![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png){kind=small}
ridequat: the phoenix rises
Nov. 30th, 2021 09:36 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
marnanelFive years pass, and the dwim project reappears.
dwim 0.1 (ridequat) is on its way, and should be available for Android by Friday. I don't have access to an iOS or Windows device, so if you do, I'd appreciate a hand with compiling it.
You can log in-- that's all. There will be more functionality later.
Here's the repo.
dwim 0.1 (ridequat) is on its way, and should be available for Android by Friday. I don't have access to an iOS or Windows device, so if you do, I'd appreciate a hand with compiling it.
You can log in-- that's all. There will be more functionality later.
Here's the repo.
Progress report: testing
Jun. 27th, 2016 06:38 pmmarnanelHello, lovely people. Here's a progress report:
Recall that dwim is based on scraping HTML. There's a class called Wrangler, currently just a stub, which does the parsing using Beautiful Soup. I'm currently putting a set of tests together for it. Given these tests, I've written a version of Wrangler in Python for rapid prototyping, which works, so I'm currently porting it to Java.
Recall that dwim is based on scraping HTML. There's a class called Wrangler, currently just a stub, which does the parsing using Beautiful Soup. I'm currently putting a set of tests together for it. Given these tests, I've written a version of Wrangler in Python for rapid prototyping, which works, so I'm currently porting it to Java.
Tentative roadmap
Apr. 1st, 2016 12:05 ammarnanelThis is a tentative list of dwim releases and the features that should go in them. I'm not attaching any dates yet. What have I forgotten? What should be in a different order?
0.1 (Ridequat)
Login and logout of one account.
View your own most recent entries.
View your full /read page.
Cuts are honoured.
Nothing works offline.
0.2 (Leesti)
Simple posting.
Share to DW.
Nothing works offline.
0.3 (Zaonce)
Posting to communities.
Choice of userpic on posting.
Choice of tags on posting.
0.4 (Heheng)
PleaseEdit userpics.
Edit filters and circles.
(Possibly: sync other users to Contacts)
Edit tags.
0.5 (Orrere)
Caching of incoming and outgoing posts.
0.9 (Diso)
View profiles.
Edit your own profile.
Inbox and messages.
1.0 (Lave)
First full release.
0.1 (Ridequat)
Login and logout of one account.
View your own most recent entries.
View your full /read page.
Cuts are honoured.
Nothing works offline.
0.2 (Leesti)
Simple posting.
Share to DW.
Nothing works offline.
0.3 (Zaonce)
Posting to communities.
Choice of userpic on posting.
Choice of tags on posting.
0.4 (Heheng)
PleaseEdit userpics.
Edit filters and circles.
(Possibly: sync other users to Contacts)
Edit tags.
0.5 (Orrere)
Caching of incoming and outgoing posts.
0.9 (Diso)
View profiles.
Edit your own profile.
Inbox and messages.
1.0 (Lave)
First full release.
Model spec
Feb. 21st, 2016 04:22 pmmarnanelHere's a draft specification for the model in Dwim. It's written abstractly rather than WRT any particular implementation. There's a lot of functionality missing because this is only for the first iteration. Suggestions and comments are very welcome.
class Fetcher
This fetches files over HTTP(S). It deals with caching according to the HTTP headers, and maintains the cache (including an upper limit on the cache's size).
It also uses Shrinker to degrade incoming images.
class Shrinker
This optionally scales incoiming and outgoing images down, to save space and bandwidth. It can be disabled in settings.
class Dwlib
This fetches HTML from Dreamwidth using Fetcher, and returns results represented by various classes, given below.
Its constructor requires the username of the account to log in with.
It caches cookies and passwords for each known account.
Any method, other than the login method and the constructor, can throw an exception if the cookies are unknown, stale, or rejected by the server.
class Post
A post. It may be someone else's post, retrieved from the server; it may be this user's post, waiting to be sent as soon as we get back online; it may be a draft post.
The class allows us to know only part of the information about the post, and to download other information about it as requested.
class Timeline
A chronological series of posts-- a user's timeline, a reading page, or a community.
Most Timelines represent the newest "n" posts for a particular timeline. But they can also represent posts made by a particular user in a particular month.
There is a method to return the number of posts currently known.
If you try to access posts beyond this number, the Timeline will attempt to fetch more.
There is a method to retrieve all posts on a timeline made since the last fetch.
class User
A user. This will only be rudimentary in the first iteration.
class UserSet
A set of Users. Used for making filters and reading-circles.
class Fetcher
This fetches files over HTTP(S). It deals with caching according to the HTTP headers, and maintains the cache (including an upper limit on the cache's size).
It also uses Shrinker to degrade incoming images.
class Shrinker
This optionally scales incoiming and outgoing images down, to save space and bandwidth. It can be disabled in settings.
class Dwlib
This fetches HTML from Dreamwidth using Fetcher, and returns results represented by various classes, given below.
Its constructor requires the username of the account to log in with.
It caches cookies and passwords for each known account.
Any method, other than the login method and the constructor, can throw an exception if the cookies are unknown, stale, or rejected by the server.
class Post
A post. It may be someone else's post, retrieved from the server; it may be this user's post, waiting to be sent as soon as we get back online; it may be a draft post.
The class allows us to know only part of the information about the post, and to download other information about it as requested.
class Timeline
A chronological series of posts-- a user's timeline, a reading page, or a community.
Most Timelines represent the newest "n" posts for a particular timeline. But they can also represent posts made by a particular user in a particular month.
There is a method to return the number of posts currently known.
If you try to access posts beyond this number, the Timeline will attempt to fetch more.
There is a method to retrieve all posts on a timeline made since the last fetch.
class User
A user. This will only be rudimentary in the first iteration.
class UserSet
A set of Users. Used for making filters and reading-circles.
marnanelSo, let's talk about UX. I'm not any kind of expert on this stuff, so this is entirely handwavey, and I'd especially welcome suggestions.
What do people think about hamburger buttons? It would make design easier, but people say it's more confusing for users. Maybe it'd be good at least for a first draft?
It seems to me that the most important screens we have to be able to display are timeline (which is both "lastn" and "read") and compose (which is "new post" and "edit post"). We'll need several others (change circle, choose reading group, view post without editing it, view comments, settings...) but those two seem to be the ones we really have to have.
I have an idea that the Tumblr app has some good ideas we might be able to steal. Like this, perhaps:
(Thanks to the two users who agreed to have their posts in that image)
Thoughts?
What do people think about hamburger buttons? It would make design easier, but people say it's more confusing for users. Maybe it'd be good at least for a first draft?
It seems to me that the most important screens we have to be able to display are timeline (which is both "lastn" and "read") and compose (which is "new post" and "edit post"). We'll need several others (change circle, choose reading group, view post without editing it, view comments, settings...) but those two seem to be the ones we really have to have.
I have an idea that the Tumblr app has some good ideas we might be able to steal. Like this, perhaps:
(Thanks to the two users who agreed to have their posts in that image)
Thoughts?
Python implementation of the back end
Jan. 18th, 2016 07:38 pmmarnanelThis is an implementation of the back end of dwim in Python, so it can be tested easily.
Early hacky draft. Handles login but nothing else. Suggestions etc welcome.
https://github.com/tthurman/dwimpy
Early hacky draft. Handles login but nothing else. Suggestions etc welcome.
https://github.com/tthurman/dwimpy
The back end
Jan. 13th, 2016 02:40 pmmarnanelThe back end will have to work by scraping, for now. If it's properly modular, we can replace it if-and-when a usable API comes along. We should use scraping for everything and not a hybrid of scraping and teh old API: a new API would use a different protocol from the old one, so we don't gain much by using the old one in the meantime.
There is a port of Beautiful Soup to Java, which will make the parsing much easier.Dealing with comments is a big question, and I think we can leave it until later iterations.
Important things it must be able to do:
- Log in, and store the cookies
- Get lists of: your friends, your subscription and access filters, accounts in your circle
- Read timelines of various kinds
- Post entries
We should never try to download all the entries of any timeline. Instead, when the user gets to the bottom of a timeline, if we don't know whether there are more entries, we should check then. (Like the way the Tumblr and Facebook clients work.)
I've been toying with putting together a Python prototype of the back end. It would allow us to debug it more easily.
Thoughts?
marnanelWhat? Dwim is (going to be) an Android client for Dreamwidth.
Why? A lot of people have been asking for this over the years. It hasn't happened because of DW's lack of workable API: the current API was inherited from LJ, and hasn't been updated to handle DW-specific features.
I have experience here. When I first wrote Joule, LJ's API didn't support the search it needed. So it worked by scraping-- that is, requesting the pages as if it was a browser, and parsing the HTML directly. Dwim should work the same way. It's especially easy with DW, because you can request pages without any styling in the way.
How? I've got a lot of ideas, but I'm carefully not diving in to coding it before we've talked about it here :) In particular we should discuss:
I'll post more in the next couple of days as I write up my notes. Feel free to post and discuss too.
Why? A lot of people have been asking for this over the years. It hasn't happened because of DW's lack of workable API: the current API was inherited from LJ, and hasn't been updated to handle DW-specific features.
I have experience here. When I first wrote Joule, LJ's API didn't support the search it needed. So it worked by scraping-- that is, requesting the pages as if it was a browser, and parsing the HTML directly. Dwim should work the same way. It's especially easy with DW, because you can request pages without any styling in the way.
How? I've got a lot of ideas, but I'm carefully not diving in to coding it before we've talked about it here :) In particular we should discuss:
- the UI
- how the back end will work
- caching
- whether and how we deal with posting while offline
- what we're going to do about including images (especially for "share to DW")
- what toolkits would be appropriate
- whether anyone wants to help with coding or design or documentation
- a roadmap of what features should appear in each iteration
I'll post more in the next couple of days as I write up my notes. Feel free to post and discuss too.
