marnanel: (Default)
[personal profile] marnanel
Hello, lovely people. Here's a progress report:

Recall that dwim is based on scraping HTML. There's a class called Wrangler, currently just a stub, which does the parsing using Beautiful Soup. I'm currently putting a set of tests together for it. Given these tests, I've written a version of Wrangler in Python for rapid prototyping, which works, so I'm currently porting it to Java.
marnanel: (Default)
[personal profile] marnanel
dwim: work is proceeding, but I realised I hadn't thought through the threading well enough, so I'm busy fixing that. Hence the recent lack of pretty screenshots!
marnanel: (Default)
[personal profile] marnanel
This is a tentative list of dwim releases and the features that should go in them. I'm not attaching any dates yet. What have I forgotten? What should be in a different order?

0.1 (Ridequat)
Login and logout of one account.
View your own most recent entries.
View your full /read page.
Cuts are honoured.
Nothing works offline.

0.2 (Leesti)
Simple posting.
Share to DW.
Nothing works offline.

0.3 (Zaonce)
Posting to communities.
Choice of userpic on posting.
Choice of tags on posting.

0.4 (Heheng)
PleaseEdit userpics.
Edit filters and circles.
(Possibly: sync other users to Contacts)
Edit tags.

0.5 (Orrere)
Caching of incoming and outgoing posts.

0.9 (Diso)
View profiles.
Edit your own profile.
Inbox and messages.

1.0 (Lave)
First full release.

Model spec

Feb. 21st, 2016 04:22 pm
marnanel: (Default)
[personal profile] marnanel
Here's a draft specification for the model in Dwim. It's written abstractly rather than WRT any particular implementation. There's a lot of functionality missing because this is only for the first iteration. Suggestions and comments are very welcome.

class Fetcher
This fetches files over HTTP(S). It deals with caching according to the HTTP headers, and maintains the cache (including an upper limit on the cache's size).
It also uses Shrinker to degrade incoming images.

class Shrinker
This optionally scales incoiming and outgoing images down, to save space and bandwidth. It can be disabled in settings.

class Dwlib
This fetches HTML from Dreamwidth using Fetcher, and returns results represented by various classes, given below.
Its constructor requires the username of the account to log in with.
It caches cookies and passwords for each known account.
Any method, other than the login method and the constructor, can throw an exception if the cookies are unknown, stale, or rejected by the server.

class Post
A post. It may be someone else's post, retrieved from the server; it may be this user's post, waiting to be sent as soon as we get back online; it may be a draft post.
The class allows us to know only part of the information about the post, and to download other information about it as requested.

class Timeline
A chronological series of posts-- a user's timeline, a reading page, or a community.
Most Timelines represent the newest "n" posts for a particular timeline. But they can also represent posts made by a particular user in a particular month.
There is a method to return the number of posts currently known.
If you try to access posts beyond this number, the Timeline will attempt to fetch more.
There is a method to retrieve all posts on a timeline made since the last fetch.

class User
A user. This will only be rudimentary in the first iteration.

class UserSet
A set of Users. Used for making filters and reading-circles.


Jan. 21st, 2016 08:38 pm
marnanel: (Default)
[personal profile] marnanel
So, let's talk about UX. I'm not any kind of expert on this stuff, so this is entirely handwavey, and I'd especially welcome suggestions.

What do people think about hamburger buttons? It would make design easier, but people say it's more confusing for users. Maybe it'd be good at least for a first draft?

It seems to me that the most important screens we have to be able to display are timeline (which is both "lastn" and "read") and compose (which is "new post" and "edit post"). We'll need several others (change circle, choose reading group, view post without editing it, view comments, settings...) but those two seem to be the ones we really have to have.

I have an idea that the Tumblr app has some good ideas we might be able to steal. Like this, perhaps:

(Thanks to the two users who agreed to have their posts in that image)

marnanel: (Default)
[personal profile] marnanel
This is an implementation of the back end of dwim in Python, so it can be tested easily.

Early hacky draft. Handles login but nothing else. Suggestions etc welcome.
marnanel: (Default)
[personal profile] marnanel

The back end will have to work by scraping, for now. If it's properly modular, we can replace it if-and-when a usable API comes along. We should use scraping for everything and not a hybrid of scraping and teh old API: a new API would use a different protocol from the old one, so we don't gain much by using the old one in the meantime.

There is a port of Beautiful Soup to Java, which will make the parsing much easier.

Dealing with comments is a big question, and I think we can leave it until later iterations.

Important things it must be able to do:
  • Log in, and store the cookies
  • Get lists of: your friends, your subscription and access filters, accounts in your circle
  • Read timelines of various kinds
  • Post entries
The timelines present a particularly interesting challenge. It's a shame that RSS/Atom feeds aren't a general solution because they're not available for /read pages. But we can request timelines in slices (?skip=...) and that will be enough.

We should never try to download all the entries of any timeline. Instead, when the user gets to the bottom of a timeline, if we don't know whether there are more entries, we should check then. (Like the way the Tumblr and Facebook clients work.)

I've been toying with putting together a Python prototype of the back end. It would allow us to debug it more easily.



Jan. 13th, 2016 12:30 pm
marnanel: (Default)
[personal profile] marnanel
What? Dwim is (going to be) an Android client for Dreamwidth.

Why? A lot of people have been asking for this over the years. It hasn't happened because of DW's lack of workable API: the current API was inherited from LJ, and hasn't been updated to handle DW-specific features.

I have experience here. When I first wrote Joule, LJ's API didn't support the search it needed. So it worked by scraping-- that is, requesting the pages as if it was a browser, and parsing the HTML directly. Dwim should work the same way. It's especially easy with DW, because you can request pages without any styling in the way.

How? I've got a lot of ideas, but I'm carefully not diving in to coding it before we've talked about it here :) In particular we should discuss:
  • the UI
  • how the back end will work
  • caching
  • whether and how we deal with posting while offline
  • what we're going to do about including images (especially for "share to DW")
  • what toolkits would be appropriate
  • whether anyone wants to help with coding or design or documentation
  • a roadmap of what features should appear in each iteration
I don't know much about iOS development, so my horizons here are limited to Android. If anyone does, that'll be useful to know.

I'll post more in the next couple of days as I write up my notes. Feel free to post and discuss too.


dwim: the dwim logo (Default)
Dwim the Android client

June 2016

56 7891011
26 27282930  


RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Oct. 20th, 2017 02:17 pm
Powered by Dreamwidth Studios