simon-weber/Unofficial-Google-Music-API — Issue #10: Unified data model
Author: simon-weber
Created: 2012-03-06 16:05:40+00:00
State: open

@dpogue: I've been thinking of a building a unified data model lately, that I think may simplify implementation and ease some of the concerns with mutation and building dictionaries. The basic idea is that we'd have a single collection of all the tracks/playlists backed with the actual representations from Google Music. The Song and Playlist models would basically be pointers into this central collection, keeping everything in sync across models. It would look something like this:

Library
---------
This is the main collection of all the songs and playlists.

It's backed by two privateish dictionaries: ``{"song id": {}}`` and ``{"playlist id": [("song id", "entryId"), ...]}``. It also exposes methods for searching and retrieving songs/playlists (I've already got some things in gmtools that'll come in handy here for searching locally. There's also the slower ``search()`` that hits the server.).

I'll show how we'd handle mutation and updates later, but my idea is that Songs and Playlists expose methods that simply change the GM representations in the Library. That way, we can leave it up to the user at runtime whether they want to auto-update or batch their changes. Keeping everything in their GM representation is a big help there; we'd just have to track which ids changed, and then push their current representation. (only thing: I'm not actually sure what we have to work with in the way of pushing playlist changes, but worst case we can always erase and create a new one with the same name).

Song
--------
This represents a single song in the Library. The only data member is the id string, which is used to index into the Library dictionary when metadata is requested.

Metadata changes could either be done by indexing into a song:

```python
song['artist'] = 'foo'
print song['album']
```

or by properties. That's just a choice between hooking ``__getitem__`` or ``__getattribute__``.

Getting/setting metadata goes through the Library to keep everything in sync, which I think makes more sense than being able to have different instances of songs:

```python
song = library.songs["id"] # not direct access to the underlying
# dictionaries; we'd control this. maybe
# a get() that support lists would be better

same_song = library.songs["id"] # returns a new Song instance

assert song.id() != same_song.id() # id() being the Python id (memory location)
assert song == same_song # comparison/hashing is just done with GM ids

print song["artist"] # => "old artist"
song["artist"] = "new artist"
print same_song["artist"] # => "new artist"
```

The setters in Song (or Library, which is probably better) would hook into the Metadata_Expectations to ensure we're only setting mutable things to the allowed values, etc.

Playlist
-----------
Playlists are a lot like songs; they'd just store their playlistId, and modify the Library collection. Tools for changing them are exposed just like songs.

Mutation
--------
Songs and Playlists wouldn't be directly manipulating the dictionaries that back Library. There'd be one level (or two, if we want to check change validity in Library instead of Song) of indirection, which handles whether or not we want to batch changes or update everything on every change. Batching changes would just mean maintaining a set of all the song/playlist ids that have been modified.

Either way, when an update occurs, we flush out all our changes and verify they went ok (metadata poses a problem, since the server doesn't actually accurately tell us if our request succeeded. I'm not really sure how to deal with this. Right now, when I test, I'm pulling down the entire library to see my changes - not ideal for large libraries. I'm thinking an optimistic approach would be best here: assume we succeeded, and fork a thread to verify at some later time that we actually did. If not, either try again or warn/raise an exception that the Library is out of date. That's just details, though.).

Assuming away the metadata verification for now, the entire process looks something like:

* user sets some piece of data in a Song, eg song["artist"] = "foo"
* Song checks to see that "foo" is allowed for artist, and that artist is mutable
* it is, so it submits a change to Library (who assumes the change is valid - it's probably better to do verification of change validity in Library instead...)
* Library gets a change.
* change recorded in the internal dictionary
* ** if we're auto-updating: **
* change pushed out to GM.
* we block until we see the change was valid
* ** if we're batching: **
* we record the id as dirty
* at some later date, changes are pushed; we push out everything that's got a dirty id and verify all changes were good.

***

Whew, that was a bit more than I thought it would - which brings me to my next point!

I'm afraid this might be a bit heavy to build in to Api. Maybe we can provide it as another abstraction layer on top of Api? This helps with two things:

* it's not a destructive change for people working with Api right now
* it shouldn't matter *how* we're pushing changes (skyjam vs web client); the model doesn't care and Api can decide

Let me know what you think! I'll have plenty of time next week to work on this if/when we decide on something.