Xion/gae-workers — Issue #1: Serializing db.Query objects
Author: Xion
Created: 2011-11-24 10:04:54+00:00
State: open

Since one of major applications of gae-workers library would be a batch processing of huge amount of datastore entities, a worker should be able to operate on db.Query objects. This involves serializing them to memcache when worker task gets shelved, and deserializing when new task is spawned.

The difficult part here is that not only the cursor has to be preserved, but also the query filters, orderings, keys_only setting etc., and there seems to be no easy way to access that information from the db.Query objects.

Alternative to messing inside this object is to devise a wrapper for it, which would independently remember the filter, order, ancestor etc. calls. We could also monkey-patch the db.Query class itself.