Package: lovely.gae
lovely.gae.asyncThis package executes jobs asynchronously, it uses the appengine
taskqueue to exectue the jobs.>>> from lovely.gae.async import defer, get_tasksThe defer function executes a handler asynchronously as a job. We
create 3 jobs that have the same signature.>>> import time
>>> for i in range(3):
...     print defer(time.sleep, [0.3])
<google.appengine.api.labs.taskqueue.taskqueue.Task object at ...>
None
NoneLet us have a look on what jobs are there. Note that there is only one
because all the 3 jobs we added were the same.>>> len(get_tasks())
1If we change the signature of the job, a new one will be added.>>> added = defer(time.sleep, [0.4])
>>> len(get_tasks())
2Normally jobs are automatically executed by the taskqueueapi, we have
a test method which executes the jobs and returns the number of jobs
done.>>> run_tasks()
2Now we can add the same signature again.>>> added = defer(time.sleep, [0.4])
>>> run_tasks()
1We can also set only_once to false to execute a worker many times with
the same signature.>>> from pprint import pprint
>>> defer(pprint, ['hello'], once_only=False)
<google.appengine.api.labs.taskqueue.taskqueue.Task object at ...>
>>> defer(pprint, ['hello'], once_only=False)
<google.appengine.api.labs.taskqueue.taskqueue.Task object at ...>
>>> run_tasks()
'hello'
'hello'
2DB Custom Property ClassesTyped ListsThis property converts model instances to keys and ensures a length.>>> from lovely.gae.db.property import TypedListProperty
>>> from google.appengine.ext import dbLet us create a model>>> class Yummy(db.Model): pass
>>> class Bummy(db.Model): passWe can now reference Yummy instances with our property. Note that we
can also use the kind name as an argument of the kind.>>> class Dummy(db.Model):
...     yummies = TypedListProperty(Yummy)
...     bummies = TypedListProperty('Bummy', length=3)The kind arguement needs to be a subclass kind name or a db.Model.>>> TypedListProperty(object)
Traceback (most recent call last):
...
ValueError: Kind needs to be a subclass of db.Model>>> dummy = Dummy()
>>> dummy_key = dummy.put()
>>> yummy1 = Yummy()
>>> yummy1_key = yummy1.put()
>>> dummy.yummies = [yummy1]We cannot set any other type.>>> bummy1 = Bummy()
>>> bummy1_key = bummy1.put()
>>> dummy.yummies = [bummy1]
Traceback (most recent call last):
...
BadValueError: Wrong kind u'Bummy'The length needs to match if defined (see above).>>> dummy.bummies = [bummy1]
Traceback (most recent call last):
...
BadValueError: Wrong length need 3 got 1>>> dummy.bummies = [bummy1, bummy1, bummy1]
>>> dummy_key == dummy.put()
TrueCase-Insensitive String PropertyThis property allows searching for the lowercase prefix in a
case-insensitive manner. This is usefull for autocomplete
implementations where we do not want to have a seperate property just
for searching.>>> from lovely.gae.db.property import IStringProperty
>>> class Foo(db.Model):
...     title = IStringProperty()>>> f1 = Foo(title='Foo 1')
>>> kf1 = f1.put()
>>> f2 = Foo(title='Foo 2')
>>> kf2 = f2.put()
>>> f3 = Foo(title='foo 3')
>>> kf3 = f3.put()
>>> f4 = Foo(title=None)
>>> kf4 = f4.put()The property does not allow the special seperator character which is
just one below the  highest unicode character,>>> f3 = Foo(title='Foo 3' + IStringProperty.SEPERATOR)
Traceback (most recent call last):
...
BadValueError: Not all characters in property titleNote that if we want to do an exact search, we have to use a special
filter that can be created by the property instance.>>> [f.title for f in Foo.all().filter('title =', 'foo 1')]
[]The “equal” filter arguments can be computed with a special method on
the property.>>> ef = Foo.title.equals_filter('Foo 1')
>>> ef
('title =', u'foo 1\xef\xbf\xbcFoo 1')>>> [f.title for f in Foo.all().filter(*ef)]
[u'Foo 1']Let us try with inequallity, e.g. prefix search. Prefix search is
normally done with two filters using the highest unicode character.Search for all that starts with “fo” case-insensitive.>>> q = Foo.all()
>>> q = q.filter('title >=', 'fo')
>>> q = q.filter('title <', 'fo' + u'\xEF\xBF\xBD')
>>> [f.title for f in q]
[u'Foo 1', u'Foo 2', u'foo 3']Search for all that starts with ‘foo 1’>>> q = Foo.all()
>>> q = q.filter('title >=', 'foo 1')
>>> q = q.filter('title <', 'foo 1' + u'\xEF\xBF\xBD')
>>> [f.title for f in q]
[u'Foo 1']>>> q = Foo.all()
>>> q = q.filter('title >=', 'foo 2')
>>> q = q.filter('title <=', 'foo 2' + u'\xEF\xBF\xBD')
>>> [f.title for f in q]
[u'Foo 2']>>> q = Foo.all()
>>> q = q.filter('title >=', 'foo 3')
>>> q = q.filter('title <=', 'foo 3' + u'\xEF\xBF\xBD')
>>> [f.title for f in q]
[u'foo 3']Pickle PropertyA pickle property can hold any pickleable python object.>>> from lovely.gae.db.property import PickleProperty>>> class Pickie(db.Model):
...     data = PickleProperty()>>> pickie = Pickie(data={})
>>> pickie.data
{}
>>> kp = pickie.put()
>>> pickie.data
{}
>>> pickie = db.get(kp)
>>> pickie.data
{}
>>> pickie.data = {'key':501*"x"}
>>> kp = pickie.put()
>>> pickie.data
{'key': 'xx...xx'}If the value is not pickleable we get a validation error.>>> pickie = Pickie(data=dict(y=lambda x:x))
Traceback (most recent call last):
BadValueError: Property 'data' must be pickleable:
(Can't pickle <function <lambda> at ...>:
it's not found as __main__.<lambda>)Safe ReferenceProperty>>> from lovely.gae.db.property import SafeReferencePropertyWe use a new model with a gae reference and our safe reference.>>> class Refie(db.Model):
...     ref   = db.ReferenceProperty(Yummy, collection_name='ref_ref')
...     sfref = SafeReferenceProperty(Yummy, collection_name='sfref_ref')>>> refie = Refie()
>>> refie.sfref is None
True
>>> refie.ref is None
TrueAn object to be referenced.>>> refyummy1 = Yummy()
>>> ignore = refyummy1.put()Set the references to our yummy object.>>> refie.sfref = refyummy1
>>> refie.sfref
<Yummy object at ...>
>>> refie.ref = refyummy1
>>> refie.ref
<Yummy object at ...>>>> refieKey = refie.put()Now we delete the referenced object.>>> refyummy1.delete()And reload our referencing object.>>> refie = db.get(refieKey)The gae reference raises an exception.>>> refie.ref
Traceback (most recent call last):
Error: ReferenceProperty failed to be resolvedWe catch the logs here.>>> import logging
>>> from StringIO import StringIO
>>> log = StringIO()
>>> handler = logging.StreamHandler(log)
>>> logger = logging.getLogger('lovely.gae.db')
>>> logger.setLevel(logging.INFO)
>>> logger.addHandler(handler)Our safe reference returns None.>>> pos = log.tell()
>>> refie.sfref is None
TrueLet’s see what the log contains.>>> log.seek(pos)
>>> print log.read()
Unresolved Reference for "Refie._sfref" set to NoneAccessing the stale property once again we will see it was reset to None:>>> pos = log.tell()
>>> refie.sfref is None
True

>>> log.seek(pos)
>>> print log.read() == ''
TrueThe property get set to None if the reference points to a dead object but only
if the property is not required:>>> class Requy(db.Model):
...     sfref = SafeReferenceProperty(Yummy, collection_name='req_sfref_ref',
...                                   required=True)

>>> refyummy1 = Yummy()
>>> ignore = refyummy1.put()

>>> requy = Requy(sfref = refyummy1)
>>> requyKey = requy.put()

>>> requy.sfref
<Yummy object at ...>

>>> refyummy1.delete()

>>> requy = db.get(requyKey)

>>> pos = log.tell()
>>> requy.sfref is None
True

>>> log.seek(pos)
>>> print log.read()
Unresolved Reference for "Requy._sfref" will remain because it is requiredBatch marker creationThis packages provides the possibility to create markers for every N
objects of a given query. This is useful to create batched html pages
or to generate jobs for every N objects.A list of attribute values are created that represent the end of a
batch at any given position in a given query. The result is stored in
memcache and the key is provided to a callback function.>>> from lovely.gae import batchLet us create some test objects.>>> from google.appengine.ext import db
>>> class Stub(db.Model):
...     c_time = db.DateTimeProperty(auto_now_add=True, required=True)
...     name = db.StringProperty()
...     age = db.IntegerProperty()
...     state = db.IntegerProperty()
...     def __repr__(self):
...         return '<Stub %s>' % self.name>>> for i in range(1,13):
...     s = Stub(name=str(i), age=i, state=i%2)
...     sk = s.put()>>> Stub.all().count()
12First we make sure that we have no tasks in the queue for testing.>>> from lovely.gae.async import get_tasks
>>> len(get_tasks())
0So for example if we want to know any 100th key of a given kind we
could calculate it like shown below. Note that we provide the pprint function
as a callback, so we get the memcache key in the output.The calculate_markers function returns the memcache key that will be
used to store the result when the calculation is completed.>>> from pprint import pprint
>>> mc_key = batch.create_markers('Stub', callback=pprint)
>>> mc_key
'create_markers:...-...-...'A task gets created.>>> tasks = get_tasks()
>>> len(tasks)
1Let us run the task.>>> run_tasks(1)
1We now have another task left for the callback, which is actually the
pprint function.>>> run_tasks()
'create_markers:...-...-...'
1We should now have a result. The result shows that we need no batches
for the given batch size (because we only have 12 objects).>>> from google.appengine.api import memcache
>>> memcache.get(mc_key)
[]Let us use another batch size. This time without callback.>>> mc_key = batch.create_markers('Stub', batchsize=1)
>>> run_tasks()
1We now have exatly 12 keys, because the batch size was 1.>>> len(memcache.get(mc_key))
12The default attributes returned are the keys.>>> memcache.get(mc_key)
[datastore_types.Key.fro...We can also use other attributes. Let us get items batched by c_time
descending. Note that it is not checked if values are not unique, so
if a non-unique attribute is used it might be the case that batch
ranges contains objects twice.>>> mc_key = batch.create_markers('Stub',
...                               attribute='c_time',
...                               order='desc',
...                               batchsize=3)
>>> run_tasks()
1
>>> markers = memcache.get(mc_key)
>>> markers
[datetime.datetime(...
>>> len(markers)
4
>>> sorted(markers, reverse=True) == markers
True>>> mc_key = batch.create_markers('Stub',
...                               attribute='c_time',
...                               order='asc',
...                               batchsize=3)
>>> run_tasks()
1
>>> markers = memcache.get(mc_key)
>>> markers
[datetime.datetime(...
>>> len(markers)
4
>>> sorted(markers) == markers
TrueWe can also pass filters to be applied to the query for the batch like this:>>> mc_key = batch.create_markers('Stub',
...                               filters=[('state', 0)],
...                               attribute='c_time',
...                               order='asc',
...                               batchsize=3)
>>> run_tasks()
1
>>> markers = memcache.get(mc_key)
>>> len(markers)
2