Package: jsonobject
jsonobjectjsonobject is a python library for handling deeply nested JSON objects
as well-schema'd python objects.jsonobject is made byDimagi, where we build, use, and contribute to OSS in our mission to reduce inequity in the world.jsonobject is inspired by and largely API compatible with
theDocument/DocumentSchemaportion ofcouchdbkit.
Because jsonobject is not only simpler and standalone, but also faster,
we also maintain a fork ofcouchdbkit,jsonobject-couchdbkit,
that is backed byjsonobjectand works seamlessly as a swap-in replacement
for the main library.It is used heavily inCommCare HQ(source),
and the API is largely stable,
but more advanced features may change in the future.Getting StartedTo install using pip, simply runpip install jsonobjectExampleThe code below defines a simple user model, and its natural mapping to JSON.fromjsonobjectimport*classUser(JsonObject):username=StringProperty()name=StringProperty()active=BooleanProperty(default=False)date_joined=DateTimeProperty()tags=ListProperty(unicode)Once it is defined, it can be used to wrap or produce deserialized JSON.>>>user1=User(name='John Doe',username='jdoe',date_joined=datetime.datetime.utcnow(),tags=['generic','anonymous'])>>>user1.to_json(){'name':u'John Doe','username':u'jdoe','active':False,'date_joined':'2013-08-05T02:46:58Z','tags':[u'generic',u'anonymous']}Notice that the datetime is converted to an ISO format string in JSON, but is a real datetime on the object:>>>user1.date_joineddatetime.datetime(2013,8,5,2,46,58)The jsonobject ConstructorA JsonObject subclass that has been defined asUserabove
comes with a lot of built-in functionality.
The basic operations areMake a new object from deserialized JSON (e.g. the output ofjson.loads)Construct a new object with given valuesModify an objectDump to deserialized json (e.g. the input ofjson.dumps)1 & 2 are accomplished with the constructor. There are two main ways to call
the constructor:User(name='John Doe',username='jdoe',date_joined=datetime.datetime.utcnow(),tags=['generic','anonymous'])as above (satisfies #2) andUser({'name':u'John Doe','username':u'jdoe','active':False,'date_joined':'2013-08-05T02:46:58Z','tags':[u'generic',u'anonymous']})(satisfies #1). These two styles can also be mixed and matched:User({'name':u'John Doe','username':u'jdoe','active':False,'tags':[u'generic',u'anonymous']},date_joined=datetime.datetime.utcnow())Notice how datetimes are stored as strings in the deserialized JSON, but asdatetime.datetimes in the nice python object—we will refer to these as the
"json" representation and the "python" representation, or alternatively the
"unwrapped" representation and the "wrapped" representation.Gotcha.
When calling the constructor, remember that the keyword argument style
requires you to pass in the "python" representation (e.g. adatetime)
while the json-wrapping style of passing in adictrequires you to give it
in the "json" representation (e.g. a datetime-formatted string).Property TypesThere are two main kinds of property types:
scalar types (like string, bool, int, datetime, etc.)
and container types (list, dict, set).
They are dealt with separately below.Scalar TypesAll scalar properties can take the valueNonein addition to
the values particular to their type (strings, bools, etc).
If set to the wrong type,
properties raise ajsonobject.exceptions.BadValueError:classFoo(jsonobject.JsonObject):b=jsonobject.BooleanProperty()>>>Foo(b=0)Traceback(mostrecentcalllast):[...]jsonobject.exceptions.BadValueError:0notoftype<type'bool'>jsonobject.StringPropertyMaps to aunicode. Usage:classFoo(jsonobject.JsonObject):s=jsonobject.StringProperty()If you set it to an asciistrit will implicitly convert tounicode:>>>Foo(s='hi')# converts to unicodeFoo(s=u'hi')If you set it to a non-asciistr, it will fail with aUnicodeDecodeError:>>>Foo(s='\xff')Traceback(mostrecentcalllast):[...]UnicodeDecodeError:'ascii'codeccan't decode byte 0xff in position 0: ordinal not in range(128)jsonobject.BooleanPropertyMaps to abool.jsonobject.IntegerPropertyMaps to anintorlong.jsonobject.FloatPropertyMaps to afloat.jsonobject.DecimalPropertyMaps to adecimal.Decimaland stored as a JSON string.
This type, unlikeFloatProperty,
stores the "human" representation of the digits. Usage:classFoo(jsonobject.JsonObject):number=jsonobject.DecimalProperty()If you set it to anintorfloat, it will implicitly convert toDecimal:>>>Foo(number=1)Foo(number=Decimal('1'))>>>Foo(number=1.2)Foo(number=Decimal('1.2'))If you set it to astrorunicode, however, it raises anAssertionError:>>>Foo(number='1.0')Traceback(mostrecentcalllast):[...]AssertionErrorTodo: this should really raise aBadValueError.If you pass in json in which the Decimal value is astrorunicode,
but it is malformed, it throws the same errors asdecimal.Decimal.>>>Foo({'number':'1.0'})Foo(number=Decimal('1.0'))>>>Foo({'number':'1.0.0'})Traceback(mostrecentcalllast):[...]decimal.InvalidOperation:InvalidliteralforDecimal:'1.0.0'jsonobject.DatePropertyMaps to adatetime.dateand stored as a JSON string of the format'%Y-%m-%d'. Usage:classFoo(jsonobject.JsonObject):date=jsonobject.DateProperty()Wrapping a badly formatted string raises aBadValueError:>>>Foo({'date':'foo'})Traceback(mostrecentcalllast):[...]jsonobject.exceptions.BadValueError:'foo'isnotadate-formattedstringjsonobject.DateTimePropertyMaps to a timezone-unawaredatetime.datetimeand stored as a JSON string of the format'%Y-%m-%dT%H:%M:%SZ'.While it works perfectly with good inputs, it is extremely sloppy when it comes
to dealing with inputs that don't match the exact specified format.
Rather than matching stricty, it simply truncates the string
to the first 19 characters and tries to parse that as'%Y-%m-%dT%H:%M:%S'.
This ignores both microseconds and, even worse,the timezone.
This is a holdover fromcouchdbkit.In newer versions of jsonboject, you may optionally specify
aDateTimePropertyasexact:classFoo(jsonobject.JsonObject):date=jsonobject.DateTimeProperty(exact=True)This provides a much cleaner conversion model
that has the following properties:It preserves microsecondsThe incoming JSON representationmustmatch'%Y-%m-%dT%H:%M:%S.%fZ'exactly. (This is similar to the default output,
except for the mandatory 6 decimal places, i.e. milliseconds.)Representations that don't match exactly will be rejected with aBadValueError.Recommendation:
If you are not locked intocouchdbkit's earlier bad behavior,
you shouldalwaysuse theexact=Trueflag onDateTimePropertys
andTimePropertys (below).jsonobject.TimePropertyMaps to adatetime.time, stored as a JSON string of the format'%H:%M:%S'.To get access to milliseconds and strict behavior, use theexact=Truesetting
which strictly accepts the format'%H:%M:%S.%f'. This is always recommended.
For more information please read the previous section onDateTimeProperty.Container TypesContainer types generally take a first argument,item_type,
specifying the type of the contained objects.jsonobject.ObjectProperty(item_type)Maps to adictthat has a schema specified byitem_type,
which must be itself a subclass ofJsonObject. Usage:classBar(jsonobject.JsonObject):name=jsonobject.StringProperty()classFoo(jsonobject.JsonObject):bar=jsonobject.ObjectProperty(Bar)If not specified, it will be set to a new object with default values:>>>Foo()Foo(bar=Bar(name=None))If you want it set toNoneyou must do so explicitly.jsonobject.ListProperty(item_type)Maps to alistwith items of typeitem_type,
which can be any of the following:aninstanceof a property class. This is the most flexible option,
and all validation (required, etc.) will be done as as specified by the property instance.a property class, which will be instantiated withrequired=Trueone of their corresponding python types (i.e.intforIntegerProperty, etc.)aJsonObjectsubclassNote that a propertyclass(as well as the related python type syntax)
will be instantiated withrequired=True,
soListProperty(IntegerProperty)andListProperty(int)do not allowNone, andListProperty(IntegerProperty())doesallowNone.The serialization behavior of whatever item type is given is recursively
applied to each member of the list.If not specified, it will be set to an empty list.jsonobject.SetProperty(item_type)Maps to asetand stored as a list (with only unique elements).
Otherwise its behavior is very much likeListProperty's.jsonobject.DictProperty(item_type)Maps to adictwith string keys and values specified byitem_type.
Otherwise its behavior is very much likeListProperty's.If not specified, it will be set to an empty dict.Otherjsonobject.DefaultProperty()This flexibly wraps any valid JSON, including all scalar and container types,
dynamically detecting the value's type and treating it
with the corresponding property.Property optionsCertain parameters may be passed in to any property.For example,requiredis one such parameter in the example below:classUser(JsonObject):username=StringProperty(required=True)Here is a complete list of properties:defaultSpecifies a default value for the propertynameThe name of the property within the JSON representation*.
This defaults to the name of the python property, but you can override it
if you wish. This can be useful, for example, to get around conflicting
with python keywords:>>>classRoute(JsonObject):...from_=StringProperty(name='from')...to=StringProperty()# name='to' by default>>>Route(from_='me',to='you').to_json(){'from':u'me','to':u'you'}Notice how an underscore is present in the python property name ('from_'),
but absent in the JSON property name ('from').\*If you're wondering how `StringProperty`'s `name` parameter
could possibly default to `to` in the example above,
when it doesn't have access to the `Route` class's properties at init time,
you're completely right.
The behavior described is implemented in `JsonObject`'s `__metaclass__`,
which *does* have access to the `Route` class's properties.choicesA list of allowed values for the property.
(Unless otherwise specified,Noneis also an allowed value.)requiredDefaults toFalse.
For scalar propertiesrequiresmeans that the valueNonemay not be used.
For container properties it means they may not be empty
or take the valueNone.exclude_if_noneDefaults toFalse. When set to true, this property will be excluded
from the JSON output when its value is falsey.
(Note that currently this is at odds with the parameter's name,
since the condition is that it is falsey, not that it isNone).validatorsA single validator function or list of validator functions.
Each validator function should raise an exception on invalid input
and do nothing otherwise.verbose_nameThis property does nothing and was added to match couchdbkit's API.Performance Comparison with CouchdbkitIn order to do a direct comparison with couchdbkit, the test suite includes a large sample schema originally written with couchdbkit. It is easy to swap in jsonobject for couchdbkit and run the tests with each. Here are the results:$ python -m unittest test.test_couchdbkit
....
----------------------------------------------------------------------
Ran 4 tests in 1.403s

OK
$ python -m unittest test.test_couchdbkit
....
----------------------------------------------------------------------
Ran 4 tests in 0.153s

OKRunning testsYou must rebuild C files for the tests to pick up your changes.  Try this for iterating:$ python setup.py build_ext --inplace && python setup.py testRecreating C source filesFor any changes in the pyx files, the corresponding C files should be recompiled with$ find jsonobject -iname '*.c' -delete
$ find jsonobject -iname '*.so' -delete
$ python setup.py build_ext --inplaceThese changes should be committed independently of the non-automated changes you made.