esnme/ultrajson — Issue #31: datetime types not handled correctly - and an alternative to the toDict idea
Author: kmanley
Created: 2011-12-27 20:42:30+00:00
State: open

JSON doesn't specify how date/time/datetime types should be serialized. The python stdlib json module raises a TypeError if you try to serialize these types.

Currently ujson tries to serialize/deserialize datetime.date and datetime.datetime (it raises an OverflowError for datetime.time) but the result is nonstandard and I think also incorrect, e.g.

>>> now = datetime.datetime.now()
>>> str(now)
'2011-12-27 15:17:54.879000'
>>> enc = ujson.dumps(now)
>>> enc
'1324999074' # hmmm, looks like a time_t?
>>> ujson.loads(_)
1324999074 # now we have just an int, not a datetime
>>> str(datetime.datetime.fromtimestamp(_))
'2011-12-27 10:17:54' # why isn't this the same as now above?

Using the toDict approach used for other unknown types would be preferable, then the user could decide how they want these types to be serialized/deserialized. Unfortunatley the toDict method for these types is tricky to implement because it's difficult to monkey patch built in types.

I wonder if instead of the toDict approach it might be better in general for ujson to look for two global functions to handle serialization and deserialization of unknown types, e.g. something like

def serialize_to_json(obj):
t = type(obj)
if t == datetime.date:
return "date:%s" % obj.isoformat()
elif t == bitset:
return "bitset:%s" % obj.to01()
elif t == numpy.ndarray:
return "ndarray:%s" % repr(obj.tolist())
...

def deserialize_from_json(s):
t, rep = s.split(":")
if t == "date":
return dateutil.parse(rep)
elif t == "bitset":
return bitset(rep)
... etc...

The serialize function should be able to return anything that ujson already knows how to serialize to JSON. The deserialize function should take str/unicode and return a Python object. This is more general than the toDict approach since there is more control over the JSON representation--it doesn't have to be a dictionary/object

Yet another approach would be to try to duplicate the JSONEncoder/JSONDecoder thing that the stdlib json package does.

Any thoughts?