simplejson/simplejson — Issue #28: Large number of nested objects triggers recursion error
Author: kumar303
Created: 2012-01-03 21:28:07+00:00
State: open

Large nested JSON objects will raise a recursion error like this:

Traceback (most recent call last):
File "stack.py", line 25, in
main()
File "stack.py", line 15, in main
dd = simplejson.loads(s)
File "/Users/kumar/tmp/json-stack/lib/python2.6/site-packages/simplejson/__init__.py", line 413, in loads
return _default_decoder.decode(s)
File "/Users/kumar/tmp/json-stack/lib/python2.6/site-packages/simplejson/decoder.py", line 402, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/kumar/tmp/json-stack/lib/python2.6/site-packages/simplejson/decoder.py", line 418, in raw_decode
obj, end = self.scan_once(s, idx)
RuntimeError: maximum recursion depth exceeded while decoding a JSON document

Here is a script to reproduce:

import json
import simplejson

def main():
s = []
num = 900
for i in range(num):
s.append('{"foo":')
s.append('1')
for i in range(num):
s.append('}')
s = ''.join(s)
try:
dd = simplejson.loads(s)
except Exception, exc:
print 'simplejson', str(exc)
try:
dd = json.loads(s)
except Exception, exc:
print 'json', str(exc)

if __name__ == '__main__':
main()

The stdlib json module fails at about 400 nested objects whereas simplejson fails at about 1000 nested objects. I know it sounds excessive but is there a way to make this not be recursive? The code we use to [validate Firefox Add-ons](https://github.com/mozilla/amo-validator/) was hitting this limit in production because of complex nested structureds (maybe locale based, I forget exactly).