Package: dirtyjson
JSON (JavaScript Object Notation) <http://json.org> is a subset of JavaScript
syntax (ECMA-262 3rd edition) used as a lightweight data interchange format.dirtyjsonis a JSON decoder meant for extracting JSON-type data from .js
files. The returned data structure includes information about line and column
numbers, so you can output more useful error messages. The input can also
include single quotes, line comments, inline comments, dangling commas,
unquoted single-word keys, and hexadecimal and octal numbers.The goal ofdirtyjsonis to read JSON objects out of files that are
littered with elements that do not fit the official JSON standard. By providing
line and column number contexts, a dirty JSON file can be used as source input
for a complex data parser or compiler.dirtyjsonexposes an API familiar to users of the standard librarymarshalandpicklemodules. However,dirtyjsonprovides
only theload(s)capability. To write JSON, use either the standardjsonlibrary orsimplejson.NoteThe code fordirtyjsonis a fairly drastically rewritten version
of the loader insimplejsonso thanks go to Bob Ippolito of thesimplejsonproject for providing such a nice starting point.Development of dirtyjson happens on Github:https://github.com/codecobblers/dirtyjsonDecoding JSON and getting position information:>>> import dirtyjson
>>> obj = [u'foo', {u'bar': [u'baz', None, 1.0, 2]}]
>>> d = dirtyjson.loads("""["foo", /* not fu*/ {bar: ['baz', null, 1.0, 2,]}] and then ignore this junk""")
>>> d == obj
True
>>> pos = d.attributes(0)  # line/column position of first element in array
>>> pos.line == 1
True
>>> pos.column == 2
True
>>> pos = d[1].attributes('bar')  # line/column position of 'bar' key/value pair
>>> pos.key.line == 1
True
>>> pos.key.column == 22
True
>>> pos.value.line == 1
True
>>> pos.value.column == 27
TrueDecoding unicode from JSON:>>> dirtyjson.loads('"\\"foo\\bar"') == u'"foo\x08ar'
TrueDecoding JSON from streams:>>> from dirtyjson.compat import StringIO
>>> io = StringIO('["streaming API"]')
>>> dirtyjson.load(io)[0] == 'streaming API'
TrueUsing Decimal instead of float:>>> import dirtyjson
>>> from decimal import Decimal
>>> dirtyjson.loads('1.1', parse_float=Decimal) == Decimal('1.1')
TrueBasic Usageload(fp[, encoding[, parse_float[, parse_int[, parse_constant[, search_for_first_object]]]]])Performs the following translations in decoding by default:JSONPythonobjectAttributedDictarrayAttributedListstringunicodenumber (int)int, longnumber (real)floattrueTruefalseFalsenullNoneIt also understandsNaN,Infinity, and-Infinityas their
correspondingfloatvalues, which is outside the JSON spec.Deserializefp(a.read()-supporting file-like object containing a JSON
document) to a Python object.dirtyjson.Errorwill be
raised if the given document is not valid.If the contents offpare encoded with an ASCII based encoding other than
UTF-8 (e.g. latin-1), then an appropriateencodingname must be specified.
Encodings that are not ASCII based (such as UCS-2) are not allowed, and
should be wrapped withcodecs.getreader(fp)(encoding), or simply decoded
to aunicodeobject and passed toloads. The default
setting of'utf-8'is fastest and should be using whenever possible.Iffp.read()returnsstrthen decoded JSON strings that contain
only ASCII characters may be parsed asstrfor performance and
memory reasons. If your code expects onlyunicodethe appropriate
solution is to wrap fp with a reader as demonstrated above.parse_float, if specified, will be called with the string of every JSON
float to be decoded. By default, this is equivalent tofloat(num_str).
This can be used to use another datatype or parser for JSON floats
(e.g.decimal.Decimal).parse_int, if specified, will be called with the int of the string of every
JSON int to be decoded. By default, this is equivalent toint(num_str).
This can be used to use another datatype or parser for JSON integers
(e.g.float).NoteUnlike the standardjsonmodule,dirtyjsonalways doesint(num_str, 0)before passing through to the converter passed is as
theparse_intparameter. This is to enable automatic handling of hex
and octal numbers.parse_constant, if specified, will be called with one of the following
strings:true,false,null,'-Infinity','Infinity','NaN'. This can be used to raise an exception if invalid JSON numbers are
encountered or to provide alternate values for any of these constants.search_for_first_object, ifTrue, will cause the parser to search for
the first occurrence of either{or[. This is very useful for
reading an object from a JavaScript file.loads(s[, encoding[, parse_float[, parse_int[, parse_constant[, search_for_first_object[, start_index]]]]])Deserializes(astrorunicodeinstance containing a JSON
document) to a Python object.dirtyjson.Errorwill be
raised if the given JSON document is not valid.Ifsis astrinstance and is encoded with an ASCII based encoding
other than UTF-8 (e.g. latin-1), then an appropriateencodingname must be
specified. Encodings that are not ASCII based (such as UCS-2) are not
allowed and should be decoded tounicodefirst.Ifsis astrthen decoded JSON strings that contain
only ASCII characters may be parsed asstrfor performance and
memory reasons. If your code expects onlyunicodethe appropriate
solution is decodestounicodeprior to calling loads.start_index, if non-zero, will cause the parser to start processing from
the specified offset, while maintaining the correct line and column numbers.
This is very useful for reading an object from the middle of a JavaScript
file.The other arguments have the same meaning as inload.Exceptionsdirtyjson.Error(msg, doc, pos)Subclass ofValueErrorwith the following additional attributes:msgThe unformatted error messagedocThe JSON document being parsedposThe start index of doc where parsing failedlinenoThe line corresponding to poscolnoThe column corresponding to posAttributedDict and AttributedListThedirtyjsonmodule usesAttributedDictandAttributedListinstead ofdictandlist. Each is actually a
subclass of its base type (dictorlist) and can be used as if they were
the standard class, but these have been enhanced to store attributes with each
element. We use those attributes to store line and column numbers. You can use
that information to refer users back to the exact location in the original
source file.Position()This is a very simple utility class that containslineandcolumn.
It is used for storing the position attributes forAttributedListandKeyValuePositionKeyValuePosition()This is another very simple utility class that containskeyandvalue. Each of those is aPositionobject specifying the
location in the original source string/file of the key and value. It is used
for storing the position attributes forAttributedDict.AttributedDict()A subclass ofdictthat behaves exactly like adictexcept that it
maintains order like anOrderedDictand allows storing attributes for
each key/value pair.add_with_attributes(self, key, value, attributes)Set thekeyin the underlyingdictto thevalueand also store
whatever is passed in asattributesfor later retrieval. In our case,
we storeKeyValuePosition.attributes(self, key)Return the attributes associated with the specifiedkeyorNoneif
no attributes exist for the key. In our case, we storeKeyValuePosition. Retrieve position info like this:pos = d.attributes(key)
key_line = pos.key.line
key_column = pos.key.column
value_line = pos.value.line
value_column = pos.value.columnAttributedList()A subclass oflistthat behaves exactly like alistexcept that it
allows storing attributes for each value.append(self, value, attributes=None):Appendsvalueto the list andattributesto the associated location.
In our case, we storePosition.attributes(self, index)Returns the attributes for the value at the givenindex. In our case,
we storePosition. Retrieve position info like this:pos = l.attributes(index)
value_line = pos.line
value_column = pos.columnNoteThis class isNOTrobust. If you insert or delete items, the attributes
will get out of sync. Making this a non-naive class would be a nice
enhancement.