Package: jdic
1. OverviewIn most projects, manipulating JSON documents requires to reinvent the
wheel on a lot of small features, or to use a varied set of external
packages. Jdic’s admitted goal is to write less code with the guarantee
to get the job done fast, flawlessly and efficiently.Jdic aims to avoid that: it is a ready-to-use type-class which eases the
manipulation of JSON-like documents, so that you can focus on logic
instead of losing time in formal document manipulations.The Jdic class provides a lot of original features but also embeds
mission-critical 3rd party libraries and unite them all within a
comprehensive easy-to-use API.2. FeaturesHere are the useful operations Jdic can do for you:Transparent JSON paths integration - for read and write operations,
through an agnostic driver model. Currently MongoDB and Jsonpath-NG
paths formats are natively supported.Fast browsing of JSON documents - browse the entire data structure
while getting useful values on each iteration: value, JSON path,
parent, parent JSON path, depth, etc.Find, Find-Keys and Find-Match features for quickly finding any key
or value, or subdocument matching MongoDB-like queries.Merge features for fusioning objects recursively, with up to 4 modes
for handling conflicting arrays (replace, merge, new, append).Diff & Patch features - so you can represent differences between two
documents in small data sets (diff), and apply those differences to
update documents (patch).JSON Schema validation - if you need it, with auto-validation on each
document change.Consistent document checksuming - natively SHA-256, it allows to get
a single checksum for the document, the checksum will always be the
same on all systems.Depth features - you can crawl your document at certain depths only.Strict features - input data will be serialized to a strict JSON
format.Agnostic dicts/lists replacement for Python’senumerate()Custom input serializer support - convert specific objects to the
JSON data you want to.Cache features with change detection to accelerate some of the API
calls.3. ExamplesInstantiationfrom jdic import jdic

o = {"a" : {"b" : {"c" : 1}}}
j = jdic(o) # Accepts dicts and lists (or any Mapping, Sequence)Find the paths for a valuepaths = [m.path for m in j.find(1)] # Find path for a value as-is
>>> ["a.b.c"]

for match in j.find(1): # Classic loop format
    print(match.path)
>>> a.b.cCrawl all the leavespaths = [m.parent_path for m in j.leaves()]  # Results include parents paths and more
>>> ["a.b"]Crawl everythingallitems = [m.value for m in j.browse()]  # The first item is always the root itself
>>> [
        {"a": {"b": {"c": 1}}},
        {"b": {"c": 1}},
        {"c": 1},
        1
]Get the depth of valuespaths = [m.depth for m in j.find({"c" : 1})] # find() target values can be objects
>>> [2]Match an object against a MongoDB-like query (useful for rules/filtering engines)paths = [m.path for m in j.find_match({"c": {"$gt": 0}}) ] # Support of Mongo-like queries
>>> ["a.b"]

paths = [m.path for m in j.find_match({  # Complex Mongo-like queries are permitted
    "$and" : [
        {"b.c": {"$exists" : True}},
        {"b.d": {"$exists" : False}}
    ]
})]
>>> ["a"]Checksum objects and sub-objects consistentlyj.checksum()
>>> ebd240a9ae435649514086d13c20d9963ec2844a1f866b313919c55a7c3f7ccb # Is consistent on all systems

j["a"].checksum() # Sub-iterables have Jdic methods / all sub-iterables implement their own checksum()
>>> 05a2013fbe17af7d58779ed96e0d74bd6fa3ce2726c1ebbd9f7dc33671b1c28e

j["a"] = None
j.checksum()
>>> 69d7d33051c5e05aa72f55a9a8e30a73da8d4afaa37127b9ea7ee29403aa9d3f # Change detection from child to parentMake diffs and patches between objectsj = jdic(o)
p = {"a" : {"e" : {"f" : -1 }}}
diff = j.diff(p)
>>> [[["a"], {"e": {"f": -1}}]] # A diff stanza - on larger documents the diffs are smaller than documents

j = j.patch(diff) # Patch does not modify the original object but returns a patched version
j == p # Jdic objects can be transparently compared with dict or list objects (or equivalents)
>>> TrueMerge objects togetherq = {"a" : {"b" : {"d" : 2}}}
j.merge(q)
>>> {"a": {"b": {"c": 1, "d": 2}}} # Handles recursive mergeValidate against a schema in real-time with the built-in change detectionj = jdic(o, schema = {'type' : 'object' , 'properties' : {'a' : {'type' : 'object'}}}) # Correct Schema
j['a'] = 3 # instant detection of schema violation (exception)
>>> Traceback (most recent call last): ...Test the Jdic object type:from jdic import Jdic # `Jdic` is the parent class, do not confuse with the function `jdic()`
>>> type(j)
<class 'jdic.JdicMapping'>
>>> isinstance(j, Jdic)
TrueChange the native enumerate()’s behavior for smoother iterations# Agnostic enumerations with a revised enumerate() function
from jdic import enumerate
y, z = [1,2,3], {'a':1, 'b':2}
for k, v in enumerate(y): # Acts just as the original enumerate() for lists
    y[k] = v
for k, v in enumerate(z): # But allows dict enumeration, raising a key instead of a counter
    z[k] = vChange the JSON path driverj = jdic({'a' : [{'b': 1}, {'b': 2}, {'b': 3}]}, driver = 'jsonpath_ng')
j['a[*].b'] = 0 # Reassign the value to all locations at once!
>>> {"a": [{"b": 0}, {"b": 0}, {"b": 0}]}

del('a[*].b') # Also works with del()
>>> {"a": [{}, {}, {}]}4. The MatchResult objectThe MatchResult object is returned for most search operations. It
contains:value: the found objectpath: the full JSON path of the found objectparent: the Jdic object containing the current found objectparent_path: the full JSON path of the parent of the found objectkey: the current key (or index) at which the object was found
within its parentdepth: the depth of the object counting from the root of the Jdic5. Jdic object instantiationjdic(obj, schema=None, serializer=None, driver=None)Instantiations of Jdic objects is made through thejdic()function
which will decide for the type of Jdic object (JdicMappingorJdicSequence) to instantiate and return. Both those types inherit
from the Jdic class (do not use this one directly, mind the lowercase).obj: any list or dictionary. Sequence and Mapping equivalents
will be casted tolistanddict.schema: optional, must be a JSON Schema in the form of adict. If provided, all changes affecting the Jdic will be
validated against the schema whenever they happen.serializer: optional, your custom serialization function. Useless
whenobjis the result of ajson.loads(). It will be called
to transform non-standard object types into standard JSON types. If
not provided, exotic types are transformed tostr. It is possible
to usesettings.serialize_custom_functioninstead, to globally
specify a serializing function for all the Jdic instances. A
serializer specified as argument will always have priority over
settings. The custom serializer function, if used, must return a JSON
compliant data type: None, bool, str, int, float, list, dict.driver: optional, a string representing the driver to use
(mongoandjsonpath_ngare natively implemented). It is
possible to usesettings.json_path_driverinstead, to globally
specify a driver. Drivers specified as argument will have priority
over settings.Note about floating point values: objects serialized as Jdic objects
will have their floating values transformed to integers whenever the
float value is equal to its integer form. This is to make the JSON dumps
and checksums consistent and avoids ‘5’ to be shown as ‘5.0’. This can
be changed by settingsettings.serialize_float_to_inttoFalse.6. Jdic objects methodsbrowse(sort=False, depth=None, maxdepth=None)Recurse on all Jdic elements, yielding aMatchResultobject on each
iteration.sort: if True all the results will be yielded with JSON paths in
alphabetical order.depth: an integer - only the results from objects atdepthwill
be yielded.maxdepth: an integer - will not recurse on documents whose depth
is abovemaxdepth.checksum(algo='sha256')Returns an ASCII checksum representing the content and data types of the
object. Checksums are consistent from an execution to another and can be
safely used for content change detection or objects comparisons. The
checksum is cached and is only recalculated if changes occured.algo: any algorithm supported by thehashlibPython librarydeepness()Returns an integer representing the deepness of the JSON structure from
wheredeepness()is called. A document with no dict or list within
it has a deepness of zero. The deepness is cached and is only
recalculated if changes occured.depth()Returns an integer representing the depth of the current document from
the root of the Jdic object. The depth of the root document is 0.diff(obj)Returns an object (a diffstanza) representing the differences between
the Jdic andobj.diff()is implemented by thejson_deltaPython library.obj: any dataenumerate(sort=False)Agnostic and non-recursive enumeration of each entry in the current
object. It yields a(k, v)tuple, wherekis either an integer
index when object is a list, and a string key when object is a dict.vis always the value.enumerate()is also available as a
standalone function within the Jdic package:from jdic import enumerate.sort: if True, sorts the dictionary keys alphabetically. Only
sort dictionary keys, not lists.find(value, sort=False, limit=None, depth=None, maxdepth=None)Searches a value within the entire Jdic. Searches are strict (==).value: the value to search for - can be a simple type (int, str,
etc.) or complex object (list, dict, Jdic, etc.)sort: if True the search results will be sorted with JSON paths
in alphabetical order.limit: an integer - terminates the search when the number of
results reacheslimit.depth: an integer - only the results from objects atdepthwill
be yielded.maxdepth: an integer - will not recurse on documents whose depth
is abovemaxdepth.find_keys(keys,mode="any",sort=False, limit=None, depth=None, maxdepth=None)Searches any sub-object containingkeys.keyscan be a single
key or a list of keys. This function aims to facilitate finding
sub-objects whose keys are known.keys: a string or list of strings. The search will be case
sensitive. Keys are for dicts and cannot be integer indexes of
arrays. Keys cannot be JSON paths.mode:"any"or"all"- if"any"then any object
matching any of the provided keys will be yielded. If"all"then
any object containing all the keys will be matched.sort: if True the search results will be sorted with JSON paths
in alphabetical order.limit: an integer - terminates the search when the number of
results reacheslimit.depth: an integer - only the results from objects atdepthwill
be yielded.maxdepth: an integer - will not recurse on documents whose depth
is abovemaxdepth.find_match(query, sort=False, limit=None, depth=None, maxdepth=None)Finds all objects matching positive againstquery. Queries forfind_match()are MongoDB-like queries, for bothmongoandjsonpath_ngdrivers. The underlying implementation is provided by
themongoqueryPython library.query: a MongoDB-like query. Please refer to the MongoDB
documentation or the examples for information on queries
structuration. Also reviewhttps://github.com/kapouille/mongoqueryfor more details onmongoqueryand its known limitations.sort: if True the search results will be sorted with JSON paths
in alphabetical order.limit: an integer - terminates the search when the number of
results reacheslimit.depth: an integer - only the results from objects atdepthwill
be yielded.maxdepth: an integer - will not recurse on documents whose depth
is abovemaxdepth.json(sort_keys=False, indent=0, ensure_ascii=False)A helper to dump Jdic objects as serialized JSON strings.sort_keys: all keys will be sorted alphabetically within their
own dicts.indent: number of spaces to add on new blocks.ensure_ascii: for a pure ASCII output (usually not recommended
for anything else than printing binary data).leaves(sort=False, depth=None, maxdepth=None)Will yield aMatchResulton each leaf encountered in the document. A
leaf is a terminal value within the JSON documents. Basically all values
are leaves, except dicts and lists.sort: if True the search results will be sorted with JSON paths
in alphabetical order.depth: an integer - only the results from objects atdepthwill
be yielded.maxdepth: an integer - will not recurse on documents whose depth
is abovemaxdepth.nb_leaves()Returns the number of leaves contained in the Jdic object. This
information is cached and is only recalculated if changes occured.match(query)ReturnsTrueorFalseif the current Jdic object matches the
Mongo-like query. Unlikefind_match()it will not recurse into
subdocuments. The currentmatch()implementation is supported by themongoqueryPython library.query: a Mongo-like query objectmerge(*objs,arr_mode="replace")Will merge the current Jdic with one or multiple other objects (dicts or
lists). It is not possible to merge a Jdic of type Mapping (dict) with a
Sequence (list) or vice-versa. This limitation does not apply to
sub-documents. Note that, unlikepatch(), the method will change the
state of the current object. If multiple args are provided then the next
obj inobjsis merged on the result of the previous merge operation,
allowing to chain the merges.objs: one or multiple objects of a similar type as the Jdic
object itself.arr_mode: determines how are handled the merging of conflicting
arrays (arrays who are on the same JSON path). 4 modes are supported:"replace": arrays in Jdic are simply replaced."append": arrays fromargsare appended to array in Jdic."new": elements of arrays fromargsare appended, but only
if they do not exist in the Jdic array."merge": a recursive merge is processed on the elements of the
same index. If there are more elements inargsarrays then
those are appended in the Jdic arrays.new()Returns an independant copy of the current Jdic, but inheriting its
driver, schema and serializer. If the Jdic is a subdocument of another
Jdic then it loses its parenthood information (detachment).parent(generation=1)Returns the Jdic parent of the current object. The root document has no
parent (None).generation: changes the generation of the parent returned. Eg.2will return the grand-parent.0always returnsNone.Noneis also returned whengenerationtargets above the root
Jdic document.patch(diff)Applies adiff stanzaas returned bydiff()and returns a patched
version of the Jdic object, without parenthood information. The original
object is not modified. The underlying implementation is provided by thejson_deltaPython library.diff: an object returned bydiff().path()Returns the full JSON path of the current Jdic object. Note that the
JSON path format will depend of the current underlying driver in use.
Eg: the root path for themongodriver is an empty string ("")
and"$"with thejsonpath_ngdriver.raw()Returns a standalone non-Jdic object representing the JSON document. The
result is alistordict, depending of the type of the Jdic
document (Sequence or Mapping). This function is useful for passing a
Jdic in the form of pure Python basic types for compatibility purposes.
The results are cached and rebuilt only if changes occured.validate(schema=None)Validates the current Jdic with any JSON schema provided. If no argument
is passed the Jdic is validated against its own schema, if it has any.
Note that callingvalidate()without argument is useless if the Jdic
is instantiated with a schema: in such case the Jdic object is
constantly validated after a change. The schema validation features are
supported by thejsonschemaPython library.schema: a JSON schema.7. SettingsAdvanced serialization settingsBy default Jdic will try to transform input floats into integers, if the
integer value is equal the float value (eg: float5.0is changed to
int5).The goal is trying to avoid unpredicted behaviors in serializations
operations, so we reduce the risk of detecting differences between two
objects who are both semantically and mathematically identical.If you want to globally prevent the float to int normalization it is
possible to setserialize_float_to_intto False:from jdic import settings
settings.serialize_float_to_int = FalseThis will apply to all classes.JSON dump formatting of Jdic objectsWhen usingstr()on a Jdic object the default behavior is to return
a nicely formatted JSON dump, whose keys are sorted and indentation set
to 4, to ease the debugging processes andprint()operations.If you wish to send or store this dump, casting it to string withstr()is not the proper way to do, prefer thejson()method
instead.If you want to change the behavior of the JSON dump throughstr(),
you can change the settings withjson_dump_sort_keysandjson_dump_indent:from jdic import settings
settings.json_dump_sort_keys = False # Disables key sorting
settings.json_dump_indent = 0 # Disables indentationThis will apply to all classes.Changing the default driverBy default the JSON path driver ismongo. Changing thejson_path_driverto another value in the settings (eg:jsonpath_ng) will change the default driver used for any future
class instantiation, unless otherwise specified injdic()parameters:from jdic import settings
settings.json_path_driver = "jsonpath_ng"8. Implementing your own JSON path driverCreate your driver as a moduleCreate a new folder within your project (eg:new_driver)Create a__init__.pyfile within the folderImplementing driversFirst, you should review an already implemented driver. Themongodriver is the best example you can use so far. It is available within
the jdic module indrivers/mongo/__init__.py.The__init__.pyfile must contain aDriverclass whose template
is:class Driver(object):
    """The driver class"""

    @classmethod
    def add_to_path(cls, path, key):
        """Adds a key at the end of a JSON path and returns the new path"""

    @classmethod
    def control_invalid_key(cls, key):
        """ Raises an exception if a key format (not JSON path) is not valid """

    @staticmethod
    def get_new_path():
        """Returns a static JSON path pointing to the root of document"""

    @classmethod
    def get_parent(cls, obj, path):
        """Returns the parent of the value pointed by JSON path"""

    @classmethod
    def get_value_at_path(cls, obj, path):
        """Returns the value pointed by JSON path"""

    @staticmethod
    def is_a_path(key):
        """True if is a JSON path, else False"""

    @classmethod
    def is_root_path(cls, path):
        """True if is a JSON path for root document, else False"""

    @staticmethod
    def keys_to_path(keys):
        """Transforms a list of keys into a proper JSON path"""

    @staticmethod
    def match(obj, query):
        """Returns True if object matches the query, else False"""

    @staticmethod
    def path_to_keys(path):
        """Transforms an expression-less JSON path into a series of keys"""Note that if you wish to benefit from already implemented functions, you
can inherit from any existing driver. For example, the current class
implementation of thejsonpath-ngdriver inherits from the Mongo
driver allowing to reimplement only the relevant features, explaining
why thematch()function is still implemented to match against Mongo
Query Language queries supported by themongodriver.class Driver(jdic.drivers.mongo.Driver):
    ...9. Related projects/libraries:json_delta:http://json-delta.readthedocs.io/en/latest/jsonschema:https://github.com/Julian/jsonschemamongoquery:https://github.com/kapouille/mongoqueryjsonpath_ng:https://github.com/h2non/jsonpath-ng10. TODO:Pip packageReadthedocs documentationMore tests (current state: 108 assertions)