Package: graphtage
GraphtageGraphtage is a command-line utility andunderlying libraryfor semantically comparing and merging tree-like structures, such as JSON, XML, HTML, YAML, plist, and CSS files. Its name is a
portmanteau of “graph” and “graftage”—the latter being the horticultural practice of joining two trees together such
that they grow as one.$echoOriginal:&&catoriginal.json&&echoModified:&&catmodified.jsonOriginal:{"foo":[1,2,3,4],"bar":"testing"}Modified:{"foo":[2,3,4,5],"zab":"testing","woo":["foobar"]}$graphtageoriginal.jsonmodified.json{"z̟b̶ab̟r̶":"testing","foo":[1̶,̶2,3,4,̟5̟],̟"̟w̟o̟o̟"̟:̟̟[̟"̟f̟o̟o̟b̟a̟r̟"̟]̟}Installation$pip3installgraphtageCommand Line UsageOutput FormattingGraphtage performs an analysis on an intermediate representation of the trees that is divorced from the filetypes of the
input files. This means, for example, that you can diff a JSON file against a YAML file. Also, the output format can be
different from the input format(s). By default, Graphtage will format the output diff in the same file format as the
first input file. But one could, for example, diff two JSON files and format the output in YAML. There are several
command-line arguments to specify these transformations, such as--format; please check the--helpoutput for more
information.By default, Graphtage pretty-prints its output with as many line breaks and indents as possible.{"foo":[1,2,3],"bar":"baz"}Use the--join-listsor-jloption to suppress linebreaks after list items:{"foo":[1,2,3],"bar":"baz"}Likewise, use the--join-dict-itemsor-jdoption to suppress linebreaks after key/value pairs in a dict:{"foo":[1,2,3],"bar":"baz"}Use--condensedor-jto apply both of these options:{"foo":[1,2,3],"bar":"baz"}The--only-editsor-eoption will print out a list of edits rather than applying them to the input file in place.The--edit-digestor-doption is like--only-editsbut prints a more concise context for each edit that is more
human-readable.Matching OptionsBy default, Graphtage tries to match all possible pairs of elements in a dictionary.Matching two dictionaries with each other is hard. Although computationally tractable, this can sometimes be onerous for
input files with huge dictionaries. Graphtage has three different strategies for matching dictionaries:--dict-strategy match(the most computationally expensive) tries to match all pairs of keys and values between the
two dictionaries, resulting in a match of minimum edit distance;--dict-strategy none(the least computationally expensive) will not attempt to match any key/value pairs unless
they have the exact same key; and--dict-strategy auto(the default) will automatically match the values of any key-value pairs that have identical
keys and then use thematchstrategy for the remainder of key/value pairs.SeePull Request #51for some examples of how these strategies
affect output.The--no-list-editsor-loption will not consider interstitial insertions and removals when comparing two lists.
The--no-list-edits-when-same-lengthor-lloption is a less drastic version of-lthat will behave normally for
lists that are of different lengths but behave like-lfor lists that are of the same length.ANSI ColorBy default, Graphtage will only use ANSI color in its output if it is run from a TTY. If, for example, you would like
to have Graphtage emit colorized output from a script or pipe, use the--coloror-cargument. To disable color even
when running on a TTY, use--no-color.HTML OutputGraphtage can optionally emit the diff in HTML with the--htmloption.$graphtage--htmloriginal.jsonmodified.json>diff.htmlStatus and LoggingBy default, Graphtage prints status messages and a progress bar to STDERR. To suppress this, use the--no-statusoption. To additionally suppress all but critical log messages, use--quiet. Fine-grained control of log messages is
via the--log-leveloption.Why does Graphtage exist?Diffing tree-like structures with unordered elements is tough. Say you want to compare two JSON files.
There arelimited tools available, which are effectively equivalent to
canonicalizing the JSON (e.g., sorting dictionary elements by key) and performing a standard diff. This is not always
sufficient. For example, if a key in a dictionary is changed but its value is not, a traditional diff
will conclude that the entire key/value pair was replaced by the new one, even though the only change was the key
itself. Seeour documentationfor more information.Using Graphtage as a LibraryGraphtage has a complete API for programmatically operating its diffing capabilities.
When using Graphtage as a library, it is also capable of diffing in-memory Python objects.
This can be useful for debugging Python code, for example, to determine a differential between two objects.
Seeour documentationfor more information.Extending GraphtageGraphtage is designed to be extensible: New filetypes can easily be defined, as well as new node types, edit types,
formatters, and printers. Seeour documentationfor
more information.Complete API documentation is availablehere.License and AcknowledgementsThis research was developed byTrail of Bitswith partial funding from the Defense
Advanced Research Projects Agency (DARPA) under the SafeDocs program as a subcontractor toGalois.
It is licensed under theGNU Lesser General Public License v3.0.Contact usif you're looking for an exception to the terms.
© 2020–2023, Trail of Bits.