Package: partial-json-parser
Partial JSON ParserSometimes we needLLM (Large Language Models)to producestructural informationinstead of natural language. The easiest way is to use JSON.But before receiving the last token of response, the JSON is broken, which means you can't useJSON.parseto decode it. But we still want to stream the data to the user.Here comespartial-json-parser, a lightweight and customizable library for parsing partial JSON strings. Here is ademo.(Note that there isa JavaScript implementationtoo)Installationpipinstallpartial-json-parser# or poetry / pdm / condapartial-json-parseris implemented purely in Python, with good type hints.UsageImporting the libraryYou can import theloadsfunction and theAllowobject from the library like this:frompartial_json_parserimportloads,AllowTheAllowobject is just an Enum for options. It determines what types can be partial. types not included inallowonly appears after its completion can be ensured.Parsing complete / partial JSON stringsTheloadsfunction works just like the built-injson.loadswhen parsing a complete JSON string:result=loads('{"key":"value"}')print(result)# Outputs: {'key': 'value'}You can parse a partial JSON string by passing an additional parameter to theloadsfunction. This parameter is abitwise ORof the constants from theAllowflag:(Note that you can directly import the constants you need frompartial-json-parser.options)frompartial_json_parserimportloads,Allowfrompartial_json_parser.optionsimportSTR,OBJresult=loads('{"key": "v',STR|OBJ)print(result)# Outputs: {'key': 'v'}In this example,Allow.STRtells the parser that it's okay if a string is incomplete, andAllow.OBJtells the parser so as a dict. The parser then try to return as much data as it can.If you don't allow partial strings, then it will not add"key"to the object because"vis not close:result=loads('{"key": "v',OBJ)print(result)# Outputs: {}result=loads('{"key": "value"',OBJ)print(result)# Outputs: {'key': 'value'}Similarity, you can parse partial lists or even partial special values if you allow it:(Note thatallowdefaults toAllow.ALL)result=loads('[ {"key1": "value1", "key2": [ "value2')print(result)# Outputs: [{'key1': 'value1', 'key2': ['value2']}]result=loads("-Inf")print(result)# Outputs: -infHandling malformed JSONIf the JSON string is malformed, theparsefunction will throw an error:loads("wrong")# MalformedJSON: Malformed node or string on line 1API Referenceloads(json_string, [allow_partial])json_string<string>: The JSON string to parse.allow_partial<Allow | int>: Specify what kind of partialness is allowed during JSON parsing (default:Allow.ALL).Returns the parsed Python value.AllowEnum class that specifies what kind of partialness is allowed during JSON parsing. It has the following members:STR: Allow partial string.NUM: Allow partial number.ARR: Allow partial array.OBJ: Allow partial object.NULL: Allow partial null.BOOL: Allow partial boolean.NAN: Allow partial NaN.INFINITY: Allow partial Infinity._INFINITY: Allow partial -Infinity.INF: Allow both partial Infinity and -Infinity.SPECIAL: Allow all special values.ATOM: Allow all atomic values.COLLECTION: Allow all collection values.ALL: Allow all values.TestingTo run the tests for this library, you should clone the repository and install the dependencies:gitclonehttps://github.com/promplate/partial-json-parser.gitcdpartial-json-parser
pdminstallThen, you can run the tests usingHypothesisandPytest:pdmtestPlease note that while we strive to cover as many edge cases as possible, it's always possible that some cases might not be covered.LicenseThis project is licensed under the MIT License.