Package: jsno
jsnoConvert Python data to and from JSON-compatible data structures.Jsno provides functions that turn any Python values into JSON-compatible
structures and back. The jsonified data can then be used wherever JSON
data is required: it can be dumped into a file, sent over the network
in an API call, or stored in a database that support JSON data.Note that jsno does not replace the standardjsonmodule - it does not
produce JSON encoded string, but instead turns arbitrary Python values into
structures thatcanbe JSON encoded.Jsno provides jsonification for most of the standard Python datatypes.
It is also easilyextensible, so that custom jsonification can be
defined for new datatypes.Jsno has special support fordataclasses, so no effort is needed to
make jsonificatiin work for classes defined as dataclasses.Basic usageData is converted to JSON strutures using thejsonifyfunction:fromdatetimeimportdatetime,dateimportjsnojsonified=jsno.jsonify({"date":datetime.utcnow().date()})assertjsonified=={"date":"2023-07-30"}Converting the jsonified data back is done using theunjsonifyfunction.
However,unjsonifycan't guess the type of the resulting value that the
JSON data should be converted to. So it needs to be given it explicitly, using
"bracket syntax":value=jsno.unjsonify[dict[str,date]](jsonified)assertvalue=={"date":datetime.date(2023,7,30)}Here the example value had to be typed as  a dictionary with date values.
When working with heterogenous data, it's usually best to define it asdataclassesinstead.Example with dataclassesFirst, let's define a custom dataclass to keep track of domain data:fromdataclassesimportdataclass@dataclassclassDomainRecord:domain:strips:list[str]enabled_at:date|None=NoneThen, let's make a couple of such records:domains=[DomainRecord(domain="example.com",ips=["93.184.216.34"],enabled_at=date(1992,1,1)),DomainRecord(domain="another.example.com",ips=[])]With jsno, these can be turned into JSON and stored in a local file:importjsonimportpathlibjsonified=jsno.jsonify(domains)pathlib.Path('domains.json').write_text(json.dumps(jsonified,indent=4))The file will look like this:[{"domain":"example.com","ips":["93.184.216.34"],"enabled_at":"1992-01-01"},{"domain":"another.example.com","ips":[]}]To read and use the domains later:jsonified=json.loads(pathlib.Path('domains.json').read_text())domains=jsno.unjsonify[list[DomainRecord]](jsonified)assertdomains[0].enabled_at.year==1992Supported typesNative JSON typesThe following types are native JSON types, and they are mapped to themselves
when jsonifying and unjsonifying:strboolintfloatNoneType (None maps to JSON's null)list (JSON array)dict (JSON object)Abstract base classesThe operations are also defined to the following abstract base classes, so
any type implementing them can be jsnoified and unjsonified:Sequence (maps to list)Mapping (maps to dict)Set (maps to list)ByteString (maps to base64 encoded string)Union typesUnion types (typing.Union and T | U) are supported. However, when unjsonising
union types, it's important that the JSON representations of two different
types do not overlap. Otherwise, jsno will choose to unjsonify ambiguous data
to the first type in the union that matches.DataclassesDataclasses are supported. They get converted to JSON objects. The name of the
class is not included in the result, unless the dataclass is a member of avariant family.Other standard Python typestuplesrangesenumsdate and, time and datetime objects (converted to ISO-formatted strings)timedelta and timezonecomplexdecimal.Decimal (converted to JSON strings)pathlib.Pathre.Patternzoneinfo.ZoneInfouuid.UUIDtypes.SimpleNamespaceLiteral (only int and str literals)NamedTupleTypedDictNewTypeDumps and loadsJsno provides shortcut functionsdumpsandloadswith interface that is
similar to the standardjsonmodule function.jsno.dumpsboth jsonifies its argument
and turns it into a JSON-encoded string, similar to standardjson.dumpsfunction. Correspondinly,jsno.loadsboth decodes and unjsonify JSON data.json=jsno.dumps(date(2023,7,30))assertjson=="2023-07-30"day=jsno.loads[date](json)assertday==date(2023,7,30)Defining custom jsonificationJsno'sjsonifyandunjsonifyare defined assingledispatchfunctions, so
implementations to new types can be registered using a decorator.For example, Python's standard random number generator can be given json representation:fromrandomimportRandom@jsno.jsonify.register(Random)def_(value):# jsonify the random number generator's statestate=value.getstate()returnjsno.jsonify(state)@jsno.unjsonify.register(Random)def_(value,as_type):# first unjsonify the statestate=jsno.unjsonify[tuple[int,tuple[int,...],float|None]](value)# create a new Random object and install the unjsonified stateresult=Random()result.setstate(state)returnresultNow it's possible to for example have a random number generator as a part of a data
structure defined using dataclasses, and have it converted to and from JSON
automatically:@dataclassclassGameState:identifier:strboard:list[tuple[int,int,str]]rng:Randomdefsave_game(database,state:GameState):database.store(state.identifier,jsonify(state))defload_game(database,identifier:str)->GameState:json=database.load(identifier)returnunjsonify[GameState](json)Unjsonify contextBy default, the unjsonify raises an error (UjsonifyError), if the JSON object
that is being converted to a dataclass contains extra keys. This is often the
most appropriate behavior, for example when converting the request body of an API
request, it's best to let the caller know that they might have mistyped property
names in their data. However, sometimes, it's best to just ignore the extra properties,
for example when reading persistent data that could have obsolete properties in it
that are not reflected in the dataclass definition.The extra key behavior can be controlled by running the unjsonification inside
the approproatecontext:withunjsonify.ignore_extra_keys:state=unjsonify[GameState](data)ConstraintsJsno support annotating types with constraints that are boolean valued functions
that must return True for the unjsonified value for it to be valid. For example,
to only accept email addresses that contain the "@" character:fromtypingimportAnnotatedfromjsnoimportConstraint@dataclassclassUser:username:stremail:Annotated[str,Constraint(lambdait:"@"init)]Constraints are always used within Annotated, so there is a shortcut operator to
make the definitions clearer. This is equivalent to the definition above:@dataclassclassUser:username:stremail:str//Constraint(lambdait:"@"init)Using the//-operator, the type of the property is directly after the colon,
not nested inside an Annotated-expression.The most typical constraints are those that limit the value or the length of a
property to a certain range. For these, jsno provides predefined shortcuts:@dataclassclassPlayer:username:str//Constraint.len(min=4,max=16)credits:int//Constraint.range(min=0)There is also a constraint for matching the value with a regular expression:@dataclassclassVariable:name:str//Constraint.regex("[A-Za-z_][A-Za-z0-9_]*")Constraints can be joined using the or-operator|:LiteralInt=Constraint.regex("[0-9]*")LiteralString=Constraint.regex('".*"')@dataclassclassLiteralValue:value:str//(LiteralInt|LiteralString)Constraints can be attached to classes as well:@dataclass@Constraint(lambdarange:range.min<=range.max)classRange:min:intmax:intThe constraint will be validated any time an instance of the class is unjsonified.Customizing field namesIt could be necessary to map the field names of a dataclass to something other
than the ones defined in the Python class. For example, the classes could be
modeling a HTTP API that uses property names that are not compatible with Python's
naming rules, like usinghyphenated-names, or names that are reserved in Python,
such as "class". The field name can be customized with anproperty_nameannotation
that can be attached to a type similar to a constraint:@dataclassclassAPIRequest:class_:str//jsno.property_name("class")instance_count:int//jsno.property_name("instance-count")request=APIRequest(class_="Request",instance_count=1)json=jsonify(request)assertjson=={"class":"Request","instance-count":1}assertunjsonify[APIRequest](json)==requestDynamic unjsonification schemasIt is possible to define ad-hoc unjsonifition schemas without declaring new types,
using thejsno.Schemaclass.For example, defining a schema for HTTP requests:schema=jsno.Schema({"method":str,"url":str,"headers":dict[str,str],"body":bytes,})Note that the types are given asvalues, not asannotationsto the schema.
Unjsonifying with the schema returns a dictionary with the shape similar to
the schema itself.Unjsonification shemas can be useful when the data format is descrirbed by dynamic
data, for example user-specific settings that are retrieved from a database,
instead of being specified in the source code. TypedDict type from python'stypingmodule can be used for similar purposes, butSchemaoffers a
couple of additional options:ignore_extra_keysargument can be given to have the unjsonification discard
unexpected keys, instead of raising an errordefault_typeargument can be given instead, to have the unjsonification accept
any key, unjsonifying the unexpected keys to the default type.extra_data_keyargument can be supplied to control where the unexpected
keys are put. By default, if default type is specified, the extra entries are
inserted to the resulting dictionary. Ifextra_data_keyis supplied, the
extra entries are inserted to a subdictionary pointed byextra_data_keyinstead.totalargument can be set to False in order to make all listed properties
non-required.totalis True by default, so all keys must be present in the
input data. Totality can be overriden for individual properties by annotating
them withtyping.Requiredortyping.NotRequiredsimilarly to their usage
withTypedDict.Function argument schemasA typical application of unjsonification schemas is converting messages encoded
in JSON into function arguments, for example converting JSON-encoded HTTP
request bodies into handler function arguments. For this purpose, jsno provides
theSchema.from_argumentsfunction that analyses the signature of a function
and produces the corresponding unjsonification schema to convert a JSON object
to the keyword argument expected by the function.As an example, here is a simple request handler function signature.defprocess_event(event_type:str,user_id:int,date:datetime):...And here is a JSON object that should be processed by the function:json_object={"event_type":"SIGN-IN","user_id":1002124,"date":"2023-08-26T06:42:11Z"}To feed the object to the handler function, first derive unjsonification schema
and use that to convert the JSON object:schema=jsno.Schema.from_arguments(process_event)kwargs=unjsonify[schema](json_object)process_event(**kwargs)Variant familiesSometimes it's useful to have a hierarchy of classes, consisting of several subclasses
that need to be stored in JSON. A typical example is defining the AST (abstract syntax
tree) for a domain-specific language. Here's the AST of a simple expression language,
defined using a hierarchy of dataclasses:classExpression:pass@dataclassclassLiteralInt(Expression):value:intdefevaluate(self,context):returnself.value@dataclassclassBinaryOperator(Expression):left:Expressionright:ExpressionclassAdd(BinaryOperator):defevaluate(self,context):returnself.left.evaluate(context)+self.right.evaluate(context)classMultiply(BinaryOperator):defevaluate(self,context):returnself.left.evaluate(context)*self.right.evaluate(context)Now, it's possible to create syntax trees, evaluate them, and convert them into JSON:ast=Add(LiteralInt(1),Multiply(LiteralInt(2),Reference("x")))assertast.evaluate({"x":3})==7json=jsno.dumps(ast,indent=4)jsonwill contain:{"left":{"value":1},"right":{"left":{"value":2},"right":{"name":"x"}}}However, reading the AST back from JSON won't work. Trying to unjsonify it as anExpressionfails:jsno.unjsonify[Expression](json)# TypeError: Unjsonify not defined for <class 'Expression'>Even if you defined the base Expression class as a dataclass, jsno wouldn't not which
of the subclasses should it choose.The solution to the problem is to mark the root of the expression as the root of avariant family, using thevariantfamilydecorator:@jsno.variantfamily(label="type")classExpression:passWhen a class is marked to form a variant family, jsno will add a label to the data
jsonfied from it's, or it's subclasses' instances, to identify the class.Adter adding the decorator, the JSON produced byjsonifycontains labels to
differentiate between the subclasses (variants):{"type":"Add","left":{"type":"LiteralInt","value":1},"right":{"type":"Multiply","left":{"type":"LiteralInt","value":2},"right":{"type":"Reference","name":"x"}}}The variant label's name (type) was given in the decorator.Now, jsno is able to unjsonify the AST:unjsonified_ast=jsno.unjsonify[Expression](json)assertunjsonified_ast==astBy default, the label for a class is taken from the class name. It's also possible
to give it explicitly using thevariantlabeldecorator:@jsno.variantlabel('mult')classMultiply(Expression):# ...InstallationInstall jsno with pip:pipinstalljsnoJsno has no 3rd party dependencies.Release NotesVersion 1.2.3 (2023-12-26)fix bug in error messages with open-ended range constraintfix ujsonifying as JSON typeturn value errors to unjsonify errors by default in unjsonifiersVersion 1.2.2 (2023-10-04)export jsno.typecheck functionfix a bug that prevented dataclasses that ignore extra keys to be jsonifiedVersion 1.2.1 (2023-09-03)allow using schemas in unjsonify type argumentshandle**kwargsin function argument schemasjsonify datetime.time as hour:minute if seconds is 0remove extra zeros from jsonified timedeltasVersion 1.2.0 (2023-08-20)ignore non-field members of dataclasses when unjsonifyingbetter error messagesspecial handling for optionalsmulti-label variantsoptional inclusion of variant labels in unjsonized dataJSON type aliasderiving unjsonification schema from function signatureVersion 1.1.5 (2023-08-14)removes a single debug printVersion 1.1.4 (2023-08-14)class constraintsVersion 1.1.3 (2023-08-14)support for types.SimpleNamespace and uuid.UUIDorphan families for tagged variants without familyunjsonification schemas without a typeVersion 1.1.2 (2023-08-09)support for Self typesupport for recursive and mutually recursive dataclasseshandle type hints in dataclasses given as stringsVersion 1.1.1 (2023-08-07)property name annotationsfix optional value jsonification when no defaultperformance improvement for dataclass jsonificationVersion 1.1.0 (2023-08-07)regular expression constraintsoperator // for constraintsjoining constraints with | operatoranonymous Record typessupport for datetime.timezone and datetime.timedeltasupport for re.Patternsupport for TypedDict and NewTypesupport for untyped tuples and namedtuplesunjsonify context for ignoring extra propertiesspecifying target for extra properties in dataclassesperformance improvements for unjsonifyVersion 1.0.7 (2023-08-04)support for typing.Annotatedconstraint annotationsVersion 1.0.6 (2023-08-03)support for datetime.time