Package: jk-jsonmodel
jk_jsonmodelIntroductionThis python module provides a special parser for JSON files that produces a data model where each value is associated with the location it originated from in the source file. The purpose of this is to simplify instantiation of objects derived from the original JSON data while providing useful error messages if something is wrong.Information about this module can be found here:github.orgpypi.python.orgThe problemSometimes you have a configuration file or you have JSON data returned by a REST API and want to instantiate an object model from this data. However, there might be something wrong with the original JSON data: Some values might be missing or some other value might have the wrong type. This is the case especially if JSON configuration files are modified by hand.If you're parsing JSON you might run in a situation where there is something wrong with your data. For example you might encounter a numeric value where you would expect a string value - or vice versa. However, if you're using the standard python JSON parser you will be able to inform the user that thereis definitivelysuch a type mismatch, but you will not be able to inform the userwhere exactlythis type mismatch occurs in the original data.The solutionThis is whatjk_jsonmodelis about. This module provides a special parser that creates special data objects that still hold the original location of each parsing token internally. Therefore the parser will not produce the standard JSON data model well known in python but substitute objects of similar structure that still maintain the location information.Having these special JSON based data objects you now can this approach: As your software knows what kind of values to expect you simply query these special objects for values of the desired type. Whenever the data encountered does not meet the expectations you express in your queries the special data objects will automatically raise an exception. This exception will then contain a clear explanationwhatwent wrong andwherethe mismatch is located in the original JSON data. By using this approach a user can immediately learn about where exactly he or she needs to correct an input file. This is important information python's classic JSON parser cannot provide.LimitationsA preliminary note. This python modulejk_jsonmodelis designed to assist in parsing JSON data in a very good way. Howerver, it currently isnotdesigned to do this in the most efficient way possible. When writing this module the focus was on convenience of use not on achieving the best JSON parsing performance possible. The latter would be a completely different goal.Please note that in respect to performance this JSON parser will be more expensive than the python's classic JSON parser. If you require a maximum of performance this python module is definitively not what you want to use. But if you require a maximum of simplicity of your code and a maximum of development speed, this python module might be the module of your choice.State of developmentIt is not recommended to use this module in production yet. This module is still under development. The API may still change significantly.There are several development goals that are in conflict with each other:convenient APImodular designviable performanceSome of the approaches currently taken could still change significantly. This could lead to an equally significant change in the API.
The first stable release is to be expected somewhere in Q2 2023.InstallationThis module can be installed via pip:$pipinstalljk_jsonmodelHow to use this moduleSelect the JSON data you want to loadIn this example we assume that the data originates from the following JSON file representing a contact:{"name":"John","surename":"Doe","birthday":{"month":5,"day":23},"phoneNumbers":{"home":"+39 123 45678","office":null,"cell":"+39 234 56789"}}ImportFirst import this module in all files necessary:import jk_jsonmodelWrite your data classesNow let's make use of this module and write a set of data classes that form a hierarchy.Let's start withContactDataRecord:classContactDataRecord(object):def__init__(self,jData:jk_jsonmodel.JMDict):assertisinstance(jData,jk_jsonmodel.JMDict)self.name=jData.getStrE("name")self.surename=jData.getStrE("surename")self.birthday=BirthdayRecord(jData.getDictE("birthday"))if"birthday"injDataelseNoneself.phoneNumbers={}fork,vinjData.getDictE("phoneNumbers").items():self.phoneNumbers[k]=PhoneNumberRecord(k,v)As you can see the class will receive aJMDictobject that represents the root node dictionary/object from the JSON file.
In__init__all data is retrieved using specialized methods and data get's converted to instances ofBirthdayRecordandPhoneNumberRecordas needed.Now for classBirthdayRecord:classBirthdayRecord(object):def__init__(self,jData:jk_jsonmodel.JMDict):assertisinstance(jData,jk_jsonmodel.JMDict)self.month=jData.getIntE("month")self.day=jData.getIntE("day")self.year=jData.getIntN("year")As you can see this is straight forward. We simply retrieve the values.
Please note that there are convenience methods ending withEandNsuch as these:getBoolE(..)getBoolN(..)getIntE(..)getIntN(..)getStrE(..)getStrN(..)...And so on. These methods differ in a single aspect only:Methods ending withEwill raise an exception if no value has been provided in the JSONMethods ending withNwill tolerate anullin JSONThis naming convention allows you to see immediately if a JSON object property is required or if it is optional.Now for classPhoneNumberRecord:classPhoneNumberRecord(object):def__init__(self,key:str,value:jk_jsonmodel.JMValue):assertisinstance(key,str)assertisinstance(value,jk_jsonmodel.JMValue)self.key=keyself.value=value.vStrN()Same thing here with methods ending withEorN.
However, this data class will receive an instance ofJMValueforvalueeven if there is anullin the JSON file.
In such a situationJMValuewill containnullas data while still maintaining the location information of the tokennull.
But because of this herevalue.vStrN()is used to retrieve the value as a string (if it is notnull).Write the main code to load the dataNow for the main part that loads the data:jData=jk_jsonmodel.loadDictModelFromFile("somedata.json")theModel=ContactDataRecord(jData)The first step is to load the JSON into an instance ofJMDict.
As we know that the JSON file should contain a JSON object we safely can callloadDictModelFromFile(..).The next step then is to convert the raw data into our data model.
This is done by passing theJMDictinstance to the constructor method ofContactDataRecordas this
constructor method has been designed specifically in the way of processing the raw data.The result is an initializedContactDataRecordobject with nested subobjects of typePhoneNumerRecordandBirthdayRecord.What if something is wrong?Now let's asume that there is something wrong with the data. We might find afloatvalue instead of aninthere:"birthday":{"month":5.0,"day":23},In this case the parser will emit the following error message:Exception: Expecting value at 'birthday.month' to be an integer (somedata.json:4:3)TODO: Verify checks. If null raise apropriate exception. If not null raise exception with appropriate location.
TODO: Note all keys not retrieved. Implement two methods:
* getSuperfluousKeys()
* errorOnSuperfluousKeys(ignoreKeys:typing.Set[str])Pretty print the instantiated data model (using jk_prettyprintobj)If we would now add somejk_prettyprintobjsuggar we can visualize the resulting data model quite easily by dumping it to STDOUT. Example:<ContactDataRecord(
        name = 'John'
        surename = 'Doe'
        birthday = <BirthdayRecord(
                month = 5
                day = 23
                year = (null)
        )>
        phoneNumbers = {
                'home' : <PhoneNumberRecord(
                        key = 'home'
                        phoneNumber = '+39 123 45678'
                )>,
                'office' : <PhoneNumberRecord(
                        key = 'office'
                        phoneNumber = (null)
                )>,
                'cell' : <PhoneNumberRecord(
                        key = 'cell'
                        phoneNumber = '+39 234 56789'
                )>,
        }
)>For more details aboutjk_prettyprintobjhave a look at the example file(s) and the following URLs:github.orgpypi.python.orgAuthor(s)Jürgen Knauth:pubsrc@binary-overflow.deLicenseThis software is provided under the following license:Apache Software License 2.0