Package: undictify
undictifyPython library providing type-checked function calls at runtimeTable of contentsIntroductionUse case: JSON deserializationDetailsRequirements and InstallationIntroductionLet's start with a toy example:deftimes_two(value):return2*valuevalue=3result=times_two(value)print(f'{value}* 2 =={result}')This is fine, it outputsoutput: 3 * 2 = 6.
But what ifvalueaccidentally is'3'instead of3?
The output will becomeoutput: 3 * 2 = 33, whichmightnot be desired.So you add something likeifnotisinstance(value,int):raiseTypeError(...)totimes_two. This will raise anTypeErrorinstead, which is better.
But you still only recognize the mistake when actually running the code.
Catching it earlier in the development process might be better.
Luckily Python allows to opt-in for static typing by offeringtype annotations.
So you add them andmypy(or your IDE) will tell you about the problem early.deftimes_two(value:int)->int:return2*valuevalue='3'result=times_two(value)# error: Argument 1 to "times_two"# has incompatible type "str"; expected "int"print(f'{value}* 2 =={result}')But you may get into a situation in which there is no useful static type information,
because of values:coming from external non-typed functions (so actually they are of typeAny)were produced by a (rogue) function that returns different types depending on some internal decision (Union[T, V])being provided as aDict[str, Any]etc.deftimes_two(value:int)->int:return2*valuedefget_value()->Any:return'3'value=get_value()result=times_two(value)print(f'{value}* 2 =={result}')At least with theappropriate settings,mypyshould dutifully complain, and now you're left with two options:Drop type-checking (for example by adding# type: ignoreto the end of theresult = times_two(value)line): This however catapults you back into the insane world where2 * 3 == 33.You manually add type checks before the call (or inside oftimes_two) likeif not isinstance(value, int):: This of course does not provide static type checking (because of the dynamic nature ofvalue), but at least guarantees sane runtime behavior.But the process of writing that boilerplate validation code can become quite cumbersome if you have multiple parameters/functions to check.
Also it is not veryDRYsince you already have the needed type information in our function signature and you just duplicated it in the check condition.This is where undictify comes into play. Simply decorate yourtimes_twofunction with@type_checked_call():fromundictifyimporttype_checked_call@type_checked_call()deftimes_two(value:int)->int:return2*valueAnd the arguments oftimes_twowill be type-checked with every call at runtime automatically. ATypeErrorwill be raised if needed.This concept ofruntime type-checks of function calls derived from static type annotationsis quite simple,
however it is very powerful and brings some highly convenient consequences with it.Use case: JSON deserializationImagine your application receives a JSON string representing an entity you need to handle:tobias_json='''{"id": 1,"name": "Tobias","heart": {"weight_in_kg": 0.31,"pulse_at_rest": 52},"friend_ids": [2, 3, 4, 5]}'''tobias=json.loads(tobias_json)Now you start to work with it. Somewhere deep in your business logic you have:name_length=len(tobias['name'])But that's only fine if the original JSON string was well-behaved.
If it had"name": 4,in it, you would get:name_length = len(tobias['name'])
TypeError: object of type 'int' has no len()at runtime, which is not nice. So you start to manually add type checking:ifisinstance(tobias['name'],str):name_length=len(tobias['name'])else:# todo: handle the situation somehowYou quickly realize that you need to separate concerns better,
in that case the business logic and the input data validation.So you start to do all checks directly after receiving the data:tobias=json.loads(...ifisinstance(tobias['id'],int):...ifisinstance(tobias['name'],str):...ifisinstance(...# *yawn*and then transfer it into a type-safe class instance:@dataclassclassHeart:weight_in_kg:floatpulse_at_rest:int@dataclassclassHuman:id:intname:strnick:Optional[str]heart:Heartfriend_ids:List[int]Having the safety provided by the static type annotations (and probably checking your code withmypy) is a great because of all the:bugs that don't make it into PRODmanual type checks (and matching unit tests) that you don't have to writehelp your IDE can now offerbetter understanding people get when reading your codeeasier and more confident refactoringsBut again, writing all that boilerplate code for data validation is tedious (and not DRY).So you decide to use a library that does JSON schema validation for you.
But now you have to manually adjust the schema every time your entity structure changes, which still is not DRY, and thus also brings with it all the typical possibilities to make mistakes.Undictify can help here too!
Annotate the classes@type_checked_constructorand their constructors will be wrapped in type-checked calls.@type_checked_constructor()@dataclassclassHeart:...@type_checked_constructor()@dataclassclassHuman:...(They do not need to bedataclasses. Deriving fromNamedTupleworks too.)Undictify will type-check the construction of objects of typeHeartandHumanautomatically.
(This works for normal classes with a manually written__init__function too.
You just need to provide the type annotations to its parameters.) So you can use the usual dictionary unpacking syntax, to safely convert your untyped dictionary (i.e.,Dict[str, Any]) resulting from the JSON string into your statically typed class:tobias=Human(**json.loads(tobias_json))(Btw this application is the origin of the name of this library.)It throws exceptions with meaningful details in their associated values in case of errors like:missing a fielda field having the wrong typeetc.It also supports optional values being omitted instead of beingNoneexplicitly (as shown in the example with thenickfield).DetailsSometimes, e.g., in case of unpacking a dictionary resulting from a JSON string,
you might want to just skip the fields in the dictionary that your function / constructor does not take as a parameter.
For these cases undictify provides@type_checked_call(skip=True).It also supports valid type conversions via@type_checked_call(convert=True),
which might for example come in handy when processing the arguments of an HTTP request you receive for example in agethandler of aflask_restful.Resourceclass:@type_checked_call(convert=True)deftarget_function(some_int:int,some_str:str)classWebController(Resource):defget(self)->Any:# request.args is something like {"some_int": "4", "some_str": "hi"}result=target_function(**flask.request.args)The values in theMultiDictrequest.argsare all strings, but the logic behind@type_checked_call(convert=True)tries to convert them into the desired target types with reasonable exceptions in case the conversion is not possible.This way a request tohttp://.../foo?some_int=4&some_str=hiwould be handled normally,
buthttp://.../foo?some_int=four&some_str=hiwould raise an appropriateTypeError.Additional flexibility is offered for cases in which you would like to not type-check all calls of a specific function / class constructor, but only some. You can usetype_checked_call()at call site instead of adding the annotation for those:fromundictifyimporttype_checked_calldeftimes_two(value:int)->int:return2*valuevalue:Any='3'resutl=type_checked_call()(times_two)(value)And last but not least, custom converters for specified parameters are also supported:importjsonfromdataclassesimportdataclassfromdatetimeimportdatetimefromundictifyimporttype_checked_constructor,optional_converterdefparse_timestamp(datetime_repr:str)->datetime:returndatetime.strptime(datetime_repr,'%Y-%m-%dT%H:%M:%SZ')@type_checked_constructor(converters={'some_timestamp':optional_converter(parse_timestamp)})@dataclassclassFoo:some_timestamp:datetimejson_repr='{"some_timestamp": "2019-06-28T07:20:34Z"}'my_foo=Foo(**json.loads(json_repr))In case the converter should be applied even if the source type
already matches the destination type, usemandatory_converterinstead ofoptional_converter.Requirements and InstallationYou need Python 3.7 or higher.python3-mpipinstallundictifyOr, if you like to use latest version from this repository:gitclonehttps://github.com/Dobiasd/undictifycdundictify
python3-mpipinstall.LicenseDistributed under the MIT License.
(See accompanying fileLICENSEor athttps://opensource.org/licenses/MIT)