Package: dycode
Dynamic Code (DyCode): a toolset for dynamic python code manipulationsHave you ever found yourself coding up boilerplates to handle different scenarios? It's most likely that you have thought about using python'sevalorexecin order to decouple some part of your code to be specified and evaluated later on at run-time. But, you have probably also come accross people who discourage you from usingeval/exec. This is because programs witheval/execare considered vulnarable and can be used to execute malicious code.While this is true in general, but in many cases, you do not care about security concerns, and your highest priority is implementing a quick solution to a general problem which could be solved by usingevalorexec. This is where DyCode comes in. It allows you to useevalandexeceffectively by providing utilities to dynamically compile code, lookup variables and lazily evaluate them.Table of ContentsDynamic Code (DyCode): a toolset for dynamic python code manipulationsInstallationUsageDynamic EvaluationVariable LookupDynamic Function EvaluationVariable AssignmentLicenseCitationInstallationpipinstalldycodeUsageDynamic EvaluationYou can usedycode.evalto combine the functionality ofdycode.eval_function(seehere) anddycode.get_value(seehere). You can do as follows:importdycodedycode.eval("math.cos")# <function <lambda> at MEM_ADDRESS> (math is imported through get_value)dycode.eval("math.cos",dynamic_args=True)(2,verbose=True)# 3, verbose is ignored (and math is imported through get_value)dycode.eval("def my_function(x): return x + 1",function_of_interest="my_function")# <function my_function at MEM_ADDRESS>dycode.eval("def my_function(x): return x + y",function_of_interest="my_function",context={"y":2})(2)# 4Variable LookupYou can usedycode.get_valueto lookup a variable as a string and then evaluate it. This is useful when you want to use a variable that is not defined in the current scope. You can do as follows:fromdycodeimportget_valueget_value("math.pi")# 3.141592653589793get_value("math.cos(0)")# won't evaluate, this is not a variable but a call to a variableget_value("math.cos")(0)# 1.0importmathget_value("cos",context=math)# math.cosget_value("something_that_does_not_exist")# raises NameErrorget_value("something_that_does_not_exist",strict=False)# Noneget_valuesupports looking up variables in a module or package in your current working directory as well (as opposed to python'simportwhich only supports looking up variables in the python standard library and installed packages). This is useful when you want to create a script that can be run from anywhere and still be able to access variables in the current working directory.For example, imagine you create your own python package with a runnable script that sorts files in a directory. You can useget_valueto lookup aconfig.sortfunction in the current working directory. This way, you can create aconfig.pyfile in the current working directory and define your own sorting function. Or use a default sorting function if theconfig.pyfile does not exist.Your code would look like this:fromdycodeimportget_valuedefsort_files():sort_function=get_value("config.sort",strict=False)ordefault_sort# do something with sortAlthough this example is somewhat contrived, such a use case is very common in data science and machine learning. Imagine writing a package for training a Deep Learning model. You can useget_valueto lookup custom Dataset classes and model implementations and this way, people can use your package without the need to modifying your code, because they can simply inject their own implementations in places where you have usedget_value.Another potential use case is defining Neural Network layers. You can useget_valueto lookup custom layers and use them in your model. Such as:fromdycodeimportget_valueimporttorchclassMyLinearBlock(torch.nn.Module):def__init__(self,in_features,out_features,activation="torch.nn.ReLU"):super().__init__()self.linear=torch.nn.Linear(in_features,out_features)self.activation=get_value(activation,strict=False)defforward(self,x):x=self.linear(x)x=self.activation(x)ifself.activationelsexreturnxThis way, you can change the activation function by simply changing theactivationargument. For example, you can usetorch.nn.Sigmoidortorch.nn.Tanhor even a custom activation function that you have defined in the current working directory or use one from a 3rd party package.Dynamic Function EvaluationWhat if you want to generate python programs dynamically? Meaning that you have a string that contains python code and you want to inject it into your program.  You can usedycode.eval_functionto evaluate a piece of code and retrieve a function. You can do as follows:fromdycodeimporteval_functioneval_function("lambda x: x + 1")# <function <lambda> at MEM_ADDRESS>eval_function("def my_function(x): return x + 1")# wont work, this is not a function,# but a code block and you need to mention your function_of_interest in that code blockeval_function("def my_function(x): return x + 1",function_of_interest="my_function")# <function my_function at MEM_ADDRESS>eval_functionaccepts three types of function descriptors:A lambda function, e.g.lambda x: x + 1, or a code which evaluates to a callable object,math.cos(in this case, the values being looked up should be present in the evaluation context, more on this later).A code block, which can contain multiple lines and functions, in which case you need to specify the name of the function of interest using thefunction_of_interestargument.eval_functionwill evaluate the code block and retrieve your function of interest.A dictionary of "code", ["context", "function_of_interest"] pairs. Useful when you are usingeval_functionon top of a configuration file, in which case you can specify the code and the context for each function of interest.When evaluating a function descriptor, you can specify a context in which the code will be evaluated. This is useful when you want to use variables that are not defined in the current scope.dycodehas a context registry that you can use as a global context for all your function evaluations.fromdycodeimporteval_function,register_contextimportmathregister_context(math,"math")# register math package as a context# you can also use register(math), which will use the name of the package as the context nameeval_function("math.cos")# <function <lambda> at MEM_ADDRESS> (math is looked up through the context registry)You can also specify a context for each function evaluation using thecontextargument. The context is a dictionary of variable names and their values. You can do as follows:eval_function("def my_function(x): return x + y",function_of_interest="my_function",context={"y":1})(2)# 3You can also optionally setdynamic_args=True, when evaluating a function. This will create a function that intelligently evaluates its arguments, by wrapping it usingdycode.dynamic_args_wrapper. Functions wrapped bydycode.dynamic_args_wrapperpreprocess arguments passed to them, and ignore arguments that are not defined in the function signature. For instance:eval_function("lambda x: x + 1",dynamic_args=True)(2,verbose=True)# 3, verbose is ignoredVariable AssignmentThere are times when you want to assign a variable in a dynamic manner. Meaning that you want to change a variable's value that is not necessarily defined in the current scope. You can usedycode.set_valueto do so.  You can do as follows:fromdycodeimportset_valueset_value("some_package.my_function",lambdax:x+1)# changing the value of pi in math packageset_value("math.pi",3.14)# now if you import math, math.pi will be 3.14importmathmath.pi# 3.14LicenseDyCode is licensed under the MIT License. SeeLICENSEfor the full license text.CitationIf you use DyCode in your research, please cite this repository as described inCITATION.cff.