Package: pragma
OverviewPyPragma is a set of tools for performing in-place code modification, inspired by compiler directives in C. These modifications are intended to make no functional changes to the execution of code. In C, this is used to increase code performance or make certain tradeoffs (often between the size of the binary and its execution speed). In Python, these changes are most applicable when leveraging code generation libraries (such as Numba or Tangent) where the use of certain Python features is disallowed. By transforming the code in-place, disallowed features can be converted to allowed syntax at runtime without sacrificing the dynamic nature of python code.For example, with Numba, it is not possible to compile a function which dynamically references and calls other functions (e.g., you may not select a function from a list and then execute it, you may only call functions by their explicit name):fns=[sin,cos,tan]@numba.jitdefcall(i,x):returnfns[i](x)# Not allowed, since it's unknown which function is getting calledIf the dynamism is static by the time the function is defined, such as in this case, then these dynamic language features can be flattened to simpler features that such code generation libraries are more likely to support (e.g., the function can be extracted into a closure variable, then called directly by that name):fns=[sin,cos,tan]fns_0=fns[0]fns_1=fns[1]fns_2=fns[2]@numba.jitdefcall(i,x):ifi==0:returnfns_0(x)ifi==1:returnfns_1(x)ifi==2:returnfns_2(x)Such a modification can only be done by the programmer if the dynamic features are knownbeforeruntime, that is, iffnsis dynamically computed, then this modification cannot be performed by the programmer, even though this example demonstrates the the original function is not inherently dynamic, it just appears so. PyPragma enables this transformation at runtime, which for this example function would look like:fns=[sin,cos,tan]@numba.jit@pragma.deindex(fns,'fns')@pragma.unroll(num_fns=len(fns))defcall(i,x):forjinrange(num_fns):ifi==j:returnfns[j](x)# Still dynamic call, but decorators convert to staticThis example is converted, in place and at runtime, to exactly the unrolled code above.DocumentationComplete documentation can be found over atRTFD.InstallationAs usual, you have the choice of installing from PyPi:pipinstallpragmaor directly from Github:pipinstallgit+https://github.com/scnerd/pypragmaUsagePyPragma has a small number of stackable decorators, each of which transforms a function in-place without changing its execution behavior. These can be imported as such:importpragmaEach decorator can be applied to a function using either the standard decorator syntax, or as a function call:@pragma.unrolldefpows(i):forxinrange(3):yieldi**xpows(5)# Is identical to...defpows(i):forxinrange(3):yieldi**xpragma.unroll(pows)(5)# Both of which become...defpows(i):yieldi**0yieldi**1yieldi**2pows(5)Each decorator can be used bare, as in the example above, or can be given initial parameters before decorating the given function. Any non-specified keyword arguments are added to the resulting function’s closure as variables. In addition, the decorated function’s closure is preserved, so external variables are also included. As a simple example, the above code could also be written as:@pragma.unroll(num_pows=3)defpows(i):forxinrange(num_pows):yieldi**x# Or...num_pows=3@pragma.unrolldefpows(i):forxinrange(num_pows):yieldi**xCertain keywords are reserved, of course, as will be defined in the documentation for each decorator. Additionally, the resulting function is an actual, proper Python function, and hence must adhere to Python syntax rules. As a result, some modifications depend upon using certain variable names, which may collide with other variable names used by your function. Every effort has been made to make this unlikely by using mangled variable names, but the possibility for collision remains.A side effect of the proper Python syntax is that functions can have their source code retrieved by any normal Pythonic reflection:In[1]:@pragma.unroll(num_pows=3)...:defpows(i):...:forxinrange(num_pows):...:yieldi**x...:In[2]:pows??Signature:pows(i)Source:defpows(i):yieldi**0yieldi**1yieldi**2File:/tmp/tmpmn5bza2jType:functionAs a utility, primarily for testing and debugging, the source code can be easily retrieved from each decoratorinsteadof the transformed function by using thereturn_source=Trueargument.Quick ExamplesCollapse LiteralsIn[1]:@pragma.collapse_literals(x=5)...:deff(y):...:z=x//2...:returny*10**z...:In[2]:f??Signature:f(y)Source:deff(y):z=2returny*100De-index ArraysIn[1]:fns=[math.sin,math.cos,math.tan]In[2]:@pragma.deindex(fns,'fns')...:defcall(i,x):...:ifi==0:...:returnfns[0](x)...:ifi==1:...:returnfns[1](x)...:ifi==2:...:returnfns[2](x)...:In[3]:call??Signature:call(i,x)Source:defcall(i,x):ifi==0:returnfns_0(x)ifi==1:returnfns_1(x)ifi==2:returnfns_2(x)Note that, while it’s not evident from the above printed source code, each variablefns_Xis assigned to the value offns[X]at the time when the decoration occurs:In[4]:call(0,math.pi)Out[4]:1.2246467991473532e-16# AKA, sin(pi) = 0In[5]:call(1,math.pi)Out[5]:-1.0# AKA, cos(pi) = -1UnrollIn[1]:p_or_m=[1,-1]In[2]:@pragma.unroll...:deff(x):...:forjinrange(3):...:forsigninp_or_m:...:yieldsign*(x+j)...:In[3]:f??Signature:f(x)Source:deff(x):yield1*(x+0)yield-1*(x+0)yield1*(x+1)yield-1*(x+1)yield1*(x+2)yield-1*(x+2)InlineIn[1]:defsqr(x):...:returnx**2...:In[2]:@pragma.inline(sqr)...:defsqr_sum(a,b):...:returnsqr(a)+sqr(b)...:In[3]:sqr_sum??Signature:sqr_sum(a,b)Source:defsqr_sum(a,b):_sqr_0=dict(x=a)# Prepare for 'sqr(a)'for____in[None]:# Wrap function in block_sqr_0['return']=_sqr_0['x']**2# Compute returned valuebreak# 'return'_sqr_return_0=_sqr_0.get('return',None)# Extract the returned valuedel_sqr_0# Delete the arguments dictionary, the function call is finished_sqr_0=dict(x=b)# Do the same thing for 'sqr(b)'for____in[None]:_sqr_0['return']=_sqr_0['x']**2break_sqr_return_1=_sqr_0.get('return',None)del_sqr_0return_sqr_return_0+_sqr_return_1# Substitute the returned values for the function callsStacking TransformationsThe above examples demonstrate how to performpragmatransformations to a function. It should be especially noted, however, that since each transformer returns a proper Python function, they can stack seamlessly:In[1]:defmake_dynamic_caller(*fns):...:@pragma.deindex(fns,'fns')...:@pragma.unroll(num_fns=len(fns))...:defdynamic_call(i,x):...:forjinrange(num_fns):...:ifi==j:...:returnfns[j](x)...:...:returndynamic_callIn[2]:f=make_dynamic_caller(math.sin,math.cos,math.tan)In[3]:f??Signature:f(i,x)Source:defdynamic_call(i,x):ifi==0:returnfns_0(x)ifi==1:returnfns_1(x)ifi==2:returnfns_2(x)File:/tmp/tmpf9tjaffiType:functionIn[4]:g=pragma.collapse_literals(i=1)(f)In[5]:g??Signature:g(i,x)Source:defdynamic_call(i,x):returnfns_1(x)File:/tmp/tmpbze5i__2Type:function