Package: lazy_python
I will write this later…What is lazy?lazyis a module for making pythonlazily
evaluated(kinda).lazyruns under python 3.5 and 3.4.Why lazy?Why not lazy?I think lazy computation is pretty cool, I also think python is pretty
cool; combining them is double cool.How to lazy?There are 3 means of using lazy code:lazy_functionrun_lazyIPython cell and line magicslazy_functionlazy_functiontakes a python function and returns a new function that is
the lazy version. This can be used as a decorator.Example:@lazy_functiondeff(a,b):returna+bCallingf(1, 2)will return athunkthat will add 1 and 2 when it
needs to be strict. Doing anything with the returned thunk will keep
chaining on more computations until it must be strictly evaluated.Lazy functions allow for lexical closures also:@lazy_functiondeff(a):defg(b):returna+breturngWhen we callf(1)we will get back athunklike we would expect;
however, this thunk is wrapping the functiong. Becausegwas created
in a lazy context, it will also be alazy_functionimplicitly. This means
thattype(f(1)(2))isthunk; but,f(1)(2)== 3.We can use strict to strictly evaluate parts of a lazy function, for example:>>>@lazy_function...defno_strict():...print('test')...>>>strict(no_strict())In this example, we never forced print, so we never saw the result of the call.
Consider this function though:>>>@lazy_function...defwith_strict():...strict(print('test'))...>>>strict(with_strict())test>>>result=with_strict()>>>strict(result)testHere we can see how strict works inside of a lazy function.strictcauses
the argument to be strictly evaluated, forcing the call to print. We can also
see that just callingwith_strictis not enough to evaluate the function,
we need to force a dependency on the result.This is implemented at the bytecode level to frontload a large part of the cost
of using the lazy machinery. There is very little overhead at function call
time as most of the overhead was spent at function creation (definiton) time.run_lazyWe can convert normal python into lazy python with therun_lazyfunction
which takes a string, the ‘name’, globals, and locals. This is likeexecorevalfor lazy python. This will mutate the provided globals and locals so
that we can access the lazily evaluated code.Example:>>>code="""
print('not lazy')
strict(print('lazy'))
""">>>run_lazy(code)lazyThis also uses the same bytecode manipulation aslazy_functionso they will
give the same results.IPython cell and line magicsIf you have IPython installed, you may use the cell and line magic machinery to
write and evaluate lazy code. For example:In[1]:fromlazyimportstrictIn[2]:%lazy2+2# line magic acts as an expressionOut[2]:4In[3]:type(_2)Out[3]:lazy._thunk.thunkIn[4]:%%lazy# cell magic is treated as a statement...:print('lazy')...:strict(print('strict'))...:strictthunkAt its core, lazy is just a way of converting expressions into a tree
of deferred computation objects calledthunks. thunks wrap normal
functions by not evaluating them until the value is needed. Athunkwrapped function can acceptthunks as arguments; this is how the
tree is built. Some computations cannot be deferred because there is some state
that is needed to construct the thunk, or the python standard defines the
return of some method to be a specific type. These are refered to as ‘strict
points’. Examples of strict points arestrandboolbecause the python
standard says that these functions must return an instance of their own
type. Most of these converters are strict; however, some other things are
strict because it solves recursion issues in the interpreter, like accessing__class__on a thunk.Custom Strictness Propertiesstrictis actually a type that cannot be put into athunk. For
example:>>>type(thunk(strict,2))intNotice that this is not a thunk, and has been strictly evaluated.To create custom strict objects, you can subclassstrict. This
prevents the object from getting wrapped in thunks allowing you to
create strict data structures.Objects may also define a__strict__method that defines how to
strictly evalueate the object. For example, an object could be defined
as:classStrictFive(object):def__strict__(self):return5This would makestrict(StrictFive())return 5 instead of an instance
ofStrictFive.undefinedundefinedis a value that cannot be strictly evaluated. It is useful as a
placeholder for computations.We can imagineundefinedin python as:@thunk.fromexpr@Exception.__new__classundefined(Exception):def__strict__(self):raiseselfThis object will raise an instance of itself when it is evaluated.
This is presented as an equivalent definition, though it is actually in c to
make nicer stack traces.Known IssuesCurrently, the following things are known to not work:Recursively definedthunksA recursively definedthunkis a thunk that appears in its own graph twice.
For example:>>>a=thunk(lambda:a)>>>strict(a)This will cause an infinite loop because in order to strictly evaluatea,
we will call the function which returnsawhich we will try to strictly
evaluate.Status: Bug, might fix.This is basically correct, for example:>>>a=lambda:a()>>>a()...RuntimeError:maximumrecursiondepthexceededThe difference in the thunk example is that we will drop into c code to preform
the recursion so it will not terminate in a reasonable amount of time.The potential fix could be to try to detect these cycles and raise some
Exception; however, this might be a very expensive check in the good case
makingthunkevaluation much slower.GotchasI opened it up in the repl, everything is strict!Because the python spec says the__repr__of an object must return astr, a call toreprmust strictly evaluate the contents so that
we can see what it is. The repl will implicitly callrepron things
to display them. We can see that this is a thunk by doing:>>>a=thunk(operator.add,2,3)>>>type(a)lazy.thunk.thunk>>>a5Again, because we need to compute something to represent it, the repl is
a bad use case for this, and might make it appear at first like this is
always strict.printdidn’t do anything!Um, what did you think it would do?If we write:@lazy_functiondeff(a,b):print('printing the sum of%sand%s'%(a,b))returna+bThen there is no reason that the print call should be executed. No
computation depends on the results, so it is casually skipped.The solution is to force a dependency:@lazy_functiondeff(a,b):strict(print('printing the sum of%sand%s'%(a,b)))returna+bstrictis a function that is used to strictly evaluate things.
Because the body of the function is interpreted as lazy python, the
function call is converted into athunk, and therefore we canstrictit.This is true foranyside-effectful function call.x is being evaluated strictly when I think it should be lazyThere are some cases where things MUST be strict based on the python
language spec. Because this is not really a new language, just an
automated way of writing really inefficient python, python’s rules must
be followed.For example,__bool__,__int__, and other converters expect that
the return type must be a the proper type. This counts as a place where
strictness is needed1.This might not be the case though, instead, I might have missed
something and you are correct, it should be lazy. If you think I missed
something, open an issue and I will try to address it as soon as
possible.Some stateful thing is brokenSorry, you are using unmanaged state and lazy evaluation, you deserve
this.thunkscache the normal form so that calling strict the second
time will refer to the cached value. If this depended on some stateful
function, then it will not work as intended.I tried to do x with athunkand it broke!The library is probably broken. This was written on a whim and I barely
thought through the use cases.Please open an issue and I will try to get back to you as soon as
possible.NotesThe function call for the constructor will be made lazy in theLazyTransformer(likethunk(int, your_thunk)), so while this
is a place where strictness is needed, it can still be ‘optimized’
away.