Package: paragraph
A pure Python micro-framework supporting seamless lazy and concurrent evaluation of computation graphs.IntroductionParagraph adds thefunctional programming paradigmto Python in a minimal fashion. One additional class,Variable, and a
function decorator,op, is all it takes to turn regular Python code into acomputation graph, i.e. a computer representation of a system of
equations:>>> import paragraph as pg
>>> import operator
>>> x, y = pg.Variable("x"), pg.Variable("y")
>>> add = pg.op(operator.add)
>>> s = add.op(x, y)The few lines above fully instantiate a computation graph, here in its simplest form with just one equation relatingx,yandsvia the functionadd. Given values for the input variablesxandy, the value ofsis resolved as follows:>>> pg.evaluate([s], {x: 5, y: 10})
[15]Key featuresThe main benefits of using paragraph stem from the following features ofpg.session.evaluate:Lazy evaluationIrrespective of the size of the computation graph, only the operations required to evaluate the output variables are executed. Consider the following
extension of the above graph:>>> z = pg.Variable("z")
>>> t = add.op(y, z)Then the statement:>>> pg.evaluate([t], {y: 10, z: 50})
[60]just ignores the variablessandxaltogether, since they do not contribute to the evaluation oft. In particular, the operationadd(x, y)is not executed.Eager curryingInvoking an op with invariable arguments (that is, arguments that are not of typeVariable) just returns an invariable value: evaluation is
eager whenever possible. If invariable arguments are provided for a subset of the input variables, the computation graph can be simplified usingsolve,
which returns a new variable:>>> u_x = pg.solve([u], {y: 10, z: 50})[0]Here,u_xis a different variable fromu: it now depends on a single input variable (x), and it knows nothing about a variableyorz,
instead storing a reference to the value of their sumt, i.e.60.Thus,pg.session.solveacts much asfunctools.partial, except it simplifies the system of equations where possible by executing dependent
operations whose arguments are invariable.Graph compositionAssume a variableydepends on a number of input variablesx_1,…,x_p, and another variablevonu_1,…,``u_q`` (not necessarily
different), andvshould be identified tox_p. The following statement:>>> y_v = pg.solve([y], args={x_p: v})[0]returns a new variabley_vthat depends onx_1,…,x_{p-1}as well as onu_1,…,u_q, as if the two computation graphs definingyandvhad been pieced together.Note that the respective input variables may overlap, with the restriction thatvshould not depend onx_pas that would result in a circular
dependency. Also, additional arguments may be added toargsin the statement above to set further values of the input variablesx_1,…,x_{p-1}. However, values cannot be set foru_1,…,u_qhere, since they are not dependencies ofy, but ofy_v.Transparent multithreadingInvokingevaluateorsolvewith an instance ofconcurrent.ThreadPoolExecutorwill allow independent blocks of the computation graph to run in
separate threads:>>> with ThreadPoolExecutor as ex:
...     res = pg.evaluate([z_t], {t: 5}, ex)This is particularly beneficial if large subsets of the graph are independent.ConstraintsSide-effectsThe features listed above come at some price, essentially because the order in which operations are actually executed generally differs from the order of
their invocations. For paragraph to guarantee that a variable always evaluates to the same value given the same inputs, as in a system of mathematical
equations, it is paramount that operations remain free of side-effects, i.e. theynevermutate an object they received as an argument, or store as an
attribute. The state sequence of the object would be, by definition, out of the control of the programmer.There is close to nothing paragraph can do to prevent such a thing happening. When in doubt, make sure to operate on a copy of the argument.TypingVariables do not carry any information regarding the type of the value they represent, which precludes binding a method of the underlying value to an
instance ofVariable: such instructions can appear only within the code of an op. Since binary operators are implemented using special methods in
Python, this also precludes such statements as:>>> s = x + yfor this would be resolved by the Python interpreter intos = x.__add__(y), thens = y.__radd__(x), yet none of these methods is defined byVariable.For more information please consult thedocumentation.