Package: artifax
artifaxartifax is a Python package to evaluate nodes in a computation graph where
the dependencies associated with each node are extracted directly from their
function signatures. Here's an example ofartifaxin action:>>>fromartifaximportbuild>>>importmath>>>build({...'x':math.pi/3,...'sin':lambdax:math.sin(x),...'cos':lambdax:math.cos(x),...'y':lambdasin,cos:2*sin-1.2*cos...}){'x':1.0471975511965976,'sin':0.8660254037844386,'cos':0.5000000000000001,'y':1.132050807568877}Install with pip:$python3-mpipinstallartifaxA computation graph can be entirely encoded in a standard python dictionary.
Each key represents a node or an artifact, that will eventually be computed
once all of its dependencies have been calculated. The value associated with
each key can be either a constant - a string, a number or an instance of a class,
or a function. In the latter case, the function arguments map to other nodes
in the computation graph to establish a direct dependency between the nodes.For example, the following dictionary:artifacts={'A':42,'B':7,'C':lambda:10,'AB':lambdaA,B:A*B,'C-B':lambdaB,C:C-B,'greeting':'Hello','message':lambdagreeting,A:'{}World! The answer is{}.'.format(greeting,A)}yields the following computation graph:Figure 1. Example of a computation graph.Thebuildfunction evalutes the entire computation graph and returns a new dictionary
with the same keys as the original one and with the calculated values for each of the nodes
in the computation graph.fromartifaximportbuildartifacts={'A':42,'B':7,'C':lambda:10,'AB':lambdaA,B:A*B,'C-B':lambdaB,C:C-B,'greeting':'Hello','message':lambdagreeting,A:'{}World! The answer is{}.'.format(greeting,A)}result=build(artifacts)fork,vinresult.items():print(f"{k:<10}:{v}")outputsA:42B:7C:10AB:294C-B:3greeting:Hello
message:HelloWorld!Theansweris42.Artifax classThebuildfunction represents the core transformation that yields artifacts.
It is entirely stateless and has no side-effects. Given the same input graph, it will always
evaluate every single node and generate the same results.Whilst these features are highly desirable from any core component, the statefulArtifaxclass can be employed to interface with the build function and provide some additional features
and performance enhancements.fromartifaximportArtifax,Atdefdouble(x):returnx*2afx=Artifax()afx.set('a',42)afx.set('b',At('a',double))# set also accepts named argumentsafx.set(c=lambdab:-b)assertlen(afx)==3assert'b'inafxresults=afx.build()fork,vinresults.items():print(k,v)# c -84# a 42# b 84Lazy buildsArtifax instances optimize sequential builds by only re-evaluating nodes that
have become stale due to an update. For example, given the graph illustrated in
Figure 1, if nodeBis updated, e.g,afx.set('B', -5)), nodesB,ABandC-Bget re-evaluated when the build method is invoked, but not any other
nodes.In the example below, the second call to thebuildmethod triggers a
re-evaluation of nodep1and all the nodes that depend on it. Nodesv2andm2, on the other hand, do not require re-evaluation since they do not depend
on the updated node.importartifaximportmathclassVector:def__init__(self,u,v):self.u=uself.v=vdefmagnitude(self):print('Calculating magnitude of vector{}...'.format(self))returnmath.sqrt(self.u**2+self.v**2)def__repr__(self):return'({},{})'.format(self.u,self.v)afx=artifax.Artifax(p1=(3,4),v1=lambdap1:Vector(*p1),m1=lambdav1:v1.magnitude(),v2=Vector(5,12),m2=lambdav2:v2.magnitude())_=afx.build()print('Updating p1...')afx.set(p1=(1,1))_=afx.build()Calculating magnitude of vector (3, 4)...
Calculating magnitude of vector (5, 12)...
Updating p1...
Calculating magnitude of vector (1, 1)...Targeted buildsThebuildmethod accepts an optional argument that specifies which node in
your computation graph should be built. Instead of returning the usual dictionary,
targeted builds return a tuple containing the value associated with each of the
target nodes.terminal_node_value=afx.build(targets='terminal_node')some_node,another_node=afx.build(targets=('node1','node2'))Targeted builds only evaluate dependencies for the target node and the target node itself.
Any other nodes in the computation graph do not get evaluated.fromartifaximportArtifaxafx=Artifax({'name':'World','punctuation':'?','greeting':lambdaname,punctuation:'Hello,{}{}'.format(name,punctuation),})greeting=afx.build(targets='greeting')print(greeting)# prints "Hello, World?"afx.set('punctuation','!')greeting,punctuation=afx.build(targets=('greeting','punctuation'))print(greeting)# prints "Hello, World!"print('Cool beans{}'.format(punctuation))# prints "Cool beans!"Targeted builds are an efficient way of retrieving certain nodes without
evaluating the entire computation graph.SolversDepending on the use case, different solvers can be employed to increase performance.
Thebuildfunction and methods accept an optionalsolverparameter which defaults tolinear.ThelinearsolverThe linear solver topologically sorts the computation graph to define a sequence
of nodes to be calculated in an order such that for any node, all of its dependencies appear
before in the sequence.TheparallelsolverTheparallelsolver consumes the computation graph starting from the nodes that have
no dependencies and processes them all in parallel. When this initial set of nodes is resolved,
their immediate neighbors make up the new frontier which also gets processed in parallel.
This procedure continues until there are no more nodes to be calculated. At any step, the
solver spawns one new process for each node at the frontier without exceeding the number of
available cores minus 1.TheasyncsolverTheasyncsolver takes the parallelism of theparallelsolver one step further. It is triggered
each time a node evaluation is completed, looking for new nodes that can be started and evaluating
them in a new process immediately.Error handlingIf the computation graph represented by the artifacts dictionary is not a DAG
(Direct Acyclic Graph), aCircularDependencyErrorexception is thrown.importartifaxtry:_=artifax.build({'x':lambdax:x+1})exceptartifax.CircularDependencyErroraserr:print('Cannot build artifacts:{}'.format(err))Cannot build artifacts: artifact graph is not a DAGIf a particular node is represented by a function for which any of its arguments isn't part
of the computation graph, anUnresolvedDependencyErrorexception is thrown._=artifax.build({'x':42,'p':lambdax,y:x+y})# raises UnresolvedDependencyError due to missing 'y' nodeHowever, sometimes this behavior might be desirable if we want nodes to resolve to partially
applied functions that can be used elsewhere. If that's the case, the exception can be suppressed
by setting theallow_partial_functionsoptional parameter tobuildtoTrue.results=artifax.build({'x':42,'p':lambdax,y:x+y},allow_partial_functions=True)print(results['p'](100))# prints 142