Package: buildgraph
Build GraphInstallingInstall withpip install buildgraphImport withfrom buildgraph import BaseStep, buildgraphIntroductionBuild Graph provides a set of tools to run build steps in order of their dependencies.Build graphs can be constructed by hand, or you can let the library construct the graph for you.Here's an example of a graph:@buildgraph()defgetGraph(prod=True):RunTest("api")RunTest("client")package=BuildPackage(prod)ifprod:UploadPackage(package)else:WritePackageToFile(package,"packages/")if__name__=="__main__":test_graph=getGraph(False,config={"code_root":"src"})test_graph.printExecutionOrder()test_graph.run()In the following examples, we'll be using this step definition:classAdder(BaseStep):"""Returns its input plus 1"""defexecute(self,n):new=n+1print(new)returnnewManual constructionDefining stepsSteps are defined by constructing a step definition and binding the required arguments.# This will create a single 'Adder' step with input 5a=Adder(5)Step arguments can be other steps:# This will provide the output from step a as input to step ba=Adder(0).alias("a")# Set an alias to identify the stepsb=Adder(a).alias("b")To run the steps, we pick the last step in the graph and call itsrunmethod....result=b.run()print(result)# 2A step from anywhere in the graph can be run, but only that step's dependencies will be executed.print(a.run())# 1 - Step b won't be runSide dependenciesSometimes you'll need to run a stepabefore stepb, buta's output won't be used byb.classPrinter(BaseStep):defexecute(self,msg):print(msg)p=Printer("Hi")a=Adder(0).alias("a")b=Adder(a).alias("b").after(p)# This ensures b will run after pb.run()Theafter(*steps)method specified steps that must be run first. If multiple steps are provided it doesn't enforce an ordering between those steps.Detatched stepsIf a step is defined but not listed as a dependency it won't be run:a=Adder(0).alias("a")b=Adder(1).alias("b")b.run()# This won't run aYou can check which steps will be run with thegetExecutionOrderandprintExecutionOrdermethods.Circular dependenciesBuildgraph will check for loops in the graph before running it and will raise an exception if one is detected.Automatic constructionThe@buildgraphdecorator builds a graph where every node is reachable from the final node.@buildgraph()defaddergraph():a=Adder(0)b=Adder(b)c=Adder(c)addergraph.run()# This will run all 3 stepsIf the steps don't have dependencies the execution order isn't guaranteed, but the steps that are defined first will be run first unless another dependency enforces a different order.Returning from a graphGraphs can return results from a step too.@buildgraph()defaddergraph():a=Adder(0)b=Adder(a)returnbresult=addergraph().run()print(result)# 2Parameterised graphsGraphs can take input which will be used to construct it.@buildgraph()defloopinggraph(loops):a=Adder(0)foriinrange(loops-1):a=Adder(a)returnalooponce=loopinggraph(1)looponce.run()# 1loopmany=loopinggraph(5)loopmany.run()# 5Graphs which take no config or parameters can be run without explicitly building the graph first:@buildgraph()defsimpleGraph():returnAdder(0)simpleGraph.run()# simpleGraph is a config-less graphsimpleGraph().run()# These two lines are equivalent@buildgraph()defconfigGraph(n):returnAdder(n)configGraph(2).run()# Graphs with config must be built by calling it as a functionNested GraphsGraphs can be nested:@buildgraph()defgetInnerGraph(p):print(f"Building inner graph with input{p}")returnAppendAndPrint(p)@buildgraph()defgetOuterGraph(p):print(f"Building outer graph with input{p}")inner1=getInnerGraph(p,config={"namespace":"inner1"})inner2=getInnerGraph(inner1,config={"namespace":"inner2"})outer=AppendAndPrint(proxy)returnouterUnpacking Return TypesGraphs can return multiple values which can be unpacked:@buildgraph()defgraph():a=Adder(1)b=Adder(a)returna,bfirst,second=graph.run()Extending stepsAll steps must inherit fromBaseStepand implement anexecutemethod.You can see example steps frombuildgraph/steps.py. These steps can also be imported and used in code.Steps can take variable positional and keyword arguments:classVarStep(BaseStep):defexecute(self,*args,x=0,**kwargs):total=sum(args)+x+sum(kwargs.values())print(total)VarStep(1,2,3,x=4,y=5,z=6).run()Shared ConfigSteps can receive a config object before running that other steps can share.classConfigStep(BaseStep):defconfigure(self,config):self.username=config['username']defexecute(self):print(f"My name is{self.username}")@buildgraph()defgetGraph():ConfigStep()ConfigStep()graph=getGraph(config={"username":"bob"})graph.run()# Both steps will print 'bob'Exception HandlingExceptions thrown inside steps will be caught, printed and the re-raised inside aStepFailedExceptionobject alongwith the
step and the arguments passed the the execute function.After handling an exception execution of further steps will stop.Type checkingBuildgraph will perform type checking when the graph is built if theexecutemethod has type annotations on its parameters.Configuring buildgraphBy default buildgraph prints coloured output. You can disable this withbuildgraph.setColor(False).ExamplesSee the scripts inexamples/for examples for more complex graphs.