buildgraph 0.0.7
Tools for building
License: MIT
Requires Python: >=3.7,<4.0

# Build Graph

## Installing

Install with `pip install buildgraph`

Import with `from buildgraph import BaseStep, buildgraph`


## Introduction


Build Graph provides a set of tools to run build steps in order of their dependencies.

Build graphs can be constructed by hand, or you can let the library construct the graph for you.


Here's an example of a graph:

```python
@buildgraph()
def getGraph(prod=True):
    RunTest("api")
    RunTest("client")
    package = BuildPackage(prod)
    if prod:
        UploadPackage(package)
    else:
        WritePackageToFile(package, "packages/")


if __name__ == "__main__":
    test_graph = getGraph(False, config={"code_root": "src"})
    test_graph.printExecutionOrder()
    test_graph.run()
```



In the following examples, we'll be using this step definition:
```python
class Adder(BaseStep):
    """
    Returns its input plus 1
    """
    def execute(self, n):
        new = n + 1
        print(new)
        return new
```

## Manual construction

### Defining steps

Steps are defined by constructing a step definition and binding the required arguments.

```python
# This will create a single 'Adder' step with input 5
a = Adder(5)
```

Step arguments can be other steps:

```python
# This will provide the output from step a as input to step b
a = Adder(0).alias("a")  # Set an alias to identify the steps
b = Adder(a).alias("b")
```

To run the steps, we pick the last step in the graph and call its `run` method.

```python
...
result = b.run()
print(result)  # 2
```

A step from anywhere in the graph can be run, but only that step's dependencies will be executed.

```python
print(a.run())  # 1 - Step b won't be run
```


### Side dependencies

Sometimes you'll need to run a step `a` before step `b`, but `a`'s output won't be used by `b`.

```python
class Printer(BaseStep):
    def execute(self, msg):
        print(msg)

p = Printer("Hi")
a = Adder(0).alias("a")
b = Adder(a).alias("b").after(p)  # This ensures b will run after p
b.run()
```

The `after(*steps)` method specified steps that must be run first. If multiple steps are provided it doesn't enforce an ordering between those steps.


### Detatched steps

If a step is defined but not listed as a dependency it won't be run:

```python
a = Adder(0).alias("a")
b = Adder(1).alias("b")
b.run()  # This won't run a
```

You can check which steps will be run with the `getExecutionOrder` and `printExecutionOrder` methods.


### Circular dependencies

Buildgraph will check for loops in the graph before running it and will raise an exception if one is detected.


## Automatic construction

The `@buildgraph` decorator builds a graph where every node is reachable from the final node.

```python
@buildgraph()
def addergraph():
    a = Adder(0)
    b = Adder(b)
    c = Adder(c)

addergraph.run()  # This will run all 3 steps
```

If the steps don't have dependencies the execution order isn't guaranteed, but the steps that are defined first will be run first unless another dependency enforces a different order.


### Returning from a graph

Graphs can return results from a step too.

```python
@buildgraph()
def addergraph():
    a = Adder(0)
    b = Adder(a)
    return b

result = addergraph().run() 
print(result)  # 2
```


### Parameterised graphs

Graphs can take input which will be used to construct it.

```python
@buildgraph()
def loopinggraph(loops):
    a = Adder(0)
    for i in range(loops-1):
        a = Adder(a)
    return a

looponce = loopinggraph(1)
looponce.run()  # 1

loopmany = loopinggraph(5)
loopmany.run()  # 5
```

Graphs which take no config or parameters can be run without explicitly building the graph first:
```python
@buildgraph()
def simpleGraph():
    return Adder(0)

simpleGraph.run()  # simpleGraph is a config-less graph
simpleGraph().run()  # These two lines are equivalent


@buildgraph()
def configGraph(n):
    return Adder(n)

configGraph(2).run()  # Graphs with config must be built by calling it as a function
```

### Nested Graphs

Graphs can be nested:
```python

@buildgraph()
def getInnerGraph(p):
    print(f"Building inner graph with input {p}")
    return AppendAndPrint(p)

@buildgraph()
def getOuterGraph(p):
    print(f"Building outer graph with input {p}")
    inner1 = getInnerGraph(p, config={"namespace": "inner1"})
    inner2 = getInnerGraph(inner1, config={"namespace": "inner2"})
    outer = AppendAndPrint(proxy)
    return outer
```

### Unpacking Return Types

Graphs can return multiple values which can be unpacked:
```python

@buildgraph()
def graph():
    a = Adder(1)
    b = Adder(a)
    return a, b

first, second = graph.run()
```


## Extending steps

All steps must inherit from `BaseStep` and implement an `execute` method.

You can see example steps from `buildgraph/steps.py`. These steps can also be imported and used in code.

Steps can take variable positional and keyword arguments:

```python
class VarStep(BaseStep):
    def execute(self, *args, x=0, **kwargs):
        total = sum(args) + x + sum(kwargs.values())
        print(total)

VarStep(1, 2, 3, x=4, y=5, z=6).run()
```


### Shared Config

Steps can receive a config object before running that other steps can share.

```python
class ConfigStep(BaseStep):
    def configure(self, config):
        self.username = config['username']

    def execute(self):
        print(f"My name is {self.username}")

@buildgraph()
def getGraph():
    ConfigStep()
    ConfigStep()

graph = getGraph(config = {"username": "bob"})
graph.run()  # Both steps will print 'bob'
```


## Exception Handling

Exceptions thrown inside steps will be caught, printed and the re-raised inside a `StepFailedException` object alongwith the 
step and the arguments passed the the execute function.

After handling an exception execution of further steps will stop.


## Type checking

Buildgraph will perform type checking when the graph is built if the `execute` method has type annotations on its parameters.


## Configuring buildgraph

By default buildgraph prints coloured output. You can disable this with `buildgraph.setColor(False)`.


## Examples

See the scripts in `examples/` for examples for more complex graphs.