node-graph-engine 0.1.0
Create node-based workflow
Requires Python: >=3.9
Keywords: graph, workflows

# NodeGraph Engine
[![PyPI version](https://badge.fury.io/py/node-graph-engine.svg)](https://badge.fury.io/py/node-graph-engine)
[![CI](https://github.com/scinode/node-graph-engine/actions/workflows/ci.yaml/badge.svg)](https://github.com/scinode/node-graph-engine/actions/workflows/ci.yaml)
[![codecov](https://codecov.io/gh/scinode/node-graph-engine/branch/main/graph/badge.svg)](https://codecov.io/gh/scinode/node-graph-engine)
[![Docs status](https://readthedocs.org/projects/node-graph-engine/badge)](http://node-graph-engine.readthedocs.io/)



A platform for designing node-based workflows.

## Supported engines

NodeGraph Engine ships with adaptors for a range of orchestration backends. Pick the
engine that best matches your deployment target using the summary below and consult the
documentation links for integration details.

| Engine | Description |
| ------ | ----------- |
| [Direct](https://node-graph-engine.readthedocs.io/en/latest/autogen/quick_start.html) | Run graphs locally inside the current Python process while capturing provenance. |
| [Airflow](https://airflow.apache.org/) | Materialise graphs as Airflow DAGs that can be scheduled and monitored in Apache Airflow. |
| [Dagster](https://dagster.io/docs) | Launch graphs as Dagster jobs and surface executions in the Dagster UI via a configured instance. |
| [Dask](https://node-graph-engine.readthedocs.io/en/latest/engines/dask.html) | Execute node jobs through Dask's threaded scheduler, keeping provenance in the local AiiDA profile while resolving dependencies and nested graphs automatically. |
| [Prefect](https://www.prefect.io/) | Execute graphs as Prefect flows while streaming provenance back to the recorder. |
| [Parsl](https://parsl.readthedocs.io/) | Dispatch tasks to Parsl executors for parallel and distributed execution without losing provenance fidelity. |
| [Redun](https://redun.dev/) | Integrate with Redun workflows while preserving provenance throughout execution. |
| [Jobflow](https://materialsproject.github.io/jobflow/) | Bridge to the Jobflow workflow system for materials science workloads with consistent provenance records. |
| [Executorlib (Pyiron)](https://executorlib.readthedocs.io/en/latest/) | Coordinate tasks with executorlib executors for provenance-rich high-throughput simulations. |

Check the [engine reference](docs/source/engines/index.rst) for a detailed overview of
each adaptor and links to upstream documentation.


```console
    pip install --upgrade --user node_graph_engine
```


## Documentation
Check the [docs](https://node-graph-engine.readthedocs.io/en/latest/) and learn about the features.


## Examples
**A simple math calculation**

```python
from node_graph import NodeGraph, node

@node()
def add(x, y):
    return x + y

@node()
def multiply(x, y):
    return x + y

@node.graph()
def AddMultiply(x, y, z):
    the_sum = add(x=x, y=y).result
    return multiply(x=the_sum, y=z).result

```

## Engines and provenance
Run graphs directly in Python:

```python
from node_graph_engine.direct import DirectEngine

graph = AddMultiply.build(x=1, y=2, z=3)

engine = DirectEngine()
results = engine.run(graph)
```

The generated provenance graph:

</div>

<p align="center">
<img src="docs/source/_static/images/add_multiply.svg" height="600" alt="Provenance Graph Example"/>
</p>


## License
[MIT](http://opensource.org/licenses/MIT)