Package: openhexa.sdk
Open-source Data integration platformOpenHEXA Python SDKOpenHEXA is an open-source data integration platform developed byBluesquare.Its goal is to facilitate data integration and analysis workflows, in particular in the context of public health
projects.Please refer to theOpenHEXA wikifor more information about OpenHEXA.This repository contains the code of the OpenHEXA SDK, a library allows you to write code for the OpenHEXA platform.
It is particularly useful to write OpenHEXA data pipelines, but can also be used in the OpenHEXA notebooks environment.The OpenHEXA wiki has a section dedicated to the SDK:Using the OpenHEXA SDK.RequirementsThe OpenHEXA SDK requires Python version 3.9 or newer, but it is not yet compatible with Python 3.12 or later versions.If you want to be able to run pipeline in a containerized environment on your machine, you will needDocker.QuickstartHere's a super minimal example to get you started. First, create a new directory and a virtual environment:mkdiropenhexa-pipelines-quickstartcdopenhexa-pipelines-quickstart
python-mvenvvenvsourcevenv/bin/activateYou can then install the OpenHEXA SDK:pipinstall--upgradeopenhexa.sdk💡New OpenHEXA SDK versions are released on a regular basis. Don't forget to update your local installations withpip install --upgradefrom times to times!Now that the SDK is installed withing your virtual environmentYou can now use theopenhexaCLI utility to create
a new pipeline:openhexapipelinesinit"My awesome pipeline"Great! As you can see in the console output, the OpenHEXA CLI has created a new directory, which contains the basic
structure required for an OpenHEXA pipeline. You can nowcdin the new pipeline directory and run the pipeline:cdmy_awesome_pipeline
pythonpipeline.pyCongratulations! You have successfully run your first pipeline locally.If you inspect the actual pipeline code, you will see that it doesn't do a lot of things, but it is still a perfectly
valid OpenHEXA pipeline.Let's publish to an actual OpenHEXA workspace so that it can run online.Using the OpenHEXA web interface, within a workspace, navigate to the Pipelines tab and click on "Create".Copy the command displayed in the popup in your terminal:openhexaworkspacesadd<workspace>You will be prompted for an authentication token, you can find it in the popup as well.After adding the workspace using the CLI, you can now push your pipeline:openhexapipelinespushAs it is the first time, the CLI will ask you to confirm the creation operation. After confirmation the console will
output the link to the pipeline screen in the OpenHEXA interface.You can now open the link and run the pipeline using the OpenHEXA web interface.ContributingThe following sections explain how you can set up a local development environment if you want to participate to the
development of the SDK.SDK development setupInstall the SDK in editable mode:python-mvenvvenv# Create a virtual environment for this projectsourcevenv/bin/activate# Activate the venvpipinstall-e".[dev]"# Necessary to be able to run the openhexa CLIUsing a local installation of OpenHEXA to run pipelinesWhile it is possible to run pipelines locally using only the SDK, if you want to run OpenHEXA in a more realistic
setting you will need to install the OpenHEXA app and frontend components. Please refer to theinstallation instructionsfor more information.You can then configure the OpenHEXA CLI to connect to your local backend:openhexaconfigset_urlhttp://localhost:8000Notes: you can monitor the status of your pipelines usinghttp://localhost:8000/pipelines/statusRunning the testsYou can run the tests using pytest:pytest