Package: infernet-ml
infernet-mlinfernet-ml is a lightweight library meant to simplify the implementation
of machine learning workflows for models intended for Web3.Library InstallationNOTE: It is strongly recommended that you use a virtual environment for development. The library is has been tested with Python 3.10.To install this library locally for development(i.e. inside an IDE), the easiest path is to runpython-mvenv.venvsource.venv/bin/activate
pipinstall"infernet-ml[all]"which will install all python dependencies, including test requirements (note you may need to installAnvildepending on the test):curl-Lhttps://foundry.paradigm.xyz|bash&&source~/.bashrc&&foundryupAlternatively, you may install with a subset of the targets [proving,classic_inf,llm_inf] depending on your use case. For example, to only install llm inference related dependencies:python-mvenv.venvsource.venv/bin/activate
pipinstall"infernet-ml[llm_inf]"In this case, to be able to run tests, you will need to install the test dependencies, located in requirements-test.txt:pipinstall-rrequirements-test.txt# install Foundry Anvilcurl-Lhttps://foundry.paradigm.xyz|bash&&source~/.bashrc&&foundryupIf you prefer to install requirements manually, we also maintain a requirements.txt file.Alternatively,if you want to modify the library source code directly, and you do not want to use pip install -e (for editable installation), you can install the dependencies from requirements.txt and set the PYTHONPATH:python3-mvenv.venvsource.venv/bin/activatecdinfernet-ml
pipinstall-rrequirements.txt# make sure to set your classpathexportPYTHONPATH=srcNote that for proving, we currently require custom build of EZKL in order to support obtaining the calldata required for verification and data attestation. See the classic_ml_proof_service.Dockerfile for instructions on how to build it.The libraryThere are broadly 2 classes of workflows: inference and training.training workflows provides various workflow base classes intended to be extended and subclassed, and leverages theMLFLowlibrary for
persisting workflow artifacts, and serving classical machine learning models.Inference workflows add hooks to help structure and organize common tasks related machine learning inference, such as preprocessing for prompt tuning or appending output with custom metadata for downstream services. Inference workflows are directly served by the Infernet ML Inference services.ml/drivers/example_workflow_driver demonstrates a simple training workflow that trains a scikit-learnDecisionTreeClassifiermodel based on theUC, Irvine balance scale dataset([balance (left, right, balanced),left_weight,left_distance,right_weight,right_distance]).Proof generation is also demonstrated, with the implementation logic mostly residing in the ml/utils/ezkl_utils.py module.For compilation:note that EZKL currently has a solc version dependency of ^0.8.20 and above.Running Tests locallyPlease install Anvil and test dependencies as needed.
We test using pytest:python3-mvenv.venvsource.venv/bin/activatecdinfernet-ml# if library not installed# pip install -r requirements.txt# pip install -r requirements-test.txt# export PYTHONPATH=srcpython-mpytesttestsRunning Workflow drivers locallypython3-mvenv.venvsource.venv/bin/activatecdinfernet-ml
pipinstall-rrequirements.txtexportPYTHONPATH=src
python3src/ml/drivers/example_workflow_driver.pyRunning mlflow uiBy default, the mlflow runs directory is stored as a subdirectory of ml. To view run artifacts through the web UI, execute the following command in the same parent directory as the mlruns directory:# Run locallymlflowuiRunning mlflow remotelyBy default, mlflow artifacts are stored locally on disk. However, a MLFLow Server instance can be instantiated either locally or remotely. To help facilitate this, a docker file is provided with dependencies for using a postgres/mysql metadata backend and Google Cloud Storage artifact backend.To ensure we connect to the remote mlflow Server, you set
the MLFLOW_TRACKING_URI environment variable with the server address. Note that dotenv files are read to help facilitate testing (place the .env file in the src
directory.)After a container is buit from the docker image, assuming we are using Google GCS and Postgres, you can issue the following command to start the server. SeeMLFLow documentationfor more details.mlflowserver\--default-artifact-rootgs://path/to/your/artifact-store\--backend-store-uripostgresql://yourusername:yourpassword@localhost:5432/postgresBuilding docker service containersIncluded with the workflow class hierarchy are several services that can be used to serve model inference and proving. Official images can be downloaded from ourdocker hub repositories, but if you wish to build them locally:# build containersdockerbuild-t"infernet_ml_llm:local"-fllm_inference_service.Dockerfile.
dockerbuild-t"infernet_ml_proof:local"-fclassic_ml_proof_service.Dockerfile.
dockerbuild-t"infernet_ml_classic:local"-fclassic_ml_inference_service.Dockerfile.Note: if you have custom workflow dependencies, you should create your own image using the provided images from docker hub as base images.To start the containers:# start containerssudodockerrun--name=llm_inf_service-d-p4999:3000--env-filellm_inference_service.env"ritualnetwork/infernet-llm-inference:0.0.4"--bind=0.0.0.0:3000--workers=2sudodockerrun--name=classic_inf_service-d-p4998:3000--env-fileclassic_ml_inference_service.env"ritualnetwork/infernet-classic-inference:0.0.4"--bind=0.0.0.0:3000--workers=2sudodockerrun--name=classic_proof_service-d-p4997:3000--env-fileclassic_proof_service.env"ritualnetwork/infernet-classic-proving:0.0.5"--bind=0.0.0.0:3000--workers=2with environment files configured as per below:Service configurationClassic ML Inference Service (classic_ml_inference_service.env)-CLASSIC_INF_WORKFLOW_CLASS="ml.workflows.inference.torch_inference_workflow.TorchInferenceWorkflow"CLASSIC_INF_WORKFLOW_POSITIONAL_ARGS=[]CLASSIC_INF_WORKFLOW_KW_ARGS={"model_name":"your_model_org/your_model_name"}# the hugging face token is required to download the modelHUGGING_FACE_HUB_TOKEN=YOUR_HUGGING_FACE_TOKENClassic ML Proof Service (classic_proof_service.env)-# The name of the model we are generating proofs forCLASSIC_PROOF_MODEL_NAME=your_model_org/your_model_name# the hugging face token is required to download necessary artifactsHUGGING_FACE_HUB_TOKEN=YOUR_HUGGING_FACE_TOKENLLM Inference Service (llm_inference_service.env)-LLM_INF_WORKFLOW_CLASS="ml.workflows.inference.llm_inference_workflow.LLMInferenceWorkflow"# path of inference HF / Prime inference backendLLM_INF_WORKFLOW_POSITIONAL_ARGS='["http://ENTER-URL-HERE"]'LLM_INF_WORKFLOW_KW_ARGS={}# the hugging face token is required to download necessary artifactsHUGGING_FACE_HUB_TOKEN=YOUR_HUGGING_FACE_TOKENDeployment UtilityFor ML Engineers that already have an existing pipeline and just want to deploy their classic machine learning models with EZKL proving support, please use the deployer.py utility script (located at /src/ml/utilities/deployer.py).If running the script directly, please ensure the infernet-ml library is installed or your PYTHONPATH (see installation instructions above) is set to the source directory. Note that a solidity compiler must be installed, currently version ^0.8.20 and above (you can use (solc-select)[https://github.com/crytic/solc-select] to manage versions):source.venv/bin/activate
solc-selectuse0.8.21--always-install
python3src/ml/utils/deployer.py--modelPATH_TO_YOUR_MODEL--sample_input_jsonPATH_TO_YOUR_INPUTYou can also leverage the docker image:sudodockerrun-it-v.:/appritualnetwork/deployer:0.0.1--modelLOCAL_MODEL.TORCH--sample_input_jsonLOCAL_SAMPLE_INPUT.jsonRun the script with --help for more info.