Package: mlflow-modal
mlflow-modalA plugin that integratesModalwith MLflow.mlflow-modalenables MLflow users to deploy their models to serverless, scalable endpoints seamlessly with zero infrastructure.Installationpipinstallmlflow-modalThe following packages are required and will be installed with the plugin:"modal-client""mlflow>=1.12.0"UsageThemlflow-modalplugin integrates Modal with the MLFlow deployments API.Before using this plugin, you must set create an account and API Token athttps://modal.com/.You must have the following environment variables configured to allow the plugin to integrate with your Modal workspace:MODAL_TOKEN_IDMODAL_TOKEN_SECRETMODAL_WORKSPACEThe API is summarized below. For full details see the MLflow deployment pluginPython APIandcommand-line interfacedocumentation.Create deploymentDeploy a model built with MLflow as a Modal webhook with the desiredconfiguration parameters; for example,gpuorkeep_warm.  Currently this plugin only supports thepython_functionflavor of MLflow models that expect a numpy array as input and return a numpy array as output. Thepython_functionflavor is the default flavor so this is not required to be specified in commands.Model deployments on Modal create a REST API endpoint athttps://<modal workspace>--mlflow-<deployment name>-predict.modal.run. The endpoint accepts POST requests with a body specifyinginputand returns a response with a body specifyingpredictions. For example:vscode@ccd6dde2dbf5:/workspaces/mlflow-modal$curl-XPOSThttps://workspace--mlflow-add5-predict.modal.run-d'{"input": [[1,2,3,4,5],[6,7,8,9,10]]}'--header"Content-Type: application/json"{"predictions":[[6.0,7.0,8.0,9.0,10.0],[11.0,12.0,13.0,14.0,15.0]]}CLImlflowdeploymentscreate-tmodal-m<modeluri>--name<deploymentname>-Cgpu=T4-Ckeep_warm=truePython APIfrommlflow.deploymentsimportget_deploy_clienttarget_uri="modal"plugin=get_deploy_client(target_uri)plugin.create_deployment(name="<deployment name>",model_uri="<model uri>",config={"gpu":"T4","keep_warm"=True})Update deploymentModify the configuration of a deployed model and/or replace the deployment with a new model URI.CLImlflowdeploymentsupdate-tmodal--name<deploymentname>-Cgpu=A100Python APIplugin.update_deployment(name="<deployment name>",config={"gpu":"A100"})Delete deploymentDelete an existing deployment.CLImlflowdeploymentsdelete-tmodal--name<deploymentname>Python APIplugin.delete_deployment(name="<deployment name>")List deploymentsList the details of all the models deployed on Modal. This includes only models deployed via this plugin.CLImlflowdeploymentslist-tmodalPython APIplugin.list_deployments()Get deployment detailsFetch the details associated with a given deployment. This includes Modalapp_id,created_attimestamp, and theendpointfor the Modal webhook.CLImlflowdeploymentsget-tmodal--name<deploymentname>Python APIplugin.get_deployment(name="<deployment name>")Run prediction on deployed modelFor the prediction inputs, the Python API expects a pandas DataFrame. To invoke via the command line, pass in the path to a JSON file containing the input.CLImlflowdeploymentspredict-tmodal--name<deploymentname>--input-path<inputfilepath>--output-path<outputfilepath>output-pathis an optional parameter. Without it, the result will be printed in the terminal.Python APIplugin.predict(name="<deployment name>",inputs=<predictioninput>)Run the model deployment "locally"Run an ephemeral deployment of your model usingmodal serve. This will behave exactly the same asmlflow deployments create -t modalhowever the app will stop running approximately 5 minutes after you hit Ctrl-C. While the app is running Modal will create a temporary URL that you can use like the normal web endpoint created by Modal.CLImlflowdeploymentsrun-local-tmodal-m<modeluri>--name<deploymentname>-Cgpu=T4-Ckeep_warm=truePlugin helpPrints the plugin help string.CLImlflowdeploymentshelp-tmodal