Package: mlflow-knative
MLflow Knative Deployment PluginMLflowplugin adding aKnativedeployment client to MLflow CLI and Python API.Note:MLServer (V2 Inference API)is enabled for all Docker builds.RequirementsPython 3.10+MLflow 2+DockerThe target Kubernetes cluster must be running Knative 1.10+.Installationpip install mlflow-knativeGetting StartedThe plugin adds support for aknativetarget scheme to themlflow deploymentsCLI.AKubernetes contextis used to define the Knative target asknative:/<context>.You can list available contexts withkubectl config get-contexts.Note:Passing only the baseknative:/(contextomitted) as target lets the Kubernetes client pick a default configuration, which may be in-cluster.KubernetesRBACmust be configured to allow the pod running this deployment client to create, list, update and delete Knative services.Make sure Docker is running locally if you intend create or update a deployment, as this is required to build an image from the MLflow model.Setting theimage_repositoryconfig key is required to make the Docker image of the model available for deployment by Knative. Additionally you may also provide an image tag with theimage_tagconfig key (defaults tolatest).mlflow deployments create \--target knative:/<context> \--name <deployment-name> \--model-uri models:/<model-name>/<model-version> \--config image_repository=<image-repository-URI> \--config image_tag=<image-tag>The plugin provides detailed target help.mlflow deployments help --target knativeAll features are also available as aPython API deployment client.frommlflow.deploymentsimportget_deploy_clientclient=get_deploy_client("knative:/my-cluster")client.create_deployment("hello-world","models:/hello-world/1",config={"image_repository":"hello-world"})Using a Private Image RepositoryTo use a private Docker image repository, simply rundocker logindefore running the deployment client, then use the full repository URI as value for theimage_repositoryconfig key.docker login --username <username> --password-stdin <private-repository-URL>#IfusingAWSECR:aws ecr get-login-password | docker login --username AWS --password-stdin <private-repository-URL>mlflow deployments create \--target knative:/<context> \--name <deployment-name> \--model-uri models:/<model-name>/<model-version> \--config image_repository=<image-repository-URI>  # e.g.: 000000000000.dkr.ecr.eu-west-3.amazonaws.com/model-nameUsing a Remote MLflow Model RegistrySet an environment variable asexport MLFLOW_TRACKING_URI=<tracking-server-uri>to use a remote MLflow model registry.
This also works with a private model registry secured with OAuth 2, using theMLflow OIDC Client Plugin.Knative Service ConfigurationThe deployment client can use any available namespace on the target cluster by setting thenamespaceconfig key. The default value isdefault.mlflow deployments create \--target knative:/<context> \--name <deployment-name> \--model-uri models:/<model-name>/<model-version> \--config image_repository=<image-repository-URI> \--config namespace=<my-namespace>To deploy a Knative service with acustom templated manifest, set theservice_templateconfig key. The value is a path to the YAML manifest you will be using.mlflow deployments create \--target knative:/<context> \--name <deployment-name> \--model-uri models:/<model-name>/<model-version> \--config image_repository=<image-repository-URI> \--config service_template=<path/to/manifest>$name,$namespaceand$imagetemplated values are respectively the deployment name, the provided namespace (or "default"), the image determined from the provided image repository and tag.Caching BehaviorOn deploymentcreation, the pluginalways builds and pushesa new Docker image for the model.On deploymentupdate, if both:an image exists on the repository with the expected tag, andtheMLflow run IDof the current deployment matches therun IDof the model to deploy (identified by its URI on the model registry)
the plugin will skip the image build and push step.Otherwise, the plugin will build and push a new image on the repository with the specified tag.Usingimmutable tagsis recommended, and we advise not relying on the defaultlatesttag.Updating a deployment will always attempt topatchthe corresponding Knative service and rely on Knative handling the changes to determine if a newrevisionis required.Thegenerationservice metadata property will be bumped when a new revision is created, which might be useful to compare service state before and after the update.LicenseThis project is licensed under the terms of the MIT license.AyzrFree and Open Source project.