Package: productionize
productionize - deploy ML models directly from Python [WIP]productionizeis an open-source lightweight ML deployment tool.You can containerize, deploy and ship your model, without everhaving to leave your beloved Python.productionizein a nutshellWhat does it do? Well, it does exactly what the catchy library name says it does.productionizehelps you to productionize your API. As a Data Scientist, most of the projects I worked on face issue when productionizing code. Often, the code is not tested, standardized or environment agnostic enough to just deploy something somewhere. This where containers come in very handy. Containerization helps you to freeze the environment and decouple your model or just your code from the host system. This makes deployment much, much easier.However, working with Docker, Kubernetes and all these other fancy tools is not as simple as one might hope. The good news though, some steps can be automated and this is exactly whatproductionizedoes. As Data Scientist you can focus on your model and the containerization and deployment is handled byproductionize.The workflow withproductionizeis very simple. First, you develop your API in Python. Next,productionizeallows you easily setup a local Kubernetes cluster that allows you to test your API. Inproductionize, this local Kubernetes cluster is called a workbench, because it is Kubernetes, with a little extra stuff to help you work. Next, you deploy your API. You don't have to change your standard API script for that,productionizewill handle that for you. Within a matter of seconds, your API is built into a container and deployed to your workbench. Here you can test your API and see if it works. If you are happy with it, you can simply export the container and deploy it to any Kubernetes cluster you like.That way,productionizemakes it super easy to turn your local API into a production-ready container and the best part: you don't even have to leave Python.Installationproductionizeis a Python library, which is hosted on PyPi. Currently, the functions are only supported onmacOS. On thedarwinplatform you can therefore download the package usingpip.pip install productionizeUsageOnce the library is properly installed from PyPi, you can source it using your standard python import command. The core
of the library are it's three main classes, those can be imported as follows:# import lib
from productionize import workbench, productThe library contains two major classes. The first one is theworkbench()class. This class allows you to setup and manage a proper ML workbench on your local machine. The second one is theproduct()class. This class allows you to deploy your ML APIs to the local workbench and to any Kubernetes cluster.Theworkbench()classOnce the main classes are sourced, you can setup your very own workbench on your local machine. The workbench consists of several tools:Docker: a container technology, which helps us to build Docker container, which are the quasi-standard in Machine Learning deployment. You can read more about Dockerhere.VirtualBox: a driver that is needed to create a VM on you local machine to host the Kubernetes cluster, which is at the heart of the workbench. You can read more about VirtualBoxhereKubectl: a cli which allows you to interact with Kubernetes. You won't have to do that, butproductionizeis running Kubernetes commands in the background.Minikube: a local implemenation of Kubernetes. Minikube runs on a VM, which is administrated by Virtualbox.Technically, the components are ensembled in a simple fashion. However, the only specialty is, that Minikube is installed on top of VirtualBox.To setup the workbench, these tools need to be installed. You can do this, by simple running thesetup()method of the workbench class. Once initiated you can call the method.# initiate class
cluster = workbench()

# install and setup components
cluster.setup()To fire up the entire workbench, you first need to login to Docker Desktop. This is installed for you, however, you need to have it running. You can easily do this, just search on your computer - if you have a Mac you just use spotlight search - for Docker and start the application.Next you will have to sign in. If you don't have an account already, you can create one for free atDocker Hub. Which is a lot like GitHub, just for containers.Once you did this, you are good to go on. You can now start the cluster using thestart_cluster()method. This method allows you to set the resource quota for the cluster. Default are two CPUs and 2GB of memory.# start the cluster
cluster.start_cluster(cpus = '2', memory = '2G')When the cluster is running, you can create a project. This helps to have a clean and well-structured cluster running. You can do this with theopen_project()method.# open project
cluster.open_project(name = "my-project")In case you want to delete the project you can use thedelete_project()method. Technically, the projects are namespaces on Kubernetes.# delete project
cluster.delete_project(name = "my-project")To stop the cluster you can simply use thestop_cluster()method. This one just idles the cluster, but doesn't remove all the components.# stop the cluster
cluster.stop_cluster()To cleanly uninstall all the components, you can just run theuninstall()method and even specify which components to delete. The default is, that the components that existed on your machine before will be not removed.# cleanly uninstall cluster components
cluster.uninstall(docker = None, kubectl = None, virtualbox = None, minikube = None, report = True)Theproduct()classWhile theworkbenchclass mainly concerns the infrastructure management, theproductclass deals with your API. Theproductclass turns your API into a deployable product. Once you have an API programmed, for instance with Flask, theproductclass will do the rest for you.Let's consider the following python script containing a Flask API:#!flask/bin/python
from flask import Flask

app = Flask(__name__)

@app.route('/hello')
def index():
    return "Hello, World!"

if __name__ == '__main__':
    app.run(port = '8000', host = '0.0.0.0')You can, of course, create any kind of API you like. You can also add new routes or whatever you need. To deploy an API to Kubernetes, you would typically need to containerize the API.productionizedoes that for you. Theproductclass contains theprepare_deployment()method. This method produces a Dockerfile from your API script and a requirements file.# initiate the class and say which project the product belongs to
my_api = product(name = "my-product",
                 project = "my-project")

# prepare the deployment
my_api.prepare_deployment(api_file = "path_to/api.py",  # path to the api file
                          requirements_file = "path_to/requirements.txt", # path to the req file
                          port = "8000") # the port your API is exposed toNote: I would advise to not do any directory stunts here. The code in this library is flexible, however, it might be a bit tricky.Once you run theprepare_deployment()method,productionizewill build a Dockerfile in your current working directory.You can, of course, modify and edit the Dockerfile. However, at your own risk. If you intend to work in an enterprise context it might be necessary to change permissions within the container. This does not have an effect onproductionize. Per default,productionizecontainers run with root.FROM python:3.7.7
RUN mkdir -p /api
COPY api.py /api/api.py
COPY requirements.txt /api/requirements.txt
RUN python -m pip install -r /api/requirements.txt
EXPOSE 8000
ENTRYPOINT ["python", "api/api.py"]Once you ran theprepare_deployment()method, you can deploy your api to the workbench. Why would you do this? Well, the workbench should serve as your local test environment. Using the deploy() method, you can easily deploy your "product" to the workbench.my_api.deploy()Per defaultdeploy()does not take any arguments. Those are not necessary as all info is stored in the my_api object afterprepare_deployment. However, if you want, you can also deploy your product on your localhost.
Technically speaking, this will just create Docker container that runs on localhost. This can be acheived with the local arg
in the method call.my_api.deploy(local = True)Once your product is deployed, the method will return the url under which you can reach your API. However, don't forget to add your custom routes.Your output should look somewhat like this:>>> my_api.deploy()

    Deployment Report:
    ------------------

    This is an automatically generated report on the status of your deployed
    product. Your API is now containerized and hosted on the workbench. You
    can access the API using:

    http://XXX.XXX.XX.XXX:XXXXX/<your_route>

    You can call the API in whatever way it is designed. If you want to get
    rid of it, just use the delete_deployment() method. If you just want to
    update the API, you can just use prepare_deployment() to create a new
    Dockerfile and then deploy() again.

              Your Product
    -----------------------
    Name:       my-product
    Project:    my-project
    Status:     deployed and healthy
    Access:     http://XXX.XXX.XX.XXX:XXXXX/<your_route>

    If you want to export the image to your local machine just use the
    export_product() method. If you want to push it to another registry,
    you can use the push_product() method.Now you know how to reach your API. In case you find out it doesn't work and you change something on the code, you can just re-runprepare_deployment()and thendeploy(). Thedeploy()will automatically realize that the "product" has already been deployed and will just update the existing one. In case you want to delete a product, you can just use thedelete_deployment()method. This will also work for local deployments.# delete product
my_api.delete_deployment(product = "my-product", project = "my-project")When you are satisfied with your API, you might want to deploy or ship it to an enterprise-ready or collaborative cluster. As the workbench is at the heart a Kubernetes cluster, everything you do on the workbench, will work on any other cluster. To give you the freedom of choice,productionizeimplements a method to deploy anywhere.This is thepush_product()method. This method pushes the product in form of a Docker image to any registry you want. Default is DockerHub. However, you can select any registry you like. In case of secure registries, you will need credentials or a token. Those will be asked from you with a prompt.# push the product
my_api.push_product(product = "my-product", registry = "my.registry:5000/image-name")This method will automatically tag the image and rundocker pushto push the image to any remote industry.Next Stepsproductionizeis far from ready and is still work in progress. I started this project around mid of May 2020, when I was super annoyed when I had to built up a new test cluster on my local machine, cause I messed up the others too much. As this all started with me sitting on my Mac, this project isat the moment only stable on macOS. I already started to work on other UNIX systems, however Windows might take a bit of time. So the next steps are the following:Release 1.0Functional Features:Ease the export of products from workbench to local machineIntegrate the push feature to external cluster registriesNon-functional Features:Update unit testing for product() classRelease 2.0Functional Features:Add workbench management featureNon-functional Features:Support latest Ubuntu versionSupport latest CentOS version