aylien-model-serving 0.0.2
Gunicorn Flask based library for serving ML Models.

# model-serving

Flask based python wrapper for deploying models as a REST based service based on 

💡 Flask 

💡 Gunicorn 

💡 Protobuf3 (optional for schema validation)

🥳 flask-caching 
 
🥳 prometheus metrics 

The repo also contains examples of registering end points and a Makefile to run the service

## Installation 
(TODO: Replace with pip)

```
 pip install https://github.com/AYLIEN/model-serving/
 cd model-serving
 pip install -e requirements.txt 
 
```

## Project Structure 

```
aylien_model_serving
│
|--- requirements.txt
|--- Makefile
│   
│
└───app
│    |-- app_factory.py
│    |-- cached_app_factory.py       
│      
│   
└───examples
    │-----example_schema.proto
    |-----example_schema_pb2.py(autogenerated by protoc)
    │-----example_serving_handler.py
    |-----example_serving_handler_cached.py
```


## How it works

* It runs a web service on the given port (defaults to `8000`).
* Any incoming request JSON will be passed to your `ServingHandler.process_request` 
* Your `ServingHandler.process_request` is expected to return a `json` 
* The request and response will be validated with a protobuf schema (optional)
* This library wraps common service code, monitoring, exception handling, etc.
 
## Usage

1. Install this library as a dependency for whatever model you want to serve.
2. Create a `ServingHandler` (see below for interface details).
3. Run the make target ` make COMMAND_UNCACHED='ServingHandler.run_app()' example-service`   

## Interfaces

The main interface to flask apps defined in [app_factory](aylien_model_serving/app_factory.py) is the `process_json` function.
This function expects to receive json input, optionally perform schema
validation, then call the `callable_handler` function using each of the fields 
in the json object as a keyword argument to the function. The function is expected to 
return an object that can be parsed to json and sent as the response.

This design allows for a very simple but powerful interface that can easily make an endpoint 
out of just about any Python function.


### Example Serving Handler

The example serving handler defined [here](aylien_model_serving/examples/example_serving_handler.py) does the following

1. Defines a method predict_lang. For the purposes of this example, this returns a static prediction. Ideally would be the prediction 
   or classification from your model. 
2. Imports a protobuf3 generated .py schema file(only if you require the json message to be schema validated)
3. Defines a function process_request that calls the wrapper function process_json with the callable from 1 and schema from 2 
4. Registers process_request and its route mapping 
5. Repeat 1-4 for a (route, callable) pair if you have more than one service end point. 

```python
import examples.example_schema_pb2 as schema
from aylien_model_serving.app_factory import FlaskAppWrapper, InvalidRequest


def predict_lang(text):
    return "en", 0.71


def predict(title=None, body=None, enrichments=None):
    if body is None:
        body = enrichments["extracted"]["value"]["body"]
    if title is None and body is None:
        raise InvalidRequest("Missing text")
    article_text = f"{title} {body}"
    detected_lang, confidence = predict_lang(article_text)
    return {
        'language': detected_lang,
        'confidence': confidence,
        'error': 'Not an error',
        'version': '0.0.1'
    }


def process_request():
    return FlaskAppWrapper.process_json(predict, schema=schema) 


def run_app():
    routes = [
        {
            "endpoint": "/",
            "callable": process_request,
            "methods": ["POST"]
        }
    ]
    return FlaskAppWrapper.create_app(routes)
```
Note that the FlaskAppWrapper accepts a callable in the process_json , and if you'd like to load a classifier or model.bin in your memory you could modify it like below 👇

```python
class ClassifyHandler:
    def __init__(self):
        self.classifier = Classifier() #this is the classifier to load , or a binary in local file storage

    def __call__(self, text):
        return self.classifier.predict(text)


def run_app():
    classify_handler = ClassifyHandler()
    routes = [
        {
            "endpoint": "/classify",
            "callable": classify_handler,
            "methods": ["POST"]
        }
    ]
    return FlaskAppWrapper.create_app(routes)
    
 ```