Leverage Watson Machine Learning Function Deployment in Data Science Workflow
By: Jade Zhou
Tags: Time Series Forecasting, Tutorial, Watson Machine Learning, Lda, Machine Learning

Leverage Watson Machine Learning Function Deployment in Data Science Workflow

Yushu Zhou, IBM Data Science Elite Team,

Wanting Wang, IBM Data Science Elite Team

Introduction

IBM Watson Machine Learning (WML) is a full-service IBM Cloud offering that makes it easy for developers and data scientists to work together to integrate predictive capabilities with their applications. WML provides two deployment options: model deployment and function deployment, so literally, when you want to integrate your model into production environment, model deployment is the reasonable choice. However, WML model deployment only supports limited number of types of models like scikit-learn and keras. In real world, apart from classification and regression models, we still have other types of models. For example, LDA is not supported by WML model deployment functionality currently. Thus, as a more flexible alternative, WML provides function deployment to be leveraged in above situations.

In this blog, we will take 2 different unsupported models as examples to introduce different ways of deploying models by WML function deployment.

Set-up

To follow this blog, we recommend you to create an IBM cloud account and instantiate your personal Cloud Object Storage (COS) and Watson Machine Learning (WML) services under ‘Catalog’.

Fig 1. WML under Catalog

Fig 2. COS under Catalog

The good thing here is we have ‘Lite’ version for these services which you can play around without any cost. After you provision these two services, go to your bluemix dashboard and choose WML instance you just created. Click on ‘view credentials’ and copy them down. We will use these credentials later.

Fig 3. Get WML Credentials

Then go to Watson Studio and create a project based on COS you just created. Finally, associate WML instance with this project under ‘Settings’.

After you finish these steps, we can start playing with WML.

How we leverage function deployment in different situations

1. You can save and deploy your model but cannot score by WML model deployment

Sci-kit learn package in Python provides numerous machine learning algorithms and related methods for data processing. Many popular algorithms from sci-kit learn are well supported by WML. One exception is Latent Dirichlet Allocation (LDA), a useful topic modeling algorithm in text analytics.

To understand why sci-kit learn LDA models are not supported and accordingly how to use sci-kit learn LDA models with WML, let’s start with training a simple LDA model on 20 news groups dataset which can be downloaded through sci-kit learn.

(1) Load data and train LDA model

import pandas as pd

from sklearn.datasets import fetch_20newsgroups

from sklearn.feature_extraction import stop_words

from sklearn.feature_extraction.text import CountVectorizer

from sklearn.decomposition import LatentDirichletAllocation ## download data

dict_news = fetch_20newsgroups(subset=’train’)

text = dict_news.data[:5000]

label = dict_news.target[:5000] ## vectorize data

stop_words = list(stop_words.ENGLISH_STOP_WORDS)

vectorizer = CountVectorizer(stop_words = stop_words, min_df = 0.01, max_df = 0.8))

text_bw = vectorizer.fit_transform(text) ## train LDA

lda = LatentDirichletAllocation(random_state=123)

lda.fit(text_bw)

As what you may expect, I use the count vectorizer to vectorize the text data, and reduce the vocabulary by setting thresholds in min_df and max_df so that words unlikely to be informative are eliminated and the training doesn’t take too much time.

Now the model is trained, I want to deploy it in WML in order for my application to use.

(2) Save and deploy to WML as a model

To make the right WML instance be aware of what is going on, I need to set up the connection to the picked WML instance.

credentials_wml = {

“apikey”: “****”,

“iam_apikey_description”: “****”,

“iam_apikey_name”: “****”,

“iam_role_crn”: “****”,

“iam_serviceid_crn”: “****”,

“instance_id”: “****”,

“password”: “****”,

“url”: “****”,

“username”: “****”

} from watson_machine_learning_client import WatsonMachineLearningAPIClient client = WatsonMachineLearningAPIClient(credentials_wml)

Credentials can be obtained from the Service Credentials page of your WML instance which we showed previously. As a recap, you can go to the resource list in your IBM Cloud dashboard (https://cloud.ibm.com/resources), find the WML instance listed under Services, click on it and go to the Service Credentials page on the menu. You may use an existing key or create a new one.

model_props = {client.repository.ModelMetaNames.NAME: “LDA model 1”} model_artifact=client.repository.store_model(lda, meta_props=model_props) published_model_uid = client.repository.get_model_uid(model_artifact) created_deployment = client.deployments.create(published_model_uid, name=”LDA model 1 deployment”)

Great, it says the deployment is successful! Before embedding the scoring endpoint into my application, I’d better test it first, just in case.

scoring_endpoint = client.deployments.get_scoring_url(created_deployment) fields = df_bw.columns.tolist() values = [df_bw.iloc[1].values.tolist()] scoring_payload = {“fields”: fields,”values”: values} predictions = client.deployments.score(scoring_endpoint, scoring_payload) print(predictions)

I pull one vectorized document (essentially one row) from the data frame, convert the values to a list, and wrap it as a payload with field names which I then send to the scoring endpoint.

Ideally I should see the result on this document given by the LDA model I just deployed, but I got the error message saying that my LDA model object does not have attribute predict.

It is clear that WML is trying to call the predict method on the LDA model object, however this operation fails simply because scikit-learn LDA models don’t have such method. Instead, when applying a scikit-learn LDA model on new data points, we need to call the transform method.

So far I have no idea if there is any way to hack into WML and change how it works — there should not, I suppose? Then we are left with the only choice: make adaptation on our side to make things work.

Thanks God, there is python deployable function we can turn to.

(3) Save and deploy to WML as a function

Python deployable function specifies what needs to be run when the function is deployed and what needs to be run when the deployment is called. In other words, you are able to customize what preparation WML does in the environment when you deploy the function, as well as what steps WML takes to generate the output when you call the API later on.

The only change we want to make for the deployment is to replace the predict method with the transform method. When it comes to the deployable function, it is straightforward but not that straightforward, as you don’t see the vanilla version of what is going on but you are asked to create your own.

What are the essential components to score a random data point? We need to a) fetch the model object back from WML, b) convert the payload to the same format as the training data, c) call the transform method on the model object over the payload, and d) construct the result to be returned.

Let’s see how these 4 components are put together.

ai_parms = { “credentials_wml” : credentials_wml,

“model_uid” : published_model_uid } def my_deployable_function( parms=ai_parms ):

## a) load model

from watson_machine_learning_client import WatsonMachineLearningAPIClient

client = WatsonMachineLearningAPIClient( parms[“credentials_wml”] )

model = client.repository.load(parms[“model_uid”]) def score( payload ):

## b) assemble input

df_payload = pd.DataFrame(payload[“values”])

df_payload.columns = payload[“fields”]

## c) obtain result from the model

res = model.transform(df_payload).tolist()

## d) assemble output

output = {“result” : res}

return output return score

The deployable function consists of the outer function (any place that is not within the score function) and the inner score function. The code sits in the outer function runs when the function is deployed, and the environment is then frozen and ready to be used whenever the API is called. The code sits in the inner score function runs when the API is called, in the environment customized by the outer function, basically takes the input json and outputs some result json.

If the score function is the only thing that WML runs when the deployment API is called, why you would even bother to care about the outer function? Why cannot just put everything in the score function?

The answer is, yes you can always put everything in the score function, but do you really want to afford the repeated cost in run time that can be isolated in environment preparation?

In this example, fetching the simple LDA model from WML may not take much time. What if you need to load a big neural network model that has hundreds of millions of parameters or some complex ensemble model that compromises dozens of weaker models? They could easily exceed 100MB or even 1GB. Loading models of such big size requires substantially longer time, and there is not benefit loading them every time when you call the API compared to loading them previously in the environment.

In addition to model loading that may take some time, other operations that are irrelevant to the scoring part (like package installation) can also be moved to the outer function.

After you configure the outer function, what needs to be done in the inner score function then is pretty easy to imagine, and actually is very much similar to how you will score a new data point with the loaded model in a plain cell in jupyter notebook.

Once the function is specified, deploying a function is as simple as deploying a model.

meta_data = { client.repository.FunctionMetaNames.NAME : ‘LDA model 1 function’ } function_details = client.repository.store_function( meta_props=meta_data, function=my_deployable_function ) function_id = function_details[“metadata”][“guid”] function_deployment_details = client.deployments.create( artifact_uid=function_id, name=’LDA odel 1 function deployment’ )

Again I see the familiar success message, although I know it does not necessarily indicate that the deployment API works as expected — from the surprise I had several minutes ago. So too early to celebrate at this moment. I have to keep it until the test is passed.

Similarly, from the deployment I obtain the scoring endpoint, and pass the same test payload to it.

scoring_endpoint_function = client.deployments.get_scoring_url( function_deployment_details ) predictions = client.deployments.score(scoring_endpoint_function, scoring_payload) print(predictions)

See what is the result?

Fig 4. Results from LDA function endpoint

2. You don’t need a fixed model

For most of time series forecast problems, simple stats models can beat machine learning without much effort, so there are many cases where we have to build not-ML models to have better predictions. Before explaining how to deploy this type of model, we have to rethink training and testing data that needs to be fed into the model.

Assume you are responsible for production and stock planning in a company, to make sure whenever clients request a specific product, your company has stock of that, you have to predict every product’s monthly demands ahead of time considering production needs time, so a gap period is needed and you have to predict demands month by month. Every time when you try to make monthly predictions, you need to refit a model because one more month data can be incorporated into your model. Thus, in this scenario, we don’t need a fixed model to be continuously called. Instead, we pass training data every time to get a new model and all predictions. To conclude, what we post to API should be training data. After that, the API should finish fitting a model and respond predictions.

Following this workflow, we should deploy a function with the ability to fit and forecast. Therefore, we already got a rough image of what should be included in the notebook.

(1) Set up WML Client

Same step as before.

(2) Define custom function — outer part

Next, create your own function. Inside function, to make sure we install all packages we need, I’ll add sub-processing here.

def es():

"""

You will deploy this function.

-------------------------------

:return score function

"""

try:



import subprocess

subprocess.check_output( "pip install -I statsmodels==0.9.0 --user", stderr=subprocess.STDOUT, shell=True)

subprocess.check_output( "pip install -I pandas==0.23.4 --user", stderr=subprocess.STDOUT, shell=True)

subprocess.check_output( "pip install -I numpy==1.15.4 --user", stderr=subprocess.STDOUT, shell=True)

import pandas as pd

import numpy as np

from statsmodels.tsa.api import SimpleExpSmoothing



except subprocess.CalledProcessError as e:



return { "error" : "subprocess.CalledProcessError:



" + "cmd:

" + e.cmd + "



output:

" + e.output.decode() }



except Exception as e:



return { "error" : repr( e ) }

(3) Define custom function — inner scoring part

Since our input is training data, we will fit a model and finish multi-step or single-step forecast inside scoring function. Then we will return forecasts in json format and outer function has to return score function.

def score(payload):

"""

To get returned values from es function, you have to rewrite your own score function here.

------------------------------------------------------------------------------------------

:return any output in json format

"""

forecast_period = int(payload['values'][0]['forecast_period'])

ts = pd.read_json(payload['values'][0]['data'])

ts.index = pd.to_datetime(ts.index)



# To put it simply, I built a simple exponential smoothing model here, but you can build more complicated models and return predictions as output field's values.

fit = SimpleExpSmoothing(ts).fit(smoothing_level=0.2,optimized=False)

fcast = fit.forecast(forecast_period) return { "output" : list(fcast)}



return score

(4) Deploy and test

Finally, we can deploy above function in our customized environment and post input data to the endpoint as a test. Remember to embed your data in json format which our function can recognize.

# store function

# replace function_name with your own name

function_name = 'Exponential Smoothing'

meta_data = {

client.repository.FunctionMetaNames.NAME: function_name#,

#Uncomment next line of code if you choose installing libraries by RUNTIME API

#client.repository.FunctionMetaNames.RUNTIME_UID: runtime_guid

} stored_function = client.repository.store_function( meta_props=meta_data, function=es) function_uid = client.repository.get_function_uid( stored_function ) # deploy function

# replace deploy_name with your own name

deploy_name = 'ES Deployment'

function_deployment_details = client.deployments.create( artifact_uid = function_uid, name = deploy_name )

Here are results if we call the deployed function:

# NOTE: manually define and pass the array(s) of values to be scored in the next line

# We're predicting next 3 months' electricity demands based on 2003 monthly data only

function_payload = {"fields": [[]], "values": [{'forecast_period': 3, 'data': pd.DataFrame(ts['2003']).to_json()}]} scoring_endpoint = client.deployments.get_scoring_url( function_deployment_details ) predictions = client.deployments.score(scoring_endpoint, function_payload)

print(predictions)

Conclusion

Now it’s time to toast! Hope this blog can bring you new ideas about how to leverage WML into data science workflow from model deployment perspective. One missing piece here is how we deploy trained models when you cannot even save it into WML model repository. For example, spark ALS model is a common tool for agile development of recommender but it’s not directly supported by WML. Our potential solution is to upload it to COS by ibmos2spark and follow same steps in first part: load back the model in outer function and score in inner function. However, since the environment for function deployment is pure python environment, it’s hard to connect to spark engine there, but it will get solved in near future. Therefore, we’ll update this blog once it’s ready.

Finally, we know to train a machine learning model, the whole pipeline is much more than just final model deployment, so Wanting will introduce how to leverage WML function deployment at other stages in part 2.

Keep an eye on our updates!