Think Lab 2124 : Build & Deploy AI/ML Models w Multiple Datasets w AutoAI - Tutorial B
Published: 2021-05-25 20:08:36

#Welcome to Tutorial Tuesday! 

We'll pick up where we left off last week with our Think Lab 2124 [Tutorial A: Build & Deploy a Data Join Experiment](https://dev.to/ibmdeveloper/think-lab-2124-build-deploy-ai-ml-models-w-multiple-datasets-w-autoai-part-a-24pk)!

##AutoAI Overview

AutoAI in Cloud Pak for Data automates ETL(Extract, Transform, and Load) and feature engineering process for relational data, saves data scientists months of manual data prep time and acheives results comparable to top performing data scientists.

The AutoAI graphical tool in Watson Studio automatically analyzes your data and generates candidate model pipelines customized for your predictive modeling problem.  These model pipelines are created iteratively as AutoAI analyzes your dataset and discovers data transformations, algorithms, and parameter settings that work best for your problem setting.  Results are displayed on a leaderboard, showing the automatically generated model pipelines ranked according to your problem optimization objective.

Collect your input data in a CSV file or files. Where possible, AutoAI will transform the data and impute missing values.

**Notes:**
* Your data source must contain a minimum of 100 records (rows).
* You can use the IBM Watson Studio Data Refinery tool to prepare and shape your data.
* Data can be a file added as connected data from a networked file system (NFS). Follow the instructions for adding a data connection of the type Mounted Volume. Choose the CSV file to add to the project so you can select it for training data.

##AutoAI Process

Using AutoAI, you can build and deploy a machine learning model with sophisticated training features and no coding. The tool does most of the work for you.

AutoAI automatically runs the following tasks to build and evaluate candidate model pipelines:
* Data pre-processing
* Automated model selection
* Automated feature engineering
* Hyperparameter optimization

In this Think Lab, you will see how to join several data sources and then build an AutoAI experiment from the joined data. The scenario we’ll explore in Part B of the Lab is for a mobile company that wants to understand the key factors that have an impact on user experience in their call center. You will use IBM AutoAI to automate data analysis for a dataset collected from a fictional call center. The objective of the analysis is to gain more insight on factors that impact customer experience so that the company can improve customer service. The data consists of historical information about customer interaction with call agents, call type, customer wireless plans and call type resolution.

##Project Requirements
[IBM Cloud (Free) Lite Tier Account](https://ibm.biz/think-2021)

##Project Setup Steps
1. [Create an IBM Cloud Lite Tier Account](https://ibm.biz/think-2021)
2. Create a Watson Studio Instance
3. Provision Watson Machine Learning & Cloud Object Storage Instances
4. Create a New Project
5. Download the Call Center Dataset from the Gallery
6. Unzip the Call Center Dataset's .zip File
7. Add the Call Center Datasets to the Project

##Project Setup
[See Tutorial A for Project Setup](https://dev.to/ibmdeveloper/think-lab-2124-build-deploy-ai-ml-models-w-multiple-datasets-w-autoai-part-a-24pk)

##Think Lab Overview
In Tutorial B of this **Think Lab**, you will use IBM AutoAI to automate data analysis for a dataset collected from a fictional call center. The objective of the analysis is to gain more insight on factors that impact customer experience so that the company can improve customer service. The data consists of historical information about customer interaction with call agents, call type, customer wireless plans and call type resolution. Each source of information is kept in a separate table (a CSV file).

Using the data join capabilities of AutoAI, you will connect the tables using common coloumns, or keys, to create a single data source, without needing to write SQL-like queries. Additionally, AutoAI will do some automated data preparation, or feature engineering on the combined data before using the data to train the model.

##About the Data

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g6hbwd14olgxa2vzaaro.png)

The data is divided as follows:
* **User_experience**: User experience reflects the satisfactory feedback from customers to each call agent daily.
* **Call_log**: Records historical information about the calls from customers to the call center in the last 3 years.
* **Call_Type**: Records call type information.
* **Wireless_Plans**: Records the kind of wireless plans customers are subscribed to.
* **Call_Resolution_Type**: Records type of call resolution.

##Steps Overview
This tutorial presents the basic steps for joining data sets then training a machine learning model using AutoAI:
* Add and join the data
* Train the experiment
* Deploy the trained model
* Test the deployed model

##Think Lab - Tutorial B Steps
1. Create a New AutoAI Experiment
2. Build the Data Join Schema
3. Run the AutoAI Experiment
4. Explore the Holdout & Training Data Insights
5. Deploy the Trained Model
6. Score the Model
7. View the Prediction Results

##Think Lab - Tutorial B: AutoAI Data Join Multi-Classification 
###1. Create a New AutoAI Experiment

####Add a New AutoAI Experiment to the Project
Click **Add to Project (+)**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kh3eq7rkjw1wzu9pf34o.png)

Select **AutoAI experiment**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fijhxp7w5a4wtqig9pmf.png)

####Associate a Machine Learning Service Instance
Select a Watson Machine Learning Service Instance from the dropdown menu. Click **Create**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fqi454maj83xad28556p.png)

####Add the Call Center Datasets
Click **Select from project**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a0ntcfg50k8fxemztean.png)

Select the call center data assets. Click **Select 5 Assets**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zqyw16u8zrqqwsn03g6h.png)

**Optional**: You can view metrics on each data source by clicking on it in the Data Source window.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a3gk7onzpf5rghv4cfqr.png)

###2. Build the Data Join Schema

The main source contains the prediction target for the experiment. Select **User_experience.csv** as the main source, then click **Configure join**.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qruuokjaeui0es83z7se.png)

In the data join canvas you will create a left join that connects all of the data sources to the main source.

####Use the Data Join Table to Build the Schema
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/od9d3bt35cpgzthpoxon.png)

Setup the nodes to match the following image to make connecting the nodes easier.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tiynzp58s5o7iqgz9jfo.png)

Drag from the node on one end of the **User_experience.csv** box to the node on the end of **Call_log.csv**.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9zblm1uat3535ze5kyn6.png)

In the panel for configuring the join, click **(+)** to add the suggested keys **Agent_ID** and **Call_Date** as the join keys.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3puv2dw8rxcjsptm312h.png)

Repeat the data join process until you have joined all the data tables.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3u3hrncbipa2bneolc1h.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x1xmlpcw32obl2jrzl41.png)

####The Completed Data Join Schema Should Look Like This:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7d37yvqhvcwedjnqwidr.png)

Choose **User_Experience** as the column to predict.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1n1zenk4j8ixy9kdc1id.png)
AutoAI analyzes your data and determines that the User_Experience column contains information making the data suitable for a **Multiclass Classification** model. The default metric for a **Multiclass Classification** model is optimized for **Accuracy & run time** automatically by AutoAI.

**Note**: 
* Based on analyzing a subset of the data set, AutoAI chooses a default model type: binary classification, multiclass classification, or regression. Binary is selected if the target column has two possible values, multiclass if it has a discrete set of 3 or more values, and regression if the target column is a continuous numeric variable. You can override this selection.
* AutoAI chooses a default metric for optimizing. For example, the default metric for a binary classification model is Accuracy.
* By default, ten percent of the training data is held out to test the performance of the model.

###3. Run the AutoAI Experiment
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hmt3vig2sg7u2dn4kxw3.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t6puqvvg3ej2wrnjr9v1.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jc5h9c86jxn6w7fo0n4l.png)

###4. Explore the Holdout & Training Data Insights
Pipeline Comparison
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/64nd1jm0ypndunl30fs6.png)
Eplore the Leading Pipeline
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6w8gnamrpk4vpknlbnij.png)
Model Evaluation
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0mdz5wu0lwd3o7nkafjf.png)
Threshold Chart
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/40yhb4pxrn31pixx6s6y.png)
Feature Importance
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b80hoxgn85kwn7qlpukb.png)
From the feature importance chart, the most important feature is count **(*Call_Type_Description*)** which is the total calls in a day.

Other important features are from the **After_Call_Work_Time**, which are talk time and queue time. These features affect users experience the most.

**The call center management team should pay attention to these features and try to figure out how to improve user’s experience by adjusting these features.**

###5. Deploy the Trained Model
####Click **Save as** and Select **Model**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8jrh9eai56q4pc9px2n3.png)

####Click **Create**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/73datfposyi3e6ck82nb.png)

####Click **View in project**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e53km5udgz06uhz2hwkw.png)

####Add the Call Center Datasets to the Deployment Space
Navigate to the **Deployment Space**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uswhkc1z2mzozjkom8am.png)

Click **browse for files to upload**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4r4aat9zu624msaio2g1.png)

Select Call Center Data Sources
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4oe5dzaknj2jv1mh1f4t.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v4lo0rmikp3a13lomkgt.png)

####Promote the Trained Model to the Deployment Space
Click **Promote to deployment space**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iwdkf5uah34pt1miw2s1.png)

Select the Deployment Space from the dropdown menu.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/id40q6ge3jhhgodajigi.png)

Check the **Go to the model in the space after promoting it** box.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x4cg9axoasv4b9ozercu.png)
Click **Promote**

####Deploy the Trained Model
Click **Create deployment**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2ks8suw5azvxc4efd1bw.png)

####Create a New Batch Deployment
Add a name for the new Batch deployment.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/occe0fep7lfyo19nv3xe.png)

####Configure the Hardware Definition
Select the **Data Join** hardware.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yn9ftydu5wppyyz308h8.png)

Select the **Model Scoring** hardware.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kajifftej4incqsuipky.png)

Click **Create**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k8g3w2s2ouehc9w95xmd.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uenyf4ouxsr71vfl6rr5.png)

###6. Score the Model
To score the model, you create a batch job that will pass new data to the model for processing, then output the predictions to a file. **Note**: For this tutorial, you will submit the training files as the scoring files as a way to demonstrate the process and view results.

####Create a New Batch Job
Click **Create Job**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uenyf4ouxsr71vfl6rr5.png)

Add a name for the new Batch job. Click **Next**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dr7390bi9flu2rd4199l.png)

Keep the current **Hardware configuration**. Click **Next**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hkncm2kyztzeulc0o7zt.png)

Keep the **Schedule off**. Click **Next**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qp1kj4lgkbxpd6pp9ao4.png)

####Add the Scoring Files
You will see the training files listed. For each training file, click the **Select data source** button, and choose the corresponding scoring file.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/an9k6uml7fr5tdkmw2yb.png)

**WARNING** : Schema mismatch. The column types in this data asset do not match the column types in the Model Schema. Click **Continue** to select anyway.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yz9za1awu1zw1emqvd5g.png)

Add **call-predictions.csv** as the Ouput file name.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jtxmh87jloviwcybu2q0.png)

####Run the Batch Job
Click **Create**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/471m9y9yxsd2miioxl3f.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cf3bvfooxdgwt1s27k8e.png)

####View the Batch Job
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/16114wzk2tu86t0tht1p.png)

####Wait for the Batch Job to Complete
Starting...
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oni9pqfdc6y27koft4va.png)

Running...
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/igt3z197nkr0wxgswrob.png)

Completed
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cm70k108n9k5b0g44p1u.png)

###7. View the Prediction Results
Download **call-predictions.csv**, and view the prediction results in Excel.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uqd34lsyl3ev5huowt4z.png)

#CONGRATULATIONS!
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t8tvevzwh914dfudp224.gif)

###You've completed Think Lab 2124: Build & Deploy AI/ML Models w Multiple Datasets w AutoAI!

Tune in next week for our next Tutorial Tuesday post.

Follow for more Cloud Native & Watson AI/ML content:
https://linktr.ee/jritten