AutoAI tutorial: Build a Binary Classification Model
AutoAI tutorial: Build a Binary Classification Model This tutorial guides you through training a model to predict if a customer is likely to buy a tent from an outdoor equipment store. Create an AutoAI experiment to build a model that analyzes your data and selects the best model type and algorithms to produce, train, and optimize pipelines. After you review the pipelines, save one as a model, deploy it, and then test it to get a prediction. Watch this video to see a preview of the steps in this tutorial. Video disclaimer: Some minor steps and graphical elements in this video might differ from your platform. This video provides a visual method to learn the concepts and tasks in this documentation. * Transcript Synchronize transcript with video Time Transcript 00:00 In this video, you will see how to build a binary classification model that assesses the likelihood that a customer of an outdoor equipment company will buy a tent. 00:11 This video uses a data set called "GoSales", which you'll find in the Gallery. 00:16 View the data set. 00:20 The feature columns are "GENDER", "AGE", "MARITAL_STATUS", and "PROFESSION" and contain the attributes on which the machine learning model will base predictions. 00:31 The label columns are "IS_TENT", "PRODUCT_LINE", and "PURCHASE_AMOUNT" and contain historical outcomes that the models could be trained to predict. 00:44 Add this data set to the "Machine Learning" project and then go to the project. 00:56 You'll find the GoSales.csv file with your other data assets. 01:02 Add to the project an "AutoAI experiment". 01:08 This project already has the Watson Machine Learning service associated. 01:13 If you haven't done that yet, first, watch the video showing how to run an AutoAI experiment based on a sample. 01:22 Just provide a name for the experiment and then click "Create". 01:30 The AutoAI experiment builder displays. 01:33 You first need to load the training data. 01:36 In this case, the data set will be from the project. 01:40 Select the GoSales.csv file from the list. 01:45 AutoAI reads the data set and lists the columns found in the data set. 01:50 Since you want the model to predict the likelihood that a given customer will purchase a tent, select "IS_TENT" as the column to predict. 01:59 Now, edit the experiment settings. 02:03 First, look at the settings for the data source. 02:06 If you have a large data set, you can run the experiment on a subsample of rows and you can configure how much of the data will be used for training and how much will be used for evaluation. 02:19 The default is a 90%/10% split, where 10% of the data is reserved for evaluation. 02:27 You can also select which columns from the data set to include when running the experiment. 02:35 On the "Prediction" panel, you can select a prediction type. 02:39 In this case, AutoAI analyzed your data and determined that the "IS_TENT" column contains true-false information, making this data suitable for a "Binary classification" model. 02:52 The positive class is "TRUE" and the recommended metric is "Accuracy". 03:01 If you'd like, you can choose specific algorithms to consider for this experiment and the number of top algorithms for AutoAI to test, which determines the number of pipelines generated. 03:16 On the "Runtime" panel, you can review other details about the experiment. 03:21 In this case, accepting the default settings makes the most sense. 03:25 Now, run the experiment. 03:28 AutoAI first loads the data set, then splits the data into training data and holdout data. 03:37 Then wait, as the "Pipeline leaderboard" fills in to show the generated pipelines using different estimators, such as XGBoost classifier, or enhancements such as hyperparameter optimization and feature engineering, with the pipelines ranked based on the accuracy metric. 03:58 Hyperparameter optimization is a mechanism for automatically exploring a search space for potential hyperparameters, building a series of models and comparing the models using metrics of interest. 04:10 Feature engineering attempts to transform the raw data into the combination of features that best represents the problem to achieve the most accurate prediction. 04:21 Okay, the run has completed. 04:24 By default, you'll see the "Relationship map". 04:28 But you can swap views to see the "Progress map". 04:32 You may want to start with comparing the pipelines. 04:36 This chart provides metrics for the eight pipelines, viewed by cross validation score or by holdout score. 04:46 You can see the pipelines ranked based on other metrics, such as average precision. 04:55 Back on the "Experiment summary" tab, expand a pipeline to view the model evaluation measures and ROC curve. 05:03 During AutoAI training, your data set is split into two parts: training data and holdout data. 05:11 The training data is used by the AutoAI training