Customer Predictive Analytics with Support Vector Machines (SVM) and Plotly
By: Nathan Worrell
Tags: Customer Analytics, Data Science, Plotly, Machine Learning, Svm

It appears that those with higher levels of education are less likely to respond to an offer. Similar to the first graph, it appears that customers with luxury vehicles are less likely to respond to an offer.

How can we further explore the seemingly negative correlation between wealth and response rate?

This graph further supports the idea that current marketing efforts are more effective on lower income customers. However, it also shows that most of the customers in this sample fall into that lower income bracket.

Before we proceed onto the classification model lets take a look at what features have the strongest correlation with our response variable.

Regression Analysis with Continuous Variables

Logistic regression is the appropriate regression analysis to conduct when the dependent variable is binary. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more independent variables.

In this case, we will be using Logistic Regression to determine which variables, if any, have an influence on the probability of the customer responding to a sales call.

We can see from the p-values of Income, Monthly Premium Auto, Months Since Last Claim, Months Since Policy Inception, Number of Open Complaints, Number of Policies, that these input variables seem to have significant relationships with the target variable Response. The “coef” column tells us that these variables are all negatively correlated with Response.

How do we interpret P-Values?

P-values evaluate how well the sample data support’s the null hypothesis. In this case, our null hypothesis is that the variables are insignificant for predicting the dependent variable, Response.

P-values are only useful when compared to an alpha-value. P-values that are less than a standard alpha-value of 0.05 can be concluded to have a significant effect on the dependent variable.

In other words, the p-value is the probability of observing this relationship between the variables in this sample if the null hypothesis were true.

If, in the population from which this sample was randomly drawn, the null hypothesis was correct, what are the chances of getting a test statistic at least as extreme as the one we got, in a sample of the size we have? -Peter Flom

Regression Analysis with Categorical Variables

Label Encoding

Most Machine learning algorithms require categorical variables to be converted to numerical values and many algorithm’s performance varies based on how Categorical variables are encoded. In this case, all of our categorical data is Nominal as opposed to Ordinal.

As you can see, each category is assigned a value from 1 through N (where N is the number of categories for the feature.)

The p-values for Marital Status, Renew Offer type, Sales channel, Vehicle size, Policy variables are significant at the 0.05 significance level. Again, these variables have negative relationships with Response.

Regression Analysis with both Continuous and Categorical Variables

I will let you check this output for yourself. Can you tell which variables are important to include in our classification model? One interesting observation to make is that customers with a higher Customer Lifetime Value seem less likely to respond to marketing calls.

So why did we spend all of this time looking for “significant relationships”? This is one part of the process known as feature engineering. If we want to use a classification algorithm to predict whether or not a future customer will respond to a marketing call, we want to look at those features (fancy name for variable) which have a significant relationship with that dependent variable. Getting rid of variables which have no effect on the outcome has many benefits. Not only does it reduce the noise in the model, but it reduces the processing power required for training of the model.

Support Vector Machine Classification Algorithm

Support Vector Machine is a supervised ML algorithm that can be used for both classification and regression. In this algorithm, each data item is plotted as a point in n-dimensional space (where n is number of features you have.) Then, a Hyper-Plane is found which best segregates the classes in N-dimensional space.

Let’s explore the concept of a Hyper-Plane with this illustration that I spent too long putting together:

Support Vectors are the data points from each class that lie closest to the hyper-plane. The goal of SVM is to maximize the Margin, or distance between the two classes. If we were to move one of the support vectors in the example above, the hyper-plane would move with it.

In cases where we don’t have linearly separable data like the example above (which is often the case in reality), the Kernel Trick comes in handy. The idea behind the Kernel Trick is to map the non-linearly separable data into a higher dimensional space where we can create a hyperplane that can actually separate the classes. Check out this awesome GIF for an illustration:

We will save the Kernel Trick for another project. For now, let’s go ahead and split our data into training and testing sets. This is a key step in the data science process which allows you to honestly evaluate the performance of your model. This process can help to protect you from overfitting your model. However, it doesn’t matter how you slice and dice your data if your sample is not representative of the population you wish to apply the model to.

Another method we will use to improve our metrics is Cross Validation. As illustrated below, it splits the data into K (in this case, K=5) sets of your choosing. K-1 subsets are used to train our data and the last subset is for testing.We take the average accuracy of each of K “folds” as our final metric.

For more information on Train/Test Split and Cross Validation check out This Article

Scikit-Learn is tried and true. Let’s see what this model can do.

As we can see, both the train/test split and cross validation techniques led to very solid metrics. Based on this output, we would feel quite confident in our predictive ability of customer response rates utilizing this model.

Conclusion

If we were given a list of customers and their demographics, we could use the insight gathered from this project to make strategic decisions about how we engage with that customer.

Thank you for reading! I hope this article was both entertaining and insightful. I had a great time putting together my first article, and a hard time letting go of my perfectionism and hitting publish, haha. I welcome any constructive feedback. Feel free to connect with me on LinkedIn!