Customers Subscription Analysis and Prediction Based on App Behavior Analysis (Logistic Regression) | by Shekhar Koirala | Towards Data Science

We will do the customers churn analysis based on the customers behavior on the website or app. We will classify what kind of customers are likely to sign up for the paid subscription of a website. After analyzing and classifying the dataset, we will be able to do the targeting based marketing or recommendation to the customers who are likely to sign up for the paid subscription plan.
For this problem we will use Logistic Regression algorithm from scikit-learn library. The dataset for this problem has been optimized with necessary feature engineering that has already been done. If you wanna find out how I cleaned the data and how the feature engineering was done, please stay tuned as i will be writing another post soon about the data cleaning process on this particular dataset.
Let’s start the process by first importing the necessary libraries and the dataset.
import numpy as np import pandas as pdimport matplotlib.pyplot as pltimport seaborn as snsfrom dateutil import parser %matplotlib qt5 
Here %matplotlib qt5 makes the visualization pop up in a new window. If you want the visualization to appear in the same notebook or console, then you can use %matplotlib inline
dataset = pd.read_csv(“appdata10.csv”) 
Let’s look at the dataset:
Now let’s separate the dataset into the features and the target variables. The target column name is “enrolled”.
X = dataset.drop(columns = 'enrolled')y = dataset['enrolled'] 
Now let’s split the dataset into training set and testing set. We need to do this so that we can only use the training set data in our model training. Splitting the dataset into training set and testing set is important in supervised machine learning as it prevents overfitting.
To do this I will use train_test_split() function from sklearn.
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 4) 
You can use any random state, it just ensures that results in different environment are same.
Let’s remove the user column for now because we don’t need it and its not a required feature. But we need it for the purpose of identifying the user that subscribed or not. For this we are going to save the user column as identifier.
train_identifier = X_train['user']X_train = X_train.drop(columns='user')test_identifier = X_test['user']X_test = X_test.drop(columns='user')
The next step is to do feature scaling. Feature scaling is done to normalize the values of different features in the dataset. Feature scaling helps us to achieve the gradient decent more quickly. If the data is more spread out, that means if it has a higher standard deviation, it will relatively take more time to calculate the gradient decent compared to the situation when we scale our data via feature scaling. So we are making all the values within a certain range. This treats all the columns equally. In the above dataset, we can see that the values of age, days of week, enrolled are not in similar range.
To do this we can use sklearn’s StandardScaler function.
from sklearn.preprocessing import StandardScalersc_X = StandardScaler()X_train2 = pd.DataFrame(sc_X.fit_transform(X_train))X_test2 = pd.DataFrame(sc_X.fit_transform(X_test))X_train2.columns = X_train.columns.valuesX_test2.columns = X_test.columns.valuesX_train.index = X_train.index.valuesX_test.index = X_test.index.valuesX_train = X_train2X_test = X_test2
Now let’s get into the exciting part; training the dataset. For this we will use LogisticRegression from sklearn library.
Logistic Regression is used for classification problem. It is the appropriate regression analysis to conduct when the dependent variable is dichotomous (binary). Like all regression analyses, the logistic regression is a predictive analysis. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables.
from sklearn.linear_model import LogisticRegression logreg = LogisticRegression(random_state=0, penalty=’l1')logreg.fit(X_train, y_train)
Here I am using penalty parameter to prevent one feature being very highly correlated to the target variable in this particular dataset, especially for the last screen variable.
Now that the model has been trained, we can do the prediction on the test set.
y_pred = logreg.predict(X_test)
After doing the prediction, we can do the model evaluation to see how our model performed. For this we will use five evaluation metrics, confusion matrix, accuracy score, f1 score, precision score and recall score. All these evaluation metrices are available in scikit learn.
from sklearn.metrics import confusion_matrix, accuracy_score, f1_score, precision_score, recall_scorecm = confusion_matrix(y_test, y_pred)accuracy = accuracy_score(y_test, y_pred)accuracy 0.7679
So we achieved the accuracy of 79%. The model has done a descent job with that accuracy.
Now let’s plot the heat map of the confusion matrix to visualize the confusion matrix.
We can see that the correctly predicted values are around 3 times more than the false positive and false negative in the confusion matrix. We can further use cross validation to evaluate the model and make sure our model is doing the right job.
from sklearn.model_selection import cross_val_scoreaccuracies = cross_val_score(estimator=logreg, X=X_train, y= y_train, cv=10)print("Logistic accuracy: %0.3f (+/- %0.3f)(accuracies.mean(),accuracies.std()*2))Output: Logistic accuracy: 0.767 (+/- 0.010)
So we can say that our model is doing the right job and a descent job with LogisticRegression.
We can further optimize our model using different parameter tuning methods. But for now I am not going to optimize this model. If you want to learn about parameter/hyper parameter tuning, you can search GridSearch, Randomized search, Backward elimination. I’ll leave those for next post.
This is my first medium post, so please feel free to give me suggestions and constructive criticism on this post and also about the method Iused to do this classification. Thank you.