Project: Modeling & Predicting of Churning Customers (in R)
By: Jarar Zaidi
Tags: R, Data Science, Modeling, Customer Success, Machine Learning

4. Modeling + Confidence Intervals

Our goal in modeling is to provide a simple low-dimensional summary of a dataset.We use models to partition data into patterns and residuals. Fitted model is just the closest model from a family of models.

Classification models are models that predict a categorical label. It will be interesting to study which characteristic(s) discriminates each category and to what extent. Predicting whether a customer will a customer will exit or stay with the company. Below we will use a logistic regression algorithm.

First we must convert the non numeric variables (discrete variables) into factors. Factors are the data objects which are used to categorize the data and store it as levels.

4 Steps: DATA PARTITION, BUILD, PREDICT & EVALUATE

```{r Modeling1}

library(tidyverse)

library(modelr)

library(plyr)

library(readr)

library(dplyr)

library(caret) # glimpse is from dplyr

# output shows that the dataset has

glimpse(dataCC4) # convert the non numeric into factors.

names <- c(2,5,7,9) # col numbers that are chars

dataCC4[,names] <- lapply(dataCC4[,names] , factor)

```

DATA PARTITIONING

```{r Modeling2}

#----------------------------

# DATA PARTITIONING

#----------------------------

#build our model on the Train dataset & evaluate its performance on #the Test dataset

# aka. holdout-validation approach to evaluating model performance. set.seed(100) # sets random seed for reproducibility of results

library(caTools) # for data partioning # create the training and test datasets. The train dataset contains # 70 percent of the data (420 observations of 10 variables) while #the #test data contains the remaining 30 percent (180 observations #of 10 variables).

spl = sample.split(dataCC4$Attrition_Flag, SplitRatio = 0.7) train = subset(dataCC4, spl==TRUE)

test = subset(dataCC4, spl==FALSE) print(dim(train)) # 4957 rows/obs 9 cols/variables

print(dim(test)) # 2124 rows/obs 9 cols/variables ```

BUILD, PREDICT & EVALUATE the Model

We fit the logistic regression model. The first step is to instantiate the algorithm. We declare binomial because 2 possible outcomes: exit/current.

```{r Modeling3}

#----------------------------

#BUILD, PREDICT & EVALUATE the Model #----------------------------

# BUILD

#---------------------------- model_glm = glm(Attrition_Flag ~ . , family="binomial",data = train)

summary(model_glm) # Baseline Accuracy

#Let's evaluate the model further,

# Since the majority class of the target (Y) variable has a #proportion of 0.84, the baseline accuracy is 84 percent.

prop.table(table(train$Attrition_Flag)) #Let's now evaluate the model performance on the training and test data, which should ideally be better than the baseline accuracy. # PREDICTIONS on the TRAIN set

predictTrain = predict(model_glm, data = train, type = "response") # confusion matrix w/ threshold of 0.5,

#which means that for probability predictions >= 0.5, the #algorithm will predict the Current response for the Y #variable.

table(train$Attrition_Flag, predictTrain >= 0.1) Evaluate: #prints the accuracy of the model on the training data, using the #confusion matrix, and the accuracy comes out to be 0 percent.

(50+774)/nrow(test) #Accuracy - 84% (50+774)/2124 -------------------------------------------------------------

#We then repeat this process on the TEST data, and the accuracy #comes out to be 84 percent. # PREDICTIONS on the TEST set

predictTest = predict(model_glm, newdata = test, type = "response") Evaluate: # Confusion matrix on test set

table(test$Attrition_Flag, predictTest >= 0.5) 1790/nrow(test) #Accuracy - 84% ```

The significance code ‘***’ in the above below output shows the relative importance of the feature variables

AIC estimates the relative amount of information lost by a given model:

the less information a model loses, the higher the quality of that model.

Lower AIC values indicate a better-fit model,

output of model_glm = glm(Attrition_Flag ~ . , family=”binomial”,data = train)

summary(model_glm)

84% Accuracy

You have learned techniques of building a classification model in R using the powerful logistic regression algorithm. The baseline accuracy for the training & test data was 84 percent. Overall, the logistic regression model is beating the baseline accuracy by a big margin on both the train and test datasets, and the results are very good.