Building your own Artificial Neural Network from scratch on Churn Modeling Dataset using Keras in Python | by Siddharth Dixit | Towards Data Science

It is no secret that customer retention is a top priority for many companies;acquiring new customers can be several times more expensive than retaining existing ones. Furthermore, gaining an understanding of the reasons customers churn and estimating the risk associated with individual customers are both powerful components of designing a data-driven retention strategy. A churn model can be the tool that brings these elements together and provides insights and outputs that drive decision making across an organization.So it is used to predict as to which customers will be leaving the bank.
Churn rate (sometimes called attrition rate), in its broadest sense, is a measure of the number of individuals or items moving out of a collective group over a specific period. It is one of two primary factors that determine the steady-state level of customers a business will support.
You can be someone who has just completed a MOOC on Machine Learning or a budding Data Scientist looking for a more Practical/Hands on Project based on Machine Learning.
This can also be relevant if you are an Economics Major trying to combine the aspects of Data Science with something related to your field.
So in this dataset we would be dealing with Churn Modeling i.e. we would be writing a Artificial Neural Network to find out reasons as to why and which customers are actually leaving the bank and their dependencies on one another.This is a classification problem i.e. a 0-1 classification(1 if customer Leaves and 0 if customer stays)We might use Theano or Tensorflow but the thing is that these libraries require us to write most of the Ml code from scratch so instead we make use of Keras which enables us writing powerful Neural Networks with a few lines of code Keras runs on Theano and Tensorflow and you can think it of as a Sklearn for Deep Learning.
# Importing the librariesimport numpy as npimport matplotlib.pyplot as pltimport pandas as pd
Save it in Pycharm Projects/Name of Project and load it using pandas.
dataset = pd.read_csv('Churn_Modelling.csv')
Looking at the features we can see that row number, name will have no relation with a customer with leaving the bank.So we drop them from X which contains the features Indexes from 3 to 12.
X = dataset.iloc[:, 3:13].values
We store the Dependent value/predicted value in y by storing the 13th index in the variable y
y = dataset.iloc[:, 13].values
Printing out the values of ...
X → Which contains the featuresy → Which contains the target variable
print(X)print(y)
Encoding categorical data... Now we encode the string values in the features to numerical values as a ML Algorithm can only work on numbers and not on string values. The only 2 values are Gender and Region which need to converted into numerical data
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
Creating label encoder object no. 1 to encode region name(index 1 in features)
labelencoder_X_1 = LabelEncoder()
Encoding region from string to just 3 no.s 0,1,2 respectively
X[:, 1] = labelencoder_X_1.fit_transform(X[:, 1])
Creating label encoder object no. 2 to encode Gender name(index 2 in features)
labelencoder_X_2 = LabelEncoder()
Encoding Gender from string to just 2 no.s 0,1(male,female) respectively
X[:, 2] = labelencoder_X_2.fit_transform(X[:, 2])
Now creating Dummy variables
onehotencoder = OneHotEncoder(categorical_features = [1])X = onehotencoder.fit_transform(X).toarray()X = X[:, 1:]
Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)
Feature Scaling
from sklearn.preprocessing import StandardScalersc = StandardScaler()X_train = sc.fit_transform(X_train)X_test = sc.transform(X_test)
Listing out the steps involved in training the ANN with Stochastic Gradient Descent:-1)Randomly initialize the weights to small numbers close to 0(But not 0).2)Input the 1st observation of your dataset in the Input Layer, each Feature in one Input Node.3)Forward-Propagation from Left to Right, the neurons are activated in a way that the impact of each neuron’s activation.is limited by the weights.Propagate the activations until getting the predicted result y.4)Compare the predicted result with the actual result. Measure the generated error.5)Back-Propagation: From Right to Left, Error is back propagated.Update the weights according to how much they areresponsible for the error.The Learning Rate tells us by how much such we update the weights.6)Repeat Steps 1 to 5 and update the weights after each observation(Reinforcement Learning).Or: Repeat Steps 1 to 5 but update the weights only after a batch of observations(Batch Learning).7)When the whole training set is passed through the ANN.That completes an Epoch. Redo more Epochs.
import keras
For building the Neural Network layer by layer
from keras.models import Sequential
To randomly initialize the weights to small numbers close to 0(But not 0)
from keras.layers import Dense
So there are actually 2 ways of initializing a deep learning model— — — 1)Defining each layer one by one— — — 2)Defining a Graph
We did not put any parameter in the Sequential object as we will be defining the Layers manually
classifier = Sequential()
This remains an unanswered question till date that how many nodes of the hidden layer do we actually need?There is no thumb rule but you can set the number of nodes in Hidden Layers as an Average of the number of Nodes in Input and Output Layer Respectively.(Works in 90% of the cases!!) — →Here avg= (11+1)/2==>6 So set Output Dim=6 — →Init will initialize the Hidden Layer weights uniformly — →Activation Function is Rectifier Activation Function(Relu)
Input dim tells us the number of nodes in the Input Layer.This is done only once and wont be specified in further layers.
classifier.add(Dense(output_dim = 6, init = 'uniform', activation = 'relu', input_dim = 11))
classifier.add(Dense(output_dim = 6, init = 'uniform', activation = 'relu'))
classifier.add(Dense(output_dim = 1, init = 'uniform', activation = 'sigmoid'))
Sigmoid activation function is used whenever we need Probabilities of 2 categories or less(Similar to Logistic Regression)Switch to Softmax when the dependent variable has more than 2 categories.
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
Fitting the ANN to the Training set
classifier.fit(X_train, y_train, batch_size = 10, nb_epoch = 100)
Predicting the Test set results
y_pred = classifier.predict(X_test)y_pred = (y_pred > 0.5)
if y_pred is larger than 0.5 it returns true(1) else false(2)
print(y_pred)
This Model when trained on the train data and when tested on the test data gives us an accuracy of around 86% in both of the cases.Which from our point of view is Great!!!
Making the Confusion Matrix
from sklearn.metrics import confusion_matrixcm = confusion_matrix(y_test, y_pred)print(cm)
Obtained from Confusion Matrix.You may change values as per what is obtained in your confusion matrix.
print(accuracy)
Congratulations! you just wrote your own Neural Network for theBank which had given you this task. You may now find out the Top 10% customers which are most likely to leave the bank(which is quite important for the bank).The Bank may now take corrective measures to avoid losing it’s customers and would not have done this without your help :P
I hope you liked my Tutorial! I am attaching the github(Now Gitlab :P) links to the code and the dataset.
github.com
References-Machine Learning course by Kirill Eremenko and Hadelin de Ponteves