Building our first neural network in keras | by Sanchit Tanwar | Towards Data Science

Signup for my live computer vision course: https://bit.ly/cv_coursem
In this article, we will make our first neural network(ANN) using keras framework. This tutorial is part of the deep learning workshop. The link to lessons will be given below as soon as I update them. Github link of this repo is here. Link to the jupyter notebook of this tutorial is here.
Index
Introduction to machine learning and deep learning.Introduction to neural networks.Introduction to python.Building our first neural network in keras. < — You are hereA comprehensive guide to CNN.Image classification with CNN.
Introduction to machine learning and deep learning.
Introduction to neural networks.
Introduction to python.
Building our first neural network in keras. < — You are here
A comprehensive guide to CNN.
Image classification with CNN.
Before starting, I would like to give an overview of how to structure any deep learning project.
Preprocess and load data- As we have already discussed data is the key for the working of neural network and we need to process it before feeding to the neural network. In this step, we will also visualize data which will help us to gain insight into the data.Define model- Now we need a neural network model. This means we need to specify the number of hidden layers in the neural network and their size, the input and output size.Loss and optimizer- Now we need to define the loss function according to our task. We also need to specify the optimizer to use with learning rate and other hyperparameters of the optimizer.Fit model- This is the training step of the neural network. Here we need to define the number of epochs for which we need to train the neural network.
Preprocess and load data- As we have already discussed data is the key for the working of neural network and we need to process it before feeding to the neural network. In this step, we will also visualize data which will help us to gain insight into the data.
Define model- Now we need a neural network model. This means we need to specify the number of hidden layers in the neural network and their size, the input and output size.
Loss and optimizer- Now we need to define the loss function according to our task. We also need to specify the optimizer to use with learning rate and other hyperparameters of the optimizer.
Fit model- This is the training step of the neural network. Here we need to define the number of epochs for which we need to train the neural network.
After fitting model, we can test it on test data to check whether the case of overfitting. We can save the weights of the model and use it later whenever required.
We will use simple data of mobile price range classifier. The dataset consists of 20 features and we need to predict the price range in which phone lies. These ranges are divided into 4 classes. The features of our dataset include
'battery_power', 'blue', 'clock_speed', 'dual_sim', 'fc', 'four_g','int_memory', 'm_dep', 'mobile_wt', 'n_cores', 'pc', 'px_height','px_width', 'ram', 'sc_h', 'sc_w', 'talk_time', 'three_g','touch_screen', 'wifi'
Before feeding data to our neural network we need it in a specific way so we need to process it accordingly. The preprocessing of data depends on the type of data. Here we will discuss how to handle tabular data and in later tutorials, we will handle image dataset. Let’s start the coding part
#Dependenciesimport numpy as npimport pandas as pd#dataset importdataset = pd.read_csv(‘data/train.csv’) #You need to change #directory accordinglydataset.head(10) #Return 10 rows of data
Our dataset looks like this.
#Changing pandas dataframe to numpy arrayX = dataset.iloc[:,:20].valuesy = dataset.iloc[:,20:21].values
This code as discussed in python module will make two arrays X and y. X contains features and y will contain classes.
#Normalizing the datafrom sklearn.preprocessing import StandardScalersc = StandardScaler()X = sc.fit_transform(X)
This step is used to normalize the data. Normalization is a technique used to change the values of an array to a common scale, without distorting differences in the ranges of values. It is an important step and you can check the difference in accuracies on our dataset by removing this step. It is mainly required in case the dataset features vary a lot as in our case the value of battery power is in the 1000’s and clock speed is less than 3. So if we feed unnormalized data to the neural network, the gradients will change differently for every column and thus the learning will oscillate. Study further from this link.
The X will now be changed to this form:
Normalized data:[-0.90259726 -0.9900495   0.83077942 -1.01918398 -0.76249466 -1.04396559 -1.38064353  0.34073951  1.34924881 -1.10197128 -1.3057501  -1.40894856 -1.14678403  0.39170341 -0.78498329  0.2831028   1.46249332 -1.78686097 -1.00601811  0.98609664]
Next step is to one hot encode the classes. One hot encoding is a process to convert integer classes into binary values. Consider an example, let’s say there are 3 classes in our dataset namely 1,2 and 3. Now we cannot directly feed this to neural network so we convert it in the form:
1- 1 0 0
2- 0 1 0
3- 0 0 1
Now there is one unique binary value for the class. The new array formed will be of shape (n, number of classes), where n is the number of samples in our dataset. We can do this using simple function by sklearn:
from sklearn.preprocessing import OneHotEncoderohe = OneHotEncoder()y = ohe.fit_transform(y).toarray()
Our dataset has 4 classes so our new label array will look like this:
One hot encoded array:[[0. 1. 0. 0.] [0. 0. 1. 0.] [0. 0. 1. 0.] [0. 0. 1. 0.] [0. 1. 0. 0.]]
Now our dataset is processed and ready to feed in the neural network.
Generally, it is better to split data into training and testing data. Training data is the data on which we will train our neural network. Test data is used to check our trained neural network. This data is totally new for our neural network and if the neural network performs well on this dataset, it shows that there is no overfitting. Read more about this here.
from sklearn.model_selection import train_test_splitX_train,X_test,y_train,y_test = train_test_split(X,y,test_size = 0.1)
This will split our dataset into training and testing. Training data will have 90% samples and test data will have 10% samples. This is specified by the test_size argument.
Now we are done with the boring part and let’s build a neural network.
Keras is a simple tool for constructing a neural network. It is a high-level framework based on tensorflow, theano or cntk backends.
In our dataset, the input is of 20 values and output is of 4 values. So the input and output layer is of 20 and 4 dimensions respectively.
#Dependenciesimport kerasfrom keras.models import Sequentialfrom keras.layers import Dense# Neural networkmodel = Sequential()model.add(Dense(16, input_dim=20, activation=’relu’))model.add(Dense(12, activation=’relu’))model.add(Dense(4, activation=’softmax’))
In our neural network, we are using two hidden layers of 16 and 12 dimension.
Now I will explain the code line by line.
Sequential specifies to keras that we are creating model sequentially and the output of each layer we add is input to the next layer we specify.
model.add is used to add a layer to our neural network. We need to specify as an argument what type of layer we want. The Dense is used to specify the fully connected layer. The arguments of Dense are output dimension which is 16 in the first case, input dimension which is 20 for input dimension and the activation function to be used which is relu in this case. The second layer is similar, we dont need to specify input dimension as we have defined the model to be sequential so keras will automatically consider input dimension to be same as the output of last layer i.e 16. In the third layer(output layer) the output dimension is 4(number of classes). Now as we have discussed earlier, the output layer takes different activation functions and for the case of multiclass classification, it is softmax.
Now we need to specify the loss function and the optimizer. It is done using compile function in keras.
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
Here loss is cross entropy loss as discussed earlier. Categorical_crossentropy specifies that we have multiple classes. The optimizer is Adam. Metrics is used to specify the way we want to judge the performance of our neural network. Here we have specified it to accuracy.
Now we are done with building a neural network and we will train it.
Training step is simple in keras. model.fit is used to train it.
history = model.fit(X_train, y_train, epochs=100, batch_size=64)
Here we need to specify the input data-> X_train, labels-> y_train, number of epochs(iterations), and batch size. It returns the history of model training. History consists of model accuracy and losses after each epoch. We will visualize it later.
Usually, the dataset is very big and we cannot fit complete data at once so we use batch size. This divides our data into batches each of size equal to batch_size. Now only this number of samples will be loaded into memory and processed. Once we are done with one batch it is flushed from memory and the next batch will be processed.
Now we have started the training of our neural network.
Epoch 1/1001600/1600 [==============================] - 1s 600us/step - loss: 1.3835 - acc: 0.3019Epoch 2/1001600/1600 [==============================] - 0s 60us/step - loss: 1.3401 - acc: 0.3369Epoch 3/1001600/1600 [==============================] - 0s 72us/step - loss: 1.2986 - acc: 0.3756Epoch 4/1001600/1600 [==============================] - 0s 63us/step - loss: 1.2525 - acc: 0.4206Epoch 5/1001600/1600 [==============================] - 0s 62us/step - loss: 1.1982 - acc: 0.4675...Epoch 97/1001600/1600 [==============================] - 0s 55us/step - loss: 0.0400 - acc: 0.9937Epoch 98/1001600/1600 [==============================] - 0s 62us/step - loss: 0.0390 - acc: 0.9950Epoch 99/1001600/1600 [==============================] - 0s 57us/step - loss: 0.0390 - acc: 0.9937Epoch 100/1001600/1600 [==============================] - 0s 60us/step - loss: 0.0380 - acc: 0.9950
It will take around a minute to train. And after 100 epochs the neural network will be trained. The training accuracy is reached 99.5 % so our model is trained.
Now we can check the model’s performance on test data:
y_pred = model.predict(X_test)#Converting predictions to labelpred = list()for i in range(len(y_pred)):    pred.append(np.argmax(y_pred[i]))#Converting one hot encoded test label to labeltest = list()for i in range(len(y_test)):    test.append(np.argmax(y_test[i]))
This step is inverse one hot encoding process. We will get integer labels using this step. We can predict on test data using a simple method of keras, model.predict(). It will take the test data as input and will return the prediction outputs as softmax.
from sklearn.metrics import accuracy_scorea = accuracy_score(pred,test)print('Accuracy is:', a*100)
We get an accuracy of 93.5%.
We can use test data as validation data and can check the accuracies after every epoch. This will give us an insight into overfitting at the time of training only and we can take steps before the completion of all epochs. We can do this by changing fit function as:
history = model.fit(X_train, y_train,validation_data = (X_test,y_test), epochs=100, batch_size=64)
Now the training step output will also contain validation accuracy.
Train on 1600 samples, validate on 400 samplesEpoch 1/1001600/1600 [==============================] - 1s 823us/step - loss: 1.4378 - acc: 0.2406 - val_loss: 1.4118 - val_acc: 0.2875Epoch 2/1001600/1600 [==============================] - 0s 67us/step - loss: 1.3852 - acc: 0.2825 - val_loss: 1.3713 - val_acc: 0.3175Epoch 3/1001600/1600 [==============================] - ETA: 0s - loss: 1.3474 - acc: 0.326 - 0s 50us/step - loss: 1.3459 - acc: 0.3231 - val_loss: 1.3349 - val_acc: 0.3650Epoch 4/1001600/1600 [==============================] - 0s 56us/step - loss: 1.3078 - acc: 0.3700 - val_loss: 1.2916 - val_acc: 0.4225Epoch 5/1001600/1600 [==============================] - 0s 74us/step - loss: 1.2600 - acc: 0.4094 - val_loss: 1.2381 - val_acc: 0.4575....Epoch 95/1001600/1600 [==============================] - 0s 37us/step - loss: 0.0615 - acc: 0.9869 - val_loss: 0.1798 - val_acc: 0.9250Epoch 96/1001600/1600 [==============================] - 0s 43us/step - loss: 0.0611 - acc: 0.9850 - val_loss: 0.1812 - val_acc: 0.9225Epoch 97/1001600/1600 [==============================] - 0s 45us/step - loss: 0.0595 - acc: 0.9894 - val_loss: 0.1813 - val_acc: 0.9275Epoch 98/1001600/1600 [==============================] - 0s 44us/step - loss: 0.0592 - acc: 0.9869 - val_loss: 0.1766 - val_acc: 0.9275Epoch 99/1001600/1600 [==============================] - 0s 43us/step - loss: 0.0575 - acc: 0.9894 - val_loss: 0.1849 - val_acc: 0.9275Epoch 100/1001600/1600 [==============================] - 0s 38us/step - loss: 0.0574 - acc: 0.9869 - val_loss: 0.1821 - val_acc: 0.9275
Our model is working fine. Now we will visualize training and validation losses and accuracies.
import matplotlib.pyplot as pltplt.plot(history.history['acc'])plt.plot(history.history['val_acc'])plt.title('Model accuracy')plt.ylabel('Accuracy')plt.xlabel('Epoch')plt.legend(['Train', 'Test'], loc='upper left')plt.show()
plt.plot(history.history['loss']) plt.plot(history.history['val_loss']) plt.title('Model loss') plt.ylabel('Loss') plt.xlabel('Epoch') plt.legend(['Train', 'Test'], loc='upper left') plt.show()
In the next chapter, we will discuss the convolution neural network(CNN) used for image data. In case of any doubt or if I have done anything wrong please comment.