PyTorch For Deep Learning — Binary Classification ( Logistic Regression )
By: Ashwin Prasad
Tags: Deep Learning, Logistic Regression, Classification Models, Pytorch Tutorial, Pytorch

This blog post is for how to create a classification neural network with PyTorch.

Note : The neural network in this post contains 2 layers with a lot of neurons. but, if the number of out features and number of layers are reduced to 1, this would just become an ordinary logistic regression

Having said that, let’s jump into the code

Importing the libraries

#importing the libraries import torch

import numpy as np

import matplotlib.pyplot as plt

2. Dataset

For this post, we are going to be using sklearn’s famous breast_cancer dataset

#importing the dataset from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()

x = data['data']

y = data['target']

print("shape of x: {}

shape of y: {}".format(x.shape,y.shape)) output:

shape of x: (569, 30)

shape of y: (569,)

3. Feature Scaling

It is important to scale the features to a standard normal before sending it to the neural network.

basically, This subtracts the mean of the column and divides by the standard deviation of a column for each value in the column ( Independent Variable)

#feature scaling from sklearn.preprocessing import StandardScaler

sc = StandardScaler()

x = sc.fit_transform(x)

4. Dataset and DataLoader

Dataset class in pytorch basically covers the data in a tuple and enables us to access the index of each data. this is necessary to create dataloader class which can be used to shuffle, apply Mini-Batch Gradient Descent and more.

Check out the previous post for more examples on how this works

#defining dataset class from torch.utils.data import Dataset, DataLoader class dataset(Dataset):

def __init__(self,x,y):

self.x = torch.tensor(x,dtype=torch.float32)

self.y = torch.tensor(y,dtype=torch.float32)

self.length = self.x.shape[0]



def __getitem__(self,idx):

return self.x[idx],self.y[idx] def __len__(self):

return self.length trainset = dataset(x,y) #DataLoader

trainloader = DataLoader(trainset,batch_size=64,shuffle=False)

5. Neural Network for Classsification

in Pytorch, neural networks are created by using Object Oriented Programming.The layers are defined in the init function and the forward pass is defined in the forward function , which is invoked automatically when the class is called.

These Functions are possible because of the class nn.Module from torch which was inherited.

The output of the neural network is between 0 and 1 as sigmoid function is applied to the output which makes the network suitable for binary classification.

#defining the network from torch import nn

from torch.nn import functional as F class Net(nn.Module):

def __init__(self,input_shape):

super(Net,self).__init__()

self.fc1 = nn.Linear(input_shape,32)

self.fc2 = nn.Linear(32,64)

self.fc3 = nn.Linear(64,1) def forward(self,x):

x = torch.relu(self.fc1(x))

x = torch.relu(self.fc2(x))

x = torch.sigmoid(self.fc3(x))

return x

If simple logistic regression is enough , the layer fc2 and fc3 could be removed.

6. Some more Parameters

Selecting various parameters such as number of epochs , loss function , learning rate and more

#hyper parameters learning_rate = 0.01

epochs = 700 # Model , Optimizer, Loss model = Net(input_shape=x.shape[1])

optimizer = torch.optim.SGD(model.parameters(),lr=learning_rate)

loss_fn = nn.BCELoss()

BCELoss is a pytorch class for Binary Cross Entropy loss which is the standard loss function used for binary classification.

Training

The Gradients that are found from the loss function are used to change the values of the weights and the process is repeated several times.

This is done to minimize the loss function and increase the accuracy

Also , the Dataset is not split into training and test set because the amount of data is already low

#forward loop losses = []

accur = [] for i in range(epochs):

for j,(x_train,y_train) in enumerate(trainloader):



#calculate output

output = model(x_train)



#calculate loss

loss = loss_fn(output,y_train.reshape(-1,1))



#accuracy

predicted = model(torch.tensor(x,dtype=torch.float32))

acc = (predicted.reshape(-1).detach().numpy().round() == y).mean() #backprop

optimizer.zero_grad()

loss.backward()

optimizer.step()

if i%50 == 0:

losses.append(loss)

accur.append(acc)

print("epoch {}\tloss : {}\t accuracy : {}".format(i,loss,acc)) output:

epoch 0 loss : 0.6731628775596619 accuracy : 0.6274165202108963 epoch 50 loss : 0.2154722362756729 accuracy : 0.9507908611599297 epoch 100 loss : 0.1049698144197464 accuracy : 0.9718804920913884 epoch 150 loss : 0.08159173280000687 accuracy : 0.9824253075571178 epoch 200 loss : 0.06935682147741318 accuracy : 0.984182776801406 epoch 250 loss : 0.0607854388654232 accuracy : 0.984182776801406 epoch 300 loss : 0.053541962057352066 accuracy : 0.9859402460456942 epoch 350 loss : 0.047878947108983994 accuracy : 0.9876977152899824 epoch 400 loss : 0.043598778545856476 accuracy : 0.9894551845342706 epoch 450 loss : 0.039949361234903336 accuracy : 0.9912126537785588 epoch 500 loss : 0.036903925240039825 accuracy : 0.9929701230228472 epoch 550 loss : 0.03445163369178772 accuracy : 0.9929701230228472 epoch 600 loss : 0.032331496477127075 accuracy : 0.9929701230228472 epoch 650 loss : 0.030418962240219116 accuracy : 0.9929701230228472

detach() function removes the requires_grad from the tensor so that it can be converted to numpy and accuracy is a list that stores the accuracy at each epoch.

Except that, Everything here is self explanatory if all the previous posts have been read.

Analysis of the Model

plotting loss and accuracy over epochs to see how it changed over training

#plotting the loss plt.plot(losses)

plt.title('Loss vs Epochs')

plt.xlabel('Epochs')

plt.ylabel('loss')

#printing the accuracy plt.plot(accur)

plt.title('Accuracy vs Epochs')

plt.xlabel('Accuracy')

plt.ylabel('loss')

This loss and accuracy plot proves that our model has learnt well.

Thank You