Practical ML Part 3: Predicting Breast Cancer with Pytorch
By: Mariana Almeida
Tags: Machine Learning, Deep Learning, Neural Networks, Data Science, Pytorch

From this preliminary we can get the following insights:

This is a quite complex model to be able to visualize the correlations.

The tumors have on average a radius of 14.12, being the smallest one 6.98 and the biggest one 28.11.

Variables related to dimensions are positively correlated (area_se and perimeter_se for example), as one is dependent on the other.

62.7% of the tumors in the dataset are benign and 37.3% are malignant.

3 — Preparing the data for Machine Learning

Changing the Labels

The values we want to predict are in the column ‘diagnosis’ — these are the labels. We want to predict whether a tumor is malignant (‘M’) or benign (‘B). All the other columns in the data frame are the predictors for the model:

The labels are represented by letters M (malignant) and B (benign). First step is to transform this information into a numerical type of data, being 0 for benign and 1 for malignant:

lab ={'B':0,'M':1}

df = df.replace({'diagnosis':lab})

df.head()

Now the data can be split into two subsets:

predictors — the information we will provide as input to the model to make the predictions

predictors = df.iloc[:, 2:31]

predictors

labels — the information we want to predict, which is whether given the information in the predictors dataset, the tumor is malignant or benign.

labels = df.iloc[:, 0:1]

labels

Transformation to tensors and splitting the data into train and test subsets

Using scikit learn, the datasets were split into training and testing data, using a ratio of 20%:

predictors_train, predictors_test, labels_train, labels_test = train_test_split(predictors,labels,test_size = 0.20)

So far the data is stored into pandas data frames. As we will implement a deep learning model using Tensor Flow, the data must be transformed into tensors. First, converting the pandas data frames into arrays:

type(np.array(predictors_train))

Out[20]:

type(np.array(labels_train))

Out[21]:

numpy.ndarray

The model will be implemented using PyTorch, so the next step is to transform the arrays into a torch element:

predictors_train = torch.tensor(np.array(predictors_train), dtype=torch.float) labels_train = torch.tensor(np.array(labels_train), dtype = torch.float) df_tf = torch.utils.data.TensorDataset(predictors_train, labels_train)

type(df_tf)

Pytorch trains the models in mini-batches. There is a class named DataLoader to perform the iterations on the dataset. The batch_size parameter gives the number of samples considered when adjusting the weights of the model:

train_loader = torch.utils.data.DataLoader(df_tf, batch_size=15, shuffle=True)

4 — Implementing and evaluating the model

In this project, a neural network with two hidden layers will be implemented. To implement such model, we need the following:

Structure of the model (defining number of layers, neurons, activation functions);

Choose a training criterion;

Choose an optimizer.

Due to the complexity and the unknowns in the problem, this type of implementation is modeled as an optimization problem. So, choosing an optimizer is part of the process.

In an optimization problem, we have the objective function (known as Cost Function or Loss Function). This is the function we want to optimize. In neural networks the objective is to minimize the error, therefore minimize the loss (or cost) function.

Structuring the model

The neural network model definition is as follows:

classifier = nn.Sequential(



nn.Linear(in_features=29, out_features=15),

nn.ReLU(),

nn.Linear(15, 15),

nn.ReLU(),

nn.Linear(15, 1),

nn.Sigmoid()

)

Input features = 29 (we have 29 features in the predictors dataset).

2 Hidden layers with 15 neurons each

Output layer with one neuron, having output labels 0 (benign tumor) and 1 (malignant tumor)

ReLu , a non linear activation function for the internal layers

, a non linear activation function for the internal layers Sigmoid, a non linear activation function for the output layer, returning a probability between 0 and 1

ReLu (Rectified Linear Unit), is a non-linear activation function that has the advantage of not activating all neurons at the same time. ReLu was chosen in a first attempt to build model more efficient in terms of computational costs.

Sigmoid reportedly works well in classification problems. As we have an output layer classifying the tumor as benign or malignant, sigmoid was chosen as the activation function.

Training Criterion

Now, we must choose the training criterion for the neural network. As this is a binary classification task, a binary cross entropy criterion was the choice:

criterion = nn.BCELoss()

Optimizer

Third, and final step is to choose the optimizer. Adam optimization algorithm is a very popular choice in deep learning problems. This is an extension of the stochastic gradient descent algorithm, but different than the SGD algorithm, Adam optimizer does not maintain the same learning rate during training.

optimizer = torch.optim.Adam(classifier.parameters(), lr=0.001, weight_decay=0.0001)

To read more about the Adam optimizer, check this article in Towards Data Science.

Training the model

The weights updated is run 100 times, as shown in the for loop below:

for epoch in range(100):

#To store the error:

run_loss = 0.



for data in train_loader:

inputs, labels = data

optimizer.zero_grad()



outputs = classifier(inputs) error = criterion(outputs, labels)

error.backward()

optimizer.step()



run_loss += error.item()

print('Epoch %3d: loss %.5f' % (epoch+1, run_loss/len(train_loader)))

The train loader loads the training data in batches (remembering that the batch size chosen for this model was of 15 samples).

zero_grad() set the gradients to zero before the backpropagation process.

set the gradients to zero before the backpropagation process. outputs computes the predictions to the model.

computes the predictions to the model. error computes the error in the calculations, comparing the predictions with the real data.

computes the error in the calculations, comparing the predictions with the real data. error.backward() is the backpropagation process in the neural network to update the weights.

is the backpropagation process in the neural network to update the weights. optimizer.step() to update the weights.

Evaluating the model

The model had an accuracy of 94.73%. Visualizing this in a confusion matrix we have:

The classifier made a total of 114 predictions, from which: