Skin cancer classification with machine learning | by Nyla Pirani | Towards Data Science

Skin cancer is the most common type of skin cancer is the US. More than 4 million cases of skin cancer are diagnosed in the US a year.
This is huge! 4 million people possibly dying a year just from skin cancer. How crazy is that to think about.
Now all those people are dying but guess what! About half of those people, maybe even more, don’t even go to the doctor during the early stages when it can be prevented.
Even if people are getting symptoms, they still don’t want to go to the doctor. This is crazy. Especially because skin cancer is much easier to treat in the early stages and it grows super fast.
Now I can’t magically make people go to the doctor. Or can I...
No, you got me. I’m not a wizard. But what I can do is get people to detect the skin cancer themselves at home. All you need is a laptop and some lines of code.
So as I said skin cancer is a huge killer in the US and all over the world really. But the thing is it’s preventable in the early stages but people just don’t want to go to the doctors.
So, I used machine learning to come up with a way for people to be able to check if they have skin cancer from the comfort of their own home.
Data set:
I used PyTorch to code this. First, you need to import the data set.
For this part, I used some code from this kernel that had all the data in it.
base_skin_dir = os.path.join('..', 'input')imageid_path_dict = {os.path.splitext(os.path.basename(x))[0]: x                     for x in glob(os.path.join(base_skin_dir, '*', '*.jpg'))}lesion_type_dict = {    'nv': 'Melanocytic nevi',    'mel': 'dermatofibroma',    'bkl': 'Benign keratosis-like lesions ',    'bcc': 'Basal cell carcinoma',    'akiec': 'Actinic keratoses',    'vasc': 'Vascular lesions',    'df': 'Dermatofibroma'tile_df = pd.read_csv(os.path.join(base_skin_dir, 'HAM10000_metadata.csv'))tile_df['path'] = tile_df['image_id'].map(imageid_path_dict.get)tile_df['cell_type'] = tile_df['dx'].map(lesion_type_dict.get) tile_df['cell_type_idx'] = pd.Categorical(tile_df['cell_type']).codestile_df[['cell_type_idx', 'cell_type']].sort_values('cell_type_idx').drop_duplicates()
Then you should get an output that looks like this. As you can see we have a nice overview of our ground truth data. The only part we need is the column ‘cell_type_idx’, because this data is what we need for the model training.
Even though only the one column is what we really need it can still be a good idea to learn what everything means. So let's look quickly at how often these other tumors that are in our table actually occur in the dataset.
tile_df['cell_type'].value_counts()
After you put in that code you should get an output that looks like this:
As you can see, the Melanocytic nevi occur nearly 58 times as often as the Dermatofibroma. It can happen that the Melanocytic nevi are preferred in the prediction compared to the Dermatofibroma. One solution could be to show less frequent classes more often in the training. But this isn't a huge problem that needs to be solved right now and we can do just fine without it.
Let’s look more at the complete table:
tile_df.sample(3)
The table above can be used to get the input data using the path. The corresponding ground truth label is already given in the same line by the column ‘cell_type_idx’. Later we will create an input batch X of several loaded images and a corresponding ground-truth value y given by the corresponding ground-truth labels.
But, before we get to that we need to do something else first. Choosing a model.
PyTorch has a feature that has well-established models. These models can optionally already be trained on the ImageNet dataset, causing the training time to be lower in general.
So, let's load a pre-trained ResNet50 and adjust the last layer a little bit.
import torchvision.models as modelsmodel_conv = models.resnet50(pretrained=True)         Downloading: "https://download.pytorch.org/models/resnet50-                  19c8e357.pth" to /tmp/.torch/models/resnet50-19c8e357.pth 100%       ██████████| 102502400/102502400 [00:01<00:00, 83469977.65it/s]print(model_conv)     ResNet(  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  (relu): ReLU(inplace)  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)  (layer1): Sequential(    (0): Bottleneck(      (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)      (downsample): Sequential(        (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      )    )    (1): Bottleneck(      (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )    (2): Bottleneck(      (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )  )  (layer2): Sequential(    (0): Bottleneck(      (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)      (downsample): Sequential(        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      )    )    (1): Bottleneck(      (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )    (2): Bottleneck(      (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )    (3): Bottleneck(      (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )  )  (layer3): Sequential(    (0): Bottleneck(      (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)      (downsample): Sequential(        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      )    )    (1): Bottleneck(      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )    (2): Bottleneck(      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )    (3): Bottleneck(      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )    (4): Bottleneck(      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )    (5): Bottleneck(      (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )  )  (layer4): Sequential(    (0): Bottleneck(      (conv1): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)      (downsample): Sequential(        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      )    )    (1): Bottleneck(      (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )    (2): Bottleneck(      (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)      (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      (relu): ReLU(inplace)    )  )  (avgpool): AvgPool2d(kernel_size=7, stride=1, padding=0)  (fc): Linear(in_features=2048, out_features=1000, bias=True))print(model_conv.fc)      Linear(in_features=2048, out_features=1000, bias=True)
So what we have to adjust the last layer (FC). The last layer is a linear layer, that has 2048 input neurons and having 1000 output neurons. This is useful if you have 1000 different classes. However, we only have to deal with 7 different classes — the 7 different tumor types — so we need to change the last layer.
num_ftrs = model_conv.fc.in_featuresmodel_conv.fc = torch.nn.Linear(num_ftrs, 7)print(model_conv.fc)Linear(in_features=2048, out_features=7, bias=True)
So now after it’s adjusted we need to move the model to the GPU because that’s where the model will be trained in the end.
Training and validation set
from sklearn.model_selection import train_test_splittrain_df, test_df = train_test_split(tile_df, test_size=0.1)# We can split the test set again in a validation set and a true test set:validation_df, test_df = train_test_split(test_df, test_size=0.5)train_df = train_df.reset_index()validation_df = validation_df.reset_index()test_df = test_df.reset_index()
Create a Class ‘Dataset’
The dataset class will allow us to easily load and transform batches of data in the background on multiple CPUs.
class Dataset(data.Dataset):    'Characterizes a dataset for PyTorch'    def __init__(self, df, transform=None):        'Initialization'        self.df = df        self.transform = transform    def __len__(self):        'Denotes the total number of samples'        return len(self.df)    def __getitem__(self, index):        'Generates one sample of data'        # Load data and get label        X = Image.open(self.df['path'][index])        y = torch.tensor(int(self.df['cell_type_idx'][index]))        if self.transform:            X = self.transform(X)        return X, y# Define the parameters for the dataloaderparams = {'batch_size': 4,          'shuffle': True,          'num_workers': 6}
Another nice thing about using the dataset class is that we can easily perform preprocessing of the data and/or data augmentation.
In this example, we only perform mirroring (RandomHorizontalFlip, RandomVerticalFlip), Crop the image to the image center, where the melanoma is most often located (CenterCrop), randomly crop from the center of the image (RandomCrop) and normalize the image according to what the pre-trained model needs (Normalize). We then transform the image to a tensor using, which is required to use it for learning with PyTorch, with the function ToTensor:
define the transformation of the images.import torchvision.transforms as trfcomposed = trf.Compose([trf.RandomHorizontalFlip(), trf.RandomVerticalFlip(), trf.CenterCrop(256), trf.RandomCrop(224),  trf.ToTensor(),                        trf.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])# Define the trainingsset using the table train_df and using our defined transitions (composed)training_set = Dataset(train_df, transform=composed)training_generator = data.DataLoader(training_set, **params)# Same for the validation set:validation_set = Dataset(validation_df, transform=composed)validation_generator = data.DataLoader(validation_set, **params)
Now we have to define the optimizer we want to use. In this case, it will be an Adam optimizer with a learning rate of 1e−61e−6.
The loss function that we will use is CrossEntropyLoss. This is the typical one chosen for multiclass classification problems.
optimizer = torch.optim.Adam(model.parameters(), lr=1e-6)criterion = torch.nn.CrossEntropyLoss()
We now have a data loader for the data in the training set, a data loader for the data in the validation set and we have the optimizer and the criterion defined. We can now start on training and testing the model.
Training and testing the model is the main part of machine learning and in my opinion the best. This is where the real stuff happens. All through the data preparation and data collection stuff are important this is the fun part and it is also super important as well.
To train the model just input this code:
max_epochs = 20trainings_error = []validation_error = []for epoch in range(max_epochs):    print('epoch:', epoch)    count_train = 0    trainings_error_tmp = []    model.train()    for data_sample, y in training_generator:        data_gpu = data_sample.to(device)        y_gpu = y.to(device)        output = model(data_gpu)        err = criterion(output, y_gpu)        err.backward()        optimizer.step()        trainings_error_tmp.append(err.item())        count_train += 1        if count_train >= 100:            count_train = 0            mean_trainings_error = np.mean(trainings_error_tmp)            trainings_error.append(mean_trainings_error)            print('trainings error:', mean_trainings_error)            break    with torch.set_grad_enabled(False):        validation_error_tmp = []        count_val = 0        model.eval()        for data_sample, y in validation_generator:            data_gpu = data_sample.to(device)            y_gpu = y.to(device)            output = model(data_gpu)            err = criterion(output, y_gpu)            validation_error_tmp.append(err.item())            count_val += 1            if count_val >= 10:                count_val = 0                mean_val_error = np.mean(validation_error_tmp)                validation_error.append(mean_val_error)                print('validation error:', mean_val_error)                breakplt.plot(trainings_error, label = 'training error')plt.plot(validation_error, label = 'validation error')plt.legend()plt.show()
Testing the model:
To test the actual ability of the model import this code:
model.eval()test_set = Dataset(validation_df, transform=composed)test_generator = data.SequentialSampler(validation_set)result_array = []gt_array = []for i in test_generator:    data_sample, y = validation_set.__getitem__(i)    data_gpu = data_sample.unsqueeze(0).to(device)    output = model(data_gpu)    result = torch.argmax(output)    result_array.append(result.item())    gt_array.append(y.item())correct_results = np.array(result_array)==np.array(gt_array)sum_correct = np.sum(correct_results)accuracy = sum_correct/test_generator.__len__()print(accuracy)0.8403193612774451
So now you can see our accuracy is pretty high but it isn’t perfect. The model isn't going to be 100% perfect yet either but it's definitely close and will get there soon.
Now with this people don’t have to go into the doctor every time they get some symptoms of skin cancer and test just test it themselves with the comfort of their own home. And now people won’t be dying from skin cancer just because they didn’t want to go to the doctor.
If you have any questions leave them in the comments below and don’t forget to leave a clap!