Predict Heart Disease With C# And A CNTK Deep Neural Network
By: Mark Farragher
Tags: Artificial Intelligence, Machine Learning, Data Science, Csharp, Programming

It’s a CSV file with 14 columns of information:

Age

Sex: 1 = male, 0 = female

Chest Pain Type: 1 = typical angina, 2 = atypical angina , 3 = non-anginal pain, 4 = asymptomatic

Resting blood pressure in mm Hg on admission to the hospital

Serum cholesterol in mg/dl

Fasting blood sugar > 120 mg/dl: 1 = true; 0 = false

Resting EKG results: 0 = normal, 1 = having ST-T wave abnormality, 2 = showing probable or definite left ventricular hypertrophy by Estes’ criteria

Maximum heart rate achieved

Exercise induced angina: 1 = yes; 0 = no

ST depression induced by exercise relative to rest

Slope of the peak exercise ST segment: 1 = up-sloping, 2 = flat, 3 = down-sloping

Number of major vessels (0–3) colored by fluoroscopy

Thallium heart scan results: 3 = normal, 6 = fixed defect, 7 = reversible defect

Diagnosis of heart disease: 0 = less than 50% diameter narrowing, 1 = more than 50% diameter narrowing

The first 13 columns are patient diagnostic information, and the last column is the diagnosis: 0 means a healthy patient, and 1 means an elevated risk of heart disease.

I will build a binary classification neural network that reads in the patient information and then makes a prediction for the heart disease risk.

Let’s get started. Download the Processed Cleveland Data file and save it as processed.cleveland.data.csv.

Next you need to set up a new console project in NET Core:

$ dotnet new console -o HeartDisease

$ cd HeartDisease

And install the required packages:

$ dotnet add package Microsoft.ML

$ dotnet add package CNTK.GPU

$ dotnet add package XPlot.Plotly

$ dotnet add package Fsharp.Core

Microsoft.ML is the Microsoft machine learning package. We will use to load and process the data from the dataset.

The CNTK.GPU library is Microsoft’s Cognitive Toolkit that can train and run deep neural networks.

And Xplot.Plotly is an awesome plotting library based on Plotly. The library is designed for F# so we also need to pull in the Fsharp.Core library.

The CNTK.GPU package will train and run deep neural networks using your GPU. You’ll need an NVidia GPU and Cuda graphics drivers for this to work.

If you don’t have an NVidia GPU or suitable drivers, the library will fall back and use the CPU instead. This will work but training neural networks will take significantly longer.

CNTK is a low-level tensor library for building, training, and running deep neural networks. The code to build deep neural network can get a bit verbose, so I’ve developed a little wrapper called CNTKUtil that will help you write code faster.

You can download the CNTKUtil files and save them in a new CNTKUtil folder at the same level as your project folder.

Then make sure you’re in the console project folder and crearte a project reference like this:

$ dotnet add reference ..\CNTKUtil\CNTKUtil.csproj

Now you are ready to start writing code. Edit the Program.cs file with Visual Studio Code and add the following code:

The HeartData class holds all the data for one single patient. Note how each field is tagged with a LoadColumn attribute that will tell the CSV data loading code from which column to import the data.

We also have a GetFeatures method that returns a subset of the data columns: the age, cholesterol level, fasting blood sugar, maximum heart rate, exercise induced angina, ST depression, and slope value of the patient.

And there’s a GetLabel method that returns 1 if the patient is a heart disease risk and 0 if the patient is not.

The features are the patient attributes that we will use to train the neural network on, and the label is the output variable that we’re trying to predict. So here we’re training on 7 patient attributes in the dataset to predict the heart disease risk.

Now it’s time to start writing the main program method:

When working with the ML.NET library we always need to set up a machine learning context represented by the MLContext class.

The code calls the LoadFromTextFile method to load the CSV data in memory. Note the HeartData type argument that tells the method which class to use to load the data.

We then use TrainTestSplit to split the data in a training partition containing 60% of the data and a testing partition containing 40% of the data.

Note that we’re deviating from the usual 80–20 split here. This is because the data file is extremely small, and so 20% of the data is simply not enough to test the neural network on.

Finally we call CreateEnumerable to convert the two partitions to an enumeration of HeartData instances. So now we have the training data in training and the testing data in testing. Both are enumerations of HeartData instances.

But CNTK can’t train on an enumeration of class instances. It requires a float[][] for features and float[] for labels.

So we need to set up four float arrays:

These LINQ expressions set up four arrays containing the feature and label data for the training and testing partitions.

Now we need to tell CNTK what shape the input data has that we’ll train the neural network on, and what shape the output data of the neural network will have:

Note the first Var method which tells CNTK that our neural network will use a 1-dimensional tensor of 7 float values as input. This shape matches the 7 values returned by the HeartData.GetFeatures method.

And the second Var method tells CNTK that we want our neural network to output a single float value. This shape matches the single value returned by the HeartData.GetLabel method.

Our next step is to design the neural network.

We will use a deep neural network with a 16-node input layer, a 128-node hidden layer, and a single-node output layer. We’ll use the ReLU activation function for the input and hidden layers, and Sigmoid activation for the output layer.

The sigmoid function forces the output of the neural network to a range of 0..1 which means we can treat it as a binary classification probability.

We can turn any regression network into a binary classification network by simply adding the sigmoid activation function to the output layer.

Here’s how to build this neural network:

Each Dense call adds a new dense feedforward layer to the network. We’re stacking two layers, both using ReLU activation, and then add a final layer with a single node using Sigmoid activation.

Then we use the ToSummary method to output a description of the architecture of the neural network to the console.

Now we need to decide which loss function to use to train the neural network, and how we are going to track the prediction error of the network during each training epoch.

For this assignment we’ll use BinaryCrossEntropy as the loss function because it’s the standard metric for measuring binary classification loss.

We’ll track the error with the BinaryClassificationError metric. This is the number of times (expressed as a percentage) that the model predictions are wrong. An error of 0 means the predictions are correct all the time, and an error of 1 means the predictions are wrong all the time.

Next we need to decide which algorithm to use to train the neural network. There are many possible algorithms derived from Gradient Descent that we can use here.

For this assignment we’re going to use the AdamLearner. You can learn more about the Adam algorithm here: https://machinelearningmastery.com/adam...

These configuration values are a good starting point for many machine learning scenarios, but you can tweak them if you like to try and improve the quality of your predictions.

We’re almost ready to train. Our final step is to set up a trainer and an evaluator for calculating the loss and the error during each training epoch:

The GetTrainer method sets up a trainer which will track the loss and the error for the training partition. And GetEvaluator will set up an evaluator that tracks the error in the test partition.

Now we’re finally ready to start training the neural network!

Add the following code:

We’re training the network for 100 epochs using a batch size of 1. During training we’ll track the loss and errors in the loss, trainingError and testingError arrays.

A batch size of one means the neural network is trained on each individual record in the dataset. This produces the best possible prediction accuracy, but also takes the longest to train. But because the dataset for this assignment is so tiny, training with batches of one record is more than fast enough here.

Once training is done, we show the final testing error on the console. This is the percentage of mistakes the network makes when predicting heart disease risk.

Note that the error and the accuracy are related: accuracy is simply 1 minus the error. So we also report the final accuracy of the neural network.

Here’s the code to train the neural network. Put this inside the for loop:

The Index().Shuffle().Batch() sequence randomizes the data and splits it up in a collection of 1-record batches. The second argument to Batch() is a function that will be called for every batch.

Inside the batch function we call GetBatch twice to get a feature batch and a corresponding label batch. Then we call TrainBatch to train the neural network on these two batches of training data.

The TrainBatch method returns the loss and error, but only for training on the 1-record batch. So we simply add up all these values and divide them by the number of batches in the dataset. That gives us the average loss and error for the predictions on the training partition during the current epoch, and we report this to the console.

So now we know the training loss and error for one single training epoch. The next step is to test the network by making predictions about the data in the testing partition and calculate the testing error.

Put this code inside the epoch loop and right below the training code:

We don’t need to shuffle the data for testing, so now we can call Batch directly. Again we’re calling GetBatch to get feature and label batches, but note that we’re now providing the testing_data and testing_labels arrays.

We call TestBatch to test the neural network on the 1-record test batch. The method returns the error for the batch, and we again add up the errors for each batch and divide by the number of batches.

That gives us the average error in the neural network predictions on the test partition for this epoch.

After training completes, the training and testing errors for each epoch will be available in the trainingError and testingError arrays. Let’s use XPlot to create a nice plot of the two error curves so we can check for overfitting:

This code creates a Plot with two Scatter graphs. The first one plots the trainingError values and the second one plots the testingError values.

Finally we use File.WriteAllText to write the plot to disk as a HTML file.

We’re now ready to build the app, so this is a good moment to save your work ;)

Go to the CNTKUtil folder and type the following:

$ dotnet build -o bin/Debug/netcoreapp3.0 -p:Platform=x64

This will build the CNKTUtil project. Note how we’re specifying the x64 platform because the CNTK library requires a 64-bit build.

Now go to the HeartDisease folder and type:

$ dotnet build -o bin/Debug/netcoreapp3.0 -p:Platform=x64

This will build your app. Note how we’re again specifying the x64 platform.

Now run the app:

$ dotnet run

The app will create the neural network, load the dataset, train the network on the data, and create a plot of the training and testing errors for each epoch.

Here’s the neural network being trained on my laptop:

And this is the training chart:

The final classification error is 0.20 on training and 0.28 on testing. That corresponds to a final accuracy on testing of 0.72. This means the neural network makes 72 correct predictions for every 100 patients.

The shape of the training curves are more or less okay, but there’s a lot of noise with big up and down fluctuations. And the training and testing curves start to diverge pretty much right from the start. This is classic overfitting.

Overfitting means the data is too complex for the model, or the model is too simple to capture the complexities of the patterns in the data.

Check out what happens when we make the data even more complex and train on all 13 columns in the data file:

The training is stalling. During each training epoch the Adam learner is unable to find any improvements. The training error is stuck at 0.44 and the test error at 0.47 with no further improvements.

What if we increased the complexity of the neural network? Let’s double the number of nodes in the input and hidden layers:

Here are the results:

We’re still overfitting and all the noise is also still present. The error is now 0.19 on training and 0.34 on testing. The final accuracy is 0.66.

How about adding a layer? We’ll halve the number of nodes but now add an extra 128-node hidden layer:

Here are the results:

Again we have noise and overfitting. The final error is 0.20 on training and 0.28 on testing, with a final accuracy of 0.72.

No improvement.

We won’t be able to get better results here. Deep neural networks are not a magical solution for every conceivable problem. The heart disease dataset is simply too complex for a feed-forward neural net to solve.

To get better results, we require either a different neural architecture or a specialized data science algorithm.

So what do you think?

Are you ready to start writing C# machine learning apps with CNTK?