Confusion Matrix — Clearly Explained
By: Indhumathy Chelliah
Tags: Statistics, Python3, Machine Learning, Artificial Intelligence, Data Science

Classification Accuracy

In Machine learning for classification models, Accuracy is one metric used to evaluate the performance of the model.

Why we need Confusion Matrix?

Example: If we have a data set[cancer prediction] of 100 records, and if the accuracy is 95% means, the model predicts 95 records correctly. 5 records are predicted wrongly.

Error is said to be 5%. We can’t predict what type of error is this? Two types of errors are possible.

1 [malignant-cancer] is predicted as 0[benign- no cancer]

0[benign-no cancer] is predicted as 1[malignant-cancer]

Out of these two errors, for this model, first, one is considered to be risky. Because if a cancer patient is mispredicted, their condition will become worse without treatment.

If a non-cancer patient is predicted as cancer means, he/she may go for another screening and will get the correct result in the next screening. So, this error is not risky.

If those 5 misclassified records belong to the first category (Cancer patients predicted as no cancer) means, it will be very risky. From the accuracy metrics, we can’t evaluate the performance of the model.

So, we go for a confusion matrix which breaks down the type of error also.

What is the Confusion Matrix?

A confusion matrix is a n*n matrix that is used for evaluating the performance of the classification model.

For Binary classification — The confusion Matrix is a 2*2 matrix.

If the target class is 3 means Confusion Matrix is 3*3 matrix and so on.

Confusion Matrix [Image by Author]

Terminologies used in Confusion Matrix

True Positive → Positive class which is predicted as positive.

→ Positive class which is predicted as positive. True Negative → Negative class which is predicted as negative.

→ Negative class which is predicted as negative. False Positive → Negative class which is predicted as positive.[Type I Error]

→ Negative class which is predicted as positive.[Type I Error] False Negative →Positive class which is predicted as negative.[Type II Error]

False Positive(FP) is also known as Type I Error.False Negative(FN) is also known as Type II Error

Metrics Calculated from Confusion Matrix

1. Recall

Recall is a measure of how many positives your model is able to recall from the data.

Out of all positive records, how many records are predicted correctly.

Recall is also known as Sensitivity or TPR (True Positive Rate)

2. Precision

Precision is the ratio of correct positive predictions to the total positive predictions.

Out of all positives been predicted, how many are actually positive.

Confusion Matrix [Image by Author]

Which metric to choose precision or recall?

Both precision and recall are used for evaluating the performance of the model. Which one to chose depends on the problem we aim to solve.

Recall — Example

Cancer Prediction-For this dataset, if the model predicts cancer records as non-cancer means it’s risky. All our cancer records should be predicted correctly.

In this example, recall metrics is more important than precision. The recall rate should be 100%. All positive records( cancer records) should be predicted correctly. False Negative should be 0.

For this cancer dataset, recall metrics is given more importance while evaluating the performance of the model.

If non-cancer records are predicted as cancer means it’s not that risky.

Example. The cancer data set has 100 records, out of which 94 are cancer records and 6 are non-cancer records. But the model is predicting 90 out of 94 cancer records correctly. Four cancer records are not predicted correctly [ 4 — FN]

Recall [Image by Author]

Precision — Example

Email Spam Filtering- For this dataset, if the model predicts good email as spam means it's risky. We don’t want any of our good emails to be predicted as Spam. So, the precision metric is given more importance while evaluating this model. False Positive should be 0.

If the spam filtering dataset has 100 records, out of which 94 are predicted as spam emails. Only 90 out of 94 records is predicted correctly. 4 good emails are classified as spam. It’s risky. The precision rate is 95%. It should be 100%. No good emails should be classified as “Spam”. False-positive should be 0 for this model.

Precision [Image by Author]

F1 Score

F1 score is a harmonic mean of precision and recall.

F1 score metric is used when you seek a balance between precision and recall.

F1 score vs Accuracy