Breast Cancer Detection Using Logistic Regression
By: Ranjani Rajamani
Tags: Logistic Regression, Precision Recall Curve, Breast Cancer, Voting Classifier, Confusion Matrix

Are you aware that 1 among 8 women in the United States is being diagnosed with breast cancer? Breast cancer is a disease in which the healthy cells of the tissue in the breast are invaded and mutated, which further grow in large numbers to form a malignant tumor. It can most likely occur at any age.

Few risk factors that contribute to breast cancer.

It occurs 100 times more in women than in men

Genetic factors such as mother, father, sister or brother had been diagnosed with ovarian or breast cancer.

· Early puberty, late menopause or child birth at an older age or no child birth can increase the risk

Dense breast tissue can also surge the risk of breast cancer. Detection can also be delayed due to the compact nature of the tissue.

Obesity, alcohol abuse, poor nutrition and exposure to frequent radiation can also increase the risk.

Early detection increases the survival chances by almost 99%. By realizing the symptoms and signs, this cancer can be detected early. Self-examination and clinical examination can be conducted for detection of lumps. Mammography is a technique where the breast is exposed to a small dose of radiation to capture the images of the affected area. It can detect cancer before the formation of lumps.

Detection of Breast Cancer Using Classification Algorithm

Unsplash image by National Cancer Institute — Mammography

Early detection of the malignancy of a lump is the key to high probability of survival. Many imaging techniques have been developed for detection and possibility of cure and decrease the mortality rate due to breast cancer. Although Mammograms are one of the best tools for breast screening, there are other imaging techniques such as Breast Ultrasound and Breast MRI.

To facilitate accurate classification of images using Machine Learning algorithms, the data processing is divided into 3 stages, namely preprocessing, feature selection and classification. Feature selection is the most important stage in algorithm building. The dataset generally contains several features and feeding too many features decreases the performance of the algorithm. Features can be extracted using Recursive Feature Extraction (RFE). Ensembling/Combining two or more algorithms could boost the performance of the model. They avoid overfitting and substantially contribute towards improving the accuracy and recall values. In the classification algorithm here, VotingClassifier has been used to combine two logistic regression models using various hyper parameters to classify the dataset. VotingClassifier combines models by giving importance to each, based on the weights assigned to them. Other boosting methods are:

AdaBoost (Strong classifier with low weights)

XGBoost (Extreme gradient boosting)

Catboost (Comes with good initial parameters and can automatically encode categorical variables)

Objective

The aim of this analysis is to use Logistic Regression to classify the data into two classes of diagnosis— Malignant & Benign. The evaluation metrics used are accuracy, ROC, confusion matrix, precision-recall.

Dataset

The Wisconsin Breast Cancer (Diagnostic) dataset has been extracted from the UCI Machine Learning Repository. Features are computed from a digitized image of a fine needle aspirate (FNA) of a breast mass. They describe characteristics of the cell nuclei present in the image.

Class distribution: 357 benign, 212 malignant

benign, malignant Number of instances: 569; Number of attributes: 32

Attributes: Ten real-valued features are computed for each cell nucleus

a) radius (mean of distances from center to points on the perimeter)

b) texture (standard deviation of gray-scale values)

c) perimeter

d) area

e) smoothness (local variation in radius lengths)

f) compactness (perimeter² / area — 1.0)

g) concavity (severity of concave portions of the contour)

h) concave points (number of concave portions of the contour)

i) symmetry

j) fractal dimension (“coastline approximation” — 1)

Preprocessing

The dataset provided is already clean and does not have any missing values. As a part of the preprocessing stage, the data is standardized using Standard Scaler library. It transforms the attributes to normal distribution

# Read data

dataset = pd.read_csv('data.csv')

# drop the last column from the data

dataset = dataset.drop(['Unnamed: 32','id'],axis = 1) # Impute the only categorical column

dataset["diagnosis"].replace(to_replace = dict(M = 1, B = 0), inplace = True)

dataset.head() #check the value count of the benign vs malignant cases

target_count = (dataset['diagnosis']).value_counts()

sns.countplot(dataset['diagnosis'])

pd.Series(dataset['diagnosis']).value_counts()

print('Proportion:', round(target_count[0] / target_count[1], 2), ': 1')

Proportion: 1.68 : 1

Correlated Features

A correlation between variables indicates that as one variable changes in value, the other variable tends to change in a specific direction. Scatter plots and heat map are good visualization techniques to view correlation between continuous variables

Correlation Matrix

Positively correlated features are the ones where the value of one variable increases as the other increases resulting in an upward slope indicating a positive correlation with all the data points falling on the line. Following are positively correlated features.

Positively Correlated Features

Negative correlated features represent cases when the value of one variable increases, the value of the other variable tends to decrease producing a downward slope.

Negatively Correlated Features

Features with high correlation are more linearly dependent and hence have almost the same effect on the target (diagnosis) variable. So, when two features have high correlation, one of the features can be dropped.

Evaluation metrics for classification

Accuracy cannot be completely relied upon as an evaluation metric for Classification.

Confusion Matrix: It is a matrix representation of the results of any binary testing. In this study, the hypothesis is to validate if the person having the lump is malignant or benign. Confusion matrix is used to represent the actual values and the prediction values

Confusion Matrix

True positive (TP): Malignant lump correctly identified as malignant

True negative (TN): Benign lump correctly identified as benign

False positive (FP): Benign lump incorrectly identified as malignant

False negative (FN): Malignant lump incorrectly identified as benign

Accuracy : Accuracy is the proportion of true results among the total number of cases sample : (TP +TN) / (TP + TN + FP +FN)

: Accuracy is the proportion of true results among the total number of cases sample : (TP +TN) / (TP + TN + FP +FN) Precision : The proportion of predicted Positives that is actually positive : TP / (TP + FP)

: The proportion of predicted Positives that is actually positive : TP / (TP + FP) Recall OR Sensitivity OR TPR : The proportion of actual positives correctly classified : TP / (TP + FN)

: The proportion of actual positives correctly classified : TP / (TP + FN) Specificity: The proportion of actual negatives correctly classified : TN/(TN + FP)

The proportion of actual negatives correctly classified : TN/(TN + FP) F1 Score: F1 score kind of maintains a balance between the precision and recall for the classifier by giving equal importance to both precision and recall. Lower the precision, lower is the F1 score. Lower the recall, lower is the F1 score. The F1 value is between 0 and 1 and is the mean of precision and recall : 2 * (Precision * Recall / Precision + Recall)

AUC ROC : Area Under Curve (AUC) is the area covered under the ROC. Receiver Operating Characteristics. ROC is the ratio of True Positive Rate (TPR) and False Positive Rate (FPR) where TPR is proportion of actual positives correctly classified by the model and FPR is the proportion of false positives classified by the model. We have the above values (TPR, FPR) from the model. We can use various threshold values between 0.1–1 to plot the sensitivity and 1-Specificity on the graph. The curve derived is the ROC curve. Lower threshold can increase the number of FP and decrease the number of FN. Higher threshold can decrease the number of FP and increase the FN.

Classification using Logistic Regression (Using RFE for feature elimination)

After splitting the data into training and test set, the training data is fit and predicted using Logistic Regression with GridSearchCV. GridSearchCV is a function that belongs to the sklearn library. It is a process to optimize the model performance by tuning the hyperparameters, automatically, by setting a few parameters in the function. Choosing optimal features for model training is an extremely difficult task to achieve. Feature selection is a key technique to improve the performance of any model. Feature selection can be done using Principal Component Analysis, but it provides us with the principal components which are achieved by a combination of features. It does not clearly declare the features, used to calculate the principal components.

Recursive Feature Elimination technique selects individual features by recursively training the data for a given model. Thus a set of features with high importance is used to improve performance of the model.

# Using GridSearchCV in Logistic regression param_grid = {'penalty' : ['l2','l1'],

'C' : [0.001, 0.01, 0.1, 1, 10, 100, 1000]

}

log_clf = LogisticRegression(random_state = 42)

CV_log_clf = GridSearchCV(estimator = log_clf, param_grid = param_grid , scoring = 'accuracy', verbose = 1, n_jobs = -1) CV_log_clf.fit(X_train, y_train) best_parameters = CV_log_clf.best_params_ log_clf = LogisticRegression(C = best_parameters['C'], penalty = best_parameters['penalty'],random_state = random_state) selector = RFE(log_clf)

selector = selector.fit(X_train, y_train) y_pred = selector.predict(X_test)

y_score = selector.predict_proba(X_test)[:,1] # Confusion maxtrix & metrics

cm = confusion_matrix(y_test, y_pred)

class_names = [0,1]

plt.figure()

print('Confusion matrix :

',cm)

plt.show() show_evaluation_metrics() # ROC curve

fpr, tpr, t = roc_curve(y_test, y_score)

plot_roc()

Confusion matrix :

[[106 2]

[ 2 61]]

Accuracy = 0.977

Precision = 0.968

Recall = 0.968

F1_score = 0.968

ROC using scoring = “accuracy” as hyper parameter

With a cross validation of 5 folds and a threshold > 0.53 and a recall = 98%, following is the performance score of the Logistic Regression model with accuracy as the hyper parameter.

Accuracy = 0.988

Precision = 0.984

Recall = 0.984

F1_score = 0.984

To further improve the performance of the model optimize the hyper parameter scoring = ’recall’ in GridSearchCV on Logistic Regression

# Using GridSearchCV in Logistic regression with Recall

param_grid = {'penalty' : ['l2','l1'],

'C' : [0.001, 0.01, 0.1, 1, 10, 100, 1000]

}

log_clf = LogisticRegression(random_state = 42)

CV_log_clf = GridSearchCV(estimator = log_clf, param_grid = param_grid , scoring = 'recall', verbose = 1, n_jobs = -1)

CV_log_clf.fit(X_train, y_train) best_parameters = CV_log_clf.best_params_

print('The best parameters for using this model is', best_parameters) CV_log_clf_1 = LogisticRegression(C = best_parameters['C'],

penalty = best_parameters['penalty'],

random_state = random_state) CV_log_clf_1.fit(X_train, y_train)

y_pred = CV_log_clf_1.predict(X_test)

y_score = CV_log_clf_1.decision_function(X_test) # Confusion maxtrix & metrics

cm = confusion_matrix(y_test, y_pred)

class_names = [0,1]

plt.figure()

print('Confusion matrix :

',cm)

plt.savefig('6')

plt.show() show_evaluation_metrics() # ROC curve

fpr, tpr, t = roc_curve(y_test, y_score)

plot_roc()

Confusion matrix :

[[106 2]

[ 1 62]]

Accuracy = 0.982

Precision = 0.969

Recall = 0.984

F1_score = 0.976

ROC using scoring = “recall” as hyper parameter

Ensembling two models by using Voting Classifier

Maximizing precision and detect all malignant tumors (high recall value) can significantly improve the performance of the classifier. This can be done by combing/ensembling two or more models with a 5-fold cross validation and selecting an optimum threshold > 0.5

Confusion matrix :

[[106 2]

[ 1 62]]

Accuracy = 0.982

Precision = 0.969

Recall = 0.984

F1_score = 0.976

ROC using VotingClassifier

Precision Recall Curve

The precision-recall curve shows the tradeoff between precision and recall for different threshold. A high area under the curve represents both high recall and high precision, where high precision relates to a low false positive rate, and high recall relates to a low false negative rate. High scores for both show that the classifier is returning accurate results (high precision), as well as returning a majority of all positive results (high recall) for a threshold > 0.5.

Precision Recall Curve

That’s it!

You can get the code here!

References