Computer Vision and Pneumonia Detection Part 1: Technical
By: Erica Gabriel
Tags: Women In Tech, Deep Learning, Data Science, Health, Healthcare

Recently, I tasked myself with analyzing chest x-ray images to determine whether or not a patient has pneumonia. The x-rays were provided by the Guangzhou Women and Children’s Medical Center in Guangzhou, China, on patients ranging from ages 1–5 years old. The image set contains images of “Normal”, “Viral” & “Bacterial” Pneumonia, classified into 2 categories: “Normal”and “Pneumonia”.

Chest X-rays depicting the lungs of pediatric patients. The areas annotated in blue show evidence of damage due to pneumonia.

The dataset was provided in ideal conditions, free of poor quality scans. Before building the model, I checked the training data for class imbalance. There were 1341 Normal scans and 3875 Pneumonia scans. Killing 2 birds with one stone, I added 2534 additional images, and augmented them to reflect real world scenarios- blurry, distorted images, shifted/rotated images, and images that are zoomed in and out of focus.

# Set criteria for image augmentation

datagen = ImageDataGenerator(

rotation_range = 40,

width_shift_range = 0.2,

height_shift_range = 0.2,

rescale = 1./255,

shear_range = 0.2,

zoom_range = 0.2,

horizontal_flip = True,

fill_mode = 'nearest') # Select an image from the dataset

img = load_img('chest_xray/train/NORMAL/IM-0757-0001.jpeg') # Convert image to an array

img_array = img_to_array(img) # Reshape the array to a (1 X n X m X 3) array

img_array = img_array.reshape((1,) + img_array.shape) # Set the path where the updated images should be saved

norm_dir = 'chest_xray/train/NORMAL/' # Add 2534 photos to the directory in batches of 150

count = 0

for batch in datagen.flow(img_array, batch_size=150, save_to_dir=norm_dir, save_prefix='IM', save_format='jpeg'):

count +=1

if count == 2534:

break



print('2534 images have been generated at', norm_dir)

Sample augmented image.

Once the images were added, I reduced the size of the images to 64 x 64, so that the sizes would be uniform and less computationally expensive to input into the model, and save the updated images to the original file directory. Using the next() function, I iterated through the image files and returned the images and labels that have been stored in the variables “__images” and “__labels”, then I normalized the images by dividing the pixels by 255.

# Train set

train_generator = ImageDataGenerator(rescale=1./255).flow_from_directory(train_dir,

target_size = (64, 64), batch_size= 7685) # Validation Set

val_generator = ImageDataGenerator(rescale=1./255).flow_from_directory(val_dir,

target_size=(64, 64), batch_size=16) # Test Set

test_generator = ImageDataGenerator(rescale=1./255).flow_from_directory(test_dir,

target_size = (64, 64), batch_size=624) # Create the image-label datasets

train_images, train_labels = next(train_generator)

val_images, val_labels = next(val_generator)

test_images, test_labels = next(test_generator)

The final preprocessing step is to resize the labels to an (l x 1) array, then preview an image and label to ensure that the correct labels are corresponding with the respective images.

train_labels_final = np.reshape(train_labels[:,0], (7685,1))

val_labels_final = np.reshape(val_labels[:,0], (16,1))

test_labels_final = np.reshape(test_labels[:,0], (624,1)) # Sanity check on image 4000

array_to_img(train_images[4000])

train_labels_final[4000, :]

Preview of sample image 4000, an augmented image.

Now on to building the model. The initial model was run with the default settings, then expounded upon, after each iteration to see which parameters helped the model to achieve optimal performance. Below is a code snippet that iterates through different optimizers while fitting to the training and validation data. After the code block is run, you can visually inspect the performance of the optimizers via a line plot.

# Create a dictionary of optimizers

optimizers = {"RMSprop": {"optimizer": RMSprop(), "color":"blue"},

"adam_01": {"optimizer": Adam(lr=0.01),"color":"red"},

"sgd": {"optimizer": SGD(), "color":"purple"},

"adadelta": {"optimizer": Adadelta(), "color":"pink"},

"adagrad": {"optimizer": Adagrad(), "color":"yellow"}} for optimizer, d in optimizers.items():

print(f'Testing {optimizer}')



# Build the CNN



model = Sequential() model.add(layers.Conv2D(32, (3, 3), activation='relu',

input_shape=(64, 64, 3))) model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Conv2D(32, (4, 4), activation='relu'))

model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu'))

model.add(layers.MaxPooling2D((2,2))) # Flattening

model.add(layers.Flatten())

model.add(layers.Dense(64, activation = 'relu'))

model.add(layers.Dense(1, activation='softmax'))

model.compile(optimizer=d['optimizer'],loss='binary_crossentropy',

metrics=['accuracy']) results = model.fit(train_images, train_labels_final,

epochs=15, batch_size=500, validation_data=(val_images,

val_labels_final)) print("Stored Results")

d['loss'] = history.history['loss']

print('='*125) # Add a partition in between optimizer results

Next, evaluate the final loss and accuracy of the model, then make predictions on the validation images. Access the total performance of the model using a confusion matrix in conjunction with a Classification Report. My final model resulted in having zero False Negatives and eight False Positives, which demonstrates a best case scenario because it’s better to flag a patient for additional testing, than to release a patient carrying pneumonia and their symptoms worsen or they spread viral pneumonia to a loved one.

# Evaluate the model on the metrics accuracy and loss

model.evaluate(train_images, train_labels_final)

model.evaluate(val_images, val_labels_final) # Make predictions on the validation set

preds = model.predict(val_images)

plot_confusion_matrix(val_labels_final, preds) # Print the Classification Report, which returns Recall, Precision, and F1-Score

classification_report =classification_report(val_labels_final, preds)

Continue to tune your hyper parameters, and train your best performing model on your testing data, then repeat. Compare your results to a pre-trained model. Below I used VGG16.

# Import Pretrained Model

from keras.applications.vgg16 import preprocess_input

from keras.applications.vgg16 import VGG16 # Edit the default image size to (64, 64, 3)

pretrain_mod = VGG16(include_top= False, input_shape=(64,64,3)) # Initialize a model

model = Sequential()

model.add(pretrain_mod)

model.add(layers.Flatten())

model.add(layers.Dense(132, activation='tanh'))

model.add(layers.Dense(1, activation='softmax')) # Freeze layers (do this to speed up

pretrain_mod.trainable = False

for layer in model.layers:

print(layer.name, layer.trainable)

print(len(model.trainable_weights)) model.compile(loss='binary_crossentropy', optimizer=Adam(lr=0.01), metrics=['accuracy']) results = model.fit(train_images, train_labels_final, epochs=11, batch_size=750, validation_data=(test_images, test_labels_final))

Don’t forget to save your best model!