Deep Learning in Healthcare — X-Ray Imaging (Part 3-Analyzing images using Python) | by Arjun Sarkar | Towards Data Science

Now that we have seen how difficult it is for an untrained professional to interpret X-ray images, lets’ look at a few techniques to view and analyze the images, their histograms, and a technique to add images and labels together, using Python programming.
The image dataset (Chest X-Rays) was obtained from Kaggle. Dataset is available on the following link — https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia/data
About the dataset — direct quote from the Kaggle challenge — The dataset is organized into 3 folders (train, test, val) and contains subfolders for each image category (Pneumonia/Normal). There are 5,863 X-Ray images (JPEG) and 2 categories (Pneumonia/Normal). Chest X-ray images (anterior-posterior) were selected from retrospective cohorts of pediatric patients of one to five years old from Guangzhou Women and Children’s Medical Center, Guangzhou. All chest X-ray imaging was performed as part of patients’ routine clinical care. For the analysis of chest x-ray images, all chest radiographs were initially screened for quality control by removing all low quality or unreadable scans. The diagnoses for the images were then graded by two expert physicians before being cleared for training the AI system. In order to account for any grading errors, the evaluation set was also checked by a third expert.
As the content clearly states, there are a total of 5863 images available in the challenge, which have been split into 2 classes, Pneumonia and Normal, and further split into train/test and validation sets. To make the challenge even harder, we have split the data into three classes, Normal, Bacterial Pneumonia, and Viral Pneumonia. In this part, we will focus only on the images — loading them with python, analyzing various important aspects of the image from a medical imaging perspective, and loading the images and labels together. Let's dive straight into it.
#importing all the necessary librariesimport numpy as np                     import matplotlib.pyplot as pltimport osimport cv2 as cvimport random
Numpy — Numpy is one of the most commonly used libraries in Python. It is used for operations on multi-dimensional arrays and matrices and doing high-level mathematical functions to operate on these arrays.
Matplotlib — A library for creating static and animated visualizations in python.
os —A module that comes built-in with python. It provides functions for interacting with the operating system.
cv2 — OpenCV (Open Source Computer Vision Library) — A very important library mainly used for computer vision. Other similar libraries are SimpleITK and Pillow (Python Imaging Library).
random — A module that generates pseudo-random numbers.
#load a single image from the bacteria folderdef load_image(path):    for img in os.listdir(bacteria_path):        print('Image name =',img)        image = cv.imread(os.path.join(bacteria_path, img))        break            return image
The images from the dataset have been split into three classes as mentioned previously. 1-Normal, 2-Bacteria (Bacterial Pneumonia), 3- Virus (Viral Pneumonia).
The above code snippet is creating a function load_image, which will be used to load a single image from the training sets, Bacteria folder. os.listdir is used to list all the files present inside that directory. In this case, it can be used to access all the images present inside the folder Bacteria. Next, it will print the name of the image. Finally, the OpenCV library is used to read the image.
Break- is necessary here, so that only the first image is accessed, otherwise the function will loop through all the images present inside the Bacteria folder.
# Investigate a single imagebacteria_path = 'H:/All Files/Kaggle/chest_xray/train/2_BACTERIA/'image = load_image(bacteria_path)plt.imshow(image, cmap='gray')plt.colorbar()plt.title('Raw Chest X Ray Image')print(f"The dimensions are {image.shape[0]} pixels height and {image.shape[1]} pixels width")print(f"The maximum pixel value is {image.max():.4f}")print(f"The minimum pixel value is {image.min():.4f}")print(f"The mean value of the pixels is {image.mean():.4f}")print(f"The standard deviation is {image.std():.4f}")
output-
In this code snippet, first, the path of the images is defined. Then the first image from the folder is loaded into variable ‘image’ by calling the function load_image. The image is then viewed by using matplotlib.imshow. After this, the dimensions of the image, the maximum pixel value, and the minimum pixel value in the grayscale bar is printed. Also the mean and standard deviation of the image pixels are calculated.
Next, we plot the histogram of all the pixels of the image. A histogram is a graphical display of data using bars of different heights.
# plot a histogramplt.hist(image.ravel(),256,[0,256]) plt.show()
Output-
Matplotlib.hist is used to plot the histogram. As the image is mostly dark, we see a huge cluster of pixels on position zero of the grayscale bar.
These are some basic functions that can be carried out on images using OpenCV and matplotlib. We will in later parts see more uses of OpenCV.
# loading the path of the train imagespath = 'H:/All Files/Kaggle/chest_xray/train/'train = os.listdir(path)
The path of the training set is defined, and the directories under the path are saved in ‘train’. In this case, there are three folders, 1_Normal, 2_Bacteria, and 3_Virus.
folders=[]folders = [f for f in sorted(os.listdir(path))]print(folders)
output-
[‘1_NORMAL’, ‘2_BACTERIA’, ‘3_VIRUS’]
We create an empty list — folders. Then, iterate over the path, using os.listdir, and sort and store the folder names in the list — ‘folders’.
labels = foldersprint (f'The labels are {labels}')# setting the size of images that we wantimage_size = 256print(f'All images to be resized into {image_size}*{image_size} pixels')
output-
The folder names are set as labels for the images, and the image size is selected to be 256*256. That is, all the images will be resized into 256*256. If we go through the dataset, we see all the images are of varying dimensions, and to feed images into a Convolutional Neural Network (CNN) it is necessary to resize the images into the same dimensions.
# defining a function to load images and labels together# this function will also resize the imagesdef load_train(path):        images = []        for label in labels:        direc = os.path.join(path, label)        class_num = labels.index(label)                for image in os.listdir(direc):            image_read = cv.imread(os.path.join(direc,image),cv.IMREAD_GRAYSCALE)            image_resized = cv.resize(image_read,(image_size,image_size))            images.append([image_resized,class_num])                return np.array(images)
Here we define a function to load in all the images according to the label names, resize them into 256*256 pixels, and return the image arrays.
An empty list is created to save all the images. Then a ‘for’ loop is run to extract all the images from all the three folders. os.path.join is used to combine paths from directories. cv.IMREAD_GRAYSCALE converts all images to grayscale format. cv.resize is used to resize images to 256*256 pixels. ‘.append’ is used to append all the images into a list, which is finally converted to an array and returned using the return statement.
#load all the training images to train_imagestrain_images = load_train(path)print(f'Shape of the training images = {train_images.shape}')
output- Shape of the training images = (5208, 2)
The function ‘load_train’ is then called, and all the training images are saved as an array in train_images. The shape of training images is (5208,2)
#loading the images and labels seperately in X and y, to be used later for trainingX = []y = []for feature, label in train_images:    X.append(feature)    y.append(label)    print (f'Length of X = {len(X)}')print (f'Length of y = {len(y)}')
output -
The images and labels need to be separated for training a neural network, and they are done so, by looping over the train_images, and by extracting the images and their corresponding labels.
# checking the number of images of each classa = 0b = 0c = 0for label in y:    if label == 0:        a += 1    if label == 1:        b += 1    if label == 2:        c += 1        print (f'Number of Normal images = {a}')print (f'Number of Bacteria images = {b}')print (f'Number of Virus images = {c}')# plotting the datax_pos = [i for i, _ in enumerate(labels)]numbers = [a,b,c]plt.bar(x_pos,numbers,color = 'green')plt.xlabel("Labels")plt.ylabel("No. of images")plt.title("Images for each label")plt.xticks(x_pos, labels)plt.show()
output-
To check the number of images in each class, a for loop was run. The results are then plotted using matplotlib.bar which is used to create bar charts.
From the data, it is clear, that there is a big difference in the number of images belonging to each label. If the network is trained with exactly these numbers of images, it might be biased towards the class with most labels. This is known as the Class Imbalance Problem. Hence it is necessary for each class to have a similar number of images, which we will talk about in the next part.
# Displays images # Extract 9 random imagesprint('Display Random Images')# Adjust the size of your imagesplt.figure(figsize=(20,10))for i in range(9):    num = random.randint(0,len(X)-1)    plt.subplot(3, 3, i + 1)        plt.imshow(X[num],cmap='gray')    plt.axis('off')    # Adjust subplot parameters to give specified paddingplt.tight_layout()
Finally, we use the random module to generate nine random images from the training set and then used matplotlib to plot these images.
This is the end of this part. As we see, for medical imaging analysis it is first very important to understand the dataset properly, in this case, X-ray images. In the next part, we will deal with the class imbalance problem and more operations using matplotlib and OpenCV.
References:
Dataset obtained from- Kermany, Daniel; Zhang, Kang; Goldbaum, Michael (2018), “Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification”, Mendeley Data, v2http://dx.doi.org/10.17632/rscbjbr9sj.Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning- (2018), Author: Daniel S. Kermany, Michael Goldbaum, Wenjia Cai, Carolina C.S. Valentim, Huiying Liang, Sally L. Baxter, Alex McKeown, Ge Yang, Xiaokang Wu, Fangbing Yan, Justin Dong, Made K. Prasadha, Jacqueline Pei, Magdalene Y.L. Ting, Jie Zhu, Christina Li, Sierra Hewett, et al., Publication: Cell Publisher: Elsevier.
Dataset obtained from- Kermany, Daniel; Zhang, Kang; Goldbaum, Michael (2018), “Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification”, Mendeley Data, v2http://dx.doi.org/10.17632/rscbjbr9sj.
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning- (2018), Author: Daniel S. Kermany, Michael Goldbaum, Wenjia Cai, Carolina C.S. Valentim, Huiying Liang, Sally L. Baxter, Alex McKeown, Ge Yang, Xiaokang Wu, Fangbing Yan, Justin Dong, Made K. Prasadha, Jacqueline Pei, Magdalene Y.L. Ting, Jie Zhu, Christina Li, Sierra Hewett, et al., Publication: Cell Publisher: Elsevier.