# Nodule Detection with Convolutional Neural Network Using Apache Spark and GPU Frameworks
## 3 Our Proposed Methodology
### Pre-Processing
The patient with a nodule size greater than 1.25 mm is diagnosed as malignant, and those who have less than 1.25 mm are labeled as benign. All the patients have an individual folder containing multiple DICOM files with their id. The training and testing set is annotated along with ground truth labels of patients in CSV format with nearly 70% of patients without cancer, and the rest 30% diagnosed otherwise. The annotation file contains 1186 nodules. ### Experimentation

In our paper, we have done experimentation to solve two problems. (1) Early Lung nodule detection (2) Minimizes the training time and speeds up the model's convergence. The initial segment explains the ROC and error rate of our proposed model CNN compared to other prevailing neural networks. The second part talks about distributed computing

\[Correntropy_{\sigma}(A,B)=E[k_{\sigma}(A-B)], \tag{2}\]0 Selected nodule candidates from the features of the lung CT scans. using Apache Spark to converge the algorithm with minimal training time. This gives a swift opportunity to tune the best hyperparameters for our model. We compared the results of performance analysis on three different platforms CPU, GPU and Apache Spark. We trained and evaluated our CNN model using four-fold cross-validation, and separate untouched data is used for testing. We randomly split the training data into four subsets of equal size for cross-validation. Hyperparameters are fine-tuned from the validation results, such as the number of filters K, their spatial extent F, the stride S, and the amount of zero padding P. TensorFlow is used as a deep learning library used in our CNN model with Nvidia GeForce GTX 750 GPU setup with 16 GB RAM. To deal with multiple format, DICOM files libraries, such as pydicom, matplotlib, and NumPy, are used. The CT image's pixel size is 512 \(\times\) 512 with a depth of 195, which is very large and cannot be fed directly into our model because of the high computational cost. Pre-processing of the dataset is done to resize the data and control uniformity among all the patient's scans. The slices provided in the dataset's scans are not uniform, even though the image is of similar size. Our proposed CNN model used three convolutional layers. ReLU is used as an activation layer in our work [39], three max-pooling layers, along with pre-processing noise removal layer. Furthermore, we handled one fully connected layer and a softmax layer as an activation function. The pixel size is resized to 99 \(\times\) 99 without padding to minimize the computing complexity. In the max-pooling layer, the lung scan's pixel size is 49 \(\times\) 49 as the upsampling layer is used to reduce the pixel values. This reduction is entirely based on the trained samples from which the region of interest is detected. We set the batch size to 160 with a complete training image set of 25,600. 15 \(\times\) 15 convolution filter and 2 \(\times\) 2 for pooling operation are applied in our algorithm, and the stride is chosen randomly with a pixel size of 7 \(\times\) 7. In the second denoising layer, the pixel size is 48 \(\times\) 48 and 24 \(\times\) 24 during max pooling for the second time. Finally, in the third convolutional layer, the lung image is 23 \(\times\) 23 and max pooled to 11 \(\times\) 11. We apply noise removal as a pre-processing step where the pixel size is reduced to 23 \(\times\) 23. Furthermore, after each convolutional layer, we use noise removal in every iteration for visual noise elimination. The softmax layer of our methodology is 2 \(\times\) 1. Our dataset size is 160 GB in total, and for training, we used 1595 patients. The quantity of slices for each patient varies according to the scans taken at the hospital. The slices per patient are put into a particular number of chunks. The input batch size is 160, and 40 iterations per epoch are used to train the samples with a learning rate of 10\({}^{-2}\) and gradually decayed over the process. The iterations are set to 10,000 with 250 epochs and four-fold cross-validation.