# Reduction of Video Capsule Endoscopy Reading Times Using Deep Learning with Small Data
## 2 Materials and Methods  
In this work, we used publicly available data from the \(\kappa\dot{a}\psi\psi\lambda\alpha\) interactive database (KID) [20; 21]. The database consists of two types of data: still frame images and videos. All data in the database was collected with a MiroCam device developed by IntroMedic Co in Seoul, Korea. The MiroCam has a recording frame rate of 3 frames per second [22]. However, the videos provided in the database are all MP4 files with a frame rate of five frames per second. Previous work [23; 24; 25; 26; 27; 28; 29; 30] using this database focused on classification on the frame only data known as KID Datasets 1 and 2. These datasets were annotated by expert clinicians and come with both classification and segmentation labels. For the videos, only the KID video 1 contains classification labels and it is the only one to have been used in previous work [31] referencing the KID dataset. The other two videos are without expert annotation; with the help of our local Veteran's Affairs (VA) Hospital, we are the first to use these previously unannotated KID videos. Our senior author, a VA physician with 20 years of GI experience, labeled videos 2 and 3 and adjusted the labels on the first video according to their findings. This resulted in three full-length capsule endoscopy videos with annotations that we used for cross-validation of our deep learning approach. The goal of this work is to reduce the physician time needed to read the VCE videos; thus, our framework focused on removing confidently predicted normal frames which are not needed for diagnoses. Before training a model, we had to pre-process the videos by separating each into frames using the five frames per second speed of the MP4 files. All frames contain a time code in the top corner as seen in Figure 1, which are what the physician uses to denote the location of any abnormalities. The ground truth for this data is a list of segments with a start and ending time of when the abnormality is visible in the video. To provide the label for each image, we used optical character recognition to read the time code and determine if the frame was indicated as abnormal by the physician. The final step before training the deep learner was to remove the time code and IntroMedic text from the images as seen in Figure 2, to prevent the model from seeing this information and using it to make predictions. We decided to use a well-known and robust convolutional neural network architecture, ResNet50 [32], which has been trained to achieve high performance on various classification benchmark datasets. ResNet50, with initial weights from training on ImageNet [33], was fine-tuned with the KID videos dataset. The classification layer of ResNet was replaced by a single fully connected layer with two neurons as seen in Figure 3. This output layer used a sigmoid activation function to classify the frames as either normal or abnormal. All models were trained using stochastic gradient descent as the optimizer using a learning rate of 0.001 with class weights to account for the abnormal frames being a minority. Typically, a network would have all its starting weights generated randomly prior to training, and learn optimal weights based on the training data. However, we had a very small dataset in this study, so we decided to use the final weights learned on the ImageNet dataset as the starting point [33]. As we only had three annotated videos to work with, it was decided that experiments would be conducted using a three-fold cross validation scheme. Thus, we would train a model with two videos, testing on the third, and repeat for all combinations. With our architecture and pre-processing pipeline chosen, our first experiment was to train a model using the videos and make predictions with that model. We found that because our dataset was small, this single model approach did not perform well.