# Automatic Detection of Airway Invasion from Videofluoroscopy via Deep Learning Technology
## 2 Materials and Methods
### Proposed Model for the Detection of Airway Invasion
#### 2.3.2 Design of the Airway Invasion Detector  
The occurrence of airway invasion is detected by judging the characteristics of the movement of a bolus when swallowed, that is whether the bolus is moving toward the airway rather than toward the esophagus. Since it is impossible to predict when airway invasion will occur for each person in a video file that is approximately 2 to 104 s long, we first determine whether airway invasions are  
Figure 3: U-Net architecture designed for the localization of the cervical spinal column. Figure 2: Images illustrating the localization of the cervical spinal column and the specification of the ROI for airway invasion detection: (**A**) ground-truth cervical spinal column, (**B**) pixels classified as belonging to the cervical spinal column by U-Net, and (**C**) ROI image with dimensions of \(299\times 299\). evident in individual image frames, and based on the results, we finally determine whether airway invasions have occurred in the corresponding video file using context information. * Deep neural network for the detection of airway invasion: As described in Section 2.3.1, the ROI image (Figure 2C) specified using U-Net is used as the input image for airway invasion detection. To identify the occurrence of airway invasion in individual image frames, we designed a DCNN (deep convolutional neural network)-based classifier. Based on the position and shape of the bolus in the image, the classifier distinguishes whether the bolus is passing normally through the esophagus (normal swallowing action) or whether airway invasion has occurred. A deep neural network contains numerous connections between neurons, and a weight is assigned to each connection. The deeper the network is, the greater the number of weights to be learned, where this increase scales exponentially; consequently, sufficient data are needed to prevent overfitting. Therefore, among the models that can be effectively trained on limited data, in this paper, we choose a CNN based on an Xception module [27]. The Xception architecture not only reduces the number of parameters to be learned by performing convolution on each channel of the image, but also has the advantage of efficiently utilizing spatial and cross-channel correlations in the image (Figure 4). Additionally, by means of residual connections, the degradation problems that can occur as the network deepens can be avoided. In the Xception architecture, data are processed through three flows, namely (1) the entry flow, (2) the middle flow, and (3) the exit flow, and this architecture consists of convolutional layers with a total of 36 3 \(\times\) 3 filters. All common convolutional and separable convolutional layers are followed by batch normalization [28]. For airway invasion detection, we defined the image frames corresponding to instances of airway invasion as the positive class (Figure 5A) and the remaining image frames as the negative class  
Figure 4: Xception module used for airway invasion detection. (Figure 5B) and designed a CNN classifier using the Xception architecture with a binary output. Among the 15,335 positive image frames and 252,413 negative image frames included among all 319 video files, seventy percent of the videos in each class were used as training data, and ten percent were used for the validation of the trained network. The remaining 20% of the videos were used as test data to evaluate the final performance of the proposed classifier. Table 1 shows the total number of image frames used in the experiment and the numbers of frames in each class used for training, validation, and testing. In this study, we used an Xception module [27] pretrained on ImageNet and fine-tuned through transfer learning [25] on videofluoscopic image data (Figure 4). The pretrained Xception module takes an image with 3 channels and dimensions of \(299\times 299\) as the input, whereas a video fluoroscopic image is a grayscale image with a single channel.