# Automatic Detection of Airway Invasion from Videofluoroscopy via Deep Learning Technology
## 2 Materials and Methods
### Proposed Model for the Detection of Airway Invasion
#### 2.3.2 Design of the Airway Invasion Detector
Therefore, we used the same grayscale ROI image resized to \(299\times 299\) as the input to each of the three channels. The transfer learning procedure increased the validation accuracy by approximately 5% compared to training from scratch. * Classification of airway invasion for a whole video: Figure 6 shows the classification results of the Xception module-based CNN for each image frame included in the videos from two subjects participating in the VFSS procedure (Figure 6A: results \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Training** & **Validation** & **Test** & **Total** \\ \hline No-Invasion Images & 137,095 & 41,322 & 73,996 & 252,413 \\ Invasion Images & 10,524 & 2010 & 2801 & 15,335 \\ Total & 147,619 & 43,322 & 76,797 & 267,748 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Statistics of videofluoscopic swallowing study (VFSS) frame data. Figure 5: Visualization of the invasion of a bolus into the larynx in VFSS image frames: (**A**) image acquired during airway invasion (positive)and (**B**) image acquired during normal swallowing (negative). for a person with normal swallowing; Figure 6B: results for a patient). In Figure 6, the horizontal axis shows the acquisition time index of the image frames, and the vertical axis shows the classification result for each image frame. A value of 1 appears for a frame that is classified as positive, and a value of 0 indicates a negative classification. Based on the classification results for the individual image frames as shown in Figure 6, we can finally determine whether airway invasion occurred in a particular video of a subject participating in the VFSS. Since the bolus-swallowing motion is a continuous motion, the images acquired during a true instance of airway invasion should be classified as positive samples continuously along the time axis. To capture the temporal context of each video in the final decision on whether airway invasion has occurred, we applied two types of filters, namely a median filter [29] and a convolution filter, to the classification results for the individual images, as shown in Figure 6.  
First, since the video files we used were recorded at 24 fps, it is reasonable to regard positive samples generated in the form of intermittent impulses, as indicated by the blue boxes in Figure 6A,B, as false positives. Therefore, we chose to correct the classification results for such  
Figure 6: Classification results for airway invasion using the Xception module-based CNN: (**A**) results for a person with normal swallowing, (**B**) results for a patient, (**C**) results for a person with normal swallowing after correction using a median filter, and (**D**) results for a patient after correction using a median filter. samples to negative results using a median filter with a length of 5. From a similar perspective, we determined that airway invasion can be considered to have occurred only when two or more of the classification results for seven consecutive image frames are classified as positive samples. Formally, let the vector \(\mathbf{r}\in R^{l}\) contain the classification results (Figure 6) for all image frames of a video composed of \(l\) frames. We consider that airway invasion has occurred only when the following \(F_{\mathbf{r}}\) is two or more:  
\[F_{\mathbf{r}}=\sum_{i=1}^{l}\sum_{k}\mathbf{r}(k)\mathbf{f}(i-k) \tag{1}\]  
Here, \(\mathbf{f}\) is a convolution filter of length 7 with a value of 1 for all components (Figure 7).