# Real-Time Multi-Label Upper Gastrointestinal Anatomy Recognition from Gastroscope Videos
## 3 Results
### Experimental Results
#### 3.2.2 GCN with LSTM Structure  
To demonstrate the importance of combining label association and temporal features for this task, we carried out a series of experiments by combining ResNet-50 with different modeling approaches, namely (1) ResNet-50 with GCN, and (2) ResNet-50 with GCN followed by LSTM. The experimental results are listed in Table 1. The scheme with LSTM achieved better results, demonstrating the importance of temporal correlation for more accurate identification. A total of 97.1% of mAP, 95.7% CF1 and 94.5% OF1 can be seen from the GL net method proposed in this paper. Specifically, compared with ResNet-GCN, our end-to-end trainable GL-Net improves the mAP, CF1 and OF1 by 4.0%, 6.1% and 7.4%, respectively. Similarly, we compared the average accuracy of the two schemes on each anatomical structure (see in Table 2). Compared with the ResNet-GCN model without the LSTM module, the accuracy of GL-Net in the anterior wall of middle-upper body, the lesser curvature of the lower body, the posterior wall of the middle-upper body, the large curvature of the middle-upper body and angulus were improved by 22.5%, 17.6%, 11.2%, 8.8% and 8.3%, respectively. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Anatomy** & **CNN-RNN** & **RNN-Attention** & **ResNet50** & **ResNet-GCN** & **GL-Net** \\ \hline esophagus & 0.903 & 0.821 & 0.998 & 0.999 & **1.0** \\ Squamocolumnar junction & 0.880 & 0.748 & 0.986 & 0.994 & **0.999** \\ Cardia I & 0.867 & 0.715 & 0.965 & 0.967 & **0.998** \\ Cardia O & 0.873 & 0.658 & 0.994 & **1.0** & **1.0** \\ Fundus & 0.855 & 0.757 & 0.978 & **0.979** & 0.902 \\ Middle-upper body A & 0.604 & 0.595 & 0.809 & 0.906 & **0.994** \\ Middle-upper body L & 0.585 & 0.594 & 0.813 & 0.872 & **0.984** \\ Middle-upper body P & 0.546 & 0.562 & 0.680 & 0.745 & **0.970** \\ Middle-upper body G & 0.586 & 0.518 & 0.601 & 0.885 & **0.904** \\ Lower body A & 0.595 & 0.548 & 0.841 & **0.918** & **0.918** \\ Lower body L & 0.575 & 0.517 & 0.713 & 0.802 & **0.821** \\ Lower body P & 0.585 & 0.454 & 0.816 & 0.876 & 0.936 \\ Lower body G & 0.541 & 0.568 & 0.731 & 0.812 & **0.988** \\ Antrum A & 0.746 & 0.727 & 0.977 & 0.979 & **0.999** \\ Antrum L & 0.732 & 0.710 & 0.987 & 0.984 & **0.997** \\ Antrum P & 0.755 & 0.722 & 0.983 & 0.985 & **0.995** \\ Antrum G & 0.706 & 0.730 & 0.976 & 0.981 & **0.997** \\ Angulus G & 0.793 & 0.756 & 0.898 & 0.905 & **0.988** \\ R-middle-upper body A & 0.621 & 0.534 & 0.855 & 0.900 & **0.980** \\ R-middle-upper body L & 0.648 & 0.542 & 0.892 & 0.925 & 0.959 \\ R-middle-upper body P & 0.770 & 0.651 & 0.919 & 0.951 & **0.956** \\ R-middle-upper body G & 0.766 & 0.636 & 0.908 & 0.945 & **0.997** \\ Duodenal bulb & 0.751 & 0.623 & 0.997 & 0.998 & **0.999** \\ Duodenal descending & 0.906 & 0.827 & 0.998 & **1.0** & **1.0** \\ Pylorus & 0.753 & 0.722 & 0.969 & 0.980 & **0.997** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of average precision of each individual anatomy identification with other methods. Bold indicates the maximum value. By introducing label association and temporal information, our GL-Net can learn features that are more discriminative than those produced by traditional CNNs that consider only visual information. Figure 8 shows the comparison of the prediction results of the two models in the video clips. It can be seen that due to the shooting angle, bubble reflection and other reasons, the variance between some classes is small and difficult to distinguish, or sometimes features are almost completely covered. Therefore, the ResNet-GCN network, which only depends on the features of a single frame image, cannot classify correctly, while GL-Net can avoid the error by considering time dependence, and each frame is identified accurately. In addition, some frames in the video have no classification results, that is, the confidence of all the predicted results does not exceed the set threshold, which may be related to the noise in the video.