# Real-Time Multi-Label Upper Gastrointestinal Anatomy Recognition from Gastroscope Videos
## 3 Results
### Experimental Results  
#### 3.2.1 GCN Structure  
The statistical results are presented in Table 1, and we compared them with related current spatial-temporal methods, including CNN-RNN [30], RNN-Attention [31], etc.  
The models involved in the comparison all used the same set of training and test data. The benchmark backbone was kept uniform for a fair comparison. It is clear to see that the GCN-based approach obtains the best classification performance, which is due to capturing the dependencies between the labels. Compared to advanced methods for capturing dependencies of frames, our method achieves better performance in almost all metrics, which proves the effectiveness of GCN. Specifically, the proposed GCN scheme obtained 93.1% of mAP, which is 21.1% higher than their method. Even using the ResNet-50 model as the backbone model, we could still achieve better results (+17.1%). This suggests that there are strong dependencies and correlations between anatomical location labels in full-coverage examinations in white-light endoscopy scenarios, and the basic backbone of CNN together with a GCN structure can capture them well.  
We further use the heatmaps to explain the model. By weighting and summing the class activation maps [54] of the final convolutional layer, the attention map can accurately highlight the areas of the image that have a high weight on recognition, thus revealing the network's implicit attention to the image and intercepting the learning information of the network [54]. The attention maps of models is shown in Figure 7.  
As illustrated in Figure 7, both in the middle and upper part of the stomach body, the GCN-based model was the best in terms of visual representation. For the other models, it is easy to randomly place weights at some locations in the image without constructing label  
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Method** & **mAP** & **CP** & **CR** & **CF1** & **OP** & **OR** & **OF1** \\ \hline CNN-RNN [30] & 0.720 & 0.772 & 0.772 & 0.772 & 0.785 & 0.770 & 0.777 \\ RNN-Attention [31] & 0.649 & 0.665 & 0.578 & 0.618 & 0.610 & 0.660 & 0.634 \\ ResNet-50 & 0.891 & 0.936 & 0.777 & 0.849 & 0.912 & 0.734 & 0.813 \\ ResNet-GCN & 0.931 & 0.941 & 0.855 & 0.896 & 0.929 & 0.820 & 0.871 \\ GL-Net & **0.971** & **0.955** & **0.959** & **0.957** & **0.937** & **0.954** & **0.945** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of average assessment results of anatomy multi-label identification with other methods. Bold indicates the maximum value.  
Figure 7: Visualization of the attention maps. The label on the left indicates the label for visual gradient back propagation and also the ground truth annotation. The upper label indicates the model name of the visualization.The visualization region is approximately close to red, indicating that more of the model inference weight tends to be in that region.  
associations. It is not possible to correctly distinguish between these anatomical structures that do not differ much between classes (greater curvature, posterior wall, anterior wall and less curvature of the body are difficult to distinguish). In contrast, the GCN-based model can pay more attention to feature regions in the image where texture features are prominent and responsive to the class. For the gastric angulus, because the structure of the angulus is prominent in the visual field, the general model can pay attention relatively accurately at this location. Compared to other models, the GCN-based model's weights are able to provide more comprehensive and complete coverage at this location, including the lesser curvature of the antral.