# Ulcerative Severity Estimation Based on Advanced CNN-Transformer Hybrid Models
## 1 Introduction
Additionally, models such as UNETR, which integrate transformer encoders with U-Net decoders, incur considerable computational and memory overhead, posing difficulties for deployment on resource-constrained medical devices, such as endoscopic image analysis systems. **Hybrid Models**: From the above analysis, it is clear that both CNN and transformer models are often task-specific, with limited generalizability and cross-modal adaptability, making them insufficient to meet the diverse and evolving demands of real-world medical applications. To overcome these limitations, hybrid CNN-transformer models have been proposed, aiming to leverage the complementary strengths of both architectures. By combining the strong local feature extraction capabilities of CNNs with the global contextual understanding of transformers, these models improve the ability to detect both fine-grained lesion features and broader spatial patterns, thereby enhancing diagnostic performance. By incorporating locality principles into transformer frameworks, some hybrid architectures achieve superior performance [19]. MedViT integrates efficient convolution blocks (ECBs) to reduce the quadratic complexity of self-attention and enable joint attention across diverse representational subspaces [20]. [PERSON] et al. proposed UC-NfNet, a hybrid CNN-transformer model with an image generation module, which demonstrated high effectiveness on a newly curated ulcerative colitis (UC) dataset of 673 images [21]. While these previously discussed models have demonstrated efficacy in their respective domains,they are not well suited for our research objectives due to their highly specialized design. MedViT, for instance, predominantly emphasizes the extraction of global structural features such as shape and overall appearance but overlooks fine-grained texture details critical for our dataset. Meanwhile, UC-NfNet's integration with an image production technique renders it incompatible with our research focus. In contrast, our study aims to identify a versatile hybrid model that seamlessly combines the strengths of CNNs and transformer architectures. Given the distinct structural patterns present in our images, CoAtNet emerges as the optimal choice. By effectively integrating local feature extraction capabilities of CNNs with the global attention mechanism of transformers, CoAtNet strikes a balance between capturing fine-grained details and understanding overall structural contexts, thereby fulfilling the specific requirements of our research. **Research Status of Intestinal Endoscopic Image Classification**: To the best of our knowledge, intestinal endoscopic image classification still largely depends on traditional deep CNNs, with most studies conducted on relatively small datasets. In contrast, the LIMUC dataset, released in 2022, is the largest publicly available collection of intestinal endoscopic images to date, and its high quality has been demonstrated in previously published studies, as discussed later in this paper. However, related research is scarce, with only two published studies. [PERSON] et al. introduced the dataset and applied regression-based deep learning to establish baseline performance, highlighting its potential for model development and external validation [22]. Another study enhances classification accuracy and quadratically weighted kappa (QWK) by proposing a Class Distance Weighted Cross-Entropy (CDW-CE) loss function based on conventional CNN architectures [23]. Despite its scale and significance, research on the LIMUC dataset has seen little progress over the past two years. This highlights the need for further exploration of this dataset and the development of more effective models tailored to its characteristics.