# Ulcerative Severity Estimation Based on Advanced CNN-Transformer Hybrid Models
## 1 Introduction  
Ulcerative colitis (UC), a chronic and relapsing subtype of inflammatory bowel disease (IBD), is characterized by continuous inflammation and ulceration confined to the mucosal layer of the colon [1]. This pathological process manifests clinically as prominent symptoms, with diarrhea and rectal bleeding being the most characteristic hallmarks [2]. However, the implications of UC extend beyond these acute symptoms; without proper management, the condition gradually increases the risk of colorectal cancer over time [3]. Given the evolving disease landscape of ulcerative colitis (UC) and its potential long-term consequences, early and accurate diagnosis is crucial for effective disease control. Conventionally, diagnosis relies on a comprehensive assessment that integrates clinical manifestations, endoscopic examinations, histopathological findings, and the exclusion of other infectious or noninfectious forms of colitis [4]. This limitation, however, is being addressed by the rapid progress of artificial intelligence (AI) in healthcare. Deep learning, in particular, has emerged as apotent force in medical image analysis, effectively showcasing its substantial potential for detecting and diagnosing diverse diseases [5]. Prevalent CNN models primarily focus on local characteristics, whereas transformers emphasize global information. In addition, although CNNs and transformers have demonstrated robust performance on large-scale datasets, they face significant challenges in the medical domain, where datasets are typically much smaller. Moreover, considering the characteristics of medical images--particularly intestinal endoscope images--pure CNNs may tend to focus excessively on specific local features, which can lead to the learning of spurious correlations and ultimately result in overfitting. On the other hand, pure transformers may fail to adequately capture subtle differences in lesion areas, thereby reducing their ability to distinguish between different degrees of pathological severity. Against the backdrop of these technological advancements, this study undertakes an in-depth exploration of cutting-edge neural network architectures, specifically focusing on their application in the classification and interpretability of intestinal endoscopic images. Specifically, the CoAtNet model inherits the generalization capability of CNNs, such as ConvNets, along with the global feature awareness of traditional vision transformers (ViTs), making it particularly well suited for medical imaging tasks such as intestinal endoscopy. Moreover, unlike other hybrid CNN-transformer models, CoAtNet adopts a vertically structured design that clearly separates the CNN and transformer stages. This design leverages prior knowledge, emphasizing the convolution network's foundational role. Notably, the original study demonstrates that CoAtNet achieves strong performance even on relatively small datasets, as evidenced by its success when trained directly on ImageNet-1k. These strengths collectively motivate our choice of CoAtNet as the backbone model for our experiments [6]. By leveraging its architectural strengths in capturing both local and global features, we aim to enhance the accuracy and reliability of clinical examination and classification. Additionally, we seek to elucidate the decision-making mechanisms of the model to validate its practical usability and trustworthiness. The main contributions of this study are summarized as follows:  
* This study evaluates the classification performance of the CoAtNet model on the LIMUC dataset, benchmarking it against cutting-edge architectures, including pure convolutional and transformer-based models. Through this comparative analysis, the distinct strengths and potential advantages of CoAtNet emerge, positioning it as a notable advancement within the field. * Through the combination of the CDW-CE loss function, we achieve further improvements in both QWK and classification accuracy.