###### Abstract  
Liquid-based cytology (LBC) plays a crucial role in the effective early detection of cervical cancer, contributing to substantially decreasing mortality rates. However, the visual examination of microscopic slides is a challenging, time-consuming, and ambiguous task. Shortages of specialized staff and equipment are increasing the interest in developing artificial intelligence (AI)-powered portable solutions to support screening programs. This paper presents a novel approach based on a RetinaNet model with a ResNet50 backbone to detect the nuclei of cervical lesions on mobile-acquired microscopic images of cytology samples, stratifying the lesions according to The Bethesda System (TBS) guidelines. This work was supported by a new dataset of images from LBC samples digitalized with a portable smartphone-based microscope, encompassing nucleus annotations of 31,698 normal squamous cells and 1395 lesions. Several experiments were conducted to optimize the model's detection performance, namely hyperparameter tuning, transfer learning, detected class adjustments, and per-class score threshold optimization. The proposed nucleus-based methodology improved the best baseline reported in the literature for detecting cervical lesions on microscopic images exclusively acquired with mobile devices coupled to the uSmartScope prototype, with per-class average precision, recall, and F1 scores up to 17.6%, 22.9%, and 16.0%, respectively. Performance improvements were obtained by transferring knowledge from networks pre-trained on a smaller dataset closer to the target application domain, as well as including normal squamous nuclei as a class detected by the model. Per-class tuning of the score threshold also allowed us to obtain a model more suitable to support screening procedures, achieving F1 score improvements in most TBS classes. While further improvements are still required to use the proposed approach in a clinical context, this work reinforces the potential of using AI-powered mobile-based solutions to support cervical cancer screening. Such solutions can significantly impact screening programs worldwide, particularly in areas with limited access and restricted healthcare resources. artificial intelligence; machine learning; deep learning; cervical cancer; cervical cytology; nucleus detection; lesion detection; computer-aided diagnosis; mobile devices +
Footnote †: journal: Medicalare increasing the interest in developing computer-aided diagnosis (CADx) systems for cervical screening. A recent review article [4] analyzed the approaches used for the tasks associated with examining microscopic images from cervical cytology smears, namely focus and adequacy assessment, region of interest segmentation, and lesion classification. Regarding segmentation and classification tasks, the authors point out that, despite the relatively good performance exhibited by binary or low-class classification approaches, the slow processing times and the considerable quantity of misclassifications or false positives reported for multi-class problems can make the algorithms unusable in a clinical environment. Additionally, the authors concluded that most works disregard adequacy assessment, while others only implement some techniques to detect and remove unwanted objects such as inflammatory cells or blood, with this topic being scarcely addressed in the literature. Considering the limitations identified in the existing literature, the authors of the current work proposed a nucleus-based approach for the automated adequacy assessment of cervical cytology smears [5]. In this work, major focus was given to the cellularity evaluation of the cytological samples since low squamous cellularity is the most common cause for the identification of specimens as unsatisfactory.