Retinalvesselsegmentationplaysamajorroleinidentifyingandscreeningeye-relateddiseasessuch as glaucoma, diabetic retinopathy, and microaneurysm. Utilising the automatic feature extraction capabilities of deep learning models, there is a potential to reduce the reliance on human exper- tise for disease diagnosis, enabling physicians to make faster and more efficient diagnoses. This study focuses on evaluating the effect of uncomplicated data augmentation techniques on retinal vessel segmentation, offering a comparative analysis across eight diverse Convolutional-based and transformer-basedarchitectures,includingUNet,Attention-UNet,Res-UNet,UNet++,UCTransNet, TransUNet, UNeXt and SwinUNet. Each model is trained on four publicly available datasets with differentresolutions,fromlow-restohigh-resimages,including 𝐷𝑅𝐼𝑉 𝐸,𝑆𝑇 𝐴𝑅𝐸,𝐶𝐻𝐴𝑆𝐸 −𝐷𝐵1, and𝐻𝑅𝐹. Subsequently, the study provides comparative results, offering insight into the efficiency of these models in retinal vessel segmentation. Obtained results indicate that even though the overall difference between models’ performance is small, convolutional models perform better than transformers when trained on lower-resolution datasets (i.e. DRIVEdataset), while transformers are best on Higher-resolution datasets (i.e. HRF). 1. Introduction Retinal vessel segmentation is a crucial component of computer-assisted screening, diagnosis, and treatment for a variety of cardiovascular and eye-related disorders, such as stroke, diabetes, hypertension, and diabetic retinopathy. Furthermore, it can be used for biometric identification by utilising the unique properties of an individual’s retinal veins. Given fundus cameras are specialised, and blood vessels are small, retinal vascular segmentation presents a unique set of challenges. The camera and outside environ- ment substantially impact image quality, which in turn af- fectssegmentationperformance.Additionaldifficultiesarise fromthefeaturesofretinalbloodvessels,whichincludetheir abundance, small size, curvature, and variability in shape. Problems include parallel or closely positioned vessels dur- ing segmentation, missing detection of small vessels, and trouble with broken vessels at bifurcations [1]. Recognisingthetime-consuminganderror-pronenature of manual segmentation performed by professionals, there has been a significant effort in automating retinal blood vessel segmentation. The majority of early automatic tech- niques for segmenting retinal vessels were concentrated on segmenting images of the fundus, examining connections between backdrop and vessel features, and extracting fea- tures from the images through the utilisation of matched filters, vessel tracking, and morphological reconstruction- based techniques[1]. Owing to the advancements in Deep Learningandtheincreasedavailabilityofcomputingpower, recent years have witnessed a substantial increase in the number of studies and enhancements in the performance ∗Corresponding author ORCID(s): 1of retinal vessel segmentation algorithms when applied to fundus images. Thefieldofbiomedicalimagesegmentationexperienced notable advancements since the introduction of the UNet network[2].Manymedicalimagesegmentationstudiesrely on UNet and its extension architectures, and the number of research works published using UNet as the baseline model has significantly increased over the past few years. Besides, to address the complexity of medical image analysis and the drawbacks of the naive UNet architecture, a variety of different extensions of this architecture have been proposed over the years [3]. For example, [4] introduced a new ar- chitecture based on the UNet by modifying skip connec- tions and sharing weights between layers, which reduced the model’s susceptibility to overfitting. [5] proposed a new network named "Octave UNet" for vessel segmentation in colour fundus images using encoder-decoder-based octave convolutionnetworks.In[6],theIterNet,anewnetwork,was introduced based on a deeper mini-UNet that uses weight sharingandskipconnectionsinretinalvesselsegmentation. Inanotherstudy,[7]proposedtheChannelAttentionResid- ual UNet to accurately segment retinal vascular and non- vascular pixels, achieving state-of-the-art performance on threedatasets,includingDRIVE,CHASEDB1andSTARE. Inanotherinterestingpaper,[8]introducedtheSuperVessel network. The method includes a super-resolution auxiliary branch for possible high-resolution detail features, which is used in training and excluded in testing. Furthermore, two modules are included to improve segmentation features: a FeatureInteractionModule(FIM)witharestrictinglossthat focusesontheregionofinterestforbettersegmentationand an Upsampling with Feature Decomposition (UFD) mod- ule. [9] utilised the UNet++ architecture, which includes a <short author list for running head>: Preprint submitted to Elsevier Page 1 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> UNet structure with intermediate nodes that perform multi- resolution aggregation of features combined with data aug- mentation techniques and achieved comparable results with state-of-the-art retinal image segmentation models. Following the introduction of the attention mechanism [10] and vision transformers [11], a variety of architectures were developed based on utilising various transformers. For example, [12] proposed a lightweight Spatial Attention UNet (SA-UNet) that employs a spatial attention module for inferring spatial dimensions and structured convolution dropout blocks to prevent overfitting. [13] conducted an empirical study on data processing, architecture design, attention mechanisms, and regularization strategies. Based on the insights gained from this review, they proposed a well-designed deep neural network that incorporated best practices and achieved high performance with AUCs of 0.98279,0.98862,and0.98724ontheDRIVE,STARE,and CHASE-DB1 datasets, respectively. However, the obtained performanceisstillslightlybelowthestate-of-the-artperfor- manceobtainedby[14]with 𝐴𝑈𝐶 −𝑅𝑂𝐶equalto0.9887, 0.9914, and 0.9887 for DRIVE, CHASE-DB1, and STARE datasets,respectively.[15]developedtheConnectionSensi- tiveAttentionUNettoenhancetheaccuracyofretinalvessel segmentation. [16] suggested SDAUNet, a retinal vascular segmentation technique that improves the UNet structure by substituting a series deformable convolution module for the convolution module. Integrating lightweight and dual attention modules into the decoder enhances extracting fea- turesfromretinopathyimages,particularlyforsmallvessels withintricatemorphology.Inanotherstudy,[17]proposeda modified UNet with a supervised attention mechanism that uses a Context Squeeze and Excitation module in the de- coding module to amplify contextual information for small blood vessels and introduces a Decoder Fusion Module for thorough feature extraction in the encoding part. A Su- pervised Fusion Mechanism produces the final result and enhancessegmentationperformancebymergingmulti-scale characteristics and gathering data from different levels. The Effect of proper data augmentation techniques on improving retinal image segmentation accuracy is investi- gated and highlighted in a series of studies, including [18] and [19]. Obtained results of [18] indicated that while the performancegapbetweendifferentarchitecturesandmodels areusuallysmall,theseminordifferencescouldbeobtained orevensurpassedthroughproperdataaugmentation.In[19], authors used affine transformations like rotate, flip, zoom Out,randomcrop,shiftandshearingandelastictransforma- tionslikeelasticdeformations,griddistortion,opticaldistor- tionandPixel-leveltransformationslikeaddingwhitenoise, gamma correction, histogram equalization, pixel Dropout, sharpening, blurring and contrast adjustment for data aug- mentation. In another study, [20] increased the accuracy of retinal vessel segmentation by introducing two new data augmentation modules based on image gamma correction and replacing the ResNet-50 as the encoder section of the UNet network architecture. In [21], authors proposed an SGLschemetoimprovetherobustnessofthemodeltrainedon noisy labels to deal with retinal vessel segmentation task’s limitations like small-scale datasets and incorrect or incomplete vessel annotations. They utilise concatenated Unet architecture consisting of the enhancement and seg- mentation modules to learn the enhancement and the seg- mentation map. They also utilised random cropped patches of 256 ×256 and applied data augmentations such as hor- izontal and vertical flips, rotations, transpose and random elastic wrapping. So far, various studies have focused on developing dif- ferentarchitecturesandobtaininghigheraccuraciesthrough utilising deeper or more advanced models and preprocess- ing and augmentation strategies. Yet, there is space for a comprehensive analysis to compare how different devel- oped architectures and models respond to the image quality andpreprocessingandaugmentationtechniques.Hence,this study evaluates the accuracy of different convolutional and transformer-based architectures in segmenting the retinal vasculartreeinthefundusimagesofretinalvessels,utilising imagepreprocessingandpatchextractiondataaugmentation strategy. Consequently this paper evaluates the accuracy of nine architectures including naive UNet [2], AttUNet [22],ResUNet[23],UCTransNet[24],TransUNet[25]with ViT-B16 and ResNet50-ViT-B16 backbones, UNet++ [26], UNeXt [27], and SwinUNet [28] over four publicly avail- able datasets, with the different image resolutions including DRIVE [29], STARE [30], CHASE-DB1 [31], and HRF [32]fromlow-resolutiontohigh-resolution,respectively.All architectures are trained over raw and preprocessed images, and the results are then compared. The remainder of this paper is hence structured as fol- lows:Section2presentsanoverviewoftheutilisedarchitec- tures, datasets, evaluation metrics, and implementation de- tails; Section 3 presents the results obtained through model training experiments; Section 4 provides a discussion on the obtained results and comparison with other studies; and finally Section 5 presents the conclusions of the study. 2. Methodology 2.1. Datasets This study performs a comparative analysis of the per- formance of different model architectures on four publicly availabledatasetswithdifferentimageresolutionsfromlow- restohigh-resimages,includingDRIVE[29],STARE[30], CHASE-DB1 [31], and HRF [32]. An overview of their specifications is provided below. DRIVE dataset comprises 40 images, each with dimen- sionsof584 ×564pixels,ofwhich20imagesareconsidered for training and 20 for testing. These images were acquired as part of a diabetic retinopathy diagnosis program in the Netherlands utilising a Canon CR5 non-mydriatic 3CCD camera with a 45 °field of view (FOV). STAREdatasetcomprises20images,eachwithdimensions of605 ×700pixels,capturedwitha35-degreefieldofview, <short author list for running head>: Preprint submitted to Elsevier Page 2 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> Figure 1: Comparison of raw (left side) and Gamma Corrected and Hue-Saturation-Value adjusted (right side) images from (a) Drive, (b) STARE, (c) CHASE-DB1, and (d) HRF datasets ofwhich5imagesareallocatedfortestingand15imagesfor training of networks. CHASE-DB1 dataset contains 28 retinal images with the resolutionof960 ×999pixelscapturedusinganNM-200-D handheld fundus camera with a 30 °field of view, of which 20imagesareallocatedfortrainingand8imagesfortesting. HRFdataset comprises 45 images and is organised as 15 subsets. Each subset contains one healthy fundus image, one image of a patient with diabetic retinopathy and one glaucoma image, each with dimensions of 3304 ×2336 pixels.Inthisstudy,weallocated30imagesfortrainingand 15 for test. 2.2. Image Preprocessing and Augmentation Image enhancement methods like gamma correction, CLAHE, and spatial transformations like Rotate and Flip have proven useful for retinal image segmentation tasks in different studies (e.g., see [19, 20]). Through a trial and error method, it is found that the random adjustment in Hue-Saturation-Value colour space can be helpful in low- qualitydatasetslikeDRIVE.Consequently,inthisstudy,we consider random Hue-Saturation-Value (HSV) Adjustment combinedwithrandomGammacorrectionasimagepractice to improve the quality of images and increase the contrast of vessels with the background. Figure 1 compares raw andpreprocessedimagesforrandomsamplesfromdifferent datasets. Theaugmentationmethodsutilizedinthisstudyinclude rotationandhorizontalandverticalFlip,whichiscarriedout utilising the Albumentations [33] library with default con- figs.Resizeaugmentationisnotutilisedinthisstudyastrial and error indicated that it leads to the loss of thin vessels.Besides, it should be mentioned that the data augmentation processisentirelyseparatedfromnetworktrainingtoensure all networks are trained with the same data. 2.3. Utilised Architectures As previously discussed, this study intends to assess the efficacyofdifferentUNet-basedandtransformer-basedseg- mentation methods on four different retinal datasets. Thus, the performance of various segmentation models, includ- ing UNet [2], AttUNet [22], ResUNet [23], UCTransNet [24],TransUNet[25]withViT-B16andResNet50-ViT-B16 backbones, UNet++ [26], UNeXt [27], and SwinUNet [28] are considered and compared. Following, we present an overview of the employed network architectures. UNet[2] is a convolutional neural network architecture designed for semantic segmentation tasks in medical image analysis. Architecture is characterised by a U-shaped net- work structure, with a contracting path to capture context and a symmetric expanding path to enable precise locali- sation. This design facilitates incorporating both high-level and low-level features, making UNet particularly effective for image segmentation tasks where detailed spatial infor- mation is crucial. AttUNet [22] is an extension of the UNet architecture that incorporatesattentionmechanismstoenhanceitssegmenta- tion performance. The network incorporates attention gates atmultiplescalestoselectivelyfocusoninformativeregions while suppressing irrelevant backgrounds. This attention mechanism allows the model to dynamically adapt its focus during the segmentation process, making it particularly ef- fectivefortaskswherethetargetstructuresmayvaryinsize, shape, or context. <short author list for running head>: Preprint submitted to Elsevier Page 3 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> Table 1 Number of trainable parameters for Utilised architectures UNet AttUNet ResUNet UCTransNet UNet++ TransUNet TransUNet UNeXt SwinUNet Number of Params ( × 106)34.53 34.87 13.04 65.6 9.16 91.67 105.27 1.47 41.38 Figure 2: Selection of 224 ×224 pixel extracted patches and their corresponding mask from DRIVE dataset utilising the primary approach ResUNet [23]buildsupontheUNetarchitectureandincor- poratesresidualconnectionsinspiredbythesuccessofresid- ual networks in image classification tasks. By addressing the vanishing gradient problem, residual connections help to overcome the challenge of training deep networks. This approach was developed for extracting roads from satellite or aerial photos, where precisely outlining road networks is critical for various applications such as urban planning and infrastructure management. UCTransNet [24]arguesthatthecurrentUNetframework with a simple skip connection scheme is still challenging to model the global multi-scale context of complex data. Consequently, this architecture proposes a Channel Trans- former (CTrans) to replace the skip connections in UNet, which consists of two modules: ( 𝑖) Channel-wise Cross Fu- sion Transformer (CCT) for the multi-scale encoder feature fusion and ( 𝑖𝑖) Channel-wise Cross Attention (CCA) for the fusion of the decoder features and the enhanced CCT features. UNet++ [26], a nested UNet architecture, represents an advancement in medical image segmentation designed to address the challenges of capturing intricate details and hierarchical features in medical images. The architecture adds stacked skip connections to the classic UNet structure, resulting in a multi-level feature hierarchy. This stackedarchitecture allows the model to gather information at sev- eral scales, from fine-grained details to wider contextual characteristics. Stacked skip connections improve the flow of information across network tiers, boosting the model’s capacity to identify structures in medical images correctly. TransUNet [25] is a novel architecture for medical image segmentation. Leveraging transformer models, traditionally successful in natural language processing, the authors em- ploy them as robust encoders for capturing intricate fea- tures in medical images. This fusion of transformers with UNetarchitectureenhancesthemodel’sabilitytounderstand contextual information and spatial relationships, proving effective for tasks in medical image analysis. It is worth mentioning that in this paper, we use two Backbonesfortheencoderwithpre-trainedweightsinTran- sUNet, namely " 𝑉 𝑖𝑇 −𝐵𝑎𝑠𝑒 − 16" and " 𝑅𝑒𝑠𝑁𝑒𝑡 50 − 𝑉 𝑖𝑇 −𝐵𝑎𝑠𝑒 − 16",whichhavesmallertrainableparameters compared with other backbones. UNeXt[27] proposes a new convolutional multilayer per- ceptron (MLP) based network for image segmentation. The paper suggests that the current state-of-the-art medical image segmentation methods, such as UNet and its latest extensions like TransUNet, are unsuitable for rapid im- age segmentation in point-of-care applications due to their parameter-heavy,computationallycomplex,andslow-to-use <short author list for running head>: Preprint submitted to Elsevier Page 4 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> Figure 3: Selection of 224 ×224 pixel patches and their corresponding mask from DRIVE dataset formed utilising the second approach nature. To address this issue, the paper proposes UNeXt, designed effectively with an early convolutional stage and an MLP stage in the latent stage. The paper proposes a tokenised MLP block that efficiently tokenises and projects the convolutional features and uses MLPs to model the representation. To further boost the performance, the paper proposes shifting the channels of the inputs while feeding into MLPs to focus on learning local dependencies. Using tokenised MLPs in the latent space reduces the number of parametersandcomputationalcomplexitywhileresultingin a better representation to help segmentation. The network also consists of skip connections between various levels of encoder and decoder. SwinUNet [28] is a pure transformer-based U-shaped ar- chitecture for medical image segmentation. The encoder, bottleneck, and decoder are all built based on the Swin Transformer block. The input medical images are split into non-overlapping image patches, tokenised and fed into the Transformer-basedU-shapedEncoder-Decoderarchitecture with skip connections for local-global semantic feature learning. The paper shows that Swin-UNet outperforms other fully convolutional or combined transformers and convolution in multi-organ and cardiac segmentation tasks. Table 1 presents an overview of the number of parame- ters for the utilised architectures to provide insight into the complexity of different models. 2.4. Evaluation Metrics Multiple metrics must be utilised to evaluate a model’s segmentation performance. The accuracy measures used inthisstudyincludetheareaunderthereceiveroperatingchar- acteristics curve (AUC-ROC), the area under the precision- recall curve (AUC-PR), F1-Score, accuracy (Acc), Sensi- tivity (Sen), Specificity (Spe), which could be calculated as follows: 𝑃𝑟𝑒 =𝑡𝑝 𝑡𝑝+𝑓𝑝(1) 𝐴𝑐𝑐 =𝑡𝑝+𝑡𝑛 𝑡𝑝+𝑡𝑛+𝑓𝑝+𝑓𝑛(2) 𝐹1 −𝑆𝑐𝑜𝑟𝑒 =2 ×𝑡𝑝 2 ×𝑡𝑝+𝑓𝑝+𝑓𝑛(3) 𝑆𝑝𝑒 =𝑡𝑛 𝑓𝑝+𝑡𝑛(4) 𝑆𝑒𝑛 =𝑡𝑝 𝑡𝑝+𝑓𝑛(5) It should be noted that all metrics in this study are calculated using Scikit-learn library [34]. 2.4.1. Patch Extraction and Implementation details Given the implementation considerations of some mod- els, all models are trained with patches with the size of 224 ×224 extracted from original images for consistency. This paper, however, has investigated two approaches for patch extraction. <short author list for running head>: Preprint submitted to Elsevier Page 5 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> The primary approach for extracting the patches from images in this study includes extracting overlapped random patcheswiththesizeof224 ×224providedthateachpatch should also be overlapped with the field of view (FOV). Performing a trial and error procedure, it was found that theefficientnumberofextractedpatchesfortrainingmodels equals1000patchesfortheDRIVEandSTAREdatasetsand 3000 patches for the CHASE-DB1. Further augmentation of data did not lead to an increase in the models’ accuracy. Specific details on the HRF dataset augmentation are pre- sentedinsection3.4.Figure2presentssamplesofextracted 224 ×224 patches and their corresponding masks. The second approach which is examined exclusively on theDRIVEdataset,includesextractingrandompatcheswith thesizeof56 ×56withtheattitudeofreducingthenumber ofpixelsoutsideofthefieldofview(FOV)aswellassaving the border of FOV in random patches. Extracted 56 ×56 patches are then randomly attached in 4 ×4 settings to form 224 ×224 pixel images for the training phase of models. However, it is generally acknowledged that this methodintroducesartificialedgestotheinputimages,which could force the model to avoid edges and potentially reduce model performance by forming misleading images with a largevarietyofHSVandGammavaluesindifferentforming patches. Consequently, image enhancements are performed subsequent to the patch extraction to overcome this draw- back. Figure 3 presents samples of formed 224 ×224 patches and their corresponding masks utilising the second approach. For all experiments, a floating learning rate that de- creases by validation loss is utilised in order to reach the highestaccuracyandglobalminima.Initiallearningratesare set to 0.01 for all networks with a scheduling factor of 0.1, decreasingthelearningrateevery5epochsifvalidationloss isnotdecreased.Performingatrialanderrorprocedure,this study utilises the 𝐴𝑑𝑎𝑚optimiser with weight decay equal to0.0001forUnet,AttUNet,UNeXt,UCTransNet,ResUnet, and Unet++. TransUnet and SwinUnet are, on the other hand, trained by utilising the SGD optimiser with weight decay equal to 0.0001 and momentum coefficient equal to 0.9.ThelossfunctionisformulatedasacompositeofCross EntropyLossandDiceLosswithLogits,bothassignedequal weights. Finally, the early stop is configured for all models conditionedonthevalidationloss,notshowingimprovement for 20 consecutive epochs. The number of epochs that each modelistrainedwithispresentedunderthe 𝑆𝑡𝑜𝑝headingin the training results of the models considered as an indicator of the model computational demand and efficiency. To train and implement models, an Nvidia 3090 GPU with 24GB of RAM and the PyTorch framework [35] are utilised. A batch size of 10 was employed for the 𝐷𝑅𝐼𝑉 𝐸 dataset, and a batch size of 16 was considered for the 𝐶𝐻𝐴𝑆𝐸 −𝐷𝐵1,𝑆𝑇 𝐴𝑅𝐸, and𝐻𝑅𝐹datasets.Table 2 Accuracy measures obtained by different models trained on unprocessed 𝐷𝑅𝐼𝑉 𝐸 dataset ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9769 0.9060 0.8073 0.9548 0.7436 0.9856 92 AttUNet 0.9771 0.9051 0.8059 0.9545 0.7413 0.9856 78 ResUnet 0.9745 0.8999 0.8035 0.9538 0.7419 0.9847 101 UCTransNet 0.9757 0.9012 0.8017 0.9536 0.7360 0.9854 65 Unet++ 0.9742 0.8986 0.8054 0.9537 0.7526 0.9830 90 TransUnet -ViT-B160.9310 0.7891 0.6943 0.9331 0.5963 0.9823 107 TransUnet -R50-ViT-B160.9658 0.8861 0.7989 0.9523 0.7448 0.9825 138 Unext 0.9710 0.8883 0.7903 0.9511 0.7245 0.9841 73 SwinUnet 0.9706 0.8888 0.7945 0.9519 0.7308 0.9841 98 3. Results This section presents the performance of all models over different datasets. For this purpose, each dataset is presentedanddiscussedseparately.Anoveralldiscussionof the obtained results and comparisons with other studies is then presented in the next section. 3.1. DRIVE Dataset As previously discussed in section 2.4.1, all models are trained on the 𝐷𝑅𝐼𝑉 𝐸 dataset utilising two distinct data augmentation and patch extraction methods. All models are initiallytrainedonthe224 ×224patchesextractedrandomly from raw images, utilising the first patching approach to compareandevaluatetheeffectivenessofthepreprocessing and data augmentation approaches. The results obtained from this configuration are presented in Table 2. In the next step, models are trained on the patches extracted before the image preprocessing, the results of which are presented in Table 3. The bold values in the tables delineate the top- performing architecture. Comparing the results, it could be concluded that the performing image preprocessing has increased the performance of all models except the UNeXt. From the model performance point of view, it could be observed that the UNet has performed better than other models in three out of six utilised accuracy metrics when trainedonpatchesextractedfrombothrawandpreprocessed images with 𝐴𝑈𝐶 −𝑅𝑜𝑐=0.9769, 𝐴𝑈𝐶 −𝑃𝑅=0.9060, 𝐹1 −𝑆𝑐𝑜𝑟𝑒=0.8073, and 𝐴𝑐𝑐=0.9548 for training on raw imagesand 𝐴𝑈𝐶 −𝑅𝑜𝑐=0.9788, 𝐴𝑈𝐶 −𝑃𝑅=0.9105,and 𝐴𝑐𝑐=0.9559 when trained on preprocessed images. How- ever, it should be highlighted that the AttUNet has obtained thehighest 𝐴𝑈𝐶 −𝑅𝑂𝐶and𝑆𝑝𝑒valuesequalto0.9771and 0.9856fortrainingonunprocessedimages.Figure4presents a qualitative comparison of the retinal vessel segmentation resultsobtainedfromdifferentmodelstrainedonthe DRIVE dataset. It could be observed that generally, UNet and At- tUNet have performed better representations of the vessels, especially for smaller and thinner vessels. In the second step, to compare the effect of different patch extraction approaches described before, all models are trained on both raw and preprocessed images utilising the second patch extraction approach described in Section 2.4.1.Theresultsobtainedthroughthissteparepresentedin <short author list for running head>: Preprint submitted to Elsevier Page 6 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> Figure 4: Qualitative comparison of the retinal vessel segmentation results on DRIVE dataset obtained utilising (a) original image, (b) preprocessed image, (c) GroundTruth, (d) UNet, (e) AttUNet, (f) ResUNet, (g)UCTransNet,(h) Unet++, (i) TransUNet-Vit, (j)TransUNet-R50, (k) UNeXt, and (l) SwinTransformer Table 3 Accuracy measures obtained by different models trained on preprocessed 𝐷𝑅𝐼𝑉 𝐸 dataset ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9788 0.9105 0.8124 0.9559 0.7501 0.9859 72 AttUNet 0.9772 0.9062 0.8087 0.9551 0.7464 0.9855 75 ResUnet 0.9779 0.9081 0.8080 0.9551 0.7416 0.9863 93 UCTransNet 0.9775 0.9069 0.8138 0.9555 0.7633 0.9836 88 Unet++ 0.9787 0.9103 0.8134 0.9559 0.7558 0.9850 93 TransUnet -ViT-B160.9332 0.7939 0.6884 0.9331 0.5802 0.9846 93 TransUnet -R50-ViT-B160.9729 0.8967 0.7969 0.9528 0.7273 0.9857 60 Unext 0.9691 0.8852 0.8013 0.9501 0.7901 0.9734 103 SwinUnet 0.9722 0.8944 0.8008 0.9531 0.7407 0.9840 118 Table4fortrainingonrawimagesandTable5fortrainingon preprocessed images. Results indicate that the AttUNet has performedbetterwhiletrainedonrawimagesobtainingbest valuesinthreeoutofsixmetricswith 𝐴𝑈𝐶 −𝑅𝑂𝐶=0.9756, 𝐴𝑈𝐶 −𝑃𝑅=0.9029,andAcc=0.9540,whichisstillslightly lower than those obtained through training models on ran- domly extracted 224 ×224 patches (first approach). On the other hand, UNet++ has performed best in five out of six utilised accuracy metrics when trained on patches extracted frombothpreprocessedimagesaugmentedthroughthesec- ond patch extraction approach with 𝐴𝑈𝐶 −𝑅𝑂𝐶=0.9789, 𝐴𝑈𝐶 −𝑃𝑅=0.9109, 𝐹1−𝑆𝑐𝑜𝑟𝑒=0.8170,and 𝐴𝑐𝑐=0.9564 which indicates a slight improvement compared to the first augmentation approach. 3.2. STARE Dataset Unlike the DRIVE dataset, STARE and other datasets considered are trained through the utilisation of the first patch extraction approach with overlapping randomly ex- tracted224 ×224patches.Thetestingresultsobtainedfrom different models trained on 𝑆𝑇 𝐴𝑅𝐸 dataset are presented in Table 6 for training on raw images and Table 7 for training on preprocessed images. Results indicate that the AttUNet has performed best while trained on raw images,Table 4 Accuracy measures obtained by different models trained on unprocessed 𝐷𝑅𝐼𝑉 𝐸 dataset through 56 ×56 pixel patch extraction technique ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9755 0.8998 0.8083 0.9540 0.7621 0.9819 90 AttUNet 0.9756 0.9029 0.8026 0.9540 0.7338 0.9862 78 ResUnet 0.9642 0.8785 0.7914 0.9504 0.7390 0.9812 180 UCTransNet 0.9747 0.8989 0.8009 0.9534 0.7363 0.9850 75 Unet++ 0.9745 0.9000 0.8095 0.9542 0.7639 0.9820 118 TransUnet -ViT-B160.9404 0.8056 0.7063 0.9349 0.6143 0.9817 76 TransUnet -R50-ViT-B160.9746 0.8973 0.8047 0.9534 0.7536 0.9825 62 Unext 0.9734 0.8949 0.7923 0.9520 0.7186 0.9861 82 SwinUnet 0.9713 0.8925 0.7956 0.9523 0.7297 0.9847 158 Table 5 Accuracy measures obtained by different models trained on preprocessed 𝐷𝑅𝐼𝑉 𝐸 dataset through 56 ×56 pixel patch extraction technique ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9780 0.9087 0.8006 0.9553 0.7415 0.9865 92 AttUNet 0.9776 0.9076 0.8058 0.9549 0.7347 0.9870 76 ResUnet 0.9767 0.9050 0.8040 0.9544 0.7348 0.9864 78 UCTransNet 0.9786 0.9095 0.8132 0.9558 0.7556 0.9850 69 Unet++ 0.9789 0.9109 0.8170 0.9564 0.7655 0.9842 63 TransUnet -ViT-B160.9259 0.7804 0.6635 0.9301 0.5413 0.9868 87 TransUnet -R50-ViT-B160.9721 0.8934 0.8014 0.9528 0.7483 0.9826 49 Unext 0.9699 0.8847 0.7924 0.9507 0.7393 0.9815 66 SwinUnet 0.9714 0.8922 0.7925 0.9520 0.7195 0.9859 118 obtainingbestvaluesinfiveoutofsixmetricswith 𝐴𝑈𝐶 − 𝑅𝑂𝐶=0.9901, 𝐴𝑈𝐶 −𝑃𝑅=0.9330, 𝐹1 −𝑆𝑐𝑜𝑟𝑒=0.8448, 𝐴𝑐𝑐=0.924, and 𝑆𝑝𝑒=0.9894, which is slightly higher than those obtained through training models on preprocessed imageswith 𝐴𝑈𝐶 −𝑅𝑂𝐶=0.9878, 𝐴𝑈𝐶 −𝑃𝑅=0.9237,in- dicatingtheineffectivenessofthepreprocessingperformed. Additionally, the naive UNet has performed comparably with the AttUNet when trained on patches extracted from <short author list for running head>: Preprint submitted to Elsevier Page 7 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> Figure 5: Qualitative comparison of the retinal vessel segmentation results on STARE dataset obtained utilising (a) original image, (b) preprocessed image, (c) GroundTruth, (d) UNet, (e) AttUNet, (f) ResUNet, (g)UCTransNet,(h) Unet++, (i) TransUNet-Vit, (j)TransUNet-R50, (k) UNeXt, and (l) SwinTransformer Table 6 Accuracy measures obtained by different models trained on unprocessed 𝑆𝑇 𝐴𝑅𝐸 dataset ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9885 0.9271 0.8427 0.9716 0.8189 0.9872 110 AttUNet 0.9901 0.9330 0.8448 0.9724 0.8068 0.9894 93 ResUnet 0.9867 0.9178 0.8322 0.9697 0.8069 0.9864 107 UCTransNet 0.9887 0.9277 0.8359 0.9713 0.7867 0.9902 81 Unet++ 0.9863 0.9235 0.8412 0.9712 0.8198 0.9867 130 TransUnet -ViT-B160.9579 0.8199 0.7295 0.9551 0.6501 0.9864 110 TransUnet -R50-ViT-B160.9849 0.9222 0.8360 0.9707 0.8016 0.9881 85 Unext 0.9853 0.9143 0.8236 0.9687 0.7867 0.9873 75 SwinUnet 0.9895 0.9276 0.8389 0.9713 0.8018 0.9887 144 Table 7 Accuracy measures obtained by different models trained on preprocessed 𝑆𝑇 𝐴𝑅𝐸 dataset ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9877 0.9230 0.8347 0.9706 0.7973 0.9884 84 AttUNet 0.9878 0.9237 0.8305 0.9705 0.7766 0.9904 79 ResUnet 0.9818 0.9010 .0.8120 0.9671 0.7640 0.9879 82 UCTransNet 0.9856 0.9161 0.8257 0.9695 0.7749 0.9895 81 Unet++ 0.9870 0.9216 0.8365 0.9703 0.8176 0.9859 97 TransUnet -ViT-B160.9566 0.8219 0.7304 0.9552 0.6520 0.9864 101 TransUnet -R50-ViT-B160.9839 0.9080 0.8229 0.9684 0.7878 0.9870 60 Unext 0.9798 0.8921 0.7997 0.9653 0.7444 0.9879 76 SwinUnet 0.9842 0.9118 0.8249 0.9689 0.7857 0.9878 85 preprocessed images. Figure 5 presents a qualitative com- parison of the retinal vessel segmentation results obtained from different models trained on the STAREdataset. 3.3. CHASE-DB1 Dataset Thetestingresultsobtainedfromdifferentmodelstrained on𝐶𝐻𝐴𝑆𝐸 −𝐷𝐵1dataset are presented in Table 8 for training on raw images and Table 9 for training on preprocessed images. Results indicate that the UCTransNet andTRansUnet-R50 haveperformedcomparablybetterthan other models while trained on raw images, each obtain- ing best values in three out of six metrics with 𝐴𝑈𝐶 −Table 8 Accuracy measures obtained by different models trained on unprocessed 𝐶𝐻𝐴𝑆𝐸 −𝐷𝐵1dataset ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9828 0.8838 0.7972 0.9642 0.7713 0.9835 108 AttUNet 0.9825 0.8826 0.7974 0.9643 0.7694 0.9839 160 ResUnet 0.9761 0.8628 0.7834 0.9617 0.7575 0.9823 182 UCTransNet 0.9832 0.8863 0.7997 0.9647 0.7722 0.9840 130 Unet++ 0.9815 0.8797 0.7930 0.9636 0.7634 0.9837 83 TransUnet -ViT-B160.9747 0.8514 0.7809 0.9614 0.7530 0.9824 115 TransUnet -R50-ViT-B160.9781 0.8750 0.8258 0.9649 0.7979 0.9817 98 Unext 0.9814 0.8769 0.7941 0.9633 0.7755 0.9821 129 SwinUnet 0.9832 0.8822 0.8023 0.9644 0.7910 0.9818 142 Table 9 Accuracy measures obtained by different models trained on preprocessed 𝐶𝐻𝐴𝑆𝐸 −𝐷𝐵1dataset ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9806 0.8799 0.7927 0.9635 0.7634 0.9836 73 AttUNet 0.9805 0.8797 0.7917 0.9634 0.7603 0.9838 60 ResUnet 0.9746 0.8581 0.7639 0.9602 0.7044 0.9859 154 UCTransNet 0.9799 0.8769 0.7902 0.9630 0.7622 0.9832 57 Unet++ 0.9791 0.8759 0.7884 0.9629 0.7571 0.9836 83 TransUnet -ViT-B160.9738 0.8476 0.7671 0.9590 0.7385 0.9812 58 TransUnet -R50-ViT-B160.9808 0.8763 0.7925 0.9637 0.7596 0.9842 40 Unext 0.9767 0.8629 0.7734 0.9608 0.7323 0.9838 84 SwinUnet 0.9807 0.8720 0.7935 0.9629 0.7811 0.9811 80 𝑅𝑂𝐶=0.9832, 𝐴𝑈𝐶 −𝑃𝑅=0.8863,and 𝐴𝑐𝑐=0.9840.How- ever, similar to the previous dataset, it is still slightly higher than those obtained through training models on preprocessed images where the highest accuracies obtained by theTRansUnet-R50 equal to 𝐴𝑈𝐶 −𝑅𝑂𝐶=0.9808, 𝐴𝑈𝐶 −𝑃𝑅=0.8763, and 𝐴𝑐𝑐=0.9637. Figure 6 presents a qualitative comparison of the retinal vessel segmenta- tion results obtained from different models trained on the CHASE-DB1 dataset. It is clear that almost all models have performedequallywellinsegmentingmainvesselsyetshow poor performance when segmenting capillary arteries. <short author list for running head>: Preprint submitted to Elsevier Page 8 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> Figure 6: Qualitative comparison of the retinal vessel segmentation results on 𝐶𝐻𝐴𝑆𝐸 −𝐷𝐵1dataset obtained utilising (a) original image, (b) preprocessed image, (c) GroundTruth, (d) UNet, (e) AttUNet, (f) ResUNet, (g)UCTransNet,(h) Unet++, (i) TransUNet-Vit, (j)TransUNet-R50, (k) UNeXt, and (l) SwinTransformer Figure 7: Qualitative comparison of the retinal vessel segmentation results on 𝐻𝑅𝐹dataset obtained utilising (a) original image, (b) preprocessed image, (c) GroundTruth, (d) UNet, (e) AttUNet, (f) ResUNet, (g)UCTransNet,(h) Unet++, (i) TransUNet-Vit, (j)TransUNet-R50, (k) UNeXt, and (l) SwinTransformer 3.4. HRF Dataset Following the patch extraction ratio used for earlier datasets would require extracting an unfeasible number of 35,000 patches for the HRF dataset, which has noticeably higher image resolution than prior datasets in this study. Given the computational and memory constraints and the objectives of our investigation, we chose a more effec- tive strategy. Early attempts to train models from scratch on this dataset resulted in poor performance (results not included). Consequently, Using pre-learned weights from models trained on raw CHASE-DB1 images, we fine-tuned the models on a subset of 3,000 random patches extracted fromtheHRFdataset.Theresultsweregreatlyenhancedby utilising this concept of transfer learning, which decreased computational costs and improved all metrics. Thetestingresultsobtainedfromdifferentmodelstrained on the 𝐻𝑅𝐹dataset are presented in Table 10 for training on raw images and Table 11 for training on preprocessed images. Results indicate that the AttUNet has performed best while trained on raw images, obtaining best values in four out of six metrics with 𝐴𝑈𝐶 −𝑅𝑂𝐶=0.9825, 𝐴𝑈𝐶 −𝑃𝑅=0.8937, 𝐴𝑐𝑐=0.9540,and 𝑆𝑝𝑒=0.9885which is generally higher than those obtained through training models preprocessed images where 𝑇 𝑟𝑎𝑛𝑠𝑈𝑛𝑒𝑡 −𝑅50has outputted better results compared to other models withTable 10 Accuracy measures obtained by different models trained on unprocessed 𝐻𝑅𝐹dataset ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9825 0.8902 0.7966 0.9642 0.7513 0.9861 40 AttUNet 0.9825 0.8937 0.7939 0.9646 0.7317 0.9885 41 ResUnet 0.9794 0.8846 0.7862 0.9634 0.7204 0.9884 50 UCTransNet 0.9820 0.8882 0.7944 0.9639 0.7464 0.9863 55 Unet++ 0.9820 0.8909 0.7965 0.9644 0.7468 0.9868 58 TransUnet -ViT-B160.9727 0.8515 0.7824 0.9614 0.7439 0.9838 35 TransUnet -R50-ViT-B160.9805 0.8828 0.7914 0.9629 0.7539 0.9844 14 Unext 0.9812 0.8879 0.7879 0.9634 0.7279 0.9877 58 SwinUnet 0.9821 0.8881 0.7951 0.9639 0.7499 0.9860 30 𝐴𝑈𝐶 −𝑅𝑂𝐶=0.9682, and 𝑆𝑒𝑛=0.72. This indicates a considerable drop in the accuracy of models trained on preprocessedimagesthroughgammacorrectionandrandom HSVadjustment.Figure7presentsaqualitativecomparison of the retinal vessel segmentation results obtained from different models trained on the HRFdataset. 4. Discussion This section provides a discussion of the results ob- tained through this study. Regarding the efficiency of the <short author list for running head>: Preprint submitted to Elsevier Page 9 of 12 <A Comparative Study of the Retinal Vessel Segmentation...> Table 11 Accuracy measures obtained by different models trained on preprocessed 𝐻𝑅𝐹dataset ArchitectureAUC -ROCAUC -PRF1 -ScoreAcc Sen Spe Stop Unet 0.9670 0.8544 0.7623 0.9601 0.6877 0.9880 51 AttUNet 0.9640 0.8432 0.7555 0.9589 0.6830 0.9872 42 ResUnet 0.9570 0.8246 0.7268 0.9558 0.6328 0.9889 73 UCTransNet 0.9669 0.8523 0.7648 0.9602 0.6940 0.9876 36 Unet++ 0.9675 0.8554 0.7620 0.9601 0.6874 0.9880 69 TransUnet -ViT-B160.9609 0.8334 0.7548 0.9579 0.6949 0.9849 14 TransUnet -R50-ViT-B160.9682 0.8550 0.7707 0.9600 0.7200 0.9847 16 UNeXt 0.9656 0.8490 0.7577 0.9592 0.6855 0.9873 43 SwinUNet 0.9297 0.7186 0.6545 0.9440 0.5732 0.9820 88 preprocessingpipelineadoptedinthisstudy,whichincludes Gamma corrections and random HSV adjustment, it could be concluded that its positive impact is limited to the low- resolution images of the DRIVEdataset where it has led to an improvement of the segmentation accuracy obtained by all models as well as faster convergence of the training procedureandrequirementforlesstrainingepoch.However, ithadanadverseeffectontheperformanceandnegligibleto no effect on model convergence of models trained on other datasets with higher resolutions. From the model performance point of view, it could be summarised that the convolutional architectures, such as naiveUNetandUNet++,haveperformedbestwhiletrained onlow-resolutionimagesofthe DRIVEdataset.Forallother datasets, transformers such as AttUNet, TransUNet and SwinUNet have performed better than convolutional-based models. Overall, the TransUNet-ViT-B16 performed worst amongallothermodels,whileAttUNetandTransUNet-R50 performed better than the rest while trained on all datasets except the DRIVE. Table 12 compares the results of this study with those obtainedbydifferentresearchers.Itcouldbeconcludedthat whileweacknowledgethatourstudydidnotoutperformthe state-of-the-artstudiesintheremainingthreedatasets,itdid achieve slightly better results in one of them, the STARE dataset. However, the results we were able to acquire for the rest of the datasets are still comparable, demonstrating competitive performance in line with existing benchmarks in the field. 5. Conclusions In conclusion, our study delves into the complicated taskofretinalvesselsegmentation,employingninedifferent Convolutional-based and transformer-based architectures. Transformers,suchasAttUNet,TransUNet,andSwinUNet, perform exceptionally well on datasets with greater reso- lutions, while convolutional models, like naive UNet and UNet++,performbetterondatasetswithlowerresolutions. Preprocessing methods have differing effects; for example, random HSV adjustment and gamma correction work well forlow-resolutionphotosbutareineffectivewhenusedwith higher-resolution images. Our findings indicate compara- ble performance with the state-of-the-art benchmarking ofthe field and minor improvements in the STARE dataset compared to other research. The conclusions highlight the importance and effectiveness of image-enhancing methods on improving model accuracy and reducing training time, especially for low-resolution datasets. CRediT authorship contribution statement Morteza Tavakol Sadrabadi: <Credit authorship de- tails>. Hamed Agahi: .