# Training Free U-Net for Retinal Vessel Segmentation

Muhammad Junaid, Laurent Moalic, Mokhtar Essaid, Lhassane Idoumghar

► To cite this version:

Muhammad Junaid, Laurent Moalic, Mokhtar Essaid, Lhassane Idoumghar. Training Free U-Net for Retinal Vessel Segmentation. Artificial Evolution 2024, Springer; Association Evolution Artificielle, Oct 2024, Bordeaux, France. **hal-04999688v2**

HAL Id: hal-04999688

https://hal.science/hal-04999688v2

Submitted on 30 Apr 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Training Free U-Net for Retinal Vessel
Segmentation

Muhammad Junaid Ali¹ *†, Laurent Moalic² †, Mokhtar Essaid³ †, and Lhassane Idoumghar⁴ †

¹muhammad-junaid.ali@uha.fr ²laurent.moalic@uha.fr ³mokhtar.essaid@uha.fr

⁴lhassane.idoumghar@uha.fr

† Université de Haute-Alsace, IRIMAS UR 7499, F-68093 Mulhouse, France

**Abstract.** Medical Image Segmentation is a crucial process in medical image analysis. It is the process of partitioning an image into several regions to segment out tumours/infectious regions or organs from medical scans. It assists in accurate diagnosis, disease progression, and treatment planning. Studies have mainly used encoder-decoder-based U-Net architectures for automatic medical image segmentation and proposed different variants of U-Net-based architectures over the years. Although these approaches are quite effective and solve numerous segmentation tasks, the design process remains difficult, time-consuming, and requires expert knowledge. To overcome this problem, researchers have proposed Neural Architecture Search (NAS), which automatically searches for a suitable architecture. NAS has been widely used to design encoder-decoder-based architectures for segmentation problems. However, the search time remains a challenge. Following this context, Zero Cost (ZC) proxies are effective performance estimation strategies used to reduce the searching time of NAS algorithms. This study proposes a training-free approach for designing U-Net architectures for retinal vessel segmentation. A Genetic Algorithm (GA) is used to search for an architecture automatically. The experimental results demonstrated that the proposed approach yields better performance when compared to existing NAS studies and hand-crafted U-Net architectures.

**Keywords:** Medical Image Segmentation · AutoML · U-NET · Neural Architecture Search · Retinal Vessel Segmentation

# 1 Introduction

Medical image segmentation is among the essential tasks in medical image analysis for accurate diagnosis of diseases. Image segmentation is a technique of image processing used to partition an image into multiple parts or regions. In the case of medical imaging, it segments out an organ or a tumorous/infected region. The latter is relatively small. Therefore, classical machine learning approaches do not guarantee a satisfactory performance on these tasks [12]. Encoder-decoder-based deep learning architectures have recently been proposed for medical image
segmentation tasks and achieved state-of-the-art results [1]. One of the famous architectures is U-Net, whose multiple variants have been proposed [13] [14]. Although deep learning architectures are successful when tackling medical imaging problems, designing them is challenging and time-consuming due to their complex algorithmic structure (multiple building blocks: convolutions, pooling, attention layer types [14]).

To automate the design process, Neural Architecture Search (NAS) is proposed to automatically search for an architecture for a given task [15]. NAS is shown to be successful for numerous medical imaging problems achieving a state-of-the-art performance regarding medical image segmentation [17], denoising [18] and classification [20]. Although NAS is efficient in terms of performance but its search time remains a difficult issue: the performance estimation of each candidate architecture is time consuming. As an attempt to reduce the computational time, numerous performance estimation strategies have been used, such as one-shot and surrogate models [21]. On the one hand, surrogate approaches are efficient but they require huge training data. On the other hand, one-shot approaches are unable to explore large search spaces [15].

Recently, studies have proposed Zero Cost (ZC) proxies, which use only a small subset of training data for evaluation. Studies have also named these proxies training-free approaches as they do not train the architecture on the whole dataset [16]. These approaches are found to be effective in different medical image analysis tasks [22] and adopting them for an automatic designing of U-Net architecture for medical image segmentation can be promising. Segmentation of retinal vessels from iris scans is important task as it helps in early diagnosis of many diseases e.g diabetic retinopathy, hypertension and age-related muscular degeneration and manifest changes in retinal blood vessels. Accurate and early detection helps in the treatment of disease. Recent studies have shown that evolutionary algorithms help design deep learning architectures for retinal vessel segmentation [8]. In this study, we proposed an efficient approach for retinal vessel segmentation using ZC proxies. The overall contribution of this study are as follows:

- Adoption of ZC proxy to reduce the search time for medical image segmentation task.

- Experiments on retinal vessel dataset to demonstrate the effectiveness of proposed approach as compared to vanilla U-Net architecture.

## 2 Related Work

Retinal vessel segmentation is an important task as it helps diagnose multiple diseases during the early stages, such as diabetes and high blood pressure. Therefore, AI-based systems assist doctors in analyzing retinal scans more effectively and improve their effectiveness. Researchers have proposed multiple machine learning and deep learning approaches for retinal vessel segmentation. The advantage of deep learning over machine learning is that it doesn't need manual
feature engineering. It can automatically extract complex features from the data
and create segmentation maps.

The main problem with deep learning architectures is the manual designing
task. Multiple NAS approaches have been proposed for retinal vessel segmen-
tation to overcome this problem. Fan et al. proposed a modified evolutionary
algorithm to automatically build encoder-decoder-based architecture for retinal
vessel segmentation. They have performed experiments on DRIVE, STARE, and
ChaseDB datasets and achieved satisfying performance with limited computa-
tional resources [19]. Furthermore, Zeki et al. proposed an evolutionary archi-
tecture optimization for retinal vessel segmentation. An opposition-based dif-
ferential evolution inspired by opposition-based learning yielded better results
than classical Differential Evolution (DE). Differential evolution is a method
used to optimize problems by iteratively improving a candidate solution. By
incorporating opposition-based learning, ODE can explore the solution space
more effectively, leading to faster and potentially better results. Essentially, it
helps the algorithm to avoid getting stuck in local optima and find the global
optimum more efficiently. Opposition-based learning (OBL) is a concept used in
optimization algorithms to improve their efficiency. The idea is to consider not
only a solution but also its opposite. For example, if you have a guess for the best
solution, you also look at the opposite guess. This way, you can explore more
possibilities and potentially find better solutions faster. Experiments on multiple
datasets show that their proposed MedUNAS outperforms existing architectures
like U-Net, using fewer parameters and less search time [23].

Moreover, Grammatical Evolution (GE) approaches have been proposed for
retinal vessel segmentation. In [8], the authors proposed to use grammatical
evolution for designing U-Net architecture, in which they formulate grammar to
evolve the U-Net architecture and its hyperparameters. They have used spatial
attention U-Net, a modified U-Net architecture that incorporates spatial atten-
tion to focus more on the relevant spatial features of the input data. This is
achieved by the attention mechanism, which assigns higher weights to the im-
portant pixels in the feature vectors, yielding better performance accuracy. Fur-
thermore, to prevent overfitting, Drop-Block regularization, a variant of dropout,
further enhances the model's performance by randomly dropping contiguous re-
gions of the feature map during training, which forces the network to learn more
robust features and prevents overfitting. The proposed approach finds the hy-
perparameters (kernel size, pooling layer, block size, padding, layer size) and
the number of parameters. Their proposed approach outperforms existing seg-
mentation approaches on the DRIVE eye retina dataset. Similar studies are also
proposed using GA for retinal vessel segmentation [6], [7] and [9].

The main shortcoming of these approaches is the searching time. To over-
come this problem, numerous performance estimation strategies have been in-
corporated, such as one-shot NAS and surrogate approaches. These approaches
reduces the searching time but they exhibit some drawbacks. For example, in the
case of surrogate approaches which uses machine learning models to predict the
fitness values during the evolution stage, their training require a huge amount
of data. Similarly, One-Shot NAS approaches first train a Super-Net and then sample small sub-networks. For evaluation of these sub-networks, weight-sharing mechanism is used to reduce the search time. However, the latter may not correctly reflects the actual performance and limits the exploration converging to a local optima. To overcome these problem, ZC proxies have been proposed in literature using a small batch of data to estimate the performance of a given architecture. [5].

Abdelfattah et al. [3] proposed using existing proxies to estimate scoring in a DL architecture including SynapticFlow (SynFlow) and SNIP. Recent studies have shown that SynFlow reduces the searching time [24]. Inspired by this study, our proposal incorporates ZC proxies into the NAS process for retinal vessel segmentation.

## 3 Proposed Methodology

This section provides the overall methodology of the proposed approach. An overview is given in Figure 1. The proposed framework mainly consists of three steps (i) population initialization (ii) crossover and mutation operations and (iii) fitness evaluation. In the first step, the population is initialized randomly and the fitness of the initialized population is evaluated on ZC proxies. In the second step, new offspring are generated from the initialized population and in the third step, the fitness is evaluated. Steps 2 and 3 are repeated until the stopping criteria is reached.

<figure><img src="image_2.png" /><figcaption>Fig. 1. The proposed Methodology diagram</figcaption></figure>

The search space is defined for representing U-Net-shaped architectures. Individual neural networks are represented using a string, as shown in Figure 4. Our focus is on an encoder-decoder-shaped architecture, which essentially
corresponds to a U-Net-shaped architecture. U-Net is a type of CNN-based architecture primarily designed for medical image segmentation. It comprises two components: an encoder (contracting path) that captures contextual information and a decoder (expanding path) that reconstructs the compressed information by up-sampling and concatenating with the encoder layers.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Proposed U-shaped encoder decoder based architecture. The encoder and decoder block consists of multiple blocks also called the depth. The encoder block consists of convolution and batch normalization layers and the decoder block consists of up-sampling layers with convolutions and attention layers.</figcaption></figure>

## 3.1 Search Space

A U-Net architecture consists of multiple blocks consisting of convolutions, normalization and pooling layers stacked one after another. The repetition of these blocks in the encoder and decoder layers is called the depth. Finding the optimal depth for a given task is a difficult task. Moreover, the type of convolutions, their kernel size, and the activation function also affect the performance of a given architecture. Our search space consists of depth, kernel size, convolution type, and attention layer as decision variables. The type of convolutions and activation functions used in the search space are shown in Figure 3.

Different types of convolutions are included in the search space: dilated convolutions, depth-wise convolutions, ghost convolutions, and squeeze and excitation convolutions. These variants of convolution layers are proposed in different studies and achieved state-of-the-art results on different classification tasks. In the encoding scheme, the type of convolutions and activation functions are represented by a discrete value between 0 and 4, the kernel size as 3 or 5, the depth is between 3 to 5. The decision to include an attention layer in the up-sampling path is represented by a binary variable, where 0 indicates the absence of the attention layer and 1 indicates its presence in the block.
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Search space utilized in this study consisting of a variety of convolutions and activation functions</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Genotype and phenotype representation of an individual</figcaption></figure>

## 3.2 Zero-Cost Proxies

Zero cost proxies are inspired by recent pruning at initialization literature [25]. First proposed by Abdelfateh et al. in [10], they are widely used in different NAS studies to reduce the search time. These proxies use a single batch of mini-data to estimate the performance of given architectures. In literature, numerous ZC proxies have been proposed. For example, Synflow, SNIP, NTK, and a number of linear regions are some of the famous ZC proxies. Among them, the NTK is quite effective in evaluating an architecture's performance [11]. NTK describes the evolution of neural networks during the training by gradient descent. Recent studies have shown that NTK predictions are highly correlated with actual performance metrics i.e accuracy [11]. We have used NTK as a performance estimation strategy to evaluate the performance of individuals in GA, aiming to reduce the search time. The fitness of an individual neural network is evaluated using the ZC proxy NTK. To evaluate NTK, the neural network and a small mini-batch of training data are passed, which returns the fitness score.

[26].

## 3.3 Experimental Settings

Python is used for the implementation of the proposed framework. For implementation of deep learning Pytroch library is used and mealpy is used for the implementation of GA. For training Adam optimizer is used with learning rate of 1e-4 and Reduce LR on plateau scheduler is used with patience of 5. Experiments have been performed on DRIVE dataset. DRIVE (Digital Retinal Images for Vessel Extraction) dataset contains images from 400 diabetic patients between 25
and 90 years old. Among these images 40 images are randomly selected. This dataset pictures are collected from diabetic patients in Netherlands [4]. For the settings of Genetic Algorithm we have performed multiple experiments to choose the best parameters which given in Table 1.

<table><caption>**Table 1.** Parameters of Genetic Algorithm used in this study</caption><thead><tr><td>Parameters</td><td>Value</td></tr></thead><tbody><tr><td>Population Size</td><td>30</td></tr><tr><td>Number of Generations</td><td>20</td></tr><tr><td>Crossover Probability</td><td>0.8</td></tr><tr><td>Mutation Probability</td><td>0.2</td></tr><tr><td>Selection Strategy</td><td>Roulette</td></tr><tr><td>Crossover Type</td><td>One-Point</td></tr><tr><td>Mutation Type</td><td>Flip</td></tr></tbody></table>

## 3.4 Experimental Results

We have evaluated the performance of the proposed approach on DRIVE retinal vessel dataset and used the train/test set distribution given in the dataset for fair comparison. The results of the proposed approach on test set are compared with manually designed U-Net. We have also compared the results of proposed approach with existing NAS studies proposed for retinal vessel segmentation. The results are given in Table 2. We have compared the proposal with multiple performance metrics including Accuracy, AUC (Area Under the Curve), Specificity, Recall, Precision, F1 Score, and Searching Time. The proposed approach keeps a balance between precision and recall as shown in F1 score which is better as compared to existing studies.

Compared to existing studies the proposed approach searches for an optimal architecture only within 20 minutes 90% less as compared to existing studies. This shows the effectiveness of utilizing ZC proxies in proposed approach in terms of time consumption. The visualization of segmented output by the best searched

<table><caption>**Table 2.** Experimental Results comparison of proposed approach with existing studies on DRIVE dataset</caption><thead><tr><td></td><td>Accuracy</td><td>AUC</td><td>Specificity</td><td>Recall</td><td>Precision</td><td>F1</td><td>Searching Time</td></tr></thead><tbody><tr><td>Manual U-Net [27]</td><td>0.9657</td><td>0.9706</td><td>0.9848</td><td>0.7692</td><td>0.8293</td><td>0.7956</td><td>-</td></tr><tr><td>Mahsa et al. [6]</td><td>0.966</td><td>0.962</td><td>0.988</td><td>0.737</td><td>0.853</td><td>0.757</td><td>-</td></tr><tr><td>H-NAS [7]</td><td>0.9546</td><td>0.9749</td><td>0.9809</td><td>-</td><td>0.8555</td><td>0.8129</td><td>3:36:59</td></tr><tr><td>Popat et. Al [9]</td><td>0.953</td><td>0.9751</td><td>0.985</td><td>-</td><td>0.8757</td><td>-</td><td>21:44:00</td></tr><tr><td>Proposed</td><td><strong>0.968</strong></td><td><strong>0.9804</strong></td><td>0.9852</td><td><strong>0.8331</strong></td><td>0.8364</td><td><strong>0.8034</strong></td><td><strong>0:20:00</strong></td></tr></tbody></table>

architecture is shown in Figure 5. The retinal scan is shown in Figure 5(a), the
ground truth is shown in Figure 5(b) and the predicted output is shown in Figure 5(c). The visual results shows that proposed approach successfully segments out the retinal vessel region from the retinal scan.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Segmentation output masks of proposed approach (a) Retinal scan (b) Ground Truth (c) Predicted Output</figcaption></figure>

The best search architecture by the proposed NAS approach is shown in Figure 6. The genotype and phenotype is shown.

<figure><img src="image_7.png" /><figcaption>**Fig. 6.** Best Searched Architecture by the proposed approach</figcaption></figure>

## 3.5 Discussion and Conclusion

In this study we have proposed an evolutionary approach for automatically designing U-shaped architecture for retinal vessel segmentation. GA is used as a search strategy to find the best performing architecture and ZC proxy named NTK is used as a performance estimation strategy to evaluate the fitness of candidate architectures. This would helps to quickly evaluate the architectures and reduce the search time of overall approach. We utilized the GA algorithm to navigate through a custom-designed search space, which included varying depths for the U-Net, different configurations for the encoder and decoder, and a range of convolutional operations and activation functions for each block. Additionally, we incorporated attention layers in the decoder cells, enhancing the model's ability to focus on critical regions and improve the accuracy of the segmentation output as used in prior approaches. This combination of NTK and GA allowed us to efficiently explore and exploit the search space, leading to the discovery
of optimized U-Net configurations tailored specifically for retinal vessel segmen-
tation. The reduced search time and improved performance metrics underscore
the potential of our approach to facilitate the development of specialized neural
networks for various medical imaging tasks.

Comparison of the proposed approach with existing NAS studies and manual
designed U-Net shows that the proposed approach have performed well in terms
of different performance measures and less searching time compared to existing
NAS studies.In the near future, the plan is to use this approach for other seg-
mentation tasks such as nuclei instance segmentation and extend the proposed
approach for multi-objective optimization problem to search for both efficient
and lightweight architectures.

4 Ackowledgement

This work was funded by ArtIC project "Artificial Intelligence for Care" (grant ANR-20-THIA-0006-01) and co-funded by IRIMAS Institute / Universit'e de Haute Alsace. The authors would like to thank the Mesocentre of Strasbourg for providing access to the GPU cluster as well as the providers of all datasets used in this paper.

References

1. Ali, M. J., Akram, M. T., Saleem, H., Raza, B., and Shahid, A. R. (2021). Glioma segmentation using ensemble of 2D/3D U-Nets and survival prediction using multiple features fusion. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part II 6 (pp. 189-199). Springer International Publishing.

2. Mookiah, Muthu Rama Krishnan, et al. "A review of machine learning methods for retinal blood vessel segmentation and artery/vein classification." Medical Image Analysis 68 (2021): 101905.

3. Abdelfattah, Mohamed S., et al. "Zero-Cost Proxies for Lightweight NAS." International Conference on Learning Representations.

4. Staal, Joes, et al. "Ridge-based vessel segmentation in color images of the retina." IEEE transactions on medical imaging 23.4 (2004): 501-509.

5. Wang, Huan, et al. "Recent Advances on Neural Network Pruning at Initialization."

6. Mahdinejad, Mahsa, et al. "Parameterising the SA-UNet using a Genetic Algorithm." IJCCI. 2022.

7. Houreh, Yassir, et al. "HNAS: hyper neural architecture search for image segmentation." (2021).

8. Mahdinejad, Mahsa, et al. "Grammar-guided evolution of the u-net." International Conference on the Applications of Evolutionary Computation (Part of EvoStar). Cham: Springer Nature Switzerland, 2023.

9. Popat, Vipul, et al. "GA-based U-Net architecture optimization applied to retina blood vessel segmentation." (2020).

10. Abdelfattah, Mohamed S., et al. "Zero-cost proxies for lightweight nas." arXiv preprint arXiv:2101.08134 (2021).
11. Chen, Wuyang, Xinyu Gong, and Zhangyang Wang. "Neural architecture search on imagenet in four gpu hours: A theoretically inspired perspective." arXiv preprint arXiv:2102.11535 (2021).

12. Elnakib, Ahmed, et al. "Medical image segmentation: a brief survey." Multi Modality State-of-the-Art Medical Image Segmentation and Registration Methodologies: Volume II (2011): 1-39.

13. Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." Medical image computing and computer-assisted intervention-MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer International Publishing, 2015.

14. Siddique, Nahian, et al. "U-net and its variants for medical image segmentation: A review of theory and applications." IEEE access 9 (2021): 82031-82057.

15. Elsken, Thomas, Jan Hendrik Metzen, and Frank Hutter. "Neural architecture search: A survey." Journal of Machine Learning Research 20.55 (2019): 1-21.

16. Poeta, Eleonora. Zero-Cost Proxies in Neural Architecture Search: A Comprehensive Study and Design of a novel hybrid proxy. Diss. Politecnico di Torino, 2023.

17. Hassanzadeh, Tahereh, Daryl Essam, and Ruhul Sarker. "Evolutionary deep attention convolutional neural networks for 2D and 3D medical image segmentation." Journal of Digital Imaging 34.6 (2021): 1387-1404.

18. Lu, Zexin, et al. "M 3 nas: Multi-scale and multi-level memory-efficient neural architecture search for low-dose ct denoising." IEEE Transactions on Medical Imaging 42.3 (2022): 850-863.

19. Fan, Zhun, et al. "Evolutionary neural architecture search for retinal vessel segmentation." arXiv preprint arXiv:2001.06678 (2020).

20. Ali, Muhammad Junaid, et al. "Designing convolutional neural networks using surrogate assisted genetic algorithm for medical image classification." Proceedings of the Companion Conference on Genetic and Evolutionary Computation. 2023.

21. Calisto, Maria Baldeon, and Susana K. Lai-Yuen. "EMONAS-Net: Efficient multi-objective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation." Artificial intelligence in medicine 119 (2021): 102154.

22. Ali, Muhammad Junaid, et al. "Evolutionary Neural Architecture Search for 2D and 3D Medical Image Classification." International Conference on Computational Science. Cham: Springer Nature Switzerland, 2024.

23. Kuş, Zeki, and Berna Kiraz. "Evolutionary architecture optimization for retinal vessel segmentation." IEEE Journal of Biomedical and Health Informatics (2023).

24. Vo, A., Pham, T.N., Luong, N.H.: Lightweight Multi-Objective and Many-Objective Problem Formulations for Evolutionary Neural Architecture Search with the Training-Free Performance Metric Synaptic Flow. In: Informatica, 47.3, (2023).

25. Wang, Huan, et al. "Recent advances on neural network pruning at initialization." arXiv preprint arXiv:2103.06460 (2021).

26. Jacot, A., Gabriel, F., & Hongler, C. (2018). Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31.

27. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention-MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18 (pp. 234-241). Springer International Publishing.