## 2 Methods
### Bayesian Hyperparameter Optimization Strategy (AutoML Framework)  
In this paper, we suggested a combined hyperparameter strategy using Bayesian optimization and Hyperband algorithms, following an AutoML approach, as illustrated in Figure 4. Our implemented strategy follows three main steps:  
1. **Configuration and model training:** This step is about automatically changing the hyperparameters of a given U-Net architecture, which we call a U-Net configuration. Then, we perform a training of the selected configuration and monitor related training metrics such as the loss function that we record and store for further use. 2. **Evaluate U-Net configuration performances:** In this step, an assessment of prediction performances of the current U-Net configuration is carried out through the computation of segmentation metrics such as IoU. 3. **Bayesian Optimization Loop (BOL):** Once a U-Net configuration is trained and evaluated, the hyperparameter optimization loop begins. In fact, when a U-Net configuration is trained, a surrogate Gaussian Process model is fitted in real time by observing the influence of input variation (U-Net hyperparameters) on the outputs, the training loss functions and the corresponding IoU score. In addition, a \"watchguard\" mechanism is implemented through the Hyperband algorithm in order to automatically address unpromising training experiments to save computing resources. Thus, the optimization loop performs several iterations on the overall process (steps 1-3) by varying the U-Net configurations until the target IoU score is reached (IoU \(\geq 0.5\)). ings, the hyperparameter search space was reduced, thanks to several assumptions that we discussed later. Tables 1 and 2 summarize the hyperparameter search space values and the combination numbers per category, respectively. The total number of hyperparameters combinations \(\mathbf{\Pi}\) is computed by the general combinatorics formula described by Equation (10):  
\[\mathbf{\Lambda}=\prod_{i=1}^{r}\mathbf{n}_{i} \tag{10}\]  
where \(\mathbf{\Lambda}\) is the search space total configurations number, r represents the hyperparameter variables, \(\mathbf{n}_{i}\) represents their associated value numbers and \(\mathbf{\xi}\) is the total average reduction factor. In deep learning, optimizer algorithms are key systems which strongly impact the training and prediction performance of the model. Such an optimization system modifies the weights of the network through the adaptation of learning rate and the minimization of a loss function by a gradient descent algorithm [61]. This optimization procedure confers to the deep learning model the ability to learn from these errors. In addition, optimizer choice is a huge research area [62; 63; 64; 65; 66; 67; 68] and a complex question strongly correlated with the type of data to analyze. We can distinguish two main optimizer families:  
* Stochastic gradient descent (SGD) with momentum (i.e., inertia added for reducing variance and increasing the convergence rate). * Adaptative stochastic gradient descent optimizer, which automatically tunes learning rate parameter \(\boldsymbol{\alpha}\) during training. Although the SGD family could be efficient, fine-tuning the learning rate, momentum and implementation of a learning rate scheduler is mandatory. Taking all of that into account, adaptative optimizers such as Adam, Adamax and RMSprop were selected. Therefore, we could reduce the range of learning parameter \(\boldsymbol{\alpha}\) between \(1\times 10^{-3}\) and \(1\times 10^{-4}\). These assumptions led to reducing the hyperparameter search space combinations by 85% from 8640 in configuration A to 1296 in B. Although this is a huge search reduction improvement, it is not sufficient and should be improved. Thus, our last degree of freedom is to reduce the activation function search space. Activation functions play a key role in deep learning networks by applying nonlinear transformations to their input weights. By doing so, they trigger/activate the learning process of the network.