## 4 Algorithm Bias
### Regularization
Apart from early stopping, in NNs there also exist two additional strategies of regularization, namely dropout and weight decay. Regarding dropout, it randomly altersthe architecture of an NN in order to allow the model to avoid pure generalization to unseen data [205]. Even though it is a simple concept, it still requires the finetuning of hyperparameters, such as the learning rate and the number of units in the NN's hidden layer(s), so that it is indeed beneficial for the model [206]. As for the weight decay, it is quite similar to dropout; however, the latter penalizes the NN's complexity exponentially, whilst dropout penalizes the NN's complexity linearly [207]. Among the different weight decay methods that have been proposed in the literature, the most recent ones are Selective Weight Decay (i.e., SWD) [208] and Adaptive Weight Decay (AdaDecay) [209]. At this point, it should be mentioned that many sources in the literature conflict weight decay with L2 regression, while others clearly distinguish each other. This should be resolved in future scholarship, in order to avoid any confusion between those two concepts. There exist several studies in the literature that utilize the aforementioned regularization techniques. In deeper detail, Lasso regression has been used alongside the RF algorithm for classifying long non-coding RNAs [210]. In [211], the authors compare early stopping with Bayesian regularization in an NN architecture that predicts the vapor pressure of pure ionic liquids, showcasing that the latter outperformed the first one in terms of generalization performance. With regard to computer vision tasks, in [212], two versions of dropout are showcased, namely Biased and Crossmap dropouts. They both offer higher performance to the CNNs that utilize them. Similarly, in [213] weight decay, which is also used in [214], and dropout are also utilized in a CNN that performs facial emotion recognition, enabling it to achieve higher accuracy. In another computer vision task related to geological data, the authors utilize two types of data augmentation as a way of regularization in their developed CNN, namely CutOut and CutMix [215]. By doing so, they managed to minimize training loss and validation loss, thus achieving a performance equal to a transfer learning method that utilized a specific domain model. Moreover, in [216] the authors utilize a derivative of Lasso for an image classification task, and they prove that their approach is not only slightly inferior to other regularization techniques but also the corresponding NN needs fewer features to achieve those inferior results. With regard to the energy domain, the authors in [217] use Lasso regularization in a linear regression model for time series forecasting while in another approach [218], a subtype of elastic net regularization named Adaptive Elastic Net is utilized in a multilevel regression model for estimating residential energy consumption. In [219], where the authors aim to predict rainfall runoff, they make use of early stopping and weight decay, thus simultaneously decreasing the model's overall training time whilst ensuring the highest accuracy of predictions possible. As for the health domain, numerous approaches can be found in the literature that make use of different types of regularization techniques. More specifically, Lasso regularization is used in [220] alongside a regression model for subgroup identification of conditioning regimens in patients with Acute Myeloid Leukemia (i.e., AML) in Complete Remission (i.e., CR). A variation of Lasso is also used in [221], where the authors perform non-linear regression for tracking the position of the head and neck based on corresponding images. Elastic net is used in [222] and in [223]. In both cases, the authors make use of Magnetic Resonance Imaging (i.e., MRI) data alongside a logistic regression model and a linear regression model, respectively, achieving remarkable results in terms of classification accuracy.