## 6 Discussion
Similarly, in [251] the authors provide a comprehensive review with regard to methods that can mitigate human bias (e.g., selection bias) that is present in the data. Another interesting comprehensive review is provided by the authors in [252], where they have gathered a noticeable amount of research studies to identify types of bias and methods to mitigate them, focusing on data bias and estimator bias. However, the above-mentioned review does not take into consideration either the fact that model builders may also introduce bias when training/evaluating their models (i.e., engineer bias), as depicted in Section 5 of this manuscript, nor the fact that the selection of an optimizer may also affect an ML model in terms of bias, as stated in Section 4. A similar comprehensive review that focuses on the same aspects as the last-mentioned review is  
Figure 7: Number of research studies per bias type. presented in [253]. What is more, in contrast to this manuscript, the above-mentioned reviews do not categorize the provided approaches per use case domain, making it difficult to identify the most appropriate approach for a specific domain. With regard to reviews that are more use-case-specific, those mostly refer to the health domain. More specifically, the authors in [254] provide a comprehensive study of the types of bias that can be encountered in health-related ML models and, more precisely, in ML models that assess the risk of cardiovascular disease (i.e., CVD). In this study, the authors also focused on data and estimator bias. In [255], the authors also investigate data bias in the context of CVD and provide ways to mitigate the bias that may be present in the electronic health records (i.e., EHRs) of patients. Similarly, the authors in [256] also provide a comprehensive list of approaches that tackle bias in radiology; however, they also refer to optimizers and regularization techniques, since they also identify that those can also introduce further bias. However, also in this case little to no attention is paid to the bias that can be introduced by the ML engineers. A summary of the above can be found in Table 12.  
The above provides a direction for future research related to ML models towards improving methodological practices to incorporate the necessary bias mitigation techniques, thus providing more complete and meaningful results. Having said that, the findings of this manuscript regarding algorithm bias show that the SVM algorithm is preferred to reduce bias whilst the SGD optimizer and the regularization method Lasso are the most widely used optimizer and regularization method, respectively. However, it should be highlighted that the above does not suggest that SVM, SGD and Lasso always produce the least bias. As it has already been stated multiple times in the manuscript, the type of data and the corresponding ML task that must be undertaken will dictate the most appropriate algorithm, optimizer and regularization method in order to reduce bias. Having this in mind, it is also quite useful to have access to other similar approaches and the methods that they use so that the model builders have some guidance regarding what techniques seem to be more appropriate for their use case. As for the model's evaluation, there also seems to be a lack of reference to the adaptation of appropriate techniques for mitigating bias and ensuring objectivity during the evaluation process. Based on the findings of this study, the three-step action plan presented in Figure 6 should become the cornerstone of the evaluation of ML models and related publications, thus leading to a widely adopted framework for achieving objective evaluation of ML approaches. With regard to any limitations that this literature review may have, those are mainly related to potential selection bias when performing the selection process of the corresponding studies.