# Making Sense of Machine Learning: A Review of Interpretation Techniques and Their Applications
## 3 Techniques for Explainable AI
### Linear and Logistic Regression  
[PERSON] [17] created regression analysis as a method to forecast one continuous variable based on another. He hypothesized that the connection between two variables could be represented by a straight line. He collected data on the heights of parents and their children and to test this idea, he plotted the data on a graph. He observed that the children's heights tended to deviate less from the average height of all children than the heights of their parents. [PERSON] called a straight line drawn through the data points the \"regression line\", and used it to predict the height of a child based on the height of their parents. [PERSON] [34] invented logistic regression by using the logistic function for statistical modeling. Both linear and logistic Regression are popular ML algorithms with a wide range of applications in AI. Linear regression predicts a continuous numerical output variable, while logistic regression predicts a categorical one. It is widely accepted that linear and logistic regression models have a clear and interpretable structure. These properties are valuable for designing AI-based decision-making systems that are trustworthy and can be used in explainable AI. [PERSON] et al. [35] provided a thorough overview of linear regression analysis and its strengths and limitations, discussed common interpretation mistakes and emphasized the necessary aspects of reporting the results to ensure the validity and reliability of the research findings. [PERSON] et al. [36] provided a clear and accessible introduction to logistic regression analysis and emphasized the importance of transparent reporting and careful interpretation of the results. These considerations are particularly important in the context of explainable AI, where the interpretability of models is essential for ensuring their trustworthiness and reliability. To improve the accuracy, stability, and generalization of regression models, they are frequently used in ensemble methods. Powerful ensemble models can be developed by combining bagging, boosting, stacking, or random forests with regression models [37; 38]. Global Sensitivity Analysis (GSA)  
GSA has become integral in understanding complex ML models, with [PERSON] et al. [39] highlighting its impact. GSA enables the quantification of input parameter importance, aiding in model optimization and critical feature identification and enhancing interpretability across domains. Additionally, GSA facilitates sensitivity assessment in hybrid models, evaluating constituent model contributions and ensuring robust, informed decision-making for improved performance. Furthermore, in engineering and risk assessment, GSA integrated into the GSAS (Global sensitivity analysis-enhanced surrogate) method allows for efficient reliability analysis, identifying critical parameters and enhancing understanding of system interactions and uncertainties, crucial for system design and maintenance [40]. The work of [PERSON] et al. [41] on gradient boosting machines (GBMs) influenced GSA application in ML. GBMs emphasized individual model contributions, aligning with GSA's focus on input impact evaluation. This integration enhanced model interpretability and robustness, particularly in ensemble-based learning techniques, contributing to improved model performance and understanding in the field of ML. The pioneering work of [PERSON] and his colleagues in developing the Bayesian network (BN) framework [42; 43] has significantly shaped the landscape of probabilistic reasoning in ML. [PERSON]'s research laid the groundwork for comprehending intricate probabilistic relationships among variables, enabling the representation of complex data structures in an intuitive manner. Emphasizing the utilization of graphical models to depict conditional dependencies, [PERSON]'s contributions facilitated the creation of transparent and interpretable models, fostering a more profound understanding of the underlying data structure.