Concept and Applications of Explainable Artificial Intelligence for Knowledge Discovery in Science

Scientific discovery, the process of generating new knowledge about the natural world, has historically relied on observations, experiments, and analyses to uncover phenomena, establish patterns, and propose explanations. While technological advancements have enabled more precise and complex experiments, they also produce massive amounts of high-dimensional data, such as videos and microscopy images. Traditional statistical and manual methods struggle to model and interpret these data, necessitating advanced approaches to uncover meaningful relationships and insights. Deep learning (DL) has demonstrated remarkable success in modeling complex relationships within such high-dimensional data; however, its adoption in scientific discovery remains limited due to its inherent “black-box” nature. Unlike conventional predictive tasks, scientific discovery demands not only accurate predictions but also an interpretable understanding of how input data influence outcomes. To overcome this limitation, Explainable AI (XAI) methods have emerged as a major opportunity, as they enhance model transparency and allow researchers to interpret predictions. This thesis explores how integrating DL and XAI jointly into the experiment process can advance scientific discovery by revealing insights within high-dimensional data from the domains of biomedicine and material science. To systematically address this challenge, the research is structured in two parts, first focusing on the evaluation and selection of suitable XAI methods before applying them to scientific discovery tasks. Part 1 focuses on the evaluation of feature importance methods and metrics in computer vision. Feature importance methods are the most commonly applied XAI methods, not only throughout the field but also within this thesis. They highlight the most important input features to a model when making a prediction. However, especially their quantitative evaluation is still in an early stage, tackling unique challenges such as the absence of ground truth or the compositionality of influencing factors. As a basis for the thesis, these challenges and the current approaches proposed by the community to solve them are structured and discussed. Subsequently, the metrics are examined thoroughly, demonstrating that existing evaluation metrics often yield conflicting results. Consequently, a more robust evaluation schema than the current state-of-the-art is proposed. Based on this schema, a large-scale feature importance benchmark is conducted, establishing which feature importance methods are most suitable for different settings, and ensuring their effective application in scientific discovery tasks. Part 2 of the thesis explores how XAI methods can be leveraged to uncover novel scientific knowledge and enhance our understanding of complex phenomena within the domains of biomedicine and material science. To this end, three distinct case studies are presented, each demonstrating the application of DL and XAI in real-world scientific experiments, leading to novel discoveries that contribute valuable insights to their respective scientific communities. The first case study focused on type-2 diabetes, demonstrates how DL and XAI methods can identify spatial regions of interest and compute histologic biomarkers from two microscopy modalities, with the results supporting existing hypotheses or generating new ones about the condition. The second case study investigates the potential of XAI and treatment effect estimation for identifying predictive imaging biomarkers, using both proof-of-concept and medical datasets, to ultimately prescribe patients the most effective treatment. The third case study applies the approach to improving the manufacturing quality of perovskite solar cells, employing various XAI methods to gain insights into the thin-film formation process and its impact on device performance. Collectively, the results of this thesis demonstrate how integrating DL and XAI into experimental workflows enables new and novel possibilities for scientific discovery, uncovering insights that would be difficult, if not impossible, to obtain through traditional statistical or manual methods alone.