# TorchEsegeta: Framework for Interpretability and Explainability of Image-Based Deep Learning Models
## 1 Introduction  
The use of artificial intelligence is widely prevalent today in medical image analysis. However, it is imperative to do away with the black-box nature of deep learning techniques to gain the trust of radiologists and clinicians, as a model's erroneous output might have a high impact in the medical domain. Interpretability vs. Explainability:Interpretability and explainability methods aim to unravel the black-box nature of decision making in machine learning and deep learning models. Explainability focuses on explaining the internal working mechanisms of the model. In contrast, interpretability focuses on observing the effects of changes in model parameters and inputs on the model prediction, hence attributing properties to the input, and it also requires a context, considering the audience who is to interpret the properties and characteristics for the model outcomes [1; 2]. It is to be noted that earlier work failed to draw a clear line of distinction between interpretability and explainability as these terms are subjective and pertain to the stakeholders, considering the audience who must understand the model and its outcomes. Interpretability and explainability nurture a sense of human-machine trust as they help the users of the machine/deep-learning models understand how certain decisions are made by the model and are not limited to statistical metric-based approaches such as accuracy or precision. In turn, this allows employing such models in mission-critical problems such as medicine, autonomous driving, legal systems, banking, etc. The basis of measuring model transparency is said to comprise simulatability, decomposability, and algorithmic transparency [2]; and model functionality, which consists of textual descriptions of the model's output and visualisations of the model's parameters. The goals to be attained through the interpretability of models [1] are trust, reliability, robustness, fairness, privacy, and causality. Explainability can be formulated as the explanation of the decisions made internally by the model that in turn generates the observable, external conclusions arrived at by the model. This promotes human understanding of the model's internal mechanisms and rationale, which in turn helps build user trust in the model [3]. The evaluation criteria for model explainability include comprehensibility by humans, fidelity and scalability of the extracted representations, and scalability and generality of the method [4]. Some models are transparent and hence inherently understandable, such as Decision Trees, K-Nearest Neighbors, Rule-based, and Bayesian models. There are even some deep learning-based models, such as GP-U-Net [5] and CA-Net [6], which are inherently trying to provide an explanation concerning their outcome. but some are opaque and require post hoc explanations. A post hoc model explanation can be through model-agnostic and model-specific techniques that aim towards explaining a black-box model in human-understandable terms. The European Union has also incorporated the transparency and accountability of models in 2016 as a criterion in the ethical guidelines for trustworthy technology AI [7; 8]. Through interpretability and explainability, to some degree, the following goals can be achieved: trustworthiness, causality, confidence, fairness, informativeness, transferability, and interactivity, which in turn can help improve the model [7]. Data scientists can generate interpretability-explainability results for their opaque models and present the model's reasoning or the model's inner mechanism to the domain experts (e.g., clinicians). If the reasoning is the same as a human domain expert would have done, then the experts can have more faith in that model, which in turn implies that the model can be incorporated into real-life workflows.