## 3 Background and Methods
### Interpretable Representation  
When using a classification in a specific domain, it is often not enough to obtain only the information in which final class instances belong (e.g., just to find out which sport is shown in the picture), but also to find out why a specific instance was classified in this class (e.g., which are the indicators that strongly indicate a specific sport). Understanding the built model also contributes to greater confidence in the model prediction. To detect possible errors, observe model behavior, and identify what significantly influenced the decision, various interpretive methods have been developed [46]. In our research, we used two of the most widely used interpretation methods, which allow the interpretation of models built above images: the Local Interpretable Model-Agnostic Explanations (LIME) and the SHapley Additive exPlanations (SHAP). In its interpretation, LIME focuses on each instance separately (this is also called a local explanation). The results of the interpretation for the individual instance are obtained by sampling its neighboring inputs and building a sparse linear model based on the predictions of these inputs. The features, included in this model, for which a high value of the coefficients was calculated are in this case considered more important for decision making [47]. The main goal of the LIME method is to provide interpretive representations for black-box classifiers. When classifying images, the LIMEprovides one or more pixel regions, indicating parts of an image that contain features, because of which the image can be classified to one of the possible classes. For its operation, LIME needs an implemented classifier function that returns classification probabilities for a list of possible classes. When we describe the above-mentioned process of generating an interpretation with Equation (6), \(x\) represents an individual instance for which we obtain interpretation, while \(g\) is a model from a set of potential interpretative models \(G\):  
\[\arg\min_{g\in G}\mathcal{L}(f,g,\pi_{x})+\Omega(g). \tag{6}\]  
The model to be explained is denoted with \(f\) while \(\pi_{x}\) represents the probability distribution around \(x\). \(\Omega(g)\) denotes how complex it is to interpret the model \(g\), which we want to minimize in order for an interpretive model to be as comprehensible as possible. \(\mathcal{L}(f,g,\pi_{x})\) denotes how similar the values of a linear model \(g\) are to the values of the built model \(f\) at the location defined by \(\pi_{x}\). For good interpretation of a classifier, we want to keep this value to a minimum [46]. The segmentation of an individual image with LIME is automated and is performed by combining similar neighboring pixels into so-called super-pixels. One super-pixel illustrates one area that represents an individual interpretive feature. Into how many of these areas the image will be divided is determined using an image segmentation algorithm such as Quick-Shift algorithm [48]. SHAP represents a game theory-based approach designed to explain the output of any machine learning model. [PERSON] and [PERSON] defined it as an optimal evaluation of the influence of individual features in association with local explanations using classical Shapley values from game theory and related extensions [49]. It implements six classes: TreeExplainer, GradientExplainer, DeepExplainer, KernelExplainer, SamplingExplainer, and PartitionExplainer. Each class is adapted to operate on a specific type of built model. TreeExplainer is intended to explain the outputs of a simple tree and tree-ensemble models, GradientExplainer explains a model using expected gradients, DeepExplainer is used to compute SHAP values in DL models, SamplingExplainer computes SHAP values under the assumption of feature independence, and PartitionExplainer allows creating common dependent variables that are available for reuse. It allows the interpretation of each instance of any classifier, as well as the interpretation of the learned model in a comprehensive form [50].