## 2 Methods
### SHapley Additive exPlanations  
The more complex the design of a machine learning model, the harder it is to understand. Deep learning is a prime example, where neural networks involve complex math that can be challenging for non-experts to grasp. As artificial intelligence and machine learning are used more and more in various applications, the need to explain how these models work has become increasingly important. This has led to the development of new techniques such as LIME [31] and SHAP [32], which bring explainability in machine learning to the field. To understand the new framework, let us first clarify what we mean by'model' in this framework. A set \(X\subset\mathbb{R}\) is called linearly separable if there exists a hyperplane \(H\) such that \(H\) can separate the set \(X\) into two subsets \(A\) and \(B\). More formally, for any two points \(x_{1},x_{2}\) in \(X\), if \(x_{1}\) belongs to subset \(A\) and \(x_{2}\) belongs to subset \(B\), then \(H\) can be represented as a linear function as follows:  
\[H(x)=w^{T}x+b \tag{8}\]  
where \(w\) is called the weight vector. If \(H(x)>0\), then \(x\) belongs to subset \(A\), otherwise, it belongs to subset \(B\). Imagine we have data to sort into two categories, like categories A and B. A common approach is to represent these data on a dimensional surface, like a plane. Each data point represents a dot, and its color would show its category. Now, we can introduce the idea of a model. In this context, a model is like a dividing line on this plane that separates the dots in category A from those in category B. Mathematically, this line can be represented by the following:  
\[m(x)=a_{1}x_{1}+a_{2}x_{2}+\ldots+a_{n}x_{n}+c \tag{9}\]  
Figure 3: The methodology of heatmap construction. Imagine all the possible dividing lines that we could draw on the plane to separate the data. These lines together form a category that we can call \"models\". Our goal is to find the optimal dividing line among them, following certain rules, often minimizing some kind of error. Now, let us introduce a new idea in machine learning: the \"explanation model\". Based on SHAP [32], it is another model itself, but simpler and easier to understand than the original complex one. We can think of it like this: sometimes it is easier to analyze a problem using a simpler version. Similarly, we can transform the original data \(x\) (such as lowering the resolution of an image) into a simpler format \(x^{\prime}\) using a transformation \(x=h_{x}(x^{\prime})\) while keeping the important details for the task. Once we have these simplified data, we can use the explanation model to understand how the original model works. This explanation model is typically simpler, making it easier to understand how the model arrives at its decisions. Our framework uses a specific type of explanation model that relies on adding up the contributions of additive feature attribution methods, which have an explanation model that is a linear function:  
\[g(z^{\prime})=\phi_{0}+\sum_{i=1}^{n}\phi_{i}z^{\prime}_{i} \tag{10}\]  
where \(z^{\prime}\) denotes binary variables, \(n\) denotes the number of simplified input features, and \(\phi_{i}\) denotes the attribute effect. In the framework of SHAP, the main aim is to understand the impact of each feature on the model's output. This involves assessing how the output of the model changes as we vary the input features. The mathematical background of this method has its origins in game theory, more specifically, Shapley regression values, which express the importance of each feature for linear models. This method calculates each attribute's effect as follows:  
\[\phi_{i}=\sum_{S\subseteq F\setminus\{i\}}\frac{|S|!(|F|-|S|-1)!}{|F|!}[f] \tag{11}\]  
where \(f_{A}\) denotes the model over the subset \(A\subseteq F\) of the feature set \(F\).