# Privacy-Preserving and Explainable AI in Industrial Applications
## 3 Explainable Industrial AI Applications
### State of the Art in Explainable AI  
To make it easier for a user to understand the logic behind the decision taken by an algorithm, a user interface (UI) should accompany it. Using UI, automation bias can be mitigated. We can categorize explainable AI methods into the following:  
* saliency maps: elements in the input that have the largest influence in the prediction are identified (e.g., LIME);
* feature attribution: attributing the classification to a small number of numeric/semantic features [47; 48];
* metric learning [49]: mapping out data structures by deriving a metric from a classifier (explicit Siamese networks are very popular);
* activation maximization: methods that are based on GAN.  
There are several methods of explainable AI that can be applied, depending on the data type. These are discussed below.  
Even though there are several explainable AI techniques, only certain methods can be applied on tabular data [50]. Techniques designed for images or text data are typically not applicable to tabular data. Tabular data can present characteristics such as correlations between features or temporal aspects, and they may contain categorical features along with continuous features. [PERSON] et al. [51] proposed one of the first methods of explainable AI, named ExplainD. It measures the importance of each input feature related to the prediction of a classifier. LIME proposed by [PERSON] et al. [52] is a model-agnostic technique used to explain the predictions of the classifier. Many other methods have been developed on the basis of LIME. The core principle is that one or more models are trained to approximate the predictions of the classification model, to determine why the classification model outputted a certain prediction. These models are trained with a dataset containing perturbed data points, which are close to the instance of interest. The newly trained model (named the surrogate model) is used to compute the proximity of each sample instance to the instance of interest. In Figure 3, the explanations for three predictions can be observed.  
For time series, saliency maps can be extracted to highlight the importance of a sequence from the input, which is related to the prediction. Class activation mapping (CAM) is a method used on convolutional neural networks (CNN) to identify input features which are representative for a class. [PERSON] et al. [53] applied this method to explain model decisions when classifying small X-ray diffraction. [PERSON] et al. [54] proposed a new method of explainable artificial intelligence (XAI) in which a data-driven approach is used to exploit distance-based data structures, without the need for making any assumption related to the data. LIME can also be employed, as well as any method that yields a heatmap, e.g., deep learning important features (DeepLIFT). Layer-wise relevance propagation (LRP) [55] can also be used, which is a method that computes feature importance by backpropagating a relevance score through the model. Figure 4 contains an example of how features are highlighted to justify model predictions.  
It is widely known that XAI presents a wide variety of methods to be used for imaging data. All methods described in the categories above can be applied to gain insight into the logic used by the model to perform the prediction. [PERSON] et al. [56] discussed the use of XAI for medical images, and [PERSON] et al. [57] described approaches and future challenges after summarizing the history of XAI.