## 1 Introduction
### State of the Art  
The term explainable ML (see [12]) refers to intrinsically complex and opaque models (so-called black-box models) where tools or methods explain their operation. As it is described in surveys [13; 14], this approach uses explanations to understand predictions, relying on external techniques and tools [9] since the model itself is not directly interpretable. As today's architectures and algorithms become increasingly complex and sophisticated, they require additional explanation mechanisms to be more effective and predictive. The explainability and interpretability of the algorithms are essential in real-life systems that have a key impact on people's lives, health, finances, security, or civil rights. It makes it possible to build trust in the technology, ensure compliance with regulations, and avoid potential risks arising from erroneous or unjustified decisions made autonomously by such systems. Because the methodology in this field has been developing rapidly in recent years, a wide variety of techniques are used in XAI to understand how ML models make decisions. According to the literature, as many \"black-box\" models utilize deep neural networks or decision tree ensembles like random forest [15; 16], it is especially those approaches that are in the scope of XAI. We can divide XAI approaches into different categories based on their nature and application: model-agnostic, which allow using the same methodology to compare various \"black-box\" approaches [17; 18; 19], or model-specific [20] techniques that work based on the details of the specific black-box model structures. The choice between those two XAIs depends on knowledge of the model's internal architecture, access to the knowledge about how a machine learning model makes individual (local) predictions (see [21; 22]) or which features are most responsible for the model output (see [22; 23]). Popular XAI techniques used in this area include LIMEs (Local Interpretable Model-agnostic Explanations) [24] which create a surrogate model such as linear regression, SHAPs (Shapley Additive Explanations) [25] that decompose the output of a model by the sums of the impact of each feature, Anchors [26] that find a decision rule that \"anchors\" the prediction sufficiently, Partial Dependence Plots (PDPs) [27] which show whether the relationship between the target and a feature is linear, monotonic, or more complex, Individual Conditional Expectation (ICE) Plots [28] that show how the instance's prediction changes when a feature changes, Feature Importance [29], Linear Model Coefficients [30], and Counterfactual Explanations [31]. Visual-based explanation of an image-domain ML model can be achieved with Saliency Maps [32] or Grad-CAM (Gradient-weighted Class Activation Mapping) [33]. Reinforcement learning (RL) [34] and its deep learning variants (DRLs) [35] are very popular branches of ML methods and serve in various fields for practical applications, especially where problems are dynamic and sequential. DRL is an agent-based model in which the algorithm learns to make optimal decisions based on the rewards and punishments it receives for its actions. Its adaptability is particularly evident in robotics (teaching robots to navigate unfamiliar terrain or perform specialized tasks), computer games (trainingartificial players [36] such as AlphaGo [37]), autonomous vehicles (taking actions in real time), and optimizing complex decision-making processes in industrial systems. In reinforcement learning, explanation of the process is complicated because the agent acts sequentially and bases its decisions on multiple stages of interaction. In practice, approaches are used in this regard which can be divided into several groups of techniques, including, among others, the following ones:  
* policy explanation: local policy approximations [38; 39; 40], i.e., approximating the agent's policy with simpler models such as decision trees or using feature importance methodology to assess the impact of individual features on the agent's decisions.