## 5 Results
### Rule-Based Explanations  
Rule-based explanations illustrate a model's decision-making process in the form of a tree or list. Figure 12c demonstrates an example of a rule-based explanation. Largely, the models producing rule-based explanations generate explanations with a global scope, i.e., of the whole model. [PERSON] et al. proposed the existing TREPAN decision tree as a surrogate model with an FFNN to generate rules depicting the flow of information within the neural network [89]. [PERSON] et al. used the Wang-Mendal (WM) algorithm to generate fuzzy rules to support recommendations with explanations [157]. A novel neuro-fuzzy system,  
Figure 12: Different forms of explanations: (**a**) numeric explanation of remaining life estimation in industry appliances [49]; (**b**) visual explanation for fault diagnosis of industrial equipment by [PERSON] et al. [140]; (**c**) example of rule-based explanation in the form of a tree [127]; and (**d**) explanation text generated with GRACE, proposed by [PERSON] et al. [58]. ALMMo-0\", was proposed by [PERSON] et al. [116]. In addition, model-specific methods have been proposed to generate rule-based explanations such as eUD3.5, an explainable version of UD3.5 [102] and Ada-WHIPS to support the AdaBoost ensemble method [112]. More methods generating rule-based explanations are listed in Table 6. The rule-based explanations are much simpler in nature than the numeric explanations that facilitate this type of explanation in supporting recommendation systems developed for general users from domains such as entertainment and finance. The use of textual explanations is found to be least common among all forms of explanations due to their higher computational complexity which requires natural language processing. The textual explanations are mostly generated at the local scope, i.e., for an individual decision. In notable works, textual explanations were generated using counterfactual sets [7; 55], template-based natural language generation [164], etc. [PERSON] et al. proposed textual CBR (TCBR) utilising patterns of input-to-output relations in order to recommend citations for academic researchers through textual explanations [156]. Unlike TCBR, interpretable confidence measures were used by [PERSON] et al. with CBR to generate textual explanations [92]. [PERSON] et al. proposed GRACE which can generate intuitive textual explanations along with the decision [58]. The textual explanations generated with GRACE were revealed to be more understandable by humans in synthetic and real experiments. Moreover, textual explanations are found to be generated at the local scope (see Figure 9) and these explanations are associated with academic research, judicial systems, etc. Table 6 lists several other proposed methods to generate textual explanations. The most common form of explanation was found to be visualisations, as shown in Table 6. With respect to the stage of adding explanations, in the majority of the cases, visual explanations in both the local and global scopes were generated using post hoc techniques and the research studies were carried out as domain-agnostic and from the healthcare domain (see Figure 9). Common visualisation techniques are class activation maps (CAM) [140; 141] and attention maps [79; 152]. CAM was further extended with gradient weights, and Grad-CAM was proposed by [PERSON] et al. [80]. [PERSON] et al. used Grad-CAM to detect COVID-19 infection based on X-rays [109]. [PERSON] and [PERSON] adopted another form pGrad-CAM to provide an explanation for banknote fraud detection [160]. Heatmaps of salient pixels were used by [PERSON] et al. as a complement to the concept-based explanation. They proposed a framework of concept attribution for deep learning to quantify the contribution of features of interest to the deep network's decision making [132].