Generating and Modifying Natural Language Explanations (2021)

Abstract: HESIP is a hybrid explanation system for image predictions that combines sub-symbolic and symbolic machine learning techniques to explain the predictions of image classification tasks. The sub-symbolic component makes a prediction for an image and the symbolic component learns probabilistic symbolic rules in order to explain that prediction. In HESIP, the explanations are generated in controlled natural language from the learned probabilistic rules using a bi-directional logic grammar. In this paper, we present an explanation modification method where a human-in-the-loop can modify an incorrect explanation generated by the HESIP system and afterwards, the modified explanation is used by HESIP to learn a better explanation.

## introduction in recent years, the development of explanation systems has gained a lot of attention. most of these explanation systems @xcite @xcite can explain predictions made by machine learning (ml) models. researchers are focusing on building explanation systems for ml models, because these models have shown excellent performance for different prediction tasks @xcite and most of these models are sub-symbolic blackbox models that are not easily understandable, and therefore lead to difficulties explaining the predictions to the users. explanation systems such as lime @xcite , @xcite and shap (lundberg and lee, 2017) use existing information of the datasets to explain predictions. however, sometime information that is not present directly in the dataset such as relation information can play an important role in the explanation; especially, in image prediction tasks as shown in lime-aleph @xcite . hesip is a hybrid explanation system for image predictions. the hesip system explains the predictions to the users using natural language explanations. the explanations are generated in a controlled natural language (cnl) @xcite using a logic programming based bi-directional grammar that is similar to @xcite . the generated explanations of the hesip system are humanunderstandable as well as machine-processable. since the explanations are represented in a natural language, they are immediately understandable by all types of users. the bi-directional grammar of the hesip system can also process a generated explanation that has been modified by the user. the hesip system aims to generate an explanation for the predicted image that represents the object information together with the relation information. it is expected that such as system is not perfect, and hesip is not an exception. hesip sometimes generates wrong explanations. to the best of our knowledge, there is no explanation system that allows a user to modify an explanation in order to improve the explanation generation process of the system. it is important that a user can modify an incorrect explanation so that the system can learn how to generate a better explanation taking the feedback from the user into consideration. in this paper, we present a method that involves a humanin-the-loop who can fix incorrect explanations by modifying them. ## hesip: system architecture hesip is a hybrid system that explains image predictions by integrating sub-symbolic and symbolic ml techniques in two separate components. for an input image, hesip makes a prediction using a sub-symbolic ml model. afterwards, hesip uses a symbolic ml technique to learn symbolic probabilistic rules that are used to explain predic-tions. based on a definition of hybrid systems introduced in kautz's classification @xcite , the hesip system follows the architecture of a type-3 hybrid system, since hesip uses a sub-symbolic component to work on a task, and then a symbolic component to finalise that task. figure 1 shows the architecture of the hesip system. more details about the hesip system can be found in @xcite . 2019 ) have developed an explanation system called lime-aleph that explains image predictions using the learned rules. the hesip system is motivated by the lime-aleph system and extends the architecture of lime-aleph in order to achieve a more generalised method. lime-aleph depends on two datasets that consist of synthetic images while the hesip system can be applied to datasets that consist of real-world images. a detail comparison between the hesip system and the lime-aleph system is provided in @xcite . the steps of the hesip system are demonstrated here using the pascal-part dataset @xcite that consists of real-world images. we want to learn the concept of a potted plant from the different parts of the concept that are present in an image. for this learning task, only those images that contain potted plants and bottles are used from the dataset. the potted plant concept has two parts: pot and plant. we say that there is a potted plant concept in an image, if it represents a pot that is located below a plant. similarly, the bottle concept consists of two parts: body and cap. there are images of bottles in the dataset that contain only a body part. in our case, we work with the bottle images that contain both parts. figure 2 shows images that contain a potted plant and a bottle. ## hesip: user interface a prototype of a graphical user interface for the hesip system has been developed to illustrate the interaction between a user and the system for generating and modifying explanations. as illustrated in figure 3 , a user clicks on the "choose file" button to select an image for predicting the image. after selecting the image, it is displayed and a new "predict & explain" button appears. when the user presses on the "predict & explain" button, the hesip system predicts the image in the sub-symbolic component and learns a probabilistic rule in the symbolic component to explain the prediction. the prediction for the image and the explanation of the prediction along with the probability of the explanation are displayed in a panel (see figure 3 ). because the interface of the hesip system displays the predicted image, the prediction and the explanation together, the user can relate and inspect them immediately and can see whether the explanation is correct or not. after showing the prediction and the explanation to the user, two buttons "modify explanation" and "confirm explanation" are displayed (see figure 3 ). after inspection, the user can either modify or confirm the explanation. if the user feels that there is something wrong with the generated explanation, then they can fix the incorrect information so that the hesip system can learn a better one. after pressing the "modify explanation" button, the hesip system shows the explanation inside a text editor that allows one to modify the explanation generated by hesip. when the user uses the text editor to modify the explanation, the editor guides the user using appropriate word suggestions according to the grammar of the cnl used for generating the explanation (see figure 4 ). there exist several projects where predictive editors have been developed to guide users for writing sentences in a cnl @xcite @xcite . when writing a cnl sentence, the next word is predicted and suggested to the user by the predictive editor. since the explanations are expressed in a cnl, the text editor of the hesip system can be developed as a predictive editor similar to the peng asp system. the user can select a word from the suggested list of words or can write the word manually. the altered parts of the explanation are displayed as coloured text so that the user can easily identify which parts of the explanation are altered. the user clicks on the "submit" button once they are done with the modification. afterwards, hesip learns a new explanatory rule for the predicted image by taking the modified explanation into account (see section 5). the modified explanation is first processed using the bi-directional grammar that listing 4: definite clause grammar rule of the bi-directional grammar np([mode:m, ctx:body, fcn:subj, def:_d, num:n, arg:x, clause:c1-c5, ante:a1-a4]) --> det([mode:m, morph:o, num:n, def:d, clause:c1-c2]), noun([mode:m, morph:_, num:n, arg:x, clause:c2-c3, ante:a1-a2]), prep([mode:m, ctx:head]), rnoun([mode:m, morph:_, num:n, arg:x, clause:c3-c4, ante:a2-a3]), { anaphora_resolution(det_noun_prep_rnoun, [m, d, x, c1, c4, c5, a1, a3, a4]) }. ## generating explanations in section 2, we showed how the hesip system learns symbolic representation for generating explanations. now let us have a closer look at how these explanations are generated. once the explanatory rule is learned in the symbolic component, the hesip system generates an explanation for the image prediction from the learned rule using a bidirectional logic grammar. the generated explanation can be processed with the same bi-directional grammar to produce a rule that is semantically equivalent to the learned rule from which the explanation was generated. this is important for the modification process, since we want to make sure that the grammar produces correct rules after processing the generated explanations as we will see in the following section. the learned rule needs to be pre-processed before it can be used by the grammar for generating a natural language explanation. in the pre-processing steps, the literals of the rule are first reordered in a linguistically-motivated way; therefore, subject grouping is applied to remove redundant information in the reordered rule; and finally, variables that serve as names are added to the rule if required in order to resolve ambiguity of definite description. after pre-processing, hesip sends the reconstructed rule to the grammar that generates the explanation. listing 3 shows a reconstructed rule for the learned rule (see listing 2) of the potted plant concept. listing 3: a reconstructed rule for the potted plant concept after pre-processing. for the reconstructed rule in listing 3, the grammar of the hesip system generates the following explanation: if an object contains an object of type pot and contains an object of type plant and the object of type pot is located under the object of type plant then the object is of type potted plant. listing 4 shows an example of a grammar rule that generates a noun phrase in the subject position for a clause pattern that occurs in the body of a rule. in the generation mode (mode:gen), this grammar rule takes a class and a type (for example, class(b, object) and type(b, pot)) as input and generates an indefinite noun phrase (an object of type pot) or a definite noun phrase (the object of type pot) as output. the argument clause holds a difference list (c1-c5) with the incoming and outgoing literals. the argument ante holds a difference list (a1-a4) with the incoming and outgoing accessible antecedents. the call to anaphora resolution/2 updates these two difference lists. it is important to note that exactly the same grammar rule can also be used in the processing mode (mode:proc), since the grammar is bi-directional. in the processing mode, the grammar rule takes the generated verbalisation as input and produces a rule as output that is semantically equivalent to the rule from which the verbalisation was generated. ## modifying explanations the bi-directional property of the grammar enables the hesip system to modify the generated explanations. in section 3, we have shown how a humanin-the-loop can alter explanations that are displayed to the user for explaining image predictions. in this section, we show the steps (see figure 5 ) that are performed by the hesip system in order to generate a new explanation from a newly learned rule for the predicted image after modification by the user. note that the explanation modification steps are completed in the symbolic component of the hesip system. let us assume that a user wants to see the prediction and the explanation for an image shown in figure 2a that represents a potted plant concept. as discussed in section 2, hesip selects sample images for a predicted image, extracts information of the sample images, represents the sample image information using an ontology. finally, hesip uses the information of the sample images as example instances in the symbolic component to learn the explanatory rule for explaining the image prediction. let us assume that hesip uses the example instances shown in listing 5 and learns the explanatory rule in listing 6 for explaining the prediction of the image in figure 2a . in this case, hesip generates the explanation "if an object contains an object of type pot and contains an object of type plant and the object of type pot is located on the object of type plant then the object is of type potted plant." from the learned rule in listing 6 using the bi-directional grammar after applying the pre-processing steps (as discussed in section 4). listing 5: three example instances that are used to learn the explanatory rule for the potted plant concept. when this explanation is displayed to a user, then the user may want to modify the explanation after noticing that the relation on between the pot and the plant objects is not correct. let us assume, the user has changed the explanation to "if an object contains an object of type pot and contains an object of type plant and the object of type pot is located under the object of type plant then the object is of type potted plant." where the preposition on is replaced by under. after submission of the modified explanation, hesip processes the explanation using the bi-directional grammar to obtain a rule. the generated rule for the modified explanation is shown in listing 7. listing 7: a rule obtained using the bi-directional grammar by processing the modified explanation for the potted plant concept. the rule (see listing 7) derived from the altered explanation is then compared with the rule previously learned by hesip (see listing 6) to identify the modifications. in this case, the user has updated the relation information and the rule in listing 7 reflects that change. after identifying the changes, the amended information is updated in the example instances. in our case, we update the relation from on to under between the pot and the plant objects for all positive example instances. we do not update any information about the negative example instances, since the user modified an explanation in order to correct it and the positive example instances represent the correct information for the concept to be learned. afterwards, an explanatory rule is learned using the updated example instances. the new rule is shown in listing 8. we can see that the new rule is different from the previous one (see listing 6) and that the preposition has been replaced. listing 8: a new explanatory rule learned for the potted plant concept after the explanation is modified by the user. once the new explanatory rule is learned, hesip first pre-processes the new learned rule as discussed in section 4 that results in a reconstructed rule. after that, hesip verbalises the reconstructed rule to obtain a new explanation for the prediction. for this scenario, hesip generates the new explanation "if an object contains an object of type pot and contains an object of type plant and the object of type pot is located under the object of type plant then the object is of type potted plant." using the new learned rule in listing 8. the new explanation is then displayed on the interface. this example illustrates the explanation modifi-cation steps for changing the relation information. the hesip system applies the same process to generate a new explanation for updating any information in the explanation. as mentioned earlier, a predictive editor is used in the hesip system to modify an explanation that supports the user in making a modification. after generating an explanation for a prediction, it is possible that the explanation may have the following incorrect information and a user can update that information in the explanation: practically, in an explanation sentence, the user can update any content word introduced by the ontology used in the system. in the modification step, the predictive editor will ensure that the explanation is grammatically correct. in the case of updating the relation information, there should not be any issue, since the relevant relation words will be suggested by the predictive editor and the user can select the relation from a list of words. however, a problem may occur while updating any object property or type information in an explanation. an explanation may contain an anaphoric reference to an object. if the user updates the property or the type information of an object that is used as an anaphoric expression, then the user has to make sure that all the other parts of the explanation are also updated. let us consider an example from a tower concept learning task @xcite to illustrate this scenario. in tower concept learning, an image consists of three squares with green, blue and pink colours, and if the image contains a square on top of another square without repeating the same colour, then the image represents a tower concept. for the tower concept, the hesip system may generate the following explanation: "if an object a contains a blue object and contains a green object and the blue object is located on the green object then the object a is of type tower." where the user may update the colour green to pink only for the first occurrence (a green object to a pink object). this will lead to incorrect information in the later part of the explanation (the noun phrase the green object should be changed to the pink object). the predictive editor usually will not identify this information as incorrect, since the sentence is grammatically correct. one possible solution to overcome this problem is to design the predictive editor in such a way that whenever a word related to an anaphoric expression will be updated, the editor will highlight all relevant anaphoric expressions in the explanation and the user can then fix the relevant words. ## evaluation we evaluate the explanation generation and modification process of the hesip system using four datasets: potted plant concept learning, house concept learning, tower concept learning and single relation learning. the lime-aleph system has used the tower concept learning and the single relation learning tasks in order to demonstrate their method. for the house concept learning task, an image consists of a triangle and a square, and if the image contains a triangle on top of a square, then the image represents a house concept. for single relation learning, if an image contains a green square on the left side of a blue square, then the image represents the left of relation. for evaluation, we use 382 test images for the potted plant concept and 1000 test images for all other concepts. the explanation generation process is evaluated in two ways. first, if the generated explanation represents the literals that correspond to the literals of the image, then we consider the explanation as a correct one. second, we check if the bi-directional grammar works in both directions using a technique known as semantic-round tripping (hossain and schwitter, 2020). using this technique, we store the formal representation r1 of an explana-tion. the explanation is then processed by the grammar that produces second formal representation r2. therefore, we compare if r1 and r2 are semantically equivalent. for the 1000 test images, hesip generates all correct explanations for single relation learning and tower concept learning, and 999 correct explanations for house concept learning leading to the accuracy of 100%, 100% and 99.9%. for the 382 test images of potted plant concept, hesip generates 310 correct explanations with an accuracy of 81.15%. to evaluate the explanation modification process, we take the test images for which the hesip system could not generate the correct explanations. hesip could not generated correct explanations for 1 test image of the house concept and for 72 test images of the potted plant concept learning. to check if the modification process works correctly, we first modify an incorrect explanation; therefore, hesip learns a new explanatory rule taking the modified explanation into consideration and finally, we check if the explanation generated from the newly learned rule is correct or not. among the 72 test images of the potted plant concept for which hesip could not generate the correct explanations, it generated 53 explanations with wrong relations. we discussed one such explanation in section 5 where the explanation is learned for the relation on instead of the relation under. for all 53 explanations, we modified the explanations with correct relations and follow the steps discussed in section 5 to generate the new explanations. we found that the hesip system generated the new explanations with correct relations for all of them. for two test images, we could not modify the explanation using a correct relation, since the images contain only plants and there was no pot in these images. we also observe that there were 17 test images of potted plants for which the hesip system learned incomplete explanatory rules and as a result, the system could not generate suitable explanations that can be modified to generate the correct explanations. we also notice a similar problem for one test image of the house concept for which the hesip system could not generate a suitable explanation. ## conclusion in this paper, we presented an explanation modification method for hesip, a hybrid explanation system for image predictions. using the prototype of the hesip system, we showed how a human working in the loop can fix an incorrect explanation generated by the system. for an image prediction, the hesip system learns an explanatory rule and generates an explanation in cnl using a bi-directional logic grammar. if the user decides that the generated explanation is wrong, then they can modify the explanation to fix it following the grammar rules of the cnl. after modifying the explanation, hesip learns a new explanatory rule taking the user's modified explanation into account and generates an explanation from the new learned rule to better explain the image prediction. the result of the evaluation shows that the modification process of the hesip system is very effective in learning better explanations in the symbolic component of the system.