R EV : I NFORMATION -T HEORETIC E VALUATION OF F REE -T EXT R ATIONALES
Anonymous authors
Paper under double-blind review
A BSTRACT
Free-text rationales are a promising step towards explainable AI, yet their evaluation remains an open research problem.
While existing metrics have mostly focused on measuring the direct association between the rationale and a given label, we argue that an ideal metric should also be able to focus on the new information uniquely provided in the rationale that is otherwise not provided in the input or the label.
We investigate this research problem from an information-theoretic perspective using the conditional V -information (Hewitt et al., 2021).
More concretely, we propose a metric called R EV (Rationale Evaluation with conditional
V -information), that can quantify the new information in a rationale supporting a given label beyond the information already available in the input or the label.
Experiments on reasoning tasks across four benchmarks, including few-shot prompting with GPT-3, demonstrate the effectiveness of R EV in evaluating different types of rationale-label pairs, compared to existing metrics.
Through several quantitative comparisons, we demonstrate the capability of R EV in providing more sensitive measurements of new information in free-text rationales with respect to a label.
Furthermore, R EV is consistent with human judgments on rationale evaluations.
Overall, when used alongside traditional performance metrics, R EV provides deeper insights into a model’s reasoning and prediction processes.
I NTRODUCTION
Model explanations have been indispensable for trust and interpretability in AI (Lipton, 2018; DoshiVelez & Kim, 2017; Kim et al., 2016; Alvarez Melis & Jaakkola, 2018).
Free-text rationales, which explain a model prediction in natural language, have been especially appealing due to their flexibility in eliciting the reasoning process behind the model’s decision making (Camburu et al., 2018; Narang et al., 2020; Rajani et al., 2019; Kumar & Talukdar, 2020b; Brahman et al., 2021), making them closer to human explanations.
However, current automatic evaluation of free-text rationales remains narrowly focused.
Existing metrics primarily measure the extent to which a rationale can help a (proxy) model predict the label it explains (i.e., accuracy based)
(Hase et al., 2020; Wiegreffe et al., 2021).
Yet, these metrics offer little understanding of the new information contained in the rationale, as added to the original input, that could explain why the label is selected —the very purpose a rationale is designed to serve.
For instance, the two rationales r ∗ 1 and ˆ r 1 ,a in Fig.
1 would be considered equally valuable under existing metrics, even though they supply different amount of novel and relevant information.
In this paper, we overcome this shortcoming by introducing an automatic evaluation for free-text rationales along two dimensions: (1) whether the rationale supports (i.e., is predictive of) the intended label, and (2) how much new information does it provide to justify the label, beyond what is contained in the input.
For example, rationale ˆ r 1 ,b in Fig.
1 violates (1) because it is not predictive of the label, “enjoy nature”.
Rationale ˆ r 1 ,a does support the label but contains no new information that justifies it, beyond what is stated in the input x ; thus, it violates (2).
Rationale r ∗ 1 is satisfied along both dimensions: it supports the label and does so by providing new and relevant information, beyond what is in the input.
Our proposed evaluation is designed to penalize both ˆ r 1 ,a and ˆ r 1 ,b , while rewarding rationales like r ∗ 1 .
People usually go hiking to enjoy nature.
Why do people go hiking?
Hiking means the activity of going for long walks especially across country, or in the nature.
People who go hiking enjoy nature.
̂ r 1, a r *̂ r 1, b ̂ r
People go hiking might end up getting lost.
Hiking is a form of exercise.
People go hiking to lose weight.
enjoy nature lose weight
People go hiking to enjoy nature.
People go hiking to lose weight.
Vacuous Baseline Rationales bb
Evaluator
High [Figure] f REV ( x , y , r ) = log f
( y | r ) − log f
( y | r ,
[Figure]
[Figure]
We introduce a R EV 1 , which adapts an information-theoretic framework from Xu et al.
(2020) for evaluating free-text rationales along the two dimensions mentioned above.
Specifically, R EV is based on conditional V -information (Hewitt et al., 2021), which quantifies the degree of information contained in a representation, beyond another (baseline) representation, accessible to a model family,
V .
As our baseline representation, we consider any vacuous rationale which simply combines an input with a given label, without providing any new information relevant to answering why the label was chosen.
R EV adapts conditional V -information to evaluate rationales, where the representation is obtained via an evaluator model trained to produce a label given the rationale.
Other metrics do not take into consideration vacuous rationales, and are hence unable to measure new, label-relevant information in rationales, beyond a vacuous baseline.
Our experiments present evaluations with R EV for rationales under two reasoning tasks, commonsense question-answering (CQA; Talmor et al., 2018) and natural language inference (NLI; Bowman et al., 2015), across four benchmarks.
Several quantitative evaluations demonstrate the capabilities of R EV in providing evaluations along new dimensions for free-text rationales, while being more consistent with human judgements compared to existing metrics.
We also provide comparisons to demonstrate the sensitivity of R EV to various degrees of input perturbations.
Additionally, our evaluation with R EV offers insights into why rationales obtained through chain-of-thought prompting (Wei et al., 2022) do not necessarily improve prediction performance.
We will make our code and data public.
R EV : I NFORMATION -T HEORETIC E VALUATION OF R ATIONALES
We introduce a new metric, R EV , Rationale Evaluation with conditional V -information, for evaluation of free-text rationales on the proposed dimensions (§2.2), based on the information-theoretic framework of conditional V -information (§2.1).
We consider the setting where we have input X ∈ X , label Y ∈ Y , and free-text rationale R ∈ R generated for label Y .
A common strategy to evaluate rationales R is through an evaluator f ∈ V based on how much R helps f predict Y given X .
The evaluator f ∶ Z → Y maps a variable
Z to a label distribution.
The definition of Z depends on the evaluation framework; e.g., Z can be a concatenation of X and R .
The evaluator f is trained on a set of input, label and rationale triples D train = {( x j , y j , r j )} , and applied to D test =
{( x i , y i , r i )} for evaluation.
The utility of R is formulated as the difference between the performance of the evaluator on predicting Y with R , and without it, i.e.
[Equation]
A larger performance gap indicates a better rationale.
Existing metrics (Hase et al., 2020; Wiegreffe et al., 2021) compute the performance gap based on prediction accuracies, measuring how much R can help the evaluator correctly predict Y given X .
However, accuracy-based evaluation can only indicate whether or not a rationale is predictive of a label, but cannot quantify how much new information the rationale provides to justify the label .
Fig.
1 illustrates this issue via an example.
Accuracy-based evaluation can distinguish between ˆ r 1 ,a and ˆ r 1 ,b since ˆ r 1 ,a supports y 1 and ˆ r 1 ,b does not.
However, it is unable to distinguish between r ∗ 1 and ˆ r 1 ,a (since both are predictive of y 1 ), despite the fact that ˆ r 1 ,a does not provide any unique and relevant information to answer why the label should be y 1 .
In practice, vacuous rationales such as ˆ r 1 ,a are commonly seen in model generations (Sun et al., 2022; Wiegreffe & Marasovi´c, 2021).
This calls for an evaluation metric which is able to identify and penalize such vacuous rationales.
2. A N I NFORMATION -T HEORETIC P ERSPECTIVE ON R ATIONALE E VALUATION
The key quantity of interest for our evaluation of rationales R is the amount of new information expressed in R (e.g., background knowledge, reasoning process) that can justify a label Y .
The mutual information between R and Y , I ( Y ; R ) can be helpful for evaluating this quantity.
However, we are not interested in the information that is already captured in the input X .
A vacuous rationale, such as ˆ r 1 ,a in Fig.
1, which simply combines the input X and the label, Y , captures all the information in X and Y without specifying any new information to help understand why Y has been chosen for
X ; let us denote such rationales as B ∈ B .
Thus, we argue that a good evaluation metric must be able to measure the amount of relevant, new information contained in a rationale beyond what is contained in any vacuous rationale, B , that leads to the prediction of Y .
Then the new information in R beyond what is available in B can be grounded with conditional mutual information (Shannon, 1948) as follows,
[Equation]
where the difference of two information quantities demonstrates the performance gap in Equation 1.
Directly computing mutual information, however, is challenging because true distributions of random variables are usually unknown, and we do not have unbounded computation.
A recently introduced information-theoretic framework called V -information circumvents this by restricting the computation to certain predictive model families, V (Xu et al., 2020).
Our approach to evaluate rationales extends this framework, following (Hewitt et al., 2021), as described below.
Conditional V -information Given a model family V that maps two random variables R and Y ,
V -information defines the usable information that can be extracted from R by models in V to predict
Y , i.e. I V ( R → Y ) .
If V generalizes to the set of all possible functions, then V -information is mutual information (Shannon, 1948).
In practice, it is feasible to estimate the usable information from R about Y by selecting any neural model without frozen parameters as V .
Following conditional mutual information in information theory (Cover & Thomas, 2006), Vinformation has been extended to conditional V -information (CVI; Hewitt et al., 2021).
CVI quantifies the V -usable information in R about Y conditioned on a variable B , i.e.
[Equation]
Here B is any vacuous rationale that leads to the prediction of Y .
In this work, we consider B simply as the concatenation of X and Y .
We leave analyzing how different baseline construction impacts our metric to future work.
H V ( ⋅ ∣ ⋅ ) is the conditional V -entropy (Xu et al., 2020; Hewitt et al., 2021;
Ethayarajh et al., 2022), defined as
[Equation]
where f [ b ] and f [ r, b ] produce a probability distribution over the labels given b and [ r, b ] as inputs respectively.
3 Further, we consider pointwise CVI for evaluating individual samples, ( r, y, b ) as
[Equation]
2. C OMPUTING R EV FOR R ATIONALE E VALUATION
Building on the framework of CVI, we propose a new metric R EV , for Rationale Evaluation with conditional V -information.
We compute R EV over a given test set, D test = {( x i , y i , r i )} , by estimating CVI over the set with an evaluator f ∈ V .
For a test example ( x, y, r ) , the R EV score denoted as R EV ( x, y, r ) is computed based on Equation 5, where b is constructed by combining x and y .
[Equation]
The R EV score for the test corpus D test , is given by the average pointwise R EV score:
[Equation]
Algorithm 1 shows the process of computing both pointwise and aggregate R EV scores.
The higher the R EV score, the more additional ( new and relevant ) information the rationale r contains to explain the label beyond the baseline rationale b .
R EV ( x, y, r ) can take positive, negative, or zero values.
When R EV ( x, y, r )) > 0 , the rationale supplies additional information for supporting the label (e.g., r ∗ 1 in Fig. 1); when R EV ( x i , y i , r i ) = 0 , the rationale provides no additional information beyond the baseline (e.g., r 1 ,a in Fig. 1); and when R EV ( x i , y i , r i ) < 0 , the rationale contains additional information which does not support the label (e.g., r 1 ,b in Fig. 1).
R EV can assign a positive score to a rationale for an incorrect prediction as long as the rationale supports it and provides additional information beyond a vacuous baseline rationale.
Thus, R EV cannot be seen as a replacement for prediction accuracy, but rather as an orthogonal metric to interpret the usefulness of a generated rationale for the model decision.
Constructing a Baseline with Vacuous Rationales Given an input x and a label y , we construct a baseline rationale b by converting x and y into a declarative sentence.
For the CQA task, we adopt a pre-trained T5-3B model fine-tuned on a set of ( question , answer , declarative sentence ) tuples (Chen et al., 2021) 4 annotated by Demszky et al.
For the NLI task, we use a template to convert ( premise , hypothesis , label ) tuple into a baseline rationale: “ premise implies / contradicts / is not related to hypothesis ”.
Table 1 shows some examples of constructed vacuous rationales.
[Table]
Training the evaluator, f
We select a generative model f ∈ V as the evaluator to learn the mapping f ∶ [ r, b ] → y , where the input [ r, b ] is the concatenation of r and b . {+
+}
In particular, we use pre-trained language models (e.g., T5; Raffel et al., 2020) and fine-tune them on the training set D train =
{( x, y ∗ , r ∗ )} , where { y ∗ } and { r ∗ } are gold labels and human-annotated rationales, respectively.
We construct baseline rationales { b ∗ } based on {( x, y ∗ )} .
The objective is to maximize the log-likelihood of y ∗ given r ∗ and b ∗ .
After training, the evaluator can be applied to evaluate a given rationale-label pair ( y, r ) w.r.t.
an input x .
The rationale-label pair ( y, r ) can be model-generated and the label may not be groundtruth (e.g., y 2 in Fig. 1), while R EV is still able to provide an assessment on the rationale along the two dimensions (§1), e.g., R EV ( x, y 2 , ˆ r 2 )
= 0 .
6 in Fig.
E XPERIMENTAL S ETUP
We outline our experimental setup by describing the reasoning tasks and datasets (§3.1), followed by the task and evaluation models (§3.2), and the baseline metrics for comparison (§3.3).
Additional details on the setup are provided in Appendix B.
3. D ATASETS
We explore two reasoning tasks, namely CommonsenseQA (CQA) and Natural Language Inference (NLI) across four datasets, all containing human-annotated free-text rationales.
For CQA task, we use ECQA (Aggarwal et al., 2021), CoS-E (v1.11; Rajani et al., 2019) and QuaRTz (Tafjord et al., 2019).
For both ECQA and CoS-E, each commonsense question is paired with five candidate choices and the task is to select an answer from the candidates.
ECQA contains higher quality human-written rationales compared to CoS-E (Aggarwal et al., 2021; Sun et al., 2022).
QuaRTz is for open-domain reasoning about textual qualitative relationships, and the task is to select an answer from two options to the question based on the textual qualitative knowledge (rationale).
e-SNLI (Camburu et al., 2018) provides explanations for the NLI, where given a premise, the task is to predict if a hypothesis entails, contradicts or is neutral to it.
More details of the datasets are in Appendix B.1.
3. T ASK AND E VALUATION M ODELS
Task models We choose T5 Large (Raffel et al., 2020) as the task model (finetuned on ground truth labels and rationales) to produce generated rationale-label pairs under three settings:
• XY ∗ → R:
Given an input text and the gold label, generate a rationale.
• X
→ YR: Given an input text, generate a label followed by a rationale.
Since T5 decodes tokens sequentially, each R is generated conditioned on the predicted Y. • X → RY: Given an input text, generate a rationale followed by a label.
Here, we compute a likelihood for each candidate Y conditioned on R, and then select the most probable candidate.
This operation can improve the model prediction accuracy, while weakening the consistency and relevance between the generated rationales and predicted labels.
After training, we collect three types of rationale-label pairs by applying the three task models on the test set of each dataset.
In addition to these three settings, we also evaluate ground-truth labels paired with crowd-sourced rationales (Y ∗ ;R ∗ ).
Evaluators Our evaluator, f
(see Equation 6 in §2) is also based on T5 Large trained on gold rationale-label pairs of the respective dataset.
We refer readers to the Appendix C.1 for results using
T5 Base, BART Large (Lewis et al., 2020) and GPT-2 Large (Radford et al., 2019) as the evaluator.
3. O THER M ETRICS FOR R ATIONALE E VALUATION
We compare with two existing automatic metrics for free-text rationale evaluation: LAS (Hase et al., 2020) and RQ (Wiegreffe et al., 2021).
Analogous to our evaluator, f , both approaches use proxy models; we use the same architecture (T5 Large) across metrics in our reported results.
Leakage-Adjusted Simulatability (LAS) Hase et al.
(2020) evaluate the quality of free-text rationales via a proxy model, trained with the task model outputs as labels and original input texts combined with rationales as input sequences.
The metric computes the difference between its prediction accuracy on the predicted label when the rationale is included into the input vs. when it is not, 1 [ ˆ y ∣ x, ˆ r ] − 1 [ ˆ y ∣ x ] , averaged over examples grouped based on whether they leak labels or not.
The final LAS score is given by the macro average across groups.
Rationale Quality (RQ) Wiegreffe et al.
(2021) propose a variant of the simulatability in Hase et al.
The main difference is that gold labels are used to train the model proxy and evaluate rationale quality.
Specifically, the quality of a rationale ˆ r is measured as 1 [ y ∗ ∣ x, ˆ r ] − 1 [ y ∗ ∣ x ] , where y ∗ is the gold label.
Similarly, RQ is the average score over all test examples.
E XPERIMENTS
We first compare R EV with existing metrics (§4.1) and human judgments (§4.2) on the ECQA dataset, as well as show R EV on other CQA and NLI benchmarks.
We then test the sensitivity of different metrics to input perturbations (§4.3).
Next, we apply R EV to generations via few-shot prompting (4.4).
Additional experiments are listed in Appendix C.
4. C OMPARISON B ETWEEN E VALUATION M ETRICS
We compare R EV to LAS and RQ, in evaluating different rationale-label pairs on the ECQA dataset.
In addition to Y ∗ ;R ∗ , XY ∗ → R, X → YR and X → RY, we also explore the evaluation on vacuous baseline rationales (Y ∗ ;B), which simply combine inputs and labels with no additional information.
Note that the scores obtained from different metrics are not directly comparable due to different comparison scales and criteria (e.g., log-probability vs. accuracy).
We mainly focus on the ranking over different types of rationale-label pairs.
The results averaged over 4 random seeds are shown in the left part of Fig.
Qualitative results are provided in Table 5 in Appendix C.2.
All three metrics agree that the crowdsourced rationales (Y ∗ ;R ∗ ) in the ECQA have the highest quality.
While by definition, R EV for vacuous rationales is low, both LAS and RQ scores for these rationales are quite high, showing that these metrics are incapable of measuring the amount of additional information in rationales.
Intuitively, we expect weaker rationale-label consistency in
X → RY setting compared to X → YR, as the labels are forcefully selected among the candidates as opposed to being freely generated by the task model (§3.2).
While R EV is able to capture this intuition and rank X → YR higher than X → RY, LAS and RQ have a different ranking.
[Table]
Next, we apply R EV to evaluate crowdsourced and model generated rationale-label pairs (Y ∗ ;R ∗ , XY ∗ → R, X → YR, X → RY) across different datasets.
For each dataset, the evaluator is trained on the training set with gold labels and crowdsourced rationales.
The results are shown in Table 2.
We observe that the gold rationales in the ECQA dataset achieve higher
R EV score than those in CoS-E. This observation is in line with the known quality issues of crowdsourced rationales in CoS-E (Aggarwal et al., 2021; Sun et al., 2022).
Moreover, training the evaluator with CoS-E results in lower R EV for all models, compared to training with ECQA.
Interestingly, model-generated rationales (XY ∗ → R) have higher R EV scores than crowdsourced rationales for CoS-E (see examples in Table 6), and similar R EV scores for ECQA, QuaRTz and e-SNLI.
For QuaRTz, the model generated rationales seem to not contain much new and relevant information over a vacuous baseline.
In the case of e-SNLI, the problem is even severer as most of the crowdsourced or generated rationales do not provide reasoning but rather follow a label-specific template e.g., A implies (that) B (Kumar & Talukdar, 2020a; Brahman et al., 2021). [Figure]
4. H UMAN E VALUATION
To understand how R EV correlates with human judgments of rationales, we conduct a crowdsourcing experiment via Amazon Mechanical Turk.
We randomly sample 230 examples from the ECQA test set and ask workers to evaluate the four types of rationale-label pairs (Y ∗ ;R ∗ , XY ∗ → R, X → YR,
X → RY) for each example.
6 We present workers with a question (input text), an answer (label) and an explanation (rationale), and ask them whether the explanation justifies the answer ( yes/no ).
If they answer yes , we further ask them to evaluate the amount of additional information supplied by the explanation that explains why the answer might have been chosen for the question.
The workers choose from none / little / some / enough , corresponding to a 4-point Likert-scale.
We collect 3 annotations per instance and use majority vote to decide whether the rationale can justify the label.
If yes , we take the average over the 3 human-annotated scores as the amount of information.
Otherwise, we give a score of -1.
More details of human evaluation are in Appendix C.6.
The results are shown in the right part of Fig.
2, where the ranking of the four types of rationalelabel pairs is Y ∗ ;R ∗ > XY ∗ → R > X
→ YR > X → RY.
While LAS and RQ rank X → RY better than X → YR (see the left part of Fig. 2), the ranking from R EV is more consistent with human judgments, suggesting its effectiveness in evaluating rationale quality.
4. I S R EV SENSITIVE TO INPUT PERTURBATIONS ?
We test the sensitivity of all automatic metrics for rationale evaluation metrics to input ( X ) perturbations in the task model, under two settings: X → YR and X → RY.
Following Wiegreffe et al.
(2021), we add zero-mean Gaussian noise N ( 0 , σ 2 ) to input word embeddings during inference, inducing task models to produce progressively degenerate rationales and labels.
A good metric should be sensitive to the change of rationales and labels and reflect their relationships under input perturbations.
R EV and RQ show similar trends as for X → RY in Fig.
3 (b) and (c).
However, LAS is less sensitive to noise for both joint models, X → RY and X → YR, in Fig.
3 (a) and (d).
Since the proxy model for LAS is trained on the task models’ predicted labels and generated rationales, it can overfit to the degenerate rationale-label pairs under input perturbations, hence being less sensitive to input noise during inference.
The largest differences between R EV and RQ are for X → YR.
We observe the task model can predict incorrect labels and then make up reasonable-sounding rationales for its wrong predictions under certain input perturbations (e.g., when σ 2 ≤ 20 ); prior work also reports this finding (Narang et al.,
2020; Wiegreffe et al., 2021).
R EV does not drop under a certain amount of input perturbations (e.g., σ 2 ≤ 20 ) in Fig.
3 (f), likely because the generated rationales still provide new information for describing both correct and incorrect labels (also see the example in Table 8).
However, as the noise exceeds the certain level, R EV decreases indicating that the task model is perhaps no longer able to make up rationales for very noisy inputs.
On the other hand, the behaviors of RQ and R EV are quite different in Fig.
3 (e) and (f).
Since RQ is computed based on gold labels (§3.3), it has reduced sensitivity to input perturbations.
When the prediction accuracy decreases, the overall evaluation of
RQ is dominated by the results on incorrect predictions, as shown in Fig.
3 (e).
We refer readers to the Table 8 in Appendix C.5 for qualitative analysis on sensitivity test.
4. E VALUATING R ATIONALES IN F EW - SHOT P ROMPTING
We test the transferability of R EV in evaluating rationales generated by few-shot prompting, and get insights on the reasoning and prediction processes of large language models (e.g., GPT-3).
GPT-3 Rationales for Gold Labels.
Wiegreffe et al.
(2022) collected 250 high quality free-text rationales generated by few-shot prompting with GPT-3 (Brown et al., 2020) for CQA (given gold labels).
Each example was assessed by 3 crowdworkers.
We focus on two aspects of their annotations: “supports the gold label” and “amount of information”.
Crowdworkers provide a yes / no answer to justify whether a rationale supports the corresponding gold label.
Only when the answer is yes , they are further asked to evaluate the amount of information contained in the rationale for justifying the label.
The amount of information is roughly categorized into 3 levels: “Not Enough”, “Enough”, “Too Much”, each annotated with a Likert-scale score.
In Fig.
4, we compare human annotation scores for amount of information 8 with the pointwise scores obtained by three automatic metrics, LAS, RQ, and R EV .
For the automatic metrics, our evaluator, f and the proxy models of
LAS and RQ are trained on the ECQA training set with gold labels and human-annotated rationales.
We observe that R EV provides finer-grained assessment of the information contained in rationales compared to LAS and RQ which only take { -1, 0, 1 } values.
When LAS and RQ are zero, it is unclear whether the rationale supports the label or not because the model proxy may predict the label based on the input only.
The judgments of R EV on whether rationales support labels (R EV >) are close to human judgments (i.e., 69% vs. 73% support rate).
The support rates of LAS and RQ are relatively low, i.e. 35% and 23% , while a large portion ( 56% and 60% respectively) corresponds to a zero LAS / RQ score.
Chain of Thought Rationales.
Wei et al.
(2022) propose chain of thought prompting to teach large language models to produce intermediate reasoning steps (rationales) before prediction, which improves their prediction performance on a range of reasoning tasks (e.g., arithmetic and symbolic reasoning).
However, the reported improvement is trivial for CQA (Wei et al., 2022), which motivates us to evaluate the intermediate rationales w.r.t.
model predictions.
We apply R EV to analyze the generated rationales during intermediate reasoning steps and final predicted labels by GPTtext-davinci-002 (Brown et al., 2020) and LaMDA 137B
(Thoppilan et al., 2022).
Fig.
5 shows the distributions of R EV for correctly and incorrectly predicted instances from GPTand LaMDA, respectively.
The prediction accuracy of GPT-3 is much higher than that of LaMDA ( 77% vs. 59% ), while the average R EV scores over all instances are close (0.80 vs. 0.84).
For both
GPT-3 and LaMDA, the R EV distributions of correct and incorrect predictions are similar and most instances have positive R EV scores.
The results demonstrate the causality between the models’ intermediate reasoning process and their final predictions, no matter whether the predicted labels are correct or incorrect.
The average R EV scores (dashed lines) over correct and incorrect predictions are close, especially for GPT-3.
This is consistent with our observation that most generated rationales from the two models are describing their predicted labels.
An insight we obtain is that the generated intermediate reasoning steps (rationales) support models’ predictions (consistent R EV scores), but cannot guarantee their correctness (discrepant accuracies).
This partially explains the minor improvement of chain of thought prompting on CQA in Wei et al.
R ELATED W ORK
Model rationales broadly fall into two categories: extractive rationales and free-text rationales.
Extractive rationales contain some important features extracted from input texts that make models produce final predictions (Lei et al., 2016; DeYoung et al., 2020; Jain et al., 2020; Schulz et al., 2019).
Free-text rationales are produced by generative models in the form of natural language.
Compared to extractive rationales, free-text rationales explain model predictions in a more human-like way and fill the gap in explaining reasoning tasks (Camburu et al., 2018; Narang et al., 2020; Rajani et al., 2019; Kumar & Talukdar, 2020b; Brahman et al., 2021).
Evaluations on extractive rationales have been well studied, generally from two perspectives — faithfulness and plausibility (DeYoung et al., 2020; Pruthi et al., 2022; Chan et al., 2022b).
Faithfulness measures to which extent rationales reflect the true reasoning process of models, while plausibility evaluates how convincing rationales are to humans (Jacovi & Goldberg, 2020).
Other perspectives include the ability of rationales in helping a student model simulate a teacher model (Pruthi et al., 2022) or bridging the communication between a classifier and a layperson (Treviso & Martins, 2020).
Existing automatic metrics for free-text rationales focus on rationale-label association, and measure the utility of a rationale based on how much it helps a model proxy predict the given label (inspired by human simulatability Doshi-Velez & Kim (2017)) (Hase et al., 2020) or the gold label (Wiegreffe et al., 2021) given the input.
Chan et al.
(2022a) further propose a framework to evaluate the automatic metrics.
However, none of them consider measuring the amount of additional information in free-text rationales.
Sun et al.
(2022) conduct a human study on the additional knowledge provided by free-text rationales.
This work is the first that proposes an automatic metric to quantify the additional information in free-text rationales.
C ONCLUSION
In this paper, we propose an information-theoretic metric, R EV , to evaluate free-text rationale.
R EV measures if a rationale contains new information that is relevant for the label of interest, beyond the information contained in the input.
We show the advantage of R EV in evaluating different types of rationale-label pairs compared to existing metrics.
We demonstrate that the evaluation of free-text rationales with R EV is consistent with human judgments.
R EV also offers insights on evaluating rationales generated via few-shot prompting.
In its current formulation, R EV might reward a rationale for an incorrect prediction as long as the rationale supports the prediction with relevant additional information.
Future work might explore evaluation that penalizes rationales which support incorrect predictions, thus bridging together predictive performance with interpretability metrics.
A P ROPERTIES OF C ONDITIONAL V - INFORMATION
As proved by Hewitt et al.
(2021), CVI has several useful properties:
Non-Negativity : I V ( R → Y ∣ B ) ≥ 0 . 2.
Independence : If Y and B are jointly independent of R , then I V ( R → Y ∣ B ) = 0 .
Monotonicity :
If U ⊆ V , then H V ( Y ∣ B ) ≤ H U
( Y ∣ B ) .
An implication from Monotonicity is complex models (e.g., pre-trained language models) can do better than simpler ones (e.g., linear models) in estimating V -usable information.
Since CVI measures the additional V -usable information in R about Y beyond what’s already extracted from B by models in V , it grounds the goal of the proposed metric R EV .
B S UPPLEMENT OF E XPERIMENTAL S ETUP B. D ATASETS
1: Input : evaluator f , test set D test =
{( x i ,y i ,r i )} 2: Initialize an empty list S 3: for ( x i ,y i ,r i )
∈ D test do 4:
Construct the baseline rationale b i 5: R EV ( x i ,y i ,r i ) = log f [ r i ,b i ]( y i ) − log f [ b i ]( y i ) 6:
S . add ( R EV ( x i ,y i ,r i )) 7: end for 8: R EV = sum( S ) /∣ S
∣ 9: Output : S , R EV For CQA task, we use ECQA (Aggarwal et al., 2021), CoS-E (v1.11) 10 (Rajani et al., 2019) and QuaRTz (Tafjord et al., 2019). Both ECQA and CoS-E originate from the CommonsenseQA dataset (Talmor et al., 2018), where each commonsense question is paired with 5 candidate choices and the task is to select an answer from the candidates.
ECQA contains higher quality free-text rationales compared to CoS-E, in terms of comprehensiveness, coherence, non-redundancy, etc.
(Aggarwal et al., 2021; Sun et al., 2022).
QuaRTz is an open-domain reasoning task about textual qualitative relationships.
Each instance contains a situated qualitative question, two answer options and a knowledge statement.
The task is to select an answer from the two options to the question based on the textual qualitative knowledge.
We use the knowledge statement as a free-text rationale since it explains why the answer is to the question.
For NLI task, we use e-SNLI (Camburu et al., 2018) which is an extension of SNLI (Bowman et al., 2015) with augmented free-text human-written rationales.
The task is to predict the entailment relationship between a premise and a hypothesis.
Table 3 shows the summary statistics of the four datasets.
[Table]
B. M ODELS
We use Huggingface Transformers (Wolf et al., 2020) to access all task models and evaluators.
We train each model for up to 30 epochs with a learning rate 5 e − 6 and a batch size 8 .
All experiments were performed on a single NVIDIA RTX 8000 GPU.
Table 4 shows input-output formattings of different task models for different tasks. [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
C S UPPLEMENT OF E XPERIMENTS C. C OMPARISON B ETWEEN E VALUATOR A RCHITECTURES
We apply R EV to evaluate different types of free-text rationales w
[Table]
r
[Table]
t. labels on the ECQA dataset.
Figure 6 shows R EV scores of the four types of rationale-label pairs evaluated by the four evaluators.
The ranking of the four groups of rationale-label pairs is consistent across the four evaluators,
i
[Table]
e.
Y ∗ ;R ∗ > XY ∗ → R > X
→ YR > X → RY.
This ranking is also consistent with human evaluation in §4.2.
Since ECQA contains high-quality crowdsourced rationales (Aggarwal et al., 2021), it is expected that the R EV of gold rationale-label pairs (Y ∗ ;R ∗ ) is the highest.
The R EV of XY ∗ → R is close to that of Y ∗ ;R ∗ , indicating the task model (T5 Large) can produce good quality rationales when it is prompted with ground-truth labels.
All four evaluators agree that the generated rationales of X → YR contain more additional background information for explaining the predicted labels than those of X → RY.
This is consistent with our design of the X → RY in §3.3, where the generated rationales and labels have weakened relevance.
For each type of rationale-label pairs, the four evaluators capture different amount of conditional V -information, while T5 Large consistently outperforms other three models.
In the reported experiments §4, we use T5 Large as the evaluator.
C. Q UALITATIVE A NALYSIS OF D IFFERENT M ETRICS
Table 5 shows the qualitative analysis of different metrics on the four types of rationale-label pairs (Y ∗ ;R ∗ , XY ∗ → R, X → YR, X → RY) on the ECQA dataset.
R EV provides more accurate evaluations on those examples than LAS and RQ.
[Table]
C. Q UALITATIVE A NALYSIS OF C O S-E R ATIONALES
Table 6 shows the exemplar of REV scores for crowdsourced and model-generated (XY ∗ → R) rationales for CoS-E. The main observation is model-generated rationales (XY ∗ → R) generally support labels, though provide limited new information, while many crowdsourced rationales in CoS-E are noisy or uninformative.
Specifically, compared to the crowdsourced rationales in CoS-E, we observe that XY ∗ → R can produce better rationales that support the labels, which also corresponds to higher
REV scores.
However, the new information contained in those rationales is still limited (please see examples).
A possible reason is the task model (XY ∗ → R) hardly learns to produce more informative rationales when trained using lower quality rationales from CoS-E, known quality issue as reported in prior work (Aggarwal et al., 2021; Sun et al., 2022).
[Table]
C. Q UALITATIVE A NALYSIS OF N EGATIVE R EV S CORES
Table 7 shows some examples of X → RY with negative R EV scores on the ECQA dataset.
When
R EV < 0 , we observe in most cases the rationale does not support the given label, while indicating other labels, or something even beyond the label candidates (e
[Table]
g., “helicopter” in the second example).
Or they could repeat the input (e.g., the first example).
The same observation holds for other types of rationale-label pairs
[Table]
C. Q UALITATIVE R ESULTS OF S ENSITIVITY T EST
Table 8 shows some examples from the sensitivity test in §4.3.
C. H UMAN E VALUATION D ETAILS
We randomly select 230 examples from the ECQA test set and conduct human evaluation on the four types of rationale-label pairs (Y ∗ ;R ∗ , XY ∗ → R, X → YR, X → RY) w.r.t.
each example through the Amazon Mechanical Turk (AMT).
Each instance is assessed by 3 workers.
We pay the workers $ 0 .
08 for assessing each instance.
Figure 7 shows the instructions we provide to workers.
In Figure 8, we show three examples, illustrating when the explanation (rationale) does not justify the answer (label), when the explanation supports the answer while not supplying additional information, and when the explanation supports the answer and provides additional information.
Figure 9 shows the interface of the actual hit for human evaluation.
For each instance, we provide a question (input), an answer (label), and an explanation (rationale), and ask the workers to answer the following two questions:
Does the Explanation justify the given Answer?
(yes or no)
The question is to ask workers to judge whether the rationale supports the label or not.
If yes, how much additional information does the Explanation have to justify the Answer beyond just reiterating what is stated in Question and Answer?
(No additional info, Little additional info, Some additional info, Enough additional info)
We only ask this question if the workers choose “yes” for the first question.
We design this question to ask workers to evaluate the extent to which the rationale provides additional information for justifying the label beyond repeating it w.r.t.
the input.
C. A DDITIONAL A NALYSIS ON L ABEL -R ELATED S ENTENCES
In some cases, a rationale contains the given label and provides new information related to the label, but does not necessarily explain why the label is selected for the input.
To evaluate such rationales, we randomly select 250 gold labels in ECQA and extract their related sentences from a large-scale knowledge base—GenericsKB (Bhakthavatsalam et al., 2020). Those sentences contain the labels, while providing little or irrelevant new information to explain the labels w.r.t. the inputs. We use them as trivial rationales for evaluation.
The average R EV scores for those trivial rationales and their crowdsourced counterparts are 0.02 and 0.87 respectively, indicating the effectiveness of R EV in identifying the new and relevant information in rationales.
Table 9 shows the REV scores of some examples and the corresponding crowdsourced rationales.
The results show that R EV can distinguish the new information in different rationales and penalize meaningless rationales.
Overall, R EV gives higher scores to crowdsourced rationales than trivial sentences from GenericsKB.
[Table]
[Table]