SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Wei Liu $^{1}$  Jun Wang $^{2*}$  Haozhao Wang $^{1*}$  Ruixuan Li $^{1*}$
Zhiying Deng<sup>1</sup> Yuankai Zhang<sup>1</sup> Yang Qiu<sup>1</sup>
$^{1}$  School of Computer Science and Technology, Huazhong University of Science and Technology
$^{2}$ iWudao Tech
$^{1}\{idc\_lw, \text{hz\_wang}, \text{rxli}, \text{dengzhiyingdd}, \text{yuankai\_zhang}, \text{anders}\} @hust.edu.cn$
$^{2}$ jwang@iwudao.tech
Rationalization is a self-explaining framework for NLP models. Conventional work typically uses the maximum mutual information (MMI) criterion to find the rationale that is most indicative of the target label. However, this criterion can be influenced by spurious features that correlate with the causal rationale or the target label. Instead of attempting to rectify the issues of the MMI criterion, we propose a novel criterion to uncover the causal rationale, termed the Minimum Conditional Dependence (MCD) criterion, which is grounded on our finding that the non-causal features and the target label are  $d$ -separated by the causal rationale. By minimizing the dependence between the unselected parts of the input and the target label conditioned on the selected rationale candidate, all the causes of the label are compelled to be selected. In this study, we employ a simple and practical measure of dependence, specifically the KL-divergence, to validate our proposed MCD criterion. Empirically, we demonstrate that MCD improves the F1 score by up to  $13.7\%$  compared to previous state-of-the-art MMI-based methods. Our code is available at: https://github.com/jugechengzi/Rationalization-MCD.
With the success of deep learning, there is growing concern about the interpretability of deep learning models, particularly as they are rapidly being deployed in various critical fields (Lipton, 2018). Ide
ally, the explanation for a prediction should be both faithful (reflecting the model's actual behavior) and plausible (aligning with human understanding) (Chan et al., 2022).
Post-hoc explanations, which are trained separately from the prediction process, may not faithfully represent an agent's decision, despite appearing plausible (Lipton, 2018). Sometimes, faithfulness should be considered a prerequisite that precedes plausibility in explanations of neural networks, especially when these networks are employed to assist in critical decision-making processes, as this factor determines the trustworthiness of the explanations. In contrast to post-hoc methods, ante-hoc (or self-explaining) techniques typically offer increased transparency (Lipton, 2018) and faithfulness (Yu et al., 2021), as the prediction is made based on the explanation itself.
Figure 1: The standard rationalization framework RNP.  $X$  is the original full text.  $X_Z$  is the selected rationale candidate and  $\hat{Y}$  is the predictor's output.
*Corresponding authors. This paper is a collaboration between Intelligent and Distributed Computing Laboratory, Huazhong University of Science and Technology and iWudao Tech.
A model-agnostic ante-hoc explanation framework, called Rationalizing Neural Predictions (RNP), was proposed by Lei et al. (2016) and is also known as rationalization. RNP utilizes a cooperative game between an explainer and a predictor, where the explainer identifies a human-interpretable subset of the input (referred to as rationale) and passes it to the subsequent predictor for making predictions, as shown in Figure 1. The explainer and predictor are trained cooperatively to maximize prediction accuracy. A significant advantage of RNP-based rationalization is its certification of exclusion, which guarantees that any unselected part of the input has no contribution to the prediction. This property ensures the maintenance of faithfulness, enabling us to focus solely on plausibility (Yu et al., 2021). Notably, although RNP was initially proposed in the field of NLP and and its enhancement schemes have primarily been validated using text data, its framework can also be applied to other domains, e.g., explaining image classification (Yuan et al., 2022) and graph neural networks (Luo et al., 2020).
Previous rationalization methods generally utilize the maximum mutual information (MMI) criterion to determine the rationale, defined as the subset most indicative of the target label. However, this criterion merely uncovers associations rather than causal relationships between the rationale and the label. Consequently, MMI is easily affected by spurious correlations and the plausibility of chosen rationales might be diminished, even though the rationales still faithfully report the predictor's behavior (Chang et al., 2020). In rationalization, there are two stages from which correlations may arise. The first type of correlation originates from the process of dataset generation, and we refer to it as feature correlation. A typical example of feature correlation, as pointed out in LIME (Ribeiro et al., 2016), is that wolves often appear together with snow. Consequently, whether the background features snow or not can serve as a strong indicator for classifying an image as depicting a wolf. Another instance of feature correlation is demonstrated in the first row of Table 1. Within a beer review, a favorable taste often correlates with an appealing aroma. Comments regarding the taste can serve as a strong indicator for the smell label. The predictor might inadvertently overfit to such correlations, leading to local optima. Consequently, the suboptimal predictor could mislead the explainer to select these spurious correlations. Another type of correlation stems from the rationale (mask) selection stage, and we call it mask correlation. An example of mask correlation is depicted in the second row of Table 1. Consider a situation where the explainer has implicitly learned the category of  $X$ , and selects a “-” for all negative inputs while excluding it from all positive inputs. In this case, the predictor only needs to determine whether the input rationale includes a “-” or not. Even though this phenomenon has not been analyzed from the perspective of spurious correlation, it has been observed and named as degeneration in prior research (Yu et al., 2019).
Some methods have been developed to address either feature correlation or degeneration separately. INVRAT (Chang et al., 2020) attempts to tackle feature correlation using invariant risk minimization (IRM) (Arjovsky et al., 2019). The main idea is to emphasize spurious (non-causal) variations by splitting the dataset into distinct environments. However, IRM-based methods have several limitations. For instance, they require strong prior knowledge about the relationships between non-causal and causal features (e.g., the extra labels of non-causal features) in order to divide the dataset (Lin et al., 2022b). Moreover, IRM-based methods are limited to addressing only a finite set of predetermined non-causal features, neglecting the potential existence of numerous unknown non-causal features. In fact, a recent study (Lin et al., 2022b) in the field of IRM has theoretically demonstrated that it is nearly impossible to partition a dataset into different environments to eliminate all non-causal features using IRM. Other challenges, such as the tendency to overfit, difficulty in applying to larger models (Zhou et al., 2022; Lin et al., 2022a), and the marginal shift risk of the input (Rosenfeld et al., 2021), have also been identified within the realm of IRM. Inter_RAT (Yue et al., 2023) attempts to eliminate feature correlation through backdoor adjustment, intervening directly with the confounders. However, it is extremely hard to measure the confounders since they are usually not observable in the dataset. As for degeneration, although not explicitly associated with spurious correlation until this study, some efforts have been tried to fix the problem. The common idea is to introduce auxiliary modules that have access the full texts to regularize the original explainer (Yu et al., 2021) or the predictor (Yu et al., 2021; Liu et al., 2022). Regularized by these auxiliary modules, the predictor can somewhat disregard the mask correlation, and degeneration is partially alleviated.
Although these methods have tried to fix the problems resulted from spurious correlations, they are still MMI-based methods and how the problems come into being is not well explored. In this study, we first identify the two stages that the spurious correlations may come from, and link the important degeneration problem to a more general mask correlation. Then, we identify that the target label  $Y$  and all the non-causal features (including the rationale masks) in the input  $X$  are  $d$ -separated by the
Table 1: The examples of feature correlation and mask correlation. Human-annotated rationales are underlined. Rationales from RNP are highlighted in red.
RNP
Dataset: Beer-Aroma. Label: Positive. Predition: Positive. Problem: feature correlation
Text: the appearance was nice . dark gold with not much of a head but nice lacing when it started to dissipate
the smell was ever so hoppy with a hint of the grapefruit flavor that ’s contained within . the taste was interesting , up front tart grapefruit , not sweet in the least . more like grapefruit rind even . ……
Dataset: Beer-Aroma. Label: Negative. Predition: Negative. Problem: mask correlation
Text: 12 oz bottle poured into a pint glass - a - pours a transparent , pale golden color . the head is pale white with no cream , one finger ’s height , and abysmal retention . i looked away for a few seconds and the head was gone s - stale cereal grains dominate . hardly any other notes to speak of . very mild in strength t
- sharp corn/grainy notes throughout it ’s entirety . ……
causal features, meaning that the non-causal features are independent of  $Y$  given the causal features. This leads to a new avenue for addressing feature correlation and mask correlation simultaneously: we only need to penalize the dependence between the target label and the unselected features conditioned on the selected rationale candidate, such that all the direct causal features will be included in the selected rationale candidate. Based on this observation, we develop a new criterion for the causal rationale, namely minimum conditional dependence (MCD). Various methods can be adopted to measure the dependence, such as mutual information, the Hilbert-Schmidt Independence Criterion (HSIC) (Gretton et al., 2007), and so on. In this paper, we adopt a simple and practical measurement for independence, the KL-divergence, to verify the effectiveness of the proposed criterion. Then, we conduct experiments on two widely used benchmarks to validate the effectiveness of MCD.
- To the best of our knowledge, we are the first to identify the degeneration problem as a form of spurious correlation. Leveraging probabilistic graphical models, we are the first to comprehensively elucidate feature correlation and degeneration under a unified perspective.
- We find that the target label and non-causal features are  $d$ -separated by the direct causal features. Based on this insight, we propose the MCD criterion, which opens a new avenue for discovering causal rationales, marking the main contribution of this study. Unlike previous methods, MCD-based methods do not require prior expert knowledge about non-causal features, thus presenting potential for broader applicability.
- We present a simple and practical architecture to develop an MCD-based method. Experiments across various datasets demonstrate that our approach achieves an improvement of up to  $13.7\%$  in F1 score compared to state-of-the-art MMI-based rationalization methods.
Rationalization. The basic cooperative framework of rationalization called RNP (Lei et al., 2016) is flexible and offers a unique advantage: certification of exclusion, which means any unselected input is guaranteed to have no contribution to the prediction (Yu et al., 2021). Based on this cooperative framework, many methods have been proposed to improve RNP from various aspects. Bao et al. (2018) used Gumbel-softmax to do the reparameterization for binarized selection. Bastings et al. (2019) replaced the Bernoulli sampling distributions with rectified Kumaraswamy distributions. Jain et al. (2020) disconnected the training regimes of the generator and predictor networks using a saliency threshold. Paranjape et al. (2020) imposed a discrete bottleneck objective to balance the task performance and the rationale length. Chang et al. (2019) tried to select class-wise rationales. Antognini et al. (2021); Antognini and Faltings (2021) tried to select rationales belonging to different aspects at once. Zheng et al. (2022) called for more rigorous evaluation of rationalization models. Fernandes et al. (2022) leveraged meta-learning techniques to improve the quality of the explanations. Havylov et al. (2019) cooperatively trained the models with standard continuous and discrete optimization schemes. Hase et al. (2020) explored better metrics for the explanations. Rajagopal et al. (2021) used phrase-based concepts to conduct a self-explaining model. Other methods like data augmentation with pretrained models (Plyler et al., 2021), training with human-annotated rationales (Chan et al., 2022), have also been tried. These methods are orthogonal to our research.
Spurious correlations. Several methods have been proposed to address the issues arising from either feature correlation or mask correlation. The impact of feature correlation is somewhat mitigated by techniques such as invariant risk minimization (Chang et al., 2020) or backdoor adjustment (Yue et al., 2023). However, as indicated in the introduction, these methods have certain limitations. To combat mask correlation, the usual strategy involves introducing an auxiliary module, which has access to the full input, to regulate the original modules and prevent them from overfitting to trivial patterns introduced by the explainer (Yu et al., 2021, 2019; Liu et al., 2022). Other methods like using multiple explainers to select diverse rationales (Liu et al., 2023a), assigning asymmetric learning rates for the two players (Liu et al., 2023b), have also been tried. Unfortunately, these methods have limited effectiveness against feature correlation in the input data. These aforementioned methods are most relevant to our research, yet we are the first to consider both feature correlation and mask correlation from a unified perspective.
We consider the text classification task, where the input is a text sequence  $X = [x_{1}, x_{2}, \dots, x_{l}]$  with  $x_{i}$  being the  $i$ -th token and  $l$  being the number of tokens. The label of  $X$  is a one-hot vector  $Y \in \{0, 1\}^{c}$ , where  $c$  is the number of categories.  $\mathcal{D}$  represents the training set. Ante-hoc rationalization consists of an explainer  $f_{E}(\cdot)$  and a predictor  $f_{P}(\cdot)$ , with  $\theta_{e}$  and  $\theta_{p}$  representing the parameters of the explainer and predictor, respectively. The goal of an MMI-based explainer is to select the most indicative pieces from the input that are related to the label.
For  $(X,Y)\sim \mathcal{D}$ , the explainer first outputs a sequence of binary mask  $M = f_{E}(X) = [m_{1},\dots,m_{l}]\in$ $\{0,1\} ^l$  (in practice, the explainer first outputs a Bernoulli distribution for each token and the mask for each token is independently sampled using gumbel-softmax). Then, it forms the rationale candidate  $X_{Z}$  by the element-wise product of  $X$  and  $M$ :
$$
X _ {Z} = M \odot X = \left[ m _ {1} x _ {1}, \dots , m _ {l} x _ {l} \right]. \tag {1}
$$
To simplify the notation, we denote  $f_{E}(X)$  as  $X_{Z}$  in the following sections, i.e.,  $f_{E}(X) = X_{Z}$ . With the generator's selection, we get a set of  $(Z,Y)$  pairs, which are generally considered to be samples taken from the distribution  $P(Z,Y)$ . Then, vanilla RNP attempts to identify the rationale by maximizing the mutual information  $I(Y;X_Z)$ :
$$
X _ {Z} ^ {*} = \underset {X _ {Z}} {\arg \max } I (Y; X _ {Z}) = \underset {X _ {Z}} {\arg \max } \left(H (Y) - H (Y | X _ {Z})\right) = \underset {X _ {Z}} {\arg \min } H (Y | X _ {Z}), \text {s . t .} X _ {Z} = f _ {E} (X). \tag {2}
$$
In practice, the entropy  $H(Y|X_Z)$  is commonly approximated by the minimum cross-entropy  $\min_{\theta_p} H_c(Y, \hat{Y} | X_Z)$ , with  $\hat{Y} = f_P(X_Z)$  representing the output of the predictor. It is essential to note that the minimum cross-entropy is equal to the entropy (please refer to Appendix B.3). Replacing  $X_Z$  with  $f_E(X)$ , the explainer and the predictor are trained cooperatively:
$$
\min  _ {\theta_ {e}, \theta_ {p}} H _ {c} (Y, f _ {P} \left(f _ {E} (X)\right) | f _ {E} (X)), \text {s . t . ,} (X, Y) \sim \mathcal {D}. \tag {3}
$$
To make the selected rationale human-intelligible, rationalization methods usually constrain the rationales by compact and coherent regularization terms. In this paper, we use the same constraints used in INVRAT (Chang et al., 2020):
$$
\Omega (M) = \lambda_ {1} \left| \frac {\| M \| _ {1}}{l} - s \right| + \lambda_ {2} \sum_ {t = 2} ^ {l} \left| m _ {t} - m _ {t - 1} \right|. \tag {4}
$$
The first term encourages that the percentage of the tokens being selected as rationales is close to a pre-defined level  $s$ . The second term encourages the rationales to be coherent.
4.1 Motivation: how spurious correlations come into being.
In this section, we consider  $X$  as a set of variables (or a multi-dimensional variables), and the selected rationale candidate  $X_Z$  is a subset (some dimensions) of it.
To begin with, in Figure 2(a), we posit a probabilistic graphical model to illustrate the corresponding data-generating process for the BeerAdvocate dataset. The input  $X$  comprises comments on three aspects:  $X_{S}$  for Smell or Aroma,  $X_{T}$  for Taste, and  $X_{A}$  for Appearance, each of which can be considered as a subset variables of  $X$ . Additionally,  $H$  signifies something that does not discuss the sentiment tendency of  $X$ . For instance,  $H$  could include the color of a bottle. The annotators assign the smell label  $Y_{S}$  by viewing the comments on aroma  $(X_{S} \rightarrow Y_{S})$ . Therefore, only  $X_{S}$  serves as the direct cause for  $Y_{S}$ . However,  $X_{S}$  is correlated with  $X_{T}$  due to a set of unobserved variables  $U$  (called confounders). For example,  $U$  may include a variable indicating whether the beer originates from a reputable brand, and a pleasant taste may imply that the beer comes from a good brand  $(U \rightarrow X_{T})$ . Moreover, a beer from a reputable brand is likely to have a pleasing smell  $(U \rightarrow X_{S})$ . Consequently,  $X_{T}$  is associated with  $Y_{S}$  via a backdoor path, as depicted by the red dotted line in Figure 2(a). In this situation,  $X_{T}$  is somewhat indicative of  $Y_{S}$ , but it signifies a statistical correlation rather than causality.
To have a more intuitive understanding of this correlation, we assume a toy example where  $U$ ,  $X_{S}$ ,  $X_{T}$ , and  $Y_{S}$  are all Bernoulli variables, with their respective probability distributions as:
$$
p (U = 1) = p (U = 0) = 0. 5,
$$
$$
p \left(X _ {T} = 1 | U = 1\right) = p \left(X _ {T} = 0 | U = 0\right) = 0. 9,
$$
$$
p \left(X _ {S} = 1 | U = 1\right) = p \left(X _ {S} = 0 | U = 0\right) = 0. 9,
$$
$$
p \left(Y _ {S} = 1 \mid X _ {S} = 1\right) = p \left(Y _ {S} = 0 \mid X _ {S} = 0\right) = 0. 9,
$$
(a) feature correlation
(b) mask correlation
Figure 2: A probabilistic graph for (a) the data generating process of BeerAdvocate and (b) the rationale (mask  $M$ ) selection process.
With some simple derivations, we can easily obtain (detailed derivation is in Appendix B.1):
$$
p \left(X _ {S} = 1\right) = p \left(X _ {T} = 1\right) = p \left(Y _ {S} = 1\right) = 0. 5. \tag {6}
$$
Then, we can further get (see Appendix B.2 for the detailed derivation of Equation 8 and 9):
$$
p (U = 1 \mid X _ {T} = 1) = \frac {p (U = 1 , X _ {T} = 1)}{p (X _ {T} = 1)} = \frac {p (X _ {T} = 1 \mid U = 1) p (U = 1)}{p (X _ {T} = 1)} = 0. 9. \tag {7}
$$
$$
p \left(X _ {S} = 1 \mid X _ {T} = 1\right) = \sum_ {U \in \{0, 1 \}} p \left(X _ {S} = 1 \mid U\right) p \left(U \mid X _ {T} = 1\right) = 0. 9 * 0. 9 + 0. 1 * 0. 1 = 0. 8 2. \tag {8}
$$
$$
p \left(Y _ {S} = 1 \mid X _ {T} = 1\right) = \sum_ {X _ {S} \in \{0, 1 \}} p \left(Y _ {S} = 1 \mid X _ {S}\right) p \left(X _ {S} \mid X _ {T} = 1\right) = 0. 8 2 * 0. 9 + 0. 1 8 * 0. 1 = 0. 7 5 6. \tag {9}
$$
Equation 8 demonstrates that  $X_{T}$  (Taste) is highly correlated with  $X_{S}$  (Smell), and Equation 9 indicates that  $X_{T}$  (Taste) is also strongly indicative of  $Y_{S}$  (Smell label). This situation can result in numerous local optima during the rationalization training process. Note that becoming trapped in a local optimum poses a significant challenge in rationalization (Yu et al., 2021; Liu et al., 2022). It is worth noting that the correlation between taste and smell here is merely one of the examples, and sometimes  $H$  can also correlate with  $Y$  in a similar fashion. For instance, in LIME (Ribeiro et al., 2016), a predictor is trained to determine whether an image contains a wolf or not based on the presence of snow in the background.
Furthermore, the degeneration problem can also be interpreted with a kind of spurious correlation (called mask correlation), as illustrated in Figure 2(b) with an example. Returning to the second example in Table 1, the variable  $M_{-}$ , denoting whether “-” is selected as part of the rationale candidate, is caused by the input  $X$  (comprising subsets  $X_{A}, X_{T}, X_{S}$  and  $H$ ) and the explainer  $E$ .  $M_{-}$  is also correlated with  $Y_{S}$  through a backdoor path, as indicated by the red line in Figure 2(b).
4.2 The conditional independent property
We first introduce an important concept in probabilistic graphical models, namely d-separation. Subsequently, we demonstrate how d-separation contributes to the identification of causal rationales.
D-Separation (Bishop, 2006):  $A$ ,  $B$ , and  $C$  denote arbitrary, non-intersecting sets of nodes (and their union might not cover all nodes of the graph) in a given probabilistic graph. Our objective is to determine whether a specific conditional independence statement  $A \perp B|C$  is implied by this graph. To do so, we examine all possible paths from any node in  $A$  to any node in  $B$ . A path is said to be blocked if it includes a node  $o$  such that either (see Appendix B.4 for why such a path is blocked)
(a) The arrows on the path meet at node  $o$ , forming either a chain (i.e.,  $\rightarrow o \rightarrow$ ) or a fork (i.e.,  $\leftarrow o \rightarrow$ ), with the node  $o$  being part of set C, or
- (b) The arrows on the path meet at node  $o$  to form a collider (i.e.,  $\rightarrow o \leftarrow$ ), and neither the node  $o$  itself nor any of its descendants are included in set C.
If all paths are blocked, then  $A$  is considered to be  $\mathbf{d}$ -separated from  $B$  by  $C$ , meaning that  $A \perp B|C$ .
Returning to our rationalization problem, the backdoor path (dotted red line) in Figure 2(a) comprises a fork  $(X_{T} \leftarrow U \rightarrow X_{S})$  and a chain  $(U \rightarrow X_{S} \rightarrow Y_{S})$ . If either  $X_{S}$  or  $U$  is included in the conditioning set, the path between  $X_{S}$  and  $Y_{S}$  becomes blocked, leading to their conditional independence, and consequently, the eradication of corresponding feature correlation. Similarly, the backdoor path (dotted red line) in Figure 2(b) forms a fork  $(M_{-} \leftarrow X_{S} \rightarrow Y_{S})$ . By including  $X_{S}$  in the conditioning set, the path between  $M_{-}$  and  $Y_{S}$  is blocked, resulting in their conditional independence and consequently, the elimination of corresponding mask correlation.
We consider the general case, where the input  $X$  is a set of variables (or features).  $X_{R}$  is a subset of  $X$  that exclusively contains all the direct causes of the target label  $Y$ , i.e., the desiderata of the rationale. We select a subset of  $X$  to serve as the rationale candidate (denoted as  $X_{Z}$ ), while the remaining unselected part is referred to as  $X_{-Z}$ . This leads us to the following properties: