SAGE: Semantic-Aware Global Explanations for Named Entity Recognition
Authors: Andrea Zugarini
Leonardo Rigutini
Venue: Submitted to ICLR 2023 (2023)
Keywords: Explainable AI
Named Entity Recognition
Language Models
Natural Language Processing

Abstract: In the last decades, deep learning approaches achieved impressive results in many research fields, 
such as Computer Vision and Natural Language Processing (NLP). 
NLP in particular has greatly benefit from unsupervised methods that allow to learn distributed representation of language.
On the race for better performances Language Models have reached hundred of billions parameters nowadays. 
Despite the remarkable results, deep models are still far from being fully exploited in real world applications. 
Indeed, these approaches are black-boxes, i.e. they are not interpretable by design nor explainable, which is often crucial to make decisions in business. 
Several task-agnostic methods have been proposed in literature to explain models' decisions. 
Most techniques rely on the "local" assumption, i.e. explanations are made example-wise.
In this paper instead, we present a post-hoc method to produce highly interpretable global rules to explain NLP classifiers. 
Rules are extracted with a data mining approach on a semantically enriched input representation, instead of using words/wordpieces solely. 
Semantic information yields more abstract and general rules that are both more explanatory and less complex, while being also better at reflecting the model behaviour.
In the experiments we focus on Named Entity Recognition, an NLP task where explainability is under-investigated. 
We explain the predictions of BERT NER classifiers trained on two popular benchmarks, CoNLL03 and Ontonotes, and compare our model against LIME.

SAGE: SEMANTIC-AWARE GLOBAL EXPLANATIONS FOR NAMED ENTITY RECOGNITION
Anonymous authors Paper under double-blind review
ABSTRACT
In the last decades, deep learning approaches achieved impressive results in many research fields, such as Computer Vision and Natural Language Processing (NLP). NLP particular has greatly benefit from unsupervised methods that allow to learn distributed representation of language. On the race for better performances Language Models have reached hundred of billions parameters nowadays. De- spite the remarkable results, deep models are still far from being fully exploited in real world applications. Indeed, these approaches are Black-boxess i.c. they are not interpretable by design nor explainable, which is often crucial to make deci- sions in business. Several task-agnostic methods have been proposed in literature explain models' decisions. Most techniques rely on the "local" assumption, i.e. explanations are made example-wise. In this paper instead, we present a post-hoc method to produce highly interpretable global rules to explain NLP classifiess. Rules are extracted with a data mining approach on semantically enriched input representation, instead of using words/wordpieces solely. Semantic information yields more abstract and general rules that are both more explanatory and less complex, while being also better at reflecting the model behaviour. In the experi- ments focus Named Entity Recognition, an NLP task where explainability is inder-investigated. We explain the predictions BERT NER classifiers trained two popular benchmarks, CoNLL03 and Ontonotes, and compare model against LIME (Ribeiro et al. 2016 and Decision Trees.
Keywords: Explainable AI, XAI, Explainable ML, Named Entity Recognition.
INTRODUCTION
In recent years, Artificial Intelligence (AI) algorithms, especially deep learning models, are emerg- ing in many applications, reporting state-of-the-art performances in many fields. In NLP, for exam- ple, the use of Large Language Models (LLM) based on huge deep neural networks achieved im- pressive results in many linguistic tasks. However, despite the remarkable results, deep approaches are still far from being fully exploited in real world applications. One major issue is the lack of interpretability and control of the models' predictions. This is often an important requirement for many industrial applications, especially in domains like medicine, defense, finance and law, where is crucial to understand the decisions and build trust in the algorithms.
The increasing need to address the problem of interpretability and improve model transparency made the "Explainable Artificial Intelligence" very popular research area in the Computer Science world. Explainable Al (XAI) or Interpretable Al or Explainable Machine Learning (XML) (Guidotti et al., 2021 is broad area of research that studies and proposes AI approaches where humans can under- stand the causes underlying the decisions and predictions made by the machine (Vilone & Longo 2021b). The AI algorithms can be usually grouped into two families (Vilone & Longo| 2021a): (a) white-box models, which include algorithms whose interpretation is given by design, and (b) black-box approaches where, on the other hand, the decision making process is "opaque" and hard to understand. White-box models such as linear regression, probabilistic classifiers or decision trees are significantly easier to explain and interpret, but, often, provide a low predictive capacity and they not always capable of modeling the inherent complexity of the task. In black-box models, on the other hand, very little knowledge is available on how the input variables influence the final decision. The relationship between input and output is often the result of complex composition of

---

mathematical functions and is not directly interpretable. Although classic ML approaches are still widespread, almost all the modern complex Al techniques, such as deep neural networks, are natu- rally opaque (Lipton 2018). Thus many new methods aimed to make new models more explainable and interpretable have been proposed or are under investigation.
Most explainability techniques rely on the "local" assumption where the descriptions are provided for each example and few approaches exist aiming at provide a interpretable description of the model as whole (glass-box). In this paper. we present SAGE (Semantic-Aware Global Explanations), method to produce highly interpretable global rules to explain NLP classifiers. Rules are extracted using a data mining algorithm which exploits a semantically enriched input representation. Semantic information yields more abstract and general rules that are both more explanatory and less complex, while being also better at reflecting the model behaviour. Entity Recognition, an NLP task where explainability is under-investigated. In particular, we aimed to explain the predictions of BERT-based (Devlin et al. 2018)) NER classifier trained and tested on two popular benchmarks, CoNLL03 (Tjong Kim Sang & De Meulder 2003) and Ontonotes (Pradhan et al. 2013). We compare the proposed model against LIME (Ribeiro et al. 2016), which currently is one of the most popular local explanation algorithm in NLP and Decision Tree classifiers
the experiments, we focus on Named
that are classic explainable by design, global explanation models.
For the assessment, exploited two commonly used metrics: fidelity and complexity. The results show that the proposed approach infers a set of rules that reproduce the behavior of the model more accurately than both LIME and Decision Trees.
The paper is organized as follow. In Section 2 we summarize related works, while in Section the proposed algorithm is described in detail. Experiments reported in Section and finally conclusions and future works are drawn in Section
2 RELATED WORK
As deep learning models have become more complex, many methods have been proposed to inter- pret and explain the predictions of model. Two main groups of XAI techniques exist: (1) local approaches, which aim to provide an interpretable explanation for each single prediction; (2) global approaches, which try to build a "white-box" version of the black-box model (thus interpretable by design).
Local Explanations. Local algorithms focus on finding an interpretable explanation of the predic- tion returned by the MI. model for given input example. Several approaches focus on interpreting the internal components of black-box model with intent of shading some lights on its decision making process. In (Csiszár et al.] 2020). authors propose the use of fuzzy logic to "explain" each Artificial Neural Network unit. Similar techniques try to identify which features in a particular in- put vector contribute the most to a neural network's ' decision. Layer-wise relevance propagation (LRP). for example, exploits back paok-propagttion similar algorithm to build a heat-map over the in- put features (Binder et al. (Moo).Moo et al., 2019).. This method proved to be very effective in Computer Vision, highlighting which pixels of image contributed most to the final classifica- tion. Other methods such as (Lundberg & Lee. 2017) and (Simonyan et al., (013) follow the same principles. Such techniques, however, are strongly related the ML models' family used and they can be employed only with neural networks. Furthermore they usually do not produce high-level explanations.
Some model-agnostic approaches were proposed in (Ribeiro et al., 2016 2018 Strumbelj & Kononenko, 2010). In particular, in (Ribeiro et al. 2016). LIME (Local Interpretable Model- agnostic Explanations) is presented. LIME trains a white-box classifier, that Iearns the black-box output distribution on neighborhood of given example. Neighbors are obtained perturbing the input features (usually randomly). The explanation can be then constructed by selecting the input features that mostly affect the model prediction. Although these models have been widely applied in explainability problems, including NLP, some limitations occur when dealing with Large Language Models (LLMs) and Named Entity Recognition. LLMs tokenize text in subwords, like wordpieces (Schuster & Nakajima 2012) in BERT, which requires particular attention in designing the text perturbation step. Masking and perturbation of the input introduce another issue for LLMs, that are contextual. Indeed random masking of tokens can produce artificial and inconsistent contexts, which
2

---

may invalidate the identification procedure of significant features. Finally, performing hundreds of input perturbation per sample, can be an expensive process, especially in problems like Named En- tity Recognition (NER), where the number of examples corresponds to the number of tokens in dataset.
Most of the interpretable explainable models for NLP is on text classification problems (Chen & Ji 2020; CCen et al. 2021). For instance, (Alvarez-Melis & Jaakkola (017) proposed a method for interpreting eqquence-o--eqquence models in Machine Translation, whereas (Tuan et al. 2021 focused on local explanations in dialog generation. To the best of our knowledge, we are the first
investigating XAI techniques in NER.
Global Explanations. While the explanation a single prediction provides the user with a ratio- nale for the classifier's behavior, it is not sufficient to convey confidence in the model as whole. Thus, in contrast to local approaches, global explanation algorithms aim to find an interpretable description of the model in the whole input space.
White-box models, such as Decision Trees (Breiman et al. 2017) and Bayesian Rule Lists (BRLs) (Letham al. 2015), are explainable by design models that used to provide global explanations. Alternatively, one could extrapolate global rules from local explanations as in (Ríbeiro et al. 2016), where authors proposed an extension of LIME for making global explanations. The resulting SP- LIME algorithm is based on the construction of an explanation matrix which collects the local importance of the interpretable components for each instance dataset X. For each component, global importance in the explanation space is evaluated and the "Submodular pick (SP)" algorithm selects subset of instances with the highest marginal coverage gain.
Our proposed method (SAGE) falls in this family. Unlike SP-LIME, it is natively global by design and aims to identify the set semantically-enrichie associative rules that best cover the behavior of the model however, the mining process specifically designed and optimized scale NLP problems like NER. Furthermore, differently from BRLs. SAGE is not used as interpretable model per-se, but it is rather an algorithm to explain globally black-box classifier predictions. To the best of knowledge, SAGE is the first method involving semantic knowledge in the generation of the explanations.
reference set (fidelity). Our algorithm shares some principles behind BRLs,
SEMANTIC-AWARE GLOBAL EXPLANATIONS
In this Section, we present SAGE (Semantic-Awar Global Explanations) algorithm, a method to discover globally significant explanation rules. To find global and interpretable description of black-box classifier, the proposed method follows a data-mining approach. In particular, given a black-box model θ BB, the general idea is apply Association Rules Discovery algorithm the learning set coupled with the model's predictions to find the most representative set of explanation rules for the original model. Although it is task agnostic, we present SAGE by exploiting the NER task. Since we are facing a NLP task, the algorithm employs semantic features to improve the generalization capabilities of the returned rules. A sketch of SAGE method is depicted in Figure []] In this section, firstly we discuss how to formulate NER in the scope of data-mining task. Next, we
y
Brack Box Moder
Predictions
x
Why?
Textual
MLP Engiioo
Association Rules Discovary
Explanation
Rules
SAGE
Fig. 1: Sketch of Semantic- Global Explanations (SAGE) method.
delincate the data mining algorithm to generate candidate rules and the strategies for pruning and selecting restricted number of explanations.
3

---

DATE
-twodd Oroond beanty. seoond sonooot PPO N:POS/oun. +1P08Nour... Derrity mestumment
Swordion, word wordiworty, swords -fwerdoot +Bworddduly wardd
Of course kends on the tventy secand
O
Of course onds on the twenty socond
B-DATE
(())
tannly
HDATE
OTHER
yean, shrther
B.ORG
soveett +0word POS
Fig. 2: Transactions creation. The function & produce transaction for each token x in the corpus. In partieular, the example shows the transactions created for the three tokens twenty, second and Of. In the example, considers (-3, +-3) context window and includes semantic/syntactic information for each token the window (e.g. the POS tags).
3.1
NER AS DATA MINING PROBLEM
Named Entity Recognition (NER) is typically addressed token classification task. Given corpus of sentences/doccmments X' and its corresponding annotations V, the goal is to predict a sequence of labels y E V. y =- (. Mo.), from sequence of tokens x E X, == (.1 Fn))
In data mining, data is organized in a collection of transactions. A transaction 4 consists in a set of n, items drawn from a specific basket B: 14 {er,1.-.., fi,,, }, where et, E B represents the jth element of the 7th transaction. transaction is often also referred to as "itemset". Given a large set of transactions T = {fi,.... J. approaches as apriori (Agrawal et al. 1994) aim to discover frequent co-occurrence patterns (frequent itemset) in form of implications rules, e.g. {Ey, Egg ez - Eg- A such rule states that when the premise is satisfied (i.e. the elements in the left part occur), the consequence should be considered (i.e. also the elements in the right part should occur).
Tackling NER with data mining requires to arrange textual data into transactions. We define the basket as the vocabulary containing the tokens from both sets X' and y. Moreover, suppose each token 24 is represented with a set of features obtained by a function Φ i - t, then each feature can be seen as an item of the token transaction. Beside its representation. also the label y is included in the basket. Consequently, we obtain corpus of transactions T where each row is associated to its target label (#(x, 1). )s). For the sake of simplicity, from now on we define ((x;) == o(x 1). In its simplest form, function &b is the identity, returning the token I; itself, 2; and a window of surrounding tokens. In general, it can be any function that returns a sequence of symbols. Since our goal is to obtain rules to predict NER classes, we interested only in implication patterns where the conclusion is the target label ys. i.e. (...) = kind. This allows us to divide the problem into independent sub-problems, one per entity class in the corpus, on a reduced set of transactions
Ty. Since each class is treated independently, the generation of frequent patterns is dramatically simplified, which crucial aspect for the scalability of apriori- methods to NER datasets.
Explaining a NER model is an analogous problem, the only difference is that labels y are the output predicted by a classifier, instead of being provided by an oracle.
3.2 SEMANTIC FEATURES
Instead of simply using plain words or wordpieces as transaction items, we enrich them with se- mantic and syntactic positional information, extracted by (z;;) that is a NLU system as shown in Figure 2 readable and reduce the overall number of explanations required to explain model. We believe that they are essential to make explanations scalable in language, where data is sparse, especially in NER, and information is also positional. For the NLP analysis employed proprietary NIPP engine") and we extract POS tagging information, entity types, concepts (e.g. synsets) and classical NER features such is diiigt ssupper__case case, is that are often important named entities indicators.
We emphasize the importance of having more general rules because they are more human-
'The expert.ai ai NLP Platform httpss:/ww.exxpert.tt
4

---

3.3 SAGE
SAGE is based on the FP-growth (Han et al. 2000) algorithm, an efficient variant of apriori (Agrawal al. 1994). Due the combinatorial nature of the problem and the relatively large amount of items in the transactions, the algorithm considers several thresholds: (1) the minimun support threshold (mingensup)) which commonly used in all data-mining algorithms to limit the outdegree of the candidate generation; (2) a minimum confidence threshold (mingencon.f) which, following (Wang) al., 2002), we introduced to retain the candidates generation the most promising frequent pat- terns only; (3) the tolerance threshold (r) which we introduced to stop adding items to a rule when confidence improvement below such value; (4) a maximum rule length threshold (mazzulelen) which limits the length of generated rules to maximum value; (5) a max number of rules to gener- ate (marnrules). which sets a maximum number of rules that can be generated. The algorithm is reported in Algorithm and consists of three main procedures, each designed to face specific step:
Algorithm Semantic-Aware Aware Global Explanations
function SAGE(X,))
for E ) do
Given т, mingeresups mingencoon
macnurlss:
Ty T T-y, C . T, mingensup, mingenconfi, mazrulelen.)
E MANDIDATES-GENCC TT.
< PRUNE(T... T-u., R)
Ey. + SELECTION(R, niters, maznrules.)
end for end function
3.3.1 CANDIDATES GENERATIONS
The core of data-mining algorithms is the candidates generations step. At each iteration, the func- tion CANDIDATES-GENC produces new set of frequent itemsets for the next step, starting from current set of transactions and using the transaction dataset as statistical reference. In particular, we exploited FP-growth (Han et al. 2000), efficient variant of apriori (Agrawal al. 1994).
3.3.2 RULES PRUNING
Rules mining can produce many patterns that often spurious and/or highly overlapping. The werrgenerattion& of rules harms both the quality and the interpretability of the explanations. As a matter of fact, simplicity is crucial to deliver clear explanations that humans can easily understand. Thus, we aim to generate a minimal rule set with limited overlaps and high faithfulness. We proceed in two steps: pruning and selection.
In the pruning procedure, we start from the observation that most of the rules co-occur in the same subset of baskets, making them pseudo duplicate rules. Hence, among them we prune less generic rules, i.c. the ones having lower support. In case of rules that triggers in the exact same set of transactions, we favor more abstract rules, i.e. the ones having less conditions and/or more high- level concepts as items. Pruning step typically drops the number of relevant patterns by an order of magnitude Pseudo code is presented Algorithm
3.3.3 RULES SELECTION
After pruning, large number of candidates still remains. Most of them have high overlap in the corpus. In the second step we aim to keep the optimal subset S" S of rules that provides the best performances. The problem is combinatorial there are 2 subsets thus, we perform a greedy strategy. For specified number of iterations Nittres we filter candidates based on their support and confidence and then sub-sclect at most maznrules such that they maximize performance metric. Support and confidence thresholds are sampled with bayesian optimization. Rules cub-selection performed greedly: the procedure starts from an empty state (no rules solution) and adds one rule
"See Appendix A for further details.
5

---

Algorithm 2 Rules pruning
function PRUNE(To; T-,, R2)
Rpruned 1 R SOTEH-SSPPDR..ADDDASSTTTTTAATTTT descending)
for r do
T. t SUPPORT- TRANSAAAIINNNNS 7 T-y:)
if Ep E Apromed TT S SUPPOR-TRANSACIOONSS. Tu: T.v;) then Rpruned f- Reruned Ur end end for return Rpraned end function
time, storing the configuration that provide the higher improvement the state. Despite the approach is heuristic, it improves the quality of the solution, both terms of fidelity and complexity reduction depicted in the section Selection procedure described in Algorithm 3
Algorithm Explanations selection
function SELECTION([R, miters, mazvrulees)
< for i = 1,... niters do
supp, conf ENNERNIIIPEBBRAAAOOTTTTAENESTEES R FILTER(R, supp, conf)
while |S| < maznrules and |S] < | do
f arg max COMPUTE-FIDELITYSS reRsS
COMPUTE- DOMPUTE-FIDELIIT(SSUU))) COWPUTO--MPOPEEEE FIDELITY(S) then
break end if
S SUT end while
if COMPUTE-FIDELIITCE > COMPUTE-FIDELITY(S) then
S"
end end for return S end function
4 EXPERIMENTS
We evaluate SAGE in different NER benchmarks, aiming to measure (1) the quality of the explanations, (2) how semantic information is vital to produce generalized rules that are both more effective and easier to understand.
4.1 METRICS
For evaluating the explanations, consider two popular criteria: fidelity and complexity.
Fidelity. Fidelity measures the ability of the explanation algorithm to mimic the predictions of the black-box classifier (e.g. BERT). To measure such index, considered the F1 score (both micro and macro). In this setting, the optimal value F1 100% means that the explainable model perfectly reflects the behavior of the black-box model.
Complexity. Complexity on the other hand, measures the simplicity resulting explanations. We define it as the number of rules per class extracted by the data-mining algorithm.

---

4.2 DATASETS
Ontonotes is a dataset containing named entity annotations for multiple languages. There are multiple versions, in the experiments used English v12 (Pradhan et al. 2013). The corpus is composed of texts from various genres. Overall, there are 18 annotated classes.
Ontonotes.
CoNLL03. CONLLO3 (Tjong Kim Sang De Meulder 2003) is a standard Named Entity Recog- nition (NER) benchmark. The dataset is collection of news articles from the Reuters corpus. Data annotated with four entity tags: PERSON, LOCATION, ORGANIZATION and MISC.
4.3 EXPERIMENTAL SETUP
Black-box classifiers. We train a BERT (Devlin et al. 2018 classifier on each benchmark (CoNLL, Ontonotes) and we explain its predictions on the test set. Predictions are made at word level, i.e. we ignore the output of word sub-tokens commonly done in NER problems with mod- els based on word-pieces like transformers. Overall, we obtain 4170 and 14227 predicted entities in CoNLL and Ontonotes, respectively.
LIME for NER. LIME is widely used for explaining models in classification problems. However, caveats when applying it in NER. Indeed, predictions at token-level, while usually the approach is applied for document-leve classification problems. Thus, explanations must be positional. Furthermore, current Language Models adopt subword-level tokenizers, while expect to have word-level explanations. Hence, wordpieces of the same words are perturbed together. For each example, we retrieve the top-k words to build an explanation explanation. Global explanations are then constructed applying submodular-pick (SP-LIME-k), In the experiments, we consider five different SP-LIME models differentiated by the choice of that varies from up to 5.
there
Decision Tree. Decision Trees (DTs) are considered to be white box classifiers, therefore they are widely used for explainability. We compare SAGE with DTs fitted with (DT) and without (DT- words) the semantic features. We fit trees with maximum depth of 100.
SAGE. Transactions are constructed as described in Section Function & produces for each token r, set of positional features in a (-3, +3) context window, as presented in 3.2 depicted in Figure 2 In the results we also report performances of a variant, called SAGE-words, that ex- ploit a simpler & function that yields only words in the (-3, +3) scope, i.e. without any semantic information. Hyper-parameterss parameters of Algorithm[[ couuld be set differently for each entity. In the experi- kept the same for all classes and both datasets.
ments, however, most of the hyper-parameters In particular, we set 7 = 0.05, mazrulelen = 5, niters = 100 and maznrules = 100 every- where. Only mingensup and mingencon, change depending on the number of occurrences of the class. The minumum confidence to further explore rule is set dinamically to mingencon
[7] whereas the minimum support is set to 10 for all entities in CoNLL, and varies between {2, 5. 15) in Ontonotes depending on the cardinality T; and such that large classes have higher mingensup and, vice versa, underreppesented entities have smaller mingensup.
4.4 RESULTS
Quality of explanations. We evaluate the quality of SAGE explanations terms of fidelity and complexity as defined in Section 4.I In particular, measure how the increase of complexity of the explanations improves the fidelity to the original model, which may be useful to determine a satisfactory trade-off between the two indicators. Results are reported in Figure 3
SAGE significantly outperforms SP-LIME global explanations. Even a single rule per-class out- shows the F1 scores for each single entity class in Ontonotes. We can see, that SAGE always outperforms SP-LIME explanations. with the exception of LAN- GUAGE and PRODUCT, two underrepresented classes where SAGE performs poorly because cannot find rules that are statistically significant
reaches any SP-LIME setup. Figure 4sows
htt https://ooggingaaecccoo/asssseea (0tt0lotts_ottoteees5ss

---

SAGE SAGELOOSS
SAGE
Fiweyy
0.3
SAUMES
63
20
40
20
100
20
60
100
Comploity
Complecty
Fig. 3: Explanation performances of our method (with and without semantics) and different models built with SP-LIME. We compare all methods in terms of j--idelity score at the varying of explanation complexity (number of explanations per class) CoNLLO3 (left) and Ontonotes (right).
SAGC
33
CRUMCS
SIUNCE
022
UEA
Fig. 4: Class-wise y-fidelity scores for each approach in Ontonotes dataset.
There are several reasons responsible for causing such a gap between SAGE and SP-LIME. First of all, LIME is local algorithm designed to provide local explanations of predicted examples, and SP-LIME combines them together to provide an overall view them. However, each explanation not designed hold outside the specific example, whereas approach aim to retrieve statis- tically relevant rules that trigger a class in the data. Furthermore, token classification tasks, such NER, introduce additional complexity because explanation must be positional, and because most of the tokens do not belong to an entity. Lastly, as we will see shortly, semantics is vital to boost performances of our approach.
One could notice however, that SP-LIME-K. > have a linear growth as the number of expla- nations increases, and may eventually surpass SAGE performances, complerxly - co. As matter of fact, this will happen, because by construction SP-LIME will retrieve one explanation per example, falling back providing local explanations. This situation occurs in Ontonotes for class LANGUAGE. Looking at Figure we can see that SP-LIME methods are performing extremely well on such entity, because the number of annotated tokens only 22, hence SP-LIME extracts one rule example. However, the number of explanations to reach and surpass SAGE is extremely high, breaking even far from a point where the explanation is easy to understand.
SAGE also outperforms Decision Trees, outlined in Table 1 also providing rules that are naturally more human- friendll than tree decision paths that may involve dozens of conditions.
The impact of semantics. Semantic information plays crucial role in providing better rules. We analyse this aspect comparing SAGE with SAGE-words, a variant where a token 2; represented only by itself and text in its context, without any external knowledge. Results outlined in Table and Figure
In all the scenarios, semantics provides significant gain in terms of fidelity. One explanation per class in SAGE outperforms 100 rules SoGE-words. Indeed, abstraction is a mean to group less general concepts into single It allows to express rules with higher coverage, and it is particularly important for language, that is characterized by the presence of many discrete symbols (words) combined together. A clear example of how semantics yields general rules can be
8

---

Explainer
CoNLL03
Ontonotes
ue-fidelity
m-fidelity
n-fidelity
m-fidelity
DTT-words
34.8
34.2
7.4
17.3
DT
76,0
74.5
32.1
32.6
SAGE-words
45.6
40.0
51.5
44.7
SAGE
80.3
78.5
57.4
Tab. 1: Evaluation of explanations in CoNLLO3 and Ontonotes datascts for approach and Decision Trees (DT), both with and without semantic features.
seen in Figure SAGE rules exploit broader concepts that are valid for much more examples. Furthermore, effectiveness of semantics is independent from the explainer algorithm. Even decision trees benefit from it (see TD.
CARDINAL
DATE symettdeliiie time wordlyear
EVENT
SAGE
Pos:ADJ A wordooe
bentEVV Pos:AR A iistitle
SAGE-words wondrwo -lwwrdhthee
Toword-Yar word-Now
SP-LIME-1
Iowurdyearr
Wond-Katiia
SAGE
istitle Laynserroo
eat:GEO
SAGE-words LewnddReand
wurd-Chim
SP-LIME-1
Road
whrd:China
woni-English
LAW
L.CC
MONEY
SAGE
Isynset rights
googrrgoii area
SAGE-words
Amendment
Middlee
-LowordsS
SP-LIME-1
word-Gram
Jsword/Mddlt
NORP
ORDINAL
ORG
SAGE
synset:Asian
word-first first
eat:COM
SAGE-words
wordeChinese
word
SP-LIME-1
Amerrcmmriiann
word lirst
wordCNN
PERCENT
PERSON
PRODUCT
SAGE
NPR.NP
wordcoole
SAGE-words
. wward. A loword:%
-2wwodd
werd:Cale
SP-LIME-1
word
QUANTITY
SAGE
MBEMEA NDU
entHOU
istidle ent-MEA
SAGE-words
L:worddeegeess
wordsbour
Lsennd/Minues
SP-LIME-1
lowordddegrees
lowordsominng
wird
Fig. 5: Explanation rules discovered by SAGE, SAGE-words and SP-LIME-I in Ontonotes with complexity Semantic features are highlighted in hold.
CONCLUSIONS
In this paper, presented SAGE, post-hoc method to extract interpretable, global explanations for NLP. A key ingredient of SAGE is the fact that it produces rules based on semantic, more abstract information. We focused on Named Entity Recognition, particularly challenging NLP problem, which was under-investigated in the scope of explainability. We compared our model against different SP-LIME variants on two BERT models trained in two popular NER benchmarks: CoNLL03 and Ontonotes. Results indicate that our global explanations more interpretable) and significantly better at mimicking the black-box classifier, although SAGE fails in making explanations of few partieularly underrepresented classes. We also prove empirically that semantic and syntactic information bring major boost to the explanation quality.
compact (hence
In the future plan to investigate possible strategies to guarantee the production of explanations for any class, regardless the amount of predictions such entity. In particular, we plan to combine local approaches, such LIME, SAGE.

---

REFERENCES
Rakesh Agrawal, Ramakrishnan Srikant, et al. Fast algorithms for mining association rules. In Proc. 20th int. conf. very large data bases, VLDB, volume 1215, pp. 487-499. Santiago, Chile, 1994.
David Alvarez-Melis and Tommi S Jaakkola. A causal framework for explaining the predictions of black-box sequence-to-sequees -equuecce models. arXiv preprint arXiv: 2017.
Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Wojciech Samek. Layer-wise relevance propagation for neural networks with local renormalization layers. In International Conference on Artificial Neural Networks, pp. 63-71. Springer, 2016.
Leo Breiman, Jerome Friedman, Richard A Olshen, and Charles J Stone. Classification and regression trees. Routledge, 2017.
Hanjie Chen and Yangfeng Ji. Learning variational word masks to improve the interpretability of neural text classifiers. arXiv preprint www.i:2010000667. 2020.
Hanjie Chen, Song Feng, Jatin Ganhotra, Hui Wan, Chulaka Gunasekara, Sachindra Joshi, and Yangfeng Ji. Explaining neural network predictions sentence pairs via learning word-group masks, arXiv preprint arXiv: 1ete22244.44488 2021.
Orsolya Csiszár, Gábor Csiszár, and József Dombi. Interpretable neural networks based on continuous- valued logic and multicriteria decision operators. Showledge-Basse Sys- tems, 199:10 2020. ISSN 0950-7051.
doi: http://dbb.orr///..000......mm URL https:/ www. www.sciencedirect com/ htt.://ceeeeee///riiiie////..a..
2020. 105972.
509507751223028889
Jacob Devlin, Ming- Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:/810.04k0 2018.
Riccardo Guidotti, Anna Monreale, Dino Pedreschi, and Fosca Giannotti. Principles of explainable artificial intelligence. In Explainable Al within the digital transformation and cyber physical systems, pp. 9-31. Springer, 2021.
Jiawei Han, Jian Pei, and Yiwen Yin. Mining frequent patterns without candidate generation. ACM sigmod record, 29(2):1-12, 2000.
Benjamin Letham, Cynthia Rudin, Tyler H McCormick, and David Madigan. Interpretable classi- fiers using rules and bayesian analysis: Building better stroke prediction model. The Annals of Applied Statistics, 355--33511 2015.
Zachary C Lipton. The mythos of model interpretability: In machine learning, the concept of inter- pretability is both important and slippery. Queue, 16(3):31-57, 2018.
Scott M Lundberg and Su-In Lee. A unified approach interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Grégoire Montavon, Alexander Binder. Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller. Layer-Wize Relevance Propagation: An Overview, pp. 193-209. Springer International Publishing, Cham, 2019. ISBN 978- 37883-030-288544 doi: 10. 1007109778---000 3-077788--------00 --0-7-8---00---2----66-- URL -00177777------00-222444-------- 10
https://doi.
Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong, Towards robust linguistic analysis using OntoNotes. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pp. 152, Sofia, Bulgaria. August 2013. Association for Computational Linguistics. URL httpps//accLantiooooy..oo///////
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin." why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference knowledge discovery and data mining, pp. 1135-1144, 2016.
10

---

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High- precision model-agnostic explanations. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Mike Schuster and Kaisuke Nakajima. Japanese and korean voice search. In 2012 IEEE interna- tional conference on acoustics, speech and signal processing (ICASSP), pp. 5149-5152. IEEE, 2012.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. arXiv preprint arXiv:/3/2.6034, 2013.
Erik Strumbelj and Igor Kononenko. An efficient explanation of individual classifications using game theory. The Journal of Machine Learning Research, 11:1-18, 2010.
Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the CONLL-2003 shared task: Language-independen named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp. 142-147, 2003. URL https:
1/aclanthology wwwpoolog.oggggleeoooo wrg/h03-0440000
Yi-Lin Tuan, Connor Pryor, Wenhu Chen, Lise Getoor, and William Yang Wang. Local explanation of dialogue response generation. Advances Neural Information Processing Systems, 34:404- 416. 2021.
Giulia Vilone and Luca Longo. Classification of explainable artificial intelligence methods through their output formats. Machine Learning and Knowledge Extraction, 3(3):615-611, 2021a.
Giulia Vilone and Luca Longo. Notions of explainability and evaluation approaches for explainable artificial intelligence. Information Fusion, 76:89-106, 106, 2021b.
Ke Wang, Liu Tang, Jiawei Han, and Junqiang Liu. Top down fp-growth for association rule mining. In Pacific-Asia conference knowledge discovery and data mining, pp. 334-340. Springer, 2002.
A SCALABILITY
Data mining approaches are combinatorial, therefore the complexity grows non-linearly with the increase of the data. One may wonder how Fp-growth adjustments, rules pruning and selection process of the algorithm contribute in controlling the potential explosion. To shed some light on this, we report in Table 2 the number of elements produced by each step in the Ontonotes dataset (the largest corpus in our experiments).
O:synset:Asian
= NORP
SAGE
Europen
NORP
O:word:Chinese = NORP
SAGE-words
Omerica - NORP
O:word:Chinese NORP
SP-LIME-1
O:word:Ameriian NORP
2:word:. O:word:Chinese
NORP
SP-LIME-2
3:word:., 0:word:American > NORP
4:word:., 1:word:North, 2:word: Koreans
NORP
SP-LIME-3
-loood:and, 16:word:., O:word:American
NORP
w:word:osittion, -Iword:the, 2:word:.,
= NORP
SP-LIME-4
2:word:aand, -3:wordtelevision, 3:word:.. Palesiiaa = NORP
8:word:., with, 6.woondintrannsieence,, I:wtrd:North, 2:word:Koreans
NORP NORP
SP-LIME-5
-2woord: -lwwod:andd 4:wood:: 2:word:., O:word:Chinese
Tab. 3: Two examples of generated rules per model for NORP entity in Ontonotes dataset. Only SAGE exploits semantics, in this case the synset features are important to build rules that more generic. We can also notice how, the increase of k in SP-LIME models makes the explanation and more wired to the example that generated
11

---

Entity
#tokens
#generated candidates
#pruning
#selection
EVENT
351
57091
3078
WORK_OFLAR
594
304724
7781
100
LAW
222
23255
1569
63
LANGUAGE
22
0
NORP
1411
14216
1926
78
FAC
398
127311
$205
LOC
504
36894
2180
30
PRODUCT
236
914
109
7
TIME
668
10184
2327
53
PERCENT
1483
181897
26277
29
MONEY
1109
46497
6871
QUANTITY
463
18337
1926
37
ORG
5604
122722
65749
100
GPE
4254
411140
24738
53
PERSON
4151
43143
7746
DATE
4739
22347
6996
100
CARDINAL
1795
14137
1621
100
ORDINAL.
322
488
91
ALL
28326
1435297
166190
1074
Tah. 2: Number of elements produced by each step. In hold classes the number of selected explanations was interrupted because the maznrules was reached.
12


Reviews:
[{"review_id": "mONXjlug8NL", "reviewer": "Reviewer_3Fgr", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper the authors introduce a new post-hoc method to create explanations for a black box classifier. They explain the steps in the method using the task of Named Entity Recognition as example. They turn the data into baskets of information , by extracting the contextual window around each word as well as some semantic features like pos tags, synsets etc and then use a data mining algorithm to extract patterns of co-occurrence corresponding to each classification label. Pruning is done over these patterns or candidate explanations based on thresholds and explanations are then selected greedily using f1 score maximization. The perform evaluation on this method by measuring the F1 score vs complexity defined by the number of explanations and they compare this to variations of LIME as well as a version of their model without semantic information\n", "summary_of_the_review": "Although the fundamental idea is good, it fails to convince that the method is robust across different datasets or NLP tasks.\nThey employ a contextual window of +/-3 tokens and that might not be transferrable to other NLP classification tasks like sentiment classifier or recommendation systems.", "strength_and_weaknesses": "Strengths:\n\n\t1. They support the necessity for each step in the algorithm\n\t2. They provide global explanations\n\t3. They are task agnostic , method can be applied to many NLP tasks\n\nWeaknesses:\n\t\n\t1. In section 3.1 they do not specify what certain notations mean  , eg the difference between the two transaction tables on the right of figure 2. \n\t2. Jump from section 3.2 to 3.3 is big especially for people who are unfamiliar with algorithms they point to such as FP-growth Han et al. (2000) and apriori Agrawal et al. (1994). They use an example for section 3.1 but then they drop the example for subsequent sections in the algortihm . \n\t3. Other evaluation metrics employed by other papers eg, fidelity to the model and comprehensibility could have been explored . Human evaluations might make a more compelling case .\n\t4. They don’t perform any study about which semantic features help and which harm the f1 score.\n\t5. Visualizaition is an important part of explainable models which this paper lacks\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "Rvwi1m_yHO-", "reviewer": "Reviewer_RoTP", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors of this work propose SAGE, a data mining method to discover global explanation rules for black box text classifiers using semantic augmented features in a post-hoc fashion.  The method first leverages FP-growth (Han et al 2000) to generate candidate implication rules of the form (t1, t2, .. tn) -> y  ( where t1, t2, etc delineate the presence of tokens and semantic features within a window and y is a class for a given task ).  They then show how to filter the generated set of rules and select a final concise set of rules.  They evaluate SAGE on the NER task, by training BERT on the task and using SAGE to explain its predictions.  They compare SAGE against SP-Lime-k which is global submodular based variant of LIME and an ablated version of SAGE which removes semantic augmentation.  They show SAGE outperforms the other methods on the CoNLL03 and OntoNotes datasets, semantic augmentation increases accuracy and additionally provide some examples of rules the system learns which highlights the utility of adding semantic augmentation.   ", "summary_of_the_review": "Its an interesting idea overall, but feels under explored in the paper and could be made stronger and more compelling if it had more robust comparison of baselines/alternative methods, ablations and qualitative analysis which really highlighted why the method should be used.\n", "strength_and_weaknesses": "**Strengths:**  \nAlthough the use of rule sets is quite common in tabular data, its less prevalent in NLP and I don’t recall seeing the construction of rule set classifiers for explaining sequence level tasks before so its quite novel ( though the authors mention Bayesian Rule Lists which is less efficient method ).  \n\n**Weakness: **  \nThe paper's argument could be made stronger.  The baseline they compare against seem pretty weak ( they allude to its limitations in the related work section ) and it seems like comparing SAGE with additional methods such as Decision trees would have more sense as well ( since SP-LIME performs very poorly in the task ).  More robust comparison of baselines/alternative methods  ( bayesian rules, global explain models, decision trees for NLP etc ) would benefit this paper along with some ablations including whether if instead of filtering as aggressively and using all rules if they'd get you 91% F1 on CoNLL?  If so, it seems perfectly acceptable as a solution if only a sparse amount of rules are used at each's instance explanation level.  \n\nDiscussing what it means for overlapping spans ( unigram, bigram, etc ) of the same word to be present in rules is also important.  Also qualitative analysis of rules extracted including human judgement on whether they are good/interpretable, because the only rules shown in the paper are of simple unigram words/features implying a NORP class.\n\nFinally, the accuracy of the BERT NER system on both tasks should be provided because thats really the upper limit of how well any explanation model could do.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "5HL3VsSp2Z-", "reviewer": "Reviewer_rFXT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors present SAGE or SEMANTIC-AWARE GLOBAL EXPLANATIONS model specifically for handling named entity recognition problems. They present a method to produce highly interpretable global rules to explain NLP classifiers. ", "summary_of_the_review": "The authors present SAGE specifically for handling named entity recognition problems and present a method to produce rules to explain NLP classifiers. The paper could have been improved by comparison to other baselines. ", "strength_and_weaknesses": "The authors have provided motivation to the problem, which is certainly important. They present their algorithm and their experiments on 2 datasets, compared against the SP-Lime baseline model. \n\nThe paper could have been improved by comparison to other baselines. Given that LIME and SP-Lime were released in 2016, more work has gone into the field of XAI. It is not clear why only LIME was chosen as the baseline. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "1bWnoxK-bD", "reviewer": "Reviewer_WCcH", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Paper presents a post-hoc method to produce interpretable global rules to explain NER classifiers. Rules are extracted with a data mining approach that gathers labeling rule patterns by FP-growth algorithms, prunes the rules by removing soft-duplicated rules, and selects rules that maximizes the F1 score. Selected rules serve well as a post-hoc global explanation for the NER model and show its better explanation quality than LIME-based global explanation.", "summary_of_the_review": "Paper presents a post-hoc method to produce interpretable global rules to explain NER classifiers and show its effectiveness.\nAlthough the paper needs more analysis to convince readers, I believe this approach can be useful to the future research of weakly-supervised learning and neural-symbolic learning for NER. I'm willing to increase the score when analysis is provided.", "strength_and_weaknesses": "Proposed approach shows its effectiveness in generating a global explanation for the NER model. I believe this approach can be useful to the future research of weakly-supervised learning and neural-symbolic learning for NER. However, I have a minor concern that the paper needs more analysis to convince readers. For example,\n\n(1) Could find that the F1 score between model prediction and rule-based prediction is over 0.4 with only one rule. It could be good to show qualitative examples of which rules are extracted “globally”, not just for the specific entity type (e.g., NORP).\n\n(2) Comparison of qualitative explanation examples between SAGE and LIME. Seems the word itself contributes a lot to labeling decision (e.g., Chinese → NORP), compared to the surrounding words or context. Then, I think word importance generated by LIME can also be a good explanation but it seems not. It would be good to show qualitative explanation examples between SAGE and LIME to show the effectiveness of SAGE.\n\n(3) How does each semantic/syntactic information (POS tag, entity types, is_digit, … etc.) contribute to the performance? Seems entity prediction gives a lot of information to create rules.\n\n(4) What is the performance of the fine-tuned BERT model as it is?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-PmEP2b3Rik", "reviewer": "Reviewer_4Q6v", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes SAGE, a method that extracts semantic-aware global explanations that can be applied to NLP in general and specifically experiments on name entity recognition.", "summary_of_the_review": "My major concerns are that (1) the paper is not written well to deliver its ideas and results, and (2) according to the paper’s statement to be general in NLP, I would anticipate to see discussion about other works on explanations of NLP in this paper, which are not mentioned. I would raise my score if these are clarified.", "strength_and_weaknesses": "### Strengths:\n* The proposed method, SAGE, is potentially useful.\n\n### Weaknesses:\n* The paper may need major revision in its writing.\n  * The paper is not easy to read for me, especially the method section and its notations without complete definitions, e.g., mingensup, mingencon, maxrulelen, maxnrules, SORT-SUPPORT-AND-ABSTRACTION, etc. I need to guess their meanings.\n  * The paper can be further proofreading. For example,\n    * in Section 2, “Finally, Performing…” has a wrong uppercase.\n    * In Section 3.1, “..explanation method, the In Algorithm 1 the …”\n  * Most of the citations in this paper are not in a correct format. Please change \\cite{} to \\citep{}.\n* The paper lacks discussion about other explanations for NLP works while this paper claims to be a general method for NLP but test on NER, such as (I'm not asking you to cite them but would like to see its position in explanation for NLP field):\n  * David Alvarez-Melis and Tommi Jaakkola. “A causal framework for explaining the predictions of black-box sequence-to-sequence models.” In EMNLP 2017.\n  * Hanjie Chen and Yangfeng Ji. “Learning variational word masks to improve the interpretability of neural text classifiers.” In EMNLP 2020.\n  * Yi-Lin Tuan, Connor Pryor, Wenhu Chen, Lise Getoor, and William Yang Wang. \"Local explanation of dialogue response generation.\" In NeurIPS 2021.\n  * Hanjie Chen, Song Feng, Jatin Ganhotra, Hui Wan, Chulaka Gunasekara, Sachindra Joshi, and Yangfeng Ji. “Explaining neural network predictions on sentence pairs via learning word-group masks.” In NAACL 2021.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject