Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: In this paper the authors introduce a new post-hoc method to create explanations for a black box classifier. They explain the steps in the method using the task of Named Entity Recognition as example. They turn the data into baskets of information , by extracting the contextual window around each word as well as some semantic features like pos tags, synsets etc and then use a data mining algorithm to extract patterns of co-occurrence corresponding to each classification label. Pruning is done over these patterns or candidate explanations based on thresholds and explanations are then selected greedily using f1 score maximization. The perform evaluation on this method by measuring the F1 score vs complexity defined by the number of explanations and they compare this to variations of LIME as well as a version of their model without semantic information

strength_and_weaknesses: Strengths:

	1. They support the necessity for each step in the algorithm
	2. They provide global explanations
	3. They are task agnostic , method can be applied to many NLP tasks

Weaknesses:
	
	1. In section 3.1 they do not specify what certain notations mean  , eg the difference between the two transaction tables on the right of figure 2. 
	2. Jump from section 3.2 to 3.3 is big especially for people who are unfamiliar with algorithms they point to such as FP-growth Han et al. (2000) and apriori Agrawal et al. (1994). They use an example for section 3.1 but then they drop the example for subsequent sections in the algortihm . 
	3. Other evaluation metrics employed by other papers eg, fidelity to the model and comprehensibility could have been explored . Human evaluations might make a more compelling case .
	4. They don’t perform any study about which semantic features help and which harm the f1 score.
	5. Visualizaition is an important part of explainable models which this paper lacks


clarity,_quality,_novelty_and_reproducibility: Clarity of the paper is a bit rocky.  They use some phrases repeatedly in multiple sections but do not elaborate them . Eg.They claim one caveat of LIME is that explanations must be positional. But their method is also dependent on a fixed contextual window .
Novelty exists , in the transforming of the data into a data mining task and incorporating semantic information.
Reproducibility is good since they list the hyperparameters used for the datasets mentioned in the paper. But it is unclear how much of finetuning of hyperparamers took place and how resilient the method is to any change in these values and its effectiveness for a different dataset or different task.
summary_of_the_review: Although the fundamental idea is good, it fails to convince that the method is robust across different datasets or NLP tasks.
They employ a contextual window of +/-3 tokens and that might not be transferrable to other NLP classification tasks like sentiment classifier or recommendation systems.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes SAGE, a method that extracts semantic-aware global explanations that can be applied to NLP in general and specifically experiments on name entity recognition.
strength_and_weaknesses: ### Strengths:
* The proposed method, SAGE, is potentially useful.

### Weaknesses:
* The paper may need major revision in its writing.
  * The paper is not easy to read for me, especially the method section and its notations without complete definitions, e.g., mingensup, mingencon, maxrulelen, maxnrules, SORT-SUPPORT-AND-ABSTRACTION, etc. I need to guess their meanings.
  * The paper can be further proofreading. For example,
    * in Section 2, “Finally, Performing…” has a wrong uppercase.
    * In Section 3.1, “..explanation method, the In Algorithm 1 the …”
  * Most of the citations in this paper are not in a correct format. Please change \cite{} to \citep{}.
* The paper lacks discussion about other explanations for NLP works while this paper claims to be a general method for NLP but test on NER, such as (I'm not asking you to cite them but would like to see its position in explanation for NLP field):
  * David Alvarez-Melis and Tommi Jaakkola. “A causal framework for explaining the predictions of black-box sequence-to-sequence models.” In EMNLP 2017.
  * Hanjie Chen and Yangfeng Ji. “Learning variational word masks to improve the interpretability of neural text classifiers.” In EMNLP 2020.
  * Yi-Lin Tuan, Connor Pryor, Wenhu Chen, Lise Getoor, and William Yang Wang. "Local explanation of dialogue response generation." In NeurIPS 2021.
  * Hanjie Chen, Song Feng, Jatin Ganhotra, Hui Wan, Chulaka Gunasekara, Sachindra Joshi, and Yangfeng Ji. “Explaining neural network predictions on sentence pairs via learning word-group masks.” In NAACL 2021.

clarity,_quality,_novelty_and_reproducibility: #### Clarity:
The paper is not easy to read for me.

#### Quality:
The paper needs major revision in its delivery. Many details are skipped or just refer to figures, tables, or algorithm blocks. I’m not sure how the F1 metric is computed in the end. Does it mean that the authors only feed the black-box classifier the explanations and use the NER F1 score to evaluate explanations?

#### Novelty:
The paper misses two lines of papers, (1) papers about explanation in NLP while claiming the method is general to explanations for NLP tasks (as listed in the weaknesses), (2) papers about another line of works often also considered as local explanation, Shapley value.
Lloyd S Shapley. “A value for n-person games.”
Erik Strumbelj and Igor Kononenko. “An efficient explanation of individual classifications using game theory.” The Journal of Machine Learning Research, 11:1–18, 2010

#### Reproducibility:
The paper may not be easy to reproduce in its current status.
summary_of_the_review: My major concerns are that (1) the paper is not written well to deliver its ideas and results, and (2) according to the paper’s statement to be general in NLP, I would anticipate to see discussion about other works on explanations of NLP in this paper, which are not mentioned. I would raise my score if these are clarified.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Paper presents a post-hoc method to produce interpretable global rules to explain NER classifiers. Rules are extracted with a data mining approach that gathers labeling rule patterns by FP-growth algorithms, prunes the rules by removing soft-duplicated rules, and selects rules that maximizes the F1 score. Selected rules serve well as a post-hoc global explanation for the NER model and show its better explanation quality than LIME-based global explanation.
strength_and_weaknesses: Proposed approach shows its effectiveness in generating a global explanation for the NER model. I believe this approach can be useful to the future research of weakly-supervised learning and neural-symbolic learning for NER. However, I have a minor concern that the paper needs more analysis to convince readers. For example,

(1) Could find that the F1 score between model prediction and rule-based prediction is over 0.4 with only one rule. It could be good to show qualitative examples of which rules are extracted “globally”, not just for the specific entity type (e.g., NORP).

(2) Comparison of qualitative explanation examples between SAGE and LIME. Seems the word itself contributes a lot to labeling decision (e.g., Chinese → NORP), compared to the surrounding words or context. Then, I think word importance generated by LIME can also be a good explanation but it seems not. It would be good to show qualitative explanation examples between SAGE and LIME to show the effectiveness of SAGE.

(3) How does each semantic/syntactic information (POS tag, entity types, is_digit, … etc.) contribute to the performance? Seems entity prediction gives a lot of information to create rules.

(4) What is the performance of the fine-tuned BERT model as it is?

clarity,_quality,_novelty_and_reproducibility: The writing is clear and the proposed approach is classic but its application is novel. 
It demonstrates an effective approach to generating a global explanation for the NER model, which is unexplored. 
I have a minor concern about the reproducibility.
summary_of_the_review: Paper presents a post-hoc method to produce interpretable global rules to explain NER classifiers and show its effectiveness.
Although the paper needs more analysis to convince readers, I believe this approach can be useful to the future research of weakly-supervised learning and neural-symbolic learning for NER. I'm willing to increase the score when analysis is provided.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors present SAGE or SEMANTIC-AWARE GLOBAL EXPLANATIONS model specifically for handling named entity recognition problems. They present a method to produce highly interpretable global rules to explain NLP classifiers. 
strength_and_weaknesses: The authors have provided motivation to the problem, which is certainly important. They present their algorithm and their experiments on 2 datasets, compared against the SP-Lime baseline model. 

The paper could have been improved by comparison to other baselines. Given that LIME and SP-Lime were released in 2016, more work has gone into the field of XAI. It is not clear why only LIME was chosen as the baseline. 

clarity,_quality,_novelty_and_reproducibility: The paper is fairly easy to read. However, most terms used in the algorithms are not defined before using them, and one has to refer to subsequent sections to understand their meaning. This reduces the readability of the paper.
The approach is seems relatively novel, but it is hard to appreciate given the lack of baselines.
There is limited details about the implementation and it is not clear if one can reproduce the results in the paper.
summary_of_the_review: The authors present SAGE specifically for handling named entity recognition problems and present a method to produce rules to explain NLP classifiers. The paper could have been improved by comparison to other baselines.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors of this work propose SAGE, a data mining method to discover global explanation rules for black box text classifiers using semantic augmented features in a post-hoc fashion.  The method first leverages FP-growth (Han et al 2000) to generate candidate implication rules of the form (t1, t2, .. tn) -> y  ( where t1, t2, etc delineate the presence of tokens and semantic features within a window and y is a class for a given task ).  They then show how to filter the generated set of rules and select a final concise set of rules.  They evaluate SAGE on the NER task, by training BERT on the task and using SAGE to explain its predictions.  They compare SAGE against SP-Lime-k which is global submodular based variant of LIME and an ablated version of SAGE which removes semantic augmentation.  They show SAGE outperforms the other methods on the CoNLL03 and OntoNotes datasets, semantic augmentation increases accuracy and additionally provide some examples of rules the system learns which highlights the utility of adding semantic augmentation.   
strength_and_weaknesses: **Strengths:**  
Although the use of rule sets is quite common in tabular data, its less prevalent in NLP and I don’t recall seeing the construction of rule set classifiers for explaining sequence level tasks before so its quite novel ( though the authors mention Bayesian Rule Lists which is less efficient method ).  

**Weakness: **  
The paper's argument could be made stronger.  The baseline they compare against seem pretty weak ( they allude to its limitations in the related work section ) and it seems like comparing SAGE with additional methods such as Decision trees would have more sense as well ( since SP-LIME performs very poorly in the task ).  More robust comparison of baselines/alternative methods  ( bayesian rules, global explain models, decision trees for NLP etc ) would benefit this paper along with some ablations including whether if instead of filtering as aggressively and using all rules if they'd get you 91% F1 on CoNLL?  If so, it seems perfectly acceptable as a solution if only a sparse amount of rules are used at each's instance explanation level.  

Discussing what it means for overlapping spans ( unigram, bigram, etc ) of the same word to be present in rules is also important.  Also qualitative analysis of rules extracted including human judgement on whether they are good/interpretable, because the only rules shown in the paper are of simple unigram words/features implying a NORP class.

Finally, the accuracy of the BERT NER system on both tasks should be provided because thats really the upper limit of how well any explanation model could do.
clarity,_quality,_novelty_and_reproducibility: The paper is mostly clearly written and easy to follow though there are few minor things that could help such as (1) explicitly mentioning Y are the blackbox predictions and not gold truth labels ( this is apparent in Fig 1, but not in section 3.1 ), (2) the last sentence of Section 3.1 right before Algorithm 1 seems cut off (3) defining minimum support and minimum confidence explicitly, etc.  The work could be reproduced for the most part.
summary_of_the_review: Its an interesting idea overall, but feels under explored in the paper and could be made stronger and more compelling if it had more robust comparison of baselines/alternative methods, ablations and qualitative analysis which really highlighted why the method should be used.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper studies explainability in NER.
It puts forward a   post-hoc method to generate interpretable global rules to explain NER classification decisions.
The reviewers agree that approach can  potentially be impactful.  However, in its current form it requires stronger baselines from the literature, ablations and qualitative analysis to help convince the reader why the approach is useful.  The authors are encouraged to continue their work, address reviewer concerns, and  resubmit at a future conference at ICLR or other venues.
justification_for_why_not_higher_score: 
There baselines are weak, and the paper needs additional analysis work.
justification_for_why_not_lower_score: n/a
Decision: Reject