Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a new LM-based method for extracting rules from a pre-trained masked language model. The key idea is fine-tuning a pre-trained language model (BART [16]) with external relational description corpora to two masked language models (MLMs). For every given rule body, the first MLM predicts ground substitutes (which are named entities) in the rule body and the second MLM predicts text sequences except the ground substitutes in the rule head. Thus, the proposed method is able to compute multiple rule heads for a given rule body. Experimental results on a manually constructed rule set and six relation-extraction datasets demonstrate that the proposed method outperforms state-of-the-art LM-based methods and is comparable to state-of-the-art KB-based methods in finding correct rules. Further results also show that the generated rules can make relation extraction more accurate than manually annotated rules do.
main_review: Learning natural language rules is useful in many real-life applications. For example, in relation extraction, the learned rules can be used as templates to discover the relationship between two entities. Thus, a more effective way to learn natural language rules is very desirable. The paper presents a LM-based method for this task and can have a significant impact. Different from the prompt-based method for generating text sequences from LMs [20], the proposed method does not directly generate text sequences in rule heads; instead, it first generates multiple ground substitutes in the given rule body, and then generates text sequences in the rule head according to the generated ground substitutes. This way in generating text sequences in rule heads is novel to the best of my knowledge. As far as I can see from the paper, the proposed method is technically sound and justified by adequate experiments, although no source code is provided in the supplemented material. The paper is generally well organized and written, but the presentation could still be improved in the following aspects. First, the restrictions on learned rules need to be given explicitly. For example, all variables appearing in learned rule heads should appear in the given rule body as well; the structure of rule heads to be learned is predefined. Second, the accuracy metric used in Table 2 needs more clarification. The estimation of this metric requires criteria for manually deciding whether a learned rule is correct or not. These criteria should be clarified. Finally, the application for applying learned rules to identify errors in LMs is problematic. On one hand, the incorrectness of a learned rule may be caused by the proposed learning method (e.g. an inappropriate value of the beam search width) rather than by the LM. Different causes are hard to be distinguished. On the other hand, even if we know that the LM has errors, we cannot correct the LM based on the learned rules. In other words, the motivation for this application is very unclear. The effectiveness of this application needs to be carefully demonstrated.
limitations_and_societal_impact: No, the authors have not adequately addressed the limitations of their work. Besides the restrictions on learned rules that should be given explicitly, the dependency between the performance of learned rules in downstream tasks and the beam search width (which determines the number of rule heads learned for one rule body) needs to be explored.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a novel method of inducing probabilistic horn clauses (x founded y) --> (x ceo of y) from the predictions of a masked language model. This is done by first mapping the premise to instances (jobs founded apple) and then decoding the predicate for the hypothesis. The approach is interesting, but the current paper is let down by the quality of the experiments, which do not give a clear picture of how accurate or useful the induced rules are.

main_review: This paper presents a method of generating probabilistic Horn cluases using a language model. The probability of a hypothesis (x is ceo of y) following a premise (x founded y) is broken down into a sum over instances of the premise, and the conditional probability of the hypothesis given these instances. This factorization permits the use of a masked language model that either predicts entities that should be substituted for some variable (instantiation of the premise) or the relation that links a pair of entities (the mapping from support to hypothesis). This is a clever way of adopting a masked language model to reason about rules that hold over sets of entity pairs, but it is also not clear to what extent the resultant probabilities properly capture the probability of a rule holding, rather than the probability of a specific verbalisation of that rule.

A BART language model is trained to model these probabilties using weak supervision via a masking scheme that focuses on pairs of entities and beam search is used to decode the instances and hypotheses. A new beam search strategy is introduced to avoid issues arising from greedy decoding of instances and hypotheses, but this method only marginally impacts the model's performance.

The approach is evaluate against the COMET system as well as an unmodified BART + prompt on a newly constructed evaluation set as well as evaluation sets that are automatically constructed from some standard relation extraction benchmarks. The proposed approach outperforms the baselines according to BLEU, ROUGE, and METEOR, but it's not clear how well these text generation metrics capture the accuracy of the generated rules. Table 3 clearly shows that the new approach is hallucinated unsupported hyptoheses and, from the examples in this table, it is not clear that COMET is a suitable baseline for the evaluation setting.

There is also a comparison to KB-based rule induction which uses human judgements of accuracy. Here the proposed approach achieves greater coverage at a small cost in terms of accuracy. I would like to see some discussion of how accuracy was determined. Almost all of the examples in Figure 1 and Table 3 are not universally true, if not completely hallucinated. A clear definition of what the authors consider to be a 'good rule' would be useful in interpreting this evaluation.

A further evaluation replaces the hand crafted explanations used in ExpBert with rules induced by ORION and shows that this matches ExpBert performance (mean performance is even slightly higher, but within variance). This is an interesting result but I am confused about the singificant discrepency between the numbers reported in Table 4 and the ExpBert paper. The authors mention that they have a more performant training scheme. but the results reported for the BERT baseline are significantly lower than those from the ExpBert paper, while the results for ExpBert + annotated rules are significantly higher. Given that this experiment is intended to demonstrate the value of the induced rules, I would like to see a direct comparison to the ExpBert system that only changes the form of the supplied rules.

----

Score raised from 5 -> 6 after response from authors. See reasoning in response to response.

limitations_and_societal_impact: This paper relies on generation metrics (BLEU, ROUGE, METEOR) to evaluate the quality of induced logical rules. The current paper does not justify this choice and these metrics do not give a clear picture of how useful the rules are, or whether the increase over COMET is due to genuinely better inference or simply task mismatch. I would like to see an expansion of the human evaluation provided in Section 5.4, as well as accuracy metrics for the comparison to COMET.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper focuses on inducing open rules (i.e., (x, r_p, y) ==> (x, r_h, y)) from the large-scale pre-trained language models (Specifically, BART) and proposes "Orion" model trained without any hand-crafted open rules. Although previous works attempt to generate these rules using language models, these approaches usually train (or fine-tune) the langauge models using annotated open rule data. The resulting langauge models can only generate "canned" rules. This paper adapts the idea of probing languge model knowledge using prompts. Two seperated langaugue models (i.e., BART) are designed to map from r_p to (x, y) and map from (x, y) to r_h, respectively. To guarantee the langauge models produce meaningful output, this paper further trains these langauge models using large-scale data annotated by on-the-shelves NER systems. This paper conducts extensive experiments to verify the quality and quantity of the inducted open rules. The models trained on the extracted rules even outperform the ones trained on manually annotated rules.
main_review: Originality: 

Open rule Induction is not a novel task. There are a few mature systems to induce open rules from open knowledge bases. The mainly novelty of this paper is to use langague models to complete this tasks without using any annotated open rules. Specifically, this includes:
    - two seperated langauge model to model r_p and r_h; 
    - a specially designed beam search algorithms to take the entity pairs (x, y) into account and 
    - using large-scale silver training data (entities annotated by on-the-shelves NER systems) to further train the langueg model to generate relavant (x, y) content.
However, the main approach is adapted from previous work that uses masking-based prompts to probe languge model knowledge. In addition, although authors claim that they mainly make use the knowledge within the langauge models, their "continuing train" phrase uses "93.63 millions" training samples. I believe there are significant amount of open rules can be found within this data. It becomes unclear to me whether this approach simply relies on these magic "93.63 millions" training samples.  It will be much better if the authors could provide more detailed ablation study on the effect of the size of this continuing training sample to the overall performance. If this approach can really effectively extract the underlying knowledge in the langauge models, the size of this continuing training sample should be relatively samll. The main goal of introducing this continuing training samples is to teach the language models to output relation-related text. In many NLP tasks, a few thousands of training samples is enough to fine-tune the langaguge models. I also notice that in evaluation, only the proposed "Orion" model is trained with this large corpus. It is interesting to see the performance of the baseline models trained with the same data. For example, The Prompt baseline model in Table 1. 

Quality & Clarity:

This paper presents a fairly complete work with various experiments, including generated relation, relation disversity and case study, verifying the authors' claims. However, the authors do not provide weakness of their proposed model. It may be interesting to see if the proposed model is too diverse in some cases given that it is trained on very large collected data. The paper is well written and easy to follow. I believe it should be straightforward to reproduce the method described in this paper. To improve the reproducibility, it will be great if the authors can open-source the continue training data (although I believe this paper lacks of experiments about its importance).

Significance:

Extracting knowledge from large-scale pre-trained langauge model should have a board impact on NLP community. This paper shows that for complicated task like open rule induction, it is important to further train the langauge models. I agree that we always need fine-tune to make langauge model follow our instructions. However, this is not something supervising for people who use pre-trained langauge models. It will be much more interesting to discuss how much cost we need to pay in order to extract the unlimited knowlege from pre-trained language models. If this paper can better address this issue, its impact will be significantly improved.

limitations_and_societal_impact: n/a
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a method to induce “open-domain” rules from pre-trained language models like BART, useful for tasks like relation extraction. Rule induction through knowledge bases is constrained by a “closed-domain”, while recent work, as discussed in the paper shows the possibility of using language models as an open-domain knowledge base. The proposed method considers rules of the form (x, r_p, y) => (x, r_h, y), where r_p and r_h are represented in natural language. The task is to predict r_h  given r_p, by aggregating over all the instantiations of (x, r_p, y). The paper presents an interesting method of utilizing the pre-trained LMs and a supported-beam-search method of aggregating the information from the top-k supporting instantiations. Experimental results look promising. However, I doubt whether the task and the kind of rules considered in this paper are general enough for wider applications.
main_review: Pros:
- Leveraging pre-trained LMs for Open-domain rule induction.
- A smart way of aggregating information from all possible instantiations using supported beam search.
- Utilizing weak supervision to train relation and instance generation modules. This helps the proposed method outperform methods like Comet (previous work) which depend on labeled/annotated data.
- Improvements over LM baseline, and Comet. 
- The inducted rules prove beneficial for downstream tasks like relation extraction. 

Limitations:
- It seems that the proposed method assumes that premises r_p are already given (E.g. lin 255, 256). It is not clear how the body of the rule (x, r_p, y) is extracted.  
- When compared to KB-based rule induction the method is somewhat worse than the best baseline, however, one benefit of the proposed method is that it outputs more rules with decent quality.
- Even though “Error identification in LMs” is posed as a utility of the proposed approach, it also seems like a limitation of the proposed method being less robust and producing over-generalized rules.

limitations_and_societal_impact: Yes.
review_title: null

Meta-review: Automatically extracting logic rules from unstructured data can be practically useful. The proposed rule extraction method based on pre-trained language models is novel and flexible. The empirical results are good on different tasks. The rebuttals clarified lots of issues raised in the initial reviews. An immediate limitation of this work is that the bias of pre-trained LMs could result in unexpected rules.
Decision: Accept (Poster)