Open Rule Induction

Abstract: Rules have a number of desirable properties. It is easy to understand,  infer new knowledge, and communicate with other inference systems. One weakness of the previous rule induction systems is that they only find rules within a knowledge base (KB) and therefore cannot generalize to more open and complex real-world rules. Recently, the language model (LM)-based rule generation are proposed to enhance the expressive power of the rules.In this paper, we revisit the differences between KB-based rule induction and LM-based rule generation. We argue that, while KB-based methods inducted rules by discovering data commonalitiess, the current LM-based methods are learning rules from rules''. This limits these methods to only producecanned'' rules whose patterns are constrained by the annotated rules, while discarding the rich expressive power of LMs for free text.Therefore, in this paper, we propose the open rule induction problem, which aims to induce open rules utilizing the knowledge in LMs. Besides, we propose the Orion (\underline{o}pen \underline{r}ule \underline{i}nducti\underline{on}) system to automatically mine open rules from LMs without supervision of annotated rules. We conducted extensive experiments to verify the quality and quantity of the inducted open rules. Surprisingly, when applying the open rules in downstream tasks (i.e. relation extraction), these automatically inducted rules even outperformed the manually annotated rules.


Reviews and author responses:

 Glad to see that our approach has convinced you more. Thank you for your advice. Our paper will focus on open rules with one atom as we defined in Definition 1 (line 82). While in addition, to clarify the problem boundary as you suggested, we will add the discussion on existential rules after the problem definition in Sec 2.2. Furthermore, we agree that the study of safe substitution and existential rules over our proposed open rules is interesting but still primitive now. We leave this as our future work.

 Thanks the authors for the detailed explanations. They make the proposed approach more convincing to me. I hope the authors will add the clarifications on which kinds of rules that the proposed approach can learn in the revised paper, as well as add some examples to support the declarations. In general, for a research work to achieve high influence, it is crucial to provide a clear boundary on what the proposal can do and what it cannot, so that followers are easier to inspire from or advance this research work. It is not hard to see that the support of existential rules will give another issue about the proposed approach, i.e., every safe rule learnt by the approach can be modified to  at least one existential rule if at least one named entity appears in the rule, making the subsequent step for selecting top rules from candidate rules harder. I do not mean that a conference paper needs to solve all problems, but some side effects can at least be mentioned in the discussion or the conclusion part.

 Thanks for the response! This is a really inspiring comment.

We noticed that the reviewer's main concern is whether our method can generate those complex existential rules with explicit new variables that can be used in a rule system. For example, in *"<mask1> has parent $\rightarrow$ <mask1> has father called <mask2>"*, the reviewer concerned that the new variable *<mask2>* makes the hypothesis more complex and difficult to generate. We will show that **our method is able to generate the above existential rules with new variables by just adding a special entity indicator token, without increasing the search space of the generation.**

Consider the rule provided by the reviewer: 

$\forall x HasParent(x) -> \exists zHasFather(x,z)$. 

It can be described by natural language:

(A) $\forall x$  *x has parents.* $\rightarrow$ $\exists z$ *x has father z.*

So far, Eq. (3) does not generate hypotheses containing new variables. It first instantiates all entities in the premise by $P(ins|r_p)$ and then generates hypotheses satisfying these instantiations. However, for the existential rule (A), we cannot instantiate $z$ since it is not in the premise. This is the main challenge in generating the existential rules.

We want to highlight that, if we only need to consider the existence of $z$, **$z$ does not need the instantiation**. Instead, **we only generate a special indicator token $<z>$ to indicate the existence of an entity $z$ in the hypothesis**. 

Specifically, we define the existential prompt for $P(r_h|ins)$ as *x <mask>$_e$*. Here the only difference between $<mask>_e$ here and the regular prompt's $<mask>$ on line 127 is that the prediction for $<mask>_e$ contains a special entity indicator $<z>$ to represent the new entity. For example, for *x= George W. Bush*, one of the prospective predictions for the existential prompt is *"George W. Bush has father <z>"*.

We have tested this method with experiments. In order to adapt to existential prompts, the language model needs to generate text with the entity indicator $<z>$. For this purpose, we slightly modify the weak supervision process in Sec 3. We transform the original sentence by replacing its second entity with $<z>$ via NER.

For example, for text *"George W. Bush has father George H. W. Bush"*, the existential prompt and the prospective output are:

**Prompt**: *George W. Bush <mask>$_e$.*

**Output**: *George W. Bush has father <z>.*

The only difference in this continue training objective is that we replace the second entity with token $<z>$. During the rebuttal phase, we only continued training BART over Wikipedia for 500 steps. After the continue-training, we used Eq. (3) to generate these existential rules as usual. Below are some top-3 hypotheses generated by Orion based on this BART. Orion successfully generated hypotheses with new variables.
```
+--------------------------------------------------------------+
|         Example of Generation with New Variable <z>          |
+-----------------------------+--------------------------------+
| r_h: [X] has parents.       | r_h: [X] is a big city.        |
|                             |                                |
| r_p:                        | r_p:                           |
|                             |                                |
| [X] was born in <z>.        | [X] is a city in <z>.          |
|                             |                                |
| [X], <z> and troop members. | [X] was the capital of <z>.    |
|                             |                                |
| [X] is the daughter of <z>. | [X] is a district in           |
|                             | the state of <z>.              |
+-----------------------------+--------------------------------+
```
We hope this would help you better understand our method.

This paper presents a novel method of inducing probabilistic horn clauses (x founded y) --> (x ceo of y) from the predictions of a masked language model. This is done by first mapping the premise to instances (jobs founded apple) and then decoding the predicate for the hypothesis. The approach is interesting, but the current paper is let down by the quality of the experiments, which do not give a clear picture of how accurate or useful the induced rules are.
  This paper presents a method of generating probabilistic Horn cluases using a language model. The probability of a hypothesis (x is ceo of y) following a premise (x founded y) is broken down into a sum over instances of the premise, and the conditional probability of the hypothesis given these instances. This factorization permits the use of a masked language model that either predicts entities that should be substituted for some variable (instantiation of the premise) or the relation that links a pair of entities (the mapping from support to hypothesis). This is a clever way of adopting a masked language model to reason about rules that hold over sets of entity pairs, but it is also not clear to what extent the resultant probabilities properly capture the probability of a rule holding, rather than the probability of a specific verbalisation of that rule.

A BART language model is trained to model these probabilties using weak supervision via a masking scheme that focuses on pairs of entities and beam search is used to decode the instances and hypotheses. A new beam search strategy is introduced to avoid issues arising from greedy decoding of instances and hypotheses, but this method only marginally impacts the model's performance.

The approach is evaluate against the COMET system as well as an unmodified BART + prompt on a newly constructed evaluation set as well as evaluation sets that are automatically constructed from some standard relation extraction benchmarks. The proposed approach outperforms the baselines according to BLEU, ROUGE, and METEOR, but it's not clear how well these text generation metrics capture the accuracy of the generated rules. Table 3 clearly shows that the new approach is hallucinated unsupported hyptoheses and, from the examples in this table, it is not clear that COMET is a suitable baseline for the evaluation setting.

There is also a comparison to KB-based rule induction which uses human judgements of accuracy. Here the proposed approach achieves greater coverage at a small cost in terms of accuracy. I would like to see some discussion of how accuracy was determined. Almost all of the examples in Figure 1 and Table 3 are not universally true, if not completely hallucinated. A clear definition of what the authors consider to be a 'good rule' would be useful in interpreting this evaluation.

A further evaluation replaces the hand crafted explanations used in ExpBert with rules induced by ORION and shows that this matches ExpBert performance (mean performance is even slightly higher, but within variance). This is an interesting result but I am confused about the singificant discrepency between the numbers reported in Table 4 and the ExpBert paper. The authors mention that they have a more performant training scheme. but the results reported for the BERT baseline are significantly lower than those from the ExpBert paper, while the results for ExpBert + annotated rules are significantly higher. Given that this experiment is intended to demonstrate the value of the induced rules, I would like to see a direct comparison to the ExpBert system that only changes the form of the supplied rules.

----

Score raised from 5 -> 6 after response from authors. See reasoning in response to response.
 This paper relies on generation metrics (BLEU, ROUGE, METEOR) to evaluate the quality of induced logical rules. The current paper does not justify this choice and these metrics do not give a clear picture of how useful the rules are, or whether the increase over COMET is due to genuinely better inference or simply task mismatch. I would like to see an expansion of the human evaluation provided in Section 5.4, as well as accuracy metrics for the comparison to COMET.


 Thanks for the clarifications and new results.

Given the fairly subjective nature of the judgements that you detail in your response to Q2, I suggest that you make these decision criteria clear in your discussion of what is needed for a 'good rule'. In fact, due to the significant differences with traditional or probabilistic logics, I would suggest you go even further and outline some of these decisions in the introduction, to prevent confusion for readers who are used to more formal definitions. 

The new results, and clarifications about experiments, are useful. Thank you for these. Together with the other clarifications, I feel that these are sufficient for me to raise my score to be in line with the other reviewers.

 Thanks for the prompt response, making the answer to Q3 more convincing to me. However, the response on Q1 still cannot address my concern due to different semantics on the following three rules.

(1) ∀xHasParent(x)→∃zHasFather(x,z)

(2) ∀x HasParent(x)→HasFather(x,f(x))

(3) ∀xHasParent(x)→HasFather(x)

Rule (1) introduces a new entity z that can be used elsewhere. The representation “<mask> has parent ->  <mask> has father.” neglects the introduction of a new entity and cannot be treated as a representation for rule (1). It seems that rule (1) could be expressed by something like "<mask1> has parent ->  <mask1> has father called <mask2>.", but I am still wondering whether the proposed approach can generate such rules.
Rule (2) uses Skolemnization to translate rule (1) into a satisfiability-preserving formula. However, these two rules are not equivalent on every aspect. For example, rule (2) cannot introduce new **named** entities but rule (1) can. More importantly, It is very hard to find natural language terms to represent a Skolemnization term like f(x).
Rule (3) is a safe rule and cannot substitute for existential rules in a rule system. Existential rules enable new entities to be considered in a rule system, while safe rules only keep the entitiy domain closed.

 **Q1. A limitation that the proposed approach cannot learn existential rules, e.g., if something has a parent, it must have a father: "(x, has parent, y) -> exists z: (x, has farther, z)".**

**Response:** This is an inspiring question. Actually, there is no need to worry about our method not being able to represent existential rules. We will show that, every rule with existential quanfier $∃$ can be converted into Skolem normal form, which only has universal first-order quantifiers $∀$, while not changing its satisfiability. This process is called **Skolemnization**. Therefore, the satisfiability of horn rules in our method has already covered the so-called existential rules.

For example, for the rule proposed by the reviewer, we can convert it into a first-order logical expression as follows:

$∀xHasParent(x)→∃zHasFather(x,z)$

which is equivalent to

$∀x∃z HasParent(x)→HasFather(x,z)$

This means for any $x$, there exists a $z$ corresponding to $x$. By the Skolem function $z = f(x)$, we have

$∀x HasParent(x)→HasFather(x,f(x))$

The right-hand side actually has only one variable, $x$. This is the same as the satisfiability of

$∀xHasParent(x)→HasFather(x)$

This is a simpler horn rule that has only one entity. We can use *“<mask> has parent. $\rightarrow$ <mask> has father.”* to represent this rule. We have mentioned how to learn such rules in the previous response.

**Q3. Considering there are multiple impact factors for the correctness of learned rules (e.g. rule templates adopted besides the beam search width), how can we find true reasons from one impact factor (namely the pretrained language model) without considering other impact factors?**

**Response:** Indeed, it is hard to explain whether an inducted rule is caused by the language model or by some other factors of Orion. However, **identifying the presence or absence of errors in the language model is simpler and more feasible than explaining their impact factors.** We can consider an inducted erroneous rule as a potential error in the language model. **By putting the rule back into the language model, the presence of the error can be identified without the need to explain its impact factors.**

For example, for the rule *$r_p$: [X] is an instance of [Y]. $\rightarrow$ $r_h$: [X] is a lower house of [Y].* in Table 6, we can use the LM (i.e. BART) to generate the top-$k$ <X,Y> pairs for $r_p$ first, and then evaluate whether these <X,Y> pairs have the "is a lower house of" relationship. We can use perplexity as the metric to avoid the influence of the decoding algorithm when generating <X,Y> pairs and evaluating their relationship. Note that in this substitution process, determining the presence or absence of potential errors is only relevant to the language model and not to Orion.

We hope that this will help you better understand our method and applications. And we are willing to discuss with you if you have further confusion about our work.


 Thanks for the response. The comments about Q1 and Q3 may be misunderstood. Q1 refers to a limitation that the proposed approach cannot learn existential rules, e.g., if something has a parent, it must have a father: "(x, has parent, y) -> exists z: (x, has farther, z)". Q3 refers to an unconvising application of learned rules, namely finding errors on the pretrained language model from incorrect learned rules. The main concern on this application is that, considering there are multiple impact factors for the correctness of learned rules (e.g. rule templates adopted besides the beam search width), how can we find true reasons from one impact factor (namely the pretrained language model) without considering other impact factors?

 We thought that the reviewer may consider the continue training corpus critical based on Table 1. We would like to further provide some examples to clarify that the continue training **plays only the role of denoising the generative process in Eq. (3)** as we illustrated in line136-142. 

For example, to train the LM for P(r_h|instance), the sentence *“Steve Jobs is the founder of Apple”* will be converted into *“Steve Jobs <mask> Apple”* via NER for training. The continue training corpus itself does not contain any annotated rules. 

Indeed, Table 1 indicates that the continue training is necessary for Orion to improve the effect. However, this is because the generative results of vanilla LM (i.e., BART) is noisy for $P(ins|r_p)$ and $P(r_h|instance)$ in Eq. (3). It often generates undesirable texts conflicting with Eq. (3). To give reviewers more direct insight, we compare the predictions for $P(r_h|instance)$ by the vanilla LM and the LM after continue training as follows:
```
+--------------------------------------------------------------------------------+
|               Top 3 Generation of P(r_h|instance = (Berlin, Germany))          |
+------------------------------------+-------------------------------------------+
| Vanilla BART:                      | continue pre-training BART:               |
+------------------------------------+-------------------------------------------+
|                                    | 1. Berlin is the largest city in Germany. |
| 1. Ber in Germany.Berlin (...)...  |                                           |
|                                    | 2. Berlin was a major center of the       |
| 2. BerGesundheit, Germany, Germany |    printing industry in Germany.          |
|                                    |                                           |
| 3. BerBerlin - Germany.com.au      | 3. Berlin has one of the highest          |
|                                    |    rates of unemployment in Germany.      |
+------------------------------------+-------------------------------------------+
```
The vanilla LM is often unable to generate relation description of two entities, namely that it is noisy for $P(r_h|instance)$. In contrast, continue training reduces the noise by ensuring the generated text to describe the relation of given entities. Therefore, the continue training is conducted only for denoising.

Thus, the continue training and its corpus are not the focus of our technique. The critical point of our method is to discover commonalities from LMs through unsupervised instantiation and generation, instead of learning rules from rules as in the previous LM-based rule generation.

We hope that this will help reviewers better understand the role of continue training for our method. And we are willing to discuss with reviewers about more specific details in our work.


 We thank the reviewer for the positive review and constructive advice. Besides, we noticed that the reviewer may concern about continue pre-training and its training corpus. Therefore, in the beginning, we would first like to clarify that such training corpus has no "magic" – it only contains NER annotations automatically generated by Spacy. The corpora come from Wikipedia and Bookcorpus, which are publicly available. No manual annotations are used by our method during the continuing pre-training. 

Regarding each of your questions:

**Q1. Whether this approach simply relies on these magic “93.63” million training corpus.**

**Response:** The corpus does not contain any annotations of rules. It only contains the automated NER annotations. The reason why our method benefits from these automated annotations are mainly because we decompose the complex rule induction problem into two simple generation problems related to entities. Other methods (e.g., prompt in Table 1) cannot benefit from the corpus because it requires the supervision of the hypothesis.

**Q2.  If this approach could effectively extract the underlying knowledge in the language models, the size of this continuing training sample should be relatively small.**

**Response:** Since the corpus uses only automated NER annotations, the cost of collecting it is very low. Therefore, it is not the focus of this paper to investigate the relationship between the size of the pre-training corpus and the effect of rule induction.

As a reference, in [a], the authors continued to pre-train the T5 model on NER annotations for 100,000 steps and found that this improved the model's effectiveness. Analogously, if we use less pre-training corpus, the induction should also be effective.

[a] how much knowledge can you pack into the language models, Roberts, A., Raffel, C. and Shazeer, N., 2020

**Q3. To improve the reproducibility, it will be great if the authors can open-source the continue training data.**

**Response:** As we stated in line 148-149, the corpora for continuing pre-training (Wikipedia and Bookcorpus) are open source. We only used Spacy's NER tagger to automatically annotate it without any manual annotations. We will open-source the code for the annotation.

**Q4. It is interesting to see the performance of the baseline models trained with the same data. For example, The Prompt baseline model in Table 1.**

**Response:** Here we would like to clarify why continuing pre-training is feasible for our method, but not for the prompt baseline. Our method contains two generation steps for P(r_h|ins) and P(ins|r_p), respectively. They are exactly predicting the information related to entity names. Thus, our method benefits from continuing pre-training over the NER annotation. However, the prompt baseline generates the hypothesis directly for a given premise. So fine-tuning it requires annotations of hypothesis, which are unavailable in an unsupervised scenario.

Inspired by the reviewer's concern, we came up with a stronger prompt baseline. We collect a set of such sentences "if sent1 then sent2" from Wikipedia and Bookcorpus and mask sent2. Then we continue to train the prompt baseline on top of this dataset. **The results are shown below. Here "new" represents the results of Prompt after continuing pre-training**. 

This new baseline still performs worse than Orion. We consider this is because it requires the training samples to contain the full if-then rules, which are uncommon in the pre-training corpus. In contrast, our method only requires the sentence to include either the body atom or the head atom, which substantially lowers the requirement for the pre-training corpus.

```
----------------------------------------------------------------------------
  OpenRule155    BLEU-1   BLEU-2   BLEU-4   ROUGE-L   METEOR   self-BlEU-2 
  Prompt         17.77    3.65     0.48     18.65     12.94      86.63     
  Prompt (new)   20.95    7.58     0.86     22.37     17.24      82.13     
  Orion          45.41    21.29    1.30     50.37     40.41      90.94     
----------------------------------------------------------------------------
```



 We thank the reviewer for the valuable and constructive feedback. Our responses to the questions of the reviewer are as below:

**Q1. It's not clear how well these text generation metrics capture the accuracy of the generated rules in Table 1.**

**Response:** Indeed, it’s hard to come up with a reasonable automatic evaluation metric. The metric BLEU used in this paper is from Comet's paper. We have done the following to overcome the difficulties of the evaluation.
1.	Manual evaluation. Please refer to Q2 below for the details.
2.	To further avoid subjectivity in the evaluation of rules, we demonstrate the effectiveness of generated heads by applying these rules to a downstream task with objective metrics (accuracy for relation extraction in Sec 6.1).

**Q2. A clear definition of what the authors consider to be a 'good rule' in Table 2 would be useful in interpreting the evaluation when comparing to KB-based rule induction.**

**Response:** During the human evaluation, we require a correct rule to be plausible, unambiguous, and fluent. For example, we label “x is a provincial capital of y $\rightarrow$ x is the largest city in y” as correct, because this inference is plausible to be valid. In contrast, we will label “x lives in y $\rightarrow$ x grew up in the east end of y” as incorrect, because the probability that x happens to grow up on the east end is too low. Besides, we will label “x is a provincial capital of y $\rightarrow$ x is a river of y” as incorrect, because the language model is ambiguous in understanding the concept of x or y on the head and body. We also give adequate examples and their evaluation results in the appendix.

Indeed, whether a rule is correct is still subjective. Therefore, to avoid subjectivity in the evaluation of rules, we demonstrate their effectiveness by applying these rules to a downstream task with objective metrics (accuracy for relation extraction in Sec 6.1).

**Q3. I would like to see a direct comparison to the ExpBERT system that only changes the form of the supplied rules.**

**Response:** Thanks for the advice! we have conducted such experiments as follow. 

While firstly, we would like to explain why we did not use the ExpBERT results directly in the paper. This is because their results were not obtained by direct fine-tuning from the vanilla BERT/ExpBERT, but was additionally modified. This modification includes fine-tuning the model on the MNLI dataset first and then freezing the parameters of BERT to conduct downstream tasks. Since this is not a common setting, we believe that the results of our paper are more general.

Secondly, **we also did the same experiment following the setup of ExpBERT. The results are shown in the table below**. Due to the pre-training on MNLI and freezing of parameters, the improvement of our method is reduced compared to BERT. But the conclusions of the new experiments are still consistent with our paper, i.e., our method is superior to BERT and the automatically inducted open rules are competitive or even slightly outperform the manually annotated rules.

```
--------------------------------------------------------
 OpenRule155                Spouse      Disease     
 BERT                       52.9(0.97)  49.7(1.01)  
 ExpBERT (annotated rules)  63.5(1.40)  52.4(1.23) 
 ExpBERT (inducted rules)   63.8(1.36)  52.6(1.20)  
--------------------------------------------------------
```


 We thank the reviewer for the positive review and constructive comments. Our responses to the questions are as below:

**Q1. It seems that all variables appearing in learned rule heads should appear in the given rule body as well.**

**Response:** The current rule requires that both body and head share a pair of entities $x$, $y$. Relaxing this restriction to require only one entity in the head (e.g., $x$) is straightforward. We could simply change the prompt in line 127 to: "x <mask>." to achieve this.

Furthermore, here we would like to discuss how rules could be extended to multiple body atoms below. 

Consider the rule: 
$body$: ($x$, is advisor of, $y$) $\land$ ($x$ graduates from $z$) $\rightarrow$ head: ($y$ works at $z$).
Given its body, in order to generate the head, a straightforward idea is to construct two separate probabilistic prompts: “$mask_x$ is an advisor of $mask_y$” and “$mask_x$ graduates from $mask_z$”, and then do logical $\land$ operation on them. However, the probability distributions of prompts are opaque, which makes the $\land$ operation intractable.

To represent the $\land$ operation, our idea is to use the power of the natural language to avoid the logical $\land$. We construct a single prompt:
“$mask_x$ is an advisor of $mask_y$ and $mask_x$ graduate from $mask_z$”
The semantics of this prompt is equivalent to the above logic $\land$ expression. Since it is a single prompt, we could still apply the decoding method proposed in this paper to generate the head: $x$ works at $z$.

Notice that the above method introduces a new problem, namely how to ensure the prompt has the same instantiation for two positions of 
$mask_x$. This could be achieved by adding a constraint to the decoding during instantiation. Besides, as the complexity of the prompt increases, whether the pre-training corpus suffers from sparsity problems will also need to be empirically verified. We leave this as future work.

**Q2. The accuracy metric used in Table 2 needs more clarification.**

**Response:** 

During the human evaluation, we require a correct rule to be plausible, unambiguous, and fluent. For example, we label “x is a provincial capital of y $\rightarrow$ x is the largest city in y” as correct, because this inference is plausible to be valid. In contrast, we will label “x lives in y $\rightarrow$ x grew up in the east end of y” as incorrect, because the probability that x happens to grow up on the east end is too low. Besides, we will label “x is a provicial capital of y $\rightarrow$ x is a river of y” as incorrect, because the language model is ambiguous in understanding the concept of x or y on the head and body. We also give adequate examples and their evaluation results in the appendix.

Indeed, whether a rule is correct is still subjective. Thus, to avoid subjectivity in the evaluation of rules, we demonstrate their effectiveness by applying these rules to a downstream task with objective metrics (accuracy for relation extraction in Sec 6.1).

**Q3. The incorrectness of a learned rule may be caused by the proposed learning method (e.g. an inappropriate value of the beam search width) rather than by the LM.**

**Response:** We believe the effect of the beam search could be easily avoided by evaluating the perplexity of the generated heads, since the perplexity will not be affected by the decoding algorithm.


 We thank the reviewer for the positive review and constructive comments. Our responses to the questions are as below:

**Q1. It is not clear how the body of the rule (x, r_p, y) is extracted (E.g. line 255, 256).**

**Response:** Premises in line255-256 are converted by the corresponding relations. See line 204-206.

**Q2. It also seems like a limitation of the proposed method being less robust and producing over-generalized rules.**

**Response:** Indeed, the bias of the language model leads to potential bias in rules. This is the limitation of our current work. We believe that this needs to be addressed by training more unbiased LMs.

However, from another perspective, since reducing the bias of LMs is a more fundamental issue, although it is not yet possible to reduce the bias of LMs by our rules, these rules help us to discover and evaluate the inferential bias of LMs.


This paper proposes a new LM-based method for extracting rules from a pre-trained masked language model. The key idea is fine-tuning a pre-trained language model (BART [16]) with external relational description corpora to two masked language models (MLMs). For every given rule body, the first MLM predicts ground substitutes (which are named entities) in the rule body and the second MLM predicts text sequences except the ground substitutes in the rule head. Thus, the proposed method is able to compute multiple rule heads for a given rule body. Experimental results on a manually constructed rule set and six relation-extraction datasets demonstrate that the proposed method outperforms state-of-the-art LM-based methods and is comparable to state-of-the-art KB-based methods in finding correct rules. Further results also show that the generated rules can make relation extraction more accurate than manually annotated rules do.  Learning natural language rules is useful in many real-life applications. For example, in relation extraction, the learned rules can be used as templates to discover the relationship between two entities. Thus, a more effective way to learn natural language rules is very desirable. The paper presents a LM-based method for this task and can have a significant impact. Different from the prompt-based method for generating text sequences from LMs [20], the proposed method does not directly generate text sequences in rule heads; instead, it first generates multiple ground substitutes in the given rule body, and then generates text sequences in the rule head according to the generated ground substitutes. This way in generating text sequences in rule heads is novel to the best of my knowledge. As far as I can see from the paper, the proposed method is technically sound and justified by adequate experiments, although no source code is provided in the supplemented material. The paper is generally well organized and written, but the presentation could still be improved in the following aspects. First, the restrictions on learned rules need to be given explicitly. For example, all variables appearing in learned rule heads should appear in the given rule body as well; the structure of rule heads to be learned is predefined. Second, the accuracy metric used in Table 2 needs more clarification. The estimation of this metric requires criteria for manually deciding whether a learned rule is correct or not. These criteria should be clarified. Finally, the application for applying learned rules to identify errors in LMs is problematic. On one hand, the incorrectness of a learned rule may be caused by the proposed learning method (e.g. an inappropriate value of the beam search width) rather than by the LM. Different causes are hard to be distinguished. On the other hand, even if we know that the LM has errors, we cannot correct the LM based on the learned rules. In other words, the motivation for this application is very unclear. The effectiveness of this application needs to be carefully demonstrated. No, the authors have not adequately addressed the limitations of their work. Besides the restrictions on learned rules that should be given explicitly, the dependency between the performance of learned rules in downstream tasks and the beam search width (which determines the number of rule heads learned for one rule body) needs to be explored.


This paper focuses on inducing open rules (i.e., (x, r_p, y) ==> (x, r_h, y)) from the large-scale pre-trained language models (Specifically, BART) and proposes "Orion" model trained without any hand-crafted open rules. Although previous works attempt to generate these rules using language models, these approaches usually train (or fine-tune) the langauge models using annotated open rule data. The resulting langauge models can only generate "canned" rules. This paper adapts the idea of probing languge model knowledge using prompts. Two seperated langaugue models (i.e., BART) are designed to map from r_p to (x, y) and map from (x, y) to r_h, respectively. To guarantee the langauge models produce meaningful output, this paper further trains these langauge models using large-scale data annotated by on-the-shelves NER systems. This paper conducts extensive experiments to verify the quality and quantity of the inducted open rules. The models trained on the extracted rules even outperform the ones trained on manually annotated rules.  Originality: 

Open rule Induction is not a novel task. There are a few mature systems to induce open rules from open knowledge bases. The mainly novelty of this paper is to use langague models to complete this tasks without using any annotated open rules. Specifically, this includes:
    - two seperated langauge model to model r_p and r_h; 
    - a specially designed beam search algorithms to take the entity pairs (x, y) into account and 
    - using large-scale silver training data (entities annotated by on-the-shelves NER systems) to further train the langueg model to generate relavant (x, y) content.
However, the main approach is adapted from previous work that uses masking-based prompts to probe languge model knowledge. In addition, although authors claim that they mainly make use the knowledge within the langauge models, their "continuing train" phrase uses "93.63 millions" training samples. I believe there are significant amount of open rules can be found within this data. It becomes unclear to me whether this approach simply relies on these magic "93.63 millions" training samples.  It will be much better if the authors could provide more detailed ablation study on the effect of the size of this continuing training sample to the overall performance. If this approach can really effectively extract the underlying knowledge in the langauge models, the size of this continuing training sample should be relatively samll. The main goal of introducing this continuing training samples is to teach the language models to output relation-related text. In many NLP tasks, a few thousands of training samples is enough to fine-tune the langaguge models. I also notice that in evaluation, only the proposed "Orion" model is trained with this large corpus. It is interesting to see the performance of the baseline models trained with the same data. For example, The Prompt baseline model in Table 1. 

Quality & Clarity:

This paper presents a fairly complete work with various experiments, including generated relation, relation disversity and case study, verifying the authors' claims. However, the authors do not provide weakness of their proposed model. It may be interesting to see if the proposed model is too diverse in some cases given that it is trained on very large collected data. The paper is well written and easy to follow. I believe it should be straightforward to reproduce the method described in this paper. To improve the reproducibility, it will be great if the authors can open-source the continue training data (although I believe this paper lacks of experiments about its importance).

Significance:

Extracting knowledge from large-scale pre-trained langauge model should have a board impact on NLP community. This paper shows that for complicated task like open rule induction, it is important to further train the langauge models. I agree that we always need fine-tune to make langauge model follow our instructions. However, this is not something supervising for people who use pre-trained langauge models. It will be much more interesting to discuss how much cost we need to pay in order to extract the unlimited knowlege from pre-trained language models. If this paper can better address this issue, its impact will be significantly improved.
 n/a

This paper proposes a method to induce “open-domain” rules from pre-trained language models like BART, useful for tasks like relation extraction. Rule induction through knowledge bases is constrained by a “closed-domain”, while recent work, as discussed in the paper shows the possibility of using language models as an open-domain knowledge base. The proposed method considers rules of the form (x, r_p, y) => (x, r_h, y), where r_p and r_h are represented in natural language. The task is to predict r_h  given r_p, by aggregating over all the instantiations of (x, r_p, y). The paper presents an interesting method of utilizing the pre-trained LMs and a supported-beam-search method of aggregating the information from the top-k supporting instantiations. Experimental results look promising. However, I doubt whether the task and the kind of rules considered in this paper are general enough for wider applications.  Pros:
- Leveraging pre-trained LMs for Open-domain rule induction.
- A smart way of aggregating information from all possible instantiations using supported beam search.
- Utilizing weak supervision to train relation and instance generation modules. This helps the proposed method outperform methods like Comet (previous work) which depend on labeled/annotated data.
- Improvements over LM baseline, and Comet. 
- The inducted rules prove beneficial for downstream tasks like relation extraction. 

Limitations:
- It seems that the proposed method assumes that premises r_p are already given (E.g. lin 255, 256). It is not clear how the body of the rule (x, r_p, y) is extracted.  
- When compared to KB-based rule induction the method is somewhat worse than the best baseline, however, one benefit of the proposed method is that it outputs more rules with decent quality.
- Even though “Error identification in LMs” is posed as a utility of the proposed approach, it also seems like a limitation of the proposed method being less robust and producing over-generalized rules.
 Yes.

Meta-review: Automatically extracting logic rules from unstructured data can be practically useful. The proposed rule extraction method based on pre-trained language models is novel and flexible. The empirical results are good on different tasks. The rebuttals clarified lots of issues raised in the initial reviews. An immediate limitation of this work is that the bias of pre-trained LMs could result in unexpected rules.

Decision: accept