Open Rule Induction
Authors: Wanyun Cui
Xingran Chen
Venue: NeurIPS 2021 Poster (2021)
Keywords: Logic rules
Open rules
Language model

Abstract: Rules have a number of desirable properties. It is easy to understand,  infer new knowledge, and communicate with other inference systems. 
One weakness of the previous rule induction systems is that they only find rules within a knowledge base (KB) and therefore cannot generalize to more open and complex real-world rules. Recently, the language model (LM)-based rule generation are proposed to enhance the expressive power of the rules.
In this paper, we revisit the differences between KB-based rule induction and LM-based rule generation. We argue that, while KB-based methods inducted rules by discovering data commonalitiess, the current LM-based methods are ``learning rules from rules''. This limits these methods to only produce ``canned'' rules whose patterns are constrained by the annotated rules, while discarding the rich expressive power of LMs for free text.

Therefore, in this paper, we propose the open rule induction problem, which aims to induce open rules utilizing the knowledge in LMs. Besides, we propose the Orion (\underline{o}pen \underline{r}ule \underline{i}nducti\underline{on}) system to automatically mine open rules from LMs without supervision of annotated rules. We conducted extensive experiments to verify the quality and quantity of the inducted open rules. Surprisingly, when applying the open rules in downstream tasks (i.e. relation extraction), these automatically inducted rules even outperformed the manually annotated rules.

Open Rule Induction
Wanyun Cui, Xingran Chen Shanghai University of Finance and Economics
wwi.wanyun@suff edu. en, xingran. chen. sufe&gma com
Abstract
Rules have number of desirable properties. It is easy to understand, infer new knowledge, and communicate with other inference systems. One weakness of the previous rule induction systems is that they only find rules within knowledge base (KB) and therefore cannot generalize to more open and complex real-world rules. Recently, the language model (LM)-basck) rule generation are proposed to enhance the expressive power of the rules. In this paper, we revisit the differences between KB-based rule induction and LM-based rule generation. We argue that, while KB-based methods inducted rules by discovering data commonalities, the current LM-based methods are "learning rules from rules". This limits these methods only produce "canned" rules whose patterns are constrained by the annotated rules, while discarding the rich expressive power of LMs for free text.
Therefore, in this paper, we propose the open rule induction problem, which aims to induce open rules utilizing the knowledge in LMs. Besides, we propose the Orion (open rule induction) system to automatically mine open rules from LMs without supervision of annotated rules. We conducted extensive experiments to verify the quality and quantity of the inducted open rules. Surprisingly, when applying the open rules in downstream tasks (i.e. relation extraction), these automatically inducted rules even outperformed the manually annotated rules.
Introduction
Rules induction is classical problem aiming to find rules from datasets 7 Previous work has focused on discovering rules within a system. For example, one of the core tasks of inductive logic programming (ILP) to mine shared rules in the form of Horn clauses from data. Early studies are mainly applied to relatively small relational datasets. Since the axioms of rules is limited to the existing entities and relations within the datasets, the expressiveness of such rules is limited and brittle. John McCarthy pointed out these rules lack commonsense and "are difficult to extend beyond the scope originally contemplated by their designers" " [8] In recent years, with the emergence of large-scale knowledge bases (KB) (3] and open information extraction [ systems, rules mined from them (e.g. AMIE+ [8]. Sherlock (26)) are built on a richer set entities and relations. Nevertheless, both the quantity of knowledge and the complexity of rules are still far weaker than the real-world
due to the expressive power of these knowledge bases.
Recently, with the rapid development of pre- pre-trained language models (LM) 6 researchers have found that pre-trained LMs can be used as high-quality open knowledge bases 21 28 and commonsense knowledge bases [27 25 Based on the expression of natural language for complex relationships, the LM as knowledge base can be used to generalize rules with more expressive power. Based on LMs, Comet [4] proposed to generate new rules (if-then clauses) for arbitrary texts. The generative model trained on annotated if-then rules in the form of natural language.
"Corresponding author
"Code and datasets are available at http://gtthahccm/cceenaaa//iiinn
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Instantiation
Inducted rules P(hypothesis/premiiss)
Premise atom
P(instancelpremisse)
support
Steve Jobs is founder of Apple
CEO of y
x is founder of
Bill Gates founder of Microsoft
member of was the secretary
Edmund Berkeley founder of ACM
Inducted rules
x worked at the brokerage Morgan Stanley for about years until 2005, when and some Morgan Stanley colleagues quit and later founded the hedge fund
is the founder and CEO of serves the board of y is currently venture capitalist at
Figure 1:
Inducting rules from LMs. We show rumning examples of Orion.
Therefore, Comet relaxes the form of knowledge in rules from a structured KB, open natural language. We know that Comet focuses on some specific domains (c.g. social interactions), but when we want to use Comet for open rule generation, training based on annotated rules will constrains the generated rules. Since there are only 23 different types of relations in Comet's s training corpus (i.e. ATOMIC2020 [14), the rules inducted by Comet are limited these types (e.g. As result, PersonX feels). Besides, the model only learns patterns from the manually annotated rules, which restricts its ability to generate novel rules.
Therefore, we revisit the differences between KB-based rule induction and LM-based rule generation methods. We argue that, leaving aside the difference in knowledge forms, their main difference is that the KB-based method inducts rules by observing common pattern of group of entities from the data, while the LM-based method only learns from the annotated rules. Namely, the former inducts rules from data, while the latter learns rules from rules. In this case, although the LM contains far more open knowledge than the KB, the current LM-based method still only generates "canned" rules. We believe that the current KB-based rule induction, e., summarize the commonality of data.
rule generation method departs from the principle of
In this paper, we propose to recapture the commonality of the data in the LM-based method, i.e., using the LM itself knowledge source and discover commonalities within it. We first proposed the open rule induction problem based on the commonality of knowledge in LMs. Then, we proposed an unsupervised open rule induction system, Orion. Instead of training the LM to generate rules that conform to human annotations, we propose to let the LM "speak" the commonalities of the data by itself. In this way, rules are not constrained by annotations.
To capture data patterns and mine rules directly from LMs, we use prompts to probe the knowledge of LMs 21). Prompts' predictions reflect the knowledge in LMs. For example, in Fig. I based on prompt: x is founder of y. we probe instances such as Steve Jobs-Apple, Bill Gates-Microsen From these instances, we further use LM to induct other expressions they support, such ofy, y. etc. Notice that unlike Comet which learns from annotated rules, we directly mine the patterns from knowledge in LMs. In addition, taking full advantage of the expressive power of LMs, inducted rules for the complex example at the bottom of Fig. D
etc.
is CEO
The significance of our proposed open rule induction in this paper is twofold. First, from the perspective of rule induction, we gain more expressive rules to approach the real-world. And by adding these rules to downstream tasks, the effect can be significantly improved. We will empirically verify this in Sec 6.1 Second, from the LM perspective, rules can identify potential errors in language models. For example, in Fig. [ if we wrongly inducted the hypothesis suggests that the LM has some kind of cognitive bias. We will elaborate this in Sec 6.2
the secretary of y. this
2 Open Rule Induction Problem
2.1 Preliminary: Rules in KB-based Methods
We refer to the definition of rules based on the Horn clause in KB-based rule induction to help define problem. In Horn clause, atom is a fact that can have variables at the subject and/or object

---

position. For example, (z, fo,foom founderOf. f. y) is an atom whose subject and object are variables. A Horn clause contains head and a body, where the head is an atom, and the body is collection of atoms. A rule is then an implication from body to head: body1 A.. body, = head. An instantiation of rule is a copy of that rule, where all variables are replaced by specific instances in the KB.
2.2 Problem Definition
We define the atom of an open rule as a natural language sentence describing some relationship between subject and object. For simplicity, we assume both subject and object are variables. For example, (z, is founder of, y) is an atom which describes the relation between and y. In defining the open rule, for simplicity, we only consider the bodies with one atom. As we will describe in See the open rule can be easily extended to more variables by slightly modifying prompts. We define an open rule as derivation from premise atom hypothesis atom.
Definition (Open rulc), An open rule is implication from the premise atom (z, Tp. y) to hypothesis atom (z, y):
(p, y) (x,, Fhe y)
(1)
where Tp and are natural language descriptions. The rule implies that instance pairs with Ty relation also (very often) has Th relation.
For example, the open rule (z, is founder of, y) + (ar, CEO of, y) means that (very often) the founder of an organization is the CEO of it. And (Steve Jobs, is founder of, Apple) (Steve Jobs, CEO of. Apple) in an instantiation of the rule. As we will show in See generalizing the open rules to more than two variables is easy. And we will discuss the potential extension of the open rules with new variables in Appendix ??.
To model to the uncertainty of the open rule, in our problem definition, we use probability to represent the open rule: PLar (rw ipp) denotes the probability of inferring hypothesis (I,, (. v) from premise (a.Tp. Tp. 3). PLM () means the probability is derived from the language model LM. For simplicity, we will use P() instead of PLA(() () in the rest of this paper.
For given premise atom (I,, Tp: y), we want induct i most relevant hypothesis atoms from the LM. Specifically, we define the problem as follows:
Problem Definition 1 (Open rule induction). For given premise atom (p.Tp: y) and k, find top Th W.E. P(rklrp)-
We compute P(ralrp) by the marginal distribution of the instantiation (ins) as below:
P(rulrp) E (or
where P(ins=(zo, (rp) denotes the conditional probability distribution of zo Yo corresponding Tp- And P(rx|ins=(4 (0)),,,) denotes the conditional probability of (xo. Th, yo) given (æo. Tp Yo) and the specific instance (xx, y). For example, PSine=(Steve Appl founder of) denotes the probability of instance Steve Jobs and y=Appe for the relation is founder of the LM. And Pira=is ceo of inse=Seve Jobs.Apple) rp ris founder of) denotes the conditional probability of (Steve Jobs, is ceo of, Apple), given that (Steve Jobs, is founder of, Apple).
Note that, when ins= Jops.Apple) is known, whether the instantiation has relation is ceo of is also known. That is, given the ins, Th is independent from The So we have:
P(rn(rp) = Σ P(ralins) P(ins(rp)
(3)
SELS
Applicability Instantiation
2.3 Rationale of the Open Rule Induction Problem
We the concept of support in KB- rule induction to explain Eq. (3) and Problem I Support is the core metric in KB-based rule induction, which quantifies the amount of instances following the rule in the dataset. We will analogize the terms in Eq. 3 to the factors of support below.
Instantiation First, we consider P(ins)rp)) as the instantiation by the language model. The probabil- ity can be considered as the typicality of this instance for Tp by the LM.
3

---

Applicability Second, we consider P(r (ins) in Eq. (3) as the applicability of ra to ins in the LM. For example. for well-trained LM, we can get P(r ceo of Steve Jobs, Apple) with a high probability.
In summary, Eq. denotes the expected number of instantiations that can be applied to the hypothesis atom w.r.t. their applicability. This is consistent with the idea of support in the KB-based rule induction. We will elaborate how to compute ((ins)rp) and P(rn lins) via the LM in Sec 3
3 Masked Language Models for Relational Descriptions
In this section, we illustrate how to calculate the two terms in Eq. (3). i.e., P(r lins) and P(instrp). For LMs, both probabilities can considered as special cases of the masked language model (MLM), ie. P(mask| masked sentence). From the MLM perspective. P(ra lins
(I, y)) is equivalent to predicting the masked text between æ and y. while P(ins = (æ, y) rp)) is equivalent to predicting the masked y for given Tp. Since MLM is typical goal of the language model pre-training, we can directly use existing pre-trained LMs to predict the masks.
Probability probing via prompts To compute the two probabilities, we construct following prompts:
For P(ri/lixx = (2r, v)): x <mask> y.
For P(ins (xr, pr): <mask>, Tp <mask>y-
For example, we use the prompt Steve Jobs <mask> Apple. to compute P(ralins (Steve Jobs,Apple)), and the prompt <mask>> is founder of <mask>y. to compute P(ins = (z, y)lr, is founder of). In our problem, we use Bart [6 as our LM, which uses seq2seq to decode one or more tokens for each mask.
mask may correspond to multiple tokens. Therefore
Although we only consider rules with two variables in this paper, our method can be easily generalized to arbitrary number of instances by modifying the prompts (e.g. from x <mask> y. to x <mask> y <mask> z).
Relational description generation via weak supervision We noticed that ML. not directly applicable to computing P(r, (ins) and P(ins(rp). For P(r, jistancce) we require the LM to generate text describing exactly the relationships between x and y. Similarly, for P(instances|r,) the generated text fragments are required to be exactly instances/entities. However, the original MLM task for language model pre-training does not have these restrictions. It may predict arbi- trary descriptions which leads to a lot of noise. For example. Bart predicts "Tokyo Metropolitan Government.JaaanJJeee Jokyo, Japan." " for prompt Tokyo <mask> Japan.
To ensure that the LM correctly predicts P(rn lins) and P(inslr,). we continue training the LM on relational description corpora. We obtain a large-scale relational description corpus using the weak supervision technique [1)., and use it to construct two separate training corpora for P(ri. jins) and P(ins)r,). respectively. For P(rx) (ins), we only mask out text except the named entities, and train the LM to predict the mask. For ((ins)rp), we only mask out named entities.
Specifically, starting from the Wikipedia and Bookcorpus 32 we searched for sentences containing one or two named entities. We consider that these sentences are (likely) describing the relationship between entities. We continue to train two Bart models each of these two corpora, which are used to predict P P(ra|ins (ins) and P(ins)rp)) respectively. We used the Spacy NER library to automatically extract the named entities. We filtered the entities related dates and numbers. The resulting dataset consists of 93.63 millions samples. The continuing train plays the role of denoising the generative
process in Eq. 33) which is also used in 24
4 Supported Beam Search for Rule Decoding
Another challenge of solving Problem T is to efficiently generate Th of a crowd of instantiations. Although we can compute P(nair))) according to Sec computing Eq. (3) is still in- tractable: (a) the search space grows exponentially according to the Tength of the generated rule; (b) we need to find the Th that has the top probability among all instantiations, rather than for a single instantiation in standard decoder.

---

Beam search for instantiation First, for the exponential number of all possible instances, we use the model for p(inslr,) trained in Sec 3 to generate the top instantiations, denoted as INS:
INS ( (P(ins(rp))
We use beam search, common and efficient way for decoding TNS. Beam search search heuristic to maintain beam of size l containing set of probable outputs. It generates from beginning to end, conditioning on the instances and
text.
We use these k instances to approximate Eq. (3)
P(n)rp)) =
Σ InSEINS
(5)
Supported beam search To solve Problem [ we need to consider the support from different instantiations. straightforward method is first generate top k rhh for ins E INS separately with beam search, and then fuse these The However, this risks making locally optimal decisions which are actually globally sub-optimal for all instantiations. These top beams for each individual instantiation may not be shared by different instantiations. Therefore, in order to decode the rules, we need to consider the support of all instantiations in our decoding heuristic.
We propose supported beam search (STS) which decodes the open rules by considering all instantia- tions. The probability of generating the next word considering all instantiations is:
P(beam' beam + w/rp) P(w P(w]beam, ry) (ateeeammm)))aaaannaaaaaaaa P(bemm(rr)
Σ
Prwlrp, beam, P beam, Tp) Pp)eamm))))))))
(Eq.
inse INS
Σ insEINS
P(w beam, ins) P(bcamiiiss
(Bayesian rule)
(6)
Σ
P(w beam, ins) (bsa))))(eemmmmsss))(((
(Independence)
InSEINS
where beam' beam + w means appending the word w at the end of beam, which forms a longer beam. (ewaee ins) can be computed via fine-tuned Bart of P(r lins) in Sec 3 (eeammiis)) is computed and updated by:
P(beam' = beam + wlins) = P(w (ins, beam) ) (beam//nns
(7)
Here P(beam' = beam + w/rp) can be considered as the global score of w, as it aggregates different instantiations. And P(beam' beam w/ins) can be considered as the local score of w, as it only considers the instantiation of ins.
Algorithm In implementation, decode different instantiations in INS simultaneously via batches. We assemble different instances ins E INS into one batch. In each step of the decoding, aggregate the local scores of compute its global score. We uniformly select the top k words w.r.t. their global scores for all instantiations. That is, we maintain identical beams for different instances in the batch, instead of maintaining individual beams for each instance individually.
Algorithm 1: Supported beam search
Function SupportedBeamSearee
(Tp. INS, k)
batch. beams + LL)
P(beam NU LL Li)ss)
for ins INS
P(beam NU LU +
for rimestep 1.. 7....Tah
Update P(beam' (ins) for len(beam') = and ins E INS by Eq. 7
Update P(beam' (rp) for len(beama) by Eq. (6) batch_beama topk (((eemm' (r,)) // Greedily select top à beams
w.r.t. their global scores.
return batch. beams
5

---

We show the pseudo-cost of supported beam search in Algo. Unlike the traditional beam search which maintains separate beams for different samples in a batch, we maintain a set of shared beams for all instances, denoted as match_beams At each timestamp, we update the local score P(beam rp ins) and global score P(beambry) in turn. Then line 8 we greedily selects the top beams w.r.t the P(beam||p)) i.e., the approximated goal of Problem
5
Experiments
All the experiments cloud of servers. Each server has 4 Nvdia Tesla V100 GPUs.
5.1 Datasets
Manual constructed dataset To evaluate the effectiveness of open rule induction, constructed our own benchmark dataset with 155 premise atoms. We call "OpenRule for convenience
in this section. First, construct premises describing different relationships between x and y. collect 121 premise relations from 6 relationship extraction datasets (Google-RE RE [21] TREx (2I], (TI]) and one knowledge graph (Yago2). We converted all relations of these datasets into premise atoms, and obtained total of 121 premise atoms after removing duplicates. We also selected 34 relations from Yago2. We select these 34 relations because they occur frequently in the bodies of inducted rules by AMIE+ and RuLEES So we think these relations have a higher inductive value. We asked the annotators annotate each premise with 5 hypothesis. We filtered out duplicates and low quality hypothesis and ended up with an average of 3.3 hypothesis atoms for each premise.
NYT10 23), WIKI8O (2) FewRel (9] SemEval
Converting relations to premise atoms We convert a relation into a premise atom via the template (z, is relation of. y) or (z, relation, y). For example, the relation <founderOf> and <helieveln> in Yago2 will be transformed into (z, is founder of. y) and (z, believe in, y), respectively.
Relation extraction datasets We also conducted experiments over relation extraction datasets to evaluate whether Orion extracts the annotated relations from given texts. We use Google-RE, TREx, NYTI0, WIKI80, FewRel, SemEval the relation extraction datasets.
5.2 Baselines
LM-based baselines We use the following LM-based rule generation baselines:
1. Comet:
The input of Comet is a premise atom, and the output is a collection of hypothesis atoms with different relations. To compare with the top 10 rules inducted by Orion, select 10 relations of Comet. See Appendix for the list of relations. Note that the hypothesis atoms generated by Comet are not always describing the relationship between and y. but may be describing about x only. For each selected relation of Comet, we generate 10 hypothesis atoms. If there are hypothesis atoms of both x and y, we choose the one with the highest probability. Otherwise, we choose the one with the highest probability among the descriptions of x.
2. Prompt:
Inspired by the work on prompt-based knowledge extraction from LMs 21. we proposed prompt-based method as baseline. We use the prompt: if rp then <mask>. and take the LM's prediction for <mask> as rh- Specifically, we use Bart as the LM and select the top 10 predictions as Th.
3. Prompt (fine-tuned): We came up with stronger baseline by fine-tuning the above prompt model. We collect a set of such sentences "if sentl then sent2" from Wikipedia and Bookcorpus and mask sent2. Then fine-tune the prompt model over these sentences.
KB-based baselines We use AMIE+ [ and RuLES 12 as the KB-based rule induction baselines.
Ablations We also considered the following ablation models.
Without continuing training P(ra|ins) or P(ins(r,) In Sec 3 we propose to generate relational descriptions by continuing training Bart. To verify its effect, we replace the models for P(r jims) r()ns)rp) with the original Bart an ablation.

---

Without STS To verify the effectiveness STS, we replace STS with the original beam search. When decoding, we first use beam search for each ins E INS separately to generate the top k Th w.r.t. P(ra/p lrp ins). Then we aggregate these hypothesis atoms according to Eq. (5)
and select the top l of them.
5.3 Main Results
Table I: Results over the OpenRule155.
Our Dataset
BLEU-1
BLEU-2
BLEU-4
ROUGE-L
METEOR
self-BLEU-2
Prompt
17.77
3.65
0.48
18.65
12.94
86.63
Prompt (fine-tuned)
20.95
7.58
0.86
22.37
17.24
82.13
Comet
21.58
8.15
1.04
23,45
5.44
90.78
Orion STS
44.92
20.24
1.21
49.72
39.68
89.84
Orion train P(nnsTr)
15.85
3.11
0.00
32,91
13.19
90.29
Orion train ((rx(ins)
19.17
3.05
0.07
34.99
10.30
83.54
Orion
45.41
21.29
1.30
50.37
40.41
90.94
We report the performance of the models on the OpenRule155 Table We use BLEU-1/24 (0). ROUGE-L [17]. and METEOR [2] to evaluate whether the model-inducted is similar to the manually annotated hypothesis. We also report the self-BIEUU2 31 of the model, which used to measure the diversity (the smaller the more diverse).
We compare Orion with the LM-based basclines. From the perspective of quality, Orion significantly outperforms the baselines. From the perspective of generated diversity, the diversity of Orion is also competitive with Comet.
Ablations We also compare with the ablation models in Table First, we find that the effectiveness of the models reduces after removing any module. This verifies the effectiveness of the proposed modules in this paper. In particular, we find that continuing train P(r (rn))ns) has the most significant effect on the model. As mentioned in Sec 3 this is because that without continuing training, Bart easily produces noisy text that does not describe the relationship between æ and y.
5.4 Comparison with the KB-based Rule Induction
In See we claimed that open rules are more flexible than rules inducted from KBs. this subsection, we verify this by comparing the results of Orion and KB-based rule induction systems.
Rules from KB-based induction Specifically, we compared the rules inducted by Orion and by the KB-based methods AMIE+ [8] and RuLES [12). We use AMIE+ and RuLES to induct rules Yago2 [3]. For fair comparison, we also only retain the rules generalized by AMIE+ and RuLES which contain exactly two variables. AMIE+ and RuLES mined 115 and 47 rules that meet the
requirement, respectively.
Comparing open rule induction with KB-based Horn rules In order to verify the inductive ability of Orion, for each Horn rule body head inducted by the KB-based methods, we converted the relation of body into a premise atom according to the conversion method in Sec 5.1 AMIE+ and RuLES have 24, 20 different bodies, respectively. For each converted premise atom, we use Orion to induct k 5, 10, 20 corresponding open rules. We manually evaluate whether these inducted rules are correct. During the human evaluation, we require a correct rule to be plausible, unambiguous, and fluent. For example,
label "[X] is provincial capital of [Y] [X] is the largest city in [Y]" as correct, because this inference is plausible to be valid. In contrast, we will label "[X] lives in [Y]
+ [X] grew up in the east end of [Y]" as incorrect, because the probability that [X] happens to grow up on the east end is too low.
The results are shown in Table It can be found that the accuracy of Orion is competitive with +MIE+ and RuLES. As k increases, Orion keeps finding new rules without decrease accuracy. Note that besides the bodies covered by AMIE+ and RuLES, Orion also generalizes rules from novel premises. This indicates that Orion finds substantially more rules than the KB-based methods with competitive quality.
7

---

Table 2: Comparisons with KB KB-based methods.
Accuracy
#Rules
AMIE+
55.7
115
RuLES
47
Orion (k=5)
50.0
120
Orion (k=10)
49.2
240
Orion (k=20)
51.3
480
5.5 Effect for Complex Premises
Orion is able to generate rules for complex premises, as the pre-training corpora of LMs contain extensive complex texts, We show two examples from TREx and FewRel in Table with (k = 5). be seen that Orion generates valid rules for complex and long texts. On the other hand, the rules generated by Comet are often only about x, not about the relationship between Z and And these rules are often about human characteristics, even if is a country in case 1. This is due the limitation of Comet's 's training data that leads to the bias of the generated rules.
Table 3: Effect of complex rule induction. Original sentence of Case 1: [X]'s emergence from international isolation has been marked through improved and expanded relations with other nations such as [Y], France, Japan, Sweden, and India. Case 2: His guitar work on the title track is credited as what first drew [X] him, who two years later invited allman to join him as part of [Y].
Orion
Comet
[X] has a long history of diplomatic relations with [Y].
<xReact>; happy.
[X] is the largest exporter of oil to [Y].
<xReason>: [X] is no longer isolated.
C1
[X]'s economy is heavily dependent on [Y].
<xWant>: to make new friends.
[X]'s foreign policy is based on its close relationship with [Y].
<isAfter>: [X] gets new job.
[X] has been the largest exporter of uranium to [Y].
<isBefore>: [X] has better relationship
with [Y].
[ guitarist and singer of [Y].
<xReact>: happy.
[X] and his band [Y].
<xReason>: his guitar work.
C2 [X] has been a fan of [Y]
<xWant>: to play song.
XI was fan of [Y].
<isAfter>: [X] plays guitar on the song.
[X] was fan the band [Y].
<isBefore>: his guitar work.
6 Application
6.1 Introducing Open Rules in Relation Extraction
Setup We apply the inducted open rules to relation extraction to verify its value. To do this, We introduce the inducted open rules as extra knowledge, and evaluate whether the inducted rules improve the effect of relation extraction models. We used ExpBERT [9 as the backbone. EXpBERT introduces several textual descriptions of all candidate relations as external knowledge. For example, for the relation spouse, ExpBERT introduces external knowledge in the form of x's husband is into the BERT model. The original descriptions of each relation in ExpBERT comes from manual annotation. To verify the effect of inducted open rules, replace these manual annotated rules with the open rules inducted by Orion.
Specifically, We follow the settings in ExpBERT and use the Spouse and Disease 10 for evaluation. For each relation, we construct the corresponding premise atom according Sec 5.1
We following
the settings of ExpBERT and use k 29, 41 hypothesis atoms inducted by Orion as for Disease and Spouse, respectively. For example, for relation spouse, Orion generates hypothesis atoms like (x, is married to, y) as the external descriptions. In addition, we modified ExpBERT to allow the training process to fine-tune the parameters that were frozen in the original ExpBERT, as we found that this will improve the model's effectiveness.

---

Results From Table 4 our automatic inducted rules even outperforms the manually annotated rules. We think that this is because Orion's 's rules are more unbiased and diverse than the manual annotations. This strongly verified the applicability of the open rules.
Table scores on relation extraction tasks. The annolated rules are from the original ExpBERT. Averaged over 5 runs.
Spouse
Discase
BERT ExpBERT annotated rules
46.43 0.84
40.20 1: 2,43
76.04 0.47
56.92 + 0.82
EXpBERT + inducted open rules
76.05 = 0.52
57.68 1.34
Coverage evaluation We also directly evaluate whether the open rules inducted by Orion cover the target relation. For the bottom example in Fig. since we can extract the relation founder from x worked founded the hedge fund y., we expect the model to also induct is founder of y given the premise.
For this purpose, we transform the samples in the relation extraction dataset into premise atoms by replacing the entity pairs with x and y, respectively. We induct rules from these premise atoms and evaluate whether the hypothesis atoms cover the corresponding relations. For each target relation, convert to hypothesis atom via the method in See and use the converted hypothesis atom as the ground truth. We compute the correlation between the inducted тa and the ground truth.
We report the results FewRel, NYTI0, WIKI80, TREx, Google-RE, and SemEval in Table 5 Since the number of samples is different for different datasets and relations, in order to get uniform evaluation, we select training samples uniformly as premise atoms for each relation each dataset. Note that there are k 10 hypothesis atoms for each premise. To evaluate the coverage, we report the one with the highest score. Orion outperforms the baseline by large margin.
Table 5: Results on relation extraction datasets.
FewRel
NYT10
WIKI80
TREx
Google- RR
SemEval
BLEU-2/4
Comet Prompt
0.60/0.00
0.73/0.00
0.36/0.00
0.82/0.11
0.60/0.00
1.80/0.00
0.95/0.00
0.64/0.00
0.87/0.00
1.45/0.11
0.79/0.00
0.90/0.00
Orion
9.08/0.08
8.34/0.51
7.24/0.30
9.03/1.14
9.22/0.00
5.69/0.00
ROUGE-L
Comet Prompt
3.02
5.70
2.44
4.77
4.19
7.82
15.35
15.10
16.38
15.80
16.44
12.54
Orion
38.72
36.72
36.75
39.51
36.93
37.90
METEOR
Comet Prompt 9.94
I.60
3.03
1.19
2.14
2.08
4.60
9.84
10.87
11.50
8.41
8.47
Orion 25.28
25.78
23.8
26.29
24.67
26.71
6.2 Application: Error Identification in Language Models
We inducted rules identify potential errors in the pre-trained LMM Some rules that defy human commonsense are incorrectly inducted. This is actually due to the bias of the language model. Specifically, found the bias of the language model caused by its pre-training corpus. We list some examples in Table 6
7 Related Work
As a classical problem, rule induction has gained extensive studies. Related researches include mining association rules [I], logical rules (29, etc. Traditional methods of rule induction tend to work only on small knowledge bases. In recent years, with the emergence of large-scale open knowledge graphs [3] and open information extraction systems 4 studies have focused on mining rules from such large-scale knowledge bases [26] 8). However, the representation capability of even

---

Table 6: Examples of identified errors
Inducted rule: [X] is the politician of [Y]. [X] was the founder and president of [Y]-
Identified error: The training corpus deseription of politician has disproportionate number
of founder president entities. This led to bias LM's perception: it assumes that
politician is always founder and president.
Inducted rule: [X] is an instance of [Y], + [X] lower house of [Y].
Identified error: The frequeney of political entities the training corpus is high. The LM tends generate political entity descriptions.
the largest knowledge bases still do not approximate the real-world, especially from the perspective of commonsense.
On the other hand, reserchers found that pre-trained LMs can be used as open knowledge bases 21 28 and commonsense knowledge bases 27 25]. Due to the unstructured form knowledge representation, the language model is much more capable for representing open knowledge. Using the language model as a basis, we want to mine open rules. Comet [14] is a relevant attempt. It learns from ATOMIC2020 [14] to generate rules in the form of if-then clauses. However, influenced by the manual annotation of ATOMIC2020, Comet's rules are "canned" and repetitive [30. In this paper, want generate rules unsupervised directly from LM's knowledge, thus achieving open rule
induction.
8 Conclusion
For rule induction, in order to break the limitation of representation in KBs, we propose to use unstructured text as atoms rules. Based on pre-trained language models, we propose the open rule induction problem. To solve this problem, we propose the Orion system, which extracts rules from the language model completely unsupervised. We also propose to optimize Orion by continuing training the language model, well as the decoding heuristic.
We conducted variety of experiments. We verified that the effectiveness of Orion exceeds that of LM-based and KB based baselines. In addition, an application to the relation extraction task found that the model with Orion's 's inductive rules even outperformed that of manually annotated rules. We also used Orion's 's rules to identify potential errors in language models.
Acknowledgments and Disclosure of Funding
We thank Wenting Ba for her valuable plotting assistance. This paper was supported by National Natu- ral Science Foundation of China (No. 619061166 16), by Shanghai Sailing Program (No. 19YF1414700).
References
[1] Rakesh Agrawal, Tomasz Imieliński. and Arun Swami. Mining association rules between sets of items large databases. In Proceedings of the 1993 ACM SIGMOD international conference on Management of data, pages 207-216, 1993.
[2] Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65-72, 2005.
[3] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247-1250. 2008
[4] Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam Hruschka, and Tom Mitchell. Toward an architecture for never-ending language learning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 24, 2010.
10

---

[5] William W Cohen and Yoram Singer. A simple. fast, and effective rule learner. AAAI/AAA,, 393335-342):3, 1999.
[6] Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4171-4186, 2019.
[7] Johannes Fürnkranz. Separate-and-conquer rule learning. Artificial Intelligence Review, 13(1):3- 54, 1999.
[8] Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M Suchanek. Fast rule mining in ontological knowledge bases with amie+. The VLDB Journal, 24 2015.
[9] Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Fewrel: large-scale supervised few-shot relation classification dataset with state-of-the-ar evaluation. In EMNLP, 2018.
[10] Braden Hancock, Martin Bringmann, Paroma Varma, Percy Liang, Stephanie Wang, and Christopher Ré. Training classifiers with natural language explanations. In Proceedings of the conference. Association for Computational Linguisties. Meeting, volume 2018, page 1884, 2018.
[11] Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. Semeval-2010 task 8: Muy classification of semantic relations between pairs of nominals. In Proceedings of the Sth International Workshop Semantic Evaluation, pages 33-38, 2010.
[12] Vinh Thinh Ho, Daria Stepanova, Mohamed H Gad-Elrab, Evgeny Kharlamov, and Gerhard Weikum. Rule learning from knowledge graphs guided by embedding models. International Semantic Web Conference, pages 72 90, 2018.
[13] Johannes Hoffart, Fabian M Suchanek, Klaus Berberich, and Gerhard Weikum. Yago2: A spatially and temporally enhanced knowledge base from wikipedia. Artificial Intelligence, 61, 2013.
[14] Jena D) Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. Comet-atomic 2020: On symbolic and neural commonsense knowl- edge graphs. arXiv preprint arXiv:2010.05955 2020.
[15] Vid Kocijan, Oana-Maria Camburu, Ana-Maria Cretu, Yordan Yordanov, Phil Blunsom, and Thomas Lukasiewicz. Wikicrem: A large unsupervised corpus for coreference resolution. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4294-4303 2019.
[16] Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising eequence-to-sequeee pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880 2020.
[17] Chin-Yew Lin and Eduard Hovy. Manual and automatic evaluation of summaries. In Proceedings of the ACL-02 Workshop on Automatic Summarization, pages 45-51, 2002.
[18] J McCarthy. Some expert systems need common sense. Annals of the New York Academy of Sciences, 426: 12 19-11777 137, 1984.
[19] Shikhar Murty, Pang Wei Koh, and Percy Liang. Expbert: Representation engineering with natural language explanations. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2106-21 2113, 2020.
[20] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311-318, 2002.

---

[21] Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin. Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNIP-IJCNIP), pages 2463-2473, 2019.
[22]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and lly Sutskever. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[23] Sebastian Riedel, Limin Yao, and Andrew McCallum. Modeling relations and their mentions without labeled text. In Joint European Conference on Machine Learning and Knowledge Discovery Databases, pages 148-163. Springer, 2010.
[24] Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 5418 5418-5426, 2020.
[25] Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. Atomic: An atlas of machine commonsense for if-then reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3027-3035, 2019.
[26] Stefan Schoenmackers, Jesse Davis, Oren Etzioni, and Daniel Weld. Learning first-order horn clauses from web text. In Proceedings of the 2010 Conference on Empirical Methods on Natural Language Processing, pages 1088-108 1098,, 2010.
[27] Trieu H Trinh and Quoc V Le. A simple method for commonsense reasoning. arXiv preprint 1w0ivv1806..2888 2018.
[28] Chenguang Wang, Xiao Liu, and Dawn Song. Language models are open knowledge graphs. arXiv preprint arXiv:2010. / 1967, 2020.
[29] Qiang Zeng, Jignesh M Patel, and David Page. Quickfoil: Scalable inductive logic programming. Proceedings the VLDB Endowment, 8(3):197-200 208, 2014.
[30] Hongming Zhang, Daniel Khashabi, Yangqiu Song, and Dan Roth. Transomes: From linguistic graphs to commonsense knowledge. arXiv preprint 2xXiv:0055000200 2020.
[31] Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1097-1100, 2018.
[32] Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE international conference on computer vision, pages 19-27, 2015.
12


Reviews:
[{"review_id": "uJk2c2b1U1f", "reviewer": "Reviewer_VhJS", "summary": "This paper proposes a method to induce “open-domain” rules from pre-trained language models like BART, useful for tasks like relation extraction. Rule induction through knowledge bases is constrained by a “closed-domain”, while recent work, as discussed in the paper shows the possibility of using language models as an open-domain knowledge base. The proposed method considers rules of the form (x, r_p, y) => (x, r_h, y), where r_p and r_h are represented in natural language. The task is to predict r_h  given r_p, by aggregating over all the instantiations of (x, r_p, y). The paper presents an interesting method of utilizing the pre-trained LMs and a supported-beam-search method of aggregating the information from the top-k supporting instantiations. Experimental results look promising. However, I doubt whether the task and the kind of rules considered in this paper are general enough for wider applications.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- Leveraging pre-trained LMs for Open-domain rule induction.\n- A smart way of aggregating information from all possible instantiations using supported beam search.\n- Utilizing weak supervision to train relation and instance generation modules. This helps the proposed method outperform methods like Comet (previous work) which depend on labeled/annotated data.\n- Improvements over LM baseline, and Comet. \n- The inducted rules prove beneficial for downstream tasks like relation extraction. \n\nLimitations:\n- It seems that the proposed method assumes that premises r_p are already given (E.g. lin 255, 256). It is not clear how the body of the rule (x, r_p, y) is extracted.  \n- When compared to KB-based rule induction the method is somewhat worse than the best baseline, however, one benefit of the proposed method is that it outputs more rules with decent quality.\n- Even though “Error identification in LMs” is posed as a utility of the proposed approach, it also seems like a limitation of the proposed method being less robust and producing over-generalized rules.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "TYGNnWBncqa", "reviewer": "Reviewer_7tur", "summary": "This paper focuses on inducing open rules (i.e., (x, r_p, y) ==> (x, r_h, y)) from the large-scale pre-trained language models (Specifically, BART) and proposes \"Orion\" model trained without any hand-crafted open rules. Although previous works attempt to generate these rules using language models, these approaches usually train (or fine-tune) the langauge models using annotated open rule data. The resulting langauge models can only generate \"canned\" rules. This paper adapts the idea of probing languge model knowledge using prompts. Two seperated langaugue models (i.e., BART) are designed to map from r_p to (x, y) and map from (x, y) to r_h, respectively. To guarantee the langauge models produce meaningful output, this paper further trains these langauge models using large-scale data annotated by on-the-shelves NER systems. This paper conducts extensive experiments to verify the quality and quantity of the inducted open rules. The models trained on the extracted rules even outperform the ones trained on manually annotated rules.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: \n\nOpen rule Induction is not a novel task. There are a few mature systems to induce open rules from open knowledge bases. The mainly novelty of this paper is to use langague models to complete this tasks without using any annotated open rules. Specifically, this includes:\n    - two seperated langauge model to model r_p and r_h; \n    - a specially designed beam search algorithms to take the entity pairs (x, y) into account and \n    - using large-scale silver training data (entities annotated by on-the-shelves NER systems) to further train the langueg model to generate relavant (x, y) content.\nHowever, the main approach is adapted from previous work that uses masking-based prompts to probe languge model knowledge. In addition, although authors claim that they mainly make use the knowledge within the langauge models, their \"continuing train\" phrase uses \"93.63 millions\" training samples. I believe there are significant amount of open rules can be found within this data. It becomes unclear to me whether this approach simply relies on these magic \"93.63 millions\" training samples.  It will be much better if the authors could provide more detailed ablation study on the effect of the size of this continuing training sample to the overall performance. If this approach can really effectively extract the underlying knowledge in the langauge models, the size of this continuing training sample should be relatively samll. The main goal of introducing this continuing training samples is to teach the language models to output relation-related text. In many NLP tasks, a few thousands of training samples is enough to fine-tune the langaguge models. I also notice that in evaluation, only the proposed \"Orion\" model is trained with this large corpus. It is interesting to see the performance of the baseline models trained with the same data. For example, The Prompt baseline model in Table 1. \n\nQuality & Clarity:\n\nThis paper presents a fairly complete work with various experiments, including generated relation, relation disversity and case study, verifying the authors' claims. However, the authors do not provide weakness of their proposed model. It may be interesting to see if the proposed model is too diverse in some cases given that it is trained on very large collected data. The paper is well written and easy to follow. I believe it should be straightforward to reproduce the method described in this paper. To improve the reproducibility, it will be great if the authors can open-source the continue training data (although I believe this paper lacks of experiments about its importance).\n\nSignificance:\n\nExtracting knowledge from large-scale pre-trained langauge model should have a board impact on NLP community. This paper shows that for complicated task like open rule induction, it is important to further train the langauge models. I agree that we always need fine-tune to make langauge model follow our instructions. However, this is not something supervising for people who use pre-trained langauge models. It will be much more interesting to discuss how much cost we need to pay in order to extract the unlimited knowlege from pre-trained language models. If this paper can better address this issue, its impact will be significantly improved.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "n/a", "review_text": "", "extra_scores": {"ethical_concerns": "n/a", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "GT_KmCK1DC", "reviewer": "Reviewer_ycgL", "summary": "This paper presents a novel method of inducing probabilistic horn clauses (x founded y) --> (x ceo of y) from the predictions of a masked language model. This is done by first mapping the premise to instances (jobs founded apple) and then decoding the predicate for the hypothesis. The approach is interesting, but the current paper is let down by the quality of the experiments, which do not give a clear picture of how accurate or useful the induced rules are.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper presents a method of generating probabilistic Horn cluases using a language model. The probability of a hypothesis (x is ceo of y) following a premise (x founded y) is broken down into a sum over instances of the premise, and the conditional probability of the hypothesis given these instances. This factorization permits the use of a masked language model that either predicts entities that should be substituted for some variable (instantiation of the premise) or the relation that links a pair of entities (the mapping from support to hypothesis). This is a clever way of adopting a masked language model to reason about rules that hold over sets of entity pairs, but it is also not clear to what extent the resultant probabilities properly capture the probability of a rule holding, rather than the probability of a specific verbalisation of that rule.\n\nA BART language model is trained to model these probabilties using weak supervision via a masking scheme that focuses on pairs of entities and beam search is used to decode the instances and hypotheses. A new beam search strategy is introduced to avoid issues arising from greedy decoding of instances and hypotheses, but this method only marginally impacts the model's performance.\n\nThe approach is evaluate against the COMET system as well as an unmodified BART + prompt on a newly constructed evaluation set as well as evaluation sets that are automatically constructed from some standard relation extraction benchmarks. The proposed approach outperforms the baselines according to BLEU, ROUGE, and METEOR, but it's not clear how well these text generation metrics capture the accuracy of the generated rules. Table 3 clearly shows that the new approach is hallucinated unsupported hyptoheses and, from the examples in this table, it is not clear that COMET is a suitable baseline for the evaluation setting.\n\nThere is also a comparison to KB-based rule induction which uses human judgements of accuracy. Here the proposed approach achieves greater coverage at a small cost in terms of accuracy. I would like to see some discussion of how accuracy was determined. Almost all of the examples in Figure 1 and Table 3 are not universally true, if not completely hallucinated. A clear definition of what the authors consider to be a 'good rule' would be useful in interpreting this evaluation.\n\nA further evaluation replaces the hand crafted explanations used in ExpBert with rules induced by ORION and shows that this matches ExpBert performance (mean performance is even slightly higher, but within variance). This is an interesting result but I am confused about the singificant discrepency between the numbers reported in Table 4 and the ExpBert paper. The authors mention that they have a more performant training scheme. but the results reported for the BERT baseline are significantly lower than those from the ExpBert paper, while the results for ExpBert + annotated rules are significantly higher. Given that this experiment is intended to demonstrate the value of the induced rules, I would like to see a direct comparison to the ExpBert system that only changes the form of the supplied rules.\n\n----\n\nScore raised from 5 -> 6 after response from authors. See reasoning in response to response.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This paper relies on generation metrics (BLEU, ROUGE, METEOR) to evaluate the quality of induced logical rules. The current paper does not justify this choice and these metrics do not give a clear picture of how useful the rules are, or whether the increase over COMET is due to genuinely better inference or simply task mismatch. I would like to see an expansion of the human evaluation provided in Section 5.4, as well as accuracy metrics for the comparison to COMET.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "4DJ00pFsPRd", "reviewer": "Reviewer_Sn8S", "summary": "This paper proposes a new LM-based method for extracting rules from a pre-trained masked language model. The key idea is fine-tuning a pre-trained language model (BART [16]) with external relational description corpora to two masked language models (MLMs). For every given rule body, the first MLM predicts ground substitutes (which are named entities) in the rule body and the second MLM predicts text sequences except the ground substitutes in the rule head. Thus, the proposed method is able to compute multiple rule heads for a given rule body. Experimental results on a manually constructed rule set and six relation-extraction datasets demonstrate that the proposed method outperforms state-of-the-art LM-based methods and is comparable to state-of-the-art KB-based methods in finding correct rules. Further results also show that the generated rules can make relation extraction more accurate than manually annotated rules do.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Learning natural language rules is useful in many real-life applications. For example, in relation extraction, the learned rules can be used as templates to discover the relationship between two entities. Thus, a more effective way to learn natural language rules is very desirable. The paper presents a LM-based method for this task and can have a significant impact. Different from the prompt-based method for generating text sequences from LMs [20], the proposed method does not directly generate text sequences in rule heads; instead, it first generates multiple ground substitutes in the given rule body, and then generates text sequences in the rule head according to the generated ground substitutes. This way in generating text sequences in rule heads is novel to the best of my knowledge. As far as I can see from the paper, the proposed method is technically sound and justified by adequate experiments, although no source code is provided in the supplemented material. The paper is generally well organized and written, but the presentation could still be improved in the following aspects. First, the restrictions on learned rules need to be given explicitly. For example, all variables appearing in learned rule heads should appear in the given rule body as well; the structure of rule heads to be learned is predefined. Second, the accuracy metric used in Table 2 needs more clarification. The estimation of this metric requires criteria for manually deciding whether a learned rule is correct or not. These criteria should be clarified. Finally, the application for applying learned rules to identify errors in LMs is problematic. On one hand, the incorrectness of a learned rule may be caused by the proposed learning method (e.g. an inappropriate value of the beam search width) rather than by the LM. Different causes are hard to be distinguished. On the other hand, even if we know that the LM has errors, we cannot correct the LM based on the learned rules. In other words, the motivation for this application is very unclear. The effectiveness of this application needs to be carefully demonstrated.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No, the authors have not adequately addressed the limitations of their work. Besides the restrictions on learned rules that should be given explicitly, the dependency between the performance of learned rules in downstream tasks and the beam search width (which determines the number of rule heads learned for one rule body) needs to be explored.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Automatically extracting logic rules from unstructured data can be practically useful. The proposed rule extraction method based on pre-trained language models is novel and flexible. The empirical results are good on different tasks. The rebuttals clarified lots of issues raised in the initial reviews. An immediate limitation of this work is that the bias of pre-trained LMs could result in unexpected rules.
Decision: Accept (Poster)