R ule BERT : Teaching Soft Rules to Pre-Trained Language Models (2021)

Abstract: While pre-trained language models (PLMs) are the go-to solution to tackle many natural language processing problems, they are still very limited in their ability to capture and to use common-sense knowledge. In fact, even if information is available in the form of approximate (soft) logical rules, it is not clear how to transfer it to a PLM in order to improve its performance for deductive reasoning tasks. Here, we aim to bridge this gap by teaching PLMs how to reason with soft Horn rules. We introduce a classification task where, given facts and soft rules, the PLM should return a prediction with a probability for a given hypothesis. We release the first dataset for this task, and we propose a revised loss function that enables the PLM to learn how to predict precise probabilities for the task. Our evaluation results show that the resulting fine-tuned models achieve very high performance, even on logical rules that were unseen at training. Moreover, we demonstrate that logical notions expressed by the rules are transferred to the fine-tuned model, yielding state-of-the-art results on external datasets.

## introduction pre-trained language models (plms) based on transformers @xcite are established tools for capturing both linguistic and factual knowledge @xcite . however, even the largest models fail on basic reasoning tasks. if we consider common relations between entities, we see that such models are not aware of negation, inversion (e.g., parent-child), symmetry (e.g., spouse), implication, and composition. while these are obvious to a human, they are challenging to learn from text corpora as they go beyond linguistic and factual knowledge @xcite . we claim that such reasoning primitives can be transferred to the plms by leveraging logical rules, such as those shown in figure 1 . ## related work plms have been shown to have some reasoning capabilities @xcite , but fail on basic reasoning tasks @xcite and are inconsistent @xcite , especially when it comes to negation @xcite . our work focuses on deductive reasoning. note that it is different from previous work, e.g., on measuring the factual knowledge of plms @xcite , on probing the commonsense capabilities of plms at the token or at the sentence level @xcite , or on testing the reasoning capabilities of plms on tasks such as age comparison and taxonomy conjunction @xcite . our work relates to task #15 in the babi dataset @xcite and to rule-takers @xcite . however, we differ (i) by using a larger subclass of first-order logic rules (with more variables and various forms), and (ii) by incorporating soft rules. our proposal is different from work on question answering (qa) with implicit reasoning based on common-sense knowledge @xcite , as we rely purely on deductive logic from explicitly stated rules. our approach also differs from methods that semantically parse natural language into a formal representation on which a formal reasoner can be applied @xcite , as we directly reason with language. yet, we are also different from natural language inference (nli) and textual entailment, which work with text directly, but cannot handle horn rules @xcite . unlike previous work @xcite @xcite , we do not design a new, ad-hoc module for neural reasoning, but we rely solely on the transformer's capability to emulate algorithms @xcite . logical rules. we rely on existing corpora of declarative horn rules mined from large rdf knowledge bases (kbs) @xcite @xcite ). an rdf kb is a database representing information with triples (or facts) p(s, o), where a predicate p connects a subject s and an object o. an atom in a rule is a predicate connecting two universally quantified variables. a horn rule (or clause) has the form: @xmath0, y), where h(x, y) is a single atom (head or conclusion of the rule) and b (body or premise of the rule) is a conjunction of atoms. positive rules identify relationships between entities, e.g., r 1 , r 2 , r 4 in figure 1 . negative rules identify contradictions, e.g., r 3 in figure 1 . rules can contain predicates comparing numerical values, such as <. for example, negative rule r 5 : birthyear(b,d) ∧ foundyear(a,c) ∧ <(c,d) → negfounder(a,b) states that any person (variable b) with a birth year (d) higher than the founding year (c) of a company (a) cannot be its founder. a fact is derived from a rule if all the variables in the rule body are replaced with constants from facts. for r 5 , facts "foundyear @xcite @xcite )" trigger the rule that derives the fact negfounder(e. musk, ford). ## dataset we start by defining the reasoning task. we then discuss example generation methods for three scenarios: single rule as input, multiple (possibly conflicting) rules that require reasoning for the same conclusion, and multiple rules that require a sequence (chain) of reasoning steps. examples of the data generation procedures are in the appendix. ## teaching plms to reason in this section, we explain how we teach a plm to reason with one or more soft rules. note that uncertainty stems from the rule confidence. one approach to teach how to estimate the probability of a prediction is to treat each confidence value (or bucket of confidence values) as a class and to model the problem as a k-way classification instance (or regression), but this is intractable when multiple rules are considered. instead, we keep the problem as a two-class one by altering how the information is propagated in the model to incorporate uncertainty from the rule confidence. let @xmath1, y i )} m @xmath2, where x i is one example of the form (context,hypothesis,confidence) and y i is a label indicating whether the hypothesis is validated or not by the context (facts and rules in english), and m is the size of the training set. a classifier f is a function that maps the input to one of the labels in the label space. let h(x, y) be a classification loss function. the empirical risk of the classifier f is we want to introduce uncertainty in our loss function, using the weights computed by the lp mln solver as a proxy to represent the probability of predicting the hypothesis as being true. to do so, we apply a revised empirical risk: when trained to minimize this risk, the model learns to assign the weights to each output class, thus predicting the confidence for the true class when given the satisfied rule head as a hypothesis. ## experiments we first describe the experimental setup (section 6.1). we then evaluate the model on single (section 6.2) and on multiple rules (sections 6.3 and 6.4). we show that a plm fine-tuned on soft rules, namely rulebert, makes accurate predictions for unseen rules (section 6.5), and it is more consistent than existing models on three external datasets (section 7). we report the values of the hyper-parameters, as well as the results for some ablation experiments in the appendix. the datasets for all experiments are summarized in ## rulebert on external datasets as our fine-tuning propagates information in the layers of the encoder, we hypothesize that rule-bert effectively "learns" logical properties of the concepts represented in the rules, such as negation and symmetry, and thus it could perform better on tasks testing such properties of plms. to study the negation of predicates, we use the negated lama datasets, which test how plms distinguish a cloze question and its negation @xcite . in most cases, plms make the same prediction both for a positive statement ("relativity was developed by einstein.") and for its negation ("relativity was not developed by einstein."). to test the symmetry relationship between predicates, we use the srl test in checklist (ribeiro et al., 2020), which focuses on behavioral testing of nlp models; we use its test set for the duplicate-question detection task (qqp) @xcite . finally, we test deductive reasoning on the babi dataset and its task #15 @xcite . ## conclusion and future work we studied whether plms could reason with soft rules over natural language. we experimented with one flavor of probabilistic answer set programming (lp mln ), but other semantics can be also used with the proposed methodology. we further explored the inference capabilities of transformerbased plms, focusing on positive and negative textual entailment. we leave non-entailment for future work. we also leave open the development of explainable models. some approaches use occlusion that removes parts of the input and checks the impact on the output @xcite or build proof iteratively using 1-hop inference @xcite .