Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming (2023)

Abstract: Pre-trained large language models (LMs) struggle to perform logical reasoning reliably despite advances in scale and compositionality. In this work, we tackle this challenge through the lens of symbolic programming. We propose DSR-LM, a Differentiable Symbolic Reasoning framework where pre-trained LMs govern the perception of factual knowledge, and a symbolic module performs deductive reasoning. In contrast to works that rely on hand-crafted logic rules, our differentiable symbolic reasoning framework efficiently learns weighted rules and applies semantic loss to further improve LMs. DSR-LM is scalable, interpretable, and allows easy integration of prior knowledge, thereby supporting extensive symbolic programming to robustly derive a logical conclusion. The results of our experiments suggest that DSR-LM improves the logical reasoning abilities of pre-trained language models, resulting in a significant increase in accuracy of over 20% on deductive reasoning benchmarks. Furthermore, DSR-LM outperforms a variety of competitive baselines when faced with systematic changes in sequence length.

## introduction complex applications in natural language processing involve dealing with two separate challenges. on one hand, there is the richness, nuances, and extensive vocabulary of natural language. on the other hand, one needs logical connectives, long reasoning chains, and domain-specific knowledge to draw logical conclusions. the systems handling these two challenges are complementary to each other and are likened to psychologist daniel kahneman's human "system 1" and "system 2" @xcite : while the former makes fast and intuitive decisions, akin to neural networks, the latter thinks more rigorously and methodically. considering lms as "system 1" and symbolic reasoners as "system 2", we summarize their respective advantages in although pre-trained lms have demonstrated remarkable predictive performance, making them an effective "system 1", they fall short when asked to perform consistent logical reasoning @xcite @xcite , which usually requires "system 2". in part, this is because lms largely lack capabilities of systematic generalization @xcite @xcite . in this work, we seek to incorporate deductive logical reasoning with lms. our approach has the same key objectives as neuro-symbolic programming @xcite : compositionality, consistency, interpretability, and easy integration of prior knowledge. we present dsr-lm, which tightly integrates a differentiable symbolic reasoning module with pre-trained lms in an end-to-end fashion. with dsr-lm, the underlying lms govern the perception of natural language and are finetuned to extract relational triplets with only weak supervision. to overcome a common limitation of symbolic reasoning systems, the reliance on human-crafted logic rules @xcite , we adapt dsr-lm to induce and finetune rules automatically. further, dsr-lm allows incorporation of semantic loss obtained by logical integrity constraints given as prior knowledge, which substantially helps the robustness. we conduct extensive experiments showing that dsr-lm can consistently improve the logical reasoning capability upon pre-trained lms. even if dsr-lm uses a roberta backbone with much less parameters and does not explicitly take triplets as supervision, it can still outperform various baselines by large margins. moreover, we show that dsr-lm can induce logic rules that are amenable to human understanding to explain decisions given only higher-order predicates. as generalization over long-range dependencies is a significant weakness of transformer-based language models @xcite , we highlight that in systematic, long-context scenarios, where most pre-trained or neural approaches fail to generalize compositionally, dsr-lm can still achieve considerable performance gains. there are other obvious remedies for lms' poor reasoning capability. ensuring that the training corpus contains a sufficient amount of exemplary episodes of sound reasoning reduces the dependency on normative biases and annotation artifacts @xcite @xcite . heuristics like data augmentation are also shown to be effective @xcite . but the above works require significant efforts for crowdsourcing and auditing training data. our method handily encodes a few prototypes/templates of logic rules and is thus more efficient in terms of human effort. moreover, our goal is fundamentally different from theirs in investigating the tight integration of neural and symbolic models in an end-to-end manner. neuro-symbolic reasoning. neuro-symbolic approaches are proposed to integrate the perception of deep neural components and the reasoning of symbolic components. representative works can be briefly categorized into regularization @xcite , program synthesis @xcite , and proof-guided probabilistic programming @xcite @xcite @xcite . to improve compositionality of lms, previous works propose to parameterize grammatical rules @xcite but show that those hybrid models are inefficient and usually underperform neural counterparts. in contrast to the above works, dsr-lm focuses on improving lms' reasoning over logical propositions with tight integration of their pre-trained knowledge in a scalable and automated way. ## experiments we evaluate dsr-lm on both clutrr and dbpedia-inf. we show that dsr-lm has accurate and generalizable long-range reasoning capability. ## concluding remarks we investigate how to improve lms' logical reasoning capability using differentiable symbolic rea-soning. through extensive experiments, we demonstrate the effectiveness of dsr-lm over challenging scenarios where widely deployed large lms fail to reason reliably. we hope our work can lay the groundwork for exploring neuro-symbolic programming techniques to improve the robustness of lms on reasoning problems.