Beta- LR : Interpretable Logical Reasoning based on Beta Distribution (2024)

Abstract: The logical information contained in text isof significant importance for logical reasoning.Previous approaches have relied on embeddingtext into a low-dimensional vector to capturelogical information and perform reasoning inEuclidean space. These methods involve constructing special graph architectures that matchlogical relations or designing data augmentation frameworks by extending texts based onsymbolic logic. However, it presents two obvious problems. 1) The logical informationreflected in the text exhibits uncertainty that isdifficult to represent using a vector. 2) Integrating logical information requires modeling logical operations (such as ∪, ∩, and ¬), while onlysimple arithmetic operations can be performedin Euclidean space. To address both the problems, we propose Beta-LR, a probabilistic embedding method to capture logical information.Specifically, we embed texts into beta distribution on each dimension to eliminate logical uncertainty. We also define neural operators thatenable interpretability and perform logical operations based on the characteristics of the betadistribution. We conduct experiments on twodatasets, ReClor and LogiQA, and our Beta-LRachieves competitive results. The experimentsdemonstrate that our method effectively captures the logical information in text for reasoning purposes. The source code is available athttps://github.com/myz12138/Beta-LR.

## introduction in recent years, there has been an increasing focus on logical reasoning @xcite @xcite , which presents a significant challenge as it necessitates the extraction of crucial information from text. figure 1 provides an example of a logic reasoning problem taken from the reclor @xcite dataset, which serves as a benchmark for evaluating logical reasoning capabilities. this dataset is in the form of a context, a * corresponding author ## methodology to enhance our ability to capture and utilize logical information, we introduce a novel probabilistic embedding method, beta-lr. the architecture of beta-lr is detailed in figure 2 . the process begins with segmenting the context into individual sentence units and encoding them using beta distributions, as discussed in section 4.1. we then proceed to develop and apply logical operators, which allow us to integrate and interpret the logical information across these sentences, creating a cohesive logical representation of the context in section 4.2. in the final step, outlined in section 4.3, we integrate this comprehensive context representation with the given question and options, enabling accurate answer prediction. ## experiment we evaluate the performance of our beta-lr on two datasets, reclor @xcite and logiqa @xcite . furthermore, we conduct an ablation study to examine the effectiveness of logical operators and crucial modules. during the training process, the adamw @xcite with β1 = 0.9 and β2 = 0.99 is taken as the optimizer and batch size is set to 8. the learning rate is 1e-6 for reclor and 5e-6 for logiqa. the model is trained for 20 epochs and 10 epochs on reclor and logiqa respectively, to obtain the optimal results. ## conclusion in this paper, we introduce beta-lr, a probabilistic embedding method for interpretable logical reasoning. our work is the first to address logical reasoning at the level of data embedding representation. through experiments conducted on two datasets, we demonstrate that beta-lr achieves competitive performances in logical reasoning tasks. the re-sults validate the effectiveness of our method in capturing and utilizing logical information for reasoning purposes. by addressing the challenges of logical uncertainty and the fusion of logical information, beta-lr provides a valuable solution for enhancing logical reasoning capabilities. limitations there could be two limitations to our approach. firstly, our beta-lr only calculates the shared logical information among all sentences, but overlooks the shared components between any two sentences. additionally, representing large amounts of textual logical information using bounded beta distributions still presents challenges. in future research, we will explore more comprehensive probabilistic embedding methods to effectively learn logical information from text.