Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning

1 INTRODUCTION . End-to-end neural models have proven to be powerful tools for an expansive set of language and vision problems by effectively emulating the input-output behavior . However , many real problems like Question Answering ( QA ) or Dialog need more interpretable models that can incorporate explicit reasoning in the inference . In this work , we focus on the most generic form of numerical reasoning over text , encompassed by the reasoning-based MRC framework . A particularly challenging setting for this task is where the answers are numerical in nature as in the popular MRC dataset , DROP ( Dua et al. , 2019 ) . Figure 1 shows the intricacies involved in the task , ( i ) passage and query language understanding , ( ii ) contextual understanding of the passage date and numbers , and ( iii ) application of quantitative reasoning ( e.g. , max , not ) over dates and numbers to reach the final numerical answer . Three broad genres of models have proven successful on the DROP numerical reasoning task . First , large-scale pretrained language models like GenBERT ( Geva et al. , 2020 ) uses a monolithic Transformer architecture and decodes numerical answers digit-by-digit . Though they deliver mediocre performance when trained only on the target data , their competency is derived from pretraining on massive synthetic data augmented with explicit supervision of the gold numerical reasoning . Second kind of models are the reasoning-free hybrid models like NumNet ( Ran et al. , 2019 ) , NAQANet ( Dua et al. , 2019 ) , NABERT+ ( Kinley & Lin , 2019 ) and MTMSN ( Hu et al. , 2019 ) , NeRd ( Chen et al. , 2020 ) . They explicitly incorporate numerical computations in the standard extractive QA pipeline by learning a multi-type answer predictor over different reasoning types ( e.g. , max/min , diff/sum , count , negate ) and directly predicting the corresponding numerical expression , instead of learning to reason . This is facilitated by exhaustively precomputing all possible outcomes of discrete operations and augmenting the training data with the reasoning-type supervision and numerical expressions that lead to the correct answer . Lastly , the most relevant class of models to consider for this work are the modular networks for reasoning . Neural Module Networks ( NMN ) ( Gupta et al. , 2020 ) is the first explicit reasoning based QA model which parses the query into a specialized program and executes it step-wise over learnable reasoning modules . However , to do so , apart from the exhaustive precomputation of all discrete operations , it also needs more fine-grained supervision of the gold program and the gold program execution , obtained heuristically , by leveraging the abundance of templatized queries in DROP . While being more pragmatic and richer at interpretability , both modular and hybrid networks are also tightly coupled with the additional supervision . For instance , the hybrid models can not learn without it , and while NMN is the first to enable learning from QA pair alone , it still needs more finer-grained supervision for at least a part of the training data . With this , it manages to supercede the SoTA models NABERT and MTMSN on a carefully chosen subset of DROP using the supervision . However , NMN generalizes poorly to more open-ended settings where such supervision is not easy to handcraft . Need for symbolic reasoning . One striking characteristic of the modular methods is to avoid discrete reasoning by employing only learnable modules with an exhaustively precomputed space of outputs . While they perform well on DROP , their modeling complexity grows arbitrarily with more complex non-linear numerical operations ( e.g. , exp , log , cos ) . Contrarily , symbolic modular networks that execute the discrete operations are possibly more robust or pragmatic in this respect by remaining unaffected by the operation complexity . Such discrete reasoning has indeed been incorporated for simpler , well-structured tasks like math word problems ( Koncel-Kedziorski et al. , 2016 ) or KB/TableQA ( Zhong et al. , 2017 ; Liang et al. , 2018 ; Saha et al. , 2019 ) , with Deep Reinforcement Learning ( RL ) for end-to-end training . MRC however needs a more generalized framework of modular neural networks involving more fuzzy reasoning over noisy entities extracted from open-ended passages . In view of this , we propose a Weakly-Supervised Neuro-Symbolic Module Network ( WNSMN ) • A first attempt at numerical reasoning based MRC , trained with answers as the sole supervision ; • Based on a generalized framework of dependency parsing of queries into noisy heuristic programs ; • End-to-end training of neuro-symbolic reasoning modules in a RL framework with discrete rewards ; To concretely compare WNSMN with contemporary NMN , consider the example in Figure 1 . In comparison to our generalized query-parsing , NMN parses the query into a program form ( MAX ( FILTER ( FIND ( ‘ Carpenter ’ ) , ‘ goal ’ ) ) , which is step-wise executed by different learnable modules with exhaustively precomputed output set . To train the network , it employs various forms of strong supervision such as gold program operations and gold query-span attention at each step of the program and gold execution i.e. , supervision of the passage numbers ( 23 , 26 , 42 ) to execute MAX operation on . While NMN can only handle the 6 reasoning categories that the supervision was tailored to , WNSMN focuses on the full DROP with numerical answers ( called DROP-num ) that involves more diverse reasoning on more open-ended questions . We empirically compare WNSMN on DROP-num with the SoTA NMN and GenBERT that allow learning with partial or no strong supervision . Our results showcase that the proposed WNSMN achieves 32 % better accuracy than NMN in absence of at least one or more types of supervision and performs 8 % better than GenBERT when the latter is fine-tuned only on DROP in a comparable setup , without additional synthetic data having explicit supervision . 2 MODEL : WEAKLY SUPERVISED NEURO-SYMBOLIC MODULE NETWORK . We now describe our proposed WNSMN that learns to infer the answer based on weak supervision of the QA pair by generating the program form of the query and executing it through explicit reasoning . Parsing Query into Programs To keep the framework generic , we use a simplified representation of the Stanford dependency parse tree ( Chen & Manning , 2014 ) of the query to get a generalized program ( Appendix A.5 ) . First , a node is constructed for the subtree rooted at each child of the root by merging its descendants in the original word order . Next an edge is added from the left-most node ( which we call the root clause ) to every other node . Then by traversing left to right , each node is organized into a step of a program having a linear flow . For example , the program obtained in Figure 1 is X1 = ( ‘ which is the longest ’ ) ; X2 = ( ‘ goal by Carpenter ’ , X1 ) ; Answer = Discrete-Reasoning ( ‘ which is the longest ’ , X2 ) . Each program step consists of two types of arguments ( i ) Query Span Argument obtained from the corresponding node , indicates the query segment referred to , in that program step e.g. , ‘ goal by Carpenter ’ in Step 2 ( ii ) Reference Argument ( s ) obtained from the incoming edges to that node , refers to the previous steps of the program that the current one depends on e.g. , X1 in Step 2 . Next , a final step of the program is added , which has the reference argument as the leaf node ( s ) obtained in the above manner and the query span argument as the root-clause . This step is specifically responsible for handling the discrete operation , enabled by the root-clause which is often indicative of the kind of discrete reasoning involved ( e.g. , max ) . However this being a noisy heuristic , the QA model needs to be robust to such noise and additionally rely on the full query representation in order to predict the discrete operation . For simplicity we limit the number of reference arguments to 2 . 2.1 PROGRAM EXECUTION . Our proposed WNSMN learns to execute the program over the passage in three steps . In the preprocessing step , it identifies numbers and dates from the passage , and maintains them as separate canonicalized entity-lists along with their mention locations . Next , it learns an entity-specific crossattention model to rank the entities w.r.t . their query-relevance ( §2.1.1 ) , and then samples relevant entities as discrete arguments ( §2.1.2 ) and executes appropriate discrete operations on them to reach the answer . An RL framework ( §2.1.3 ) trains it end-to-end with the answer as the sole supervision . 2.1.1 ENTITY-SPECIFIC CROSS ATTENTION FOR INFORMATION EXTRACTION . To rank the query-relevant passage entities , we model the passage , program and entities jointly . Modeling interaction between program and passage This module ( Figure 2 , left ) learns to associate query span arguments of the program with the passage . For this , similar to NMN , we use a BERT-base pretrained encoder ( Devlin et al. , 2018 ) to get contextualized token embeddings of the passage and query span argument of each program step , respectively denoted by Pk and Qk for the k ’ th program step . Based on it , we learn a similarity matrix S 2 l⇥n⇥m between the program and passage , where l , n , and m respectively are the program length and query span argument and passage length ( in tokens ) . Each Sk 2 n⇥m represents the affinity over the passage tokens for the k ’ th program argument and is defined as Sk ( i , j ) = wT [ Qki ; Pkj ; Qki Pkj ] , where w is a learnable parameter and is element-wise multiplication . From this , an attention map Ak is computed over the passage tokens for the k ’ th program argument as Ak ( i , j ) = softmaxj ( Sk ( i , j ) ) = exp ( Sk ( i , j ) ) P j exp ( Sk ( i , j ) ) . Similarly , for the i ’ th token of the k ’ th program argument the cumulative attention aki w.r.t . the passage is aki = softmaxi ( P j Sk ( i , j ) ) . A linear combination of the attention map Ak ( i , · ) weighted by aki gives the expected passage attention for the k ’ th step , ↵̄k = P i akiAk ( i , · ) 2 m. Span-level smoothed attention . To facilitate information spotting and extraction over contiguous spans of text , we regularize the passage attention so that the attention on a passage token is high if the attention over its neighbors is so . We achieve this by adopting a heuristic smoothing technique ( Huang et al. , 2020 ) , taking a sliding window of different lengths ! = { 1 , 2 , . . . 10 } over the passage , and replacing the token-level attention with the attention averaged over the window . This results in 10 different attention maps over the passage for the k ’ th step of the program : { ↵̄ ! k | ! 2 { 1 , 2 , . . . , 10 } } . Soft span prediction . This network takes a multi-scaled ( Gupta et al. , 2020 ) version of ↵̄ ! k , by multiplying the attention map with |s| different scaling factors ( s = { 1 , 2 , 5 , 10 } ) , yielding a |s|dimensional representation for each passage token , i.e. , ↵̄ ! k 2 m⇥|s| . This is then passed through a L-layered stacked self-attention transformer block ( Vaswani et al. , 2017 ) , which encodes it to m⇥ d dimension , followed by a linear layer of dimension d⇥ 1 , to obtain the span prediction logits : ↵ ! k = Linear ( Transformer ( MultiScaling ( ↵̄ ! k ) ) 2 m. Further the span prediction logits at each program step ( say k ) is additively combined with those from the previous steps referenced in the current one , through the reference argument ( ref ( k ) ) at step k , i.e. , ↵ ! k = ↵ ! k + P k02ref ( k ) ↵ ! k0 . Modeling interaction between program and number/date entities This module ( Figure 2 , right ) facilitates an entity-based information spotting capability , that is , given a passage mention of a number/date entity relevant to the query , the model should be able to attend to the neighborhood around it . To do this , for each program step , we first compute a passage tokens to number tokens attention map Anum 2 l⇥m⇥N , where N is the number of unique number entities . Note that this attention map is different for each program step as the contextual BERT encoding of the passage tokens ( Pk ) is coupled with the program ’ s span argument of that step . At the k-th step , the row Anumk ( i , · ) denotes the probability distribution over the N unique number tokens w.r.t . the i-th passage token . The attention maps are obtained by a softmax normalization of each row of the corresponding passage tokens to number tokens similarity matrix , Snumk 2 m⇥N for k = { 1 . . . l } , where the elements of Snumk are computed as S num k ( i , j ) = P T kiWnPknj with Wn 2 d⇥d being a learnable projection matrix and nj being the passage location of the j-th number token . These similarity scores are additively aggregated over all mentions of the same number entity in the passage . The relation between program and entities is then modeled as ⌧ ! k = softmax ( P i ↵ ! kiA num k ( i , · ) ) 2 N , which gives the expected distribution over the N number tokens for the k-th program step and using ! as the smoothing window size . The final stacked attention map obtained for the different windows is T numk = { ⌧ ! k | ! 2 { 1 , 2 , . . . 10 } } . Similarly , for each program step k , we also compute a separate stacked attention map T datek over the unique date tokens , parameterized by a different Wd . A critical requirement for a meaningful attention over entities is to incorporate information extraction capability in the number and date attention maps Anum and Adate , by enabling the model to attend over the neighborhood of the relevant entity mentions . This is achieved by minimizing the unsupervised auxiliary losses Lnumaux and Ldateaux in the training objective , which impose an inductive bias over the number and date entities , similar to Gupta et al . ( 2020 ) . Its purpose is to ensure that the passage attention is densely distributed inside the neighborhood of ± ⌦ ( a hyperparameter , e.g. , 10 ) of the passage location of the entity mention , without imposing any bias on the attention distribution outside the neighborhood . Consequently , it maximises the log-form of cumulative likelihood of the attention distribution inside the window and the entropy of the attention distribution outside of it . ( 1 ) Lnumaux = 1 l lX k=1 mX i=1 [ log ( NX j=1 nj2 [ i± ⌦ ] a num kij ) NX j=1 nj 62 [ i± ⌦ ] a num kij log ( a num kij ) ] where is indicator function and anumkij = A num k ( i , j ) . Ldateaux for date entities is similarly defined .

Summary: The paper proposes a new model for numerical reasoning in machine comprehension. Given a passage and a query, the model outputs an arithmetic expression over numbers/dates in the passage (e.g. max(23, 26, 42)). The model is trained with weak supervision in the form of numerical answers only. This weak supervision is used to define reward for reinforcement learning training. A key claimed advantage of the model compared to the prior art is that it trains end-to-end from the rewards as the only form of supervision. This is contrasted to  neural module networks, which require program supervision for good performance, as well as GenBERT, which requires additional synthetic training data for pretraining. Two key quantitative results include: