Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: 
The paper proposes a new model for numerical reasoning in machine comprehension. Given a passage and a query, the model outputs an arithmetic expression over numbers/dates in the passage (e.g. max(23, 26, 42)). The model is trained with weak supervision in the form of numerical answers only. This weak supervision is used to define reward for reinforcement learning training. A key claimed advantage of the model compared to the prior art is that it trains end-to-end from the rewards as the only form of supervision. This is contrasted to  neural module networks, which require program supervision for good performance, as well as GenBERT, which requires additional synthetic training data for pretraining. Two key quantitative results include: 
better performance on the DROP-num datasets, compared to NMNs with less supervision and GenBERT without data augmentation
comparable to strongly-supevised NMN performance on DROP-Pruned-num.

The general approach is quite elegant and makes sense. It is encouraging that the paper reports successful training with RL. It is also important to build models that use less extra supervision. 

That said, I have some concerns regarding the paper’s positioning. The introduction, as well as many other places in the text categorizes some of the prior art as “learning a multi-type answer predictor over different reasoning types (e.g., max/min, diff/sum, count, negate) and directly predicting the corresponding numerical expression, instead of learning to reason”. What is learning to reason then? For example, in NAQANet the model predicts whether the numbers in the passage should be summed or subtracted from each other, why is this not learning to reason? Second, calling this model a “module network” is misleading, in my opinion. In neural module networks modules learn to do things, and here the key modules of discrete reasoning are predefined. The model also contains “modules” that are conditioned on different question spans. But there is no experiment checking whether having multiple such modules is actually useful. There is furthermore no qualitative explanation of what these modules are supposed to do and what they are actually doing. Lastly, I am not sure the use of synthetic data for GenBERT can be called “strong supervision”. Data augmentation and strong supervision are not the same thing. 

The paper is very dense and is quite hard to read. A lot of space is allocated to a very detailed technical presentation of “modelling interactions”, while the high-level picture of how the model functions is still hard to grasp. For example, Figure 2 is confusing because it has a “Stacked Span Prediction” pathway that leads to nowhere. Reading the text I find that apparently the output of this part is actually used for “modelling interaction between programs and number entities”, which is in the right part of the figure. These basic high-level architectural decisions are hard to understand as the reader is overwhelmed by technical details, such as sliding windows and scaling factors for various attentions. 

The way the results are displayed in the table is somewhat confusing. The fully-supervised NMN baseline is shown in Table 2 for DROP-Pruned-num but is not shown in Table 1 for Table-num. Instead, it can be found in Table 3. I would recommend presenting all results in one table, even if it shows that the current model performs worse than others with more supervision or data augmentation. Furthermore, I think that comparing top-k accuracies for the proposed model and top-1 accuracies for other models, as it is done in Table 3, does not make sense.

In summary, while I think the paper might be proposing an interesting model with promising results, I also think that presentation needs work. It would be great to see a clearer high-level explanation of the difference between the proposed model and the prior work. To this end, the prior work should be better discussed (notably there is no Related Work section at the moment). Besides, more quantitative and/or qualitative results are needed to support the hypothesis that the model performs “noisy query execution”. 

Other minor comments:
- it is the first time I see the word skyline used to mean “the baseline from above” 
- there is a lot of really long sentences in the paper, which makes the reading very hard. I’d recommend to try and break them up into shorter ones. 

review_title: interesting model, but there are positioning and presentation issues

final_score: rating: 7: Good paper, accept
review_content: ICLR 2021 Review

Title: Elaborate modeling with cutting-edge techniques, but limited on generalizability and lacks analysis.

This paper introduces a method to train neuro-symbolic module networks with weak supervision signal, and out-performs previous models on datasets containing numerical problems. Dislike previous works, WNSMN (Weakly-Supervised Neuro-Symbolic Module Network) doesn't rely on strong supervision and annotations but incorporates many inductive biases for numerical operations. It learns to execute programs that are extracted from dependency parse with noisy heuristics and are trained end-to-end in a reinforcement learning framework. 

Neural Module Networks are a very promising trend for machine reading comprehension for that it explicitly involves the reasoning process, but training such a system requires expensive annotations. Weakly supervised learning was infeasible because of the exponentially large action space, but this paper manages to address this issue by some inductive biases. To make the searching efficient, WNSMN designs modules for the interaction between program and passage, as well as programs and entities. These inductive biases are implemented with an attention mechanism and make the action size tractable. Besides, the sampling process is also learnable, with two carefully designed samplers -- operator and argument samplers. 

The training technique also plays an important role. Instead of training with discrete binary rewards and from scratch, WNSMN starts with hard EM objective with heuristic silver answers. The training process with the reinforcement learning objective adopts a multi-arm bandit assumption and entropy penalty. These techniques alleviate the problems of instability and overfitting.

Overall this paper is brilliant, adopts proper techniques, and elaborately designs the architecture. However, it's also limited in generalizability and needs more analysis.

Compared to the two baseline works, i.e. Neural Module Networks (NMNs) and GenBERT, WNSMN only works for numerical problems. The modules designed in the model, like the entity attention and operator sampler, are ad-hoc for problems involving dates and numbers. It's hard to extend WNSMN to more general cases. So it's questionable whether this framework could be applied to other problems of DROP like PERSON or VERB PHRASE.

One of the crucial pathologies of weakly supervised learning is the spurious prediction, where the model outputs the correct with wrong steps. However, this paper doesn't analyze this problem with WNSMN. So I'm wondering what percentage of the correct predictions of WNSMN is derived from the correct reasoning process? This could be studied with a small subset of human-annotated data and is helpful to rule out the possibility that the model learns some un-interpretable patterns.

The first step of WNSMN, i.e. query parser, is rule-based and makes less sense in contrast to other parts. It severely limits the ability of the model by just incorporating the first layer of the dependency parse tree. How can this model deal with complex patterns that are not step-by-step? How do you guarantee that the dependency relationship between steps is from left to right? I'm not convinced by the explanation "the QA model needs to be robust to such noise and additionally rely on the full query representation in order to predict the discrete operation." It will be clear if you can post statistics on the percentage covered by this simple heuristic.

Questions:

1. Is there a specific reason that you focus on numerical problems?
2. During hard EM, you choose the best action based on your current policy. How many good actions do you have in the first step? If there are too many good actions, it's still hard for the model to learn from correct signals.
3. Continued from question 2, why do you choose the best action, instead of smoothed over all the good actions?
4. To train the in-house baseline models, do you train them on the whole dataset or just the subset that involves numbers and dates? Is that possible that NMNs or GenBERT could learn useful patterns from other types of questions to answer numerical questions?

Presentation suggestions. The paper presentation is clear and dense overall. My suggestions would be:
1. Limit the content of describing the interactions. Section 2.1.1 is overwhelming and intimidating to me, with a lot of definitions and equations. The idea behind the interactions, however, is not that complex. Some descriptions of the methods could either be abridged or moved to the appendices, like the multi sliding windows and multi-scaling tricks given they're not your contribution. Also, the interactions between program/passage and program/entity are very similar, and you might be able to refactor these two paragraphs.
2. Curly brackets should be used for the set instead of ordered vectors. E.g. stacked attention windows in section 2.1.1.
3. Words in equations should be surrounded by \mathrm or \mathit to make them less like the production of many variables.
review_title: Elaborate modeling with cutting-edge techniques, but limited on generalizability and lacks analysis.

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes a neurosymbolic module network that predicts a program structure following a dependency parse, populates that program's arguments, and executes it to answer numerical reasoning questions over text.  They claim that compared to Gupta et al. (2020), this approach doesn't require as many domain-specific heuristics to find gold programs or as much precomputation -- it is learned with weak supervision only (just the answers). The model has a number of pieces allowing the model to reference entities, numbers, and dates in a cross-attentive fashion. Results show that on numerical questions from the DROP dataset, the model outperforms that of Gupta et al. and is competitive with other approaches when appropriate assumptions are made.

In general, I like what this paper is trying to do. In a vacuum, I might recommend accepting it. However, my most pressing concern with this paper is the fairly fundamental similarity to this ICLR 2020 spotlight paper:

https://openreview.net/pdf?id=ryxjnREFwH
Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension
Xinyun Chen, Chen Liang, Adams Wei Yu, Denny Zhou, Dawn Song, Quoc V. Le

I will refer to this paper as Chen et al. and their system as NeRd.

The authors failed to cite this paper, so I presume they are not aware of it. There are many similarities, including the broad strokes behind the DSL and the learning approach with weak supervision. I will discuss this current paper in contrast to that one to highlight the differences and the contributions.

Model: My favorite aspect of the current work is the fact that it leverages dependency representations. I like this idea a lot and buy that it might be a path to more general models of this form. However, it's hard for me to evaluate whether this is really more general due to the evaluation focused on this dataset, so this contribution feels somewhat theoretical. Would this dependency-based formalism work well elsewhere, or does it just happen to work well for DROP? It feels hard to make this work for larger, more compositional problems where the syntax-semantics divide is going to manifest itself more strongly (e.g., surrounding quantification).

The thing I like least about this paper is the complexity of the model.  The NeRd programmer is substantially simpler, as a basic seq2seq model. While you can argue that they have a more aggressive approach for warm-starting, like Gupta et al., there's less inductive bias built into the model directly.  By contrast, the current paper has a large number of bespoke modules for handling things from the DROP dataset.  I'm okay with things like treating entities as first-class concepts and allowing attention over them. But specialized attention maps targeting numbers and dates starts to feel very tailored to this dataset. Given the focus on numerical reasoning, perhaps some of this is to be expected, but I believe the other past approaches (NeRd, Gupta) have architectures that are a bit more general.

DSL: The DSLs between these two approaches are quite similar, but this is to be expected as the task is the same.

Weak supervision: Chen et al. use some heuristics as well as data augmentation to start learning off by giving the model access to a larger number of programs which find the right answer right off the bat. To handle spurious programs, they use hard EM wth an additional thresholding step to avoid training on programs that are extremely unlikely.  The present work initializes with a hard EM-style learning before switching to REINFORCE. In my view, there is not much of a conceptual advantage to this approach over NeRd. I don't think this paper has a particularly keen insight to solve the cold-start problem for RL here, and the NeRd-style data augmentation is not that big of a weakness. So I don't see a fundamental contribution here over NeRd.

Results: Table 1 looks convincing but I'm having a hard time fully understanding it. It seems like the low numbers for NMN in DROP-num-Test are because this includes many question types explicitly excluded by the original Gupta et al. paper -- is this correct? This is fair but I think should be more explicit in the table.

As for Table 2, the ablations of different attention methods are interesting, but I'm not sure this is a fully fair comparison to Gupta et al. These pieces can't be mapped in an apples-to-apples fashion to aspects of the current approach. Gupta et al.  could just as easily say: well, let me ablate your specialized date handling module and do a comparison on that axis. In a new setting, we can claim such sources of supervision might be unavailable, but it's not clear to me that the system with this component deleted is the fairest point of comparison.

It also seems based on the Table 3 comparisons, we do not expect WNSMN to perform well compared to NeRd (see Table 4 in that paper).

OVERALL

Normally I am not a stickler for novelty over prior work. However, in this case, I feel compelled to judge this paper and directly compare it to Chen et al., as the motivation, evaluation conditions, and technical details of these papers bear striking similarities. (That paper also evaluates on MathQA, which is another point in its favor.) And in summary, this work does not offer enough new material to justify acceptance.  It is quite well-done work and has some interesting insights, though it suffers from an overly complex neural model. But the insights are not new, and the results are not enough to convince me that it should be accepted.
review_title: Missing critical comparison to NeRd

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposed a weakly supervised module networks to solve numeric reasoning problems. The model is trained with reinforcement learning over the modules. The model solved a few challenges in training the RL model. They evaluated their model on a subset of the DROP dataset. 

This paper was clearly presented and the results was promising. My only concern is that they only evaluate the model on a carefully constructed subset. I am not sure how it compare to other models in more general settings and/or on other datasets. 

The authors mentioned that most questions in the dataset only requires 1-hop reasoning, so they reduced their model to 1-step MDP. Will your model work on questions that involves multi-step reasoning? Can you show that?

Did you detect the query span as a preprocessing step? What if the query spans are noisy/missing/duplicated? 

* I did not check the correctness of the equations.
review_title: A well written paper with good experiment results.

Meta-review: This paper proposes a weakly supervised model for numerical reasoning. After discussion with the reviewers it seems that it is already known that training NMNs directly on DROP is not successful and requires taking additional measures. Past work (NERD) has resorted to using data augmentation, and this work encodes it directly to the model. This paper needs to show the advantages of their approach and that it generalizes better to other scenarios. Other minor issues include (a) clarity fo writing (b) focus on a subset of questions (c) no evaluation on other numerical datasets (d) mild inaccuracies w.r.t prior work (GenBERT)
Decision: Reject