Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes to linearize GRU and LSTM cells (as error terms should be negligible when inputs are small in magnitude). Putting these linearized, or, really, affine, RNN cells together into a single-layer sequence processor, thanks to the affine-ness, we can decompose the score that is obtained by taking dot products with a query at each timestep into contribution by immediate unigram features and all subsequences leading to this unigram. The authors evaluate these scores, showing that they do and don't capture phenomena in a synthetic dataset and proceeding to show that when both training and evaluating with this simplified network on SST yields strong results.

#### Strengths/what I loved:

- Motivation and Related Work seemed nicely done, set this paper up nicely, and made me excited to read on!
- I like the idea of testing double negation and omitting it during training and it was interesting to see the networks then fail to pick up on it (assuming the synthetic dataset is reasonable).
- The visualizations of sequence-level scores (Figures 2 and 4) are very cleverly chosen and powerful, even if they take a bit of getting used to.
- Figures 3 and 5 are striking: even on SST unigrams and perhaps bigrams seem to get you most of the way if you trust the approximate interpretation.

#### Criticism/weaknesses:

- It is unclear how the simplified/affine-ized architecture relates to vanilla RNNs---those, too, can be decomposed like that and one could just as well look at these features. To paraphrase: I don't see why any of the analysis and results in this paper is only true for gated cells and I wouldn't be surprised to see a vanilla RNN yield equally good approximate cells, since after all the task is very simplistic (Section 5.2 concludes as much, as even without sequence information the task is easy to solve, but the conclusion drawn from that result, namely that there is something about GRUs and LSTMs to read from this doesn't follow in my opinion).
- The synthetic dataset is a mystery: yes, it contains sentences that contain the words shown in Table 6 (Appendix A.2), but... what are these sentences? Are they actual text from some dataset? Text sampled from some model or grammar? Random words without any sequential coherence? What is the vocabulary size? What does appearing "mostly" in positive or negative instances really mean? Just giving one or two examples would have made me a lot less worried and confused about what is going on here, but to base many if not most of the results on it, this dataset is woefully underdescribed.

#### Questions:

- Notation/definitions in section 5.1.1 are either very unclear or flat-out wrong: Sun & Lu (2020) do indeed define a notion of a "token-level polarity score" for each output class, but the notion of output classes is not mentioned at all here, in fact instead of the output embedding matrix W that Sun & Lu (2020) posit, this paper speaks only of a single vector w. I assume that that is the vector associated with the positive class and the prediction thus is strictly binary---very much unlike Sun & Lu (2020). In addition to that, the notion of a "sequence-level polarity score" that here complements the token-level score is not at all mentioned in Sun & Lu (2020) as far as I can see, so to say that their methodology is used is misleading in more than one way. Finally, the model described in Sun & Lu (2020) is one that uses attention, i.e., that does mean-pooling after the linear layer that is transforming the individual states. This paper mentions the linear layer, but not the pooling/attention, so it's unclear if that is a poor paraphrase of Sun & Lu or whether this paper here too diverges from the paper it claims to build on.
- Are Figures 2 and 4 cherry-picked? Since nothing else is stated, I would assume so. In light of that, the longest subsequence on the right of Figure 2 being negative is rather strange. Do you have an explanation? Is "those" a negative word?
- I think it would've been very interesting to see whether or not this simplification is legitimate, that is whether the assumptions made in the derivation are justified: train with the original cell formulation, but then evaluate *quantitatively* using the approximate cell to essentially create a table like Table 2 or perhaps even a scatter plot for individual logits to see how much things change or don't change. That would go a long way to convince me that the approximation is at least reasonable.
- A.3: "there's an apparent difference" What is that difference? I don't see any.
- A.4: What is the change here, can you highlight it or motivate it? The results certainly aren't particularly impressive, so I'm tempted to say this section hurts more than it helps...

#### Typos and other small things:

- 5.1: As you have multiple runs, which one was selected? The best according to some metric? The median or mean somehow? Or randomly chosen? Either way, I strongly feel that empirical results should always come with a sense of stability: what was the variance between runs? Between hyperparameters? How sensitive are results and what can we say about statistical significance?
- The overlaid histograms definitely need to have some transparency or be shown another way---right now it is impossible to see what is "happening" in the blue bars as they are hidden by the orange bars.
- The last sentence of Section 5 should link to A.4, I guess?


---

I read and appreciated the response, but my overall rating is still leaning negative.
review_title: Linearizing GRUs and LSTMs allows for decomposing into unigram- and substring contributions and that does well enough on sentiment analysis

final_score: rating: 3: Clear rejection
review_content: This paper examines n-gram level features encoded within the hidden states of recurrent neural models. The proposed method approximates the hidden states of LSTMs and GRUs with a first-order Taylor series, which the authors claim is an adequate approximation with small enough inputs. The authors apply their method to models trained on synthetic sentiment analysis and language modeling datasets. The paper is difficult to understand, and many assumptions are not properly justified. The experiments are also not convincing, as a large portion of the analysis focuses on small synthetic data, and some of them do not have clear takeaways or explanations as to why they were conducted in the first place. Overall, I cannot recommend the paper's acceptance in its current form.

comments/questions:
- is the scenario of extremely low input magnitudes realistic? how generalizable are these findings to standard initializations used in NLP architectures
-  similarly, on page 3 the authors assume that the higher order terms of h_{t-1} are "insignificant"; in practice, it is unclear how often this is true. i wish the paper would contain more justification behind these assumptions, as they are critical for judging how faithful the approximation is and thus how useful the proposed method is for diagnosing RNNs.
- can't the authors actually show quantitatively how good the approximations are? if the higher-order terms indeed do not affect the quality of the approximation, that could be justified by some experiments. i'm not really convinced by Table 2: the approximations could be quite different from the original model but still yield good downstream accuracy.
- what is a "polarity score" (bottom of page 5)? I didn't quite understand what this is supposed to represent, is it how predictive of a label a particular span is? 
- why are synthetic datasets used at all here? experiments on a small set of sentences with a tiny vocabulary and artificial "double negatives" are not compelling. Appendix A.2 does not fully specify this dataset (nor motivate why it was created); what is e.g., its average sentence length?
- the results on a real sentiment dataset (SST2) are confusing (sec 5.1.3): what does figure 4 show me that I couldn't already learn by simply passing those two phrases into the model as separate inputs and looking at the model's prediction? 
- what is the point of training models with the approximations instead of the original GRU/LSTM cell equations?  i don't understand the significance of Table 3.
review_title: nice idea but makes a lot of questionable assumptions

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper attempts to add a contribution on understanding how gated recurrent neural networks like GRUs and LSTMs can learn the representation of n-grams. The authors expand the sigmoid function and the hyperbolic tangent function using Taylor series to obtain approximated closed-form mathematical expression of hidden representation when using the GRU or the LSTM as the update rules. The approximated hidden representation of the GRU and the LSTM update rules can be separated into two terms, (1) the current token-level input feature and (2) the sequence-level feature, which is a weighted sum of all previous tokens. As the hidden representation consists of two feature terms, one can take each feature (either token-level or sequence-level) separately for a downstream task, e.g., evaluate how good when sequence-level feature is used for predicting polarity score in sentiment analysis.
 
The idea of improving theoretical understanding on how n-grams are modelled by gated recurrent activation functions is sound. However, I am not entirely satisfied with what has been investigated after obtaining the approximated closed-form expression of gated recurrent activation functions. The tasks that were used in the experiments are sentiment analysis and language modelling. In sentiment analysis, most of the plots were there to show how token-level features or sequence-level features align with the polarity score, and we can observe some sort of individual implication from each term. However, it is predictable that sequence-level feature should be meaningful. I don't see much of insights by showing that the polarity score from sequence-level features indeed align with this prediction. If we can to apply Taylor expansion to simple recurrent neural networks (RNNs), such that we can expand the hidden representation of a standard RNN into two terms: the current token-level input feature and sequence-level feature, how would the results look like and how can we relate them with what were reported in this paper? Is this paper particularly showing how gated RNNs are modelling n-grams or RNNs in general? A comparison would be nice to show how sequence features get improved in gated RNNs.

It is interesting to see that the approximated versions of GRUs and LSTMs can perform on a par with the original models on language modelling tasks, however, these results don't necessarily improve our understanding on how gated RNNs are capable of learning good representations of n-grams. They confirm that sequence features are indeed helpful though.

In Section 5.1, if there were multiple trials of experiments on the same task, why not report the average and the variance of the results instead of one set out of multiple results?

In Section 5.2, Adpative softmax (Joulin et al., 2017) was used for Wikitext-130. -> Adpative softmax (Joulin et al., 2017) was used for
Wikitext-103.
review_title: An interesting attempt to improve theoretical understanding on gated RNNs.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper provides a reliable interpretation of modern RNN models, through unrolling GRU and LSTM cells. The approximate state representations include a token-level term that only depends on the current input token and a sentence-level term that depends on all inputs until the current token. The deriving process is clear and illuminating. The experiment section shows that the approximation shares similar behavior and performance as the original model.

Overall, the paper is well written and easy to follow. Although GRU and LSTM are no longer the default model for SOTA performance in the NLP community. I believe that this study still provides interesting insights for those who want to develop better recurrent or non-recurrent models in the future.

My major concern is that the language model experiment didn’t include a stronger baseline method, such as AWD-LSTM, which provides a significantly lower ppl compared to these in the paper.

It would be interesting to see a more detailed ablation study, that studies the importance of each term in A(x_t).

review_title: Solid reasoning and Interesting Results

final_score: rating: 4: Ok but not good enough - rejection
review_content: DISCLAIMER: this is not my field of research. With strong arguments I could be persuaded to change my score.

This paper introduces a method to unroll the Gated Recurrent Unit (GRU) and the Long Short-Term Memory (LSTM) unit using a taylor expansion. Essentially a linearization of the GRU and LSTM. Given the simplifications, the authors argue that their model captures N-gram information for the sequential information. The paper presents results suggesting that this approximation is able to capture much of the same sequential information as the LSTM and GRU on benchmarks such as SST-2, PDB, Wikitext-2, and Wikitext-103.

What this paper excels at is a thorough theoretical formulation of the proposed approximation, and a comparison with an approximation without the sequential information. This shows that the approximate version is able to capture sequential information.

However, the relevance of this paper is not clear to me, and the introduction and related works does little more to explain that relevance than stating: “understanding the essential features captured by GRU/LSTM”. The tasks that the method is tested on are synthetic data for sentiment and tasks/models that haven't been relevant since 2016. The motivation for these tasks, and the qualitative analysis is hard for me to understand.

The paper could use some reformulations and more emphasis on what exactly the purpose of the paper is, in particular I find the lack of consistency in present/past tense disrupts the reading experience.

Below I have made a few comments/questions:

Abstract:
“Sequential data in practice” - unclear what this means
“sequence-level representations brought in by the gating mechanism” - I dont understand this sentence
“essential components” - vaquely defined
“Based on such a finding” - rephrase

Introduction:
“gradient vanishing or explosion issues” - vanishing or exploding gradients
“While such models ...” - this whole sentence is a little vague

Related work:
“With the variants” - its variants
General comment (also for introduction): While you mention many interesting findings in recent years, it is difficult for me to assess how exactly your work differs. Please use the related works to emphasize what you are doing differently than previous work in your field.

LSTM:
“A LSTM cell” - An LSTM cell

Experiments:
“Figure 2” I don’t get what each bar represents
“Subphrase labels” - are subphrase labels the node annotation?

Why is negation, and a synthetic variant, important to explain the relationship between N-grams and LSTMs?
Why do you choose the datasets you do? Why is SST-2 and benchmarking against older language models of interest? Why is an N-gram comparison interesting? Perhaps the authors should look into contemporary research on formal methods in sequential models for inspiration of tasks and where an interesting hypothesis might be: https://arxiv.org/abs/1906.03648

Update:
I have read the rebuttal and the updated paper. I don't see my issue of relevance addressed. My score remains the same.
review_title: Linear LSTMs

Meta-review: the authors demonstrated that vanilla RNN, GRU and LSTM compute at each timestep a hidden state which is the sum of the current input and the weighted sum of the previous hidden states (weights can be either unit or complicated functions), when sigmoid and tanh functions are replaced by their second-order taylor series each. they refer to the first term as token-level and the second term as sequence-level, and claim that the latter can be thought of as summing n-gram features in the case of GRU & LSTM due to the complicated weight matrices used for the weighted sum, largely arising from the gating mechanisms. 

the reviewers are largely unsure about the significance of the findings in this paper due to a couple of reasons with which i agree. first, it is unclear whether the proposed approximation scheme is enough to capture much of what happens within either GRU or LSTM. if we consider a single step, it's likely fine to ignore the O(x^3) term arising from either sigmoid or tanh, but when unrolled over time, it's unclear whether these error terms will accumulate or cancel each other. without either empirically or theoretically verifying the sanity of this approximation, it's difficult to judge whether the authors' findings are specific to this approximation scheme or do indeed reflect what happens within GRU/LSTM. 

second, because the authors have used relative simple benchmarks to demonstrate their points, it is difficult, if not impossible, to tell whether the authors' findings are about the datasets themselves (which are all well known to be easily solvable or solvable very well with n-gram classification models and n-gram language models) or about GRU/LSTM, which is related to the first weakness shared by the reviewer. the observations that n-gram models and simplified GRU/LSTM models work as well as the original GRU/LSTM models on these datasets might simply imply that these datasets don't require any complicated interaction among the tokens beyond counting n-grams, which lead to the original GRU/LSTM trained to be simplified (n-gram detectors.) 

that said, i still believe this direction is important and is filled with many interesting observations to be made. i suggest the authors (1) verify the efficacy of their approximation scheme (probably empirical validation is enough, and (2) demonstrate their point with more sophisticated problems (carefully designed synthetic datasets are perfectly fine.)



Decision: Reject