SlowGuess/NAACL_2022_OCR huggingface dataset:
We employ the sentiment analysis, relation classification, named entity recognition (NER) and language modeling tasks as testbeds. The first task is often used in investigating  $n$ -gram phenomena (Yessenalina and Cardie, 2011; Li et al., 2016) while the others are often used in examining how capable an encoder is when extracting features from texts (Grave et al., 2018; Zhou et al., 2016; Lample et al., 2016).
Datasets For sentiment analysis, we considered the Stanford Sentiment Treebank (SST) (Socher et al., 2013), the IMDB (Maas et al., 2011), and the AG-news topic classification<sup>7</sup> (Zhang et al.,
For brevity, we suppress biases following Merrill et al. (2020).
7AG-news can be viewed as a special sentiment analysis dataset.
Definition	Parameterization
Eiunian	ht=tanh(Winxt+Wihhht-1)	A(xt)=diag(tanh'(Winxt))Wih	g(xt)=tanh(Winxt).
GRU	rt=σ(Wirxt+Whrht-1)zt=σ(Wizxt+Whzht-1)nt=tanh(Winxt+rtCWht-1)ht=(1-zt)Cn+ztCht-1	A(xt)=diag[fn(xt) ⊕[1-gz(xt)] ⊕gr(xt)]Wh	where:gr(xt)=σ(Wirxt),f_r(t)=g_r'(xt),gz(xt)=σ(Wizxt),f_z(xt)=g_z'(xt),gn(xt)=tanh(Winxt),fn(xt)=gn'(xt).
- diag[gn(xt) ⊕ fz(xt)]Whz
+ diag[gs(xt)]
g(xt)=[1-gz(xt)] ⊕ gn(xt)
LSTM	it=σ(Wiixt+Whiht-1)ft=σ(Wifxt+Whfht-1)ot=σ(Wioxt+Whoht-1)ctm=tanh(Wicxt+Whcht-1)ct=ft ⊙ct-1+itCtmtht=ot tanh(ct)	A(xt)=[BtEtF], g(xt)=[[gc(xt)gh(xt)]	where:gc(xt)=gi(xt) ⊕gc^m(xt),gh(xt)=go(xt) ⊕tanh[gc(xt)],gi(xt)=σ(Wixt),f_i(xt)=g_i'(xt),gf(xt)=σ(Wifxt),f_f(xt)=g_f'(xt),go(xt)=σ(Wioxt),fo(xt)=g_o'(xt),gc^m(xt)=tanh(Wicxt),fc^m(xt)=tanh'(Wicxt).
Bt=diag[gt(xt)]
Et=diag[go(xt) ⊕tanh'[gc(xt)]Bt
Dt=diag[gc^m(xt) ⊕ fi(xt)]Wh
+diag[gt(xt) ⊕ fc^m(t)]Wh
Ft=diag[go(xt) ⊕tanh'[gc(xt)]Dt
+diag[fo(xt) ⊕tanh[gc(xt)]]Wh
2015) datasets. The first dataset has sufficient labels for phrase-level analysis, the second dataset has instances with relatively longer lengths, and the third one is multi-class. For relation classification and NER, we considered the SemEval 2010 Task 8 (Hendrickx et al., 2010) and CoNLL-2003 (Tjong Kim Sang and De Meulder, 2003) datasets respectively. For language modeling, we considered the Penn Treebank (PTB) dataset (Marcus et al., 1993), the Wikitext-2 (Wiki2) dataset and the Wikitext-103 (Wiki103) dataset (Merit et al., 2016). PTB is relatively small while Wiki103 is large. The statistics are shown in Tables 6 and 7 in the appendix.
Baselines The  $n$ -gram representations (together with their corresponding context representations) discussed in the literature are considered as baselines, which are listed in Table 1 along with the MVMA and MVM models. MVM(A)-G/L/E refers to the MVM(A) model created with the  $A$  and  $g$  functions derived from GRU/LSTM/Elman, but are trained directly from data. The  $A$  and  $g$  functions for GRU, LSTM and Elman are listed in Table 2.
Additionally, to understand whether the complexity of  $A$  affects the expressive power, we created a new model called MVMA-ME, which comes with an  $A$  function that is slightly more complex than that of MVMA-E but less complex than those of MVMA-G and MVMA-L:  $A(x_{t}) = 0.25\mathrm{diag}[\tanh (Wx_{t})]M + 0.5I$  and  $g(x_{t}) = \tanh (W^{\prime}x_{t})$  (here,  $W$ ,  $M$  and  $W^{\prime}$  are learnable weight matrices). The  $g$  function is the same as that of MVMA-E.
Setup For sentiment analysis, relation classification and language modeling, models consist of one embedding layer, one RNN layer, and one fully-connected layer. The Adagrad optimizer (Duchi et al., 2011) was used along with dropout (Srivastava et al., 2014) for sentiment analysis $^{8}$  and rela
Table 2: Parameterization of  $A$  and  $g$  by Elman RNN, GRU, and LSTM.  $x_{t}$  is the representation of the input token  $x_{t}$  and  $W_{**}$  refers to a weight matrix.  $\sigma$  and  $\tanh$  are the element-wise sigmoid and tanh functions respectively.  $g'$ ,  $\tanh'$  and  $f'$  refer to the element-wise derivative. The diag operation converts a vector into a diagonal matrix.
Model	SST-2	AG-news	IMDB
dev	test	dev	test	dev	test
MM	86.0±1.3	85.6±0.4	-	-	-	-
VA-W	80.6±1.6	80.4±1.4	90.3±0.4	90.0±0.3	88.0±0.6	88.0±0.4
VA-EW	82.6±0.3	82.0±0.3	-	-	-	-
MV-MG	84.9±0.5	85.0±1.0	84.9±4.0	84.4±4.0	50.9±0.0	50.2±0.1
MV-L	85.4±0.4	84.9±0.8	86.9±1.7	86.5±1.7	51.0±0.1	50.2±0.1
MVM-E	59.6±1.6	59.5±1.1	-	-	-	-
MVMA-G	87.0±0.4	85.3±0.5	91.6±0.5	91.3±0.3	90.5±0.5	89.6±0.7
MVMA-L	86.7±1.0	85.4±1.0	91.4±0.5	91.3±0.5	89.4±0.6	89.2±0.6
MVMA-E	81.4±1.1	80.8±1.5	-	-	-	-
MVMA-ME	83.2±0.5	81.9±0.3	90.6±0.5	90.2±0.3	80.6±0.5	80.1±1.1
GRU	84.9±0.9	84.9±0.5	92.1±0.1	91.6±0.3	87.7±0.2	87.2±0.3
LSTM	84.3±0.8	84.4±0.3	91.9±0.4	91.5±0.5	89.0±0.1	88.7±0.4
Elman	79.1±0.3	79.7±1.4	87.5±0.5	87.5±0.6	67.0±1.9	66.7±0.9
Table 3: Accuracy percentage (↑) on sentiment analysis (text classification) datasets (averaged over 3 runs). “-” means the model failed to converge.
tion classification. For language modeling, models were trained with the Adam optimizer (Kingma and Ba, 2014). We ran word-level models with truncated backpropagation through time (Williams and Peng, 1990) where the truncated length was set to 35. Adaptive softmax (Joulin et al., 2017) was used for Wiki103. For NER, models consist of one embedding layer, one bidirectional RNN layer, one projection layer and one conditional random field (CRF) layer. The SGD optimizer was used. Final models were chosen based on the best validation results. More implementation details can be found in the appendix.
5.1 Comparison of Representation Models
We investigate how baseline  $n$ -gram representation models<sup>9</sup>, the MVM model, and the MVMA model perform on the aforementioned testbeds. We also compare with the standard RNN models.
Sentiment Analysis Apart from the GRU and LSTM models, it can be observed that our MVMA-G and MVMA-L models are also able to achieve competitive results on three sentiment analysis datasets, as we can see from Table 3, demonstrating the efficacy of those recurrence-induced  $n$ -gram
spectral normalization (Miyato et al., 2018) was used on the weight matrices  $\mathbf{W}_{h*}$  for standard RNNs.
We excluded VM, which we found was hard to train. We also excluded MVM-R which only considers bigrams.
<sup>8</sup>We investigated the approximation between RNNs and their corresponding recurrence relations in Appendix B.2. The
representations. Although Elman RNN and its corresponding MVMA-E and MVM-E models also have a mechanism for capturing  $n$ -gram information (similar to GRU and LSTM), they did not perform well, which may be attributed to a limited expressive power of their  $A$  and  $g$  functions when used for defining  $n$ -grams as described previously.
Both MM and VA-EW fail to converge on AGnews and IMDB, showing challenges for them to handle long instances. This may be explained by the lengthy matrix multiplication involved in their representations, which may result in vanishing/exploding gradient issues. Interestingly, MVM-G and MVM-L, which solely rely on the longest  $n$ -gram representation, are also able to achieve good results on SST-2, indicating a reasonable expressive power of such  $n$ -gram representations alone. However, they fail to catch up with MVMA-G and MVMA-L on IMDB which contains much longer instances, confirming the significance of the context representation, which captures  $n$ -grams of varying lengths.
Unlike MVMA-E, the MVMA-ME model does not suffer from loss stagnation on AG-news and IMDB but the performance on IMDB obviously falls behind MVMA-G and MVMA-L as shown in Table 3. This indicates a sufficiently expressive  $A(x_{t})$  (such as the Jacobian matrices of GRU and LSTM) may be needed to handle long instances.
Relation Classification & NER For relation classification, context representations (or final hidden states) are used for classification. For NER, we use the concatenated context representations (or hidden states) at each position of bidirectional models to predict entities and their types. Table 4 shows that MVMA-G and MVMA-L outperform the MVM-G and MVM-L models respectively on both tasks, again confirming the effectiveness of the context representations. MVM(A)-E did not perform as well as MVM(A)-G and MVM(A)-L, which demonstrates the significance of expressive power for the  $A$  and  $g$  functions. Similar to the results in sentiment analysis, MVMA-ME did not perform as well as MVMA-G and MVMA-L. However, to our surprise, MVMA-ME did not outperform VA-EW on NER, suggesting that a delicate choice of  $A$  can be important for this task. The poor performance of VA-W on NER might be explained by a weak expressive power of its  $n$ -gram representations. MM fails to converge on the relation classification task, which implies it is not robust across different datasets. Interestingly, it is remarkable that MVMA-G, MVMA-L and MVMA-E could yield competitive results compared to GRU, LSTM
Model	Relation Classification	NER
dev	test	dev	test
MM	-	-	33.9±0.6	30.8±0.4
VA-W	41.2±0.2	37.9±0.9	17.6±0.6	16.5±1.6
VA-EW	39.7±1.1	38.3±0.7	70.8±0.7	63.4±1.0
MVM-G	51.2±0.5	52.6±0.7	54.2±1.6	47.6±2.2
MVM-L	48.8±1.3	50.5±1.5	53.8±1.7	46.6±1.6
MVM-E	-	-	27.8±0.9	25.6±0.9
MVMA-G	62.2±1.0	59.7±0.1	75.0±0.4	67.7±0.5
MVMA-L	57.5±0.3	56.2±0.8	75.6±0.2	67.9±0.3
MVMA-E	27.8±0.9	25.6±0.9	69.0±0.4	61.7±0.1
MVMA-ME	46.3±0.9	46.2±0.6	67.0±0.5	57.6±0.8
GRU	67.2±0.6	62.2±0.2	75.6±0.5	67.9±0.5
LSTM	65.2±0.9	61.3±1.4	76.3±0.5	68.1±0.5
Elman	27.8±0.9	25.6±0.9	67.1±0.9	58.6±0.6
Table 4: F1 scores  $(\uparrow)$  (averaged over 3 runs) on the relation classification and NER tasks. “-” means the model failed to converge.
Model	PTB	Wiki2	Wiki103
GRU	dev	118.4±0.4	146.1±0.4	109.4±0.6
test	110.1±0.4	136.8±0.1	113.3±0.8
MVMA-G	dev	119.8±0.4	150.3±0.8	111.8±0.5
test	111.1±0.2	140.2±1.0	115.2±0.5
MVM-G	dev	146.5±1.3	170.1±2.8	-
test	138.8 ± 1.0	160.0±2.6	-
LSTM	valid	118.6±0.4	150.6±0.6	108.3±0.6
test	109.8±0.4	140.4±0.8	112.4±0.8
MVMA-L	dev	121.5±0.5	152.0±0.5	109.1±0.6
test	113.2±0.5	142.5±0.7	112.6±0.6
MVM-L	dev	124.3±1.5	155.6±0.9	-
test	117.0±1.0	145.7±1.6	-
MVMA-ME	dev	140.7±0.9	169.0±1.0	153.1±4.2
test	134.0±1.0	158.4±1.4	157.4±4.3
Table 5: Perplexities  $(\downarrow)$  on language modeling (averaged over 5 runs). " -": the model failed to converge.
and Elman on NER, implying such  $n$ -gram representations could be crucial for our NER task.
Language Modeling For the language modeling task, we choose MVMA-G, MVMA-L, MVM-G and MVM-L for experiments. We also run MVMA-ME. As we can see from Table 5, there are performance gaps between the MVMA models and the standard RNNs - though the gaps often do not appear to be particularly large. This indicates there may be extra information within higher order terms of the standard RNN functions useful for such a task. Yet, such information cannot be captured by the MVMA models that employ simplified functions. The gaps between the MVM models and MVMA models are remarkable, which again indicates that the correct way of defining the left context representation can be crucial for the task of next word prediction. MVMA-ME did not perform well on the language modeling task, which might be attributed to the less expressive power of its functions  $A$  and  $g$ .
5.2 Interpretation Analysis
We conduct some further analysis to examine the interpretability of  $n$ -gram representations. Specifically, we examine whether the models are able to capture certain linguistic phenomena such as negation, which is important for sentiment analysis (Ribeiro et al., 2020). We also additionally
made comparisons with the vanilla Transformer (Vaswani et al., 2017) here<sup>10</sup> despite the fact that it remains largely unclear how it precisely captures sequence features such as  $n$ -grams.
We could also obtain the  $n$ -gram representations and the corresponding context representations from the learned standard RNN models, based on their learned parameters. We denote such  $n$ -gram representations as  $\mathrm{RNN}_{n\text{-gram}}$ , and the context representations as  $\mathrm{RNN}_{\text{context}}$ , where "RNN" can be GRU, LSTM or Elman. As  $n$ -gram representations are vectors, a common approach is to transform them into scalars with learnable parameters (Murdoch et al., 2018; Sun and Lu, 2020). We define the  $n$ -gram polarity score to quantify the polarity information as captured by an  $n$ -gram representation  $\pmb{v}_{i:t}$  from time step  $i$  to  $t$ , which is calculated as:
$$
s _ {i: t} ^ {\boldsymbol {v}} = \boldsymbol {w} ^ {\top} \boldsymbol {v} _ {i: t}, \tag {14}
$$
where  $\pmb{w}$  is the learnable weight vector of the final fully-connected layer. We also define the context polarity score for the context as  $\sum_{i=1}^{t} s_{i:t}^{\pmb{v}}$ .
We trained RNNs and baseline models on SST-2 and automatically extracted 73 positive adjectives (e.g., "nice" and "enjoyable") and 47 negative adjectives (e.g., "bad" and "tedious") from the vocabulary<sup>11</sup>.  $N$ -gram polarity scores were calculated for those adjective unigrams and their negation bigrams formed by preponding "not" to them. For VA-EW and VA-W, their  $n$ -gram representations do not involve tokens other than the last token. Such limitations prevent them from capturing any negation information. We therefore calculate the context polarity scores using their context representations instead (which in this case is a bigram). This also applies to Transformer for the same reason.
We observed that, for the GRU and LSTM models along with their corresponding MVMA models, the  $n$ -gram representations are generally able to learn the negation for both the adjective and their negation bigrams as shown in Figures 2a and  $2\mathrm{b}^{12}$ , preponding "not" to an adjective will likely reverse the polarity. This might be a reason why they could achieve relatively higher accuracy on the sentiment analysis tasks. Interestingly, MVM-G could also capture negation as shown in Figure 2c, again suggesting the impressive expressive power of such  $n$ -gram representations alone.
However, as shown in Figure 2, models such as VA-W, MVMA-E, and MM are struggling to capture negation for negative adjectives, again implying a weaker expressive power of their  $n$ -gram representations. Specifically, MVMA-E fails to capture negation for negative adjectives, which may be attributed to a relatively weaker Jacobian matrix function  $A$  (as compared to those of GRU and LSTM) preventing them from pursuing optimal conditions.
Figure 2e shows that the MVMA-ME model, which has a function  $A$  less complex than the ones from MVMA-G and MVMA-L but more complex than the one from MVMA-E, still can generally learn negation of negative adjectives better than the MVMA-E model. This demonstrates the necessity of choosing more expressive  $A$  and  $g$  functions.
Interestingly, both VA-W and Transformer are struggling with capturing the negation phenomenon for negative adjectives in our experiments as shown in Figures 2g and 2h, which suggests that they may have a weaker capability in modeling sequential features in our setup. However, we found they could still achieve good performances on the AGnews and IMDB datasets<sup>13</sup>. We hypothesize this is because the nature of SST-2 makes these two models suffer more on this dataset - it has rich linguistic phenomena such as negation cases while the other two datasets do not.
Additionally, we examined the ability for GRU, LSTM, MVMA-G and MVMA-L to capture both the negation and intensification phenomena. For such experiments, instead of using SST-2, we trained the models on SST-5, which comes with polarity intensity information. Polarity intensities were mapped into values of  $\{-2, -1, 0, +1, +2\}$ , ranging from extremely negative to extremely positive. We conducted some experiments based on the same setup above for capturing negation on SST-2. To our surprise, our preliminary results show that all models were performing substantially worse in terms of capturing intensification than capturing negations. We hypothesize that this is caused by the imbalance between negation phrases and intensification phrases. Specifically, the intensification word "very" (1,729 times) was exposed less than the negation word "not" (4,601 times) in the training set of SST-5.
One approach proposed in the literature for sentence classification is to consider all the hidden states of an RNN in an instance (Bahdanau et al., 2015). We believe this may actually be able to al
<sup>13</sup>We conducted additional experiments for Transformers on sentiment analysis. Results are in appendix.
10 The mean of output representations was treated as the context representation for Transformer during training. We also tried to use the concatenation of the first and last token, following (Luan et al., 2019), which yielded similar results.
11 Such adjectives and detailed automatic extraction process can be found in the appendix.
12 Results of LSTM are similar to GRU, which can be found in the appendix.
(a)  $\mathrm{GRU}_{n\cdot \text{gram}}$
(b) MVMA-G
(c) MVM-G
(d) MVMA-E
(e) MVMA-ME
(f) MM
(g) VA-W
(h) Transformer
Figure 2: Distribution of polarity scores for adjectives and their negation bigrams on SST-2.  $p$ -adj and  $n$ -adj refer to the positive and negative adjectives respectively. [-] refers to the negation operation (prepending the word "not"). Circles refer to outliers. More results can be found in the appendix.
Figure 3: Context polarity scores (MVMA-G, SST-5) for positive (L) and negative (R) adjectives along with their negation and intensification bigrams.
leviate the above issue as it allows more  $n$ -grams within an instance to be exposed to the label information. Thus, we followed their approach for training our MVMA and MVM models<sup>14</sup>.
We can see that the negation and intensification phenomena can be explained by both the context representations in Figure 3<sup>15</sup>. Specifically, preponding either positive or negative adjectives with "very" will likely strengthen their polarity while adding "not" will likely weaken their polarity. These results suggest that RNNs are able to capture information of linguistic significance within the sequence, and our identified  $n$ -gram representations within their hidden states appear to be playing a salient role.
From the experiments above, we can see that our introduced  $n$ -gram representations, coupled with the corresponding context representations, are powerful in practice in capturing  $n$ -gram information better than the baseline compositional models introduced in the literature. We also found that RNNs can induce such representations due to their recurrence mechanism $^{16}$ .
However, there can be several factors that affect the efficacy of different representations. First, through comparisons with different variants of
MVMA, we can see that the precise way of parameterizing the functions  $A(x_{t})$  and  $g(x_{t})$  matter. Second, through the comparison between MVMA and MVM, we can see that defining an appropriate context representation that incorporates a correct set of  $n$ -grams is also important. Third, for models which do not capture such explicit  $n$ -gram features like ours, interestingly, they may still be able to yield good performances on certain tasks. For example, though VA-W and Transformer did not perform well on SST-2, they yielded results competitive to GRU and LSTM on AG-news and IMDB. This observation indicates there could be other useful features captured by such models that can contribute towards their overall modeling power.
Although in this work we did not aim to propose novel or more powerful architectures, we believe our work can be a step towards better understanding of RNN models. We also hope it can provide inspiration for our community to design more interpretable yet efficient architectures.
In this work, we focused on investigating the underlying mechanism of RNNs in terms of handling sequential information from a theoretical perspective. Our analysis reveals that RNNs contain a mechanism where each hidden state encodes a weighted combination of salient components, each of which can be interpreted as a representation of a classical  $n$ -gram. Through a series of comprehensive empirical studies on different tasks, we confirm our understandings on such interpretations of these components. With the analysis coupled with experiments, we provide findings on how RNNs learn to handle certain linguistic phenomena such as negation and intensification. Further investigations on understanding how the identified mechanism may capture a wider range of linguistic phenomena such as multiword expressions (Schneider et al., 2014) could an interesting future direction.
<sup>14</sup>However, for simplicity, in this work we only used the mean context representations (or hidden states) instead of a weighted sum of them.
15 More results are in the appendix.
<sup>16</sup>We also visualized the context representations and  $n$ -gram representations in the appendix, which provide intuitive understanding of them.
We would like to thank the anonymous reviewers and our ARR action editor for their constructive comments. This research/project is supported by the Ministry of Education, Singapore, under its Tier 3 Programme (The Award No.: MOET32020-0004). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not reflect the views of the Ministry of Education, Singapore.
Leila Arras, José Arjona-Medina, Michael Widrich, Grégoire Montavon, Michael Gillhofer, Klaus-Robert Müller, Sepp Hochreiter, and Wojciech Samek. 2019. Explaining and interpreting lstms. In *Explainable ai: Interpreting, explaining and visualizing deep learning*, pages 211-238. Springer.
Leila Arras, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2017. Explaining recurrent neural network predictions in sentiment analysis. In Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR.
Marco Baroni and Roberto Zamparelli. 2010. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of EMNLP.
Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2017. What do neural machine translation models learn about morphology? In Proceedings of ACL.
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. The journal of machine learning research, 3:1137-1155.
Satwik Bhattachamishra, Kabir Ahuja, and Navin Goyal. 2020. On the Ability and Limitations of Transformers to Recognize Formal Languages. In Proceedings of EMNLP.
Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2016. Deep neural networks for learning graph representations. In Proceedings of AAAI.
Hanjie Chen, Guangtao Zheng, and Yangfeng Ji. 2020. Generating hierarchical explanations on text classification via feature interaction detection. In Proceedings of ACL.