Deepening Hidden Representations from Pre-trained Language Models
Authors: Junjie Yang
hai zhao
Venue:  (2021)
Keywords: Natural Language Processing
Representation Learning

Abstract: Transformer-based pre-trained language models have proven to be effective for learning contextualized language representation. However, current approaches only take advantage of the output of the encoder's final layer when fine-tuning the downstream tasks. We argue that only taking single layer's output restricts the power of pre-trained representation. Thus we deepen the representation learned by the model by fusing the hidden representation in terms of an explicit HIdden Representation Extractor (HIRE), which automatically absorbs the complementary representation with respect to the output from the final layer. Utilizing RoBERTa as the backbone encoder, our proposed improvement over the pre-trained models is shown effective on multiple natural language understanding tasks and help our model rival with the state-of-the-art models on the GLUE benchmark.

Under review as a conference paper at ICLR 2021
DEEPENING HIDDEN REPRESENTATIONS FROM LANGUAGE MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Transformer-based pre-trained language models have proven be effective for learning contextualized language representation. However, current approaches only take advantage of the output of the encoder's final layer when fine-tuning the downstream tasks. We argue that only taking single layer's output restricts the power of pre-trained representation. Thus we deepen the representation learned by the model by fusing the hidden representation in terms of an explicit HIdden Rep- resentation Extractor (HIRE), which automatically absorbs the complementary representation with respect to the output from the final layer. Utilizing RoBERTa as the backbone encoder, our proposed improvement over the pre-trained models is shown effective multiple natural language understanding tasks and help our model rival with the state-of of-thc-art models on the GLUE benchmark.
INTRODUCTION
Language representation is essential to the understanding of text. Recently, pre-trained language models based on Transformer (Vaswani et al. 2017) such GPT (Radford al., 2018). BERT (Devlin et al. 2019), XLNet (Yang et al. 2011)) and RoBERTa eLiu al. 2019c) have been shown to be effective for learning contextualized language representation. These models have since continued to achieve new state-of-the-ait art results on a variety of natural language processing tasks. They include question answering (Rajpurkar et al. 2018 Lai et al., 2017), natural language inference (Williams) al. 2018 Bowman et 20.55) named entity recognition (Tjong Kim Sang & De Meulder 2003). sentiment analysis (Socher et al. 2013) and semantic textual similarity (Cer et al. 2017
Dolan &
Brockett 2005).
Normally, Transformer-basee models are pre-trained on large-scale unlabeled corpus in an unsuper- vised manner, and then fine-tuned on the downstream tasks through introducing task-specific output layer. When fine-tuning on the supervised downstream tasks, the models pass directly the output of Transformer encoder's 's final layer, which is considered the contextualized representation of input text, to the task-specific layer.
However, due to the numerous layers (i.e., Transformer blocks) and considerable depth of these pre- trained models, we argue that the output of the last layer may not always be the best representation of the input text during the fine-tuning for downstream tasks. Devlin et al. (2019) shows diverse combinations of different layers' outputs of the pre-trained BERT result in distinct performance on CoNLL-2003 Named Entity Recognition (NER) task (Tjong Kim Sang & De Meulder 2003). Peters et al. (2118) points out for pre-trained language models, including Transformer, the most transferable contextualized representations of input text tend to occur in the middle layers, while the top layers specialize for language modeling. Therefore, the onefold use of the last layer's output may restrict the power the pre-trained representation.
In this paper, propose an extra network component design for Transformer-baseee model, which is capable of adaptively leveraging the hidden information in the Transformer's 's hidden layers to refine the language representation. Our introduced additional components include two main additional components:
HIdden Representation Extractor (HIRE) dynamically learns a complementary representa- tion which contains the information that the final layer's output fails to capture.

---

Under review as a conference paper at ICLR 2021
2. Fusion network integrates the hidden information extracted by the HIRE with Transformer final layer's 's output through two steps of functionalities, leading to refined contextualized language representation.
Taking advantage of the robustness of RoBERTa by using it as our backbone teatsfforer--basee encoder (Liu et al., 2019c), we conduct experiments on GLUE benchmark (Wang al., 2018). which consists of nine Natural Language Understanding (NLU) tasks. With the help of HIRE, our model outperforms the baseline on 5/9 of them and advances the state-of-the-art on SST-2 dataset. Keeping the backbone Transformer model unchanged its architecture, pre-training procedure and training objectives, we get comparable performance with other state-of-the-ar models on the GLUE leaderboard, which verifies the effectiveness of the proposed HIRE enhancement over Transformer model.
MODEL
Tedeopecticc oulpat laver
2layer BI-GRU
Tusien Network
Concaltration
HIRE
Mathhul
Layer An
...
Ennder
Liner
Exporit
God
stand
Figure 1: Architecture of our model. HIRE denotes HIdden Representation Extractor, which the Bi-GRUs share the same parameters for each layer's 's output.
2.1 TRANSFORMER-BASEI ENCODER
Transformer- basedd encoder responsible for encoding input text into contextualized representation. Let [wi, ..., ) represent sequence of n words of input text, Transformer-bace encoder encodes the input sequence into universal contextualized representation Raxd,
R Encoder( (wa,...,, w,,)
where d is the hidden size of the encoder and R is the output of Transformer-based encoder's last layer which has the same length as the input text. We call it preliminary representation in this paper to distinguish it with the one that we introduce in section 2.2 Here, we omit a rather extensive formulations of Transformer and refer readers to Vaswani et al. (017). Radford et (2018) and Devlin et al. (019) for more details.
2.2 HIDDEN REPRESENTATION EXTRACTOR
Transformer-based encoder normally has many structure-identicn layers stacked together, for ex- ample, BERT and XLNNetARRGE all contain 24 layers of the identical structure, either outputs
2

---

Under review as a conference paper at ICLR 2021
from these hidden layers the last layer, but not only limited to the latter, may be extremely help- ful for specific downstream task. To make full use of the representations from these hidden layers, introduce extra component attached to the original encoder, HIdden Representation Extractor (HIRE) to capture the complementary information that the output of the last layer fails to capture. Since each layer does take the same importance represent certain input sequence for different downstream tasks, we design an adaptive mechanism that can compute the importance dynamically. We measure the importance by importance score.
The input to the HIRE is (Ho....,HH) Hj...., Hi} where represents the number of layers in the encoder. Here Ho is the initial embedding of input text, which the input of the encoder's first layer but is updated during training and H, IR nxd is the hidden-state of the encoder at the output of layer j. For the sake of simplicity, we call them hidden-state afterwards.
We use the same 2-layer Bidirectional Gated Recurrent Unit (GRU) (Cho et al. 2014) to summarize each hidden-state of the encoder. Instead of taking the whole output of GRU as the representation of the hidden state, concatenate GRU's each layer and each direction's 's final state together. In this way, we manage to summarize the hidden-state into a fixed-sized vector. Hence, we obtain U R (++ xdd with u, the summarized vector of H,:
u, Bi-GRU R
(2)
where 0 < sisl. I. Then the importance value o; for hidden-state H, is calculated by:
0; ReLUUW u, + b) R
(3)
where W E R and b E IR are trainable parameters. Let a=(=)) be normalized into a probability distribution s through a softmax layer:
softmax E R
(4)
where 8; is the normalized weight of hidden-state when computing the representation. Subse- quently, obtain the input sequence's new representation A by:
A H. Raxd
With the same shape the output of Transformer-bes bbssd encoder's final layer, HIRE's output A is expected to contain the additional useful information from the encoder's hidden-states and we call it complementary representation.
2.3 FUSION NETWORK
This module fuses the information contained in the output of Transformer-basee encoder and the one extracted from encoders' hidden states by HIRE.
Given the preliminary representation R, instead of letting it flow directly into task-specfic output layer, we combine together with the complementary representation A yield M, defined by:
M = A; R A; R A] E Rnx4d
(6)
where o is element-wise multiplication (Hadamard Product) and [:] concatenation across the last dimension.
Later, two-layer bidirectional GRU, with the output size of for each direction, is used to fully fuse the information contained in the preliminary representation and the complementary representation. We concatenate the outputs of the GRUs in two dimensions together for the final contextualized representation:
F = Bi-GRU(M) E Rdx2d
2.4 OUTPUT LAYER
The output layer is task-specific. The following are the concerned implementation details on two tasks, classification and regression.
3

---

Under review as a conference paper at ICLR 2021
For classification task. given the input text's contextualized representation F, following Devlin et al. (2019), we take the first row с E I of corresponding to the first input token (< 8 >) as the aggregated representation. Let m be the number of labels in the datasets, pass through feed-forward network (FFN):
q W tanhhW c +1) + b2 E R"
with W1 e Rddxd W2 Ráxm, bj R2 and b2 R the only parameters that we introduce in output layer. Finally, the probability distribution of predicted label is computed as:
softmax(q)
R
For regression task, we obtain y the same manner with m 1, and take y as the predicted value.
2.5 TRAINING
For classification task, the training loss to be minimized is defined by the Cross- Cross-Entropy:
L( - - Σ log(pr.))
(10)
where 0 is the of all parameters in the model, T is the number of examples in the dataset, Pi,e the predicted probability of gold class e for example i.
For regression task, define the training loss by squared error (MSE):
T Σ
L(
q. 9²)
(11)
where yi the predicted value for example and yi is the ground truth value for example i.
EXPERIMENTS
3.1 DATASET
We conducted the experiments on the General Language Understanding Evaluation (GLUE) bench- mark (Wang et al., 2018) to evaluate our proposed method. GLUE collection of 9 diverse datasets for training, evaluating, and analyzing natural language understanding models.
3.2
MAIN RESULTS
Single Sentence CoLA
Similarity and Paraphrase
Natural Language Inference MNLI-m/mm
Model
SST-2
MRPC
QQP
STS-B
QNLI
RTE
(Mc)
(Acc)
(Acc)
(Acc)
(Pearson)
(Ace)
(Acc)
(Acc)
MT-DNN
63.5
94.3
87.5
91.9
90.7
87.1/86.7
92.9
83.4
XLNET-
69.0
97.0
90.8
92.3
92.5
90.8/90.8
94.9
85.9
ALBERT-XXlarge (1.5M)
71.4
96.9
90.9
92.2
93.0
90.8/-
95.3
89.2
RoBERTa-large
68.0
96.4
90.9
92.2
92.4
90.2/90.2
94.7
86.6
RoBERTa+HIRE
69.7
96.8
90.9
92.0
92.4
90.7/90.4
95.0
86.6
(+1.7)
(+0.4)
(-0.2)
(+0.5/+0.2)
(+0.3)
Table 1: GLUE Dev results. Our results based on single model trained with single task and a median over five runs with different random seed but the same hyperparameter reported for each task. The results of MT-DNN, XLNEETARGEE. ALBERT and RoBERTa from Liu et al. (2019)). Yang et al.] (2019). Lan et al.] (2020) and Liu et al. (2019c). See the lower-most row for the performance of our approach. Me, acc and pearson denote Matthews correlation, accuracy and Person correlation coefficient respectively.
'lll the datasets he obtained from httppss///guubeeemmmmrrrcco////
4

---

Under review as a conference paper at ICLR 2021
Model
Params (M)
Shared (M)
ALBERT-xxlarge
235
MT-DNN
350
340
RoBERTa
355
RoBERTa+HIRE
437
Table 2: Parameter Comparison.
Table compares our method with list of Transformer-based models on the development set. Model parameter comparison shown in Table 2 To obtain direct and fair comparison with our baseline model RoBERTa, following the original paper (Liu et al. (019c). we fine-tune RoBERTa+HIRE separately for each of the GLUE tasks, using only task-specific training data. The single-model results for each task are reported. We run our model with five different random seeds but the same hyperparameters and take the median value. Due to the problematic nature of WNLI dataset, we exclude its results in this table. The results shows that RoBERTa+HIRE consistently outperforms RoBERTa on of the GLUE task development sets, with an improvement of 1.7 points, 0.4 points, 0.5/0.2 points, 0.3 points CoLA, SST-2, MNLI and QNLI respectively. And on the MRPC, STS-B and RTE task, our model get the same result as RoBERTa. It should be noted that the improvement is entirely attributed to the introduction of HIdden Representation Extractor and fusion network in our model.
Single Senfence CoLA
Similarity and Paraphrase
Natural Language Inference
Model
SST-2
MRPC
QQP
STS-B
MNLI-m/mm
QNLI
RTE
WNLI
Score
8.5k
67k
3.7k
364k
7k
393k
108k
2.5k
634
BERT
60.5
94.9
89.3/85.4
72. 1/81/89.3
87.6/86.5
86.7/85.9
92.7
70.1
65.1
80.5
MT-DNN
68.4
96.5
92.7/90.3
73.7/89.9
91.1/90.7
87.9/87.4
96.0
89.0
B-RoBETTT
68.0
96.8
93.1/90.8
74.8/90.3
92.3/92.1
91.1/90.7
95.6
88.7
89.0
88.4
XI. Nart
70.2
97.1
92.9/90.5
74.7/90.4
93.0/92.6
90.9/90.9
88.5
92.5
ALBERT
69.1
97.1
93.4/91.2
74.2/90.5
92.5/92.0
91.3/91.0
89.2
91.8
RoBERTa/
67.8
96.7
92.3/89.8
74.3/90.2
92.2/91.9
90.8/90.2
95.4
88.2
89.0
88.1
RoBERTa+HIRE
68.6
97.1
93.0/90.7
74.3/90.2
92.4/92.0
90.7/90.4
95.5
87.9
89.0
(+0.8)
(+0.4)
(+0.7/40.9)
(+0,2140.1)
(-0.1140.2)
(+0.1)
(-0.3)
(+0.2)
Table 3: GLUE Test results, scored by the official evaluation server. All the results are obtained from radeerbbrr)) The number below each task's s name indicates the size of training dataset. Recently the GLUE benchmark has forbidden all the submissions to treat the QNLI as a ranking task, which results in the missing some models'
GLUE leaderboard (https://g webennchaaaae
com/
accuracies the ONLI Devlin et al. 2201); Liu et al. (2019b);
'Zhu et al. (0020); Yang et al.
2019
Lan et al. 2020
Liu et al. (2019c).
the results of HIRE enhancement and other models on the test set that have been sub- mitted the GLUE leaderboard. Following Liu al. (2019c), finettue STS-B and MRPC start- ing from the MNLI single-task model. Given the simplicity between RTE, WNLI and MNLI, and the large-scale nature of MNLI dataset (393k), we also initialize ROBERTa+HIRE with the weights of MNLI single-task model before fine-tuning on RTE and WNLI. We submitted the ensemble-model results to the leaderboard. The results show that HOBERTa+HIRE still boosts the strong RoBERTa baseline model on the test set. To be specific, ROBERTa+HIRE outperforms RoBERTa over CoLA, SST-2, MRPC, SST-B, MNLI-mm and QNLI with an improvement of 0.8 points, 0.4 points, 0.7/0.9 points, 0.2/0.1 points, 0.2 points and 0.1 points respectively. In the meantime, RoBERTa+HIRE gets the same results as RoBERTa on QQP and WNLL By category, ROBERTa+HIRE has better performance than RoBERTa on the single sentence tasks, similarity and paraphrase tasks. It is worth noting that our model obtains maae-offthee-al results on SST-2 dataset, with a score 97.1. The re- sults are quite promising since HIRE does not modify the encoder internal architecture (Yang al. 2019) or redefine the pre-training procedure (Liu et al. 2019c). getting the comparable results with them.
5

---

Under review as a conference paper at ICLR 2021
4 ABLATION STUDY
In this section, we perform a set of ablation experiments to understand the effects of our proposed techniques during fine-tuning, All the results reported in this section are a median of five random runs.
4.1 MODEL DESIGN CONSIDERATION
CoLA (Mc)
STS-B (Pearson)
Model
Our model
69.7
92.4
w/o HIRE
68.5
91.5
w/o Fusion network
68.2
92.2
Table 4: Ablation study over model design consideration on the development set of CoLA and STS- B. The result for each model median of five random runs. Both HIRE and fusion network significantly improve the model performance on all two datasets.
To individually evaluate the importance of HIRE and fusion network, we vary our model in the following ways and conduct the experiments on the development set of CoLA and STS-B. The results are Table
We remove the HIRE from model and pass the preliminary representation R directly into the fusion network. In order to keep the fusion network, we fuse the preliminary representation with itself, which means define instead M by:
M = R. R: R + Ro R E Raadd
(12)
The results are presented in row 2.
We remove the fusion network, and take the outputs of HIRE as the final representation of the input and flow it directly into the output layer and present the results in row 3.
As can be seen from the table, to use HIRE to extract the hidden information is crucial to the model: Matthews correlation of CoLA and Pearson correlation coefficient of STS-B drop dramat- ically by 1.2 and 0.9 points if it's removed. We also observe that fusion network an important component that contributes 1. 1.5/0.2 gains the CoLA and STS-B tasks. The results are aligned with our assumption that HIRE extracts the complementary information from the hidden layers while the fusion network fuses it with the preliminary representation.
4.2 EFFECT OF DYNAMIC MECHANISM
Method
Mc
HIRE (dynamic)
69.7
mean, 25 layers
68.9
(-0.8)
mean, last layers
68.0
(-1.7)
mean, first 6 layers
68.8
(-0.9)
random, all 25 layers
69.3
(-0.4)
random, last layers
68.1
(-1.6)
Table Effect dynamic mechanism when computing the importance scores. A median Matthews correlation of five random runs is reported for CoLA the development set.
We investigate how the adaptive assignment mechanism the importance score affects the model's performance. CoLAA is chosen the downstream task. Table 5 compares our proposed mechanism

---

Under review as a conference paper at ICLR 2021
Number
Mc
68.5
69.3
69.7
68.5
Table Ablation study over GRU layer number in fusion network on the development set CoLA. The results median Matthews correlation of five random runs. The best result is in bold.
with diverse fixed combinations of importance scores different range of layers: first 6, last 6 or all 25 layers. Two strategies are studied: mean and random. In the mean situation, we suppose all the layers contribute exactly the same. On the contrary, under the random condition, generate score for each layer randomly and do the SoftMax operation across all the layers.
From the table, observe that fixing each layer's importance score for all examples hurts the model performance, Across all rows, we find that none of single strategy can yield the best performance. The results enable to conclude that the hidden-state weighting mechanism introduced by HIRE may indeed adapt to diverse downstream tasks for better performance.
4.3 EFFECT OF GRU LAYER NUMBER IN FUSION NETWORK
To investigate the effect of GRU layer number in the fusion network, we set the GRU layer number from to 3 and conduct the ablation experiments on the development dataset of CoLA. Table shows the results. We observe that the modest number 2 would be a better choice.
ANALYSIS
357-2
040
D0P
18114
0.16
000
CHU
Figure 2: Left: Distribution of importance scores over different layers when computing the com- plementary representation for various NLU tasks. The numbers on the abscissa axis indicate the corresponding layer with () being the first layer and 24 being the last layer. Right: Distribution of importance scores over different layers for each example of SST-2 dataset. The number on the ordinate axis denotes the index of the example.
We compare the importance score's distribution of different NLU tasks. For each task, we run our best single model the development and the results calculated by averaging the values across all the examples within each dataset. The results are presented in the left of Figure From the top the bottom of the heatmap, the results are placed in the following order: single-sentence tasks, similarity and paraphrase tasks, and natural language inference tasks. From it, we find that the distribution differs among the different tasks, which demonstrates HIRE's dynamic ability to adapt for distinct tasks when computing the complimentary representation. The most important contribu- tion occurs below the final layer for all the tasks except STS-B and RTE, All layers have a close contribution for STS-B and RTE task. In addition, we find that QNLI, classification task which requires the model to determine whether the context sentence contains the answer to the question, relies on layers 2-4 to phrase-level (or span-level) information in the lower layers and that this information gets gradually diluted in higher layers. Since QNLI is derived from the span extraction question answering dataset
extent. In fact, Jawahar al. (2019) state that BERT mostly captures

---

Under review as a conference paper at ICLR 2021
SQuAD, where the answer is span of the input text, we speculate that HIRE takes advantage of this information when making the prediction.
The right of Figure 2 presents the distribution of importance scores over different layers for each example of SST-2 dataset. The number on the ordinate axis denotes the index of the example. It shows that HIRE can adapt not only distinct tasks but also the different examples. In the meantime, we observe also that even though there are subtle differences among these examples, they follow certain same patterns when calculating the complementary representation, for example, layers 21 and 22 contribute the most for almost all the examples and also the layers around them. But the figure shows also that for some examples, all layers contribute nearly equally.
RELATED WORK
Transformer model is empowered by self-attention mechanism and has been applied as an effec- tive model architecture design in quite lot of pre-trained language models (Vaswani et al. (017). OpenAl GPT (Radford et al. 2018) is the first model that introduced Transformer architecture into unsupervised pre-training- But instead of unidirectional training like GPT, BERT (Devlin et al.) 2019) adopts Masked LM objective when pre-training, which enables the representation to incor- porate context from both direction. The next sentence prediction (NSP) objective is also used by BERT to better model the relationship between sentences. Trained with dynamic masking, large mini-batches and a larger byte-level BPE, full-sentences without NSP, RoBERTa (Liu et al., 2019c) improves BERT's performance on the downstream tasks from a better BERT re-implementation training on BooksCorpus(Zhu et al. 2013), CC-News, Openwebtext and Stories. terms of fine- tuning downstream tasks, all these powerful Transformer- based models help various NLP tasks continuously achieve new saae-of-the-ane results. Diverse new methods have been proposed recently for fine-tuning the downstream tasks, including multi-task learning (Liu et al. 2019b), adversar- ial training (Zhu et al. 2020) or incorporating semantic information into language representation ZZang et al. (020))
Traditionally, downstream tasks or back-end part of models take representations from the last layer of the pre-trained language models as the default input. However, recent studies trigger researchers' interest on the intermediate layers' representation learned by pre-trained models. When studying the linguistic knowledge and transfcrability of contextualized word representations with a series of seventeen diverse probing tasks, Liu et al. (2019a) observe that Transformers tend to encode transferable features in their intermediate layers, aligned with the results by |Peters et al. (0188)) Jawahar et al. (2019) show rich hierarchy of linguistic information is encoded by BERT, with surface features, syntactic features and semantic features lying from bottom to top in the layers. So far, to our best knowledge, exploiting the representations learned by hidden layers is still stuck with limited empirical observations, which motivates us to propose a general solution for fully exploiting all levels of representations learned by the pre-trained language models. However, want to note that a freezing/fine srrzzinn/iinettninng discrepancy may exist, since previous observations are made by adopting feature extracting approach while our work is conducted in the fine-tuning scenario.
Several works have made attempts to utilize the information from the intermediate layers of deep models (Zhao al. 2013 Peters et al. 2018 Sa Zhu et al. 2018 Tenney al. 2019). Peters et al. 2018a) propose a deep contextualized word representation called ELMo, which is a task-specific linear combination of the intermediate layer representations in the bidirectional LSTM language model. [Tenney et al.] (2019) find that using ELMo-style scalar mixing of layer activations, both deep Transformer models (BERT and GPT) gain significant performance improvement on a novel edge probing task. Zhu et al. (0118) adopt a linear combination of embeddings from different layers in BERT encode tokens for conversational question answering but in a feature-basee way. Compared with these works, our approach still presents the following innovations: 1) Our dynamic layer weighting mechanism calculates importance scores for each example over different layers, which is distinct from previous works (e.g., ELMo), where the contribution score (in the form of parameter) for each layer is always same for all the examples of the same task. 2) Our approach fuses the representation weighted from the hidden layers and the one from the last layer to get refined representation. 3) To our best knowledge, our work is the first one that takes advantage of the pre-trained language model's intermediate information in the context of fine-tuning
8

---

Under review as a conference paper at ICLR 2021
7 CONCLUSION
This paper presents HIdden Representation Extractor (HIRE), novel enhancement component that refines language representation by adaptively leveraging the Transformer- based model's hidden lay- ers. In our proposed model design, HIRE dynamically generates complementary representation from all hidden layers other than that from the default last layer. lite fusion network then incorpo- rates the outputs of HIRE into those of the original model. The experimental results demonstrate the effectiveness of refined language representation for natural language understanding. The analysis highlights the distinct contribution of cach layer's output for diverse tasks and different examples.
REFERENCES
Luisa Bentivogli, Ido Dagan, Hoa Trang Dang, Danilo Giampiccolo, and Bernardo Magnini. The fifth PASCAL recognizing textual entailment challenge. In Proceedings of Text Analysis Confer- ence (TAC 2009), 2009.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 632-642, September 2015.
Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 1-14, August 2017.
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder- for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724-1734, October 2014.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The PASCAL recognising textual entailment challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pp. 177-190, 2006. ISBN 978-3-540-33428- 6.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), Volume 1 (Long and Short Papers), pp. 011444111188 June 2019.
William B. Dolan and Chris Brockett. Automatically constructing corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005.
Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. The third PASCAL recog- nizing textual entailment challenge. In Proceedings of the ACI-PASCAL Workshop on Textual Entailment and Paraphrasing, pp- 1-9, June 2007.
Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. What does BERT learn about the structure of language? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACI.), pp. 3651 136511 July 2019.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Interna- tional Conference on Learning Representations (ICLR), may 2015.
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 785-794, September 2017.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Sori- cut. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference Learning Representations (ICLR), 2020.

---

Under review as a conference paper at ICLR 2021
Hector J. Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. In Pro- ceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning, KR 12, pp. 552-561, 2012. ISBN 978-1-57735-5600
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. Lin- guistic knowledge and transferability of contextual representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Hu- man Language Technologies (NAACL), Volume (Long and Short Papers), pp. 1073-1094, June 2019a.
Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. Multi-task deep neural networks for natural language understanding. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 4487-4496 July 2019b.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized bert pretraining 2019c.
approach. arXiv preprint 1xXiv:997.11666 2019c.
B.W. Matthews.
Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA) Protein Structure, 405(2):442 451, 1975. ISSN 0005-2795.
Matthew Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Proceedings of the 2018 Con- ference of the North American Chapter the Association for Computational Linguisties; Human Language Technologies (NAACL.), Volume (Long Pupers), pp. 2227-2237, June 2018a.
Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of the 2018 Conference on Em- pirical Methods Natural Language Processing (EMNLP), pp. 1509, Noveeber 2018b.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language under- standing by generative tre-training. Technical report, OpenAI, 2018. URL/ttpps: //openai./// com/ htmm/blog/ http://oogllgggggeeeeaaaa www..aagggguuuuurrrrrraeeeeeeeee
Alec Radford, Jeffrey Wu, Dario Amodei, Daniela Amodei, Jack Clark, Miles Brundage, and Ilya Sutskever. Better language models and their implications. Technical report, OpenAI, 2019. URL
httpss://ppen:: cww/coomoooooooooooooooocoooccoo og http:bbeteerrrre wtwpbeeeeeee-lagggg-e--e-eoeeeoe
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2383 2392, November 2016.
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't 't know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 2: Short Papers), pp. 784-789, July 2018.
BarHaim Roy, Dagan Ido, Dolan Bill, Ferro Lisa, Giampiccolo Danilo, Magnini Bernardo, and Szpektor Idan. The second recognising textual entailment challenge. In In P Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, 2006.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1631-1642, October 2013.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research (JMLR), 15:1999-1958, 2014.
10

---

Under review as a conference paper at ICLR 2021
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das, and Ellie Pavlick. What do you learn from context? Probing for sentence structure in contextualized word representations. In 7th International Conference on Learning Representations (ICI.R,, may 2019.
Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the CoNLL-2003 shared task: Language-independee named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp. 142-147, 2003.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser. and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30 (NIPS), pp- 5998-6008. 2017.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353-355, November 2018.
Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. Neural network acceptability judgments. arXiv preprint arXiv: 1805. 2477, 2018.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad croad-coverage challenge corpus for sen- tence understanding through inference. Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), Volume (Long Papers), pp. 1112-1122, June 2018.
Zhilin Yang. Zihang Dai, Yiming Yang. Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. XLNet: Generalized autoregressive pretraining for language understanding, Advances in Neu- ral Information Processing Systems 32 (NIPS), pp. 5754-5764. 2019.
Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang. Xi Zhou, and Xiang Zhou. Sewantics-aware bert for language understanding. In Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-2020), 2020.
Han Zhao, Zhengdong Lu, and Pascal Poupart. Self-Adaptive Hierarchical Sentence Model. In Proceedings of the 24th International Joint Conference on Artificial Intelligence, pp. 4069-4076,
2015.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. FreeLB: Enhanced Ad- versarial Training for Natural Language Understanding. International Conference on Learning Representations (ICLR), 2020.
Chenguang Zhu, Michael Zeng, and Xuedong Huang. SDNet: Contextulized attention-based deep 2018.
network for conversational question answering. arXiv preprint
Y. Zhu, R. Kiros, R. Zemel, R. Salakhutdinov, R. Urtasun, A. Torralba, and S. Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In 2015 IEEE International Conference Computer Vision (ICCV), pp. 19-27. Dec 2015.
A APPENDIX
A.1 DETAILED DESCRIPTIONS FOR THE GLUE BENCHMARK
Three different tasks are presented in GLUE benchmark (Wang et al. 2018) according to the original paper:
Single-sentence tasks:
The Corpus of Linguistic Acceptability (CoLA) Warstadt et al., 2018 requires the model to determine whether sentence is grammatically acceptable; the Stanford Senti- ment Treebank (SST-2) (Socher et al. 2013) is to predict the sentiment of movie reviews with label of positive negative.
11

---

Under review as a conference paper at ICLR 2021
Similarity and paraphrase tasks: Similarity and paraphrase tasks to predict whether each pair of sentences captures a paraphrase/semantic equivalence relationship. The Microsoft Research Paraphrase Corpus (MRPC) (Dolan & Brockett 2005). the Quora Question Pairs (QQP) and the Semantic Textual Similarity Benchmark (STS-BB (Cer et al. 2017 are presented in this category.
Natural Language Inference (NLI) tasks: Natural language inference is the task of determining whether a "hypothesis" is true (entailment), false (contradiction), or undetermined (neutral) given "premise". GLUE benchmark contains the following tasks: the Multi-Genre Natural Language Inference Corpus (MNLI) (Williams et al., (018), the converted version of the Stanford Question Answering Dataset (QNLI) (Rajpurkar et 2016). the Recognizing Textual Entailment (RTE) (Dagan et al. 2006) Roy et al. (2066] Giampiccolo et al.] [2007] Benttvogli et al. 2009) and the Winograd Schema Challenge (WNLII) Levesque et 20122.
Four official metrics are adopted to evaluate the model performance: Matthews correlation (Mathhews 1975). accuracy, Fl score, Pearson and Spearman correlation coefficients.
A.2 IMPLEMENTATION DETAILS
Our implementation of HIRE and its related fusion network is based on the PyTorch implementation of Transformer
Following Liu et al. (019c), adopt GPT-2 (Radford et al. 2019) tokenizer with a Byte-Pair Encoding (BPE) vocabulary of subword units size 50K. The maximum length input sequence is 128 tokens.
Preprocessing:
We use ROBERTaLARGE as the cansformer-based encoder and load the pre- training weights of RoBERTa (Liu al. 2019c). Like BERT LARGe, RoBERTajARGGE model contains 24 Transformer-block, with the hidden size being 1024 and the number of self-attention heads being 16 (Liu et al. 2019c Devlin et al. 2019).
Model configurations:
Optimization: We use Adam optimizer (Kingma & Ba 2015) with 31 = 0.9, B2 = 0.98 and 10 and the learning rate selected amongst [1--5 2c-5} with a warmup rate of 0.06 depending on the nature of the task. The number of training epochs ranges from 3 to 27 with the early stop and the batch size selected amongst {16, 32, 48). addition to that, we clip the gradient norm within 1 to prevent exploding gradients problem occurring in the recurrent neural networks in our model.
Regularization: We employ two types of regularization methods during training. We apply dropout (Srivastava et al., 2014) of rate 0.1 to all layers in the Transformer-basse encoder and GRUs in the HIRE and fusion network. additionally adopt L2 weight decay of 0. during training.
htttps:/www.
12


Reviews:
[{"review_id": "srwFlBgZ_G3", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a new mechanism, called HIRE, to extract more information from the intermediate layers of pre-trained models, which will be further fused with the last layer of pre-trained models. The main contribution of this work is the newly proposed dynamic feature extractor HIRE and the fusion network. Experiments confirmed the effectiveness of the proposed method, and some interesting observations on the importance of different layers for different tasks were given (i.e. Figure 2). \n\nHowever, the proposed HIRE and Fusion modules can be viewed as the combination of some widely used deep learning mechanisms (e.g. Bi-GRU, softmax), using Bi-GRE to represent a sequence is a widely adopted choice, so the contribution in modeling is quite limited. Some existing works, like Self-Adaptive Hierarchical Sentence Model https://arxiv.org/pdf/1504.05070.pdf, have already proposed similar ideas to use more information from the intermediate layers of deep models. \n\nThe paper is well-written and organized. I have several concerns:\n1. All the models are evaluated on the GLUE dataset, experiments on more challenging tasks like QA (e.g. SQuAD 1.1/2.0) should be added. It would be interesting to visualize the importance scores of each layer for this challenging task. \n\n2. As illustrated in Figure 2, different downstream task highly depends on different intermediate layers, the authors are suggested to conduct experiments under the multi-task setting, since the HIRE can adaptively select the intermediate layers, the proposed method should have a big advantage for this setting. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "ik-lwhK9Or1", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method to improve the downstream performance of a pretrained Transformer on NLP tasks. The core idea is to not only use the output of the last Transformer layer for prediction, but let the model decide how to fuse the information from intermediate layers as well. To dynamically decide which intermediate layers to use depending on the input example, the model uses a mechanism conceptually similar to self-attention, which yields a normalized importance score for each layer. The importance-weighted sum then yields a complementary representation to the last layer. Lastly, another network produces a final, integrated representation from the output at the last layer and the complementary representation, which is then used for prediction.\nThe model is evaluated on the GLUE benchmark.\n\nStrengths:\nThe paper is well written and the experimental evaluation seems correct. The paper has a nice ablation study which shows that the learned importance scores, the complementary representations, and the fusion network are needed to reach the model's full performance.\n\nWeaknesses:\nThe main weakness is that the proposed extension to the baseline is relatively complex and rather heavy-weight in terms of new parameters (introducing ~25% more parameters compared to the baseline according to Table 2), yet only achieves a very marginal relative improvement of 0.2 percent over the baseline. It seems likely that this improvement could be achieved through much simpler means, e.g., additional self-attention layers on top of the last pretrained layer. This is supported by the fact that the paper's analysis of the proposed importance score mechanism doesn't show comprehensible patterns (or the paper doesn't talk about it).\n\nI lean slightly towards rejection, because, although the proposed model is reasonable and could have potential, the experiments do not currently demonstrate that potential, and I hence expect it hard for the community to learn from this paper.\n\nQuestions:\n\n* You cite several papers which demonstrate that different layers of pretrained Transformers encode different information, which is the motivation for your architecture. However, the cited sources use a feature extracting approach, i.e., they don't fine tune the encoder on their target task. In the finetuning scenario (yours), can't we expect the model to learn to simply forward the relevant information from intermediate layers to the last layer?\n\n* In Section 5 you analyze which intermediate layers are used for which task. You state that we can see that the model learns to prefer different layers for each task, but don't go into detail. Can you relate the nature of the task to the layers it produces? For example, why does QNLI rely on layers 2-4 to some extent despite being a rather high-level understanding task?\n\n* In Figure 2, right side you observe that different examples tend to different layers. Did you qualitatively inspect the examples and try to find a pattern/link to their preferred intermediate layers?\n\nSuggestions:\n\n* RoBERTa already achieves super-human results on the GLUE benchmark, meaning that it is probably already close to some upper limit. This makes it even more difficult for your model to substantially improve upon it. I would consider evaluating it on the SuperGLUE benchmark instead, since there RoBERTa is not so close to human performance yet.\n\n* Your model is designed to make use of representations at lower layers if a task requires it. But all the tasks in your experiments are evaluated on high-level natural language understanding tasks, which typically require representations at higher layers. This makes it more likely that your model does not improve much over the baseline, because the last layer will arguably already contain much of the information needed for the tasks. I think you should consider different tasks in your experiments to increase the chance of significant improvements.\n\n* In the current state, it is unclear whether your improvements are coming from the specific model you chose or merely from the fact that you added a lot of parameters. As a control, you could include an experiment where you simply add several self-attention layers on top of RoBERTa such that you match the number of parameters of your model.", "extra_scores": {}, "extra_text": {}}, {"review_id": "UHtN1YWqVb", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n### Summary \nIn this paper, the authors propose a fine-tuning scheme for pre-trained language models, called Hidden Representation Extractor (HIRE). The proposed method consists of two components.\n1) Hidden representation extractor (HIRE): a two-layer bi-GRU model summarizes the hidden representation of the encoder with a weighted-sum scheme.\n2) Fusion network: combine the weighed representation with original representation via the bi-GRU model.\nThe authors apply the proposed approach on top of RoBERTa and evaluated on general benchmark GLUE. The experimental results show that the proposed method could give moderate improvements over RoBERTa baseline system.\n\nThe authors' main contribution lies in two components: HIRE and fusion network. However, unfortunately, both components are not novel to me.\n1) The idea of HIRE model is very similar to utilizing ELMo in down-stream tasks, where the hidden representations of each layer are weighted.\n2) The central part of the fusion network utilizes a concatenation scheme of [A;B;A+B;A*B], which is also a widely used approach, similar to Chen et al. (2017), Seo et al. (2017), Hu et al. (2019), etc.\n\nBesides the novelty of the proposed components, the experiments are also not well-designed. I assume the proposed method could apply to various pre-trained language models (PLMs), as this is an approach that aims to apply on the fine-tuning stage. However, the authors only present the results on top of RoBERTa model, which cannot demonstrate that the proposed model can be generally applied to various PLMs. I would suggest the authors evaluate their approach in a more general way, such as build on top of various PLMs, or apply on PLM in another language. These would have helped to better demonstrate the effectiveness of the proposed method.\n\nConsidering the novelty and generalizability of the proposed method, I recommend rejection for this paper.\n\n[1] Chen et al. Enhanced LSTM for Natural Language Inference.  \n[2] Seo et al. Bidirectional attention flow for machine comprehension.  \n[3] Hu et al. Read+ verify: Machine reading comprehension with unanswerable questions.  \n\n\n### PROs\n1. A possible useful framework for fine-tuning down-stream tasks for pre-trained language models.\n\n\n### CONs\n1. The design of the components is not novel.\n2. The experiments are not well-designed, and the results only show marginal improvements over RoBERTa.\n3. The generality of the proposed method is not well-studied, as the experiments are only performed on top of RoBERTa.\n\n\n### Questions\n1. Unlike Transformer-based models, RNN models (such as Bi-GRU) are not computationally efficient. How about your training time compared to the baselines? According to Table 2, using HIRE will increase the total parameter from 355 to 437, which will slow down the inference speed.\n2. The authors did not cite ELECTRA (Clark et al., 2020) for comparison. After comparing the results, the proposed model is not competitive against ELECTRA, while it would cost more training and inference time, which will weaken the impact of the paper. I strongly recommend the authors also carry out experiments on top of ELECTRA (or ALBERT-xxlarge, etc.) to see if your approach generalizes well on various PLMs.\n\n\n### Minor Reviews\nI did not go through every detail of the writing and only list a few of the issues here.\n1. page 1, section 1: CoNNL-2003 -> CoNLL-2003\n2. page 1, section 1: (Peter et al., 2018b) points out ... -> Peter et al. (2018b) points out ...\n3. page 3, section 2.3: Tansformer -> Transformer; yeild -> yield; elementwise -> element-wise;\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes  a new mechanism, called HIRE, to  improve the down-stream performance of a pre-trained Transformer on NLP tasks. Different from directly using the last layer of transformer, the proposed model allows the system to dynamically decide which intermediate layers to use based on the input through some sort of gating. The model is evaluated on GLUE, a benchmark for natural language understanding.  My major concerns are the following 
1.  the gating mechanism on using intermediate sentence representation is not new,  as pointed by some reviewers, although its implementation on transformers is still interesting.
2.  the empirical part is not convincing enough:  a) GLUE data set is relatively simple, the authors should try something more complex, b）the improvement over baseline is rather modest, which could be achieved with simpler modification.

I'd suggest to reject this paper.
Decision: Reject