Title: Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks
Authors: Zhilin Yang
Ruslan Salakhutdinov
William W. Cohen
Venue: ICLR 2017
Keywords: Natural language processing
Deep learning
Transfer Learning

Abstract: Recent papers have shown that neural networks obtain state-of-the-art performance on several different sequence tagging tasks. One appealing property of such systems is their generality, as excellent performance can be achieved with a unified architecture and without task-specific feature engineering.  However, it is unclear if such systems can be used for tasks without large amounts of training data. In this paper we explore the problem of transfer learning for neural sequence taggers, where a source task with plentiful annotations (e.g., POS tagging on Penn Treebank) is used to improve performance on a target task with fewer available annotations (e.g., POS tagging for microblogs). We examine the effects of transfer learning for deep hierarchical recurrent networks across domains, applications, and languages, and show that significant improvement can often be obtained.  These improvements lead to improvements over the current state-of-the-art on several well-studied tasks.


Review discussion:
I am curious about what happens if the same task which could be implemented as T-A is treated as T-B or T-C. I'd expect the performance to fall, but unclear by how much.
We agree that this would be an interesting study. We will include corresponding experimental results in our later version.
As the training framework is important in this paper, can you elaborate on the training details?
How the task pair <s,t> is sampled given the binomial distribution? Do you sample <s,t> out of a pool of {<s,t>} or sample s and t separately out of {s} and {t} and then pair them? Are the representations shared across the pairs, e.g., the char/word embeddings that are the same for different pairs <s_1,t_1> and <s_2,t_2>?
What is the stop criterion? With many <s,t> pairs, certain pair <s_i,t_i> may converge earlier than others.
So how this issue is dealt?
Although the way of selecting T-A/T-B/T-C is reasonable, it will be interesting to show some results when one certain task which is best-suited for T-A is treated as T-B/T-C.
1.
Regarding the training framework.
The s and t in Section 3.3 refer to the indices of the source and target tasks. At each iteration, we sample either s or t from a set {s, t} using a binomial distribution.
For example, in most cases we have p(X=s)=0.5 and p(X=t)=0.5, where X is a binary random variable. If X=s, we sample a batch of instances from task s, and then do an update step. If X=t, we sample a batch of instances from task t, and then do an update step.
Representation sharing follows the illustration in Figure 1, where different architectures share different sets of parameters.
We do early stopping on the development set of the target task.
The source and target tasks indeed might have different convergence rates. This can be alleviated by tuning the binomial distribution mentioned above.
However, in most cases, we simply let p(X=s)=p(X=t)=0.5 and it works fine.
Sorry about the confusion, and we will clarify this in our later version.
2.
Regarding comparison between T-A/T-B/T-C on one task.
We agree that this would be an interesting study. We will include corresponding experimental results in our later version.
Did you check the variance that you obtain with different runs, in Table 3?
The variances for POS tagging, chunking, and NER are 1.7e-7, 1.0e-7, and 4.0e-7, respectively.
This paper presents a clear hierarchical taxonomy of transfer learning methods as applicable to sequence tagging problems. This contextualizes and unifies previous work on specific instances of this taxonomy.
Moreover, the paper shows that previously unexplored places in this taxonomy are competitive with or superior to the state of the art in key benchmark problems.
It'd be nice to see this explored further, such as highlighting what is the loss as you move from the more restrictive to the less restrictive transfer learning approaches, but I believe this paper is interesting and acceptable as-is.
Thank you for the comments. We have updated the paper to include the results of the study (applying different models to the same task pair).
Authors' response well answered my questions. Thanks!
Evaluation not changed.
This paper proposes a hierarchical framework of transfer learning for sequence tagging, which is expected to help the target task with the source task, by sharing as many levels of representation as possible. It is a general framework for various neural models. The paper has extensive and solid experiments, and the performance is competitive with the state of the art on multiple benchmark datasets. The framework is clear by itself, except that more details about training procedure, i.e. sec-3.3, need to be added.
The experimental results show that for some task pairs {s,t}, this framework can help low-resource target task t, and the improvement increases with more levels of representations can be shared.
Firstly, I suggest that the terms *source* and *target* should be more precisely defined in the current framework, because, due to Sec-3.3, the s and t in each pair are sort of interchangeable.
That is, either of them can be the *source* or *target* task, especially when p(X=s)=p(X=t)=0.5 is used in the task sampling. The difference is: one is low-resourced and the other is not.
Thus it could be thought of as multi-tasking between tasks with imbalanced resource.
So one question is: does this framework simultaneously help both tasks in the pair, by learning more generalizable representations for different domains/applications/languages? Or is it mostly likely to only help the low-resourced one? Does it come with sacrifice on the high-resourced side?
Secondly, as the paper shows that the low-resourced tasks are improved for the selected task pairs, it would also be interesting and helpful to know how often this could happen.
That is, when the tasks are randomly paired (one chosen from a low-resource pool and the other from a high resource pool), how often could this framework help the low-resourced one?
Moreover, the choice of T-A/T-B/T-C lies intuitively in how many levels of representation *could* be shared as possible. This implicitly assumes share more, help more.
Although I tend to believe so, it would be interesting to have some empirical comparison.
For example, one could perhaps select some cross-domain pair, and see if T-A > T-B > T-C on such pairs, as mentioned in the author’s answer to the pre-review question.
In general, I think this is a solid paper, and more exploration could be done in this direction.
So I tend to accept this paper.
Thank you for your feedback. We would like to respond to several questions raised by the reviewer.
Q1
The objective function is symmetric for both the source and target tasks, which can be viewed as a multi-task learning framework.
However, the training process is not symmetric as we perform early-stopping on the low-resourced task. It usually does not improve the performance on the high-resourced task by much (as can be seen in Figure 2 with 1.0 labeling rates), because when sufficient (or possibly infinite) labels are given, it is difficult to use auxiliary data from another task to improve the performance because of shifted distribution (e.g., covariate shift and concept drift).
Q2
We tried to address the question “when does transfer learning help” in the paper. As shown in Figure 2, Table 2, and analysis in Section 4.2, with task pairs systematically picked, we observe that the following factors are crucial for the performance of our transfer learning approach: a)  label abundance for the target task, b) relatedness between the source and target tasks, and c) the number of parameters that can be shared.
Our study covered a wide range of task pairs. Transfer learning helps for those task pairs, and the extent to which it helps depends on the above factors.
Q3
We have also updated the paper to include the results of the study (applying different models to the same task pair).
The authors propose transfer learning variants for neural-net-based models, applied to a bunch of NLP tagging tasks.
The field of multi-tasking is huge, and the approaches proposed here do not seem to be very novel in terms of machine learning: parts of a general architecture for NLP are shared, the amount of shared "layers" being dependent of the task of interest.
The novelty lies in the type of architecture which is used in the particular setup of NLP tagging tasks.
The experimental results show that the approach seems to work well when there is not much labeled data available (Figure 2). Table 3 show some limited improvement at full scale.
Figure 2 results are debatable though: it seems the authors fixed the architecture size while varying the amount of labeled data; it is very likely that tuning the architecture for each size would have led to better results.
Overall, while the paper reads well, the novelty seems a bit limited and the experimental section seems a bit disappointing.
Thank you for the comments. We would like to clarify the settings in Figure 2.
We believe your comments on Figure 2 are based on the assumption that in practice, the training data is fixed and architecture is tuned around that.  In transfer settings, that need not be so: in practice one can make choices about what additional (off-target) data to use as well as learning-method choices and of course new data can also be labeled, often at lower cost than for the target domain. A grid search over both data and architectural choices would be expensive, so we argue that the Figure 2 experiments are appropriate.
In particular, we fixed the hyper-parameters for different labeling rates so that we can study the performance gain of transfer learning. Since we use the same hyper-parameters for both transfer and non-transfer settings, we believe it is a fair comparison.
We further performed experiments where we fine-tuned the hyper-parameters for a specific labeling rate, and found that similar improvements can be obtained.
For example, with the optimal hyper-parameters (evaluated on the dev set), transfer learning from PTB to Genia with the T-B model leads to a performance gain of 0.06, similar to the results with default hyper-parameters as reported in the paper.
We study the effects of using T-A, T-B, and T-C when transferring from PTB to Genia. The results are included in the lower part of Table 2. It is clear that the performance gain decreases when less parameters are shared.

Meta-review:
role: PC
data: comment: One weak and one positive review without much concrete substance. The third review is positive, but the experiments are not that convincing: the gains from transfer are small in table 3 and in table 2 it is unclear how strong the baselines are. Given how competitive ICLR is, the area chair has no alternative than to unfortunately reject this paper.

Decision: Accept (Poster)