Title: A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
Authors: Kazuma Hashimoto
Caiming Xiong
Yoshimasa Tsuruoka
Richard Socher
Venue: ICLR 2017
Keywords: Natural language processing
Deep learning

Abstract: Transfer and multi-task learning have traditionally focused on either a single source-target pair or very few, similar tasks. Ideally, the linguistic levels of morphology, syntax and semantics would benefit each other by being trained in a single model. We introduce such a joint many-task model together with a strategy for successively growing its depth to solve increasingly complex tasks. All layers include shortcut connections to both word representations and lower-level task predictions. We use a simple regularization term to allow for optimizing all model weights to improve one task's loss without exhibiting catastrophic interference of the other tasks. Our single end-to-end trainable model obtains state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment. It also performs competitively on POS tagging. Our dependency parsing layer relies only on a single feed-forward pass and does not require a beam search.

TL;DR: A single deep multi-task learning model for five different NLP tasks.

Review discussion:
questions:
- without output label embeddings how do you connect two tasks?
- depth and shortcut connections effect relatedness and entailment tasks too much. how do you explain this? could this be related to using the same corpus (PTB) for all the previous tasks?
- for the first three tasks section splittings are not the same to compare with literature (I guess). Would not it be safer to use the same section split so that a gold-tag information would not flow one task to the other?
- table 12,13 reports higher numbers for chunkings. I guess those are word level accuracies not F1 scores. is this correct?
suggestions:
- related work would be more thorough if you include joint learning studies from NLP literature such as https://transacl.org/ojs/index.php/tacl/article/viewFile/412/81
- predicting the head and the dependency label does not guarantee producing well-formed dependency trees. did you try using a decoder like eisner's algorithm or maximum-spanning-tree algorithm?
Thank you very much for your questions and suggestions.
Let me respond to each of the questions.
>> - without output label embeddings how do you connect two tasks?
It is true that the tasks are not always directly connected.
For example, the POS information is not directly fed into the dependency parsing layer.
However, in our model, there are vertical connections in the multi-layer bi-LSTMs, as explained in the example of the input vector g^{(2)}_t in Section 2.3.
Therefore, all of the hidden states in the multi-layer bi-LSTMs are propagated to higher layers even when we do not use the output label embeddings.
>> - depth and shortcut connections effect relatedness and entailment tasks too much. how do you explain this? could this be related to using the same corpus (PTB) for all the previous tasks?
First, in the case of depth (Table 7), the size of training data is an important factor.
For example, the training data for dependency parsing includes 39,832 sentences, and a dependency annotation is assigned for each word in the sentences.
By contrast, the training data for the relatedness and entailment tasks includes only 4,500 sentences, and a gold annotation is assigned for each sentence.
Thus, the size of the training data for the relatedness and entailment tasks is much smaller than that for the other three tasks.
Our results in Table 7 show that such small datasets are not sufficient to train deep networks (4 layer bi-LSTMs for the relatedness task and 5 layer bi-LSTMs for the entailment task).
Second, in the case of shortcut connections (Table 9), the difference of the tasks is an important factor.
The first three tasks (POS tagging, chunking, and dependency parsing) are tasks primarily related to syntax, and semantic similarities between words are not always important in these tasks.
By contrast, capturing semantic similarities is very important in the semantic tasks (relatedness and entailment).
Without the shortcut connections, the word embeddings are only fed into the first (POS) layer, and the relatedness and entailment layers receive the word information modified by the first three layers.
Therefore, the semantic similarities captured by the word embeddings could be washed out before handling the two semantic tasks.
Please also refer to Appendix D to see how the word embeddings are modified according to each task.
We will add the above discussions to the manuscript.
>> - for the first three tasks section splittings are not the same to compare with literature (I guess). Would not it be safer to use the same section split so that a gold-tag information would not flow one task to the other?
As described in Section 5.1, we used the commonly-used data splits for POS tagging, chunking, and dependency parsing.
Therefore, some of the sections are used only in one task.
We will make this setting clearer in the next version.
>> - table 12,13 reports higher numbers for chunkings. I guess those are word level accuracies not F1 scores. is this correct?
They are indeed the F1 scores.
As explained in Footnote 5, the development and test sentences are included in the training data of dependency parsing, following the standard data splits.
Therefore, it is more like a situation where some of the training data is included in the test data, so the results of chunking cannot be compared with previously published results.
In contrast, the results of the Single and JMT_AB settings can be compared with previously reported results.
>> - related work would be more thorough if you include joint learning studies from NLP literature such as https://transacl.org/ojs/index.php/tacl/article/viewFile/412/81
Thank you for the pointer.
We will expand the related work section in the next version.
>> - predicting the head and the dependency label does not guarantee producing well-formed dependency trees. did you try using a decoder like eisner's algorithm or maximum-spanning-tree algorithm?
We did not try to make all of the dependency results become well-formed trees.
In our primary analysis on the dependency parsing results in Appendix B, we found 22 sentences with multiple root nodes or no root nodes out of the 1,700 development sentences.
The other possibility is the issue of cyclic graphs.
Thank you for the suggestion and we will try this direction.
1. From table 1, it seems training with high level tasks (jointly) will always hurt the low level tasks performance. Any explanation for that?
2. From table 1, training with only DE seems outperform training the JMT_all system. The POS/parsing information doesn't help the high level task?
3. The training/test sets in POS/parsing are different (but overlapped). Is it possible for parsing to get gold POS information indirectly in this training method?
Thank you very much for your questions, and let me respond to each of the questions.
>> 1. From table 1, it seems training with high level tasks (jointly) will always hurt the low level tasks performance. Any explanation for that?
First, the accuracy of POS tagging does not drop when the POS component is jointly trained with the high-level semantic tasks.
In the case of dependency parsing, the accuracy may seem to drop, but the difference is very small (UAS: 94.67 and 94.71, LAS: 92.90 and 92.92), and indeed there is no significant difference.
We expected that the higher-level tasks were helpful in improving the lower-level tasks, but currently, we did not observe positive results.
Our work is the first step towards handling many different types of NLP tasks in a single model, achieving competitive results with those of state-of-the-art models specifically tuned only for single tasks.
>> 2. From table 1, training with only DE seems outperform training the JMT_all system. The POS/parsing information doesn't help the high level task?
In the task of semantic relatedness, we used the Mean Squared Error (MSE) as the evaluation metric, and thus, lower numbers indicate better accuracy.
Therefore, the MSE score of JMT_all (0.233) is better than that of JMT_DE (0.238).
In the case of the task of textual entailment (a 3-class classification task), the difference of the scores 86.2 and 86.8 is small, and the score of JMT_all is comparable with the state-of-the-art results.
Related to that, we will describe the size of the training data for each task in the next version.
In our model, five different types of tasks are handled in the single model, and it is not obvious when to stop the training while trying to maximize the scores of all the tasks.
As the first step, we focused on maximizing the accuracy of dependency parsing on the development data.
However, the sizes of the training data are different across the different tasks;
for example, the semantic relatedness and entailment tasks include only 4,500 sentence pairs for training, and the dependency parsing dataset includes 39,832 sentences with word-level annotations.
Thus, in general, the dependency parsing task requires more training epochs than the semantic tasks, but currently, our model trains all of the tasks for the same training epochs.
We observed that better scores on the development sets of the semantic tasks can be achieved before the accuracy of dependency parsing reaches the best score.
Therefore, it is an interesting research direction to investigate some methods for achieving the best scores for all of the tasks at the same time.
>> 3. The training/test sets in POS/parsing are different (but overlapped). Is it possible for parsing to get gold POS information indirectly in this training method?
That is right, and the POS component of the dependency parsing model is trained by using the gold POS information of the overlapped data, although the gold POS information is not used when training the dependency parsing component.
It should be noted that the sentences in the development and test sets of dependency parsing are not included in POS and chunking training data.
Did the authors try to directly feed the label scores (instead of building an embedding)? -- Table 10 reports results without LE, but it is likely that one would try first this kind of approach by simply feeding the scores in the next architectures;
Also, have you tried with the label information, but not the hidden state one? I wonder which one is actually important, or if they are redundant (in that respect, the drop observed in Table 10 is minimal when removing LE).
Thank you very much for your detailed questions, and let me respond to each of the questions.
>> Did the authors try to directly feed the label scores (instead of building an embedding)? -- Table 10 reports results without LE, but it is likely that one would try first this kind of approach by simply feeding the scores in the next architectures;
We used the label embeddings because previous work (e.g., [1], [2], and [3]) used the embeddings for representing POS labels as well as words.
Thank you for the suggestion and we will try to directly use the label scores for comparison.
References:
[1] Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In EMNLP.
[2] David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured Training for Neural Network Transition-Based Parsing. In ACL.
[3] Makoto Miwa and Mohit Bansal. 2016. End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures. In ACL.
>> Also, have you tried with the label information, but not the hidden state one? I wonder which one is actually important, or if they are redundant (in that respect, the drop observed in Table 10 is minimal when removing LE).
We have actually performed additional experiments regarding this question, and the results are reported in Table 13 in the latest version of our submission.
As shown in the table, removing the hidden states from the input vectors has smaller effects than removing the label embedding connections.
That is, the vertical connections in the deep bi-LSTMs might become redundant if we can explicitly make use of the output information about the lower-level tasks.
When it is not obvious how to directly use output information, then the vertical connections can be helpful.
We will add more detailed discussions in the next version.
The paper introduce a way to train joint models for many NLP tasks. Traditionally, we treat these tasks as “pipeline” — the later tasks will depending on the output of the previous tasks. Here, the authors propose a neural approach which includes all the tasks in one single model. The higher level tasks takes (1) the predictions from the lower level tasks and (2) the hidden representations of the lower level tasks.
Also proposed in this paper, is the successive regularization. Intuitively, this means that, when training the high level tasks, we don’t want to change the model in the lower levels by too much so that the lower level tasks can keep a reasonable accuracy of prediction.
On the modeling side, I think the proposed model is very similar comparing to (Zhang and Weiss, ACL 2016) and SPINN (Bowman et al, 2016) in a even simpler way. The number of the experiments are good.
But I am not sure I am convinced by the numbers in Table 1 since the patterns are not very clear there — sometimes, the performance of the higher level tasks even goes down when training with more tasks (sometimes it does go up, but also not very significant and stable). The dependency scores, although I don’t think this is a serious problem, comparing the UAS/LAS when the output is not guaranteed to be a well-formed tree isn’t strictly speaking fair.
I admit that the successive regularization make sense intuitively and is a very interesting direction to try.
However, without a careful study of the training schema of such model, the current results on successive regularization do not convince me that it should be the right thing to do in such models (the current results are not strong enough to show that). The training methods need to be explored here including things as iteratively train on different tasks, and the relationship between the number of training iterations of a task and it’s training set size (and loss on this task etc).
Thank you very much for your comments.
>> On the modeling side, I think the proposed model is very similar comparing to (Zhang and Weiss, ACL 2016) and SPINN (Bowman et al, 2016) in a even simpler way.
Thank you for the pointer.
I will consider mentioning the relationship or difference in our paper.
>> But I am not sure I am convinced by the numbers in Table 1 since the patterns are not very clear there
As mentioned in the preliminary response, in our model, five different types of tasks are handled in the single model, and it is not obvious when to stop the training while trying to maximize the scores of all the tasks.
As the first step, we focused on maximizing the accuracy of dependency parsing on the development data.
However, the sizes of the training data are different across the different tasks;
for example, the semantic relatedness and entailment tasks include only 4,500 sentence pairs for training, and the dependency parsing dataset includes 39,832 sentences with word-level annotations.
Thus, in general, dependency parsing requires more training epochs than the semantic tasks, but currently, our model trains all of the tasks for the same training epochs.
We observed that better scores on the development sets of the semantic tasks can be achieved before the accuracy of dependency parsing reaches the best score.
Therefore, it should be an important research direction to investigate a method for achieving the best scores for all of the tasks at the same time.
>> The dependency scores, although I don’t think this is a serious problem, comparing the UAS/LAS when the output is not guaranteed to be a well-formed tree isn’t strictly speaking fair.
This aspect should be further discussed in our paper.
As discussed in Appendix B, 22 parsing results are not well-formed trees in terms of ROOT nodes on the development data including 1,700 sentences.
In addition, we have found that 61 parsing results have cycles.
In total, more than 95% of the greedy parsing results are well-formed trees.
I applied 1st order Eisner's algorithm to the 83 parsing results which are not well-formed trees, and as a result, the overall accuracy does not significantly change.
For example, in the case of JMT_ABC shown in Table 12, the UAS is now 94.53% (previously, 94.52%), and the LAS is 92.62% (previously, 92.61%).
Now , all parsing results can be well-formed trees while keeping the accuracy.
this work investigates a joint learning setup where tasks are stacked based on their complexity. to this end, experimental evaluation is done on pos tagging, chunking, dependency parsing, semantic relatedness, and textual entailment. the end-to-end model improves over models trained solely on target tasks.
although the hypothesis of this work is an important one, the experimental evaluation lacks thoroughness:
first, a very simple multi-task learning baseline [1] should be implemented where there is no hierarchy of tasks to test the hypothesis of the tasks should be ordered in terms of complexity.
second, since the test set of chunking is included in training data of dependency parsing, the results related to chunking with JMT_all are not informative.
third, since the model does not guarantee well-formed dependency trees, thus, results in table 4 are not fair.
minor issue:
- chunking is not a word-level task although the annotation is word-level. chunking is a structured prediction task where we would like to learn a structured annotation over a sequence [2].
[1] http://ronan.collobert.com/pub/matos/2011_nlp_jmlr.pdf
[2] http://www.cs.cmu.edu/~nasmith/LSP/
Thank you for the comments.
>> first, a very simple multi-task learning baseline [1] should be implemented where there is no hierarchy of tasks to test the hypothesis of the tasks should be ordered in terms of complexity.
That is right.
To show the importance of the linguistic hierarchy, we provided the results with and without using the hierarchy of the multi-layer RNNs in Table 12 with the paragraph "Different layers for different tasks".
Although the model architecture is different from the suggested baseline [1], the results in Table 12 show that the use of the hierarchy is more important than the number of the model parameters.
>> second, since the test set of chunking is included in training data of dependency parsing, the results related to chunking with JMT_all are not informative.
That is right.
We put more weight on higher-level tasks (dependency parsing, relatedness, entailment) in the JMT_all setting, while the chunking results with JMT_AB on the test set can be comparable with other published results.
>> third, since the model does not guarantee well-formed dependency trees, thus, results in table 4 are not fair.
This aspect should be further discussed in our paper.
As discussed in Appendix B, 22 parsing results are not well-formed trees in terms of ROOT nodes on the development data including 1,700 sentences.
In addition, we have found that 61 parsing results have cycles.
In total, more than 95% of the greedy parsing results are well-formed trees.
I applied 1st order Eisner's algorithm to the 83 parsing results which are not well-formed trees, and as a result, the overall accuracy does not significantly change.
For example, in the case of JMT_ABC shown in Table 12, the UAS is now 94.53% (previously, 94.52%), and the LAS is 92.62% (previously, 92.61%).
Now, all parsing results can be well-formed trees while keeping the accuracy.
The authors propose a transfer learning approach applied to a number of NLP tasks; the set of tasks appear to have an order in terms of complexity (from easy syntactic tasks to somewhat harder semantic tasks).
Novelty: the way the authors propose to do transfer learning is by plugging models corresponding to each task, in a way that respects the known hierarchy (in terms of NLP "complexity") of those tasks. In that respect, the overall architecture looks more like a cascaded architecture than a transfer learning one. There are some existing literature in the area (first two Google results found: https://arxiv.org/pdf/1512.04412v1.pdf, (computer vision) and https://www.aclweb.org/anthology/P/P16/P16-1147.pdf (NLP)).
In addition to the architecture, the authors propose a regularization technique they call "successive regularization".
Experiments:
- The authors performed a number of experimental analysis to clarify what parts of their architecture are important, which is very valuable;
- The information "transferred" from one task to the next one is represented both using a smooth label embedding and the hidden representation of the previous task. At this point there is no analysis of which one is actually important, or if they are redundant (update: the authors mentioned they would add something there).
Also, it is likely one would have tried first to feed label scores from one task to the next one, instead of using the trick of the label embedding -- it is unclear what the latter is actually bringing.
- The successive regularization does not appear to be important in Table 8; a variance analysis would help to conclude.
Thank you very much for your comments.
According to your comments, we will improve our paper as much as possible.
>> At this point there is no analysis of which one is actually important, or if they are redundant (update: the authors mentioned they would add something there).
Thank you for pointing out this key observation.
Indeed, we have discussed the results shown in Table 13 corresponding to the paragraph "Vertical connections" in Section 6.3, and we will explain the results more clearly.
>> Also, it is likely one would have tried first to feed label scores from one task to the next one, instead of using the trick of the label embedding -- it is unclear what the latter is actually bringing.
Thank you for the suggestion.
We try to compare the results.
The use of the label embeddings is similar to using the word embeddings;
for example, the label embeddings capture similarities between labels as shown in [1].
Thus, the POS and chunking label embeddings could be effective in using the output information in different computation units (e.g., the four different matrix multiplications in Equation (1)).
[1] Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In EMNLP.

Meta-review:
role: PC
data: comment: There is a bit of spread in the reviewer scores, but ultimately the paper does not meet the high bar for acceptance to ICLR. The lack of author responses to the reviews does not help either.

Decision: Reject