Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: 1) As the proposed soft-sharing utilizes task dependent weights, it would be great to discuss the increase of model size compared to the conventional hard-sharing. 

Furthermore, experiments that shown the gains of the proposed method are not coming from the extra params would make the paper more convincing. 


2) The paper focus on MTL, but the factorization approach can be used more generically, not restricted to MTL tasks. A simple way is to cluster the training data and each cluster can be your "task". In speech community, there's some work on cluster adaptive training of DNN, which does similar factorization for each DNN layer, such as "Cluster Adaptive Training for Deep Neural Network Based Acoustic Model" (http://ieeexplore.ieee.org/document/7364207/)
Discussions on the similarities or differences between them would be interesting.


review_title: model size and cluster adaptive training 

final_score: rating: 7: Good paper, accept
review_content: The paper proposed a tensor factorization approach for MTL to learn cross task structures for better generalization. The presentation is clean and clear and experimental justification is convincing. 

As mentioned, including discussions on the effect of model size vs. performance would be useful in the final version and also work in other fields related to this. 

One question on Sec. 3.3, to build the DMTRL, one DNN per-task is trained with the same architecture. How important is this pretraining? Would random initialization also work here? If the data is unbalanced, namely, some classes have very few examples, how would that affect the model?




review_title: null

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper proposed a nice framework leveraging Tucker and Tensor train low-rank tensor factorization to induce parameter sharing for multi-task learning.

The framework is nice and appealing. 

However, MTL is a very well studied problem and the paper considers simple task for different classification, and it is not clear if we really need ``Deep Learning" for these simple datasets. A comparison with existing shallow MTL is necessary to show the benefits of the proposed methods (and in particular being deep) on the dataset. The authors ignore them on the basis of speculation and it is not clear if the proposed framework is really superior to simple regularizations like the nuclear norm. The idea of nuclear norm regularization can also be extended to deep learning as gradient descent are popular in all methods. 
review_title: Comparison with other standard MTL methods is missing

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper proposed a deep multi-task representation learning framework that learns cross-task sharing structure at every layer in a deep network with tensor factorization and end-to-end knowledge sharing. This approach removed the requirement of a user-deﬁned multi-task sharing strategy in conventional approach. Their experimental results indicate that their approach can achieve higher accuracy with fewer design choices.

Although factorization ideas have been exploited in the past for other tasks I think applying it to MTL is interesting. The only thing I want to point out is that the saving of parameter is from the low-rank factorization. In the conventional MTL each layer's weight size can also be reduced if SVD is used. 

BTW, recent neural network MTL was explored first (earlier than 2014, 2015 work cited) in speech recognition community. see, e.g., 

Huang, J.T., Li, J., Yu, D., Deng, L. and Gong, Y., 2013, May. Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 7304-7308). IEEE.
review_title: null

Meta-review: The reviews for this paper were quite mixed, with one strong accept and a marginal reject. A fourth reviewer with strong expertise in multi-task learning and deep learning was brought in to read the latest manuscript. Due to time constraints, this fourth review was not entered in the system but communicated through personal communication. 
 
 Pros:
 - Reviewers in general found the paper clear and well written.
 - Multi-task learning in deep models is of interest to the community
 - The approach is sensible and the experiments show that it seems to work
 
 Cons:
 - Factorization methods have been used extensively in deep learning, so the reviewers may have found the approach incremental
 - One reviewer was not convinced that the proposed method would work better than existing multi-task learning methods
 - Not all reviewers were convinced by the experiments
 - The fourth reviewer found the approach very sensible but was not excited enough to champion the paper
 
 The paper was highly regarded by at least one reviewer and two thought it should be accepted. The PCs also agree that this paper deserves to appear at the conference.
Decision: Accept (Poster)