From: NIPS.cc/2018/Workshop/IRASL/Paper36/Authors
title: Thanks for your comments!
comment: We really appreciate the time and effort you put into reviewing our paper! Thanks!

From: NIPS.cc/2018/Workshop/IRASL/Paper36/Authors
title: Thanks for your review!
comment: Thanks for your effort and your comments! 

1/ We believe that our work is tied to the main theme of the workshop, as the robustness of our model is demonstrated by various comparisons in the ablation study.  Aligning representations from two distinctive information processing views ensures that the ensemble of two views after learning provides better performance than each one of them. Simply averaging two representations without aligning them during learning could provide poor performance and it is worse than the linear model solely on unsupervised evaluation tasks, as presented in Table 6. We will include more details into our paper.

2/ We will revise the notations in Eqs. 1 and 3.

Cheers,

From: NIPS.cc/2018/Workshop/IRASL/Paper36/Authors
title: Thanks for your interesting and helpful comments, and we will try to address some questions here.
comment: Q1: It is hard to design an invertible RNN decoder, whose inverse can be applied in the testing time. We believe that it is possible to design an invertible RNN decoder, but given the findings in [52], it doesn't seem to be helpful to learn an autoregressive/non-autoregressive RNN decoder.

Q2: Good point! Attention can certainly be applied in our model as both the RNN and the linear encoders are capable of generating a sequence of hidden states, such as the BiDAF architecure.  

In our case, we proposed a simple prototype to verify the idea that, by aligning the representations produced from two views/encoding functions, each view gets improved compared to single-view trained counterparts. Additional attention mechanism could be helpful in terms of getting better performance, but it doesn't necessarily provide explicit evidence for verifying our idea.

L95: Yes! The finding in [52] is indeed based on the nature of the downstream tasks. However, as the downstream tasks evaluated in [52] cover various aspects of natural language understanding, the finding itself is generalisable to the situation when the goal is to learn general purpose sentence representations in an unsupervised fashion.

L105: The usage of our models is discussed in Table 2 in our paper.



Q about Table 2:

We discussed the reason in the text in Section 3, as the representation pooling methods for supervised and unsupervised tasks are proposed in previous work, and we directly adopted them.

From: NIPS.cc/2018/Workshop/IRASL
title: Acceptance Decision
decision: Accept

From: NIPS.cc/2018/Workshop/IRASL/Paper36/AnonReviewer3
title: Multi-view frameworks with RNNs and Linear models for Semantic tasks
review: The key idea is to use multi-view models consisting of an RNN encoder and a linear encoder trained using either a generative or a discriminative criterion. In the generative framework, a sentence generates another sentence in the neighboring context whereas in the discriminative framework a classifier distinguishes whether or not two sentences belong to neighboring contexts.  The paper reports improvements on an unsupervised textual similarity task and competitive performance supervised semantic tasks. 

The finding that multiview methods perform better than single-view methods is interesting and a useful contribution.

Some comments:
It would be very useful to see examples from both unsupervised and supervised settings where the multi-view model does better than the baseline and vice-versa. 

Some questions:
1) how would an average-on-word encoder and an RNN decoder perform? 

2) In the current method, only the hidden state at the last time step is used. However from work on neural machine translation, it is known that attention based representations are a lot more powerful. It might be interesting to discuss how attention based models could be used in this multi-view scenario. For example, is it possible that the current approach without attention works better only on short sentences?

L95: "Given the finding [52] that neither an autoregressive nor an RNN decoder is necessary for learning sentence representations that excel on downstream tasks, our learning framework only learns to predict words in the next sentence." 
Wouldn't this finding be tied to the nature of the downstream task? 

L105. Discuss how this model is used at test time.

It would be useful for researchers if this code is released.

(Table 2) Why are separate pooling representations used for unsupervised and supervised? It would be best to compare the same set of pooling representations in both cases.









rating: 4: Top 50% of accepted papers, clear accept
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/IRASL/Paper36/AnonReviewer1
title: Multi-view learning for improved sentence representation
review: The authors introduce a multi-view training framework for sentence representation. It consists of an RNN encoder and linear word average model that could be paired and trained generatively together in an encoder-decoder fashion, or trained discriminatively.

The paper is well written, easy to follow, with extensive experimentation. What is lacking is more insight and analysis in the text about the presented numeric results, e.g. the superiority of the discriminative training, mixed results of with and without inevitability constraint. 

Also, the paper is not tightly related to the main theme of the workshop around interpretability and robustness. On the other hand, multi-view learning could improve robustness but this link need to be developed in the text.

comments:
-in equation 1, make it clear that w_j is from sentence i+1 using a superscript
-in equation 3, make it clear that the negative samples in n are outside the context c of i

rating: 4: Top 50% of accepted papers, clear accept
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/IRASL/Paper36/AnonReviewer2
title: Improving Sentence Representations with Multi-view Frameworks
review: This paper proposes a novel training for obtaining sentence representations.
For encoding sentences, two views, the RNN-based sentence encoder and the avg-on-word-vectors sentence encoder, were combined.
These endoders were trained with generative and discriminative objectives.
The advantage of these two-by-two combination has been evaluated and confirmed with a wide range of unsupervised and supervised downstream tasks.
Additional ablation study and detailed discussion further made the proposed approach convicing.

Various papers have been showing effectiveness of pre-training and transfer learning for natural language understanding.
While the concept is similar, the proposed method in this paper is focusing on sentence embedding and can be regarded as another standard method.
rating: 4: Top 50% of accepted papers, clear accept
confidence: 2: The reviewer is fairly confident that the evaluation is correct