From: NIPS.cc/2018/Workshop/IRASL
title: Acceptance Decision
decision: Reject

From: NIPS.cc/2018/Workshop/IRASL/Paper60/AnonReviewer3
title: Results not very exciting and missing baselines, qualitative results
review: The paper proposes a method to induce polysemy to any pretrained word embedding. Sense embeddings are obtained by running word2vec on a sense tagged corpora. These sense embeddings are then converted into word specific sense embeddings and then combined with a pretrained embedding via a concatenation. 

First off, several baselines are missing. The idea of concatenating embeddings coming from 2 different sources can be applied to any off the shelf sense specific embedding and comparisons with those would help convince the reader of the efficacy of the proposed solution. 

From Table-1, in the WS-S dataset, it appears that using CoKE only helps Glove vectors and actually hurts the spearman correlation scores for all other pretrained embeddings. An understanding of why this happens could be very useful and strengthen the paper.  In Table-4, the Glove+COKE numbers are missing. It would be useful to see some qualitative analysis to understand where the performance boost on extrinsic evaluation is coming from. 

Moreover, although the paper is concerned with inducing polysemy to any given pretrained word embeddings, comparisons with other SoTA models (from Table-3) show that the proposed solution is poorer. 

Pros:
1) Tackles a somewhat well motivated problem of inducing polysemy to any pretrained word embedding

Cons:
1) The writing of the paper is very rough. Spaces have been omitted in a lot of places, and the figures are extremely rough as well.
2) Section 2.4 is not particularly clear. Important qualitative analysis to understand how CoKE actually helps is missing
3) No experiments show if vocabulary/"societal" bias has been adequately addressed. 


rating: 2: Marginally below acceptance threshold
confidence: 3: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: NIPS.cc/2018/Workshop/IRASL/Paper60/AnonReviewer2
title: new approach of inducing polysemy to word embedding
review: This paper introduces a new approach that tries to induce polysemy to word embedding, by grounding contextualized sense representation and word embeddings to a knowledge base. The main idea in this paper follows in line with the group of methods using extra vector to represent word senses and augment the word embedding.

My main concern about this paper is its clarity. The authors try to explain a lot of things in the main parts, but I am having a hard time understanding it. It is hard to understand how the sense embeddings are generated from WordNet. It is also hard to understand the variables in the equations due to lack of explanation.

Pros:
- The experiments part is pretty complete. 
- The model outperforms its baselines by a large margin in some tasks like word similarity task.

Cons:
- The societal bias part seems not very relevant to the topic of the paper. There are no experiments considering this as well.
- The paper needs a lot more work to be clear for readers.

rating: 2: Marginally below acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/IRASL/Paper60/AnonReviewer1
title: Promising results but poorly written
review: This work builds upon previous work in word sense disambiguation and proposes an approach that merges the pre-trained word embedding with specifically learnt word sense embeddings. The paper shows the promising results on word similarity and relatedness benchmarks. They also do an extrinsic evaluation of these embeddings on SST and TREC dataset but performance gain in those cases is marginal.

 The main contribution is the word sense embeddings learnt using sense tag corporas which are then concatenated with pre-trained embedding to lead better performance on intrinsic tasks. My concerns with the paper are as follows:

1.  Due to poor writing, it is not very clear how sense embeddings are being generated. How are WordNet nodes extended? Also, it is not very clear how Word2Vec objective is used to produce "sense-form" embeddings. This according to me is the crucial contribution of the paper but due to lack of clarity, it is hard to understand what exactly was done here.
2. Section 2.4 can be re-written for clarity. You motivate using a function of ratio's as a mixing coefficients but then go on to use ranks. It is not very clear how words not in WordNet are taken care of in this case as you use ranks from WordNet for calculating \alpha.
3. "By conditioning rank on structure, this leads to ontology grounded sense forms.". It is not clear what do you mean by this sentence. There are a lot more examples of such sentences. 
4. COKE(C) and COKE(M) is not mentioned in the paper anywhere. In general, I will strongly suggest keeping the captions of your tables and figures as informative as possible by mentioning all non trivial details like what is RANDOM, COKE(M) and COKE(C) etc mean. 
5. In Table 1 and Table 2. You switch from using full Skip-Gram to SG.This can be confusing and should be avoided.
6. It is not clear to me why averaging all senses of the embedding be different from not using any sense at all. I don't see an advantage of doing this over using pre-trained embedding directly. Is there a justification why this should help? 

The paper is hard to read due to poor writing, vague sentences and verbosity.  There are a plenty of grammatical and formatting mistakes. A lot of sections are vaguely written and leave the reader scratching their head about what is being said there. 

Originality: This work is a bit derivative in nature but strong results on word similarity and relatedness task do show a merit in this line of work.

Pros:
1. Strong experimental results.
2. In general, experiments seems to be well done. 
3. This paper addresses an important problem of inducing polysemy in word embeddings.

Cons:
1. Poorly written
2. See above mentioned points.

rating: 2: Marginally below acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct