Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper presents an interesting idea to word embeddings that it combines a few base vectors to generate new word embeddings. It also adopts an interesting multicodebook approach for encoding than binary embeddings. 

The paper presents the proposed approach to a few NLP problems and have shown that this is able to significant reduce the size, increase compression ratio, and still achieved good accuracy.

The experiments are convincing and solid. Overall I am weakly inclined to accept this paper.
review_title: Review

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper proposed a new method to compress the space complexity of word embedding vectors by introducing summation composition over a limited number of basis vectors, and representing each embedding as a list of the basis indices. The proposed method can reduce more than 90% memory consumption while keeping original model accuracy in both the sentiment analysis task and the machine translation tasks.

Overall, the paper is well-written. The motivation is clear, the idea and approaches look suitable and the results clearly follow the motivation.

I think it is better to clarify in the paper that the proposed method can reduce only the complexity of the input embedding layer. For example, the model does not guarantee to be able to convert resulting "indices" to actual words (i.e., there are multiple words that have completely same indices, such as rows 4 and 6 in Table 5), and also there is no trivial method to restore the original indices from the composite vector. As a result, the model couldn't be used also as the proxy of the word prediction (softmax) layer, which is another but usually more critical bottleneck of the machine translation task.
For reader's comprehension, it would like to add results about whole memory consumption of each model as well.
Also, although this paper is focused on only the input embeddings, authors should refer some recent papers that tackle to reduce the complexity of the softmax layer. There are also many studies, and citing similar approaches may help readers to comprehend overall region of these studies.

Furthermore, I would like to see two additional analysis. First, if we trained the proposed model with starting from "zero" (e.g., randomly settling each index value), what results are obtained? Second, What kind of information is distributed in each trained basis vector? Are there any common/different things between bases trained by different tasks?
review_title: Effective work

final_score: rating: 7: Good paper, accept
review_content: The authors proposed to compress word embeddings by approximate matrix factorization, and to solve the problem with the Gumbel-soft trick. The proposed method achieved compression rate 98% in a sentiment analysis task, and compression rate over 94% in machine translation tasks, without a performance loss. 

This paper is well-written and easy to follow.  The motivation is clear and the idea is simple and effective.

It would be better to provide deeper analysis in Subsection 6.1. The current analysis is too simple. It may be interesting to explain the meanings of individual components. Does each component is related to a certain topic? Is it meaningful to perform ADD or SUBSTRACT on the leaned code? 

It may also be interesting to provide suitable theoretical analysis, e.g., relationships with the SVD of the embedding matrix.

review_title: Simple yet effective work

Decision: Accept (Poster)