Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a model to determine word embeddings by composing morpheme embeddings. This is important research that mixes pre-existing knowledge and corpus-derived knowledge. 

strengths_and_weaknesses: Pro:

- the idea is sound
- results are convincing

Cons:

- it is unclear the use of the tensor product
questions: How does the method deal with tensors of increasing dimensions?

limitations: 
The paper fails to be clear with respect to the use of the tensor product. Indeed, the tensor product increases the dimensions of the tensors at each step. The dimension of the product of two tensors is the sum of dimensions of the two tensors, that is, if two matrices are the arguments of the tensor product, the results is a four-dimensional tensor. Hence, it is unclear how this model generalizes.

The authors should reduce the use of the bold and the italics.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces a new word embedding compression method with morphological augmentation, which is called "morphologically-enhanced tensorized embeddings (MorphTE)". The general idea is that, similar to how each word's meaning is a combination of its morphemes, each word embedding is a the product of the word's individual morpheme embeddings. 
The authors conduct experiments on machine translation, question answering, and natural language inference tasks and show that MorphTE can compress word embedding parameters by a lot without performance loss, significantly outperforming related embedding compression methods.
strengths_and_weaknesses: Strengths:
- The method is well motivated *and* performs well.
- The paper is very clearly written.
- The experimental setup is well done and the proposed approach is compared to relevant baselines.

Weaknesses:
- No big ones; one could consider using newer NLI datasets like MNLI, but this isn't really a weakness.
questions: - What are the limitations of your work?
- How does the performance of your proposed approach depend on the performance of the morphological segmenter?
limitations: I don't see any direct negative societal impact of this work. The authors don't address limitations explicitly.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes to incorporate morphemes while compressing word embeddings and achieve lossless compression while decreasing the number of parameters. The authors compare their method to several compression techniques and show that even with a 40x compression ratio, the proposed embeddings are on par with the baseline, and on 20x they can outperform previous baselines.
strengths_and_weaknesses: Strengths:
- Well-written paper
- Provide results on several tasks and multiple dataset for the task
- Embeddings can be use on resource-limited devices while still keep the performance

Weaknesses:
- Usage on resource-limited devices is used as motivation, however there is no further discussion and speed/memory comparison
- Ambiguity in terms which sometimes affects clarity: word/morpheme/bpe embeddings

questions: - I did not quite understand how do you train morpheme embeddings? Is it a linear layer in the transformer? And on which step do you divide a word into morphemes?
- Do you use words as inputs and split them into morphemes on the fly, or dataset is preprocessed with morphemes? Do you have any computational overhead, especially since your sequences are longer with morphemes representations?
- line 233: <...>**bpe**<...> - when and for what do you use bpe? Is this only for **Original**? 
- do you still use positional embeddings? Does it somehow change compared to the original? 

limitations: Limitations are discussed in analysis
review_title: null

Meta-review: metareview: The paper presents a method to compress word embeddings using morphologically-enhanced tensorized embeddings. The main idea is decomposing a word embedding into a tensor product of multiple small vectors. This idea has been explored in previous work [1], the main contribution of this is using *morphological segmentation*, aiming to capture morphological features. 

The proposed method is compared to prior work on multiple tasks (machine translation, qeustion answering, etc), overall exhibiting strong performance in terms of compression rate & performance (though sometimes gains are pretty smaller compared to Word2ket (e.g., Table 2)). 

During the discussion phase, the authors provided the results with random, non-morpholgically inspired segmenation, which was useful. I’d recommend putting them into the main paper. Experimenting on morphologically rich languages (e.g., Finnish) would be useful. As well as sensitive study towards hyper parameters (e.g., dimensionality of morpheme embeddings) which can easily impact the compression rate & performance. 

Overall, the reviewers and AC is positive about the paper. 

[1] word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement
https://arxiv.org/abs/1911.04975
confidence: Certain
Decision: Accept