Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper introduces a technique to project n gram statistic vectors into a lower dimensional space in order to improve memory efficiency and lower training time. The paper is motivated by the important problem of trying to improve efficiency of existing language models which can be extremely resource intensive. The authors then compare the performance of n gram statistics with HD vectors on 4 datasets to demonstrate that embedding into HD vectors can preserve performance while reducing resource utilization.

	1. The main methodological contribution (using HD vectors) are a nice contribution. I would like the authors to clarify why they chose those 3 operations to operate on vectors for this particular task. 
	2. Lack of a competitive baseline : I would like to see how this method compares with existing techniques to speedup n grams such as Pibiri et al. (https://arxiv.org/pdf/1806.09447.pdf). I believe there are other works which work on speeding up n gram models as well but not comparison is presented.  
	3. Minor comment: the authors mention using a SGD classifier but I fail to understand what they mean by this. SGD is an optimization technique and not a classifier so I would like the authors to correct this in the paper.  

As it currently stands, the lack of strong baselines and the incremental nature of the contribution lead me to believe that this paper does not represent a sufficient advance to warrant publication. I would advise the authors to consider submitting to a more specialized venue (in NLP).

review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: This paper proposes the use of hyperdimensional (HD) vectors to represent n-gram statistics. The HD vectors are first generated from the whole corpus. Then, it is aggregated or bundled to a vector for each sample as an input of a classifier training. The evaluation is conducted on four datasets: Chatbot, AskUbuntu, WebApplication and 20 News Group using a bunch of classifier including KNN, Random Forest, MLP etc.

It is interesting to see how to hash/project the high dimensional n-gram vector into a lower space for efficiency. The approach is useful in online production systems, and it is eco-friendly. However, there are a few concerns detailed as follows:

1. Can it be generalized to contemporary learned embeddings, e.g., word2vec and GloVe? 

2. Lack of proper baselines for comparison. Word2vec, GloVe are trained on large corpora once and can be applied directly to other tasks, and they should be served as baselines. Furthermore, the simple bag of word/TF-IDF should be included as baselines as well.

3. Lack of analysis: it is hard to understand what kind of HD vectors are generated. Are these n-grams semantically related projected nearby in the HD space? This helps readers to understand the constructed embeddings.

4. The SentEval benchmark is popular in sentence level representation learning and it is well known. It is better to see some evaluations on it as well. http://www.lrec-conf.org/proceedings/lrec2018/pdf/757.pdf

Minor comments: 
1. The Subword Semantic Hashing is originally from DSSM published in 2013.  (https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/cikm2013_DSSM_fullversion.pdf) 
2. What is $v_c$ in 4.2?

review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: This paper shows a trade-off relationship between computational cost (memory usage, train/test time) and performance on several NLP machine learning algorithms that use n-gram statistics. The authors claim that a simple n-gram representation vector with a conventional classifier (MLP, SVC, Naive Bayes...) is computationally efficient.

The large set of experiments on various conventional NLP models and n-gram statistics provide detail information about the trade-off relation between performance and computational cost. My concern is that the computational efficiency of the conventional NLP model is well known to NLP researchers. It would be nice if the authors provide a more persuasive explanation for the importance of this research question.
review_title: Official Blind Review #3

Meta-review: The reviewers were unanimous that this submission is not ready for publication at ICLR in its present form.

Concerns raised included lack of relevant baselines, and lack of sufficient justification of the novelty and impact of the approach.
Decision: Reject