Venue: ICLR 2018 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The goal of this paper is to study generalisation to novel classes. This paper stipulates some interesting ideas, using an idea of expansion layers (using a form of hard distillation, where the weights of known classes are fixed), a GMM to model the already learned classes (to reduce storage), and a form of gradient dropout (updating just a subset of the weights using a dropout mask). All of these assume a fixed representation, trained on the base classifier, then only the final classification layer is adjusted for the novel examples. 

The major drawback is that none of these ideas are fully explored. Given fixed representation, for example the influence of forgetting on base classes, the number of components used in the GMM, the influence of the low-shot, the dropout rate, etc etc.  The second major drawback is that the experimental setting seems very unrealistic: 5 base classes and 2 novel classes. 

To conclude: the ideas in this paper are very interesting, but difficult to gather insights given the focus of the experiments.

Minor remarks
- Sect 4.1 "The randomly ... 5 novel classes" is not a correct sentence.
- The extended version of NCM (4.2.1), here uses as prototype-kNN (Hastie 2001) has also been explored in the paper of NCM, using k-means per class to extract prototypes.
- Given fixed representations, plenty of work has focused on few-shot (linear) learning, this work should be compared to these. 
review_title: Interesting, yet (by far) not sufficiently explored

final_score: rating: 6: Marginally above acceptance threshold
review_content: On few-shot learning problem, this paper presents a simple yet powerful distillation method where the base network is augmented with additional weights to classify the novel classes, while keeping the weights of the base network unchanged. Thus the so-called hard distillation is proposed. This paper is well-written and well organized. The good points are as follows,

1. The paper proposes a well-performance method for the important low-shot learning problem based on the transform learning.
2. The Gen-LSNE maintains a small memory footprint using a generative model for base examples and requires a few more parameters to avoid overfitting and take less time to train.
3. This paper builds up a benchmark for low-shot network expansion.

There are some problems,
1. There still is drop in accuracy on the base classes after adding new classes, and the accuracy may still drop as adding more classes due to the fixed parameters corresponding to the base classes. This is slightly undesired.
2. Grammatical mistake: page 3, line 5(“a additional layers”)

review_title: a paper proposing hard-distillation for few-shot learning 

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes a method for adapting a pre-trained network, trained on a fixed number of
classes, to incorporate novel classes for doing classification, especially when the novel classes
only have a few training examples available. They propose to do a `hard' distillation, i.e. they
introduce new nodes and parameters to the network to add the new classes, but only fine-tune the new
networks without modifying the original parameters. This ensures that, in the new expanded and
fine-tuned network, the class confusions will only be between the old and new classes and not
between the old classes, thus avoiding catastrophic forgetting. In addition they use GMMs trained on
the old classes during the fine-tuning process, thus avoiding saving all the original training data.
They show experiments on public benchmarks with three different scenarios, i.e.  base and novel
classes from different domains, base and novel classes from the same domain and novel classes have
similarities among themselves, and base and novel classes from the same domain and each novel class
has similarities with at least one of the base class.                        
                                                                             
- The paper is generally well written and it is clear what is being done     
- The idea is simple and novel; to the best of my knowledge it has not been tested before
- The method is compared with Nearest Class Means (NCM) and Prototype-kNN with soft distillation
  (iCARL; where all weights are fine-tuned). The proposed method performs better in low-shot
  settings and comparably when large number of training examples of the novel classes are available
- My main criticism will be the limited dataset size on which the method is validated. The ILSVRC12
  subset contains 5 base and 5 novel classes and the UT-Zappos50K subset also has 10 classes. The
  idea is simple and novel, which is good, but the validation is limited and far from any realistic
  use. Having only O(10) classes is not convincing, especially when the datasets used do have large
  number of classes. I agree that this will not allow or will takes some involved manual effort to
  curate subsets for the settings proposed, but it is necessary for being convincing.
review_title: The paper proposes a network/classifier expansion method to learn to classify with additional novel  classes in the future, without re-training with all the original data. It fine tunes the new parameters added with the new data (from novel classes), and with sampled examples from  simple generative models of the old classes. Overall the paper has a simple idea which is validated on limited settings (~10 classes only). 

Meta-review: Two reviewers recommended rejection, and one is slightly more positive. The main concern is that the experiments are not convincing (ie, the number of base and added classes is very small). Furthermore, while the paper introduces several interesting ideas, the AC agrees with the second reviewer that each of these could be explored in more detail. This work seems preliminary. The authors are encouraged to resubmit to a future conference.
Decision: Reject