Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: One of promising approach to tackle the few-shot problems is to use meta-learning so that the learner can quickly generalize to an unseen task. One-class classification requires only a set of positive examples to discriminate negative examples from positive examples. The current paper addresses a method of meta-training one-class classifiers in the MAML framework when only a handful of positive examples are available. 

---Strength---
- Few-shot one-class classification is a timely subject, which has not be studied yet.
- Meta-training one-class classifiers in the MAML framework seems to be sound.

---Weakness---
- MAML is quite a general meta-training framework, which can be used when parameterized base-learners are updated using gradient methods.  Thus, when parameterized models for one-class classification are used, it is rather easy to meta-train one-class classifiers in the MAML framework.
- Regarding episodic training, in contrast to few-shot classification problems, support sets in episodes have similar positive examples. Thus, fine-tuning baseline method could work well, even without using MAML. Please compare it with the fine-tuning method.

---Comments---
- I assume that the query set in each episode include negative examples, while support sets have only positive examples. Right? What is the value of c (class imbalance rate) in the query set?
- Wouldn't it be  better to focus on experiments with c=0% since one-class classification requires the training with only positive examples?
- What was the baseline one-class classifier? One-class SVM?
- It was mentioned that CLEAR was an earlier work. Then, the empirical comparison with CLEAR should be included when image data is considered.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: In this paper, the authors have investigated the few shot one classification problem. They have presented a meta-learning approach that requires only few data examples from only one class to adapt to unseen tasks. The proposed method builds upon the model-agnostic meta-learning (MAML) algorithm. I think the topic itself is interesting and I have the following concerns.
(1) The first is about the real requirement of this learning scenario. Although the authors have pointed out some real applications, I think they have been introduced separated. In other words, since this setting is the combination of two previous areas, i.e., one class classification and few-shot learning, I fell that the authors have introduced it by just a combination. What are the unique challenges of this problem? I think these problems should be clarified at first.
(2) The second one is the algorithms itself. Although I have not checked the details, I fell that the authors have prepared this paper in a rough way. The authors have only described the method, without deep analyses answering the question why. For example, the method seems heuristic, without theoretical analysis. In summary, I think this paper likes a technical report, not a research paper.
(3) Although I can catch the main meaning of this paper, it seems that the writing style is not so fluently. I suggested the authors to recognize the presentation.

review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: 

 This paper tackles an interesting problem, one-class classification or anomaly detection, using a meta-learning approach. The main contribution is to introduce a parameter such that the inner-loop of the meta-learning algorithm better reflects the imbalance which occurs during meta-testing. Results are shown comparing a few simple baselines to both MAML and the modified variant, on a few datasets such as image-based ones (MNIST, miniImageNet), a synthetic dataset, and a real-world time-series example from CNC milling machines.

Overall, the paper presents an interesting problem and awareness that meta-learning might be general enough to solve it well, but provides no real novelty in the approach. The datasets and comparison to other state of art methods (including both other anomaly detection methods and out of distribution methods) is lacking. I suggest the authors perform more rigorous experimentation and focus the paper to be a paper about an understudied problem with rigorous experiments/findings, or improve their method beond the small modification made. Due to these weaknesses, I vote for rejection at this time. Detailed comments are below. 

Strengths
 
  - The problem is interesting and under-studied in the context of deep learning and transferable methods from similar ML problems (e.g. few-shot learning)

  - The method is simple and adapts a state of art in few-shot learning (meta-learning, and specifically MAML)

Weaknesses

  - While I enjoyed reading the paper since it tackles an under-explored problem, it is hard to justify publishing the method/approach at a top machine learning conference. Changing the balance in meta-learning is a relatively obvious modification that one would do to better reflect the problem; I don't think it results in general scientific/ML principles that can be used elsewhere. 

  - The relationship to out-of-distribution detection (which some of the experiments, e.g. Multi-task MNIST and miniImagenet essentially test) is not discussed or compared to. How are anomalies defined and is it really different than just being out-of-distribution? 

  - The datasets are limited. The MNIST dataset seems to choose a fixed two specific categories for meta-validation and meta-testing, as opposed to doing cross-validation. Results on just one meta-testing seems limited in this case with just one class. In terms of time-series, anomaly detection has been studied for a long time; is there a reason that the authors create a new synthetic dataset? For the milling example, how were anomalies provoked?

  - The baselines do not represent any state of art anomaly detection (e.g. density based, isolation forests, etc.) nor out of distribution detection; the latter especially would likely do extremely well for the simple image examples. 

  - There is no analysis of what the difference is in representation (initialization) learning due to the differences between the OCC and FS setup. What are the characteristics of the improved initialization?


One minor comment not reflecting the decision:
  - Exposition: Define the one-class classification problem; it's not common so it would be good to define in the abstract, or mention anomaly detection which is a better-known term. 


review_title: Official Blind Review #2

Meta-review: The authors present a combination of few-shot learning with one-class classification model of problems. The authors use the existing MAML algorithm and build upon it to present a learning algorithm for the problem. As pointed out by the reviewers, the technical contributions of the paper are quite minimal and after the author response period the reviewers have not changed their minds. However, the authors have significantly changed the paper from its initial submission and as of now it needs to be reviewed again. I recommend authors to resubmit their paper to another conference. As of now, I recommend rejection.
Decision: Reject