Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper presents an architecture to learn an active learning procedure that can be applied to different tasks of the same domain. An example practical usage is preference elicitation for recommendation, in which one wants to learn the series of questions to ask to users to efficiently collect ratings for recommendation. The algorithm is based on the idea that a "question" is a rating/label of an item in a pre-specified support set. The support set depends on the task at hand. The overall algorithm is trained using policy gradient, and some experiments on Omniglot and MovieLens show the algorithm performs better than a few baselines.

The idea makes sense and the problem is interesting since there is no clear solution to the "cold-start" problem in the literature. The work is still fairly preliminary (for instance, a simple baseline would be to learn the representations offline using standard supervised learning and apply an existing active learning algorithm at test time), but it may be sufficient for acceptance to the workshop.

remarks:
- It seems that the MovieLens experiments do not correspond to a realistic scenario and I am not sure of the conclusion of the experiments. The support set in these experiments contains only movies that were rated by the user (without the ratings). It may seem to make sense in lab experiments because we cannot ask the user to rate additional movies. In practice however,  knowing what kind of movies the user rates is already informative of the user interests, even without the ratings (see e.g., Marlin et al. "Collaborative Prediction and Ranking with Non-Random Missing Data": people tend to rate items they like). Thus, the model in the experiments has access to more information that what would be available in practice. A more realistic evaluation would be to take the entire set of movies as support set, but to ignore the query if it is not a movie that the user rated. 

- Related question: what is the bi-LSTM useful for in the encoder of the support set? I suppose this is used to encode individual items in the context of the whole support set. But once again, in the MovieLens/cold-start recommendation setting, I am not sure that it can really be used in practice.

- In Fig. 2 (c), it seems that much of the improvement compared to the baselines comes at the first query. Do the authors have an idea of the rule found by the model to make the first selection? 
review_title: review

final_score: rating: 6: Marginally above acceptance threshold
review_content: The topic of this paper is to present a novel active learning mechanism
based on reinforcement learning and one-shot learning, where active learning
is seen as a sequential decision process of selecting one example at a time
to be labeled and update the model accordingly to better select the next
example.

I did not quite understand the motivation example of the cold start problem
in recommendation tasks: it's only "cold" for the first movie, not afterward,
so how does it differ from normal recommendation approaches?

More importantly, I did not quite understand the precise proposed model:
the figure helped but was not enough, and the text itself said "a detailed
description... is beyond the scope of this extended abstract". So the only
thing I got was the loss function and a very rough idea of the overall process.
How does it differ from other active learning approaches for instance?

Finally, experiments look good but are not compared to any other active
learning approaches (apart from random selection which is rather naive).

I understand this is a workshop submission and it's only 3 pages but if
I don't understand the main idea and the results, it's hard for me to give
a good score.

*improving score after revision.

review_title: not enough details?

Decision: Accept