Venue: ICLR 2018 conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This article clearly describes how they designed and actively trained 2 models for question reformulation and answer selection during question answering episodes. The reformulation component is trained using a policy gradient over a sequence-to-sequence model (original vs. reformulated questions). The model is first pre-trained using a bidirectional LSTM on multilingual pairs of sentences. A small monolingual bitext corpus is the uses to improve the quality of the results. A CNN binary classifier performs answer selection. 

The paper is well written and the approach is well described. I was first skeptical by the use of this technique but as the authors mention in their paper, it seems that the sequence-to-sequence translation model generate sequence of words that enables the black box environment to find meaningful answers, even though the questions are not semantically correct. Experimental clearly indicates that training both selection and reformulation components with the proposed active scheme clearly improves the performance of the Q&A system. 
review_title: Well written paper that clearly describes a RL-based active approach for question reformulation and answer selection with interesting experimental results. 

final_score: rating: 7: Good paper, accept
review_content: This paper proposes active question answering via a reinforcement learning approach that can learn to rephrase the original questions in a way that can provide the best possible answers. Evaluation on the SearchQA dataset shows significant improvement over the state-of-the-art model that uses the original questions. 

In general, the paper is well-written (although there are a lot of typos and grammatical errors that need to be corrected), and the main ideas are clear. It would have been useful to provide some more details and carry out additional experiments to strengthen the merit of the proposed model. 

Especially, in Section 4.2, more details about the quality of paraphrasing after training with the multilingual, monolingual, and refined models would be helpful. Which evaluation metrics were used to evaluate the quality? Also, more monolingual experiments could have been conducted with state-of-the-art neural paraphrasing models on WikiQA and Quora datasets (e.g. see https://arxiv.org/pdf/1610.03098.pdf and https://arxiv.org/pdf/1709.05074.pdf). 

More details with examples should be provided about the variants of AQA along with the oracle model. Especially, step-by-step examples (for all alternative models) from input (original question) to question reformulations to output (answer/candidate answers) would be useful to understand how each module/variation is having an impact towards the best possible answer/ground truth.

Although experiments on SearchQA demonstrate good results, I think it would be also interesting to see the results on additional datasets e.g. MS MARCO (Nguyen et al., 2016), which is very similar to the SearchQA dataset, in order to confirm the generalizability of the proposed approach. 

---------------------------------------------
Thanks for revising the paper, I am happy to update my scores.
review_title: good, but could be better with more details and experiments

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper formulates the Jeopardy QA as a query reformulation task that leverages a search engine.  In particular, a user will try a sequence of alternative queries based on the original question in order to find the answer.  The RL formulation essentially tries to mimic this process.  Although this is an interesting formulation, as promoted by some recent work, this paper does not provide compelling reasons why it's a good formulation.  The lack of serious comparisons to baseline methods makes it hard to judge the value of this work.

Detailed comments/questions:
	1. I am actually quite confused on why it's a good RL setting. For a human user, having a series of queries to search for the right answer is a natural process, but it's not natural for a computer program.  For instance, each query can be viewed as different formulation of the same question and can be issued concurrently. Although formulated as an RL problem,  it is not clear to me whether the search result after each episode has been used as the immediate environment feedback. As a result, the dependency between actions seems rather weak.
	2. I also feel that the comparisons to other baselines (not just the variation of the proposed system) are not entirely fair. For instance, the baseline BiDAF model has only one shot, namely using the original question as query.  In this case, AQA should be allowed to use the same budget -- only one query.  Another more realistic baseline is to follow the existing work on query formulation in the IR community.  For example, 20 shorter queries generated by methods like [1] can be used to compare the queries created by AQA.

[1] Kumaran & Carvalho. "Reducing Long Queries Using Query Quality Predictors". SIGIR-09
	
Pros:
	1. An interesting RL formulation for query reformulation

Cons:
	1. The use of RL is not properly justified
	2. The empirical result is not convincing that the proposed method is indeed advantageous 

---------------------------------------

After reading the author response and checking the revised paper, I'm both delighted and surprised that the authors improved the submission substantially and presented stronger results.  I believe the updated version has reached the bar and recommend accepting this paper.  
review_title: RL formulation for question rewrite, but with weak justification

Meta-review: this submission presents a novel way in which a neural machine reader could be improved. that is, by learning to reformulate a question specifically for the downstream machine reader. all the reviewers found it positive, and so do i.
Decision: Accept (Oral)