Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose a training technique for information retrieval models in the context of (open domain) question answering. Assuming the existence of some reader model, the idea is to use internal information of that model as a training signal for a retriever. Specifically, they use the attention activations over the input documents as synthetic labels for the retriever.
 
The paper is written well, and proposes an interesting idea. The technique is well motivated, I particularly like that they motivate the use of the cross-attention score through a simple experiment, where they compare the top 10 DPR passages with the top 10 passages by attention score.
The results over the SOTA seem moderate though, and the number of iterations seems to be an important (and potentially underexplored) variable there.
 
I support the acceptance of the paper, because I believe the technique and the choice of model score (cross-attention score) are both interesting contributions.
 
As the authors say, training retrieval systems is tricky, since there is usually not sufficient labelled data available and it might depend heavily on the task. The iterative training that exploits internal state of the “downstream” model that they describe is an interesting idea that deserves attention from the community.
 
In Table, only up to 4 iterations in the table, which still show large improvements from one to the next. It would be interesting to know at what point no additional gains are seen.

review_title: Interesting technique

final_score: rating: 7: Good paper, accept
review_content: The paper targets an important problem in open-domain QA - the training of the retriever for the purpose of determining a segment that may contain the answer. In the most traditional setting, the retriever is just a traditional IR system such as BM25. In some existing work, the retriever has been trained to locate the documents containing the answer (e.g. inverse cloze task, or DPR). This paper goes in the same direction. The difference is that it uses the attention weights as relevance signals to train the retriever, instead of the inclusion of the answer in the passage.

Overall, the paper describes an interesting contribution to the problem. I vote for accepting it.

Strengths:
The proposed idea is interesting and complements those used in the existing work. In the experiments, it is shown that the idea can indeed improves the existing ones: it takes the existing work DPR as the starting point, and shows that the iterative process incorporating the attention weights can improve the QA performance.

The paper provides solid experiments that support the claim.

Weaknesses:
While the basic idea sounds interesting, the paper does not provide a strong intuition behind it. One could imagine that strong attention could be paid to wrong passages, as the reader does not know the ground truth. In contrast, the idea used in the previous work - the inclusion of the answer in a passage is a positive signal - sounds more intuitive. It would be interesting to estimate how the attentions correlate to with the distribution of correct answer in the passages. This could provide some empirical indication that attention weights point to the right direction. The paper contains Table 1, which partly show this correlation. A stronger intuitive motivation could make the paper stronger.


review_title: The paper is about open domain QA. It describes an interesting approach to train the retriever using attention of the reader as weak supervision signal. This helps solve the problem of lack of QA-oriented text retrieval relevance judgments. The general assumption is that the reader can provide useful signals back to the reader. The experiments on 2 datasets confirm that such a process can indeed help increase the performance.

final_score: rating: 7: Good paper, accept
review_content: Paper Summary:
* This paper proposes a technique to learn retriever models for question answering that does not require annotated pairs of query and documents. The proposed technique uses  attention scores of a reader model to obtain synthetic labels for the retriever.  Experimental results with NaturalQuestions, TriviaQA, and NarrativeQA show that the technique achieved state-of-the-art results.

Strengths:
* This paper is well organized and well written.
* The proposed method is novel.
* Experimental results are strong.

Comments:
* We can see from Table 1 of this paper and Table 2 of the DPR paper that the P@20 and P@100 scores of the proposed retriever, which used iterative training starting with documents retrieved with BM25, outperformed those of the DPR model trained in a supervised manner.  This is a surprising result. I think this paper can benefit a lot with a more analysis with this point.  How much false positives and negatives are included in the reader's attention scores? Why are the readers' attention scores better than supervised  labels?
* I also would like to see the results of the proposed system that uses supervised labels of relevant passages (e.g. a cross entropy loss) in addition to the KL divergence loss.  Moreover, I would like you to add the results of iterative training starting with documents retrieved with DPR to Table 1.
* There is a concurrent study.  I think it would be very useful to discuss the study: "Is Retriever Merely an Approximator of Reader?" https://openreview.net/forum?id=dvXFpV6boX

Update:
Thank you for the answers to my questions and additional experiments!
review_title: Official Blind Review #3

final_score: rating: 7: Good paper, accept
review_content: Many Information Retrieval systems rely on two components: a retriever that identifies a small set of "support" documents from a large corpus, followed by a reader that re-scores these support documents more finely. For retrievers, metrics like BM25 were once common but they are increasingly replaced by machine learned components. However, most datasets do not provide direct supervision information for the retriever.

Assuming a Transformer reader, this paper proposes to train the reader and retriever iteratively, using cross-attention scores from an increasingly better reader as a way to identify increasingly better sets of support documents. The novelty of this paper is the use of reader cross-attention scores as a proxy to train the retriever. The models are otherwise standard.

This paper is well motivated and clearly written. It seems to be improving the state-of-the-art on three datasets and over several recent baselines. The training procedure (including hyper-parameters) is detailed, facilitating reproducibility.

Questions and comments:
- Since DPR plays a central role in this paper, it deserves a longer introduction, either in Section 2 or later. Further discussion on the use of already trained models as starting point would be interesting.
- Similarly, it would be nice to summarize the negative example sampling strategy in Section 3.5.
- On the NarrativeQA dataset, the authors mention "it is not straightforward to train the retriever with heuristics using Q-A pairs". Isn't BM25 (used for the initial set of documents) a heuristic? Can this be clarified?
- I assume the gains against the baselines are statistically significant, but it would be nice to mention in writing.
- An iterative training procedure is going to be slower than other approaches. Some discussion on this point would be interesting. Or referencing a discussion in a previous paper would also be OK.
- Are there any other downsides to this approach?

Minor comments:
- In the third paragraph of the first section, it sounds like the authors introduce the retriever/reader architecture, which is standard (and otherwise clear from the rest of the paper).
- Typo, page 2: well know -> well known
- Please introduce d (embedding dimensionality) in page 3.
- Please add a citation in the first row of Table 3.
review_title: An interesting contribution to supervised training of retriever models

Meta-review: The paper attempts to improve retrieval in open domain question answering systems, which is a very important problem. In this regards, the authors propose to utilize cross-attention scores from a seq2seq reader models as signal for training retrieval systems. This approach overcomes typical low amount of labelled data available for retriever model. The reviewers reached a consensus that the proposed approach are interesting and novel. The proposed approach establish new state-of-the-art performance on three QA datasets, although the improvements over previous methods are marginal. Overall, reviewers agree that the paper will be beneficial to the community and thus I recommend an acceptance to ICLR.

Decision: Accept (Poster)