  We study multi-answer retrieval, an under-explored problem that requires
retrieving passages to cover multiple distinct answers for a given question.
This task requires joint modeling of retrieved passages, as models should not
repeatedly retrieve passages containing the same answer at the cost of missing
a different valid answer. Prior work focusing on single-answer retrieval is
limited as it cannot reason about the set of passages jointly. In this paper,
we introduce JPR, a joint passage retrieval model focusing on reranking. To
model the joint probability of the retrieved passages, JPR makes use of an
autoregressive reranker that selects a sequence of passages, equipped with
novel training and decoding algorithms. Compared to prior approaches, JPR
achieves significantly better answer coverage on three multi-answer datasets.
When combined with downstream question answering, the improved retrieval
enables larger answer generation models since they need to consider fewer
passages, establishing a new state-of-the-art.
