  Neural information retrieval systems typically use a cascading pipeline, in
which a first-stage model retrieves a candidate set of documents and one or
more subsequent stages re-rank this set using contextualized language models
such as BERT. In this paper, we propose DeepImpact, a new document
term-weighting scheme suitable for efficient retrieval using a standard
inverted index. Compared to existing methods, DeepImpact improves impact-score
modeling and tackles the vocabulary-mismatch problem. In particular, DeepImpact
leverages DocT5Query to enrich the document collection and, using a
contextualized language model, directly estimates the semantic importance of
tokens in a document, producing a single-value representation for each token in
each document. Our experiments show that DeepImpact significantly outperforms
prior first-stage retrieval approaches by up to 17% on effectiveness metrics
w.r.t. DocT5Query, and, when deployed in a re-ranking scenario, can reach the
same effectiveness of state-of-the-art approaches with up to 5.1x speedup in
efficiency.
