SlowGuess/ACL_2020_OCR huggingface dataset:
In Table 5, we show how we consistently improve over DENSPI when SPARC is added in different search strategies. Note that on CuratedTREC where the questions more resemble real user queries, DFS outperforms SFS showing the effectiveness of dense search when not knowing which documents to read.
Models using Bi-LSTM as Base Encoders
Model	# of Docs. to Read	# of Docs. to Re-rank	s/Q
DrQA	5	None	35
Paragraph Ranker	≈ 3	20	161
Models using BERT as Base Encoders
Model	# of Docs. to Read	# of Docs. to Re-rank	Max. Sequence Length/Doc. Stride	s/Q
BertSerini	100 paragraphs*	100 paragraphs*	384/128	115
Multi-Passage BERT	30 passages**	100 passages**	100/50	84
ORQA	k blocks†	None	384/128	8
* Assumed 1 paragraph = 200 BERT tokens.
$^{**}$  1 passage = 100 BERT tokens.
† 1 block = 288 BERT tokens. Assumed  $k = 10$ .
Table 6: Details on how the inference speed of each open-domain QA model is estimated using a single CPU. We use the default 384/128 (Max. Sequence Length/Doc. Stride) setting for both BertSerini and ORQA.