SlowGuess/NAACL_2021_OCR huggingface dataset:
Web search results, while most accurately correlates with human judgment, also reflect sensitivity of the retrieval algorithm to the query formulation as well as the collection-specific selectivity of the query terms. The resulting scores for our sample rewrites are R@10("When is Robert Downey Jr birthday", "When is Robert Downey Jrs birthday")=0.6 and R@10("When did Gabriel Garcia die", "when did Gabriel Garcia Marquez die") = 0.78.
D Examples of Query Rewrites
In Table 9 we show sample question rewrites from top 3 QR models along with conversational context.
E Examples of Answers Found
In Table 10 we provide two sample answers found by the baseline model. In the first example, the baseline system picked the same passage as the human annotator, but extracted a different answer span from this passage. In the second example, the baseline system picked a different passage than the human annotator. While the produced answers are not exact matches to the corresponding human answers, we consider them as correct upon manual inspection. This observation that a single question in QReCC can have multiple correct answers poses an important challenge for standard approaches to QA evaluation.
Table 9: A sample of conversation snippets from the QReCC test set with the question rewrites produced by Transformer++, CopyTransformer, and GECOR models.
Conversational context	Transformer++	CopyTransformer	GECOR
Q1: Did Nadia Comănesci win any Gold medals in the Olympics?
A1: Nadia Comănesci is a five-time Olympic gold medalist.
Q2: What about Silver?	Did Nadia Comănesci win any Silver medals?	Did Nadia win Silver medals?	What about Silver medals?
Q1: What is range in statistics?
A1: The Range is the difference between the lowest and highest values.
Q2: Describe some different metrics to interpret it.	Describe some different metrics to interpret range in statistics.	Describe some different metrics to interpret range in statistics.	Describe some different metrics to interpret range.
Q1: What is nominal GDP?
A1: Nominal GDP is GDP evaluated at current market prices and includes all of the changes in market prices that have occurred during the current year due to inflation or deflation.
Q2: What is the difference with real?	What is the difference between nominal and real GDP?	What is the difference with real GDP?	What is the difference with real GDP?
Q1: Tell me about lavender plants?
A1: Lavandula is a genus of 47 known species of flowering plants in the mint family, Lamiaceae. It is native to the Old World and is found from Cape Verde and the Canary Islands, Europe across to northern and eastern Africa, the Mediterranean, southwest Asia to southeast India.
Q2: What are the different types?	What are the different types of lavender plants?	What are the different types of plants?	What are the different types of plants?