  Recent studies on Question Answering (QA) and Conversational QA (ConvQA)
emphasize the role of retrieval: a system first retrieves evidence from a large
collection and then extracts answers. This open-retrieval ConvQA setting
typically assumes that each question is answerable by a single span of text
within a particular passage (a span answer). The supervision signal is thus
derived from whether or not the system can recover an exact match of this
ground-truth answer span from the retrieved passages. This method is referred
to as span-match weak supervision. However, information-seeking conversations
are challenging for this span-match method since long answers, especially
freeform answers, are not necessarily strict spans of any passage. Therefore,
we introduce a learned weak supervision approach that can identify a
paraphrased span of the known answer in a passage. Our experiments on QuAC and
CoQA datasets show that the span-match weak supervisor can only handle
conversations with span answers, and has less satisfactory results for freeform
answers generated by people. Our method is more flexible as it can handle both
span answers and freeform answers. Moreover, our method can be more powerful
when combined with the span-match method which shows it is complementary to the
span-match method. We also conduct in-depth analyses to show more insights on
open-retrieval ConvQA under a weak supervision setting.
