Open-ended human learning and information-seeking are increasingly mediated by technologies like digital assistants. However, such systems often fail to account for the user's pre-existing knowledge , which is a powerful way to increase engagement and to improve retention . Assuming a correlation between engagement and user responses such as "liking" messages or asking followup questions, we design a Wizard of Oz dialog task that tests the hypothesis that engagement increases when users are presented with facts that relate to their existing knowledge . Through crowd-sourcing of this experimental task we collected and now open-source 14K dialogs (181K utterances) where users and assistants converse about various aspects related to geographic entities . This dataset is annotated with pre-existing user knowledge, message-level dialog acts, message grounding to Wikipedia,  user reactions to messages , and per-dialog ratings. Our analysis shows that responses which incorporate a user's prior knowledge do increase engagement. We incorporate this knowledge into a state-of-the-art multi-task model that reproduces human assistant policies , improving over content selection baselines by 13  points.
Open-ended human learning and information-seeking are increasingly mediated by digital assistants. However, such systems often ignore the user's pre-existing knowledge . Assuming a correlation between engagement and user responses such as "liking" messages or asking followup questions, we design a Wizard-of-Oz dialog task that tests the hypothesis that engagement increases when users are presented with facts related to what they know . Through crowd-sourcing of this experiment, we collect and release 14K dialogs (181K utterances) where users and assistants converse about geographic topics like geopolitical entities and locations . This dataset is annotated with pre-existing user knowledge, message-level dialog acts, grounding to Wikipedia, and user reactions to messages . Responses using a user's prior knowledge increase engagement. We incorporate this knowledge into a multi-task model that reproduces human assistant policies and improves over a BERT content model by 13 mean reciprocal rank points.
D
technologies like
R
fail to account for
ignore
D
, which is a powerful way to increase engagement and to improve retention
R
Wizard of Oz
Wizard-of-Oz
R
that relate to their existing knowledge
related to what they know
R
experimental task we collected and now open-source
experiment, we collect and release
R
various aspects related to geographic entities
geographic topics like geopolitical entities and locations
D
message
A
and
R
, and per-dialog ratings. Our analysis shows that responses which incorporate
. Responses using
D
do
D
state-of-the-art
R
, improving over content selection baselines
and improves over a BERT content model
A
mean reciprocal rank