Abstract: This is the application document for the 2019 Amazon Alexa competition. We give an overall vision of our conversational experience, as well as a sample conversation that we would like our dialog system to achieve by the end of the competition. We believe personalization, knowledge, and self-play are important components towards better chatbots. These are further highlighted by our detailed system architecture proposal and novelty section. Finally, we describe how we would ensure an engaging experience, how this research would impact the field, and related work.

Introduction: Prompt: What is your team’s vision for your Socialbot? How do you want your customers to feel at the end of an interaction with your socialbot? How would your team measure success in competition? Our vision is made up of the following main points: 1. A natural, engaging, and knowledge-powered conversational experience. Made possible by a socialbot that can handle all kinds of topics and topic switching more naturally than current Alexa bots. Our goal is not necessarily for the user to feel like they are talking to a human. 2. More natural topic handling and topic switching. Incorporating knowledge into neural models BIBREF0 and using the Amazon topical chat dataset can help improve current socialbots in this aspect. 3. Building a deeper, more personalized connection with the user. We believe that offering a personalized experience is equally as important as being able to talk about a wide range of topics BIBREF1. 4. Consistency. Consistency is another important aspect of conversations which we want to take into account through our user models. 5. Diversity and interestingness. The socialbot should give diverse and interesting responses, and the user should never feel like it is merely repeating what it has said earlier. At the end of an interaction customers should feel like they just had a fun conversation, maybe learned something new, and are thrilled to talk to the bot again. Throughout the dialog, customers should feel like the socialbot is interested in them and their topics, and can offer valuable insight and opinions. It is also important for it to suggest relevant topics in an engaging way. Users should never feel like the bot is not interested or can’t continue a conversation. This is a reason behind classifying and calculating our metrics for each user input, to get an idea of user engagement in the current conversation. Our main measures for success are: - User feedback. - Comparison to other dialog systems in A/B tests. - Automatic metrics. We would measure success partly by looking at the user feedback. We expect our socialbot’s ratings to constantly increase, and verbal feedback to get more positive throughout the competition. We plan to classify verbal feedback with a simple sentiment classifier to quantitatively see the rate of improvement. Working back from the customer and constantly improving the conversational experience based on feedback is important to us. Success would also be measured by comparing our system to previous socialbots or other dialog systems in A/B tests with crowdsourced evaluators. Our goal is to have long and high-quality conversations, but the longevity shouldn’t come from awkwardly long, specific, and forced replies, as is the case with some of the current socialbots. While generally, a longer conversation is better, it is not the only metric that we wish to consider. Besides user ratings we also have a plethora of automatic metrics that we want to improve on, like metrics measuring topic depth and breadth BIBREF2, entropy metrics measuring diversity, or embedding metrics measuring coherence BIBREF3. Different metrics measure different aspects of responses, thus it is important to not solely look at metrics individually.

Question: How large is the Dialog State Tracking Dataset?
Answer: 1,618 training dialogs, 500 validation dialogs, and 1,117 test dialogs