We investigate the use of machine learning in combination with feature engineering techniques to explore human multimodal clarification strategies and the use of those strategies for dialogue systems. We learn from data collected in a Wizardof-Oz study where different wizards could decide whether to ask a clarification request in a multimodal manner or else use speech alone. We show that there is a uniform strategy across wizards which is based on multiple features in the context. These are generic runtime features which can be implemented in dialogue systems. Our prediction models achieve a weighted f-score of 85.3% (which is a 25.5% improvement over a one-rule baseline). To assess the effects of models, feature discretisation, and selection, we also conduct a regression analysis. We then interpret and discuss the use of the learnt strategy for dialogue systems. Throughout the investigation we discuss the issues arising from using small initial Wizard-of-Oz data sets, and we show that feature engineering is an essential step when learning from such limited data.
For multimodal clarification dialog, REF investigated the use of machine learning to explore human multimodal clarification strategies and the use of those strategies to decide, based on the current dialogue context.
Using Machine Learning To Explore Human Multimodal Clarification Strategies
International Conference On Computational Linguistics And Annual Meeting Of The Association For Computational Linguistics - Poster Sessions
Computer Science