Improvements in the quality, usability and acceptability of spoken dialog systems can be facilitated by better evaluation methods. To support early and efficient evaluation of dialog systems and their components, this paper presents a tripartite framework describing the evaluation problem. One part models the behavior of user and system during the interaction, the second one the perception and judgment processes taking place inside the user, and the third part models what matters to system designers and service providers. The paper reviews available approaches for some of the model parts, and indicates how anticipated improvements may serve not only developers and users but also researchers working on advanced dialog functions and features.
Möller and Ward REF proposed a tripartite framework (behavior of user and system during the interaction; perception and judgment processes taking place inside the user; and what matters to system designers and service providers) to describe the evaluation problem in detail.
A Framework for Model-based Evaluation of Spoken Dialog Systems
Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
Computer Science