  Computer voice is experiencing a renaissance through the growing popularity
of voice-based interfaces, agents, and environments. Yet, how to measure the
user experience (UX) of voice-based systems remains an open and urgent
question, especially given that their form factors and interaction styles tend
to be non-visual, intangible, and often considered disembodied or "body-less."
As a first step, we surveyed the ACM and IEEE literatures to determine which
quantitative measures and measurements have been deemed important for voice UX.
Our findings show that there is little consensus, even with similar situations
and systems, as well as an overreliance on lab work and unvalidated scales. In
response, we offer two high-level descriptive frameworks for guiding future
research, developing standardized instruments, and informing ongoing review
work. Our work highlights the current strengths and weaknesses of voice UX
research and charts a path towards measuring voice UX in a more comprehensive
way.
