The Voice in the Machine: Building Computers That Understand Speech

In The Voice in the Machine, Roberto Pieraccini takes the reader on a journey through the history of speech technologies, from the speaking machine of von Kempelen in 1804 through the current uses of statistical machine learning that make possible speech recognition, artificial speech production, speaker recognition, and dialog management. Although this is more of a popular science book about the history of speech technologies than a book dedicated to voice user interface (VUI) design, HCI researchers and VUI designers will very likely find the information accessible and valuable. The fundamental goal of human factors engineering is to strive for optimal allocation of tasks to machines and humans, and you can’t do that if you don’t have a good understanding of machine and human capability. Without getting into the complex mathematics of machine learning, Pieraccini does a superb job of communicating how machines work with speech and provides enough background about human speech to aid an understanding of the capabilities and limitations of current systems. The first chapter provides background in the operating characteristics of human speech and communication. Part of the reason that no one has yet built a machine that has the same speech capabilities as humans is that there are aspects of human language that no one completely understands. Despite this, there is much about human language that we do know, and this chapter does an excellent job of summarizing that research and knowledge base. The next six chapters of the book cover the major speech technology topics, including different approaches to speech recognition, artificial speech production, statistical language models, and dialog management. There are already books and seminal papers that provide the mathematics behind modern speech engineering for technical audiences (e.g., Jelinek, 1997; Manning & Schütze, 1999; Rabiner, 1989). For the more general audience of a popular science book, Pieraccini provides figures, graphs, and accessible analogies to guide readers though complex topics such as endpoint (silence) detection; formantbased template matching; feature vectors and dynamic time warping for isolated-word recognition; and their extensions to continuous speech recognition, artificial speech production, and dialog management. The vast majority of commercial speech systems have components based on training hidden Markov models for statistical acoustic and language modeling (and for more advanced natural language understanding, dialog management). This may seem counterintuitive, but throughout the book Pieraccini shows consistent historical patterns of researchers, starting with attempts to directly model speech input and output with rules, but with continued research eventually establishing the superiority of statistical methods. The tests sponsored by the Defense Advanced Research Project Agency with evaluations by the National Institute of Standards and Technology played a key role in standardizing the comparison of different approaches to speech recognition and natural language processing, primarily through coordinating the collection of a common set of audio (a corpus) to use to train and evaluate the systems under test. From 1988 through 2006 the National Institute of Standards and Technology conducted evaluations of speech read in noise-free environments, spontaneous interactive speech, conversational speech, broadcast speech, and meeting speech (in which the system not only transcribes the speech but must recognize who is speaking). Through 2001, large corporate research centers and universities made much of the investment in developing speech technologies with the goal of eventually creating systems fluent in human speech. In Chapter 8 (“Becoming Real”), Pieraccini moves away from technology to a history of how speech recognition technologies became real programs serving real customers of real companies. The commercialization of speech technologies followed the realization that the technologies did not have to perfectly match human capabilities to be useful. With appropriate design and a constrained domain, it is possible to build applications that are actually useful. Beginning in the 1990s, a number of products became available that would let users dictate to their personal computers, initially one word at a time (e.g., Dragon Dictate, IBM VoiceType), and later allowing continuous speech with interspersed commands (e.g., Dragon NaturallySpeaking, IBM ViaVoice). During the same decade, conversational systems started to appear, beginning with Wildfire, an automated personal telephone assistant. The secret to Wildfire’s success was the way its prompts constrained what callers said in response, limiting the required capability of the speech recognizer—an early example of VUI design. In the first decade of the 21st century, the number of deployed speech applications exploded. Pieraccini notes several key drivers of this explosion. One was the past proliferation of poorly designed interactive voice response applications using