Voice as the userinterface – a new erain speech processing
Debbie Greenstreet
Product Marketing Manager
Digital Signal Processors
Texas Instruments

John Smrstik
Marketing Manager
Digital Signal Processors
Texas Instruments

As the evolution of computing continues from thedesktop era and into the mobile arena, the technology
advancements that have led to these changes are
impressive to witness.
It is hard to imagine that in nearly every person’s pocket is several times the computing
power once needed to get a crew of American astronauts to the moon and safely backhome. This computing power isn’t just embedded in our phones – it’s in our tablets,
music players, watches and appliances, and is becoming more pervasive all the time.
If technology is going to be part of everyday life,

comfortable to use, and consumers are relying more

making electronic gadgets easy to use with aon the capabilities that voice recognition has to offer.

simple, intuitive, natural user interface is necessary.
Historically, designers addressed user interfaces

Voice as the user interface

through hardware devices (a keyboard and mouse),

The focus for creating a voice-based user interface

graphical interfaces (icons, buttons), and touchmust be to make the user experience simple,

screens that combine both.

comfortable and reliable. To deliver a good

Several next-generation alternatives are emerging

experience, designers need to consider questions
such as: where the interaction will take place, how

to make interaction with computing devices moremuch information is being communicated, whether

accessible, easier and safer. Some of these are

the user is near-by or far away from the device andfamiliar, like gesture-based interfaces (Nintendo® Wii

how the interaction should happen. Each type ofor Kinect gaming systems). Others serve specificvoice-enabled system may have different answers

market niches, like a brain-computer interface thatto these questions, and any system designer shouldmaps electric signals of specific commands toattempt to account for the most likely use cases andhelp paralyzed patients interact with the world.

then design to meet the expected challenges.

The user interface technology that is arguably
showing the most promise today has been around

Voice recognition systems

for many years – voice and speech recognition.

Voice or speech recognition systems fall into two

While intelligent voice recognition (IVR) systems

main categories.

have been around since at least the 1980s,

The first and simplest is called “grammar-based”

many frustrations with the technology have kept

recognition and is based on programming thevoice as the user interface limited to applications

recognition engine with a few pre-defined rules tolike call-center management. These frustrations

drive the activity. Sometimes called discrete word

include inaccuracy and misinterpretation in speechsystems, these systems have a finite vocabulary anddetection, and the limited vocabulary used in

may not efficiently separate commands from othermenu-driven systems. As designers continue tospeech that it does not understand.

address these frustrations, they have become more

Voice as the user interface – a new era in speech processingI2

May 2017

Consider a voice-controlled portable media playercase requires the resources of a cloud infrastructure,

without a Wi-Fi or cellular connection. This player

while a smartphone is acting as a voice capture andmight respond to commands like “play,” “pause” orplayback device.

“skip.” The vocabulary of commands for this type

Some more sophisticated systems today employ®

of system could be only a few words, or potentiallyboth techniques. These systems use grammar-

dozens of words and phrases (“track nine,” “raise

based engines as a trigger for certain actions,volume,” “new playlist”). The recognition engine

and use natural language processing and cloudand its vocabulary would need to be embedded

computing resources to deliver more advancedin the device and run from local memory. These

responses. A simple command library processes

recognition engines use a pattern-matchingactions that are handled locally and that have only

algorithm that performs a “best fit” of the digitala local context. An example could be to ask asignature of the captured phrase with a reference

voice-enabled thermostat to “raise the temperaturecommand signature.

to 74 degrees.” Decoding and taking action on

The second category is “natural language”

this phrase could be handled locally with no otherrecognition. In natural language systems, theintervention. And in this example, the thermostat

recognition engine can process continuous speech,

does not need to provide an audio acknowledgment

identify key words and in some systems interpretof the temperature change.

simple context. Natural language systems are

Employing cloud speech recognition services could

more resource-intensive to implement than discrete

add richness to this interaction. As an example,

word systems, but have the potential to provide

consider this string of speech: “Thermostat, raisea richer user experience. If the system’s local

the temperature to 74 degrees. It seems cold

processing, memory and storage resources are… what is the forecast today in Dallas?” Theselarge (for example, in a PC), then a natural language

sentences could represent a simple command

system can be embedded without the need for(“raise the temperature”) while also acting as aadditional resources. However, as electronics

trigger (“thermostat”) to the system to listen toscale smaller, connected resources such as cloud-the full speech and send it to the cloud to retrieve

computing resources may be necessary in order tocurrent weather forecast information.

accommodate natural language.

This experience requires a few additional things.

A good example of a natural language system

First, the thermostat needs to be connected to theis OK Google, which requires a connection to

Internet to access the cloud. Second, there would

Google Cloud resources to provide its voice userneed to be a return path for the response, mostinterface. Locally, the voice is sampled, digitized

likely via audio playback. Although the electronics

and compressed, and then sent over a wirelessare a bit more complicated, if it improves the user

network to the cloud infrastructure where the voiceinterface of the thermostat and provides additional

processing engine resides. The voice processing

value (the weather forecast), the additional costs

engine decodes, processes and analyzes the voice

can be worth the effort.

for content, matching it to an index for a response.

Amazon Echo with Alexa Voice Service and Google

The response is re-encoded and sent back overHome are two types of systems that employ trigger

the network, and the local device plays back thephrases as well as connectivity to cloud-enabled

response. Delivering a rich user experience in this

Voice as the user interface – a new era in speech processingI

3

May 2017

voice services that comprehend natural language.

system also needs to account for echo effects

These devices are marketed as home automationthat may be present. Fortunately, digital signal

hubs that coordinate with other cooperating devices

processing (DSP) techniques commonly found inbuilt within the same software ecosystem.

traditional communication systems can improve theperformance of voice recognition systems.

Capturing the voice signal
A system’s ability to recognize speech, interpret

The far-field effectcommands, process natural language and provide

Smartphone users take for granted the performancea relevant response in proper context will makeof accurate speech processing. However, accuratelyinteractions with computing systems much simpler

filtering a voice from other audio sources among

and much more tangible for many types of users

adjacent noise – when that voice is meters awayof all abilities. The computing power stored in thefrom the microphones – is another level of processing

cloud will enable these systems to evolve further

complexity that requires a technology known as far-to produce even more enriched experiences.

field microphone arrays. This technology also employs

As evidenced by products available today, voice

signal-processing techniques such as beamforming

recognition provides a comfortable, valuable userand spatial filtering, which are commonly used tointerface for all kinds of systems.

reliably receive a directional signal.

But interpreting the voice signal is only part of the

A signal received across an array of sensor elements

technical challenge. For voice recognition systems– in this case, microphones – that are spatially

to work well, the input to the recognition engine –separated, has geometric characteristics that can

the core algorithm that maps the incoming signal

be used to improve reception of the audio signal.

to a known signature – must be clean and give theEach element receives the signal at slightly different

recognition engine the best opportunity to do thetimes based on the relative positions of the sensors

job it was created to do. Background noise and the(microphones) and the source (voice) signal. Piecing

location of the voice relative to the microphones canthese time-shifted signals together determines thecontribute to attenuation of the target signal andlocation of the source relative to the sensors.

produce errors. The presence of an audio signal in

Applying adaptive filtering techniques can effectively

the playback path, the reflection of the original voice,tune the system to listen more closely to certainand even acoustic coupling from the vibrations of

microphone sensors, while ignoring sensors farthera loudspeaker can bring additional impairments.

away from the voice. This strengthens the input

Speakerphones and other conferencing systems

signal, especially when compared to a single elementthat handle audio have dealt with many of thesethat only listens to audio coming from all directions

considerations.

(the omni-directional case). While there are certainly

Ideally, to provide a smooth user experience, adiminishing returns to the number of microphone

voice recognition system should work effectively inelements added to a system, more microphones can

the presence of ambient noise (and occasionallytheoretically provide better input that is more immune

loud background noise) as well as when theto ambient noise.

speaker’s voice is coming from relatively long

Selecting how many microphones to use and theirdistances, such as from across the room. The

Voice as the user interface – a new era in speech processing

configuration requires consideration of the expectedI

4

May 2017

use case, balanced against the practical realities ofcomponent, when subtracted from the composite

system cost. For example, a system mounted to asignal, reveals the best possible representation ofwall could assume that the voice signal of interest

the actual speech signal.

is most likely coming from a 180-degree field in

Combining spectral noise reduction withfront of the system. In this case, you might use abeamforming can improve the system’s signal-linear array of two to four microphones with outward

to-noise ratio and produce a high-quality input

pointing beams (at 90 degrees relative to thesignal to a speech recognition engine. End users

microphone array). A system designed to be used inwill experience better performance in noisy

the center of a room in which the target voice could

environments and/or while speaking farther away

be coming from a 360-degree field might employ

from the system – the far-field effect that manya circular array of six or more microphones, assystems are striving for. Dealing with environmental

shown in Figure 1. Well-thought-out choices for thefactors addresses one of the key attributes of anumber, configuration and placement of microphone

good user experience.

elements should consider how the system is likely to
be used, and can make a significant impact on the

Echo and reverberation

user experience.

Many systems implementing voice recognition also
incorporate an audio-playback path. This represents
another challenge to voice recognition systems,
as it adds other noise components. Acoustic echo
occurs when the audio being played back is picked
up by the microphone elements and sent back into
the system. It can be exacerbated when the audio
playback volume is high and when the speaker andmicrophone elements are in close proximity.
Reverberation occurs when audio-signal reflections
are fed into the microphone.
A system can’t work well without eliminating theeffects created by audio playback. See Figure 2.
Acoustic echo-cancellation algorithms and de-Figure 1. Example of a working eight-microphone system configured ina circular array delivering digital audio output.

reverberation filters are signal-processing routines
traditionally employed in conferencing, mobile and

Implementing spectral noise reduction behind the

speakerphone applications. They are finding a new

microphone array and beamforming contributes

application in speech recognition systems.

to further improvement. Spectral noise reductionuses the frequency domain to effectively remove

The embedded voice-trigger andrecognition systemambient and unwanted noise components from anaudio signal, making it easier to hear the desired

An embedded voice system can be broken up into

signal. Using adaptive filtering techniques creates anthree different functional areas. On the front end

estimate of the signal’s noise component. This noise

Voice as the user interface – a new era in speech processingI

5

May 2017

Direct speech
Reverberated speech
Echo, and reverberated Echo

Figure 2. Echo and reverberation complicate the capture of a clean voice signal at the microphone.

of the system are signal-processing components

The speech signal, once processed, is passed toresponsible for acquiring and improving the quality

an embedded speech recognition engine. Thisof the input speech signal. As discussed, thesespeech recognition engine implements a small

can include beamforming, spectral noise reduction,

vocabulary of commands or trigger words that kick off

acoustic echo cancellation and de-reverberation.

specific actions. Actions may include a set of locally

DSPs can efficiently implement these common

executed commands like “turn on ...” and usuallyalgorithms. An example of such system is shown ininclude a trigger alerting the rest of the system to

Figure 3 below.

begin listening. Siri, OK Google and Alexa are threecommon triggers in use today.

BF 1 (0°)+(ASNR or AER)

BF 2 (30°)+ (ASNR or AER)
I2S/
McBSP
/DMA

MSS

VAD
DRC

I2S
EDMA

To
SpeechRec
EngineBF 12 (330°)+(ASNR or AER)

Figure 3. Functional diagram of a far-field voice pre-processing application running on a DSP.

Voice as the user interface – a new era in speech processingI

6

May 2017

Upon acceptance of the system trigger, an Internet

in Figure 4. Depending on the desired quality ofconnection (or network connection) needs to beaudio playback (for example, playing back music),established in order to send the incoming speech to

you may want to employ audio post-processing

a network-based natural language processor. Cloud-and equalization technology to improve the listening

based systems have the amount of processing ca-

experience. There are many alternatives for audio

pability necessary to apply to recognition and context

processing; again, DSPs or standard microprocessors

interpretation, an ability to learn and update algorithms

can implement most of them easily and efficiently.

for accuracy and robustness, and the storage available
for the massive amounts of speech data that passPutting the system all together

through the system to accelerate learning. Examples

The applications for this technology are diverse,of cloud-based systems include Google Voice, Alexa

rapidly growing and expandable beyond consumer

Voice Service with Amazon Web Services (AWS) andproducts like in-home virtual assistants. HomeIBM Watson. The cloud-recognition engine processes

security, building automation and industrial-based

natural language and translates it to actions that canuses of this technology abound. However, the range

be fulfilled through web integration or applications builtof solutions needed to address voice as the user

to work with that service.

interface applications varies; no one size fits all. From

The cloud returns a response based on the action

the number of microphones, to the sophisticationof beamforming, to the need for acoustic echo

requested; in many cases this is an audio response.

cancellation, the elements are highly dependent on

An audio playback path is often part of the system,

the product requirements.

including a speaker and amplifier solution, as shown

Software Libraries
BF

MSS

ASNR

VAD

AER

DRC

SpeechRecognition

DSP

C6000™
Multicore DSP

Cloud server
for speechprocessing

Microphone arraywith ADC

Audio playback
speaker

DSP

Network module

DAC

Figure 4. A complete voice user interface with cloud-recognition services and an audio playback path.

Voice as the user interface – a new era in speech processingI

7

May 2017

Fortunately, TI has a suite of silicon and software

hardware/software suite offers a scalable productcomponents, from the analog front end to processedline that adds voice as the user interface. Finally, thespeech output, that you can integrate into a

TI Designs reference design library, including thecomplete yet efficient solution for a given application.

66AK2G02 based reference design, can help you

A variety of audio-specific analog-to-digitalquickly establish an evaluation and demonstration

converters (ADCs) are applicable to different system

platform.

needs. TI’s portfolio offers high-dynamic-range,

Summary

low-noise products that can handle single, dual
or four microphone designs in a single package.
These software-controlled ADCs feature advanced
features such as energy signal detection and infinite
impulse response (IIR) linear-phase first impulse
response (FIR) filters for interfacing with analog ordigital microphones.
TI DSPs enable speech audio pre-processing
received from multi-microphone systems. The ultralow-power C5000™ DSP family is great for batterypowered applications and capable of performing
routines like beamforming, array signal-to-noise
ratio (ASNR) and dynamic range compression (DRC)
signal processing, acoustic echo cancellation andde-reverberation, and some level of speech recognition.
The floating-point-capable C6000™ family handles
more complex applications that require more
microphones, more sophisticated noise-reduction
algorithms and even audio post-processing functions
for a playback path. These DSP families are supported
by a processor software development kit (SDK) thatprovides a common software basis for all products
and enables easier movement across the portfolio.
TI also offers field-proven software libraries that
include beamforming, ASNR, DRC, voice activity
detection (VAD) and acoustic echo removal (AER) for
incorporating and tuning to your solutions as needed.

All types of embedded systems are consideringinteraction with electronic systems with voice as
the user interface, from consumer audio devices, tohome automation systems, to industrial automationequipment. Services like Siri, Cortana, Google Now
and Alexa have shown that the user experience canbe dramatically improved relative to voice response
systems we have used in the past (consider anairline’s voice response enabled customer serviceline) to be very natural and seamless. Cloudcomputing and speech recognition services areeasily accessible, powerful and constantly improving.
Internet connectivity is nearly ubiquitous and very
easy and cost-effective to embed in systems.
Signal-processing techniques that have been usedeffectively in traditional communication systems for
decades are getting new life cleaning and enhancing
input voice signals for speech recognition engines.
These facts have encouraged device manufacturers
to add voice as the user interface and have shown
consumers that the products can work well andreliably. In the coming years, we can expect tostart interacting with more systems via voice
and observing the integration and interaction ofsystems based on patterns that we determine orare automatically learned. Playing music or retrieving
trivia answers are only the beginning; it will beCollectively, these devices and corresponding

exciting to see how voice as the user interface

software can enable audio pre-processing solutions

unlocks innovation across the industry.

optimized for your particular application. ThisFor more information please visit: www.ti.com/audio

Important Notice: The products and services of Texas Instruments Incorporated and its subsidiaries described herein are sold subject to TI’s standard terms andconditions of sale. Customers are advised to obtain the most current and complete information about TI products and services before placing orders. TI assumes
no liability for applications assistance, customer’s applications or product designs, software performance, or infringement of patents. The publication of information
regarding any other company’s products or services does not constitute TI’s approval, warranty or endorsement thereof.
The platform bar is a trademark of Texas Instruments. All other trademarks are the property of their respective owners.
© 2017 Texas Instruments Incorporated

SLYY116

IMPORTANT NOTICE FOR TI DESIGN INFORMATION AND RESOURCES
Texas Instruments Incorporated (‘TI”) technical, application or other design advice, services or information, including, but not limited to,reference designs and materials relating to evaluation modules, (collectively, “TI Resources”) are intended to assist designers who aredeveloping applications that incorporate TI products; by downloading, accessing or using any particular TI Resource in any way, you(individually or, if you are acting on behalf of a company, your company) agree to use it solely for this purpose and subject to the terms ofthis Notice.
TI’s provision of TI Resources does not expand or otherwise alter TI’s applicable published warranties or warranty disclaimers for TI
products, and no additional obligations or liabilities arise from TI providing such TI Resources. TI reserves the right to make corrections,enhancements, improvements and other changes to its TI Resources.
You understand and agree that you remain responsible for using your independent analysis, evaluation and judgment in designing yourapplications and that you have full and exclusive responsibility to assure the safety of your applications and compliance of your applications(and of all TI products used in or for your applications) with all applicable regulations, laws and other applicable requirements. You
represent that, with respect to your applications, you have all the necessary expertise to create and implement safeguards that (1)anticipate dangerous consequences of failures, (2) monitor failures and their consequences, and (3) lessen the likelihood of failures thatmight cause harm and take appropriate actions. You agree that prior to using or distributing any applications that include TI products, youwill thoroughly test such applications and the functionality of such TI products as used in such applications. TI has not conducted any
testing other than that specifically described in the published documentation for a particular TI Resource.
You are authorized to use, copy and modify any individual TI Resource only in connection with the development of applications that include
the TI product(s) identified in such TI Resource. NO OTHER LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE TOANY OTHER TI INTELLECTUAL PROPERTY RIGHT, AND NO LICENSE TO ANY TECHNOLOGY OR INTELLECTUAL PROPERTYRIGHT OF TI OR ANY THIRD PARTY IS GRANTED HEREIN, including but not limited to any patent right, copyright, mask work right, orother intellectual property right relating to any combination, machine, or process in which TI products or services are used. Information
regarding or referencing third-party products or services does not constitute a license to use such products or services, or a warranty or
endorsement thereof. Use of TI Resources may require a license from a third party under the patents or other intellectual property of thethird party, or a license from TI under the patents or other intellectual property of TI.
TI RESOURCES ARE PROVIDED “AS IS” AND WITH ALL FAULTS. TI DISCLAIMS ALL OTHER WARRANTIES OR
REPRESENTATIONS, EXPRESS OR IMPLIED, REGARDING TI RESOURCES OR USE THEREOF, INCLUDING BUT NOT LIMITED TO
ACCURACY OR COMPLETENESS, TITLE, ANY EPIDEMIC FAILURE WARRANTY AND ANY IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT OF ANY THIRD PARTY INTELLECTUAL
PROPERTY RIGHTS.
TI SHALL NOT BE LIABLE FOR AND SHALL NOT DEFEND OR INDEMNIFY YOU AGAINST ANY CLAIM, INCLUDING BUT NOT
LIMITED TO ANY INFRINGEMENT CLAIM THAT RELATES TO OR IS BASED ON ANY COMBINATION OF PRODUCTS EVEN IF
DESCRIBED IN TI RESOURCES OR OTHERWISE. IN NO EVENT SHALL TI BE LIABLE FOR ANY ACTUAL, DIRECT, SPECIAL,COLLATERAL, INDIRECT, PUNITIVE, INCIDENTAL, CONSEQUENTIAL OR EXEMPLARY DAMAGES IN CONNECTION WITH OR
ARISING OUT OF TI RESOURCES OR USE THEREOF, AND REGARDLESS OF WHETHER TI HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
You agree to fully indemnify TI and its representatives against any damages, costs, losses, and/or liabilities arising out of your noncompliance with the terms and provisions of this Notice.
This Notice applies to TI Resources. Additional terms apply to the use and purchase of certain types of materials, TI products and services.
These include; without limitation, TI’s standard terms for semiconductor products http://www.ti.com/sc/docs/stdterms.htm), evaluation
modules, and samples (http://www.ti.com/sc/docs/sampterms.htm).

Mailing Address: Texas Instruments, Post Office Box 655303, Dallas, Texas 75265
Copyright © 2017, Texas Instruments Incorporated