Dynamic acoustic model for vehicle

A vehicle voice processor includes a processing device and a data storage medium. The processing device is programmed to receive identification information from a wearable device, identify a speaker from the identification information, identify a dialect associated with the speaker from the identification information, select a predetermined acoustic model, and adjust the predetermined acoustic model based at least in part on the dialect identified.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to PCT/US2015/046473 titled “Dynamic Acoustic Model for Vehicle” filed on 24 Aug. 2015, the contents of which are hereby incorporated by reference in its entirety.

BACKGROUND

Vehicles with voice recognition allow an occupant to control certain vehicle functions using voice commands. Voice commands allow the occupant to control the infotainment system, the entertainment system, the climate control system, etc., by speaking certain commands understandable to the vehicle. The vehicle will process and carry out the voice commands by outputting various control signals in accordance with the voice commands received.

DETAILED DESCRIPTION

Improving how occupants interact with vehicles via speech would enhance the in-vehicle experience. A natural speech model, which allows occupants to feel as though they are communicating with their vehicle rather than simply giving it a command, is one way to improve occupant-vehicle interaction. Before natural speech models can become ubiquitous in vehicles, the vehicle must be able to recognize speech more consistently and more accurately.

One way to increase the consistency and accuracy of vehicle voice recognition systems includes modifying the way the voice recognition system processes speech. Traditional acoustic models are static and trained under a variety of conditions that would be considered typical for the automatic speech recognition (ASR) use cases. That is, traditional acoustic models are trained according to a generic person's expected speech patterns. It would be cost prohibitive, if not impossible, to include an acoustic model for every possible dialect and accent. Moreover, background noise makes it difficult for traditional acoustic models to accurately process speech.

Wearable devices may allow the vehicle to better identify and understand a particular occupant's speech patterns. An example vehicle voice processor, that can customize an acoustic model for a particular person based on data from that person's wearable device, includes a processing device and a data storage medium. The processing device is programmed to receive identification information from the wearable device, identify a speaker from the identification information, identify a dialect associated with the speaker from the identification information, select a predetermined acoustic model, and adjust the predetermined acoustic model based at least in part on the dialect identified.

Accordingly, the voice processor can dynamically reweight the acoustic model as a function of identification information by the wearable device. Since traditional acoustic models are built as a linear combination of feature vectors derived from training sets under a variety of appropriate combinations, and since many static models do a poor job of handling accented speech, the voice processor can calibrate the acoustic model to the particular speaker identified by the wearable device. Calibrating the acoustic model can include, e.g., selecting and weighting applicable feature vectors.

Thus voice processor can leverage data collected by the wearable device. The data can include basic classification information, such as race, ethnicity, primary language, etc., volunteered by the user when setting up the wearable device. Alternatively or in addition, the data can include classifications performed by machine learning algorithms to specifically identify how the user's phoneme distributions are biased. With this information, the voice processor can reweight the acoustic model to the optimal linear combination of feature vectors for the speaker, greatly improving speech recognition. Additionally, voice recognition (identification of the speaker by voice) could be used with adaptive learning paradigms built into, e.g., the infotainment system to further enhance recognition, as the infotainment system can build a profile that continually augments the feature vector weights.

The elements shown may take many different forms and include multiple and/or alternate components and facilities. The example components illustrated are not intended to be limiting. Indeed, additional or alternative components and/or implementations may be used. Further, the elements shown are not necessarily drawn to scale unless explicitly stated as such.

As illustrated inFIG. 1, the host vehicle100includes a voice processing system105in communication with a wearable device110. Although illustrated as a sedan, the host vehicle100may include any passenger or commercial automobile such as a car, a truck, a sport utility vehicle, a crossover vehicle, a van, a minivan, a taxi, a bus, etc. In some possible approaches, the host vehicle100is an autonomous vehicle configured to operate in an autonomous (e.g., driverless) mode, a partially autonomous mode, and/or a non-autonomous mode.

The wearable device110may include any number of circuits or components that permit the wearable device110to wirelessly communicate with the voice processing system105. The wearable device110may be configured to communicate using any number of wireless communication technologies such as, e.g., Bluetooth®, Bluetooth Low Energy®, Wi-Fi, Wi-Fi Direct, etc. The wearable device110may be programmed to pair with the voice processing system105, which may permit the wearable device110and the voice processing system105to exchange data. For instance, the wearable device110may be programmed to transmit identification information, associated with the person wearing the wearable device110, to the voice processing system105. The identification information may include, e.g., an identity of the speaker. The identity of the speaker may be based on, e.g., information provided by the speaker when the wearable device110is set up. Setting up the wearable device110may include generating a profile and associating the profile to the wearable device110. The identification may include, e.g., a unique identifier associated with the speaker, and the unique identifier may be transmitted to the host vehicle100along with the identification information.

The identification information may further include dialect information. For instance, the wearable device110may engage in an ongoing “training” function where the wearable device110continuously attempts to understand the user's speech by, e.g., matching spoken phonemes to expected phonemes. The differences between the spoken phonemes and the expected phonemes may be characterized as the speaker's dialect. The dialect information, therefore, may identify the speaker's dialect or another representation of the spoken phonemes relative to the expected phonemes.

The voice processing system105may be programmed to pair with, and receive identification information from, the wearable device110. The voice processing system105may process the identification information to identify the speaker. With the speaker identified, the voice processing system105may select an acoustic model. The acoustic model, which may be referred to as a “predetermined acoustic model,” may be a standard model incorporated into the voice processing system105.

The voice processing system105may further identify a dialect of the speaker. The dialect may be determined from the identity of the speaker or other information about the speaker, including the dialect information transmitted from the wearable device110. For instance, dialects may be associated with different geographic regions, which could include a present geographic region of the speaker or a previous geographic region of the speaker (the geographic region where the speaker grew up or spent the most time). By way of example, one dialect may be selected for a speaker who spent the majority of his life near Boston and a different dialect may be selected for a speaker who spent the majority of his life in the southern United States. Instead of or in addition to geography, the voice processing system105may determine the speaker's dialect based on the “training” performed by the wearable device110.

The voice processing system105may adjust the predetermined acoustic model, to create a calibrated acoustic model, based on the dialect identified. Adjusting the predetermined acoustic model may include, e.g., selecting a voice feature among multiple voice features. Each voice feature may be associated with a particular phoneme. Adjusting the predetermined acoustic model may further include adjusting a weight applied to the selected voice feature. The weight applied may indicate how much influence a particular phoneme should receive in interpreting the speaker's speech. Thus, increasing the weight may make the feature more influential while decreasing the weight may make the feature less influential.

The voice processing system105may receive an acoustic signal (i.e., speech uttered by the person wearing the wearable device110) and apply the calibrated acoustic model to the acoustic signal. The voice processing system105may process the acoustic signal according to the calibrated acoustic model and generate appropriate commands, consistent with the voice commands represented by the acoustic signal, to one or more vehicle subsystems115(seeFIG. 2).

FIG. 2is a block diagram showing example components of the voice processing system105. As shown, the voice processing system105includes a communication device120, a microphone125, a data storage medium130, and a voice processor135.

The communication device120may include any number of circuits or components that facilitate communication between the wearable device110and the voice processor135. The communication device120may be programmed to communicate with the wearable device110via any number of wireless communication technologies such as, e.g., Bluetooth®, Bluetooth Low Energy®, Wi-Fi, Wi-Fi Direct, etc. The communication device120may be programmed to pair with the wearable device110and wirelessly receive identification information, including dialect information, from the wearable device110. The communication device120may be programmed to transmit the identification information to, e.g., the voice processor135.

The microphone125may include any number of circuits or components that can receive an acoustic signal, such as speech, and convert the acoustic signal into an electrical signal, which may be referred to as an “analog acoustic signal.” For instance, the microphone125may include a transducer that generates the analog acoustic signal in accordance with the speech. The microphone125may be located in, e.g., the passenger compartment of the host vehicle100. In some possible implementations, the microphone125may be configured or programmed to output the analog acoustic signal to, e.g., a signal converter so that the analog acoustic signal may be converted to a digital acoustic signal.

The data storage medium130may include any number of circuits or components that can store electronic data. In one possible approach, the data storage medium130may include computer-executable instructions. The data storage medium130may also or alternatively store acoustic models. The acoustic models may include, e.g., any number of predetermined acoustic models, which as discussed above may be standard models incorporated into the voice processing system105. Further, the data storage medium130may be programmed or configured to store one or more calibrated acoustic models.

The voice processor135may include any number of circuits or components configured or programmed to process speech. In one possible approach, the voice processor135may be programmed to receive the identification information from the communication device120and identify a speaker (i.e., the person wearing the wearable device110) from the identification information. The voice processor135may be further configured to identify a dialect associated with the speaker. The dialect may be determined from the identification information, which as discussed above may include dialect information. The voice processor135may be programmed to select one of the predetermined acoustic models stored in the data storage medium130. The selection of the predetermined acoustic model may be based on, e.g., the identification information. Further, the voice processor135may be programmed to adjust the selected predetermined acoustic model based on, e.g., the dialect information either received from the wearable device110or inferred from the identification information (e.g., inferred form a geographic region associated with the person wearing the wearable device110). As discussed above, the adjusted predetermined acoustic model may be referred to as a calibrated acoustic model, and the voice processor135may be programmed to store the calibrated acoustic model in the data storage medium130. With the calibrated acoustic model generated, the voice processor135may receive analog or digital acoustic signals in real time and apply the calibrated acoustic model to any received acoustic signals to better understand the utterances of the person speaking. If the speech includes voice commands, the voice processor135may generate and output command signals to one or more vehicle subsystems115that carry out the voice command.

FIG. 3is a block diagram300illustrating an example data flow. The wearable device110sends the identification information to the voice processing system105. At block305, the voice processing system105identifies the speaker, and at block310, the voice processing system105determines the speaker's dialect. At block315, the voice processing system105adjusts the acoustic model according to the dialect to generate the calibrated acoustic model, which is shown at block320. Speech is received at the voice processing system105, via the microphone125, and converted to the acoustic signal. The acoustic signal is passed through the calibrated acoustic model, which helps the voice processing system105to better process and interpret the speech. If the speech includes a voice command, the voice processing system105may output a command to one or more vehicle subsystems115.

FIG. 4is a block diagram400illustrating an example adjustment of an acoustic model that may be incorporated into the voice processing system105. The calibrated (i.e., adjusted) acoustic model is shown at block405. The voice processor135may apply a calibration signal410to the predetermined acoustic model. The calibration signal may identify particular changes to be made to the weight applied to one or more features, shown in blocks415A-415N. Block420indicates the programming of the voice processor135to reweight each of the features415A-415N according to the calibration signal. As discussed above, reweighting the features415A-415N may include selecting one or more of the voice features415A-415N, where each voice feature is associated with a particular phoneme, and adjusting a weight applied to any of the selected voice features. The weight applied may indicate how much influence a particular phoneme should receive in interpreting the speaker's speech. Thus, increasing the weight may make the feature more influential while decreasing the weight may make the feature less influential. The acoustic signal, represented by block425, may be passed through the calibrated acoustic model, and the output of the calibrated acoustic model405, shown at block430, may include the recognized speech.

FIG. 5is a flowchart of an example process500that may be executed by the voice processing system105to account for a particular user's speech patterns. The process500may be executed while the host vehicle100is running. For instance, the process500may begin when the host vehicle100is first turned on and continue to execute until the host vehicle100is turned off, until all passengers have exited the host vehicle100, until no wearable devices110are paired with the host vehicle100, or until the host vehicle100is otherwise no longer able to receive and process voice commands.

At block505, the voice processing system105may pair with the wearable device110. The voice processing system105may pair with the wearable device110associated with, e.g., the driver or another vehicle occupant. The pairing may be facilitated by, e.g., the communication device120.

At block510, the voice processing system105may receive identification information from the wearable device110. The identification information may, in one possible approach, include dialect information. The identification information may be received via the communication device120and transmitted to, e.g., the voice processor135.

At block515, the voice processing system105may identify the speaker. That is, the voice processing system105may process the identification information to determine who is wearing the wearable device110. In some instances, the voice processor135may identify the speaker and select a profile associated with the speaker from the data storage medium130.

At block520, the voice processing system105may identify a dialect associated with the person identified at block515. The voice processor135may, in one possible implementation, determine the dialect from, e.g., the identification information.

At block525, the voice processing system105may select one of the predetermined acoustic models. Multiple predetermined acoustic models may be stored in the data storage medium130, and the voice processor135may select one of the predetermined acoustic models from among those stored.

At block530, the voice processing system105may adjust the predetermined acoustic model selected at block525. For instance, the voice processor135may, using the dialect identified at block520or possibly other information received from the wearable device110, adjust the predetermined acoustic model to generate the calibrated acoustic model. One way to adjust the predetermined acoustic model includes selecting one or more voice feature, from among multiple voice features, and adjusting the weight that is applied to one or more of the voice features. As discussed above, each voice feature is associated with a phoneme, so adjusting the weight for a voice feature indicates the amount of influence that should be given to each phoneme. Increasing the weight may mean a more influential phoneme while reducing the weight may indicate a less influential phoneme.

At block535, the voice processing system105may receive an acoustic signal. The acoustic signal may be received via the microphone125and may represent speech uttered in the passenger compartment of the host vehicle100.

At block540, the voice processing system105may apply the calibrated model to the acoustic signal. For instance, the voice processor135may receive the acoustic signal from the microphone and apply the calibrated acoustic model generated at block530to the acoustic signal.

At decision block545, the voice processing system105may determine whether the acoustic signal includes any voice commands. The voice processor135, for instance, may make such a determination by comparing utterances represented by the acoustic signal to the weighted features, and determining whether the utterances represent phonemes associated with voice commands. If the acoustic signal includes voice commands, the process500may proceed to block550. Otherwise, the process500may return to block535.

At block550, the voice processing system105may generate and output an appropriate command signal. The voice processor135may generate the command signal associated with the voice command detected at block545. Further, the voice processor135may output the command signal to the appropriate vehicle subsystem so that the voice command can be carried out. The process500may continue at block535.

Accordingly, the disclosed voice processing system105can dynamically reweight the predetermined acoustic model as a function of identification information provided by the wearable device110. Since traditional acoustic models are built as a linear combination of feature vectors derived from training sets under a variety of appropriate combinations, and since many static models do a poor job of handling accented speech, the voice processing system105can calibrate the acoustic model to the particular speaker identified by the wearable device110. Calibrating the acoustic model can include, e.g., selecting and weighting applicable feature vectors.