Source: https://patents.google.com/patent/US9972320B2/en
Timestamp: 2019-06-20 18:32:18
Document Index: 97016777

Matched Legal Cases: ['Application No. 15781811', 'Application No. 16207288', 'Application No. 10', 'Application No. 10', 'Application No. 2016', 'Application No. 10', 'Application No. 2016', 'Application No. 10', 'Application No. 10']

US9972320B2 - Hotword detection on multiple devices - Google Patents
Hotword detection on multiple devices Download PDF
US9972320B2
US9972320B2 US15/278,269 US201615278269A US9972320B2 US 9972320 B2 US9972320 B2 US 9972320B2 US 201615278269 A US201615278269 A US 201615278269A US 9972320 B2 US9972320 B2 US 9972320B2
predefined hotword
US15/278,269
US20180061419A1 (en
2016-08-24 Priority to US201662378869P priority Critical
2016-09-28 Application filed by Google LLC filed Critical Google LLC
2016-09-28 Assigned to GOOGLE INC. reassignment GOOGLE INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Gruenstein, Alexander H., MELENDO CASADO, DIEGO, FOERSTER, JAKOB NICOLAUS
2016-09-28 Priority to US15/278,269 priority patent/US9972320B2/en
2018-03-01 Publication of US20180061419A1 publication Critical patent/US20180061419A1/en
2018-05-15 Publication of US9972320B2 publication Critical patent/US9972320B2/en
For a speech-enabled system, the users' manner of interacting with the system is designed to be primarily, if not exclusively, by means of voice input. Consequently, the system, which potentially picks up all utterances made in the surrounding environment including those not directed to the system, must have some way of discerning when any given utterance is directed at the system as opposed, e.g., to being directed at an individual present in the environment. One way to accomplish this is to use a hotword, which by agreement among the users in the environment, is reserved as a predetermined word that is spoken to invoke the attention of the system. In an example environment, the hotword used to invoke the system's attention are the words “OK computer.” Consequently, each time the words “OK computer” are spoken, it is picked up by a microphone, conveyed to the system, which may perform speech recognition techniques or use audio features and neural networks to determine whether the hotword was spoken and, if so, awaits an ensuing command or query. Accordingly, utterances directed at the system take the general form [HOTWORD] [QUERY], where “HOTWORD” in this example is “OK computer” and “QUERY” can be any question, command, declaration, or other request that can be speech recognized, parsed and acted on by the system, either alone or in conjunction with the server via the network.
As systems that are configured to respond to hotwords become more prevalent, the likelihood of more than one being near a user when the user speaks the hotword increases. For example, a user may be in the living room watching television. A tablet may be on the table, and a phone may be on the couch next to the user. Upon speaking, “OK computer, play music,” the user likely does not want the phone, tablet, and television to begin playing music. Instead, the user only wants one of the devices to respond. When the phone, tablet, and television detect a hotword, they each communicate with a server and notify the server that the device detected a hotword. With each notification sent to the server, each device transmits a group identifier and the type of device. The group identifier is data that identifies a device group from which only one device should respond to a hotword. For example, devices in the same groom will share a group identifier.
When the server receives data from each of the phone, tablet, and television, the server compares the context data of each device to determine which one should respond to the hotword. The context data may include the capabilities of the devices, the time since each device was previously used, the previous action preformed, the location of the device, and the loudness of the detected hotword. Based on this context data, the server selects one of the devices to respond to the hotword. For example, the server may select the tablet to respond to the hotword based on the devices being located at the user's home and the fact that the user most commonly uses the tablet while at home. The server then provides an instruction to the tablet to process the user's speech. The tablet processes the “play music” command and begins to play music. The server also provides instructions to the television and phone not to process the user's speech. The phone returns to its previous state, and the television continues to play the television show without interruption.
In the example shown in FIG. 1, the user 102 is in a room with both computing device 106, for example, a mobile phone, and computing device 108, for example, a television. When the user 102 speaks a voice command, both the phone 106 and the television 108 receive through their respective microphones the audio data that corresponds to the user's voice command. Both the phone 106 and the television 108 are configured to respond to voice commands, but it is unlikely that when the user 102 speaks “OK computer, play music” that the user 102 wishes for both the phone 106 and the television 108 to begin playing music. To avoid the situation of both devices responding to a voice command, the computing devices 106 and 108 may begin by identifying other devices in the nearby vicinity that are configured to respond to voice commands that include a hotword. The nearby vicinity may include devices that are on the same local network, devices that can communicate through a short range radio signals, or devices that are in a same room.
At stage B, the user 102 speaks the utterance 104, “OK computer, play music.” The utterance 104 includes both a hotword 116, “OK computer,” and a voice command 118, “play music.” Because the user 102 is close enough to the phone 106 and the television 108, both the phone 106 and the television 108 receive, through their respective microphones, the audio data that corresponds to the utterance 102.
As the phone 106 calculates a hotword confidence score for the audio data in the buffer 120, the phone 106 compares the hotword confidence score to a hotword confidence score threshold. If the hotword confidence score satisfies the hotword confidence score threshold, then the phone 106 proceeds to stage C1. For example, if the hotword confidence score is 0.8 and the threshold is 0.7, then the phone 106 proceeds to stage C1. The hotword confidence score threshold represents a hotword confidence score that likely corresponds to detecting a hotword. In instances where a user speaks, “OK computer” clearly and is near the receiving device, the hotword confidence score may be 0.9. Hotword confidence scores may decrease if the speaker is farther from the devices or speaks less clearly. In instances where a user speaks, “Call Alice,” the hotword confidence score may be 0.1.
In some implementations and as illustrated in stage C1′, the phone 106 sends additional data to the server. The additional data may include data that indicates a time elapsed since the phone 106 was previously used and how the phone 106 was used. For example, the user 102 may have previously used the phone 106 to send an SMS message two minutes prior to the phone 106 detecting the hotword 116. The phone 106 may store a history of the prior usage of the phone 106 in the usage data 132. Each entry in the usage data 132 may include any combination of an application used, an action performed by the phone 106, an action performed by the user 102, and a timestamp. In some implementations, the phone 106 may send additional data related to more than one previous action, for example, the previous three actions.
In some implementations and in stage C1′, the phone 106 sends data that indicates a current state of the phone 106 when the phone 106 detected the hotword 116. For example, the current state may be that the phone 106 is asleep, the phone 106 is awake with the screen off or dimmed, or the phone 106 is awake and the user 102 is using the email application of the phone 106. In the example shown in FIG. 1, the phone 106 is currently in a sleep state when it detects the hotword 116.
In some implementations and as illustrated in stage C1′, the phone 106 sends data to the server 110 that indicates the loudness of the detected hotword 116. The loudness scorer 134 accesses the buffer 120 and measures the loudness of the portion of audio data that corresponds to the hotword 116. In some implementations, the loudness scorer 134 calculates the power or the root mean square of the audio data that corresponds to the hotword 116. For example, the loudness scorer 134 may calculate the loudness to be fifty decibels. In some implementations, the loudness scorer 134 normalizes the power or the root mean square of the audio data that corresponds to the hotword 116 by subtracting the power or the root mean square of the audio data in the buffer 120 that does not correspond to the hotword 116. The audio data in the buffer 120 that does not correspond to the hotword 116 may correspond to the ambient noise around the phone 108. For example, the loudness scorer 134 calculates the power or the root mean square of about one second of audio data before the hotword 116 to be thirty decibels. Therefore, the loudness difference is ten decibels. The phone 106 may then send the loudness data to the server 110. In some implementations, the phone 106 may calculate the loudness of the entire utterance 104 or of only the voice command 116. The phone 106 may then transmit data indicating the loudness of the entire utterance 104 or of only the voice command 116 to the server 110.
In some implementations and as illustrated in stage C1′, the phone 106 sends the hotword confidence score to the server 110. For example, the phone 106 calculates a hotword confidence score of 0.8 for the hotword 116 and transmits the score to the server 110. In some implementations, the phone 106 sends location data to the server 110. For example, the GPS unit on the phone 106 may detect a any combination of geographic location, geographic speed, and geographic movement, and the phone 106 may transmit that geographic location, geographic speed, and geographic movement to the server 110. As another example, the phone 106 may use triangulation to determine a geographic location. In some implementations, the speech recognizer 136 of the phone 106 may perform speech recognition on the audio data that is stored in the buffer 120 including the audio data that correspond to the hotword 116 or the voice command 118 or both. The speech recognizer 136 may calculate a speech recognition confidence score and transmit the speech recognition confidence score to the server 110. The speech recognition confidence score may reflect the likelihood that the transcription generated by the speech recognizer 136 is accurate. For example, the speech recognizer 136 may calculate a speech recognition confidence score of 0.7 and transmit the score of 0.7 to the server 110. In some implementations, the phone 106 transmits, to the server, 110 the transcription of the utterance 104 or only a portion of the utterance 104, for example, the hotword 116 or the voice command 118. For example, the speech recognizer 136 transcribes the voice command 118 and transmits “play music” to the server 110. In some implementations, the phone 106 also sends data that indicates the applications installed on the phone 106. For example, if the phone 106 has the application “Instant Movies” and “Love Music” installed, then the phone 106 may send data identifying “Instant Movies” and “Love Music.” In some implementations, the phone 106 may have many applications installed. In this case, the phone 106 may only send data identifying the most used applications, for example, the ten most frequently used applications.
In some implementations and as illustrated in stage C2′, the television 108 sends additional data to the server. The television 108 sends additional data that is similar to the additional data described above with respect to the phone 106 in stage C1′. The television 108 may store usage data 138 that is related to previous uses of the television 108 and transmit the usage data to the server 110. For example, the television 108 may transmit data indicating that the user 102 previously used the “Instant Movies” application to watch “Love Story” two hours ago. The television 108 may transmit data that indicates it current state. For example, the television 108 may transmit data indicating that the television 108 is currently showing the World News Channel and has been for the past fifteen minutes.
In some implementations, the television 108 sends a hotword confidence score. For example, the hotworder 126 may calculate a hotword confidence score of 0.75, and the television 108 transmits the score of 0.75 to the server 110. The television 108 may send its geographic location to the server 110. The speech recognizer 142 may perform speech recognition on the audio data in the buffer 124. The speech recognizer 142 may generate a transcription as well as a speech recognition confidence score. The television 108 may send the transcription or the speech recognition confidence score or both to the server 110. The television 108 may send data that indicates the applications installed on the television 108 or the most frequently installed applications. For example, the television 108 may send data that identifies the installed application of “Jungle Movies” and “Social Pictures.”
In some implementations, the server 110 may receive a transcription of the voice command 118. In this instance, the devices have already performed speech recognition on the voice command 118 and the device selector 146 selects a device to act on the voice command 118. With the transcription of the voice command, the device selector 146 can access the context data 148 to select an appropriate device to respond to the voice command. For example, if the voice command is “Call Alice,” then the device selector 146 would select the phone 106 to process the voice command because the user 102 typically uses the phone 106 to call even in the case where the television 108 is configured to process voice calls or because the phone 106 is the only device configured to process voice calls.
In some implementations, the device selector 146 may use the current or previous use of the device to select a device to respond to the voice command. The current or previous use may be related to the voice query. For example, the voice command may be, “stop playing music.” If the device selector 146 has access to context data 148 that describes one of the devices as playing music, then the device selector 146 selects the device that is playing music to respond to the voice command. As another example, the voice command may be, “show the music video again.” If the device selector 146 has access to context data 148 that indicates that one of the devices played a music video for “Rock Group” in the last ten minutes, then the device selector 146 will select that device to respond to that voice command.
In some implementations, the server 110 may receive context data related to the applications installed on the phone 106 and television 108. The device selector 146 may use the context data related to the installed applications to select a device to respond to the voice query. For example, the voice command may be “play Super Ninja.” The device selector 146 may have context information that the phone 106 has Super Ninja installed. Therefore, the device selector 146 selects the phone 106 to respond to the “play Super Ninja” command. In some implementations, the server 110 may receive data indicated installed applications if the user downloads the applications from an application server in the cloud 154 and the application server communicates with the server 110 regarding the installed applications.
In stage E, the server 110 selects a device to perform speech recognition on the voice command and then perform the voice command. In the case where the server 110 receives a transcription of the voice command, the server 110 selects a device to perform the voice command. In the example show in FIG. 1, the server 110 selects the phone 106 based on the television context data 152 that indicates that the television is currently being used, so the phone 106 may be better to respond to a new voice command. In instances where the server 110 receives a transcription of the voice command 118, the server may use context data that indicates the television 108 is showing a video while the phone 106 is idle to determine that the phone should respond to the “play music” the voice command. The device selector 146 may the use any of the criteria described above in any combination and weight each factor differently to select a device.
In some implementations, the voice command 118 associated with a single instance of a hotword may include more than one command. In certain instances, it may be appropriate for more than one device to process the commands, either serially or in parallel. For example, a user 102 may speak, “OK computer, play Love Story and call mom.” In this instance, it may be better for the phone 106 to handle the call and the television 108 to handle playing the movie “Love Story”. For the server 110 to provide instructions to both the phone 106 and the television 108, the server 110 may need the transcription of the utterance 104. With the transcription of the utterance 104, the device selector 146 may identify the two commands in the utterance 104 and the nearby devices that are best equipped to handle the commands based on the context data of each device and the capabilities data of each device. In this example, the device selector 146 may determine that the television 108 is better equipped to show “Love Story” because the television 108 has a larger screen and more powerful speakers than the phone 106. The device selector 146 may determine that the phone 106 is better equipped to handle the telephone call because the phone 106 has a cellular radio. The server 110 may then send instructions to the phone 106 to call mom and to the television 108 to play the movie “Love Story.”
In some implementations, the phone 106 or the television 108 or any other device may present an interface that permits the user 102 to enter a preference for which device responds to a hotword. For example, the user 102 may enter into the interface that when the user 102 is with the phone 106 and the television 108, then the user 102 prefers for the phone 106 to respond to a hotword. The user 102 may also be able to select an option that allows the server 110 to override the user's preference in instances where a device may be incapable of performing the voice command. In instances where the server 110 does not receive the transcription of the utterance, then the server 110 may select the device according to the user preference. As an example of a server override, the server 110 may receive hotword detection data packet from both the phone 106 and the television 108. The hotword detection data packets include the transcription of the voice command, which may be “OK computer, turn on “World News Now.” The user 102 has previously selected that when the user 102 is near the television 108 and the phone 106, that the phone 106 should respond to a voice command. However, in this case, the phone 106 is not configured to show the news program “World News Now” because the phone 106 does not have access to the service that provides that news program. Accordingly, the server 110 instructs the television 108 to respond to the hotword and voice command and the phone 106 not to respond to the hotword and voice command.
The system receives audio data that corresponds to an utterance (210). For example, the user may receive audio data that corresponds to the user speaking, “OK computer, give me directions home.” In some implementations, the system may receive from another system, data indicating that the other system is configured to respond to hotwords. The system and the other system may negotiate a group identifier that each can include in data packets to send to the server when either the system or the other system detects a hotword. For example, the group identifier may be an email address of the user logged into one or both of the devices. In some implementations, the system receives audio data when it is in a sleep state and with the screen off. In this case, the system may remain in a sleep state and still receive and process the audio data. In some implementations, the system may be in an active state, such as playing a video. In this case, the system receives and processes the audio data and continues to play the video.
The system determines that the utterance likely includes a particular, predefined hotword (220). For example, the system may determine that the utterance likely includes “OK computer.” In some implementations, the system may be in a sleep state and upon receipt of identifying the hotword, the screen of the system activates, possibly on a dimmed low power state, and indicates the system is processing audio data.
The system receives, from the server, an instruction to commence speech recognition processing on the audio data (240). For example, the system performs speech recognition on the audio data that does not correspond to the hotword and determines the transcription, “give me directions home.” In some implementations, the system may receive an instruction not to commence speech recognition on the audio data. In some implementations, the system may have already have performed speech recognition on the audio data and transmitted the transcription to the server. In this instance the system may receive an instruction to process the voice command in the transcription or not to process the voice command. If the system receives an instruction not to process the voice command, then the system may return to its previous state such as a sleep state or displaying a video. The system, in response to receiving the instruction to commence speech recognition processing on the audio data, processes at least a portion of the audio data using an automated speech recognizer on the computing device (250). The system may then execute any voice command spoken by the user. For example, the system may provide the user directions to the user's home.
The system, based on the context data of the computing device, determines that the computing device commence speech recognition processing on audio data associated with the particular, predefined hotword (330). In some implementations, the audio data corresponds to an utterance, such as, a user speaking, “OK computer, give me directions home.” In some implementations, the system selects the computing device based on any combination of the capabilities of the device, the location of the device, the time since the device was last used, the last action performed by the device, or the loudness of the audio data. In some implementations, the system may not have enough information to select a computing device. In this instance, the system may prompt the computing device to preform speech recognition on the utterance and provide a transcription of the utterance. The system may then use the context data and the transcription to select the computing device. In some implementations, the system may prompt the computing device for additional context information instead of or in addition to prompting for the transcription. For example, the system may prompt the computing device for confidence scores of the hotword and the speech recognition.
receiving, by a first computing device that is configured to respond to a particular, predefined hotword and from a second computing device that is in a vicinity of the first computing device, data indicating that the second computing device is configured to respond to the particular, predefined hotword;
transmitting, to the second computing device and by the first computing device, data indicating that the first computing device is configured to respond to the particular, predefined hotword;
determining a group identifier that identifies the first computing device and the second computing device;
receiving, by the first computing device, audio data that corresponds to an utterance;
determining that the utterance likely includes the particular, predefined hotword;
in response to determining that the utterance likely includes the particular, predefined hotword, transmitting, to a server, (i) data indicating that the first computing device likely received the particular, predefined hotword, (ii) data identifying the first computing device, and (iii) the group identifier that identifies the first computing device and the second computing device;
receiving, from the server, an instruction to commence speech recognition processing on the audio data; and
in response to receiving the instruction to commence speech recognition processing on the audio data, processing at least a portion of the audio data using an automated speech recognizer on the first computing device.
receiving, by the first computing device, additional audio data that corresponds to an additional utterance;
determining that the additional utterance likely includes the particular, predefined hotword;
in response to determining that the utterance likely includes the particular, predefined hotword, transmitting, to the server, (i) the data indicating that the first computing device likely received the particular, predefined hotword, (ii) the data identifying the first computing device, and (iii) the data identifying the group of nearby computing devices that includes the first computing device;
receiving, from the server, an instruction to not commence speech recognition processing on the audio data; and
in response to receiving the instruction to not commence speech recognition processing on the audio data, ceasing processing of the audio data using the automated speech recognizer on the first computing device.
determining a loudness of the audio data associated with the particular, predefined hotword; and
in response to determining that the utterance likely includes the particular, predefined hotword, transmitting, to the server, the loudness of the audio data associated with the particular, predefined hotword.
4. The method of claim 3, wherein determining a loudness of the audio data associated with the particular, predefined hotword comprises:
determining a power of the audio data associated with the particular, predefined hotword; and
determining a power of audio data that is not associated with the particular, predefined hotword and that the first computing device received before the audio data associated with the particular, predefined hotword,
wherein the loudness of the audio data associated with the particular, predefined hotword is based on the power of the audio data associated with the particular, predefined hotword and the power of the audio data that is not associated with the particular, predefined hotword and that the first computing device received before the audio data associated with the particular, predefined hotword.
determining a confidence score that reflects a likelihood that the audio data associated with the particular, predefined hotword corresponds to the particular, predefined hotword; and
in response to determining that the utterance likely includes the particular, predefined hotword, transmitting, to the server, confidence score.
6. The method of claim 5, wherein determining a confidence score that reflects a likelihood that the audio data associated with the particular, predefined hotword corresponds to the particular, predefined hotword comprises:
determining audio features from the audio data associated with the particular, predefined hotword; and
based on the audio features, determining, using a neural network, the confidence score.
in response to determining that the utterance likely includes the particular, predefined hotword, transmitting, to the server, data indicating a location of the first computing device.
in response to determining that the utterance likely includes the particular, predefined hotword, transmitting, to the server, data indicating an elapsed time since a previous use of the first computing device.
in response to determining that the utterance likely includes the particular, predefined hotword, transmitting, to the server, data indicating a previous action performed by the first computing device.
receiving, by a server and from a first computing device, data indicating that the first computing device is configured to respond to a particular, predefined hotword;
receiving, by the server and from a second computing device that is in a vicinity of the first computing device, data indicating that the second computing device is configured to respond to the particular, predefined hotword;
receiving, by a server and from the first device, (i) data indicating that the first computing device likely received the particular, predefined hotword, (ii) data identifying the first computing device, and (iii) the group identifier that identifies the first computing device and the second computing device;
accessing context data that indicates a context of the first computing device;
based on the context data of the first computing device, determining that the first computing device commence speech recognition processing on audio data associated with the particular, predefined hotword; and
transmitting, to the first computing device, an instruction to commence speech recognition processing on the audio data associated with the particular, predefined hotword.
receiving, by the server and from the second computing device, (i) data indicating that the second computing device likely received the particular, predefined hotword, (ii) data identifying the second computing device, and (iii) the group identifier that identifies the first computing device and the second computing device;
accessing context data that indicates a context of the second computing device;
based on the context data of the first computing device and the context data of the second computing device, determining that the second computing device not commence speech recognition processing on the audio data associated with the particular, predefined hotword; and
transmitting, to the second computing device, an instruction to not commence speech recognition processing on the audio data associated with the particular, predefined hotword.
13. The method of claim 10, wherein the context data that indicates a context of the first computing device comprises data indicating one of more capabilities of the first computing device,
wherein determining that the first computing device commence speech recognition processing on audio data associated with the particular, predefined hotword is based on the one of more capabilities of the first computing device.
14. The method of claim 10, wherein the context data that indicates a context of the first computing device comprises data indicating an elapsed time since a previous use of the first computing device,
wherein determining that the first computing device commence speech recognition processing on audio data associated with the particular, predefined hotword is based on the elapsed time since the previous use of the first computing device.
15. The method of claim 10, wherein context data that indicates a context of the first computing device comprises data indicating a previous action performed by the first computing device,
wherein determining that the first computing device commence speech recognition processing on audio data associated with the particular, predefined hotword is based on the previous action performed by the first computing device.
receiving, from the first computing device data indicating a loudness of the audio data associated with the particular, predefined hotword,
wherein determining that the first computing device commence speech recognition processing on the audio data associated with the particular, predefined hotword is further based on the loudness of the audio data associated with the particular, predefined hotword.
receiving, by the server and from the first computing device, data indicating a location of the first computing device,
wherein determining that the first computing device commence speech recognition processing on audio data associated with the particular, predefined hotword is further based on the location of the first computing device.
receiving, from the first computing device, data indicating an action associated with the audio data and data indicating actions performed by the first computing device within a particular period of time after receiving the instruction to commence speech recognition processing on the audio data associated with the particular, predefined hotword; and
updating the context data of the first computing device based on the data indicating the action associated with the audio data and the data indicating the actions performed by the first computing device within the particular period of time after receiving the instruction to commence speech recognition processing on the audio data associated with the particular, predefined hotword.
US15/278,269 2016-08-24 2016-09-28 Hotword detection on multiple devices Active US9972320B2 (en)
US201662378869P true 2016-08-24 2016-08-24
US15/278,269 US9972320B2 (en) 2016-08-24 2016-09-28 Hotword detection on multiple devices
PCT/US2017/045123 WO2018038888A1 (en) 2016-08-24 2017-08-02 Hotword detection on multiple devices
CN201780052132.0A CN109791763A (en) 2016-08-24 2017-08-02 Hot word detection in more equipment
DE202017104895.0U DE202017104895U1 (en) 2016-08-24 2017-08-15 Hotword detection at a plurality of devices
US15/952,434 US10242676B2 (en) 2016-08-24 2018-04-13 Hotword detection on multiple devices
US15/952,434 Continuation US10242676B2 (en) 2016-08-24 2018-04-13 Hotword detection on multiple devices
US20180061419A1 US20180061419A1 (en) 2018-03-01
US9972320B2 true US9972320B2 (en) 2018-05-15
ID=59579964
US15/278,269 Active US9972320B2 (en) 2016-08-24 2016-09-28 Hotword detection on multiple devices
US15/952,434 Active US10242676B2 (en) 2016-08-24 2018-04-13 Hotword detection on multiple devices
US (2) US9972320B2 (en)
CN (1) CN109791763A (en)
DE (1) DE202017104895U1 (en)
WO (1) WO2018038888A1 (en)
JPH11231896A (en) 1998-02-19 1999-08-27 Fujitsu Ten Ltd Speech startup system
JP2000310999A (en) 1999-04-26 2000-11-07 Asahi Chem Ind Co Ltd Facilities control system
JP2006227634A (en) 2006-03-29 2006-08-31 Seiko Epson Corp Equipment control method using voice recognition, equipment control system using voice recognition and recording medium which records equipment control program using voice recognition
US20070198849A1 (en) 2001-06-05 2007-08-23 Sensory, Incorporated Client-server security system and method
US20140337032A1 (en) 2013-05-13 2014-11-13 Google Inc. Multiple Recognizer Speech Recognition
US20150379989A1 (en) * 2014-06-26 2015-12-31 Nuance Communications, Inc. Voice-controlled information exchange platform, such as for providing information to supplement advertising
US20160189715A1 (en) * 2014-12-26 2016-06-30 Panasonic Intellectual Property Corporation Of America Speech recognition device and method
US20160217790A1 (en) 2014-10-09 2016-07-28 Google Inc. Hotword detection on multiple devices
2016-09-28 US US15/278,269 patent/US9972320B2/en active Active
2017-08-02 CN CN201780052132.0A patent/CN109791763A/en unknown
2017-08-02 WO PCT/US2017/045123 patent/WO2018038888A1/en active Application Filing
2017-08-15 DE DE202017104895.0U patent/DE202017104895U1/en active Active
2018-04-13 US US15/952,434 patent/US10242676B2/en active Active
KR20140031391A (en) 2011-07-01 2014-03-12 퀄컴 인코포레이티드 Identifying people that are proximate to a mobile device user via social graphs, speech models, and user context
"Speaker Recognition", Wikipedia, the free encyclopedia, 5 pages, last modified on Jan. 30, 2014; downloaded from the internet on May 15, 2014, http://en.wikipedia.org/wiki/Speaker_recognition.
Auckenthaler et al. "Score Normalization for Text-independent Speaker Verification System," Digital Signal Processing, vol. 10, 2000, 13 pages.
European Office Action issued in Application No. 15781811.3, dated Aug. 23, 2017, 6 pages.
Extended European Search Report in European Application No. 16207288.8-1914, dated Apr. 4, 2017, 8 pages.
Hurmalainen, Antti et al., "Detection, Separation and Recognition of Speech from Continuous Signals Using Spectral Factorisation", 20th European Signal Processing Conference (EUSIPCO 2012), Bucharest Romania, Aug. 27-31, 2012, 5 pages.
International Preliminary Report on Patentability in International Application No. PCT/US2015/052860, dated Apr. 11, 2017, 9 pages.
International Search Report and Written Opinion issued in International Application No. PCT/US2017/045123, dated Nov. 8, 2017, 15 pages.
Jae-Seung, Choi, "Text-dependent Speaker Recognition using Characteristic Vectors in Speech Signal and Normalized Recognition Method," Journal of Korean Institute of Information Technology, 10(5), May 2012, 7 pages (English Abstract).
Korean Office Action issued in Application No. 10-2016-7027999, dated Aug. 28, 2017, 9 pages (with English Translation.
Korean Office Action issued in Application No. 10-2016-7033161, dated Dec. 19, 2017, 10 pages (with English Translation).
Notice of Allowance in Japanese Application No. 2016-551250, dated Aug. 7, 2017, 3 pages (English Translation).
Notice of Allowance in Korean Application No. 10-2016-7021778, dated Apr. 25, 2017, (English Translation) 3 pages.
Notice of Allowance in U.S. Appl. No. 15/201,972, dated Jun. 1, 2017, 10 pages.
Office Action in Japanese Application No. 2016-551250, dated Apr. 10, 2017, 6 pages (English Translation).
Office Action issued in Korean Application No. 10-2016-7022701, dated Aug. 17, 2017, 4 pages (English Translation).
Office Action issued in Korean Application No. 10-2016-7027999, dated Apr. 12, 2017, 9 pages (English Translation).
Office Action issued in U.S. Appl. No. 15/346,914, dated Jun. 20, 2017, 25 pages.
Ramirez, Javier et al., "Statistical Voice Activity Detection Using a Multiple Observation Likelihood Ratio Test", IEEE Signal Processing Letters, vol. 12, No. 10, Oct. 2005, 4 pages.
US10255920B2 (en) * 2016-02-24 2019-04-09 Google Llc Methods and systems for detecting and processing speech signals
US10242676B2 (en) 2019-03-26
CN109791763A (en) 2019-05-21
US20180061419A1 (en) 2018-03-01
US20180286406A1 (en) 2018-10-04
DE202017104895U1 (en) 2017-11-27
WO2018038888A1 (en) 2018-03-01
AU2018201834A1 (en) 2018-04-12 Intelligent assistant for home automation
CN102971725A (en) 2013-03-13 Word-level correction of speech input
EP3347894A1 (en) 2018-07-18 Arbitration between voice-enabled devices
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MELENDO CASADO, DIEGO;GRUENSTEIN, ALEXANDER H.;FOERSTER, JAKOB NICOLAUS;SIGNING DATES FROM 20160926 TO 20160927;REEL/FRAME:039877/0853