PATENT DOCUMENT

Publication Number: US-10417405-B2
Application Number: US-201816128337-A
Country: US
Kind Code: B2

Title: Device access using voice authentication

Abstract:
A device can be configured to receive speech input from a user. The speech input can include a command for accessing a restricted feature of the device. The speech input can be compared to a voiceprint (e.g., text-independent voiceprint) of the user&#39;s voice to authenticate the user to the device. Responsive to successful authentication of the user to the device, the user is allowed access to the restricted feature without the user having to perform additional authentication steps or speaking the command again. If the user is not successfully authenticated to the device, additional authentication steps can be request by the device (e.g., request a password).

Claims:
What is claimed is: 
     
       1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:
 receive a first speech input at the electronic device, the first speech input including a command associated with a feature of the electronic device; 
 while generating an initial text-independent voiceprint based on the first speech input, determine an action corresponding to the command; 
 store the generated text-independent voice print; 
 provide access to the feature of the device according to the command; 
 receive a second speech input at the electronic device; and 
 determine whether voice characteristics of the second speech input match voice characteristics of the stored text-independent voice print. 
 
     
     
       2. The computer-readable storage medium of  claim 1 ,
 wherein the instructions further cause the electronic device to: 
 generate voice samples based on the first speech input; 
 store the voice samples on the device; and 
 generate the initial text-independent voiceprint based on the voice samples. 
 
     
     
       3. The computer-readable storage medium of  claim 1 ,
 wherein generating an initial text-independent voiceprint based on the first speech input further comprises: 
 determining one or more phonemes based on the first speech input; and 
 generating a model based on the one or more phonemes. 
 
     
     
       4. The computer-readable storage medium of  claim 3 , wherein generating a model based on the one or more phonemes includes identifying one or more characteristics associated with a user pronunciation of the determined one or more phonemes. 
     
     
       5. The computer-readable storage medium of  claim 4 , wherein generating a model based on the one or more phonemes includes vector quantization. 
     
     
       6. The computer-readable storage medium of  claim 4 ,
 wherein generating a model based on the one or more phonemes further comprises: 
 determining one or more transitions between states associated with voice characteristics of the first speech input. 
 
     
     
       7. The computer-readable storage medium of  claim 6 , wherein the one or more transitions include stochastic Markovian transitions. 
     
     
       8. The computer-readable storage medium of  claim 6 , wherein the one or more transitions are associated with at least one voice characteristic, silence characteristic, nasal characteristic, liquid characteristic, or frication characteristic. 
     
     
       9. The computer-readable storage medium of  claim 1 ,
 wherein the initial text-independent voiceprint is generated while the device is in an unlocked state. 
 
     
     
       10. The computer-readable storage medium of  claim 1 , wherein the electronic device receives the second speech input while the electronic device is locked. 
     
     
       11. The computer-readable storage medium of  claim 1 ,
 wherein the instructions further cause the electronic device to: 
 in accordance with a determination that characteristics of the second speech input match characteristics of the stored text-independent voiceprint, perform a task; and 
 in accordance with a determination that characteristics of the second speech input do not match characteristics of the stored text-independent voiceprint, provide a notification. 
 
     
     
       12. The computer-readable storage medium of  claim 11 , wherein performing a task includes executing a command included in the second speech input. 
     
     
       13. The computer-readable storage medium of  claim 11 , wherein performing a task includes unlocking the electronic device. 
     
     
       14. The computer-readable storage medium of  claim 11 , wherein the notification includes at least one of a sound or a vibration. 
     
     
       15. The computer-readable storage medium of  claim 11 , wherein the notification includes a prompt requesting a user to provide a password or code. 
     
     
       16. The computer-readable storage medium of  claim 11 , wherein performing a task further comprises:
 obtaining biometric data associated with a user; 
 determining whether the user is authorized based on the biometric data; 
 in accordance with a determination that the user is authorized based on the biometric data, performing a task associated with the second speech input. 
 
     
     
       17. The computer-readable storage medium of  claim 16 , wherein the biometric data includes at least one of facial recognition data or finger print data. 
     
     
       18. The computer-readable storage medium of  claim 1 ,
 wherein the initial text-independent voiceprint is generated by a voiceprint service. 
 
     
     
       19. The non-transitory computer-readable storage medium of  claim 1 , wherein the instructions further cause the electronic device to:
 cause the initial text-independent voiceprint to be generated by a network server, wherein the generated text-independent voiceprint is stored at the network server. 
 
     
     
       20. The non-transitory computer-readable storage medium of  claim 1 , wherein the instructions further cause the electronic device to:
 transmit, to a network server, a first plurality of voice samples at a first time; 
 transmit, to the network server, a second plurality of voice samples at a second time; and 
 cause the initial text-independent voiceprint to be generated by the network server based on the first plurality of voice samples and the second plurality of voice samples. 
 
     
     
       21. The non-transitory computer-readable storage medium of  claim 20 , wherein the first plurality of voice samples and the second plurality of voice samples are transmitted to the network server during a period when the electronic device is idle or when the electronic device is using less than a predetermined amount of resources. 
     
     
       22. An electronic device, comprising:
 one or more processors; 
 a memory; and 
 one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
 receiving a first speech input at the electronic device, the first speech input including a command associated with a feature of the electronic device; 
 while generating an initial text-independent voiceprint based on the first speech input, determining an action corresponding to the command; 
 storing the generated text-independent voice print; 
 providing access to the feature of the device according to the command; 
 receiving a second speech input at the electronic device; and 
 determining whether voice characteristics of the second speech input match voice characteristics of the stored text-independent voice print. 
 
 
     
     
       23. The electronic device of  claim 22 , wherein the one or more programs further include instructions for:
 generating voice samples based on the first speech input; 
 storing the voice samples on the device; and 
 generating the initial text-independent voiceprint based on the voice samples. 
 
     
     
       24. The electronic device of  claim 22 , wherein generating the initial text-independent voiceprint based on the first speech input further comprises:
 determining one or more phonemes based on the first speech input; and 
 generating a model based on the one or more phonemes. 
 
     
     
       25. The electronic device of  claim 24 , wherein generating a model based on the one or more phonemes includes identifying one or more characteristics associated with a user pronunciation of the determined one or more phonemes. 
     
     
       26. The electronic device of  claim 25 , wherein generating a model based on the one or more phonemes includes vector quantization. 
     
     
       27. The electronic device of  claim 25 , wherein generating a model based on the one or more phonemes further comprises:
 determining one or more transitions between states associated with voice characteristics of the first speech input. 
 
     
     
       28. The electronic device of  claim 27 , wherein the one or more transitions include stochastic Markovian transitions. 
     
     
       29. The electronic device of  claim 27 , wherein the one or more transitions are associated with at least one voice characteristic, silence characteristic, nasal characteristic, liquid characteristic, or frication characteristic. 
     
     
       30. The electronic device of  claim 22 , wherein the initial text-independent voiceprint is generated while the device is in an unlocked state. 
     
     
       31. The electronic device of  claim 22 , wherein the electronic device receives the second speech input while the electronic device is locked. 
     
     
       32. The electronic device of  claim 22 , wherein the one or more programs further include instructions for:
 in accordance with a determination that characteristics of the second speech input match characteristics of the stored text-independent voiceprint, performing a task; and 
 in accordance with a determination that characteristics of the second speech input do not match characteristics of the stored text-independent voiceprint, providing a notification. 
 
     
     
       33. The electronic device of  claim 32 , wherein performing a task includes executing a command included in the second speech input. 
     
     
       34. The electronic device of  claim 32 , wherein performing a task includes unlocking the electronic device. 
     
     
       35. The electronic device of  claim 32 , wherein the notification includes at least one of a sound or a vibration. 
     
     
       36. The electronic device of  claim 32 , wherein the notification includes a prompt requesting a user to provide a password or code. 
     
     
       37. The electronic device of  claim 32 , wherein performing a task further comprises:
 obtaining biometric data associated with a user; 
 determining whether the user is authorized based on the biometric data; 
 in accordance with a determination that the user is authorized based on the biometric data, performing a task associated with the second speech input. 
 
     
     
       38. The electronic device of  claim 37 , wherein the biometric data includes at least one of facial recognition data or finger print data. 
     
     
       39. The electronic device of  claim 22 , wherein the initial text-independent voiceprint is generated by a voiceprint service. 
     
     
       40. The electronic device of  claim 22 , wherein the one or more programs further include instructions for:
 causing the initial text-independent voiceprint to be generated by a network server, wherein the generated text-independent voiceprint is stored at the network server. 
 
     
     
       41. The electronic device of  claim 22 , wherein the one or more programs further include instructions for:
 transmitting, to a network server, a first plurality of voice samples at a first time; 
 transmitting, to the network server, a second plurality of voice samples at a second time; and 
 causing the initial text-independent voiceprint to be generated by the network server based on the first plurality of voice samples and the second plurality of voice samples. 
 
     
     
       42. The electronic device of  claim 41 , wherein the first plurality of voice samples and the second plurality of voice samples are transmitted to the network server during a period when the electronic device is idle or when the electronic device is using less than a predetermined amount of resources. 
     
     
       43. A method, comprising:
 at an electronic device with one or more processors and memory:
 receiving a first speech input at the electronic device, the first speech input including a command associated with a feature of the electronic device; 
 while generating an initial text-independent voiceprint based on the first speech input, determining an action corresponding to the command; 
 storing the generated text-independent voice print; 
 providing access to the feature of the device according to the command; 
 receiving a second speech input at the electronic device; and 
 determining whether voice characteristics of the second speech input match voice characteristics of the stored text-independent voice print. 
 
 
     
     
       44. The method of  claim 43 , further comprising:
 generating voice samples based on the first speech input; 
 storing the voice samples on the device; and 
 generating the initial text-independent voiceprint based on the voice samples. 
 
     
     
       45. The method of  claim 43 , wherein generating the initial text-independent voiceprint based on the first speech input further comprises:
 determining one or more phonemes based on the first speech input; and 
 generating a model based on the one or more phonemes. 
 
     
     
       46. The method of  claim 45 , wherein generating a model based on the one or more phonemes includes identifying one or more characteristics associated with a user pronunciation of the determined one or more phonemes. 
     
     
       47. The method of  claim 46 , wherein generating a model based on the one or more phonemes includes vector quantization. 
     
     
       48. The method of  claim 46 , wherein generating a model based on the one or more phonemes further comprises:
 determining one or more transitions between states associated with voice characteristics of the first speech input. 
 
     
     
       49. The method of  claim 48 , wherein the one or more transitions include stochastic Markovian transitions. 
     
     
       50. The method of  claim 48 , wherein the one or more transitions are associated with at least one voice characteristic, silence characteristic, nasal characteristic, liquid characteristic, or frication characteristic. 
     
     
       51. The method of  claim 43 , wherein the initial text-independent voiceprint is generated while the device is in an unlocked state. 
     
     
       52. The method of  claim 43 , wherein the electronic device receives the second speech input while the electronic device is locked. 
     
     
       53. The method of  claim 43 , further comprising:
 in accordance with a determination that characteristics of the second speech input match characteristics of the stored text-independent voiceprint, performing a task; and 
 in accordance with a determination that characteristics of the second speech input do not match characteristics of the stored text-independent voiceprint, providing a notification. 
 
     
     
       54. The method of  claim 53 , wherein performing a task includes executing a command included in the second speech input. 
     
     
       55. The method of  claim 53 , wherein performing a task includes unlocking the electro device. 
     
     
       56. The method of  claim 53 , wherein the notification includes at least one of a sound or a vibration. 
     
     
       57. The method of  claim 53 , wherein the notification includes a prompt requesting a user to provide a password or code. 
     
     
       58. The method of  claim 53 , wherein performing a task further comprises:
 obtaining biometric data associated with a user; 
 determining whether the user is authorized based on the biometric data; 
 in accordance with a determination that the user is authorized based on the biometric data, performing a task associated with the second speech input. 
 
     
     
       59. The method of  claim 58 , wherein the biometric data includes at least one of facial recognition data or finger print data. 
     
     
       60. The method of  claim 43 , wherein the initial text-independent voiceprint is generated by a voiceprint service. 
     
     
       61. The method of  claim 43 , further comprising:
 causing the initial text-independent voiceprint to be generated by a network server, wherein the generated text-independent voiceprint is stored at the network server. 
 
     
     
       62. The method of  claim 43 , further comprising:
 transmitting, to a network server, a first plurality of voice samples at a first time; 
 transmitting, to the network server, a second plurality of voice samples at a second time; and 
 causing the initial text-independent voiceprint to be generated by the network server based on the first plurality of voice samples and the second plurality of voice samples. 
 
     
     
       63. The method of  claim 62 , wherein the first plurality of voice samples and the second plurality of voice samples are transmitted to the network server during a period when the electronic device is idle or when the electronic device is using less than a predetermined amount of resources.

Description:
CROSS REFERENCE TO RELATED APPLICATION 
     This application is a continuation of U.S. patent application Ser. No. 14/987,580, filed Jan. 4, 2016, entitled DEVICE ACCESS USING VOICE AUTHENTICATION, which is a continuation of U.S. patent application Ser. No. 13/053,144, filed. Mar. 21, 2011, entitled DEVICE ACCESS USING VOICE AUTHENTICATION, the contents of each of which are hereby incorporated by reference in their entirety for all purposes. 
    
    
     TECHNICAL FIELD 
     The disclosure generally relates to techniques for controlling user access to features of an electronic device. 
     BACKGROUND 
     Many of today&#39;s computers and other electronic devices include a feature that allows a user to lock the computer or device from access by others. Some of the devices provide a mechanism for unlocking a locked device through a graphical user interface of the device. For example, the graphical user interface can provide a mechanism that allows a user to input authentication information, such as a password or code. 
     Some computers and other electronic devices can provide voice command features. For example, a user of a device can speak a voice command into a microphone coupled to the device. When the voice command is received by the device, the device can recognize and execute the voice command. 
     SUMMARY 
     A device can be configured to receive speech input from a user. The speech input can include a command for accessing a restricted feature of the device. The speech input can be compared to a voiceprint (e.g., text-independent voiceprint) of the user&#39;s voice to authenticate the user to the device. Responsive to successful authentication of the user to the device, the user is allowed access to the restricted feature without the user having to perform additional authentication steps or speaking the command again. If the user is not successfully authenticated to the device, additional authentication steps can be requested by the device (e.g., request a password). 
     In some implementations, a voiceprint can be generated for an authorized user of a device. For example, one or more samples of the user&#39;s voice can be collected as the user speaks voice commands into the device. A voiceprint can be generated based on the one or more voice samples. The voiceprint can be generated locally on the device or by a network voiceprint service (e.g., network server). The voiceprint can be used with a text-independent voice authentication process running on the device or hosted by the network service to authenticate the user to the device. 
     Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages. A device can include a more user-friendly authentication process for accessing a locked device. A user&#39;s voice can be authenticated at the same time that a voice command is processed; no separate authentication step is required. The device can generate a voiceprint while the user speaks voice commands into the device; no separate speaker recognition training step is required. The voice authentication features disclosed below can provide fast and secure voice control access to any/all features of the device. 
     Details of one or more implementations are set forth in the accompanying drawings and the description below. Other features, aspects, and potential advantages will be apparent from the description and drawings, and from the claims. 
    
    
     
       DESCRIPTION OF DRAWINGS 
         FIG. 1  illustrates an example device configured for processing voice commands. 
         FIG. 2  is flow diagram of an example process for generating a voiceprint. 
         FIG. 3  illustrates an example locked device that can be configured for voice authentication. 
         FIG. 4  is a flow diagram of an example process for voice authentication. 
         FIG. 5  is a block diagram of an example network operating environment. 
         FIG. 6  is a block diagram of an example implementation of the mobile device of  FIGS. 1-4 . 
     
    
    
     DETAILED DESCRIPTION 
     Voice Commands 
       FIG. 1  illustrates an example device  100  configured for processing voice commands. For example, device  100  can be a mobile device, such as a cell phone, smart phone, electronic tablet, television system, personal data assistant, a laptop or any other mobile device. Device  100  can be a desktop computer or any other device that can require a user to authenticate the user to the device  100 . In some implementations, device  100  can receive speech input, determine a command based on the speech input, and execute the command. For example, a user can activate a voice control feature of device  100  by pressing and holding down button  102 . When activated, the voice control feature can display a voice control graphical user interface on touch sensitive display  104 , for example, as displayed in  FIG. 1 . A user can cancel the voice control feature by pressing cancel button  106  displayed in menu bar  114 . 
     In some implementations, when the voice control feature is activated, device  100  can receive speech input from a user through microphone  108 . In some implementations, the speech input can be translated into text representing the words spoken in the speech input. For example, speech recognition analysis or modeling (e.g., Hidden Markov modeling (HMM), dynamic time warping (DTW), etc.) can be performed on the speech input to generate text that represents the content of the speech input. 
     In some implementations, the text generated from the speech input can be analyzed to determine a command to invoke a feature of device  100 . For example, if the text includes the word “call,” device  100  can determine that the user wants to make a phone call and can invoke a telephony application. If the text includes the word “play,” device  100  can determine that the user wants to play media stored on device  100  and can invoke a media player to play content, such as music or a movie, for example. 
     In some implementations, the voice control feature of device  100  can provide feedback to the user to indicate the success or failure of device  100  to determine the command. For example, the feedback (e.g., audio, visual, vibration) can indicate to the user what command is about to be executed on the device, whether the device  100  was successful in determining a command based on the speech input, and/or whether the command was successfully executed by device  100 . For example, a voice generated by the device can tell the user what command is about to be executed by the device. 
     In some implementations, voice control features of device  100  can only be accessed when the device is in an unlocked state (e.g., when the user accessing the device has been authenticated). 
     Voiceprinting 
       FIG. 2  is flow diagram of an example process  200  for generating a voiceprint. In some implementations, device  100  can be configured to generate a voiceprint for a user based on speech inputs received by device  100 . For example, device  100  can collect one or more samples of the user&#39;s voice white the user is interacting with voice control features of device  100 . In some implementations, device  100  can use the voiceprint in a text-independent voice authentication process to authenticate a user to device  100 . 
     In some implementations, generating a voiceprint can be performed only when device  100  is in an unlocked state. For example, generating a voiceprint can be performed only when the user providing the speech input has been authenticated to device  100  as the owner or an authorized user of device  100  to prevent generating a voiceprint based on an unauthorized user&#39;s or intruder&#39;s voice. 
     At step  202 , a speech input is obtained. In some implementations, device  100  can be configured to receive speech input through microphone  102  coupled to device  100 . Microphone  102  can generate audio data from the speech input. In some implementations, device  100  can be configured to collect one or more voice samples from the audio data and transmit the voice samples to a remote voiceprint service. 
     At step  204 , a voiceprint is generated. For example, the one or more voice samples can be analyzed and/or modeled to generate a voiceprint of an authorized user of device  100  based on unique information about the user&#39;s vocal tract and the behavior of the user&#39;s speaking patterns. In some implementations, the voiceprint can be generated at device  100 . For example, the audio data can be processed by device  100  to generate a voiceprint that can be used to recognize an authorized user&#39;s voice during speaker authentication. In some implementations, the voiceprint can be generated at a remote or networked service. For example, device  100  can be configured to collect one or more voice samples from audio data and transmit the voice samples to voiceprint service  508  of  FIG. 5 . For example, voice samples can be collected over time from multiple speech inputs and the voice samples can be transmitted in batches to voiceprint service  508 . The voice sample batches can be transmitted to voiceprint service  508  during periods when device  100  is idle or experiencing low resource usage, for example. Voiceprint service  508  can be configured to generate a voiceprint (e.g., a text-independent voiceprint) based on the samples received from device  100 . Voiceprint service  508  can transmit the generated voiceprint to device  100  to be used by device  100  when authenticating a user using speaker recognition analysis. 
     In some implementations, device  100 , or remote voiceprint service  508 , can include a voiceprint module that can learn the “signature” or “print” of a person&#39;s voice in a text-independent way. For example, statistical models of the characteristics of the spectral features present in a user&#39;s pronunciation of various phonemes can be built to distinguish voice characteristics of different user&#39;s voices. For example, Vector Quantization (VQ) codebook-based techniques can be employed to generate a voiceprint. Ergodic-HMM-based methods that analyze the stochastic Marchovian transitions between states to build learned models of voice characteristics such as voicing, silence, stop burst, nasal/liquid, frication, etc., can be used to generate a voiceprint, for example. In some implementations, a two-pass speaker recognition approach can be used that first explicitly determines phonemes or phoneme classes from the audio data from a speech input and then performs speaker verification by a weighted combination of matches for each recognized phoneme category. 
     The text-independent speaker authentication processes described above can provide voice authentication without requiring a specific passphrase or particular word for voice authentication. By contrast, text-dependent speaker verification processes often require specific passphrases or word utterances to perform speaker recognition and, therefore, often require a separate authentication step (e.g., challenge-response step) that requires a user to speak a particular word or phrase. The text-independent authentication process does not require a separate challenge-response authentication step. 
     In some implementations, once the voiceprint is generated, the voiceprint can be stored at device  100 . For example, if device  100  generates the voiceprint, the voiceprint can be stored in memory or non-volatile storage (e.g., a hard drive) coupled to device  100 . If the voiceprint is generated by a network server (e.g., by the voiceprint service  508 ), device  100  can receive the network generated voiceprint and store the voiceprint in memory or non-volatile storage. The network server can also store voiceprints that it generates. 
     At step  206 , a command is determined based on the speech input. In some implementations, the speech input can be processed to determine a command corresponding to the voiceprint. For example, the speech input can be translated into text using speech-to-text processing and the text can be analyzed to identify a command using speech recognition processing. For example, once the speech input is translated into text, the text of the speech input can be compared to text associated with commands known to device  100  to determine if any of the speech input text corresponds (e.g., matches) to the command text. If a textual correspondence is found, in whole or in part, in the speech input, device  100  can execute the command corresponding to the command text that corresponds to the speech input text. 
     In some implementations, the command can be determined while the voiceprint is generated. For example, once the speech input is received by device  100 , the speech input can be processed to (e.g., processed in parallel) generate a voiceprint and determine a voice command. Thus, a single speech input can be used to generate a voiceprint and to issue a voice command. 
     At step  208 , the command is executed. For example, once a command is determined based on the speech input, the command can be executed by device  400 . 
     Security Features 
       FIG. 3  illustrates an example locked device  100  that can be configured for voice authentication. For example, device  100  can be locked (e.g., in a state requiring authentication of a user) to prevent unauthorized access to features (e.g., the entire device, individual applications, etc.) or information stored on device  100 . In some implementations, individual features of device  100  can be locked. For example, individual features of device  100  can require authentication of a user before device  100  allows access to the features. Authentication of a user can be required by the device to ensure that the user accessing the device is the owner or an authorized user of the device. 
     In some implementations, device  100  can require a user to authenticate that the user is an authorized user of device  100  before granting access to device  100  or individual features of device  100 . For example, touch sensitive display  104  can display a user interface that allows a user to enter a passcode to unlock device  100 . A user can enter a passcode (e.g., a four digit number, word, sequence of characters) using touch sensitive key pad  302  to cause device  100  to unlock. Other user authentication and device unlocking mechanisms (e.g., voice authentication, face recognition, fingerprint recognition) are also possible. 
     In some implementations, when an unauthenticated user (e.g., a user that has not been authenticated yet) attempts to access features of or provide input to device  100 , authentication of the user can be performed. For example, when a user attempts to place a telephone call, access an e-mail application, address book or calendar on a password locked device, the user interface of  FIG. 3  can be presented to the user to allow the user to enter a password, code, or other user authenticating input. In some implementations, if the user enters a password or code that is known to device  100 , the user can be authenticated and the device  100  and/or features of device  100  can be unlocked. If the user enters a password or code that is unknown to the device  100 , the user cannot be authenticated and device  100  and/or features of device  100  can remain locked. In some implementations, device  100  can be configured to perform voice authentication of a user, as described with reference to  FIG. 4 . 
     Voice Authentication 
       FIG. 4  is a flow diagram of an example process  400  for voice authentication. For example, voice authentication of a user can be performed when a speech input is received at a locked device by performing speaker recognition analysis on the speech input. Authentication of a user can be performed using text-independent voice authentication techniques, as described above. 
     The voice authentication features described herein can allow for fast and secure access to all of the features of and data stored on device  100 . For example, these voice authentication features can enable a user of device  100  to access features and information on device  100  in a secure way and without having to enter a passcode every time the user attempts to access device  100 . Without these voice authentication features, user access to a device can be slowed by separate authentication steps, sensitive or private user data stored on a device can be accessed by an unauthorized user or intruder to the device, or the functionality that a user can access using voice control features of the device may have to be limited to just non-private, non-sensitive information and commands, for example. 
     At step  402 , a speech input is obtained. For example, a user of locked device  100  can press and hold button  102  to activate voice control features of device  100 , even when device  100  is locked. In some implementations, device  100  can receive a speech input through microphone  108  when voice control features of device  100  are activated. 
     At step  404 , the speech input is used to perform user authentication. In some implementations, the speech input can be used to authenticate a user to device  100  using speaker recognition analysis. For example, if device  100  is locked, the voice of the speech input can be analyzed using speaker recognition analysis to determine if the user issuing the speech input is an authorized user of device  100 . For example, the voice characteristics of the voice in the speech input can be compared to voice characteristics of a voiceprint of an authorized user stored on device  100  or by a network service. If the voice can be matched to the voiceprint, the user can be authenticated as an authorized user of device  100 . If the voice cannot be matched to the voiceprint, the user will not be authenticated as an authorized user of device  100 . If a user cannot be authenticated to device  100  based on the speech input, an error message can be presented (e.g., audibly and/or visually, vibration) to the user. For example, if the user cannot be authenticated based on the speech input, device  100  can notify the user of the authentication error with sound (e.g., alarm or synthesized voice message) presented through speaker  110  or loud speaker  112  or a vibration provided by a vibrating source. Device  100  can present a visual error by presenting on touch interface  104  a prompt to the user to provide additional authentication information (e.g., password, code, touch pattern, etc.). 
     At step  406 , a command can be determined based on the speech input. As described above, the speech input can be translated to text and the text can be processed to determine a command present in the speech input. In some implementations, a user can be authenticated based on the speech input while the speech input is processed to determine the command in the speech input. That is, the user can submit a single speech input to device  100  and that single speech input can be processed to both authenticate the user and to determine which command the user wants the device to execute. 
     At step  408 , the command can be executed when the voice is authenticated. In some implementations, if the user&#39;s voice in the speech input can be matched to a voiceprint of an authorized user, the user&#39;s voice can be authenticated and the device can execute the determined command. In some implementations, device  100  can execute the determined command while device  100  is locked. For example, device  100  can remain locked while device  100  executes the command such that additional voice (or non-voice) input received by device  100  will require authentication of the user providing such input. In some implementations, locked device  100  can be unlocked in response to authenticating a user to locked device  100  using voice authentication processes described above. For example, locked, device  100  can be unlocked when a user&#39;s voice is authenticated as belonging to an authorized user of device  100  such that subsequent input or commands do not require additional authentication. 
     In some implementations, other biometric data (e.g., other than a user&#39;s voice) can be used to authenticate a user to a device or confirm the result of a voice authentication to provide more confidence of a successful voice authentication. For example, front facing camera  116  of mobile device  100  can be used to collect images of a user&#39;s face that can be used to recognize an authorized user of the device based on facial recognition analysis. As another example, the touch-sensitive display  104 , or button  120 , can be configured to collect finger print data for a user and the finger print data can be used to authenticate a user to the device. 
     In some implementations, authenticating a user using other types of biometric data can be performed passively. For example, authentication of a user can be performed while the user is interacting with the device in non-authentication-specific ways. For example, the user&#39;s fingerprint can be authenticated when the user touches the touch-sensitive display to interact with the music player object  124 . Front facing camera  116 , for example, can collect images of the user&#39;s face as the user interacts with video chat features of device  100 . Front facing camera  116  can collect images for face recognition analysis and authentication while the user is operating device  100  in other ways, such a web browsing. The collected images can be used to authenticate the user using facial recognition analysis. In some implementations, a combination of biometric data can be collected a used to authenticate a user when the user attempts to access device  100 . For example, a combination of speaker recognition, face recognition, fingerprint matching, or other biometric data can be used to authenticate a user to device  100 . 
     Example Network Operating Environment 
       FIG. 5  is a block diagram of an example network operating environment  500 . In  FIG. 5 , mobile devices  502   a  and  502   b  each can represent mobile device  100 . Mobile devices  502   a  and  502   b  can, for example, communicate over one or more wired and/or wireless networks  510  in data communication. For example, a wireless network  512 , e.g., a cellular network, can communicate with a wide area network (WAN)  514 , such as the Internet, by use of a gateway  516 . Likewise, an access device  518 , such as an 802.11g wireless access device, can provide communication access to the wide area network  514 . In some implementations, both voice and data communications can be established over the wireless network  512  and the access device  518 . For example, the mobile device  502   a  can place and receive phone calls (e.g., using VoIP protocols), send and receive e-mail messages (e.g., using POP3 protocol), and retrieve electronic documents and/or streams, such as web pages, photographs, and videos, over the wireless network  512 , gateway  516 , and wide area network  514  (e.g., using TCP/IP or UDP protocols). Likewise, in some implementations, the mobile device  502   b  can place and receive phone calls, send and receive e-mail messages, and retrieve electronic documents over the access device  518  and the wide area network  514 . In some implementations, the mobile device  502   a  or  502   b  can be physically connected to the access device  518  using one or more cables and the access device  518  can be a personal computer. In this configuration, the mobile device  502   a  or  502   b  can be referred to as a “tethered” device. 
     The mobile devices  502   a  and  502   b  can also establish communications by other means. For example, the wireless device  502   a  can communicate with other wireless devices, e.g., other mobile devices  502   a  or  502   b , cell phones, etc., over the wireless network  512 . Likewise, the mobile devices  502   a  and  502   b  can establish peer-to-peer communications  520 , e.g., a personal area network, by use of one or more communication subsystems, such as the Bluetooth™ communication devices. Other communication protocols and topologies can also be implemented. 
     The mobile device  502   a  or  502   b  can, for example, communicate with one or more services  530 ,  540 ,  550 ,  560 ,  570  and  580  over the one or more wired and/or wireless networks  510 . For example, a navigation service  530  can provide navigation information, e.g., map information, location information, route information, and other information, to the mobile device  502   a  or  502   b . A user of the mobile device  502   b  can invoke a map functionality and can request and receive a map for a particular location. 
     A messaging service  540  can, for example, provide e-mail and/or other messaging services. A media service  550  can, for example, provide access to media files, such as song files, audio books, movie files, video clips, and other media data. In some implementations, separate audio and video services (not shown) can provide access to the respective types of media files. A syncing service  560  can, for example, perform syncing services (e.g., sync files). An activation service  570  can, for example, perform an activation process for activating the mobile device  502   a  or  502   b.    
     A voiceprint service  580  can, for example, generate voiceprints that can be used to authenticate users of mobile device  502   a  or  502   b . For example, voiceprint service  580  can receive samples of a user&#39;s voice from mobile device  502   a  or  502   b  and generate a voiceprint based on the voice samples. Mobile device  502   a  or  502   b  can, for example, collect the voice samples as a user is interacting with various voice features (e.g., voice control, telephone, voice recorder, etc.) of mobile device  502   a  or  502   b . Once voiceprint service  580  has generated a voiceprint for a user, voiceprint service  580  can transmit the voiceprint to mobile device  502   a  or  502   b . Once the voiceprint is received at mobile device  502   a  or  502   b , the voiceprint can be used by the mobile device to authenticate a user based on the user&#39;s voice. The voiceprints generated by voiceprint service  580  can be text-independent voiceprints, for example. 
     Other services can also be provided, including a software update service that automatically determines whether software updates exist for software on the mobile device  502   a  or  502   b , then downloads the software updates to the mobile device  502   a  or  502   b  where the software updates can be manually or automatically unpacked and/or installed. 
     The mobile device  502   a  or  502   b  can also access other data and content over the one or more wired and/or wireless networks  510 . For example, content publishers, such as news sites, RSS feeds, web sites, blogs, social networking sites, developer networks, etc., can be accessed by the mobile device  502   a  or  502   b . Such access can be provided by invocation of a web browsing function or application (e.g., a browser) of mobile device  502   a  or  502   b.    
     Example Mobile Device Architecture 
       FIG. 6  is a block diagram  600  of an example implementation of the mobile device  100  of  FIGS. 1-4 . The mobile device  100  can include a memory interface  602 , one or more data processors, image processors and/or central processing units  604 , and a peripherals interface  606 . The memory interface  602 , the one or more processors  604  and/or the peripherals interface  606  can be separate components or can be integrated in one or more integrated circuits. The various components in the mobile device  100  can be coupled by one or more communication buses or signal lines. 
     Sensors, devices, and subsystems can be coupled to the peripherals interface  606  to facilitate multiple functionalities. For example, a motion sensor  610 , a light sensor  612 , and a proximity sensor  614  can be coupled to the peripherals interface  606  to facilitate orientation, lighting, and proximity functions. Other sensors  616  can also be connected to the peripherals interface  606 , such as a positioning system (e.g., GPS receiver), a temperature sensor, a biometric sensor, or other sensing device, to facilitate related functionalities. 
     A camera subsystem  620  and an optical sensor  622 , e.g., a charged coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) optical sensor, can be utilized to facilitate camera functions, such as recording photographs and video clips. The camera subsystem  620  and the optical sensor  622  can be used to collect images of a user to be used during authentication of a user, e.g., by performing facial recognition analysis. 
     Communication functions can be facilitated through one or more wireless communication subsystems  624 , which can include radio frequency receivers and transmitters and/or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystem  624  can depend on the communication network(s) over which the mobile device  100  is intended to operate. For example, a mobile device  100  can include communication subsystems  624  designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi or WiMax network, and a Bluetooth™ network. In particular, the wireless communication subsystems  624  can include hosting protocols such that the device  100  can be configured as a base station for other wireless devices. 
     An audio subsystem  626  can be coupled to a speaker  628  and a microphone  630  to facilitate voice-enabled functions, such as speaker recognition, voice replication, digital recording, and telephony functions. The audio subsystem  626  can be configured to facilitate processing voice commands, voiceprinting and voice authentication, as described above with reference to  FIGS. 1-4 . 
     The I/O subsystem  640  can include a touch screen controller  642  and/or other input controller(s)  644 . The touch-screen controller  642  can be coupled to a touch screen  646 . The touch screen  646  and touch screen controller  642  can, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen  646 . 
     The other input controller(s)  644  can be coupled to other input/control devices  648 , such as one or more buttons, rocker switches, thumb-wheel, infrared port, USB port, and/or a pointer device such as a stylus. The one or more buttons (not shown) can include an up/down button for volume control of the speaker  628  and/or the microphone  630 . 
     In one implementation, a pressing of the button for a first duration can disengage a lock of the touch screen  646 ; and a pressing of the button for a second duration that is longer than the first duration can turn power to the mobile device  100  on or off. Pressing the button for a third duration can activate a voice control, or voice command, module that enables the user to speak commands into the microphone  630  to cause the device to execute the spoken command. The user can customize a functionality of one or more of the buttons. The touch screen  646  can, for example, also be used to implement virtual or soft buttons and/or a keyboard. 
     In some implementations, the mobile device  100  can present recorded audio and/or video files, such as MP3, AAC, and MPEG files. In some implementations, the mobile device  100  can include the functionality of an MP3 player, such as an iPod™. The mobile device  100  can, therefore, include a 36-pin connector that is compatible with the iPod. Other input/output and control devices can also be used. 
     The memory interface  602  can be coupled to memory  650 . The memory  650  can include high-speed random access memory and/or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and/or flash memory (e.g., NAND, NOR). The memory  650  can store an operating system  652 , such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks. 
     The operating system  652  can include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, the operating system  652  can be a kernel (e.g., UNIX kernel). In some implementations, the operating system  652  can include instructions for performing voice authentication. For example, operating system  652  can implement the security lockout and voice authentication features as described with reference to  FIGS. 1-4 . Operating system  352  can implement the voiceprint and voice authentication features described with reference to  FIGS. 1-4 . 
     The memory  650  can also store communication instructions  654  to facilitate communicating with one or more additional devices, one or more computers and/or one or more servers. The memory  650  can include graphical user interface instructions  656  to facilitate graphic user interface processing; sensor processing instructions  658  to facilitate sensor-related processing and functions; phone instructions  660  to facilitate phone-related processes and functions; electronic messaging instructions  662  to facilitate electronic-messaging related processes and functions; web browsing instructions  664  to facilitate web browsing-related processes and functions; media processing instructions  666  to facilitate media processing-related processes and functions; GPS/Navigation instructions  668  to facilitate GPS and navigation-related processes and instructions; and/or camera instructions  670  to facilitate camera-related processes and functions. 
     The memory  650  can store other software instructions  672  to facilitate other processes and functions, such as the security and/or authentication processes and functions as described with reference to  FIGS. 1-4 . For example, the software instructions can include instructions for performing voice authentication on a per application or per feature basis and for allowing a user to configure authentication requirements of each application or feature available on device  100 . 
     The memory  650  can also store other software instructions (not shown), such as web video instructions to facilitate web video-related processes and functions; and/or web shopping instructions to facilitate web shopping-related processes and functions. In some implementations, the media processing instructions  666  are divided into audio processing instructions and video processing instructions to facilitate audio processing-related processes and functions and video processing-related processes and functions, respectively. An activation record and International Mobile Equipment Identity (IMEI)  674  or similar hardware identifier can also be stored in memory  650 . 
     Each of the above identified instructions and applications can correspond to a set of instructions for performing one or more functions described above. These instructions need not be implemented as separate software programs, procedures, or modules. The memory  650  can include additional instructions or fewer instructions. Furthermore, various functions of the mobile device  100  can be implemented in hardware and/or in software, including in one or more signal processing and/or application specific integrated circuits.

Metadata:
Filing Date: 20180911
Publication Date: 20190917
Grant Date: 20190917
Priority Date: 20110321
Inventors: CHEYER, ADAM JOHN
Assignee: APPLE INC
CPC Classifications: [{"code": "G10L17/00", "inventive": true, "first": false, "tree": "[]"}, {"code": "H04L63/0861", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F21/32", "inventive": true, "first": true, "tree": "[]"}, {"code": "H04L63/10", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L17/24", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F21/32", "inventive": true, "first": true, "tree": "[]"}, {"code": "G10L17/24", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F21/32", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F21/31", "inventive": true, "first": true, "tree": "[]"}, {"code": "G10L17/24", "inventive": true, "first": false, "tree": "[]"}, {"code": "H04L63/10", "inventive": true, "first": false, "tree": "[]"}, {"code": "H04L63/0861", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F21/32", "inventive": true, "first": true, "tree": "[]"}]
Family ID: 46025882