PATENT DOCUMENT

Publication Number: US-10438595-B2
Application Number: US-201816155662-A
Country: US
Kind Code: B2

Title: Speaker identification and unsupervised speaker adaptation techniques

Abstract:
Systems and processes for generating a speaker profile for use in performing speaker identification for a virtual assistant are provided. One example process can include receiving an audio input including user speech and determining whether a speaker of the user speech is a predetermined user based on a speaker profile for the predetermined user. In response to determining that the speaker of the user speech is the predetermined user, the user speech can be added to the speaker profile and operation of the virtual assistant can be triggered. In response to determining that the speaker of the user speech is not the predetermined user, the user speech can be added to an alternate speaker profile and operation of the virtual assistant may not be triggered. In some examples, contextual information can be used to verify results produced by the speaker identification process.

Claims:
What is claimed is: 
     
       1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions for operating a virtual assistant, which when executed by one or more processors of an electronic device, cause the device to:
 receive current user speech for activating the virtual assistant, wherein the current user speech is associated with current contextual data; 
 select, based on the current contextual data, a first set of stored voiceprints from a plurality of sets of stored voiceprints in a speaker profile of the device, wherein the first set of stored voiceprints is annotated to indicate first contextual data; 
 determine whether a current voiceprint derived from the current user speech matches the first set of stored voiceprints within a predetermined threshold; and 
 in accordance with a determination that the current voiceprint matches the first set of stored voiceprints within the predetermined threshold:
 add the current voiceprint to the first set of stored voiceprints in the speaker profile; 
 annotate the current voiceprint to indicate the first contextual data; and 
 activate the virtual assistant to process a spoken command received subsequent to the user speech. 
 
 
     
     
       2. The non-transitory computer-readable storage medium of  claim 1 , wherein the speaker profile is associated with a user providing the current user speech. 
     
     
       3. The non-transitory computer-readable storage medium of  claim 1 , wherein each voiceprint of the plurality of sets of stored voiceprints of the speaker profile was generated from user speech received prior to receiving the current user speech. 
     
     
       4. The non-transitory computer-readable storage medium of  claim 1 , wherein the instructions, when executed by the one or more processors, further cause the device to:
 in accordance with a determination that the current voiceprint does not match the first set of stored voiceprints within the predetermined threshold, add the current voiceprint to an alternate speaker profile. 
 
     
     
       5. The non-transitory computer-readable storage medium of  claim 1 , wherein the instructions, when executed by the one or more processors, further cause the device to:
 in accordance with a determination that the current voiceprint does not match the first set of stored voiceprints within the predetermined threshold:
 determine whether the current voiceprint derived from the current user speech matches, within a second predetermined threshold, a second set of stored voiceprints in a second speaker profile of the device; and 
 in accordance with a determination that the current voiceprint matches the second set of stored voiceprints within the second predetermined threshold, add the current voiceprint to the second set of stored voiceprints in the second speaker profile. 
 
 
     
     
       6. The non-transitory computer-readable storage medium of  claim 5 , wherein the second speaker profile is associated with a user different from the user associated with the speaker profile. 
     
     
       7. The non-transitory computer-readable storage medium of  claim 1 , wherein the instructions, when executed by the one or more processors, further cause the device to:
 in accordance with a determination that the current voiceprint matches the first set of stored voiceprints within the predetermined threshold:
 perform speech-to-text conversion on the spoken command; 
 determine a user intent based on the spoken command; 
 determine a task to be performed based on the user intent; 
 determine a parameter for the task to be performed based on the spoken command; and 
 perform the task in accordance with the determined parameter. 
 
 
     
     
       8. The non-transitory computer-readable storage medium of  claim 1 , wherein the predetermined threshold is based on a number of voiceprints in the first set of stored voiceprints. 
     
     
       9. The non-transitory computer-readable storage medium of  claim 1 , wherein the instructions, when executed by the one or more processors, further cause the device to:
 after adding the current voiceprint to the first set of stored voiceprints in the speaker profile:
 determine whether a predetermined condition indicating an erroneous speaker determination is met; and 
 in accordance with a determination that the predetermined condition is met, remove the current voiceprint from the first set of stored voiceprints in the speaker profile. 
 
 
     
     
       10. The non-transitory computer-readable storage medium of  claim 9 , wherein determining whether the predetermined condition is met comprises receiving a user request to cancel activation of the virtual assistant. 
     
     
       11. A method for operating a virtual assistant, the method comprising:
 at an electronic device having a processor and memory:
 receiving current user speech for activating the virtual assistant, wherein the current user speech is associated with current contextual data; 
 selecting, based on the current contextual data, a first set of stored voiceprints from a plurality of sets of stored voiceprints in a speaker profile of the device, wherein the first set of stored voiceprints is annotated to indicate first contextual data; 
 determining whether a current voiceprint derived from the current user speech matches the first set of stored voiceprints within a predetermined threshold; and 
 in accordance with a determination that the current voiceprint matches the first set of stored voiceprints within the predetermined threshold:
 adding the current voiceprint to the first set of stored voiceprints in the speaker profile; 
 annotating the current voiceprint to indicate the first contextual data; and 
 activating the virtual assistant to process a spoken command received subsequent to the user speech. 
 
 
 
     
     
       12. The method of  claim 11 , wherein the speaker profile is associated with a user providing the current user speech. 
     
     
       13. The method of  claim 11 , wherein each voiceprint of the plurality of sets of stored voiceprints of the speaker profile was generated from user speech received prior to the current user speech. 
     
     
       14. The method of  claim 11 , further comprising:
 in accordance with a determination that the current voiceprint does not match the first set of stored voiceprints within the predetermined threshold, adding the current voiceprint to an alternate speaker profile. 
 
     
     
       15. The method of  claim 11 , further comprising:
 in accordance with a determination that the current voiceprint does not match the first set of stored voiceprints within the predetermined threshold:
 determining whether the current voiceprint derived from the current user speech matches, within a second predetermined threshold, a second set of stored voiceprints in a second speaker profile of the device; and 
 in accordance with a determination that the current voiceprint matches the second set of stored voiceprints within the second predetermined threshold, adding the current voiceprint to the second set of stored voiceprints in the second speaker profile. 
 
 
     
     
       16. The method of  claim 15 , wherein the second speaker profile is associated with a user different from the user associated with the speaker profile. 
     
     
       17. The method of  claim 11 , further comprising:
 in accordance with a determination that the current voiceprint matches the first set of stored voiceprints within the predetermined threshold:
 performing speech-to-text conversion on the spoken command; 
 determining a user intent based on the spoken command; 
 determining a task to be performed based on the user intent; 
 determining a parameter for the task to be performed based on the spoken command; and 
 performing the task in accordance with the determined parameter. 
 
 
     
     
       18. The method of  claim 11 , wherein the predetermined threshold is based on a number of voiceprints in the first set of stored voiceprints. 
     
     
       19. The method of  claim 11 , further comprising:
 after adding the current voiceprint to the first set of stored voiceprints in the speaker profile:
 determining whether a predetermined condition indicating an erroneous speaker determination is met; and 
 in accordance with a determination that the predetermined condition is met, removing the current voiceprint from the first set of stored voiceprints in the speaker profile. 
 
 
     
     
       20. The method of  claim 19 , wherein determining whether the predetermined condition is met comprises receiving a user request to cancel activation of the virtual assistant. 
     
     
       21. An electronic device, comprising:
 one or more processors; 
 a memory; and 
 one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
 receiving current user speech for activating the virtual assistant, wherein the current user speech is associated with current contextual data; 
 selecting, based on the current contextual data, a first set of stored voiceprints from a plurality of sets of stored voiceprints in a speaker profile of the device, wherein the first set of stored voiceprints is annotated to indicate first contextual data; 
 determining whether a current voiceprint derived from the current user speech matches the first set of stored voiceprints within a predetermined threshold; and 
 in accordance with a determination that the current voiceprint matches the first set of stored voiceprints within the predetermined threshold:
 adding the current voiceprint to the first set of stored voiceprints in the speaker profile; 
 annotating the current voiceprint to indicate the first contextual data; and 
 activating the virtual assistant to process a spoken command received subsequent to the user speech. 
 
 
 
     
     
       22. The electronic device of  claim 21 , wherein the speaker profile is associated with a user providing the current user speech. 
     
     
       23. The electronic device of  claim 21 , wherein each of the stored voice prints of the speaker profile was generated from user speech received prior to the current user speech. 
     
     
       24. The electronic device of  claim 21 , wherein the one or more programs further include instructions for:
 in accordance with a determination that the current voiceprint does not match the first set of stored voiceprints within the predetermined threshold, adding the current voiceprint to an alternate speaker profile. 
 
     
     
       25. The electronic device of  claim 21 , wherein the one or more programs further include instructions for:
 in accordance with a determination that the current voiceprint does not match the first set of stored voiceprints within the predetermined threshold:
 determining whether the current voiceprint derived from the current user speech matches, within a second predetermined threshold, a second set of stored voiceprints in a second speaker profile of the device; and 
 in accordance with a determination that the current voiceprint matches the second set of stored voiceprints within the second predetermined threshold, adding the current voiceprint to the second set of stored voiceprints in the second speaker profile. 
 
 
     
     
       26. The electronic device of  claim 25 , wherein the second speaker profile is associated with a user different from the user associated with the speaker profile. 
     
     
       27. The electronic device of  claim 21 , wherein the one or more programs further include instructions for:
 in accordance with a determination that the current voiceprint matches the first set of stored voiceprints within the predetermined threshold:
 performing speech-to-text conversion on the spoken command; 
 determining a user intent based on the spoken command; 
 determining a task to be performed based on the user intent; 
 determining a parameter for the task to be performed based on the spoken command; and 
 performing the task in accordance with the determined parameter. 
 
 
     
     
       28. The electronic device of  claim 21 , wherein the predetermined threshold is based on a number of voiceprints in the first set of stored voiceprints. 
     
     
       29. The electronic device of  claim 21 , wherein the one or more programs further include instructions for:
 after adding the current voiceprint to the first set of stored voiceprints in the speaker profile:
 determining whether a predetermined condition indicating an erroneous speaker determination is met; and 
 in accordance with a determination that the predetermined condition is met, removing the current voiceprint from the first set of stored voiceprints in the speaker profile. 
 
 
     
     
       30. The electronic device of  claim 21 , wherein determining whether the predetermined condition is met comprises receiving a user request to cancel activation of the virtual assistant. 
     
     
       31. The non-transitory computer-readable storage medium of  claim 1 , wherein the first set of stored voiceprints is selected in accordance with a determination that the current contextual data matches the first contextual data associated with the first set of stored voice prints. 
     
     
       32. The method of  claim 11 , wherein the first set of stored voiceprints is selected in accordance with a determination that the current contextual data matches the first contextual data associated with the first set of stored voice prints. 
     
     
       33. The electronic device of  claim 21 , wherein the first set of stored voiceprints is selected in accordance with a determination that the current contextual data matches the first contextual data associated with the first set of stored voice prints.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application is a continuation of U.S. patent application Ser. No. 14/835,169, filed Aug. 25, 2015, entitled SPEAKER IDENTIFICATION AND UNSUPERVISED SPEAKER ADAPTATION TECHNIQUES, which claims priority from U.S. Provisional Ser. No. 62/057,990, filed on Sep. 30, 2014, entitled SPEAKER IDENTIFICATION AND UNSUPERVISED SPEAKER ADAPTATION TECHNIQUES, which is hereby incorporated by reference in its entirety for all purposes. 
    
    
     FIELD 
     This relates generally to natural language processing and, more specifically, to performing speaker identification for natural language processing. 
     BACKGROUND 
     Intelligent automated assistants (or virtual assistants) provide an intuitive interface between users and electronic devices. These assistants can allow users to interact with devices or systems using natural language in spoken and/or text forms. For example, a user can access the services of an electronic device by providing a spoken user input in natural language form to a virtual assistant associated with the electronic device. The virtual assistant can perform natural language processing on the spoken user input to infer the user&#39;s intent and operationalize the user&#39;s intent into tasks. The tasks can then be performed by executing one or more functions of the electronic device, and a relevant output can be returned to the user in natural language form. 
     Some natural language processing systems can perform speaker identification to verify the identity of a user. These systems typically require the user to perform an enrollment process during which the user speaks a series of predetermined words or phrases to allow the natural language processing system to model the user&#39;s voice. While this process can be used to effectively model the user&#39;s voice, it can produce unreliable results if the user speaks in an unnatural manner during the enrollment process and/or if the user performs the enrollment process in an acoustic environment that is different than those in which the speaker identification is later performed. Thus, improved processes for modeling a user&#39;s voice are desired. 
     SUMMARY 
     Systems and processes for generating a speaker profile for use in performing speaker identification for a virtual assistant are provided. One example process can include receiving an audio input including user speech and determining whether a speaker of the user speech is a predetermined user based on a speaker profile for the predetermined user. In response to determining that the speaker of the user speech is the predetermined user, the user speech can be added to the speaker profile and operation of the virtual assistant can be triggered. In response to determining that the speaker of the user speech is not the predetermined user, the user speech can be added to an alternate speaker profile and operation of the virtual assistant may not be triggered. In some examples, contextual information can be used to verify results produced by the speaker identification process. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  illustrates an exemplary environment in which a virtual assistant can operate according to various examples. 
         FIG. 2  illustrates an exemplary user device according to various examples. 
         FIG. 3  illustrates an exemplary process for operating a virtual assistant according to various examples. 
         FIG. 4  illustrates an exemplary process for operating a virtual assistant in a speaker profile generating mode according to various examples. 
         FIG. 5  illustrates an exemplary process for operating a virtual assistant in a speaker profile modifying mode according to various examples. 
         FIG. 6  illustrates an exemplary process for operating a virtual assistant in a static speaker profile mode according to various examples. 
         FIG. 7  illustrates an exemplary process for performing speaker identification according to various examples. 
         FIG. 8  illustrates a functional block diagram of an electronic device configured to generate a speaker profile based on a spoken user input according to various examples. 
     
    
    
     DETAILED DESCRIPTION 
     In the following description of examples, reference is made to the accompanying drawings in which it is shown by way of illustration specific examples that can be practiced. It is to be understood that other examples can be used and structural changes can be made without departing from the scope of the various examples. 
     This relates to generating a speaker profile for use in performing speaker identification for a virtual assistant. One example process can include receiving an audio input including user speech and determining whether a speaker of the user speech is a predetermined user based on a speaker profile for the predetermined user. In response to determining that the speaker of the user speech is the predetermined user, the user speech can be added to the speaker profile and operation of the virtual assistant can be triggered. In response to determining that the speaker of the user speech is not the predetermined user, the user speech can be added to an alternate speaker profile and operation of the virtual assistant may not be triggered. In some examples, contextual information can be used to verify results produced by the speaker identification process. 
     System Overview 
       FIG. 1  illustrates exemplary system  100  for implementing a virtual assistant according to various examples. The terms “virtual assistant,” “digital assistant,” “intelligent automated assistant,” or “automatic digital assistant” can refer to any information processing system that interprets natural language input in spoken and/or textual form to infer user intent, and performs actions based on the inferred user intent. For example, to act on an inferred user intent, the system can perform one or more of the following: identifying a task flow with steps and parameters designed to accomplish the inferred user intent; inputting specific requirements from the inferred user intent into the task flow; executing the task flow by invoking programs, methods, services, APIs, or the like; and generating output responses to the user in an audible (e.g., speech) and/or visual form. 
     A virtual assistant can be capable of accepting a user request at least partially in the form of a natural language command, request, statement, narrative, and/or inquiry. Typically, the user request seeks either an informational answer or performance of a task by the virtual assistant. A satisfactory response to the user request can include provision of the requested informational answer, performance of the requested task, or a combination of the two. For example, a user can ask the virtual assistant a question, such as “Where am I right now?” Based on the user&#39;s current location, the virtual assistant can answer, “You are in Central Park.” The user can also request the performance of a task, for example, “Please remind me to call Mom at 4 p.m. today.” In response, the virtual assistant can acknowledge the request and then create an appropriate reminder item in the user&#39;s electronic schedule. During the performance of a requested task, the virtual assistant can sometimes interact with the user in a continuous dialogue involving multiple exchanges of information over an extended period of time. There are numerous other ways of interacting with a virtual assistant to request information or performance of various tasks. In addition to providing verbal responses and taking programmed actions, the virtual assistant can also provide responses in other visual or audio forms (e.g., as text, alerts, music, videos, animations, etc.). 
     An example of a virtual assistant is described in Applicants&#39; U.S. Utility application Ser. No. 12/987,982 for “Intelligent Automated Assistant,” filed Jan. 10, 2011, the entire disclosure of which is incorporated herein by reference. 
     As shown in  FIG. 1 , in some examples, a virtual assistant can be implemented according to a client-server model. The virtual assistant can include a client-side portion executed on a user device  102 , and a server-side portion executed on a server system  110 . User device  102  can include any electronic device, such as a mobile phone, tablet computer, portable media player, desktop computer, laptop computer, PDA, television, television set-top box, wearable electronic device, or the like, and can communicate with server system  110  through one or more networks  108 , which can include the Internet, an intranet, or any other wired or wireless public or private network. The client-side portion executed on user device  102  can provide client-side functionalities, such as user-facing input and output processing and communications with server system  110 . Server system  110  can provide server-side functionalities for any number of clients residing on a respective user device  102 . 
     Server system  110  can include one or more virtual assistant servers  114  that can include a client-facing I/O interface  122 , one or more processing modules  118 , data and model storage  120 , and an I/O interface to external services  116 . The client-facing I/O interface  122  can facilitate the client-facing input and output processing for virtual assistant server  114 . The one or more processing modules  118  can utilize data and model storage  120  to determine the user&#39;s intent based on natural language input, and perform task execution based on inferred user intent. In some examples, virtual assistant server  114  can communicate with external services  124 , such as telephony services, calendar services, information services, messaging services, navigation services, and the like, through network(s)  108  for task completion or information acquisition. The I/O interface to external services  116  can facilitate such communications. 
     Server system  110  can be implemented on one or more standalone data processing devices or a distributed network of computers. In some examples, server system  110  can employ various virtual devices and/or services of third party service providers (e.g., third-party cloud service providers) to provide the underlying computing resources and/or infrastructure resources of server system  110 . 
     Although the functionality of the virtual assistant is shown in  FIG. 1  as including both a client-side portion and a server-side portion, in some examples, the functions of the assistant can be implemented as a standalone application installed on a user device. In addition, the division of functionalities between the client and server portions of the virtual assistant can vary in different examples. For instance, in some examples, the client executed on user device  102  can be a thin-client that provides only user-facing input and output processing functions, and delegates all other functionalities of the virtual assistant to a backend server. 
     User Device 
       FIG. 2  is a block diagram of a user-device  102  according to various examples. As shown, user device  102  can include a memory interface  202 , one or more processors  204 , and a peripherals interface  206 . The various components in user device  102  can be coupled together by one or more communication buses or signal lines. User device  102  can further include various sensors, subsystems, and peripheral devices that are coupled to the peripherals interface  206 . The sensors, subsystems, and peripheral devices gather information and/or facilitate various functionalities of user device  102 . 
     For example, user device  102  can include a motion sensor  210 , a light sensor  212 , and a proximity sensor  214  coupled to peripherals interface  206  to facilitate orientation, light, and proximity sensing functions. One or more other sensors  216 , such as a positioning system (e.g., a GPS receiver), a temperature sensor, a biometric sensor, a gyroscope, a compass, an accelerometer, and the like, are also connected to peripherals interface  206 , to facilitate related functionalities. 
     In some examples, a camera subsystem  220  and an optical sensor  222  can be utilized to facilitate camera functions, such as taking photographs and recording video clips. Communication functions can be facilitated through one or more wired and/or wireless communication subsystems  224 , which can include various communication ports, radio frequency receivers and transmitters, and/or optical (e.g., infrared) receivers and transmitters. An audio subsystem  226  can be coupled to speakers  228  and a microphone  230  to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording, and telephony functions. 
     In some examples, user device  102  can further include an I/O subsystem  240  coupled to peripherals interface  206 . I/O subsystem  240  can include a touch screen controller  242  and/or other input controller(s)  244 . Touch-screen controller  242  can be coupled to a touch screen  246 . Touch screen  246  and the touch screen controller  242  can, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, such as capacitive, resistive, infrared, and surface acoustic wave technologies, proximity sensor arrays, and the like. Other input controller(s)  244  can be coupled to other input/control devices  248 , such as one or more buttons, rocker switches, a thumb-wheel, an infrared port, a USB port, and/or a pointer device such as a stylus. 
     In some examples, user device  102  can further include a memory interface  202  coupled to memory  250 . Memory  250  can include any electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, a portable computer diskette (magnetic), a random access memory (RAM) (magnetic), a read-only memory (ROM) (magnetic), an erasable programmable read-only memory (EPROM) (magnetic), a portable optical disc such as CD, CD-R, CD-RW, DVD, DVD-R, or DVD-RW, or flash memory such as compact flash cards, secured digital cards, USB memory devices, memory sticks, and the like. In some examples, a non-transitory computer-readable storage medium of memory  250  can be used to store instructions (e.g., for performing some or all of process  300 ,  400 ,  500 ,  600 , or  700 , described below) for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device, and execute the instructions. In other examples, the instructions (e.g., for performing process  300 ,  400 ,  500 ,  600 , or  700 , described below) can be stored on a non-transitory computer-readable storage medium of server system  110 , or can be divided between the non-transitory computer-readable storage medium of memory  250  and the non-transitory computer-readable storage medium of server system  110 . In the context of this document, a “non-transitory computer readable storage medium” can be any medium that can contain or store the program for use by or in connection with the instruction execution system, apparatus, or device. 
     In some examples, the memory  250  can store an operating system  252 , a communication module  254 , a graphical user interface module  256 , a sensor processing module  258 , a phone module  260 , and applications  262 . Operating system  252  can include instructions for handling basic system services and for performing hardware dependent tasks. Communication module  254  can facilitate communicating with one or more additional devices, one or more computers, and/or one or more servers. Graphical user interface module  256  can facilitate graphic user interface processing. Sensor processing module  258  can facilitate sensor related processing and functions. Phone module  260  can facilitate phone-related processes and functions. Application module  262  can facilitate various functionalities of user applications, such as electronic-messaging, web browsing, media processing, navigation, imaging, and/or other processes and functions. 
     As described herein, memory  250  can also store client-side virtual assistant instructions (e.g., in a virtual assistant client module  264 ) and various user data  266  (e.g., user-specific vocabulary data, preference data, and/or other data, such as the user&#39;s electronic address book, to-do lists, shopping lists, etc.) to provide the client-side functionalities of the virtual assistant. 
     In various examples, virtual assistant client module  264  can be capable of accepting voice input (e.g., speech input), text input, touch input, and/or gestural input through various user interfaces (e.g., I/O subsystem  240 , audio subsystem  226 , or the like) of user device  102 . Virtual assistant client module  264  can also be capable of providing output in audio (e.g., speech output), visual, and/or tactile forms. For example, output can be provided as voice, sound, alerts, text messages, menus, graphics, videos, animations, vibrations, and/or combinations of two or more of the above. During operation, virtual assistant client module  264  can communicate with the virtual assistant server using communication subsystem  224 . 
     In some examples, virtual assistant client module  264  can utilize the various sensors, subsystems, and peripheral devices to gather additional information from the surrounding environment of user device  102  to establish a context associated with a user, the current user interaction, and/or the current user input. In some examples, virtual assistant client module  264  can provide the contextual information or a subset thereof with the user input to the virtual assistant server to help infer the user&#39;s intent. The virtual assistant can also use the contextual information to determine how to prepare and deliver outputs to the user. 
     In some examples, the contextual information that accompanies the user input can include sensor information, such as lighting, ambient noise, ambient temperature, images or videos of the surrounding environment, distance to another object, and the like. The contextual information can further include information associated with the physical state of user device  102  (e.g., device orientation, device location, device temperature, power level, speed, acceleration, motion patterns, cellular signal strength, etc.) or the software state of user device  102  (e.g., running processes, installed programs, past and present network activities, background services, error logs, resources usage, etc.). Any of these types of contextual information can be provided to the virtual assistant server  114  as contextual information associated with a user input. 
     In some examples, virtual assistant client module  264  can selectively provide information (e.g., user data  266 ) stored on user device  102  in response to requests from the virtual assistant server  114 . Virtual assistant client module  264  can also elicit additional input from the user via a natural language dialogue or other user interfaces upon request by virtual assistant server  114 . Virtual assistant client module  264  can pass the additional input to virtual assistant server  114  to help virtual assistant server  114  in intent inference and/or fulfillment of the user&#39;s intent expressed in the user request. 
     In various examples, memory  250  can include additional instructions or fewer instructions. Furthermore, various functions of user device  102  can be implemented in hardware and/or in firmware, including in one or more signal processing and/or application specific integrated circuits. 
     Speaker Identification 
       FIG. 3  illustrates an exemplary process  300  for operating a virtual assistant using speaker identification according to various examples. In some examples, process  300  can be performed by a system similar or identical to system  100  having a user device similar or identical to user device  102  configured to implement a virtual assistant capable of continuously (or intermittently over an extended period of time) monitoring an audio input for a receipt of a trigger phrase that initiates activation of the virtual assistant. For example, a user device implementing the virtual assistant can continuously or intermittently monitor sounds, speech, and the like detected by a microphone of the user device without performing an action, such as performing a task flow, generating an output response in an audible (e.g., speech) and/or visual form, or the like, in response to the monitored sounds and speech. However, in response to detecting the trigger phrase, the virtual assistant can perform a speaker identification process to ensure that the speaker of the trigger phrase is the intended operator of the virtual assistant. Upon verification of the identity of the speaker, the virtual assistant can be activated, causing the virtual assistant to process a subsequently received word or phrase and to respond accordingly. A more detailed description of process  300  is provided below. 
     At block  302  of process  300 , an audio input including user speech can be received at a user device. In some examples, a user device (e.g., user device  102 ) can receive the audio input including user speech via a microphone (e.g., microphone  230 ). The microphone can convert the audio input into an analog or digital representation, and provide audio data representing the audio input to one or more processors (e.g., processor(s)  204 ) of the user device. 
     At block  304 , speech-to-text conversion can be performed on the audio input received at block  302  to determine whether the audio input includes user speech containing a predetermined trigger phrase. The trigger phrase can include any desired set of one or more predetermined words, such as “Hey Siri.” The trigger phrase can be used to activate the virtual assistant and signal to the virtual assistant that a user input, such as a request, command, or the like, will be subsequently provided. For example, a user may utter the trigger phrase “Hey Siri,” followed by the command “Call Mom,” to activate the virtual assistant and request that the virtual assistant initiate a phone call to the phone number associated with “Mom” in the user&#39;s contact list. In some examples, the speech-to-text conversion performed at block  304  can be performed locally on the user device (e.g., user device  102 ). In other examples, the speech-to-text conversion can be performed at a remote sever (e.g., server system  110 ). In these examples, the user device can transmit the audio input received at block  302  to the remote server through one or more networks (e.g., network  108 ). The remote server can perform the speech-to-text conversion on the audio input and return a textual representation of the audio input to the user device. If it is determined at block  304  that the audio input does not include the trigger phrase, process  300  can return to block  302 . Blocks  302  and  304  can be repeatedly performed at any desired interval of time to monitor the audio input for the trigger phrase. If it is instead determined at block  304  that the audio input received at block  302  includes the trigger phrase, process  300  can proceed to block  305 . 
     At block  305 , the user device can generate a speaker profile, selectively perform speaker recognition using the speaker profile, and selectively activate the virtual assistant in response to positively identifying the speaker using speaker recognition. In some examples, the speaker profile can generally include one or more voice prints generated from an audio recording of a speaker&#39;s voice. The voice prints can be generated using any desired speech recognition technique, such as by generating i-vectors to represent speaker utterances. Speaker recognition can be performed using the voice prints of a speaker profile by comparing an audio input containing user speech with the voice prints in the speaker profile. As discussed in greater detail below, block  305  can include blocks  306 ,  308 ,  310 , and  312  for allowing the user device to operate in multiple modes of operation based on a status of the speaker profile. 
     Specifically, at block  306 , the user device can select one of multiple modes in which to operate. In some examples, the multiple modes can include a speaker profile building mode (represented by block  308 ) in which a speaker&#39;s voice can be modeled to generate a speaker profile, a speaker profile modifying mode (represented by block  310 ) in which a speaker profile can be used to verify the identity of a user and in which the speaker profile can be updated based on newly received user speech, and a static speaker profile mode in which an existing speaker profile can be used to verify the identity of a user and in which the speaker profile may not be changed based on newly received user speech. A more detailed description of each of these modes is provided below with respect to  FIGS. 4-7 . 
     In some examples, the speaker profile building mode (represented by block  308 ) can be selected at block  306  if a speaker profile for a user&#39;s voice does not exist or if the speaker profile includes less than a lower threshold number of voice prints (e.g., 1, 5, 10, or other number of voice prints). In some examples, the speaker profile modifying mode (represented by block  310 ) can be selected at block  306  if the speaker profile for the user&#39;s voice includes a number of voice prints between the lower threshold number of voice prints and an upper threshold number of voice prints (e.g., 15, 20, 25, 30, or other number of voice prints). In some examples, the static speaker profile mode (represented by block  312 ) can be selected at block  306  if the speaker profile for the user&#39;s voice includes more than the upper threshold number of voice prints. 
     If it is determined at block  306  that the appropriate mode of operation is the speaker profile building mode (e.g., because no speaker profile exists or because the speaker profile includes an insufficient number of voice prints), process  300  can proceed to block  308  where the virtual assistant can operate in the speaker profile building mode.  FIG. 4  illustrates an exemplary process  400  for operating the virtual assistant in this mode. 
     At block  402 , the user device can add the audio input that was received at block  302  and determined to include the trigger phrase at block  304  to an existing speaker profile (if such a profile exists), or can be used create a new speaker profile (if no speaker profile currently exists). Adding the audio input to the speaker profile can generally include generating a voice print from the audio input (e.g., by generating an i-vector representation of the audio input) and storing the voice print in association with the speaker profile. In some examples, the stored voice print can be annotated with contextual data, such as a location where the audio input used to generate the voice print was received (e.g., based on location data generated by the user device), a time that the audio input used to generate the voice print was received, or the like. 
     At block  404 , the user device can activate the virtual assistant by processing audio input received subsequent to the audio input containing the trigger phrase. For example, block  404  can include receiving the subsequent audio input, performing speech-to-text conversion on the subsequently received audio input to generate a textual representation of user speech contained in the subsequently received audio input, determining a user intent based on the textual representation, an acting on the determined user intent by performing one or more of the following: identifying a task flow with steps and parameters designed to accomplish the determined user intent; inputting specific requirements from the determined user intent into the task flow; executing the task flow by invoking programs, methods, services, APIs, or the like; and generating output responses to the user in an audible (e.g., speech) and/or visual form. 
     After completing block  404 , process  400  can end and the operation of the user device can return to block  302  of process  300 . Blocks  302  and  304  can then be repeatedly performed until detecting another occurrence of the trigger phrase in the audio input. In response, block  306  can again be performed to determine the appropriate mode in which to operate. In some examples, the profile building mode represented by block  308  and process  400  can continue to be selected until a sufficient number of voice prints are generated for the speaker profile. As mentioned above, this can be 1, 5, 10, or any other desired number of voice prints. Thus, blocks  302 ,  304 ,  306 , and  308  (e.g., process  400 ) can repeatedly be performed until the speaker profile includes this number of voice prints. 
     Using process  400 , the virtual assistant can advantageously use utterances of the trigger phrase received at block  302  to build a speaker profile without forcing the user to perform a separate enrollment process. Instead, the virtual assistant can use speech from the user&#39;s natural interaction with the virtual assistant to generate this speaker profile. This can result in a speaker profile that more accurately represents the user&#39;s voice. Additionally, while using process  400  in the speaker profile building mode, the virtual assistant can process subsequently received audio input at block  404  without previously performing speaker identification on the user&#39;s utterance of the trigger phrase. This can be done because either a speaker profile does not exist or because an existing speaker profile contains an insufficient number of voice prints to reliably identify the speaker. 
     Once the speaker profile generated using process  400  includes the desired number of voice prints, receipt of an audio input containing the trigger phrase at block  302  can cause the user device to determine at block  306  that the appropriate mode of operation is the speaker profile modifying mode represented by block  310 . As mentioned above, in some examples, the speaker profile modifying mode can be selected if the speaker profile for the user includes a number of voice prints between a lower threshold number of voice prints (e.g., 1, 5, 10, or other number of voice prints) and an upper threshold number of voice prints (e.g., 15, 20, 25, 30, or other number of voice prints).  FIG. 5  illustrates an exemplary process  500  for operating the virtual assistant in this mode. 
     At block  502 , the user device can perform a speaker identification process on the audio input received at block  302  of process  300  to determine whether the speaker is a predetermined user (e.g., an authorized user of the device). Any desired speaker identification process can be used, such as an i-vector speaker identification process. In some examples, block  502  can include comparing the audio input received at block  302  with some or all of the voice prints of the speaker profile generated using process  400 . For instance, in some examples, block  502  can include determining whether the audio input received at block  302  matches (e.g., matches within an accepted tolerance) each of the voice prints of the speaker profile. If it is determined that the audio input matches a threshold number of the voice prints of the speaker profile, it can be determined that the speaker of the utterance contained in the audio input is the user represented by the speaker profile. The threshold number of matching voice prints can be any desired number, such as 1 voice print, 50% of the voice prints, 75% of the voice prints, all of the voice prints, or any other number depending on the desired confidence that the speaker is actually the user represented by the speaker profile. In some examples, the threshold number of matching voice prints can depend on the number of voice prints contained in the speaker profile. For example, if the speaker profile includes a large number of voice prints, block  502  can require a larger threshold number or percentage to require a closer match to the speaker profile. Conversely, if the speaker profile includes a small number of voice prints, block  502  can require a lower threshold number or percentage to require less stringent matching to the speaker profile 
     In other examples where the voice prints are annotated with contextual information, block  502  can include comparing the audio input received at block  302  with a subset of the voice prints of the speaker profile that match contextual information associated with the audio input. For example, if the audio input received at block  302  was received at a location corresponding to the user&#39;s office, block  502  can include determining whether the audio input received at block  302  matches the voice prints generated from audio data received at a location corresponding to the user&#39;s office, but can exclude determining whether the audio input received at block  302  matches the voice prints generated from audio data received at a location corresponding to the user&#39;s home or in the user&#39;s vehicle. Comparing the audio input to the speaker profile in this way can advantageously result in greater speaker identification accuracy since the conditions in which the audio input and the audio input used to generate the voice prints are more closely related. 
     In yet other examples, the user device can include a speaker profile for more than one user. In these examples, block  502  can include comparing the audio input received at block  302  with some or all of the voice prints of some or all of the speaker profiles. The determined identity of the speaker can be the user associated with the speaker profile that most closely matches the audio input. 
     Using any of the speaker identification processes described above, if it is determined at block  502  that the speaker of the audio input is the predetermined user represented by the speaker profile, process  500  can proceed to block  504 . 
     At block  504 , the audio input can be added to the speaker profile in a manner similar or identical to block  402  of process  400 . In some examples, the audio input can be added to the speaker profile to increase the total number of voice prints contained in the speaker profile. In other examples, the speaker profile can include a maximum number of voice prints and block  504  can include adding the most recently received audio input to the speaker profile and removing the voice print associated with the oldest received audio input from the speaker profile. At block  506 , the virtual assistant can be activated and subsequently received audio input can be processed in a manner similar or identical to block  404  of process  400 . After completing block  506 , the user device can return to block  302  of process  300 . 
     Referring back to block  502  of process  500 , if it is instead determined that the speaker of the audio input is not the predetermined user represented by the speaker profile, process  500  can proceed from block  502  to block  508 . At block  508 , the user device can add the audio input that was received at block  302  of process  300  and determined to not be made by the predetermined user represented by the speaker profile to an alternate speaker profile (if such a profile exists), or can be used create a new alternate speaker profile (if no alternate speaker profile currently exists). The alternate speaker profile can represent the voice of one or more users that are different than the predetermined user represented by the speaker profile. Similar to adding the audio input to the speaker profile at block  402  or  504 , adding the audio input to the alternate speaker profile can include generating a voice print from the audio input (e.g., by generating an i-vector representation of the audio input) and storing the voice print in association with the alternate speaker profile. In some examples, the stored voice print can be annotated with contextual data, such as a location where the audio input used to generate the voice print was received (e.g., based on location data generated by the user device), a time that the audio input used to generate the voice print was received, or the like. 
     In some examples, a single alternate speaker profile can be generated at block  508 . In these examples, all audio inputs determined to not be spoken by the predetermined user at block  502  can be added to this alternate speaker profile at block  508 . In other examples, separate alternate speaker profiles can be generated for each uniquely identified speaker. For example, block  508  can include comparing the audio input received at block  302  with all alternate speaker profiles to determine if the speaker of the audio input matches an existing speaker profile. If it is determined at block  508  that the speaker of the audio input matches one of the existing alternate speaker profiles, the audio input can be added to that alternate speaker profile. If it is instead determined at block  508  that the speaker of the audio input does not match one of the existing alternate speaker profiles, a new alternate speaker profile can be generated using the audio input. 
     In some examples, after one or more alternate speaker profiles are generated at block  508 , subsequent performances of block  502  can include comparing the audio input received at block  302  with the voice prints of both the speaker profile and alternate speaker profile(s). For example, it can be determined whether the audio input received at block  302  matches (e.g., within an accepted tolerance) each of the voice prints of the speaker profile and each of the voice prints of the alternate speaker profile(s). In some examples, if it is determined that the audio input matches a greater number or percentage of voice prints from the speaker profile than a number or percentage of voice prints from the alternate speaker profile(s), it can be determined that the speaker of the utterance contained in the audio input is the predetermined user represented by the speaker profile. If it is instead determined that the audio input matches a greater number or percentage of voice prints from the alternate speaker profile(s) than a number or percentage of voice prints from the speaker profile, it can be determined that the speaker of the utterance contained in the audio input is not the predetermined user represented by the speaker profile. 
     After completing block  508 , the user device can return to block  302  of process  300  without activating the virtual assistant and without processing subsequently received audio inputs. Using process  500 , the virtual assistant can perform speaker identification using the speaker profile generated using process  400  to verify that the speaker of the trigger phrase is the authorized or intended user of the virtual assistant. This can advantageously be performed to prevent unauthorized or unintended triggering of the virtual assistant. For example, performing of speaker identification can be useful in situations where the user device is located in a room with similar devices configured to activate a virtual assistant in response to the same trigger phrase. In this situation, it can be undesirable for one user&#39;s virtual assistant to activate in response to an utterance of the trigger phrase by another user. Additionally, process  500  can be used to modify or update the speaker profile with newly received speech from the user&#39;s natural interaction with the virtual assistant to update the speaker profile. Updating the speaker profile can advantageously be performed to allow the speaker profile to adapt to changes in the user&#39;s voice over time and/or account for different conditions in which the user&#39;s speech is received. 
     After completing block  506  or  508 , the operation of the user device can return to block  302  of process  300 . Blocks  302  and  304  can repeatedly be performed until detecting another occurrence of the trigger phrase in the audio input. In response, block  306  can be performed to determine the appropriate mode in which to operate. In some examples, the speaker profile modifying mode represented by block  310  and process  500  can continue to be selected until a sufficient number of voice prints are generated for the speaker profile. As mentioned above, this can be 15, 20, 25, 30, or any other desired number of voice prints. Thus, blocks  302 ,  304 ,  306 , and  310  can repeatedly be performed until the speaker profile includes this number of voice prints. 
     Once the speaker profile includes the desired number of voice prints, receipt of an audio input containing the trigger phrase at block  302  can cause the user device to determine at block  306  that the appropriate mode of operation is the static speaker profile mode represented by block  312 . As mentioned above, in some examples, the static speaker profile mode can be selected if the speaker profile for the user&#39;s voice includes a number of voice prints equal to or greater than an upper threshold number of voice prints (e.g., 15, 20, 25, 30, or other desired number of voice prints).  FIG. 6  illustrates an exemplary process  600  for operating the virtual assistant in this mode. 
     At block  602 , the user device can perform a speaker identification process on the audio input received at block  302  of process  300  in a manner similar or identical to that of block  502  of process  500 . If it is determined that the speaker of the audio input is the predetermined user represented by the speaker profile, then process  600  can proceed to block  604  without adding the audio input to a speaker profile in a manner similar or identical to block  402  of process  400  or block  504  of process  500 . At block  604 , the virtual assistant can be activated and subsequently received audio input can be processed in a manner similar or identical to block  404  of process  400  or block  506  of process  500 . If it was instead determined at block  602  that the speaker of the audio input was not the predetermined user represented by the speaker profile, process  600  can end without adding the audio input to a speaker profile and without processing subsequently received audio input. Instead, the operation of the user device can return to block  302  of process  300 . 
     Using process  600 , the virtual assistant can perform speaker identification using the speaker profile generated using process  400  and/or  500  to verify that the speaker of the trigger phrase is the authorized or intended user of the virtual assistant. This can advantageously be performed to prevent unauthorized or unintended triggering of the virtual assistant. For example, performing of speaker identification can be useful in situations where the user device is located in a room with similar devices configured to activate a virtual assistant in response to the same trigger phrase. In this situation, it can be undesirable for one user&#39;s virtual assistant to activate in response to an utterance of the trigger phrase by another user. Additionally, process  600  can forgo the adding of audio input to a speaker profile. This can be performed in situations where the speaker profile has a sufficient number of voice prints and additional voice prints may not be necessary. This can advantageously reduce the amount of storage required to store the speaker profile(s) and reduce the processing power required to add voice prints to existing speaker profile(s). 
     In some examples, the speaker identification performed at block  502  and/or  602  can include determining whether the speaker identification made using the speaker profile and/or alternate speaker profile(s) was erroneous based on contextual information.  FIG. 7  illustrates an exemplary process  700  for performing speaker identification using contextual information that can be performed at block  502  or  602 . At block  702 , a speaker identification process can be performed on the audio input received at block  302  of process  300  in a manner similar or identical to block  502  of process  500  or block  602  of process  600 . If it is determined that the speaker of the audio input is the predetermined user represented by the speaker profile, process  700  can proceed to block  704 . 
     At block  704 , it can be determined based on contextual information whether the determination made at block  702  was likely erroneous. In some examples, this can include determining whether a request to cancel an operation is received (e.g., via a verbal request from the user, via a button or other input mechanism of the user device, or the like). The occurrence of the request to cancel can indicate that the determination made at block  702  was likely incorrect because the user is now canceling the operation that an utterance of the trigger phrase was intended to initiate. This can occur, for example, in a situation where multiple users are located in the same room and an utterance from one user triggers activation of the virtual assistant on another user&#39;s device. In other examples, block  704  can include determining whether subsequent audio input including user speech is not received within a threshold length of time. The threshold length of time can be any desired length of time, but, in some examples, can represent the amount of time during which a user is likely to provide a request or command to the virtual assistant after speaking the trigger phrase. The absence of an audio input including user speech within this threshold length of time can indicate that the determination made at block  702  was likely incorrect because the user is not providing the type of input expected after utterance of a trigger phrase, suggesting that the user did not intend to trigger the virtual assistant. In other examples, other types of contextual information that can be indicative of the determination made at block  702  being correct or incorrect can be used. If it is determined at block  704  that the determination made at block  702  was not erroneous, process  700  can proceed to block  708  where a final determination that the speaker of the audio input is the predetermined user represented by the speaker profile can be made. Alternatively, if it is instead determined at block  704  that the determination made at block  702  was likely erroneous, process  700  can proceed to block  710  where a final determination that the speaker of the audio input is not the predetermined user represented by the speaker profile can be made. 
     Referring back to block  702 , if it is instead determined that the speaker of the audio input is not the predetermined user represented by the speaker profile, process  700  can proceed to block  706 . At block  706 , it can be determined based on contextual information whether the determination made at block  702  was likely erroneous. In some examples, this can include determining whether another audio input including the trigger phrase is received within a threshold length of time. The threshold length of time can be any desired length of time. The receipt of another trigger phrase can indicate that the determination made at block  702  was likely incorrect because the user is again attempting to trigger the virtual assistant. In other examples, block  706  can include determining whether another form of a request to trigger the virtual assistant (e.g., via a selection of a mechanical button or other input mechanism) is received within a threshold length of time. The threshold length of time can be any desired length of time. The receipt of another trigger phrase can indicate that the determination made at block  702  was likely incorrect because the user is continuing to attempt to trigger the virtual assistant using means other than the trigger phrase. In other examples, other types of contextual information that can be indicative of the determination made at block  702  being correct or incorrect can be used. If it is determined at block  706  that the determination made at block  702  was not erroneous, the process can proceed to block  710  where a final determination that the speaker of the audio input is not the predetermined user represented by the speaker profile can be made. Alternatively, if it is instead determined at block  706  that the determination made at block  702  was likely erroneous, process  700  can proceed to block  708  where a final determination that the speaker of the audio input is the predetermined user represented by the speaker profile can be made. 
     Processes  300 ,  400 ,  500 ,  600 , and  700  can continue to be repeated any number of times in accordance with the description above to process user inputs and to generate/modify speaker profiles for performing speaker identification. In some examples, the speaker profiles can be reset or deleted in response to a request from the user, a request to change a language of the virtual assistant, or other input. In these examples, process  300  can again operate in the speaker profile generating mode (represented by block  308 ) and continue as described above. 
     While specific modes of operation and criteria for selecting each mode (e.g., at block  306 ) are described above, it should be appreciated that any other criteria can be used depending on the desired operation of the virtual assistant. Moreover, it should be appreciated that process  300  can include alternate, additional, or fewer modes of operation. For instance, in some examples, process  300  can include only the speaker profile modifying mode. In these examples, blocks  306 ,  308 , and  312  can be omitted. In other examples, process  300  can include the speaker profile building mode and the speaker profile modifying mode, but can exclude the static speaker profile mode. In these examples, process  300  can include block  306 ,  308 , and  310 , but can exclude block  312 . It should be appreciated that other combinations of these and other modes are possible. 
     It should be appreciated that the blocks of processes  300 ,  400 ,  500 ,  600 , and  700  can be performed on user device  102 , server system  110 , or a combination of user device  102  and server system  110 . For instance, in some examples, all blocks of processes  300 ,  400 ,  500 ,  600 , and  700  can be performed on user device  102 . In other examples, some blocks of processes  300 ,  400 ,  500 ,  600 , and  700  can be performed at user device  102 , while other blocks of processes  300 ,  400 ,  500 ,  600 , and  700  can be performed at server system  110 . In yet other examples, all blocks of processes  300 ,  400 ,  500 ,  600 , and  700  can be performed at server system  110 . 
     Electronic Device 
     In accordance with some examples,  FIG. 8  shows a functional block diagram of an electronic device  800  configured in accordance with the principles of the various described examples. The functional blocks of the device can be implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described examples. It is understood by persons of skill in the art that the functional blocks described in  FIG. 8  can be combined or separated into sub-blocks to implement the principles of the various described examples. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein. 
     As shown in  FIG. 8 , electronic device  800  can include a touch screen display unit  802  configured to display a user interface and to receive touch input, and a sound receiving unit  804  configured to receive sound input. In some examples, electronic device  800  can include a speaker unit  806  configured to generate sound. Electronic device  800  can further include a processing unit  808  coupled to touch screen display unit  802  and sound receiving unit  804  (and, optionally, coupled to speaker unit  806 ). In some examples, processing unit  808  can include receiving unit  810 , determining unit  812 , adding unit  814 , speech-to-text unit  816 , intent determining unit  818 , task determining unit  820 , parameter determining unit  822 , and task performing unit  824 . 
     Processing unit  808  can be configured to receive (e.g., from audio receiving unit  804 ) an audio input comprising user speech. Determining unit  810  can be configured to determine whether a speaker of the user speech is a predetermined user based at least in part on a speaker profile for the predetermined user. Adding unit  812  can be configured to add, in accordance with a determination that the speaker of the user speech is the predetermined user, the audio input comprising user speech to the speaker profile for the predetermined user. 
     In some examples, the speaker profile for the predetermined user comprises a plurality of voice prints. 
     In some examples, each of the plurality of voice prints of the speaker profile for the predetermined user was generated from previously received audio inputs comprising user speech. 
     In some examples, determining whether the speaker of the user speech is the predetermined user based at least in part on the speaker profile for the predetermined user comprises: determining whether the audio input comprising user speech matches at least a threshold number of the plurality of voice prints; in accordance with a determination that the audio input comprising user speech matches at least the threshold number of the plurality of voice prints, determining that the speaker of the user speech is the predetermined user; and in accordance with a determination that the audio input comprising user speech does not match at least the threshold number of the plurality of voice prints, determining that the speaker of the user speech is not the predetermined user. 
     In some examples, determining whether the speaker of the user speech is the predetermined user based at least in part on the speaker profile for the predetermined user comprises: determining whether the audio input comprising user speech matches at least a threshold number of the plurality of voice prints; in accordance with a determination that the audio input comprising user speech matches at least the threshold number of the plurality of voice prints: determining whether an erroneous speaker determination was made based on contextual data; in accordance with a determination that an erroneous speaker determination was not made based on contextual data, determining that the speaker of the user speech is the predetermined user; and in accordance with a determination that an erroneous speaker determination was made based on contextual data, determining that the speaker of the user speech is not the predetermined user; and in accordance with a determination that the audio input comprising user speech does not match at least the threshold number of the plurality of voice prints: determining whether an erroneous speaker determination was made based on contextual data; in accordance with a determination that an erroneous speaker determination was not made based on contextual data, determining that the speaker of the user speech is not the predetermined user; and in accordance with a determination that an erroneous speaker determination was made based on contextual data, determining that the speaker of the user speech is the predetermined user. 
     In some examples, adding the audio input comprising user speech to the speaker profile for the predetermined user comprises: generating a voice print from the audio input comprising user speech; and storing the voice print in association with the speaker profile for the predetermined user. 
     In some examples, adding unit  814  can be further configured to add, in accordance with a determination that the speaker of the user speech is not the predetermined user, the audio input comprising user speech to a speaker profile for an alternate user. 
     In some examples, the speaker profile for the alternate user comprises a plurality of voice prints. 
     In some examples, each of the plurality of voice prints of the speaker profile for the alternate user was generated from previously received audio inputs comprising user speech. 
     In some examples, determining whether the speaker of the user speech is the predetermined user is further based at least in part on the speaker profile for the alternate user. 
     In some examples, determining whether the speaker of the user speech is the predetermined user comprises: determining whether the audio input comprising user speech matches a greater number of voice prints of the speaker profile for the predetermined user than a number of voice prints of the speaker profile for the alternate user; in accordance with a determination that the audio input comprising user speech matches a greater number of voice prints of the speaker profile for the predetermined user than a number of voice prints of the speaker profile for the alternate user, determining that the speaker of the user speech is the predetermined user; and in accordance with a determination that the audio input comprising user speech does not match a greater number of voice prints of the speaker profile for the predetermined user than a number of voice prints of the speaker profile for the alternate user, determining that the speaker of the user speech is not the predetermined user. 
     In some examples, determining whether the speaker of the user speech is the predetermined user comprises: determining whether the audio input comprising user speech matches a greater number of voice prints of the speaker profile for the predetermined user than a number of voice prints of the speaker profile for the alternate user; in accordance with a determination that the audio input comprising user speech matches a greater number of voice prints of the speaker profile for the predetermined user than a number of voice prints of the speaker profile for the alternate user: determining whether an erroneous speaker determination was made based on contextual data; in accordance with a determination that an erroneous speaker determination was not made based on contextual data, determining that the speaker of the user speech is the predetermined user; and in accordance with a determination that an erroneous speaker determination was made based on contextual data, determining that the speaker of the user speech is not the predetermined user; and in accordance with a determination that the audio input comprising user speech does not match a greater number of voice prints of the speaker profile for the predetermined user than a number of voice prints of the speaker profile for the alternate user: determining whether an erroneous speaker determination was made based on contextual data; in accordance with a determination that an erroneous speaker determination was not made based on contextual data, determining that the speaker of the user speech is not the predetermined user; and in accordance with a determination that an erroneous speaker determination was made based on contextual data, determining that the speaker of the user speech is the predetermined user. 
     In some examples, speech-to-text unit  816  can be configured to perform, in accordance with a determination that the speaker of the user speech is the predetermined user, speech-to-text conversion on a second audio input comprising a second user speech, wherein the second audio input is received after receiving the audio input comprising user speech. Intent determining unit  818  can be configured to determine, in accordance with a determination that the speaker of the user speech is the predetermined user, a user intent based on the second user speech. Task determining unit  820  can be configured to determine, in accordance with a determination that the speaker of the user speech is the predetermined user, a task to be performed based on the second user speech. Task performing unit  824  can be configured to perform, in accordance with a determination that the speaker of the user speech is the predetermined user, the task to be performed in accordance with the determined parameter. 
     As described above, one aspect of the present technology is the gathering and use of data available from various sources to improve the delivery to users of invitational content or any other content that may be of interest to them. The present disclosure contemplates that in some instances, this gathered data can include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data can include demographic data, location-based data, telephone numbers, email addresses, home addresses, or any other identifying information. 
     The present disclosure recognizes that the use of such personal information data, in the present technology, can be used to the benefit of users. For example, the personal information data can be used to deliver targeted content that is of greater interest to the user. Accordingly, use of such personal information data enables calculated control of the delivered content. Further, other uses for personal information data that benefit the user are also contemplated by the present disclosure. 
     The present disclosure further contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and/or privacy practices. In particular, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining personal information data private and secure. For example, personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Further, such collection should occur only after receiving the informed consent of the users. Additionally, such entities would take any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices. 
     Despite the foregoing, the present disclosure also contemplates examples in which users selectively block the use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements can be provided to prevent or block access to such personal information data. For example, in the case of advertisement delivery services, the present technology can be configured to allow users to select to “opt in” or “opt out” of participation in the collection of personal information data during registration for services. In another example, users can select not to provide location information for targeted content delivery services. In yet another example, users can select to not provide precise location information, but permit the transfer of location zone information. 
     Therefore, although the present disclosure broadly covers use of personal information data to implement one or more various disclosed examples, the present disclosure also contemplates that the various examples can also be implemented without the need for accessing such personal information data. That is, the various examples of the present technology are not rendered inoperable due to the lack of all or a portion of such personal information data. For example, content can be selected and delivered to users by inferring preferences based on non-personal information data or a bare minimum amount of personal information, such as the content being requested by the device associated with a user, other non-personal information available to the content delivery services, or publicly available information. 
     Although examples have been fully described with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications are to be understood as being included within the scope of the various examples as defined by the appended claims.

Metadata:
Filing Date: 20181009
Publication Date: 20191008
Grant Date: 20191008
Priority Date: 20140930
Inventors: KIM, YOON
KAJAREKAR, SACHIN S.
Assignee: APPLE INC
CPC Classifications: [{"code": "G10L15/1822", "inventive": false, "first": false, "tree": "[]"}, {"code": "G10L15/26", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L15/26", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L17/06", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L17/06", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L15/1822", "inventive": false, "first": false, "tree": "[]"}, {"code": "G10L17/26", "inventive": true, "first": true, "tree": "[]"}, {"code": "G10L17/04", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L17/06", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L15/26", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L15/1822", "inventive": false, "first": false, "tree": "[]"}, {"code": "G10L17/04", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L17/26", "inventive": true, "first": true, "tree": "[]"}, {"code": "G10L17/04", "inventive": true, "first": true, "tree": "[]"}]
Family ID: 55585151