VIRTUAL AGENT TRANSPARENT USER AUTHENTICATION

Certain aspects of the present disclosure provide techniques for receiving audio data comprising a user voice command; determining a task to be completed by a remote service based on the user voice command; determining that a reference voice print associated with the user is stored in a user account; authenticating the user by determining that a sample voice print based on the user voice command matches the reference voice print associated with the user; storing authentication evidence associated with the task; and providing proof of user authentication to the remote service in order to initiate the task with the remote service.

INTRODUCTION

Aspects of the present disclosure relate to transparently authenticating a user of a virtual assistant.

A virtual assistant, sometimes also referred to as an intelligent virtual assistant (IVA), is a software agent that assists users by performing tasks or services based on questions or commands provided by a user. Users often interact with virtual assistants on a user device through natural language, spoken or written words. For example, users may ask a virtual assistant to retrieve information or complete a transaction. A virtual assistant may interpret the user's request and determine a particular task for the virtual assistant to perform. A virtual assistant may then perform the task on behalf of the user. Virtual assistants improve users' lives by automating and performing tasks on behalf of the user. Increasingly, virtual assistants are being used by consumers and businesses alike.

In some instances, a user may request a virtual assistant to perform a task requiring the virtual assistant to interact with a third-party, such as a remote service. Ideally, the virtual assistant would connect to and perform a task with a third-party without requiring any further action from the user. However, many third-party services have security measures that prevent performing certain tasks without first authenticating the requesting party. This presents a technical problem in which the third-party service cannot directly verify the identity of the user for which the virtual assistant is acting and thus cannot complete the requested task without user intervention. Even when a virtual assistant is associated with a user account or specific user device, a third-party service cannot be certain an authentic user is directing the virtual assistant. Conventionally, the third-party will require the actual user to authenticate directly with the third-party, which defeats the purpose and convenience of using the virtual assistant. Practically, this means that virtual assistants are often unable to perform tasks without user intervention, which reduces their utility.

Accordingly, there is a need for improved methods for authenticating users of virtual assistants.

SUMMARY

Certain embodiments provide a method comprising: receiving audio data comprising a user voice command; determining a task to be completed by a remote service based on the user voice command; determining that a reference voice print associated with the user is stored in a user account; authenticating the user by determining that a sample voice print based on the user voice command matches the reference voice print associated with the user; storing authentication evidence associated with the task; and providing proof of user authentication to the remote service in order to initiate the task with the remote service.

Certain other embodiments provide a method comprising: receiving audio data comprising a user voice command; determining a task to be completed by a remote service based on the user voice command; determining that a sample voice print based on the user voice command cannot authenticate the user; authenticating the user based on a non-voice authentication method; storing the audio data as a user authentication model training sample; storing authentication evidence associated with the task; and providing proof of user authentication to the remote service in order to initiate the task with the remote service.

DETAILED DESCRIPTION

Aspects of the present disclosure provide apparatuses, methods, processing systems, and computer-readable mediums for transparently authenticating users of a virtual assistant application to establish trust for a virtual assistant to perform tasks on behalf of the user.

Virtual assistants, such as Intelligent Virtual Assistants (IVAs), may perform many different tasks on behalf of users by contacting and exchanging information with third-party services. For example, a user may request a virtual assistant to make a dinner reservation, book a hotel room, or update account information through a natural language request uttered to the virtual assistant.

However, third-party services often bear the risk of fraudulent activity through their service and thus generally require a means of verifying that a virtual assistant is acting on behalf of an authentic user. By way of example, a third-party service may not authorize a transaction requested by a virtual assistant until the end user directing the virtual assistant is authenticated by the third-party. Consequently, a user often needs to interact directly with the third-party service to complete a request made through the virtual assistant, which greatly reduces the benefit of the virtual assistant. Moreover, this inconvenience may cause the user to avoid one or both of the virtual assistant and the third-party service.

Thus, there exists a technical problem of how a third-party can authenticate a user without directly interacting with the user in order to perform tasks initiated by a virtual assistant on behalf of the user. In other words, how can the third-party establish a trust relationship with the virtual assistant so that it may act autonomously on behalf of the user? Moreover, because a virtual assistant may be capable of requesting and performing a large variety of tasks, the third-party may further need to determine the extent to which the virtual assistant is allowed to act on behalf of the user (e.g., to perform certain tasks on behalf of the user, but not others), even if a trust relationship exists. For example, the third-party may be authorized to provide account information, such as a balance of a bank account, but not to change account information, such as changing contact details associated with the account.

Aspects described herein provide technical solutions to the aforementioned technical problems. As described in more detail below, a user may be transparently authenticated when using a device that interacts with a third-party service based on a voice print of the user. Beneficially, the voice print may be established and then used for verification without user interaction. This technical solution enables a virtual assistant to authenticate a user when interacting with third-party services so that a user need not intervene to complete a wide variety of tasks, thus alleviating the technical problems in conventional systems.

Example Process Flow for Transparently Authenticating a User of a Virtual Assistant

FIG.1depicts an example process flow100for transparently authenticating a user of a virtual assistant. Generally, aspects of process flow100may be performed by virtual assistant trust system200, as described below with respect toFIG.2.

Flow100begins at step102with a user speaking to a user device (e.g., user device202, as described with respect toFIG.2), instructing a virtual assistant to begin a new task on behalf of the user with a remote service (e.g., remote service222inFIG.2). Generally speaking, when a user requests a virtual assistant to perform a task on its behalf, that may be referred to as a user voice command. Note that a remote service in this example is an example of a third-party service, which is to say a service hosted separately from the user's device and the virtual assistant in this example.

In some embodiments, a virtual assistant may use Natural Language Processing (NLP) techniques to interpret the user's spoken instructions and determine a task to be performed. For example, a user may instruct, “Update my billing information with my utility provider,” and the virtual assistant determines the task to perform is updating the user's billing information with the utility provider. An audio device or peripheral, such as a microphone, may be incorporated into a user device for receiving the user's spoken instructions and generating audio data to be used by the system. Note that while audio data is used in various examples described herein, other forms of data may be generated to assist a virtual assistant in performing tasks. For example, a device may receive commands by detecting signs using image data or by receiving textual instructions via a user interface and/or input device.

Flow100then proceeds to step104with determining whether the user needs to be authenticated to perform the task. In some embodiments, a user is authenticated every time the user initiates a new task with the virtual assistant. For example, a user may be authenticated when the user launches a virtual assistant and remains authenticated for a single session with the virtual assistant, which beneficially protects confidential information associated with the virtual assistant, such as the virtual assistant task history or status. In some embodiments, a user is authenticated when a task requires a user to be authenticated, for example, when the task requires a user to be logged into a user account for tasks such as manipulating a user's calendar, sending text messages, executing purchases, etc. For example, if a user is using a virtual assistant on a smart speaker device, the smart speaker device may always be on and authenticate the user when the task requires the user to be authenticated. If the user does not need to be authenticated, then the virtual assistant proceeds to step130with completing the task with the remote service, as discussed in further detail below.

Flow100then proceeds to step106with determining whether a reference voice print is stored for the user, such as in user account database218inFIG.2, discussed below. A voice print, for example, may be a voice signature, voice model, and the like, associated with a user's voice. A voice print, in some embodiments, may be data representing the unique vocal characteristics of an individual's voice, such as accent, cadence, tone, rate of speed, spectral characteristics, etc. A reference voice print may be obtained, for example, from a user authentication model, such as described in further detail with respect to step118below.

If there is a reference voice print stored for the user, then flow100proceeds to step108with determining whether a sample voice print matches the reference voice print. A sample voice print, in some embodiments, comprises audio data of a user voice command, such as audio data of the user speaking to the user device instructing the virtual assistant to begin the task.

In some embodiments, a match between a sample voice print and the reference voice print may be determined by a voice print matching component of a user authentication model. In some embodiments, the user authentication model determines an acoustic signature associated with a user voice print, such as a sample voice print or a reference voice print. The acoustic signature associated with a user voice print may be a signal representation of user phoneme production and inflections that define an accent or style of speech. In some embodiments, the user voice print matching component compares the acoustic signature associated with the sample voice print and the acoustic signature associated with the reference voice print and determines a score representing the probability that the sample voice print and the reference voice print belong to the same person based on the level of overlap between the acoustic signatures. For example, if there is significant overlap between the acoustic signature associated with the sample voice print and the acoustic signature associated with the reference voice print, then the score indicates a high probability the sample voice print and the reference voice print belong to the same person. Thus, the sample voice print and the reference voice print are determined to match and the user is authenticated. In some embodiments, the probability that the sample voice print and the reference voice print belong to the same person may be determined through various modeling techniques, for example, hidden Markov models, neural network models, mismatch compensation, score-to-likelihood-ratio conversion, and the like. In some embodiments, a user authentication model may be trained based on historical user audio, such as user audio saved as a training example at step116, discussed below, to determine a reference acoustic signature and determine a match between a sample voice print and a reference voice print. In some embodiments, a sample voice print may be determined to match a reference voice print through other voice comparison methods to authenticate a user. If the sample voice print matches the reference voice print, then the user is authenticated and flow100proceeds to step120.

If at step106, there is not a reference voice print stored for the user, or at step108, the sample voice print does not match the reference voice print, then the user must be manually authenticated at step110. A user may be manually authenticated, for example, through password-based authentication, multi-factor authentication, certificate-based authentication, biometric authentication, token-based authentication, and others.

If the user is determined to be unauthenticated at step112, then the virtual assistant is denied from fulfilling the task at114and the task is terminated. For example, where the user is not authenticated, the virtual assistant may not perform the task on behalf of the user. In some cases, the failed authentication may be stored and/or reported, such as to ensure compliance and security of the identity provider service (e.g., identity provider service220inFIG.2). Audio data associated with a failed authentication may also be stored and/or reported.

If the user is determined to be authenticated at step112, then the audio data associated with the task instruction (received at step102) is saved a training example for a user authentication model.

Flow100then proceeds to step118with training (or retraining/tuning) the user authentication model. For example, where there was no reference voice print stored, such as at step106, a reference voice print may be generated through an authentication model for the user. The new reference voice print may then be available for subsequent user authentication. In another example, where the sample voice print failed to match the reference voice print, an authentication model for the user may be retrained or tuned and a reference voice print updated. The updated reference voice print may then be available for subsequent user authentication.

If the user is authenticated at step108or112, flow100proceeds to step120with storing authentication evidence associated with the specific task. In some embodiments, evidence of the user authentication may be stored, for example, with an identity provider service, such as identity provider service220discussed with respect toFIG.2, below. In some embodiments, the authentication evidence comprises one or more of: a task-specific authentication token, a time-stamp associated with the audio data, an IP address associated with the user, a user-ID and password, or other authentication evidence showing the user has permission to access the task. A task-specific authentication token may be, for example, an authentication token identifies a user as authenticated for a particular task to be performed by the virtual assistant. A time-stamp associated with the audio data may be, for example, a time-stamp indicating when the audio data was generated. An IP address associated with the user may be, for example, an IP address associated with the user device, such as user device202inFIG.2. Evidence of user authentication may be used, for example, to audit actions taken by a virtual assistant.

Flow100then proceeds to step122with determining whether the identity provider service (IdP) is trusted by the remote service. For example, the identity provider service may be trusted by the remote service when the identity provider service and the remote service use identity federation. In identity federation, a service provider relies on an identity provider to authenticate users and convey identity information to authorize user access to the service provider.

If the identity provider service is trusted by the remote service, flow100proceeds directly to step126, which is discussed below.

If the identity provider service is not trusted by the remote service, then flow100proceeds to step124with authenticating with an external identity provider service. In some cases, the remote service may not trust one identity provider service, such as an internal identity provider service, but the remote service does trust an external identity provider service. In such cases, the internal identity provider service may authenticate the user with the external identity provider service and obtain proof of user authentication from the external identity provider service. Once the user has been authenticated with the external identity provider service, flow100continues to step126.

At step126, the virtual assistant initiates the task with the remote service. In some embodiments, the virtual assistant may initiate the task through a service handler to contact the remote service, such as described in more detail below with respect toFIG.2.

Flow100then proceeds at step128with providing the proof of user authentication to the remote service. In some embodiments, the proof of user authentication comprises an indication that the virtual assistant is authorized to perform the task on behalf of the user. Further, the proof of user authentication may indicate specific tasks the virtual assistant can perform for the user, or in other words, the scope for which the virtual assistant has permission from the user to act.

In some embodiments, the proof of user authentication comprises a token, such as an authentication token, a connected token, a contactless token, a disconnected token, a software token, a web token, and the like. In some embodiments, the token may be signed and expire after a set time period, which beneficially reduces the chance of an initially authorized but later compromised virtual assistant performing an unauthorized task.

Flow100then proceeds at step130with completing the task with the remote service.

Note that flow100is just one example, and other flows having additional, fewer, alternative, or differently ordered steps may be implemented.

Example System for Establishing Trust with a Virtual Assistant

FIG.2depicts an example system200for establishing trust with a virtual assistant.

Generally, a user wishing to have a virtual assistant perform some task with a remote service222on their behalf may interact with user device202(e.g., utter or enter a user voice command into user device202). In response, user device202connects with a virtual assistant service206. In the depicted embodiment, virtual assistant service206is communicatively coupled with user device202through mobile API service204. Thus, in the depicted embodiment, mobile API service204routes all communication and data between user device202and virtual assistant service206.

Note that while user device202is depicted throughout as a mobile smart device, in this case a smart phone, any electronic device with the capability to interact with a user (e.g., to receive a user command or query) and connect with virtual assistant service206might similarly implement the methods described herein. For example, user device202could be a tablet computer, desktop computer, a smart home device (e.g., a smart speaker), a smart wearable device, or generally any computer processing device with the ability to receive data from a user (e.g., through a user interface) and connect to virtual assistant service206.

Mobile API service204is configured to receive audio data, such as audio data associated with task instruction by a user speaking to a virtual assistant, as described with respect to step102inFIG.1. Mobile API service204is further configured to save the audio data as a sample voice print associated with the specific task the virtual assistant is to perform with user account database218.

Virtual assistant service206is configured to determine an appropriate channel to contact remote service222. Virtual assistant service206may use various communication channels, including, for example, phone calls, emails, social media communications, text messages, live chats, and other communication channels.

In some embodiments, virtual assistant service206determines a communication channel to contact remote service222based on a characteristic of a task, for example, the type or complexity of a task. In some embodiments, virtual assistant service206determines a communication channel to contact remote service222based on which communication channels the virtual assistant has been trained on. For example, virtual assistant service206may be trained to book a flight over a conversational communication channel such as a phone call, text message, or live chat, but not through social media or email. In some embodiments, virtual assistant service206determines a communication channel to contact a remote service222based on a characteristic of the remote service222, such as the type of the remote service, availability of various communication channels for the remote service, or preferred communication channels of the remote service. In some embodiments, virtual assistant service206determines a communication channel to contact a remote service222based on current expected wait times across various communication channels. In some embodiments, virtual assistant service206determines a communication channel to contact a remote service222based on a cost to remote service222associated with a communication channel. For example, virtual assistant service206contacting remote service222via a phone call may be higher cost than a live chat for remote service222and virtual assistant service206may user the lower cost live chat.

Virtual assistant service206is further configured to register a communication session with identity provider service220. In some embodiments, registering a communication session includes indicating the selected communication channel (e.g., phone call, email, social media communication, text message, or live chat) through which virtual assistant service206will contact the remote service222to perform a task on behalf of the user. Generally, virtual assistant service206then uses an appropriate service handler208to contact the remote service222on the selected communication channel, as described with respect to step126inFIG.1.

For example, virtual assistant service206may use outbound call service210to contact remote service222by way of a phone call or text message, email client service212to contact remote service222by way of an email, social channel service214to contact remote service222by way of a social media communication (e.g., a direct message, a post, etc.), live chat service216to contact remote service222by way of chat messages, and the like.

Identity provider service220is configured to authenticate the user. Identity provider service220is further configured to access a reference voice print associated with the user, such as described with respect to step106inFIG.1, which may be stored in user account database218. If a reference voice print is stored for the user, then identity provider service220determines whether the sample voice print matches the reference voice print, described with respect to step108inFIG.1. If the reference voice print and the sample voice print match, then identity provider service220stores authentication evidence, as described with respect to step120inFIG.1.

Identity provider service220is configured to manually authenticate a user if no reference voice print is stored in user account database218, or if identity provider service220determines the reference voice print does not match the sample voice print, as described with respect to step110inFIG.1. Identity provider service220may terminate a task and deny virtual assistant service206from fulfilling the task, if the user cannot be authenticated.

Identity provider service220is further configured to train (or retain/tune) a reference authentication model to obtain a reference voice print or update a reference voice print if the user is successfully authenticated, as described with respect to step118inFIG.1. Identity provider service220may store the reference voice print in user account database218. Further, authentication evidence may be stored identity provider service220, as described with respect to step120inFIG.1.

Identity provider service220is configured to determine whether the remote service222trusts identity provider service220, as described with respect to step122inFIG.1. If identity provider service220is trusted, then identity provider service220obtains the proof of user authentication stored in user account database218. If identity provider service220is not trusted by the remote service, then identity provider service authenticates with an external identity provider service and obtain proof of user authentication from the external provider service, as described with respect to step124.

Identity provider service220is further configured to receive a communication session authentication query from the remote service222, where the query is used to determine whether virtual assistant service206is associated with an authenticated user. Remote service222provides contact information for the selected communication channel (e.g., an origin telephone number, email account, social media account, chat identity, or the like) associated with the communication session as part of the communication session authentication query. Identity provider service220determines whether the communication session was registered with identity provider service220based on the provided contact information for the selected communication channel.

If identity provider service220determines that no communication session was registered with identity provider service220based on the provided contact information for the selected communication channel, then identity provider service220sends an indication to remote service222that the communication session is not expected.

If identity provider service220determines that a communication session was registered with identity provider service220based on the provided contact information for the selected communication channel, then identity provider service220sends an indication to remote service222that the communication session is expected.

Identity provider service220is further configured to determine that the user associated with the registered communication session has been authenticated. In some cases, identity provider service220obtains a proof of user authentication stored in user account database218. Identity provider service220can then send the proof of user authentication to remote service222to prove that the user that initiated the registered communication session is authentic, as described with respect to step128.

Once proof of user authentication is sent to remote service222, trust is established between virtual assistant service206and remote service222, allowing virtual assistant service206to perform the task on behalf of the user, such as at step130inFIG.1. For example, the task may include completing a transaction, retrieving account information, updating account information, enabling or disabling an account service, querying the status of a shipment or transaction, scheduling a service or action, and the like.

In some cases, virtual assistant service206may commence, but not complete a task on behalf of the user. For example, where the task is to call a utility provider to inquire about a recent bill, virtual assistant service206may establish a trust relationship with the utility provider customer service department (e.g., remote service222), and transfer the call to the user when a customer service department agent has been reached. In this example, virtual assistant service206initiates contact with the customer service department, authenticates the user with the customer service department, and transfers the call between departments all without user interaction and without the user waiting on hold. Beneficially, this reduces the time the user is on the call and the user may proceed to directly inquiring about their recent bill.

FIG.3depicts further aspects of a remote service for establishing trust with a virtual agent through an example data flow300for establishing trust of a virtual assistant.

Generally, all like numbered aspects ofFIG.3are as described inFIG.2. In the depicted example, remote service222comprises queue service326, customer authentication service324, customer account database328, customer service agent device330, and backend services332. In the depicted example, datacenter302comprises identity provider service220and service handler208.

Queue service326is generally configured to receive a communication session from service handler208on a communication channel, such as a call, email, social media communication, text message, or live chat from a virtual assistant.

Queue service326is further configured to query customer authentication service324to determine whether the user is authenticated before sending the communication session to a customer service agent using a customer service agent device330. In some embodiments, to determine whether a user is authenticated, customer authentication service324queries identity provider service220. A communication session authentication query sent by customer authentication service324, asks identity service provider whether the communication session was registered with identity provider service220. Customer authentication service324provides contact information for the selected communication channel (e.g., an origin telephone number, email account, social media account, or chat identity) associated with the communication session with the communication session authentication query. Identity provider service220then determines whether the communication session was registered with identity provider service220based on the provided contact information for the selected communication channel.

If the communication session was not registered with identity provider service220, then customer authentication service324receives, from identity provider service220, an indication that the communication session is unexpected and the user is unauthenticated. Customer authentication service324then indicates to queue service326the user is unauthenticated. Queue service326may then transfer the communication session to a customer service agent device330as unauthenticated. In some embodiments, a customer service agent using customer service agent device330may then directly authenticate the user.

If the contact information is associated with a registered communication session, then customer authentication service324receives proof of user authentication from identity provider service. Customer authentication service324may then trust the user was authenticated by identity provider service220, without needed to authenticate the user directly. This reduces the number of times a user needs to be authenticated while maintaining security. If identity provider service220is trusted, then the received proof of user authentication is associated with identity provider service220. If identity provider service220is not trusted by the remote service, then the proof of user authentication is associated with an external identity provider service, as described with respect to step124inFIG.1.

In some embodiments, the proof of user authentication authorizes the performance of a task on behalf of the user. For example, the proof of user authentication may authorize access to a customer account or authorize a transaction with that account.

In some embodiments, the proof of user authentication comprises an indication that a virtual assistant (e.g., virtual assistant service206inFIG.2), is authorized to perform a specific task on behalf of the user. For example, the proof of user authentication may indicate the virtual assistant is authorized to make a particular payment to a particular recipient on a customer account. Customer authentication service324may use the proof of user authentication to look up a user in a customer account database328to find customer account information. Customer account information and the proof of user authentication are then sent to a customer service agent device330and backend services332, to establish trust for the communication session. In some embodiments, a customer service agent may use customer account information and the proof of user authentication during a trusted communication session to perform an authorized task.

Example Method for Providing Proof of User Authentication by a Virtual Assistant

FIG.4depicts an example method400for providing proof of user authentication by a virtual assistant.

Method400begins at step402with receiving audio data comprising a user voice command, such as when a user speaking to a virtual assistant as described above with respect toFIG.1.

Method400then proceeds to step404with determining a task to be completed by a remote service based on the user voice command, such as through NLP techniques as described above with respect toFIG.1.

In some embodiments, method400further comprises determining that the task requires proof of user authentication, such as described with respect to step104ofFIG.1.

Method400then proceeds to step406with determining that a reference voice print associated with the user is stored in a user account, such as in user account database218inFIG.2.

Method400then proceeds to step408with authenticating the user by determining that a sample voice print based on the user voice command matches the reference voice print associated with the user, such as described above with respect toFIG.1.

Method400then proceeds to step410with storing authentication evidence associated with the task. For example, authentication evidence may be stored in a user account, such as in user account database218inFIG.2.

In some embodiments, the authentication evidence comprises one or more of: a task-specific authentication token; a time-stamp associated with the audio data; or an IP address associated with the user.

Method400proceeds to step412with providing proof of user authentication to the remote service in order to initiate the task with the remote service. For example, proof of user authentication may be provided to customer authentication service324in response to a query, as described above with respect toFIG.3.

In some embodiments, method400further comprises determining that a trust relationship exists with the remote service, wherein the proof of user authentication comprises a task-specific authentication token stored with the authentication evidence.

In some embodiments, method400further comprises determining that a trust relationship does not exist with the remote service; authenticating with an external identification provider; and receiving from the external identification provider the proof of user authentication, such as described with respect to step124ofFIG.1.

In some embodiments, the proof of user authentication comprises a username and user password, such as associated with a customer account in customer account database328inFIG.3.

Note that method400is one example, and other flows including additional, alternative, fewer steps, or steps in a different order, are possible consistent with various aspects described herein.

Example Method for Training a Virtual Assistant to Authenticate a User Based on Voice

FIG.5depicts an example method500for training a virtual assistant to authenticate a user based on a user's voice.

Method500begins at step502with receiving audio data comprising a user voice command, such as when a user speaks to a virtual assistant, as described above with respect toFIG.1.

Method500then proceeds to step504with determining a task to be completed by a remote service based on the user voice command, such as through NLP techniques as described above with respect toFIG.1.

In some embodiments, method500further comprises determining that the task requires proof of user authentication, such as described with respect to step104ofFIG.1.

Method500then proceeds to step506with determining that a sample voice print based on the user voice command cannot authenticate the user.

In some embodiments, determining that the sample voice print based on the user voice command cannot authenticate the user comprises determining that no reference voice print associated with the user is stored in a user account, such as described with respect to step106inFIG.1.

In some embodiments, determining that the sample voice print based on the user voice command cannot authenticate the user comprises determining that a reference voice print associated with the user stored in a user account does not match the sample voice print, such as described with respect to step108inFIG.1.

Method500then proceeds to step508with authenticating the user based on a non-voice authentication method, such as described with respect to step110inFIG.1.

Method500then proceeds to step510with storing the audio data as a user authentication model training sample, such as described with respect to step116inFIG.1.

Method500then proceeds to step512with training a user authentication model based on the user authentication model training sample, such as described with respect to step118inFIG.1.

In some embodiments, method500further comprises storing the trained user authentication model in a user account. For example, the trained user authentication model may be stored in user account, such as in user account database218inFIG.2.

In some embodiments, method500further comprises storing authentication evidence associated with the task; and providing proof of user authentication to the remote service in order to initiate the task with the remote service. For example, authentication evidence may be stored in user account, such as in user account database218inFIG.2, and proof of user authentication may be provided to customer authentication service324in response to a query, as described above with respect toFIG.3.

In some embodiments, the authentication evidence comprises one or more of: a task-specific authentication token; a time-stamp associated with the audio data; or an IP address associated with the user.

In some embodiments, method500further comprises determining that a trust relationship exists with the remote service, wherein the proof of user authentication comprises a task-specific authentication token stored with the authentication evidence.

In some embodiments, method500further comprises determining that a trust relationship does not exist with the remote service; authenticating with an external identification provider; and receiving from the external identification provider the proof of user authentication, such as described with respect to step124ofFIG.1.

In some embodiments, method500further comprises determining that the task requires proof of user authentication.

In some embodiments, the proof of user authentication comprises a username and user password.

Note that method500is one example, and other methods including additional, alternative, or fewer steps, or steps in a different order, are possible consistent with the various aspects described herein.

Example Processing System

FIG.6depicts an example processing system600configured to perform the methods described herein.

Processing system600includes one or more processors602. Generally, a processor602is configured to execute computer-executable instructions (e.g., software code) to perform various functions, as described herein.

Processing system600further includes network interface604, which generally provides data access to any sort of data network, including local area networks (LANs), wide area networks (WANs), the Internet, and the like.

Processing system600further includes input(s) and output(s)606, which generally provide means for providing data to and from processing system600, such as via connection to computing device peripherals, including user interface peripherals.

Processing system600further includes a memory610comprising various components. In this example, memory610includes a virtual assistant component620, an identity provider service component621, a mobile API service component622, a service handler component623, a user authentication model component624, a voice print matching component625, audio data626, user account data627, user voice command data628, authentication evidence629, sample voice print data630, proof of user authentication631, reference voice print data632, and task data633.

Processing system600may be implemented in various ways. For example, processing system600may be implemented within on-site, remote, or cloud-based processing equipment. Note that in various implementations, certain aspects may be omitted, added, or substituted from processing system600.

Example Clauses

Clause 1: A method, comprising: receiving audio data comprising a user voice command; determining a task to be completed by a remote service based on the user voice command; determining that a reference voice print associated with the user is stored in a user account; authenticating the user by determining that a sample voice print based on the user voice command matches the reference voice print associated with the user; storing authentication evidence associated with the task; and providing proof of user authentication to the remote service in order to initiate the task with the remote service.

Clause 2: The method of Clause 1, further comprising: determining that a trust relationship exists with the remote service, wherein the proof of user authentication comprises a task-specific authentication token stored with the authentication evidence.

Clause 3: The method of Clause 1, further comprising: determining that a trust relationship does not exist with the remote service; authenticating with an external identity provider service; and receiving from the identity provider service the proof of user authentication.

Clause 4: The method of any one of Clause 1-3, further comprising determining that the task requires proof of user authentication.

Clause 5: The method of any one of Clause 1-4, wherein the proof of user authentication comprises a username and user password.

Clause 6: The method of any one of Clause 1-5, wherein the authentication evidence comprises one or more of: a task-specific authentication token; a time-stamp associated with the audio data; or an IP address associated with the user.

Clause 7: A method, comprising: receiving audio data comprising a user voice command; determining a task to be completed by a remote service based on the user voice command; determining that a sample voice print based on the user voice command cannot authenticate the user; authenticating the user based on a non-voice authentication method; storing the audio data as a user authentication model training sample; storing the audio data as a user authentication model training sample; and training a user authentication model based on the user authentication model training sample.

Clause 8: The method of Clause 7, further comprising storing authentication evidence associated with the task; and providing proof of user authentication to the remote service in order to initiate the task with the remote service.

Clause 9: The method of any one of Clause 7-8, further comprising storing the trained user authentication model in a user account.

Clause 10: The method of any one of Clause 7-9, wherein determining that the sample voice print based on the user voice command cannot authenticate the user comprises determining that no reference voice print associated with the user is stored in a user account.

Clause 11: The method of any one of Clause 7-9, wherein determining that the sample voice print based on the user voice command cannot authenticate the user comprises determining that a reference voice print associated with the user stored in a user account does not match the sample voice print.

Clause 12: The method of any one of Clause 7-11, further comprising: determining that a trust relationship exists with the remote service, wherein the proof of user authentication comprises a task-specific authentication token stored with the authentication evidence.

Clause 13: The method of any one of Clause 7-11, further comprising: determining that a trust relationship does not exist with the remote service; authenticating with an external identity provider service; and receiving from the identity provider service the proof of user authentication.

Clause 14: The method Clause any one of 7-13, further comprising determining that the task requires proof of user authentication.

Clause 15: The method of Clause any one of 7-14, wherein the proof of user authentication comprises a username and user password.

Clause 16: The method of Clause any one of 7-15, wherein the authentication evidence comprises one or more of: a task-specific authentication token; a time-stamp associated with the audio data; or an IP address associated with the user.

Clause 17: An apparatus, comprising: a memory comprising computer-executable instructions; a processor configured to execute the computer-executable instructions and cause the apparatus to perform a method in accordance with any one of Clauses 1-16.

Clause 18: An apparatus, comprising means for performing a method in accordance with any one of Clauses 1-16.

Clause 19: A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by a processor of a processing system, cause the processing system to perform a method in accordance with any one of Clauses 1-16.

Additional Considerations