Flexible security level for device interaction

A system and method for accommodating various device and application security levels collects authentication data such as voice and fingerprint in advance of opening an application or process potentially requiring different security than the device itself requires. The device is then able to execute a user's voiced command, request or query without further user actions if the command, request or query is related to an application or process for which the already-gathered authentication data is sufficient.

TECHNICAL FIELD

The present disclosure is related generally to mobile communication devices, and, more particularly, to a system and method for variable security enforcement on a mobile communications device based on the application context in which access occurs.

BACKGROUND

Consumer use of technology continues to grow at an astounding rate, and users are now likely to attend to most or many interactive aspects of their lives via their computer or mobile device. However, the increased adoption of technology for private or financial exchanges or activities has raised the risk and consequences of a security breach, with the severity of the consequences depending upon the application in which security has been breached. Thus, many applications now include security capabilities, but these capabilities differ based on many factors.

For example, an application installed on a device may require a different level of security than an application hosted on a web site. Similarly, within an application, different actions may require different levels of security; in banking, making transfers requires a higher level of security than viewing a transaction history; viewing a transaction history requires a higher level of security than viewing the current balance, etc. With each application or function exposing security requirements that may be inconsistent with other applications or functions, the task of managing security on a mobile device can be daunting.

While the present disclosure is directed to a system that can eliminate certain shortcomings, it should be appreciated that such a benefit is neither a limitation on the scope of the disclosed principles nor of the attached claims, except to the extent expressly noted in the claims. Additionally, the discussion of technology in this Background section is reflective of the inventors' own observations, considerations, and thoughts, and is in no way intended to accurately catalog or comprehensively summarize the art currently in the public domain. As such, the inventors expressly disclaim this section as admitted or assumed prior art. Moreover, any identification or implication above or otherwise herein of a desirable course of action reflects the inventors' own observations and ideas, and should not be assumed to indicate an art-recognized desirability.

SUMMARY

In keeping with an embodiment of the disclosed principles, a system and method for accommodating various device and application security levels collects authentication data such as voice and fingerprint in advance of opening an application or process potentially requiring different security than the device itself requires. The device is then able to execute a user's voiced command, request or query without further user actions if the command, request or query is related to an application or process for which the already-gathered authentication data is sufficient.

In keeping with another embodiment of the disclosed principles, a process of processing a user interaction with a mobile electronic communication device is provided. The method entails receiving a user fingerprint input while the device is locked, unlocking the device and creating a fingerprint authorization score based on the received user fingerprint. If a user utterance has been received, the process then generates a voice authentication score associated with a probability that the user is an authorized user, converts the user utterance into informational content specifying a user-desired action, and determines that a combined score based on the fingerprint authorization score and the voice authentication score is sufficient to authorize the user-desired action. The user-desired action is then authorized and executed.

In keeping with yet another embodiment of the disclosed principles, a mobile electronic communication device is provided having a fingerprint sensor, a voice sensor and a processor configured to authenticate device user via the fingerprint sensor to unlock the device, gather voice data via the voice sensor to authenticate the user for a function on the device having a higher security requirement than for unlocking the device, interpret the voice data to identify the function and one of a command, request and query contained therein. The device then authenticates the user for the function, and executes the command, request or query.

In keeping with yet another embodiment of the disclosed principles, a method of processing a user interaction with a mobile electronic communication device is provided. This method entails receiving first user information, authenticating a user to the device and unlocking the device based on first user information and receiving second user information that both identifies a function having a higher security requirement than the device itself, and also identifies a user-desired action. The user is authenticated to the identified function using the first and second user information and the user-desired action is executed via the function.

Other features and aspects of embodiments of the disclosed principles will be appreciated from the detailed disclosure taken in conjunction with the included figures.

DETAILED DESCRIPTION

Before discussing the details of various embodiments of the disclosed principles, a brief overview is given to aid the reader. When considering the touch UI (user interface) of a portable electronic device such as a cellphone, smartphone, laptop, touchpad, etc., the system does not have information regarding the user's intent when an interaction begins; for example the user's intent may be to see the transaction history of his or her account, but when the user starts opening the banking application the system does not have any information indicating that the user is opening the application in order to then view the transaction history. Thus, the user may be authenticated once at the beginning of the interaction (when launching the application) with that authentication being valid for all the following user interactions within the application until the application exits.

That is, the user is not generally re-authenticated during the active life of the application. For that reason, the security level enforced at the beginning of the interaction is the highest level of all possible user interactions. Thus, for example, an interaction sequence to execute several tasks with the security level FP (Fingerprint) might be as follows:

Task A: The user wants to see the current balance on his bank account, on a phone with the screen OFF:

1) FP touch: unlocks the phone, turns on the screen.

2) Touch sequence on the screen: opens Bank application.

3) FP touch: logs the user in bank application, the user sees the balance.

Task B: User wants to see the last deposit from Employer on his bank account, on a phone with screen OFF:

1) FP touch: unlocks the phone, turns on the screen.

2) Touch sequence on the screen: opens bank application.

3) FP touch: logs the user in bank application.

4) Touch sequence on the screen: the user navigates to the recent transactions and views the transaction of interest.

As can be seen, the incidence of redundant and inappropriate security measures is high when the device and the application both manage their own security. In an embodiment of the disclosed principles, flexible security levels are conveniently provided. In one embodiment, applied to voice UI, the system is given the user's intent when the user starts the interaction. For example, a command of “open my recent transactions at bankname,” tells the device the user's intent before the device launches the application, and at the same time, the system can authenticate the user by his or her voice.

Since both the user intent and user's authentication are available before the action is executed, the obtained authentication level can be compared with the required security level for the given action. In this way the voice UI allows using flexible security levels for different actions. If the recognized intent requires a higher level of security than the level obtained from the voice authentication, the system can request additional authentication such as Fingerprint (FP) authentication. Alternatively, the user may know from previous interactions that the action requires a high level of security and the user may then perform Fingerprint authentication in parallel with uttering the voice command.

FIG. 1Bis an authentication level pyramid191showing the manner in which increasingly sensitive applications require increased levels of security. At Level1of the pyramid, interactions such as checking weather, using a phone utility (e.g., camera or flashlight) and playing a game are secured at the same level as the device itself, for example, by voice authentication. At Level2, applications such as banking, virtual private network interactions, email access and so on require heighted security which, in this example, consists of combined voiceprint and fingerprint. It is possible that yet more sensitive applications such as secure database access (Level3) may require not only voiceprint and fingerprint but also a secure token.

In an embodiment, the device employs “always on” technologies, such that speech processing, speaker authentication, and fingerprint authentication are always on. This allows the user to have seamless and fast interactions. As a result, the previously mentioned tasks A and B can be accomplished in a faster manner and with much improved security level as follows:

1) User speaks “open bank” or “show me bank balance” while touching FP; the user sees the balance.

1) User speaks “open my recent transactions at bank” while touching FP; the user sees the transaction of interest in recent transactions.

Similarly, for accessing banking information on a laptop, the user may execute Task A by speaking to the phone “open bank on laptop” or “show me bank balance on laptop” while touching FP; the user then is shown the balance on the laptop. With respect to Task B, the user may speak to the phone “open my recent transactions at bank on laptop” while touching FP; the user is then shown his or her recent transactions, including the transaction of interest on laptop. Coordination between the phone and the laptop may be over a larger network such as WiFi, WAN and so on, or may be via a short range protocol such as Bluetooth.

Turning now to a more detailed discussion in conjunction with the attached figures, the techniques of the present disclosure are illustrated as being implemented in a suitable computing environment. The following device description is based on embodiments and examples of the disclosed principles and should not be taken as limiting the claims with regard to alternative embodiments that are not explicitly described herein. Thus, for example, whileFIG. 1Billustrates an example mobile device within which embodiments of the disclosed principles may be implemented, it will be appreciated that other device configurations or types may be used.

The schematic diagram ofFIG. 1Ashows an exemplary device110forming part of an environment within which aspects of the present disclosure may be implemented. It will be appreciated that additional or alternative components may be used in a given implementation depending upon user preference, component availability, price point, and other considerations.

In the illustrated embodiment, the components110include a display screen120, applications (e.g., programs)130, a processor140, a memory150, one or more input components160(user input receiver) such as speech and text input facilities, and one or more output components170such as text and audible output facilities, e.g., one or more speakers. In an embodiment, the input components160include a microphone and the device110further includes an authentication data processing module180. In various embodiments motion sensors, proximity sensors, camera/IR sensors and other types of sensors may be used to collect certain types of input information such as user presence, user gestures and so on.

The processor140may be any of a microprocessor, microcomputer, application-specific integrated circuit, and like structures. For example, the processor140can be implemented by one or more microprocessors or controllers from any desired family or manufacturer. Similarly, the memory150may reside on the same integrated circuit as the processor140. Additionally or alternatively, the memory150may be accessed via a network, e.g., via cloud-based storage. The memory150may include a random access memory (i.e., Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRM) or any other type of random access memory device or system). Additionally or alternatively, the memory150may include a read only memory (i.e., a hard drive, flash memory or any other desired type of memory device).

The information that is stored by the memory150can include program code associated with one or more operating systems or applications as well as informational data, e.g., program parameters, process data, etc. The operating system and applications are typically implemented via executable instructions stored in a non-transitory computer readable medium (e.g., memory150) to control basic functions of the electronic device. Such functions may include, for example, interaction among various internal components and storage and retrieval of applications and data to and from the memory150.

Further with respect to the applications130, these typically utilize the operating system to provide more specific functionality, such as file system services and handling of protected and unprotected data stored in the memory150. Although some applications may provide standard or required functionality of the user device110, in other cases applications provide optional or specialized functionality, and may be supplied by third party vendors or the device manufacturer.

Finally, with respect to informational data, e.g., program parameters and process data, this non-executable information can be referenced, manipulated, or written by the operating system or an application. Such informational data can include, for example, data that are preprogrammed into the device during manufacture, data that are created by the device or added by the user, or any of a variety of types of information that are uploaded to, downloaded from, or otherwise accessed at servers or other devices with which the device is in communication during its ongoing operation.

In an embodiment, a power supply190, such as a battery or fuel cell, is included for providing power to the device110and its components. All or some of the internal components communicate with one another by way of one or more shared or dedicated internal communication links195, e.g., an internal bus.

In various embodiments, the device110is programmed such that the processor140and memory150interact with the other components of the device110to perform certain functions. The processor140may include or implement various modules and execute programs for initiating different activities such as launching an application, transferring data, and toggling through various graphical user interface objects (e.g., toggling through various display icons that are linked to executable applications).

Applications and software reside on a tangible non-transitory medium, e.g., RAM, ROM or flash memory, as computer-readable instructions. The device110, via its processor140, runs the applications and software by retrieving and executing the appropriate computer-readable instructions.

Turning toFIG. 2, this figure illustrates an application view of a system for providing flexible security levels for device interaction. The device200may be the same device110shown inFIG. 1A, and includes memory150and output components170. In the illustrated embodiment, the input facilities170include a mic (microphone)201, e.g., for voiceprint authentication, as well as a fingerprint input203, which may include a fingerprint sensor and associated circuitry and logic. Alternatively, the computational tasks associated with the fingerprint and voice data processing may be executed by the main processor140.

The illustrated device also includes, in memory150, an operating system (OS)205and a banking application207. A remote machine209, such as a remote server, hosts one or more web sites, one of which is a web site associated with a bank, which is likewise associated with the banking application207. Note that the banking application and OS may run from and may use device memory150for execution as shown, or may be only stored in memory105, or may be stored in and run from a different memory entirely.

In operation, the user of the device200places his or her finger on the fingerprint sensor of the fingerprint input203while uttering an utterance that has both informational value and authentication value and that is received by the mic201. For example, the utterance may be “show me my bank balance.” The device200via its processor140and OS205authenticates the user using both biometric factors, i.e., voice and fingerprint.

Subsequently, based on the informational value of the utterance, the OS205opens the banking application207and then passes the authentication data or, in a trusted system, a confirmation of authentication, to the banking application207. The banking application207then interacts with the banking web site211in its ordinary manner, e.g., by passing either the authentication data (voice and fingerprint) or an authentication confirmation to the banking web site211.

The banking application207then requests the user's account balance from the banking web site211and shortly thereafter receives the user's account balance from the banking web site211. Finally, the banking application207displays the user's account balance on the display (120,FIG. 1) of the device200.

Since the user has been authenticated for both the device200and the banking web site211, the user may utter further queries or commands to the banking application207via the device without further authentication. If the user utters a command or query that requires a higher level of authentication, the banking application207may request the additional required information or data.

Turning toFIG. 3, the device200stores authentication input data (voice and fingerprint)300and uses it to execute further user commands or requests regarding other applications. As shown in the example ofFIG. 3, the user may utter a further command, such as “Buy 2 tickets for 2:00 showing of Movie X.” The OS205interprets this utterance, authenticates the user, opens the movie ticket application301, which then sends the request to the movie ticket web site303on its server305.

At this point, the user is already fully authenticated with respect to the movie ticket web site303, and hence the movie ticket application301executes the user's ticket request via the website303. It should be noted that if additional security is required, e.g., not just voice and fingerprint but also password, the user may be prompted by the movie ticket application301to supply the additional data by speaking or otherwise entering the movie ticket application password, without having to reenter voice and fingerprint data.

Although certain process flows will be apparent from the foregoing,FIG. 4shows a generalized process400for device behavior in providing a variable and conveniently met security level on a mobile electronic communication device such as the device110(200). The illustrated flow assumes an always on device, but it will be appreciated that additional steps, e.g., device waking and the like, may be added if needed in the context of a device that is not always on.

Continuing, the device receives data representative of a user fingerprint at stage401of the process400, and receives an utterance from the user at stage403. Stages401and403may occur simultaneously or serially. The device then attempts to authenticate the user for the device itself using both voice authentication and fingerprint authentication at stage405. If the authentication attempt is successful, the process400continues to stage407. Otherwise the process400returns to stage401and awaits another device authentication attempt by the user.

Continuing at stage407after a successful authentication to the device itself, the device interprets the user utterance informationally, meaning that the device converts the utterance to language and interprets the language to identify informational content such as a command, request or query. If the command, request or query refers or relates to a device function amenable to voice control, the process400flows to stage409, wherein the device responds accordingly to the command, request or query. If instead the command, request or query refers to an application on the device, the process400flows to stage411, wherein the device opens the indicated application. In this example, the application has a security requirement that is the same as or less than the device, i.e., the application requires voice and fingerprint authentication or requires voice-only or fingerprint-only authentication.

The device then authenticates the user to the application at stage413by passing the already-obtained voice and fingerprint data to the application. Since the device has already authenticated the user via this same data, the application will likely also successfully authenticate the user, and the process will flow to stage415, wherein the device passes the command, request or query to the application for execution. It will be appreciated that the application may use other resources to authenticate the user and/or to fulfill the user's command, request or query. For example, in the bank account example discussed above, the banking application207used an associated bank website211to retrieve the information requested by the user. Similarly, the bank website211may or may not provide authentication services for the application.

The foregoing example is directed to a focused use, wherein the user initially uses the phone for the express purpose of using the particular application. However, it is also possible for the user to authenticate themselves to the device for such an express purpose, but then continue on to another task requiring authentication. Alternatively, the user may be authenticated by the device to use it for general purposes but then at some point decide to use an application requiring the same or different security.

In this connection,FIG. 5is a flowchart illustrating a more general process500for providing a variable security level on a mobile electronic communication device in the context of further user interaction after device authentication, in a case where the device security level requires fingerprint authentication, and the user's subsequent requested action requires a higher level (fingerprint plus voice authentication). As above, the illustrated flow assumes an always-on device, but such is not a requirement in all embodiments.

At stage501of the process500, the device receives data representative of the user's fingerprint. This occurs, e.g., when the user picks up the device and places their finger on the fingerprint sensor pad. The device determines at this point whether any voice input is received at stage503. If voice input is not detected, the device takes its default opening action at stage505, e.g., displaying a background, displaying an array of application icons, etc. The device then waits for a predetermined period at stage507, e.g., N seconds, for any voice input to be detected. If voice input is still not detected at stage507, the process500ends. Otherwise, the process500flows to stage511. Similarly, if voice input was initially detected at stage503, then stages505-509are bypassed and the process500flows directly to stage511.

At stage511, the device generates a voice authentication score, e.g., reflecting the likelihood that the speaker is the authorized party they purport to be. The voice input is then converted to extract the user's intent, e.g., “view account balance,” “open secure office portal,” etc. The device then fuses the voice authentication score and a fingerprint authentication score at stage515. If the combined authentication data score meets or exceeds the security requirements in place to fulfill the user's intent, as checked at stage517, then the device executes the desired action at stage521and the process500exits.

If instead the combined authentication data score does not meet the security requirements in place to fulfill the user's intent, then the process flows to stage519wherein the device rechecks the user's intent and repeats the authorization steps.

Referring now toFIG. 6, this figure shows a process600of voice-initiated authentication. At stage601of the process600, the device, which is executing always-on voice detection, detects a voice input from user. The voice input is interpreted at stage603to identify the user's intent, e.g., a stated request to view a bank balance or send an email. At stage605, the device scores the voice input for user authentication to produce a voiceprint authentication score, e.g., by comparing the voice characteristics to stored voice characteristics of authorized users to determine to what extent the speaker's voice matches a known authorized user. For example a score of 95 may indicate a high likelihood that the speaker is a known authorized user whereas a score of 50 may indicate that the user is equally likely to be unauthorized or authorized.

Regardless of the exact scoring system used, the process600continues to stage607wherein the device determines whether the interpreted intent of the user requires fingerprint authorization in addition to voice authentication. As noted above, certain tasks (e.g., viewing a bank account balance) may require greater security than other tasks (e.g., checking the local weather). If it is determined at stage607that fingerprint authentication is required to fulfill the user intent, then the process flows to stage609, wherein the device determines whether fingerprint data is available.

If fingerprint data is available, then the process600flows to stage613, wherein the device processes the fingerprint data to produce a fingerprint authentication score, and then to stage615wherein the device combines the fingerprint authentication score and the voiceprint authentication score to produce a fused score. The fused score may be produce by simple addition, weighted addition, averaging and so on. If it was determined at stage609that fingerprint data was not available, then the process600would flow to stage611to collect fingerprint data before continuing at stage613.

From stage615, the process flows to stage617. The process can also arrive at stage617from stage607if it was determined at stage607that the user intent does not require fingerprint authorization. At stage617, the device determines whether the authentication score (either the fused score from stage615or the voiceprint score from stage607) is sufficient to allow fulfillment of the user intent. If the authentication score is sufficient to allow fulfillment of the user intent then the process flows to stage619wherein the user intent is fulfilled. Otherwise, the device may recheck the user's authentication level at stage621.

It will be appreciated that a system and method for variable level device and application security have been described herein. However, in view of the many possible embodiments to which the principles of the present disclosure may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the claims. Therefore, the techniques as described herein contemplate all such embodiments as may come within the scope of the following claims and equivalents thereof.