Abstract:
A system and method is provided for visual authentication and authorization of a user for mobile devices, the system having: a login display on a mobile selection device displaying a visual pattern, a data collection engine whereby selection features are obtained from a plurality of user selection events to the mobile selection device with reference to the visual pattern, the selection attributes comprise measured selection attributes and derived selection attributes calculated from the measured selection attributes; an authentication engine whereby the selection attributes are compared to projected user selection attributes derived from user selection attribute values obtained during prior successful logins.

Description:
RELATED APPLICATIONS 
     This application claims the benefit of U.S. Provisional Application No. 61/829,354, filed May 31, 2013. This application is a continuation in part of U.S. application Ser. No. 13/481,106 filed on May 25, 2012 which claims benefit to U.S. Provisional Application No. 61/490,924, filed May 27, 2011. These applications are herein incorporated by reference in their entirety for all purposes. 
    
    
     FIELD OF THE INVENTION 
     The invention relates to user authentication systems, and more particularly, to an optical authentication and authorization system for mobile devices. 
     BACKGROUND OF THE INVENTION 
     The keyboard event models are inconsistent across different Smartphone and mobile platforms and identifying a dwell time for a key click on a touch screen seems unfeasible at this time. The programmatic UITouch class in Apple iOS, for example, cannot be utilized due to the fact that a keyboard automatically pops up and disables touch events when user tries to type in the iOS. The keystroke capture measurement of a user&#39;s typing flight time alone is not distinguishable to succinctly identify an individual, and would therefore offer very limited value in the authentication scheme. Many keystroke algorithms are built around the consistency of dwell values, which is how long a user presses a key, for a particular user. Therefore, the QWERTY keyboard biometrics, especially dwell time, are unsuited to touch screen displays and a color pattern usage is more intuitive for a natural user interaction. 
     Keystroke algorithms do not utilize features of touch screen devices that could provide further insight in to the identity of the owner. While known systems employ the idea of using graphical and/or touch (gesture) passwords for authentication, they are not known to use the attributes of the particular touch events. Such systems use continuous gestures as passwords, comparing the drawn gesture to previously trained gestures. They do not use discrete selection events, each with a range of characterizing elements. 
     There are additionally previous patents using discrete selections, but use the selections to perfectly recreate a previous set of selections. For example, selecting a series of points on a displayed photograph or selecting the correct subset from a grid of icons. The user is granted access if the match is perfect. There is no intelligent confidence factor based on how similar the current login touches are to previous touches. 
     In some instances it is advantageous for the user to operate the mobile device without recourse to their hands or speech, either as a result of incapacity or the increased security as a result of the increased difficulty for an observer to recognize non-verbal hands free activities. 
     What is needed, therefore, are techniques for biometrically authenticating a user based on attributes of a sequence of selection events. 
     SUMMARY OF THE INVENTION 
     One embodiment of the present invention provides a system for visual authentication and authorization of a user for mobile devices, the system having: a login display on a mobile device displaying a visual pattern; a data collection engine whereby selection features are obtained from a plurality of user selection events to the mobile device with reference to the visual pattern, the selection attributes comprise measured selection attributes and derived selection attributes calculated from the measured selection attributes; an authentication engine whereby the selection attributes are compared to projected user selection attributes derived from user selection attribute values obtained during prior successful logins. 
     Another embodiment of the present invention provides such a system wherein the visual pattern further comprises inclusion of decoy regions. 
     A further embodiment of the present invention provides such a system wherein the selection attributes comprise a selection attribute selected from the group consisting of exchange, intensity, globularity, selection feature movement velocity, selection feature movement location spread, selection intensity, and selection feature size. 
     Yet another embodiment of the present invention provides such a system wherein the visual pattern is a color grid. 
     A yet further embodiment of the present invention provides such a system wherein the visual pattern is a color wheel. 
     Still another embodiment of the present invention provides such a system wherein the visual pattern comprises a custom key arrangement. 
     A still further embodiment of the present invention provides such a system wherein the data collection engine is configured to collect device attributes selected from the group of device attributes consisting of host name, device type, serial number, International Mobile Equipment Identity (IMEI), electronic serial number (ESN), integrated circuit card identification (ICCID), modem firmware, device unique ID, name of device, model, system version, screen resolution, IP Address, Time Zone, GPS position, and compares the device attributes to device attributes of the user stored in the system. 
     Even another embodiment of the present invention provides such a system embodiment of the present invention provides such a system wherein the projected user selection attributes are averages of the user selection attribute values during prior successful logins. 
     An even further embodiment of the present invention provides such a system wherein the projected user selection attributes are regression analysis outputs of the user selection attribute values during prior successful logins. 
     Still yet another embodiment of the present invention provides such a system wherein at least one user selection event in the plurality is discrete. 
     One embodiment of the present invention provides a method for the authentication of a user of a system, the method having: requiring the user to enter a sequence of selection events to a display displaying a visual pattern; comparing the sequence of selections to a stored selection pattern; capturing a plurality of measured selection attributes; calculating a plurality of derived selection attributes; comparing the derived selection attributes to projected values for the selection attributes of the user; and allowing at least limited access by the user to the system if the derived selection attributes are consistent with the projected values. 
     Another embodiment of the present invention provides such a method further having prompting a user to select a user selection pattern sequence. 
     A further embodiment of the present invention provides such a method further prompting a user to enter the user selection pattern sequence a plurality of times so as to train the system to authenticate the user. 
     Yet another embodiment of the present invention provides such a method wherein the derived selection attributes comprise exchange. 
     A yet further embodiment of the present invention provides such a method wherein the derived selection attributes comprise intensity, 
     Still another embodiment of the present invention provides such a method wherein the derived selection attributes comprise globularity. 
     Another embodiment of the present invention provides such a method wherein the derived selection attributes comprise selection feature movement velocity. 
     Another embodiment of the present invention provides such a method wherein the derived selection attributes comprise selection feature location spread. 
     Another embodiment of the present invention provides such a method wherein the derived selection attributes comprise selection intensity. 
     Another embodiment of the present invention provides such a method wherein the derived selection attributes comprise facial feature size. 
     A still further embodiment of the present invention provides such a method wherein at least one selection event in the sequence of selection events is discrete. 
     The features and advantages described herein are not all-inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and not to limit the scope of the inventive subject matter. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a flow chart illustrating a visual authentication and authorization system for mobile devices configured in accordance with one embodiment of the present invention. 
         FIG. 2  is a flow chart illustrating a user enrollment and training sequence for use in a visual authentication and authorization system for mobile devices configured in accordance with one embodiment of the present invention. 
         FIG. 3  is a block diagram illustrating a display pattern of visual authentication and authorization system for mobile devices configured in accordance with one embodiment of the present invention in which a plurality of color pallets are presented to the user in which the color pallet access code is obscured. 
         FIG. 4  is a block diagram illustrating a visual authentication and authorization system for mobile devices configured in accordance with one embodiment of the present invention with colorized QWERTY keyboards. 
         FIG. 5  is a block diagram illustrating a visual authentication and authorization system for mobile devices configured in accordance with eye movement activation. 
         FIG. 6  is a perspective view illustrating a visual authentication and authorization system for mobile devices configured in accordance with one embodiment of the present invention with eye closure activation. 
         FIG. 7  is a perspective view illustrating a visual authentication and authorization system for mobile devices configured in accordance with one embodiment of the present invention with eye closure activation shown mid selection. 
     
    
    
     DETAILED DESCRIPTION 
     As illustrated in the attached figures, a functional flow representation of one embodiment of the present invention implemented on a mobile native architecture or app like Apple iOS or Android. In such an embodiment, a new and innovative selection pattern identifying a user is supplied as an alternative to the QWERTY Keyboard biometric analysis. In one such embodiment the pattern may be a color pattern, while those skilled in the art will appreciate that other visual cues or images may be used to elicit a response from the user. A color pattern idea is especially useful in a Smartphone scenario because of the following. 
     Another possibility is to utilize visual capture features of the devices to capture selections made by a variation in a selection feature, without any touch or keystroke happening on the device. The movement of a facial feature or a body part can signify selections; each with a range of characterizing elements, which can be compared to previously trained selections. 
     The system configured according to one embodiment will not misbehave in scenarios like, when the user is traveling, walking, driving or doing some other “work or bodies in motion” and using the mobile device simultaneously. A Keyboard biometric pattern based analysis in these cases would have yielded inconsistent values; identifying or authenticating a user would have been problematic. 
     In embodiments of the present invention having selection Pattern analysis there will be changes involved in the functional flow and integration as compared to the existing systems. In such a system, illustrated in  FIG. 1 , a primary sequence is entered and initiated as a user accesses something requiring authentication  12  in which a user is prompted to enter a “password-like” sequence of selections (via touch or other selection mode) to the screen  14 . Each selection event in the sequence may comprise contact or hands free interaction of the user with the display at one or more point. In one embodiment each selection event may be discrete, that is, being a contact that is substantially non-moving in the x-y coordinate plane of the display. Raw data or measured quantities or measure selection attributes are collect while the user enters the data  16 . The pattern or selection sequence is tested against a stored pattern  18  and if the sequence is inaccurate, user access is denied  20  without further biometric selection analysis. Visual patterns displayed by the device may include solid color fields, icons, images, photo, geometric patterns, animations or videos, or simulation of device turnoff. In instances where the sequence is correct, further analysis is conducted during which derived quantities are calculated from measured selection attributes  22 . Measured and derives quantities are then compared to projected values derived from prior successful logins  24 . While in one embodiment the projected value may be an average of all previous login, in other embodiments a small sample or subset of logins could be used. In a further embodiment, the projected value may use a regression analysis of past logins. If the measured and derived quantities are consistent with projected values, the user is allowed access  26  but is otherwise denied access  20 . 
     Where a user has not been previously enrolled or is opening a new authentication account, the new user is prompted to set up the authentication features in an enrollment procedure such as are illustrated in  FIG. 2 . The user is prompted with the need to access the system  28 . The user is then asked to select a selection pattern  30 . While the prompt may be any image or pattern, one embodiment of the present invention uses a color palette. Such an embodiment is described for clarity. In an embodiment where a color palette is the prompt, a palette is selected by the user  30 , during the process of selecting a palette, the Device, and Geographic (such as global positioning data) attributes are recorded  32 . 
     Once enrolled, a user may either be prompted to train the device to recognize the user, or allowed access, while the selection pattern recognition is itself only activated once sufficient data has been collected. The decision as to whether training is required, may, in embodiments designed for enterprises be made without consultation of the user, while in consumer embodiments, the user may be allowed to decide if they consider training necessary for their own security requirements. The embodiment illustrated in  FIG. 2  is shown as a consumer embodiment. In such an embodiment the user decides if training of the system is required  34 . If it is, the user is prompted to repeatedly enter the selection pattern  36 . Once the user enters a number of iterations of the selection pattern  38  the user reaches an iteration threshold completing the training. The number of iterations in an iteration threshold may in some embodiments be set by the user or be pre-defined by the system. If training is either declined or the threshold iterations have been reached, the user is allowed access  40 . 
     As illustrated in  FIG. 1 , where a user is subject to selection pattern recognition requirements, measured data or measured selection attributes are collected from the selection pattern submitted as the primary sequence. Examples of measured selection attributes, which may be used in various embodiments, are provided in Table 1. As noted some of these items will be more readily useful in the context than others. 
     
       
         
               
               
               
               
             
               
               
               
               
             
           
               
                   
                 TABLE 1 
               
               
                   
                   
               
               
                   
                 Measured 
                   
                   
               
               
                   
                 Quantities 
                 Android 
                 iOS 
               
               
                   
                   
               
             
             
               
                   
               
             
          
           
               
                   
                 Touch or stare 
                 Yes 
                 yes 
               
               
                   
                 start time 
                   
                   
               
               
                   
                 Touch or stare 
                 Yes 
                 yes 
               
               
                   
                 end time 
                   
                   
               
               
                   
                 X coordinate 
                 Yes 
                 yes 
               
               
                   
                 Y coordinate 
                 Yes 
                 yes 
               
               
                   
                 Size 
                 Yes 
                 no 
               
               
                   
                 Pressure 
                 Yes 
                 no 
               
               
                   
                 Touchmajor 
                 Yes 
                 no 
               
               
                   
                 Touchminor 
                 Yes 
                 no 
               
               
                   
                 Orientation of 
                 Yes 
                 no 
               
               
                   
                 touch or glance 
                   
                   
               
               
                   
                 X accelerometer 
                 Yes 
                 yes 
               
               
                   
                 Y accelerometer 
                 Yes 
                 yes 
               
               
                   
                 Z accelerometer 
                 Yes 
                 yes 
               
               
                   
                 X orientation (of 
                 yes 
                 yes 
               
               
                   
                 device via 
                   
                   
               
               
                   
                 gyroscope) 
                   
                   
               
               
                   
                 Y orientation 
                 Yes 
                 yes 
               
               
                   
                 (gyroscope) 
                   
                   
               
               
                   
                 Z orientation 
                 Yes 
                 Yes 
               
               
                   
                 (gyroscope) 
                   
                   
               
               
                   
                 Selection 
                 Yes 
                 Yes 
               
               
                   
                 Feature 
                   
                   
               
               
                   
                 Movement 
                   
                   
               
               
                   
                 Velocity 
                   
                   
               
               
                   
                 Selection 
                 Yes 
                 Yes 
               
               
                   
                 Feature 
                   
                   
               
               
                   
                 Location Spread 
                   
                   
               
               
                   
                 Selection 
                 Yes 
                 Yes 
               
               
                   
                 Intensity 
                   
                   
               
               
                   
                 Selection 
                 Yes 
                 Yes 
               
               
                   
                 Feature Size 
               
               
                   
                   
               
             
          
         
       
     
     Examples of additional derived quantities or characteristics are provided in Table 2. 
     
       
         
               
               
               
             
           
               
                 TABLE 2 
               
               
                   
               
               
                 Derived 
                   
                   
               
               
                 Quantities 
                 Definition 
                 Description 
               
               
                   
               
             
             
               
                 Dwell 
                 end(event)-start(event) 
                 The duration of each 
               
               
                   
                   
                 individual selection 
               
               
                   
                   
                 event. 
               
               
                 Flight 
                 start(next event)- 
                 The time between each 
               
               
                   
                 end(event) 
                 individual selection 
               
               
                   
                   
                 event. 
               
               
                 &lt;x&gt; 
                 Average x-coordinate 
                 Average horizontal 
               
               
                   
                 of event 
                 position of each 
               
               
                   
                   
                 individual selection 
               
               
                   
                   
                 event. 
               
               
                 &lt;y&gt; 
                 Average y-coordinate 
                 Average vertical 
               
               
                   
                 of event 
                 position of each 
               
               
                   
                   
                 individual selection 
               
               
                   
                   
                 event. 
               
               
                 δx 
                 difference between x 
                 The difference in 
               
               
                   
                 coordinates of 
                 horizontal position 
               
               
                   
                 successive events 
                 between successive 
               
               
                   
                   
                 selection events. 
               
               
                 δy 
                 difference between y 
                 The difference in 
               
               
                   
                 coordinates of 
                 vertical position 
               
               
                   
                 successive events 
                 between successive 
               
               
                   
                   
                 selection events. 
               
               
                 &lt;δx&gt; 
                 average of δx 
                 The average horizontal 
               
               
                   
                   
                 position difference 
               
               
                   
                   
                 between successive 
               
               
                   
                   
                 selection events. 
               
               
                 &lt;δy&gt; 
                 average of δy 
                 The average vertical 
               
               
                   
                   
                 position difference 
               
               
                   
                   
                 between successive 
               
               
                   
                   
                 selection events. 
               
               
                 Location 
                 {square root over ([(x-&lt;x&gt;) 2 +(y-&lt;y&gt;) 2 ])} 
                 The distance from a  
               
               
                   
                   
                 particular selection and 
               
               
                   
                   
                 the average location of 
               
               
                   
                   
                 that selection event. 
               
               
                 Exchange 
                 {square root over ([(δx-&lt;δx&gt;) 2  + (δy-)} 
                 The difference in 
               
               
                   
                 (δy- &lt;δy&gt;) 2 ] 
                 location between 
               
               
                   
                   
                 successive selection 
               
               
                   
                   
                 events compared to the 
               
               
                   
                   
                 expected difference in 
               
               
                   
                   
                 location. 
               
               
                 Intensity 
                 Time series of 
                 How consistent the 
               
               
                   
                 pressure sampled 
                 position of a selection 
               
               
                   
                 repeatedly during 
                 even is throughout an 
               
               
                   
                 selection event 
                 individual selection 
               
               
                   
                   
                 event 
               
               
                 Globularity 
                 Spatial extent of 
                 The total area of the  
               
               
                   
                 Selection Event 
                 selection screen being 
               
               
                   
                   
                 selected during an 
               
               
                   
                   
                 individual selection 
               
               
                   
                   
                 event. 
               
               
                 X-force 
                 Time series of x 
                 The X direction 
               
               
                   
                 accelerometer sampled 
                 acceleration of the 
               
               
                   
                 repeatedly during 
                 mobile device during an 
               
               
                   
                 event 
                 individual selection 
               
               
                   
                   
                 event. 
               
               
                 Y-force 
                 Time series of y 
                 The Y direction 
               
               
                   
                 accelerometer sampled 
                 acceleration of the 
               
               
                   
                 repeatedly during 
                 mobile device during an 
               
               
                   
                 event 
                 individual selection 
               
               
                   
                   
                 event. 
               
               
                 Z-force 
                 Time series of z 
                 The Z direction 
               
               
                   
                 accelerometer sampled 
                 acceleration of the 
               
               
                   
                 repeatedly during 
                 mobile device during an 
               
               
                   
                 event 
                 individual selection 
               
               
                   
                   
                 event. 
               
               
                 X-motion 
                 Time series of x 
                 The component of 
               
               
                   
                 orientation 
                 device orientation in 
               
               
                   
                   
                 the X direction during 
               
               
                   
                   
                 an individual selection 
               
               
                   
                   
                 event. 
               
               
                 Y-motion 
                 Time series of y 
                 The component of 
               
               
                   
                 orientation 
                 device orientation in 
               
               
                   
                   
                 the Y direction during 
               
               
                   
                   
                 an individual selection 
               
               
                   
                   
                 event.  
               
               
                 Z-motion 
                 Time series of z 
                 The component of 
               
               
                   
                 orientation 
                 device orientation in 
               
               
                   
                   
                 the Z direction during  
               
               
                   
                   
                 an individual selection  
               
               
                   
                   
                 event.  
               
               
                 Active shift 
                 Initial acceleration 
                 The initial acceleration  
               
               
                   
                 and subsequent motion 
                 and subsequent 
               
               
                   
                 along direction 
                 movement caused by  
               
               
                   
                 perpendicular to and 
                 pressing on the screen 
               
               
                   
                 into screen 
                 during a selection 
               
               
                   
                   
                 event.  
               
               
                 Reactive 
                 Reactive acceleration 
                 The slowdown and 
               
               
                 shift 
                 and subsequent motion 
                 movement back to the 
               
               
                   
                 along direction 
                 original device position 
               
               
                   
                 perpendicular to and 
                 once a particular 
               
               
                   
                 out of screen 
                 selection event is over. 
               
               
                 &lt;v&gt; 
                 Average selection 
                 The speed at which a 
               
               
                   
                 feature movement 
                 selection feature is in 
               
               
                   
                 velocity 
                 motion between 
               
               
                   
                   
                 selection events. 
               
               
                 &lt;δs&gt; 
                 Average selection 
                 The difference in 
               
               
                   
                 feature location 
                 location between 
               
               
                   
                 spread 
                 multiple simultaneous 
               
               
                   
                   
                 selection features. 
               
               
                 &lt;μ&gt; 
                 Average selection 
                 How radical the change 
               
               
                   
                 intensity 
                 to signify selection is. 
               
               
                 &lt;Σ&gt; 
                 Determined selection 
                 The ratio of sizes 
               
               
                   
                 feature size 
                 between detected  
               
               
                   
                   
                 selection features. 
               
               
                   
               
             
          
         
       
     
     In one embodiment, each iteration of the selection pattern allows further data collection and further refines the system&#39;s profile of a particular user allowing for greater accuracy in identification. 
     In alternative embodiments such as that of  FIG. 3  a series of palettes  42  may be offered in the login, one of which would contain the 4 color “pin”. Alternatively, as in  FIG. 4  a QWERTY capability may be offered in one of two colors  44 ,  46  doubling the number of possible combinations for each password. As in  FIG. 5 , a color wheel  48  may be used. A color wheel could be equipped with any number of possible colors increasing the number of possible combinations. 
     One embodiment of the present invention, illustrated in  FIGS. 6-7 , provides a system in which colors or image  42  or sequence may be selected by movement of an eye  50  tracked via a camera or selection tracker  52  disposed in the mobile device. In one embodiment a system of eye or selection tracking may be used such as that sold under the TOBII gaze interaction mark. No claim is made to TOBII gaze interaction, which is a trademark of TOBII Technology. One skilled in the art would appreciate that other suitable APIs or proprietary software may be used which provide for facial or facial feature recognition. In some such systems eyes are detected by the system through the use of multiple near IR microprojectors, which create reflection patters on subject&#39;s eyes, then register the reflected points via image sensors. The reflected images allow the system to identify the position of the eyes in real time and track gaze direction. Systems may be configured to identify physical characteristics of a user eye, monitor for eye closure movement and patterns of movement and dwell point, pupil dilation. 
     When a user selection event occurs, the variation in a selection feature points at a particular spot  54  on the displayed visual pattern or image  42 . The distance from this particular selection point and the average position of this selection point is the location attribute. The exchange is calculated as the difference in this location between successive collection points compared to the expected difference. The intensity of a selection event is how consistent the selection point is throughout an individual selection event. Since the selection feature can move during a selection event, the total area that the selection covered during an individual selection event is the globularity of that selection event. 
     In such a system “key strokes” may be identified with either prolonged eye positioning (i.e. a stare) at a particular item to be selected, or through the use of blinks or eye to signal the device that the user selects that particular color or key. In one embodiment the choice between blinking or timed staring to identify the selected color or symbol is a setting selected by the user. A user may elect one or the other method depending on their personal preference or taking into considerations environmental factors like allergens, bright sunlight, or wind. 
     The foregoing description of the embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of this disclosure. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.