Abstract:
Advantage is taken of the fact that as a player becomes uncomfortable, visually observable aspects of the player&#39;s physiological state, such as the frequency of eye blinks, may change. By analyzing such changes and/or other image changes, as obtained from signals generated by an input device, the system is able to determine changes in a player&#39;s comfort level and change the game accordingly.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    The present application is related to concurrently filed, co-pending, and commonly-assigned U.S. patent application Ser. No. ______, Attorney Docket No. 70051558-01, entitled “SYSTEM AND METHOD FOR USING WAVELET ANALYSIS OF A USER INTERFACE SIGNAL FOR PROGRAM CONTROL,” the disclosure of which is hereby incorporated herein by reference. 
     
     TECHNICAL FIELD 
       [0002]    The present invention relates generally to interactive computer systems and methods, and more particularly to systems and methods for using image analysis of a user interface signal to control a program. 
       BACKGROUND OF THE INVENTION 
       [0003]    Electronic games using computers, game consoles or hand held consoles typically employ an input device, a processor, and a visual display. The input device can be a mouse, joystick or other form of controller which allow the player to input responses into the processor. The processor can be a part of a computer system or it can be a dedicated game system, such as XBOX® and PLAYSTATION® game systems. The processor communicates with the display to show visual and/or audio images of the game. 
         [0004]    These electronic games typically have different levels of play in order to provide a level of play which is entertaining for the player. For many games, the level of play is selected by the user prior to playing the game, and changing the level of play will require the user to restart the game. A player will often have to try out a variety of levels before finding a level which is suitable for his/her level of play. 
         [0005]    Games typically place the player in situations where quick and/or agile response is required and a player&#39;s score is typically reflective of his/her ability to respond quickly to the challenges presented by the game. In many situations, a mouse, joystick or a controller is used to relay positional information which is used by the game to move an object on the display. 
         [0006]    Playing at an appropriate level of play will ensure that the player is challenged by the game, and minimizes the risk of overwhelming or boring the player. However, when user skill levels must be set before a game is started there is not a convenient way in which the user can adjust the skill level short of restarting the game. Often, the user does not even know that the skill level is wrong for that user. 
       BRIEF SUMMARY OF THE INVENTION 
       [0007]    Advantage is taken of the fact that as a player becomes uncomfortable, visually observable aspects of the player&#39;s physiological state, such as the frequency of eye blinks, may change. By analyzing such changes and/or other image changes, as obtained from signals generated by an input device, the system is able to determine changes in a player&#39;s comfort level and change the game accordingly. 
         [0008]    The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantage of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the inventions as set forth in the appended claims. The novel features which are believed to be characteristic of the invention, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention. 
     
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0009]    For a more complete understanding of the present invention, reference is now made to the following description taken in conjunction with the accompanying drawings, in which: 
           [0010]      FIG. 1  is a diagram of one embodiment of the present invention; 
           [0011]      FIGS. 2A and 2B  are overviews of embodiments of the invention; 
           [0012]      FIG. 3  shows one example of image processing used in the invention; and 
           [0013]      FIG. 4  illustrates a block diagram of one embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0014]    As shown in  FIG. 1 , one embodiment of the present invention includes a user visual interface, such as device  22 , which communicates with processor  12  via processing circuit  30 . The program code or algorithm for a game program may be stored in a memory device, such as memory  14 , and executed by processor  12 . The processing shown in circuit  30  (as will be discussed with respect to  FIG. 3 ) can be accomplished in circuitry, in software as part of an algorithm, or partially in each. The processing or circuitry can be separate from, or integral to, processor  12 . 
         [0015]    User interface  22  may be a stand alone device as shown in  FIG. 2  where the visual input device is shown marked on top of display  201 , or it can be incorporated into a computer mouse, a game controller, a joystick, or any other device, such as keyboard  202 , through which the player is able to input his/her actions into processor  12 . Device  22  can be an analog or digital camera or an LED display for capturing images, such as eye blinks, head movements, frequency of face touches, swinging motion of arms or legs, head nods, etc. Processor  12  may include memory devices, microprocessors, and any components which are utilized to execute the program code for a game program. The processor system may be a part of a computer system, a video game console, or a hand held gaming device. 
         [0016]    The visual image, for example, eye blinks, head nods, changes in facial images, etc., can be used as an indicator of a player&#39;s response level. For example, fast eye blinks may suggest a player becoming frustrated or confused. Changes of a player&#39;s response level during game play can be used to change a game program aspect, for example the play difficulty could be adjusted. 
         [0017]    As shown in  FIG. 3 , in one embodiment, amplifier  31  of image processing circuit  30  receives the image signal, for example, from device  22  shown in  FIGS. 2A and 2B . This signal would typically, but not always, be in analog form and would be amplified by amplifier  31  and filtered by filter  32 . The filtering would remove signals outside certain ranges as desired. This filtered signal would be conditioned by circuit  33  and, if necessary, converted to digital format by A/D converter  34 . This signal would then be presented to processor  12 . 
         [0018]    Note that as part of the filtering and/or conditioning, changes of the image could be detected and signaled to the processor, or, if desired, the entire conditioned signal could be presented to the processor for a determination of image change and a determination of what such an image change means in terms of user discomfort, etc. 
         [0019]    Once the player&#39;s response level is determined, processor  12  may modify the system, for example by adjusting the difficulty level of the game being played. In one embodiment of the invention, this includes switching between preset difficulty levels defined in the game or to use a difficulty level that is indicated by a database or algorithm to be appropriate with the determined response level. In another embodiment, the processing system may increase or decrease the speed with which it executes the game algorithm. In a still further embodiment, the difficulty can be continuously adjusted until the user response moves within a defined range. 
         [0020]      FIG. 4  represents a flowchart of one embodiment  40  of the system used to adjust skill level of a game, or other application, based on received visual images. Process  402  acquires a signal from user interface  22  and by image analysis of the signal as discussed with respect to  FIG. 3 , as performed by process  403  and process  404 , a determination is made by process  405  if some significant feature of the image has changed. If it has changed from a previous reading (or a previous set of readings as stored in memory), then process  406  determines the response level of the player based on the nature of the received visual images. Process  402  then adjusts the skill level of the game or other parameters based on the received visual image data. Images can be stored for a period of time in a memory (not shown) and an image can be compared against the previous image, or against a composite (or average) of previous images to determine of the user is agitated, bored, wondering, intense, etc. Rapid eye changes could, for example, mean the user is agitated. If desired, data can be maintained on a user by user basis such that for a particular user a correlation can be made between visual image changes and action (poor, good, etc.) taken by that user. This correlation then can be the foundation of changes made to the program upon future detection of similar visual changes. 
         [0021]    Note that there could be a table established of user levels such that the received visual images are averaged (or otherwise saved) over a period of time can be used to determine an actual graded level of user comfort. Based on the gradation at any point in time the same (or other) program can be adjusted harder or easier. 
         [0022]    Note that while a game system has been described, the concepts taught herein can be used to control the operation aspects of any program running on any processor. For example, a spread sheet program can have different instructions presented based on the frequency of eye movement or other visual image. 
         [0023]    Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods, and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, process, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same results as the corresponding embodiments described herein may be utilized according to the present invention. Accordingly, the appended claims are intended to include within their scope such process, machines, manufacture, composition of matter, means, methods or steps.