Abstract:
In a user-interactive system, such as an interactive voice response (IVR) system, the duration and/or force of key-presses by the user and/or the speed of response to prompts by the user are used to make inferences about the user, such as whether the user is impaired or is growing frustrated. The user interface is modified based on the inferences.

Description:
TECHNICAL FIELD 
     This invention relates generally to interactive systems, and specifically to interactive communications systems. 
     BACKGROUND OF THE INVENTION 
     Callers who experience difficulty with an interactive voice response (IVR) system may become so frustrated that they hang up before accomplishing their objective. This can result in toward, and lost revenue for, the called party. Thus, there is a need to detect when callers are having trouble, and intervene on their behalf. The like considerations exist not only in IVR systems, but in any systems that require interaction with a user, such as Automatic Call Distribution (ACD) systems, for example. 
     In this context, users with cognitive and/or physical disabilities might have a harder time navigating a typical IVR or other interactive user interface than the general population. Therefore, disability access rules enforced by the FCC require support for people who are cognitively impaired. Specifically, the Code of Federal Regulations, 36 C.F.R. Part 1193.41(i), states the following: “Input, control, and mechanical functions shall be operable with limited cognitive skills. Provide at least one mode that minimizes the cognitive, memory, language, and learning skills required of the user.” This regulation implies that IVR systems should adjust their behavior to accommodate the needs of cognitively impaired users. Unfortunately, it is often not practical—and perhaps even illegal—to maintain a database that identifies specific caller IDs or specific accounts as belonging to someone who has a disability. The problem then is, how can an IVR system detect automatically that a user is impaired? A way of detecting need for assistance is to monitor a user&#39;s interaction with the system. Indeed, some IVR systems monitor key-press sequences in order to detect that a user is having problems. For example, it may be assumed that users need additional help when they frequently ask for menus to be repeated, or if they are observed to go down a branch of a menu, come back up, go down a different branch, and then come back up again. Key-press duration has been used to determine the action to be taken. For example, on an electric typewriter, it is common for the “key repeat” function to be triggered by holding down a key longer than is necessary for a single character to be typed. And on electronic watches, it is common for a programming mode change to be triggered by holding down a button. Such time-dependent controls are explicit components of the user interface. In the mid 1980s, studies were conducted by Telcordia Technologies (then named Bell Communications Research, Inc.) to determine whether measurements of inter-digit pauses while personal identification numbers (PINs) are being entered might help to detect when the PINs are being entered by someone other than their owners. 
     SUMMARY OF THE INVENTION 
     According to one aspect of the invention, the duration of actuations of an actuator (e.g., key-presses) by a user are used to make inferences about the user, such as whether or not the user is having difficulty with the user interface and needs assistance. According to this aspect, the duration of actuation of an actuator is determined; in response to a long duration, the user interface is modified; and in response to a not-long duration, the interface is not so modified. According to another aspect of the invention, the speed of response to prompts (i.e., the delay between prompts to a user and the user&#39;s actuations of an actuator in response to those prompts) is used to make inferences about the user. According to this aspect, the speed of response to a prompt is determined; in response to a slow speed, the user interface is modified; and in response to a not-slow speed, the interface is not so modified. According to yet another aspect of the invention, the force, pressure, of actuation of an actuator is also used to make inferences about the user, such as whether the user is growing frustrated. According to this aspect, the force of actuation of an actuator is determined; in response to a high force, the user interface is modified; and in response to a not high force, the interface is not so modified. Preferably, a system uses the duration of the actuation, the force of the actuation, and/or delay data, in combination with other metrics (for example, the number of times that a user asks for a menu to be repeated) in order to improve the accuracy of the inference. 
     The user interface is adjusted based on the inferences. For example, if it is detected that a user&#39;s key-press durations change during a single IVR session from a normal level to an extended level, it is inferred that the user is having trouble or is growing frustrated, and the system can respond by transferring the call to a human agent. Or, if it is detected that the key-press durations are extremely long at the start of the session, and remain long during the session, it is inferred that the user has a disability of some sort. This could be a physical disability, such as a motor-skill impairment, or a cognitive disability. It is possible to distinguish between the two by examining other metrics; for example, frequent requests for menus to be repeated is assumed to indicate cognitive or language issues, rather than a physical disability. Or, the user can be questioned in order to determine the nature of the user&#39;s disability. On the one hand, when a cognitive or language problem is assumed to be present, the IVR system can adjust the user interface by, for example, slowing down the playback rate of the speech, adjusting the menu structure so that fewer options are presented at each node (thereby making the available options easier to remember), explaining each option in greater detail, and/or requesting confirmation of the user&#39;s entries from the user. On the other hand, an accommodation for people with motor impairments is an automatic lengthening of the call-terminating time-out period, or of the system-imposed inter-digit time-out period, when users are entering a series of characters (e.g., dialing a phone number). 
     Conversely, if it is detected that a user&#39;s key-presses have unusually short durations, it may be inferred that the user is an experienced user who is familiar with the interface, and the system can respond by making the interface navigable by the user more quickly, such as by giving the user terse prompts and more choices at each prompt, and playing the prompts at a faster speed. 
     Although these examples describe telephony applications, the idea is useful in other domains, such as with interfaces to portable electronic devices and to computers, and with Internet applications. For example, the technique is as useful when applied to button-press durations on a mouse or stylus-press durations on a computer as it is for dial-pad key-press durations. 
     While the invention has been characterized above as a method, it also includes apparatus that performs the method, as well as a computer-readable medium containing instructions which, when executed by a computer, cause the computer to perform the method. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWING 
       These and other features and advantages of the invention will become more apparent from considering the following description of an illustrative embodiment of the invention together with the drawing, in which: 
         FIG. 1  is a block diagram of a system that includes an illustrative embodiment of the invention; 
         FIG. 2  is a functional flow diagram of a first illustrative embodiment of an interface application of the system of  FIG. 1 ; and 
         FIG. 3  is a functional flow diagram of a second illustrative embodiment of the interface application of the system of  FIG. 1 . 
     
    
    
     DETAILED DESCRIPTION 
       FIG. 1  is a block diagram of a system that includes an illustrative embodiment of the invention. The system includes a key entry device  116  by means of which a user enters information by pressing keys, a timing device  110  for timing key-press durations and/or delays, and a responsive device  100  that responds to the key-press durations and/or delays. Device  100  is typically computerized and comprises a processor  120  and a memory  122  or any other computer-readable medium storing data and/or programs for use by processor  120 ,  1  including an interface application program  122  that implements a user interface  125  for a user of key entry device  116 . 
     System  100  is described generically, and may take many different forms in actual implementation. For example, the system of  FIG. 1  may be a telecommunication system where key entry device  116  is the dial keypad of a telephone, timing device  110  is a timing function of a telephone switch, and responsive device  100  is an IVR system. Or, the system of  FIG. 1  may be a computer or a personal digital assistant (PDA) where key entry device  116  is a keyboard, mouse, stylus, and/or a touch-sensitive screen, timing device  110  is a timing function of the computer or PDA, and responsive device  100  comprises a display of the computer or PDA. Or, the system of  FIG. 1  may be a game controller where key entry device  116  is a joystick and button game control, timing device  110  is a timing function of the game controller, and responsive device  100  is the speaker and display or display output of the game controller. These are just a few illustrative examples of the myriad forms that the system of  FIG. 1  can take. 
       FIG. 2  shows functionality of one illustrative embodiment the system of  FIG. 1  where key entry device  116  is a wireless phone and responsive device  100  is an IVR system executing interface application  124 . The wireless phone detects the speed of its user&#39;s response to IVR prompts, the inter-digit intervals between key-presses, and the key-press durations and force, at step  200 , and reports them to the IVR system, for example, via the tone-duration field of system messages of the IETF RPC 2833 signaling protocol. The IVR system compares the response speed and inter-digit intervals against historical records for either this user or the population at large to determine if the response speed is slow and inter-digit time is long, at step  202 . If so, the IVR system compares the key-press durations and force against historical records to determine if the durations are long or the force is high, at step  204 . If so, the system infers that the user is distracted, perhaps because he or she is using the telephone while driving, and therefore it lengthens the inter-digit time-out and call-terminating time-out periods, at step  206 . Returning to step  204 , if the IVR system determines that the key-press durations are long or forceful, it infers that the user is impaired, and therefore it simplifies the menus that it presents to the user, at step  210 , and offers assistance of an automatic speech recognition (ASR) interface to the user, at step  212 . 
     Returning to step  202 , if the IVR system determines that the response time and the inter-digit time are not slow, it compares the key-press durations and force against historical records to determine if the durations are long or the force is high, at step  214 . If so, the system infers that the user is frustrated, and therefore connects the user to a human agent, at step  216 . If the IVR system determines at step  214  that the key-presses are not long or forceful, it infers that the user needs no assistance with the user interface, and therefore it continues to present the standard, unmodified, interface to the user, at step  218 . 
       FIG. 3  shows functionality of a second illustrative embodiment the system of  FIG. 1  where key entry device  116  is a fixed phone and responsive device  100  is an IVR system executing interface application  124 . The switching system to which the phone is connected acts as timing device  110  and monitors the user&#39;s key-press durations, at step  300 , which it then reports to the IVR system. The IVR system stores the duration information. If the IVR system detects that the key-press duration is consistently longer-than-normal, at step  302 , it compares the newly-arriving duration information against the stored information to determine if the durations of key-presses are progressively getting longer, at step  304 . If so, the IVR system infers that the user is getting frustrated, and so it connects the user to an agent, at step  306 ; if not, IVR system queries the user to determine what the problem may be, at step  310 . For example, it may query whether it would help the user to slow down the playing speed of the menu prompts. It might also query the user whether fewer options per menu would be helpful. Other variations in the user interface could be offered and implemented based on feedback from the user. It then uses the user&#39;s responses to determine if the user is physically or cognitively impaired, at step  312 . If the user is believed to be cognitively impaired, the IVR system changes the prompt-menu structure to present simpler choices to the user, and also slows down the playback speed of the prompts, at step  314 . The actual difference between a distracted user and one with cognitive impairment may not matter in terms of the options offered to the user, since both can require the same action. The IVR system also starts to provide confirmations of the user&#39;s responses to the user, at step  316 . If the user is found, at step  312 , to have a physical impairment, the IVR system queries the user for whether he or she would prefer to use an ASR interface, or to speak to an agent, at step  318 . 
     Returning to step  302 , if the key-press durations are not consistently found to be longer-than-normal, the IVR system checks whether the key-press durations are consistently shorter-than-normal, at step  330 . If so, the IVR system infers that it is interacting with an experienced user, or one that has normal abilities and is in a hurry, and so it compresses the prompts that it gives to the user, at step  334 , for example, by making the prompts terser, presenting more choices in each announcement, and increasing the playback speed of each announcement. If the IVR system keeps historical records on past interactions with users, the IVR system may also offer to skip the menu and short-cut the user to a cached menu of the user&#39;s favorite choices, at step  336 . If the IVR system finds at step  330  that the key-press durations are not consistently shorter-than-normal, it just continues to present the standard interface to the user, at step  332 . 
     Of course, various changes and modifications to the illustrative embodiment described above will be apparent to those skilled in the art. For example, other environments in which the invention may be advantageously employed include interacting with an ACD system, a game where the controller adjusts the sensitivity of the joy stick and/or the user feedback based on the input observed at a previous level in the game so that the game learns to be tougher as the user&#39;s skills at the game gets better, interacting with a burglar alarm keypad where the user gets better over timer at the use of the system so as to benefit from speed-up but where the user on occasion may be slower than typical because he or she is carrying something, and interacting with a TV remote control where detection that the operator is having problems may result in the TV screen being used to display to the operator the steps involved in using the remote to reach a particular program and/or activation of an easier interface that requires the operator to use just the arrow keys and an “enter” button. Such changes and modifications can be made without departing from the spirit and the scope of the invention and without diminishing its attendant advantages. It is therefore intended that such changes and modifications be covered by the following claims except insofar as limited by the prior art.