Abstract:
Method and apparatus for identifying locations on a touchscreen of at least two touch events that occur within a predetermined time of one another comprises monitoring the touchscreen for touch events. Each touch event occurs at a discrete location on the touchscreen defined by an XY coordinate pair. A coordinate series is generated including at least two X coordinates and at least two Y coordinates when first and second touch events occur within a predetermined time of one another. When a release event occurs, the release event is correlated with one of the X coordinates and one of the Y coordinates in the coordinate series to form a first XY coordinate pair corresponding to the first touch event. The first XY coordinate pair associated with the first touch event is output.

Description:
BACKGROUND OF THE INVENTION  
       [0001]     This invention relates generally to touch input systems, and more particularly, to touch input systems in which there can be multiple touches overlapping in time, and to methods and apparatus for identifying the locations of multiple touch inputs  
         [0002]     Touch input systems have become ubiquitous throughout industrialized countries. These systems have replaced or supplemented conventional input systems, such as a keyboard or mouse in many applications, including for example, information kiosks, retail point of sale, order input (e.g. restaurants), and industrial line operations. Various sensing technologies are applied in touch input systems currently in the marketplace, including acoustic, resistive, capacitive and infrared. A touch input system is typically used in conjunction with some type of information display system that may include a computer. When a user touches a displayed object, the touch input system communicates the location of the touch to the system.  
         [0003]      FIGS. 1 and 2  show conventional touch sensor systems and touch input systems. The touch sensor system  100  generally comprises a touchscreen  105  (also called a touch screen), an example of which may be a touch sensor having a transparent substrate. The system  100  also comprises a lead  111  coupling a controller  110  to the touchscreen  105 . A touchscreen system comprising the touchscreen  105  and controller  110  may be used in conjunction with a display device  115 . The touch sensor system  100  is configured to respond to a touch on the touchscreen  105  by causing acoustic waves to be transmitted across the touchscreen  105 , one or more of which are modulated in the presence of the touch. The controller  110  in turn uses the modulated signal from the waves to identify the location of the touch on the touchscreen  105 . The controller  110  also uses the modulated signal to distinguish between valid touches and invalid signals (e.g., signals generated by contamination on the surface of the screen). If the controller  110  identifies a touch as valid, it transmits the touch&#39;s location to a host computer (not shown) that then implements a corresponding computer finction to display the pertinent information, e.g., graphics, on the display device  115 . Graphics or other information may be displayed on the display device  115  in response to an operator&#39;s command, e.g. touching a particular area of the touchscreen  105 .  
         [0004]      FIG. 2  illustrates an acoustic wave touch input system  102 . A transparent sensor substrate  120  having a surface  122  covers a screen of a display system. The transparent sensor substrate  120  is typically made of glass. The wave energy is directed along one or more paths that form an invisible XY grid overlaying the substrate surface  122  wherein a touch to the surface  122  causes wave energy to be attenuated.  
         [0005]     A first transmitting transducer  125  and a first receiving transducer  135  are provided in two corners of the substrate  120 , with the corners being located on a first vertical side of the substrate  120 . The first transmitting transducer  125  transmits acoustic waves in the horizontal right direction to be received by the first receiving transducer  135 . A second transmitting transducer  130  and a second receiving transducer  140  are oriented perpendicularly to the first transmitting and receiving transducers  125  and  135  on a first horizontal side of the substrate  120 . Both the transmitting transducers  125  and  130  and the receiving transducers  135  and  140  may be, for example, piezoelectric transducers. Two reflector arrays  200  and  205  are provided on both horizontal sides of the substrate  120 , and two reflector arrays  210  and  215  are provided on both vertical sides of the substrate  120 . The reflector arrays partially reflect waves from the transmitting transducers to the receiving transducers.  
         [0006]     The controller  110  sends signals to the transmitting transducers  125  and  130  through lines  160  and  165 , and the transmitting transducers  125  and  130  generate acoustic energy that is launched across the substrate  120  and reflected by the reflector arrays. The controller  110  accepts signals from the receiving transducers  135  and  140  through lines  190  and  195 , and the received signals include timing and signal amplitude. The controller  110  comprises coded instructions (stored, for example, in a memory of a microprocessor), which when executed, perform steps to control and process the relevant signals. The controller  110  need not comprise a computer, but may be implemented in hardware, firmware, software or any combination thereof. The time the wave takes to travel from the transmitting transducers  125  and  130  to the receiving transducers  135  and  140  via the reflector arrays  200 ,  205 ,  210  and  215  is dependent on the path length, and therefore the position of an attenuation within the wave can be correlated to the time at which it was received relative to the time it was launched. Waves are periodically and repetitively propagated in both the X and Y directions of the substrate  120  in order to allow the detection of coordinates of a touch event location  250 . The time between the repetitive propagation of waves is the sampling time.  
         [0007]     One disadvantage of touch input systems incorporating the propagation and detection of acoustic waves is that if two or more points are pressed or touched concurrently or within a specific same sampling period of the system, the receiving transducers  135  and  140  will detect multiple X coordinates and multiple Y coordinates within a single time interval in which the coordinates are read, and as such the touch location may be identified by multiple distinct coordinate pairs. This is illustrated in  FIG. 3  for the case of two concurrent touch events indicated at locations  250  and  251 . In the example shown in  FIG. 3 , there are two possible combinations of X and Y pairs which could indicate touch locations  252  and  253 , which are not the actual touch locations. Therefore, for applications that need the capability to sense multiple concurrent touches, improvements over conventional systems are desired.  
         [0008]     Multiple touches that overlap in time may be detected as simultaneous events. Simultaneous touches occur when the start times for two touches are the same within the time resolution of the system (e.g., the time resolution of the microchip controller of the system). Features of the system that can limit time resolution include analog to digital sampling rate, wave propagation velocity, bandwidth of analog circuits, and the like. For example, if the controller  110  samples the touchscreen  105  at a rate of  100  times per second, then touch events arriving within 0.01 second of each another cannot be resolved in time. In some applications, it is likely that two touches will occur somewhere in the screen within 0.01 second. For example, in a video game involving head-to-head competition, this probability may be very high.  
         [0009]     Therefore, a need exists for a method and apparatus for identifying the locations of touch events occurring within the same time period. Certain embodiments of the present invention are intended to meet these needs and other objectives that will become apparent from the description and drawings set forth below.  
       BRIEF SUMMARY OF THE INVENTION  
       [0010]     In one embodiment, a method for identifying locations on a touchscreen of at least two touch events that occur within a predetermined time of one another comprises monitoring the touchscreen for touch events. Each touch event occurs at a discrete location on the touchscreen defined by an XY coordinate pair. A coordinate series is generated including at least two X coordinates and at least two Y coordinates when first and second touch events occur within a predetermined time of one another. When a release event occurs, the release event is correlated with one of the X coordinates and one of the Y coordinates in the coordinate series to form a first XY coordinate pair corresponding to the first touch event. The first XY coordinate pair corresponding to the first touch event is output.  
         [0011]     In another embodiment, an apparatus for correlating coordinates representative of at least two touch events on a touchscreen that occur within a predetermined time of one another comprises a touchscreen having a touch surface for receiving touch events. Each touch event occurs at a discrete location on the touch surface defined by an XY coordinate pair. A touchscreen controller monitors the touch surface for the touch events. The touchscreen controller identifies at least two X coordinates and at least two Y coordinates when at least two touch events occur within a predetermined time of one another. A buffer receives at least two X coordinates and at least two Y coordinates from the touchscreen controller. The touchscreen controller forms a first XY coordinate pair based on a release event associated with a first touch.  
         [0012]     In another embodiment, a method for pairing coordinates representative of multiple touch events on a touch apparatus that occur within the same measurement period comprises receiving a first set of signals representative of coordinate locations along a first axis. A second set of signals representative of coordinate locations along a second axis is received. Consecutively received sets of signals are compared to the first and second sets of signals to identify a missing signal component in the consecutively received sets of signals. Coordinate pairs are identified based on the missing signal component. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0013]      FIG. 1  shows a conventional touch sensor system.  
         [0014]      FIG. 2  illustrates an acoustic wave touch input system.  
         [0015]      FIG. 3  illustrates the case of two concurrent touch events.  
         [0016]      FIG. 4  illustrates a touch sensor system capable of resolving multiple touch situations in accordance with an embodiment of the present invention.  
         [0017]      FIG. 5  illustrates an acoustic wave touch input system in accordance with an embodiment of the present invention.  
         [0018]      FIG. 6  illustrates a method for resolving multiple touch situations in accordance with an embodiment of the present invention. 
     
    
       [0019]     The foregoing summary, as well as the following detailed description of certain embodiments of the present invention, will be better understood when read in conjunction with the appended drawings. The figures illustrate diagrams of the functional blocks of various embodiments. The functional blocks are not necessarily indicative of the division between hardware circuitry. Thus, for example, one or more of the functional blocks (e.g., processors or memories) may be implemented in a single piece of hardware (e.g., a general purpose signal processor or a block or random access memory, hard disk, or the like). Similarly, the programs may be stand alone programs, may be incorporated as subroutines in an operating system, may be functions in an installed imaging software package, and the like. It should be understood that the various embodiments are not limited to the arrangements and instrumentality shown in the drawings.  
       DETAILED DESCRIPTION OF THE INVENTION  
       [0020]      FIG. 4  illustrates a touch sensor system  260  capable of resolving multiple touch situations in accordance with an embodiment of the present invention. The touch sensor system  260  comprises the display device  115  with the touchscreen  105  and transparent sensor substrate  120  as previously discussed. A controller  262  is interconnected with the touchscreen  105  with the lead  111 . The controller  262  further comprises at least one buffer  264  and  266  for temporarily storing coordinate information and/or signals representative of coordinate information.  
         [0021]     A microprocessor  268  may receive signals from the touchscreen  105  and determine the coordinate information of touch events as discussed below. The microprocessor  268  may then output the coordinate information to another device such as a central or host computer  272  via lead  270 . It should be understood that the coordinate information passed through the lead  270  is representative only. In addition, information may be output in many forms and formats by the computer  272 , such as text or graphics on the display device  115 , a different display device or monitor, a light, a bell, an initiation or termination of an action, and the like. Therefore, the information passed through the lead  270  may change based on the purpose of the touch sensor system  260 . Optionally, the controller  262  may be located within a monitor or the display device  115 , in a separate unit as illustrated, or within the computer  272 .  
         [0022]      FIG. 5  illustrates an acoustic wave touch input system  280  in accordance with an embodiment of the present invention. Elements in common with  FIGS. 2 and 3  are labeled with like item numbers. Although surface acoustic waves (SAW) are illustrated, it should be understood that other sensing technologies may also be used, including, but not limited to, acoustic, resistive, capacitive and infrared.  
         [0023]      FIG. 6  illustrates a method for resolving multiple touch situations in accordance with an embodiment of the present invention. FIGS.  4  to  6  will be discussed together.  
         [0024]     In step  300 , the controller  262  begins the scan process to continuously monitor the touchscreen  105  for touch events. For example, the controller  262  may send a signal to the first transmitting transducer  125  via line  160 . The first receiving transducer  135  sends a first returning signal via line  190  to the controller  262 . The controller  262  then sends a signal to the second transmitting transducer  130  via line  165 . The second receiving transducer  140  sends a second returning signal via line  195  to the controller  262 . As stated previously, the returning signal includes timing and signal amplitude information representative of touch events, if present. Therefore, controller  262  constantly sends and receives signals in both the X and Y directions in order to detect the coordinates of one or more touch events. The time between the repetitive propagation of waves is the sampling rate or time. A measurement period may be determined as the time period for the microprocessor  268  to send and receive the first and second sets of signals.  
         [0025]     In step  302 , the microprocessor  268  analyzes the first and second returning signals to determine whether one or more X and Y coordinates are detected. If no X or Y coordinates are detected, the first and second returning signal information may be discarded. If at least one X and at least one Y coordinate are detected, flow passes to step  304 . It should be understood that steps  300  and  302  are repeatedly performed so that the touchscreen  105  is continuously monitored for touch events.  
         [0026]     In step  304 , the microprocessor  268  stores the detected X and Y coordinates in one or more buffers  264  and  266 . For example, a first coordinate series of X coordinates may be stored in a memory or buffer  264  and a second coordinate series of Y coordinates may be stored in a memory or buffer  266 . Alternatively, a single buffer  264  may be used to store all detected coordinates. Optionally, sets of signals representative of the coordinates may be stored, wherein the microprocessor  268  or other device may identify the actual X and Y coordinate locations later.  
         [0027]     In step  306 , the microprocessor  268  determines whether the pairing of the X and Y coordinates can be determined; indicating that a discrete location has been touched on the touchscreen  105 . For example, if a single touch occurs at touch location  282 , an X 1  coordinate and a Y 1  coordinate are returned. The microprocessor  268  forms the coordinate pair (X 1 , Y 1 ), and in step  308 , the microprocessor  268  transmits the XY coordinate pair, (X 1 , Y 1 ) and clears the buffers  264  and  266 . The XY coordinate pair may be transmitted to a central or host computer  272  for implementation of the desired function.  
         [0028]     However, if touch events occur at touch locations  282  and  284  such that, in step  302 , the microprocessor  268  detects coordinate series X 1 , X 2  and Y 1 , Y 2  within a predetermined time or measurement period of one another, the pairing of the X and Y coordinates cannot be determined and flow passes to step  310 . The predetermined time may, for example, be based on a sampling rate or time in which the touchscreen  105  is monitored for touch events (step  300 ). It should be understood that more than two touch events may be detected at the same time, resulting in additional X and Y coordinates to be paired. For example, touch location  288  (X 4 , Y 4 ) may be detected at the same time as touch locations  282  and  284 .  
         [0029]     In step  310 , the microprocessor  268  delays the transmission of any coordinates. Continuing the example above of touch locations  282  and  284 , the coordinate series X 1 , X 2  and Y 1 , Y 2  are retained in the buffers  264  and  266 . The microprocessor  268  continues to scan for touch events, such as in step  300 .  
         [0030]     In step  312 , the microprocessor  268  compares the currently detected coordinates (such as a consecutively acquired coordinate series or sets of signals) with the coordinates and/or signals saved in the buffers  264  and  266  to determine if a change has been detected. If the same coordinates, X 1 , X 2  and Y 1 , Y 2  are detected, the microprocessor  268  determines that continuous touches have occurred and flow returns to step  310 . No coordinates are transmitted, the current coordinates remain in the buffers  264  and  266 , and the microprocessor  268  continues to scan for touch events. Optionally, the microprocessor  268  may identify the coordinates as unchanged when within a tolerance, such as to account for a slight finger movement or roll of the user&#39;s finger along the touch surface.  
         [0031]     Returning to step  312 , the microprocessor  268  may also determine that a change has occurred based on one of relative timing of the touch events, absolute touch intensity, rate of change of touch intensity, correlation of touch intensity over multiple measurement cycles, and touch movement (i.e. dragging or rolling finger). These changes may allow the microprocessor  268  to pair coordinates by using other comparison methods in addition to the method of  FIG. 6 .  
         [0032]     If the microprocessor  268  detects one additional coordinate, either an X or Y coordinate, but not both, flow passes to step  314 . This may occur if touch location  286 , having the coordinates (X 1 , Y 3 ), is detected. Therefore, the X coordinate locations of touch locations  282  and  286  are the same, and the coordinates cannot be paired. The Y 3  coordinate is stored, such as in the buffer  266 , and flow returns to step  310 . Alternatively, the microprocessor  268  may discard or disregard the additional coordinate depending upon the application.  
         [0033]     If the microprocessor  268  detects an additional touch event, such as at touch location  290  having coordinates (X 5 , Y 5 ), flow passes from step  312  to step  316 . The microprocessor  268  can pair the new set of coordinates (X 1   5 , Y 5 ), however, depending upon the processing algorithms and system implementation, the microprocessor  268  may transmit the paired coordinates (X 5 , Y 5 ), save the paired coordinates (X 5 , Y 5 ) in one of the buffers  264  and  266 , or discard the paired coordinates (X 5 , Y 5 ).  
         [0034]     If the microprocessor  268  detects that one less X and one less Y coordinate is present in a subsequent returned signal, a release event has occurred and flow passes from step  312  to step  318 . This may occur when a user lifts a finger or stylus from the touchscreen  105 . In step  318 , the microprocessor  268  correlates the release event with one of the touch events, such as by comparing the subsequently returned signals to the coordinates or signals stored in the buffers  264  and  266  to identify the missing X and Y coordinates. The missing X and Y coordinates or signal components correlate to a touch location and can be paired. Therefore, if the microprocessor  268  identifies that the returned signals now contain only the X 2  and Y 2  coordinates, the microprocessor  268  can pair the previously identified coordinates (X 1 , Y 1 ) and (X 2 , Y 2 ), which were stored in the buffers  264  and  266 .  
         [0035]     In step  320 , the microprocessor  268  determines whether additional coordinates are to be paired. For example, if touch events occurred at the touch locations  282 ,  284  and  288  and were detected in step  302  at substantially the same time or within a predetermined time of one another, in step  318  the microprocessor  268  would be able to pair only the X and Y coordinates associated with the lift off event. Using the example above, the microprocessor  268  has paired the coordinates of touch location  282  (X 1 , Y 1 ) (step  318 ) and returns to step  310  if the additional coordinates are to be paired. Additional coordinates may be paired by detecting a second lift off or release event. Depending upon the processing algorithms being used, the microprocessor  268  may output the paired coordinates or save the paired coordinates in one of the buffers  264  and  266 . The unpaired coordinates remain stored in the buffers  264  and  266 .  
         [0036]     In step  320 , if no additional coordinates are to be paired, flow passes to step  322 , and the XY coordinate pair(s) are output or transmitted to the central or host computer  272  for implementation of the desired function. Optionally, the microprocessor  268  may also identify and/or transmit the coordinate pair associated with the lift off, and/or identify and/or organize the sets of coordinates based on a predetermined hierarchy.  
         [0037]     In addition to video games, dual or multiple touch situations may also be encountered when using keyboards simulated on a touch display, such as when selecting a particular option, object or key combination on a keyboard, such as the shift key in combination with another key to create a capital letter or characters used in emoticons. Also, international keyboards have the need to resolve multiple touch situations to create character combinations. In addition, dual or multiple touch capability may be desired to implement critical situations, where it is required to select certain combinations of keys or inputs to initiate or terminate an action, such as simultaneous selection of two keys or touch points to confirm the start of a potentially dangerous operation in a factory.  
         [0038]     While the invention has been described in terms of various specific embodiments, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the claims.