Abstract:
An interactive input system comprises imaging devices with different viewpoints and having at least partially overlapping fields of view encompassing a region of interest. At least two of the imaging devices have different focal lengths. Processing structure processes image data acquired by the imaging devices to detect the existence of a pointer and determine the location of the pointer within the region of interest.

Description:
FIELD OF THE INVENTION 
       [0001]    The present invention relates to an interactive input or touch system and method. 
       BACKGROUND OF THE INVENTION 
       [0002]    Touch systems are well known in the art and typically include a touch screen or panel having a touch surface on which contacts are made using a pointer in order to generate user input. Pointer contacts with the touch surface are detected and are used to generate corresponding output depending on areas of the touch surface where the contacts are made. Common touch systems utilize analog resistive, electromagnetic, capacitive, acoustic or machine vision to identify pointer interactions with the touch surface. 
         [0003]    For example, International PCT Application No. PCT/CA01/00980 filed on Jul. 5, 2001 and published under No. WO 02/03316 on Jan. 10, 2002, assigned to SMART Technologies Inc., assignee of the present application, discloses a camera-based touch system comprising a touch screen that defines a touch surface on which a computer-generated image is presented. Depending on the application, a front or rear projection device may be used to project the image that is visible on the touch surface. A rectangular bezel or frame surrounds the touch surface and supports wide-angle digital cameras at its corners. The digital cameras have overlapping fields of view that encompass and look generally across the entire touch surface from different viewpoints. The digital cameras acquire images looking across the touch surface and generate image data. Image data acquired by the digital cameras is processed by digital signal processors to determine if a pointer exists in the captured image data. When it is determined that a pointer exists in the captured image data, the digital signal processors convey pointer characteristic data to a master controller, which in turn processes the pointer characteristic data to determine the location of the pointer in (x,y)-coordinates relative to the touch surface using triangulation. The pointer coordinate data is conveyed to a computer executing one or more applications programs. The computer uses the pointer coordinate data to update the computer-generated image that is presented on the touch surface. Pointer contacts on the touch surface can therefore be recorded as writing or drawing or used to control execution of applications programs executed by the computer. 
         [0004]    In many environments such as in teaching institutions, large scale touch systems are desired so that visible presentations can be made to large groups. To satisfy this need, a large scale touch system as disclosed in U.S. patent application Ser. No. 10/750,219 to Hill et al. and assigned to SMART Technologies Inc., assignee of the subject application, has been developed. This large scale touch system includes a touch panel having a plurality of input sub-regions. The input sub-regions overlap to define a generally contiguous input surface. Each coordinate input sub-region comprises a set of wide-angle digital cameras having different viewpoints that look across an associated portion of the input surface. Each input sub-region processes image data captured by the digital cameras and generates pointer coordinate data in response to pointer contacts on the associated portion of the input surface. The pointer coordinate data is processed to update image data presented on the input surface. When a pointer contact is made on a coordinate input sub-region that does not overlap with an adjacent coordinate input sub-region, the coordinate input sub-region processes acquired images to derive pointer data and triangulates the position of the pointer using the derived pointer data thereby to determine the position of the pointer contact relative to the input surface. When a pointer contact is made on a coordinate input sub-region that overlaps with an adjacent coordinate input sub-region, each overlapping coordinate input sub-regions processes acquired images to derive pointer data and triangulates the position of the pointer using the derived pointer data. Thereafter, the triangulated positions generated by the overlapping coordinate input sub-regions are processed in accordance with defined logic thereby to determine the position of the pointer contact relative to the input surface. 
         [0005]    Although the above touch systems work extremely well, resolution issues arise as the size of the touch surface increases. Typically cameras with very wide fields of view are employed so that each camera sees the entire touch surface. However, when a pointer is brought into contact with the touch surface at a location that is far from one or more of the cameras, the pointer may appear very small to those cameras. In fact, to the cameras, the pointer may appear to be only one (1) or two (2) pixels wide making pointer detection difficult and unreliable. As will be appreciated, there is a need to improve pointer detection especially in touch systems having very large touch surfaces. 
         [0006]    It is therefore an object of the present invention to provide a novel interactive input system and method. 
       SUMMARY OF THE INVENTION 
       [0007]    Accordingly, in one aspect there is provided in an interactive input system comprising: 
         [0008]    imaging devices with different viewpoints and having at least partially overlapping fields of view encompassing a region of interest, at least two of said imaging devices having different focal lengths; and 
         [0009]    processing structure processing image data acquired by the imaging devices to detect the existence of a pointer and determine the location of the pointer within the region of interest. 
         [0010]    In one embodiment, at least some of the imaging devices are arranged in pairs. One imaging device of each pair has a wide field of view and the other imaging device of each pair has a narrow field of view. The wide field of view fully encompasses the narrow field of view. The imaging devices of each pair may be stacked vertically or arranged side-by-side. 
         [0011]    In one embodiment, a touch surface is associated with the region of interest. Pairs of imaging devices are positioned adjacent corners of the touch surface. The imaging devices of each pair look generally across the touch surface. For each imaging device pair, the processing structure processes image data acquired by each imaging device of the pair to determine if a pointer is believed to exist in the image data with a desired level of confidence and further processes that image data to determine the location of the pointer. The desired level of confidence is existence of a pointer beyond a threshold size. The processing structure processes the image data acquired by the imaging devices of each pair to verify at least one of pointer existence and pointer location. 
         [0012]    According to another aspect there is provided a touch system comprising: 
         [0013]    a touch surface on which an image is visible; 
         [0014]    imaging assemblies about the periphery of said touch surface, said imaging assemblies having at least partially overlapping fields of view encompassing said touch surface, each imaging assembly comprising at least two imaging devices with each imaging device having a different focal length; and 
         [0015]    processing structure processing data generated by the imaging assemblies to determine the location of at least one pointer relative to the touch surface. 
         [0016]    According to yet another aspect there is provided an interactive input system comprising: 
         [0017]    camera assemblies with different viewpoints and having fields of view encompassing a region of interest, each camera assembly comprising at least two image sensors with the image sensors having different focal lengths; and 
         [0018]    processing structure processing image data acquired by said camera assemblies to detect one or more pointers in said region of interest. 
         [0019]    The interactive input system and method provides advantages in that reliable pointer detection can be achieved even in instances where the pointer is remote from one or more of the imaging devices. In addition, as in some instances imaging devices of different focal lengths see the same object, data extracted from the images acquired by the imaging devices can be used to calibrate the imaging devices and verify the pointer location. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0020]    Embodiments will now be described more fully with reference to the accompanying drawings in which: 
           [0021]      FIG. 1  is a front plan view of a camera-based interactive input system; 
           [0022]      FIG. 2  is a schematic diagram of the interactive input system of  FIG. 1 ; 
           [0023]      FIG. 3  is an enlarged front plan view of a corner of the touch panel of  FIG. 2 ; 
           [0024]      FIG. 4  is a front plan view of a camera assembly forming part of the touch panel of  FIG. 2 ; and 
           [0025]      FIG. 5  is a front plan view of the touch panel of  FIG. 2  showing the fields of view of the camera assemblies. 
       
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
       [0026]    Referring now to  FIGS. 1 and 2 , a camera-based touch system is shown and is generally identified by reference numeral  50 . Camera-based touch system  50  is similar to that disclosed in previously referenced International PCT Application Serial No. WO 02/03316, assigned to SMART Technologies Inc., assignee of the subject application, the content of which is incorporated herein by reference. 
         [0027]    As can be seen, touch system  50  includes a touch panel  52  coupled to a digital signal processor (DSP)-based master controller  54 . Master controller  54  is also coupled to a computer  56 . Computer  56  executes one or more application programs and provides computer-generated image output to a projection device  58 . Projection device  58  in turn presents a computer-generated image that is visible on the surface  60  of the touch screen  52 . The touch panel  52 , master controller  54 , computer  56  and projection device  58  form a closed-loop so that pointer contacts on the touch surface  60  can be recorded as writing or drawing or used to control execution of application programs executed by the computer  56 . 
         [0028]    The touch surface  60  is bordered by a bezel or frame  62  similar to that disclosed in U.S. Pat. No. 6,972,401 to Akitt et al. issued on Dec. 6, 2005, assigned to SMART Technologies, Inc. assignee of the subject application, the content of which is incorporated herein by reference. A DSP-based digital camera assembly  70  having on-board processing capabilities, best seen in  FIGS. 3 and 4 , is positioned adjacent each top corner of the touch surface  60  and is accommodated by the bezel  62 . In this embodiment, each digital camera assembly  70  comprises a pair of camera sensors  72  and  74  that look across the touch surface  60  and a processing unit (not shown) communicating with the camera sensors. The focal lengths of the camera sensors  72  and  74  are different as will be described. The camera sensors  72  and  74  of each digital camera assembly  70  are vertically stacked on top of one another. The optical axes of the camera sensors  72  and  74  are in line with the diagonals of the touch surface  60  and thus, the optical axes bisect the diagonally opposite corners of the touch surface  60 . 
         [0029]    The lower camera sensor  72  of each digital camera assembly  70  has a wide angle lens giving the camera sensor  72  a wide field of view so that the lower camera sensor  72  sees the entire touch surface  60 . The upper camera sensor  74  of each digital camera assembly  70  has a narrow angle lens giving the camera sensor  74  a long and narrow field of view so that the upper camera sensor  74  sees only a portion of the touch surface  60 . In this embodiment, the lower camera sensor  72  has a field of view equal to about 95°. The upper camera sensor  74  has a field of view in the range of from about 30° to 60°. Those of skill in the art will however appreciate that other combinations of fields of view for the lower and upper camera sensors  72  and  74  can be selected. As the camera sensors  72  and  74  are stacked on top of one another, the field of view (FOV 1 ) of the lower camera sensor  72  fully encompasses the field of view (FOV 2 ) of the upper camera sensor  74  as shown in  FIG. 5 . In this manner, the upper camera sensors  74  are best suited to detect accurately pointers that are distant from the digital camera assemblies  70  while the lower camera sensors  72  are best suited to detect accurately pointers that are proximate to the digital camera assemblies  70 . 
         [0030]    During operation of the touch system  50 , the camera sensors  72  and  74  of each digital camera assembly  70 , look across the touch surface  60  and acquire images. For each digital camera assembly  70 , image data acquired by each camera sensor  72  and  74  thereof, is processed by the processing unit to determine if a pointer is believed to exist in each captured image with a desired level of confidence (i.e. the pointer is above a threshold size in the captured image). As will be appreciated, when the pointer is remote from a digital camera assembly  70 , only its upper camera sensor  74  will detect the existence of a pointer with the desired level of accuracy and when the pointer is near the digital camera assembly  70 , only its lower camera sensor  72  will detect the existence of the pointer with the desired level of accuracy. When a pointer is determined to exist in one of the captured images with the desired level of confidence, pointer characteristic data is derived from that captured image identifying the pointer position in the captured image. If the pointer is determined to exist in both captured images with the desired level of confidence, the pointer characteristic data is derived from the captured image in which the pointer appears the largest. 
         [0031]    The pointer characteristic data derived by each digital camera assembly  70  is then conveyed to the master controller  54 , which in turn processes the pointer characteristic data in a manner similar to that described in U.S. Pat. No. 6,954,197 to Morrison et al. issued on Oct. 4, 2005, assigned to SMART Technologies Inc., assignee of the subject application, the content of which is incorporated by reference, so that a bounding box surrounding the pointer contact on the touch surface  60  is determined allowing the location of the pointer in (x,y)-coordinates to be calculated. 
         [0032]    The pointer coordinate data is then reported to the computer  56 , which in turn records the pointer coordinate data as writing or drawing if the pointer contact is a write event or injects the pointer coordinate data into the active application program being run by the computer  56  if the pointer contact is a mouse event. As mentioned above, the computer  56  also updates the image data conveyed to the projection device  58  so that the image presented on the touch surface  60  reflects the pointer activity. 
         [0033]    If desired, the image processing results during pointer existence determination for both the upper and lower camera sensors  72  and  74  of each digital camera assembly  70  can be compared to verify the existence of the pointer. Pointer characteristic data for each captured image can also be generated and compared to verify the location of the pointer within the captured images. Also, as the camera sensors  72  and  74  of each digital camera assembly  70  both see the same pointer when the pointer is brought towards the touch surface  60 , pointer data derived from acquired images can be used to calibrate the camera sensors  72  and  74  of the digital camera assemblies  70 . 
         [0034]    Although the digital camera assemblies  70  are described as having vertically stacked camera sensors  72  and  74  with the field of view of the wide angle camera sensor  72  fully encompassing the field of view of the narrow angle camera sensor  74 , those of skill in the art will appreciate that other camera assembly arrangements are possible. For example, the camera sensors  72  and  74  of the digital camera assemblies  70  may be arranged side-by-side with the field of view of the wide angle camera sensors  72  still fully encompassing the field of view of the narrow angle camera sensors  74 . Of course, other camera sensor orientations are possible. The field of view of the wide angle camera sensor  72  need not fully encompass the field of view of the narrow angle camera sensor  74 . The fields of view of the wide angle and narrow angle camera sensors can of course only partially overlap. As will be appreciated, in this arrangement there is less redundancy. 
         [0035]    In the embodiment described above, although each camera assembly  70  is described as comprising two camera sensors  72  and  74  communicating with a single processing unit, each camera sensor may communicate with an associated processing unit. In this case, the processing units of each camera assembly  70  communicate to determine which processing unit is to provide pointer data to the master controller  54 . In situations where a pointer is seen best by one camera sensor but the pointer is moving in a direction that is better viewed by the other camera sensor, the processing units can communicate pointer data between one another to ensure accurate pointer tracking as responsibility for tracking the pointer is handed from one processing unit to the other. 
         [0036]    The touch system  50  as described above comprises a pair of digital camera assemblies  70  positioned adjacent the top corners of the touch surface  60 . Those of skill in the art will appreciate that additional camera assemblies  70  may be disposed about the periphery of the touch surface  60 , especially when the touch surface is very large as described in aforementioned U.S. patent Ser. No. 10/750,219 to Hill et al. 
         [0037]    As will be appreciated by those of skill in the art, the pointer may be a finger, a passive or active stylus or other object, a spot of light or other radiation or other indicator that can be seen by the cameras. Although the touch system is described as including digital cameras, other imaging devices such as for example linear optical sensors that are capable of generating an image may be employed. 
         [0038]    In the embodiments described above, pointer contacts made on a touch surface are detected and tracked. Those of skill in the art will appreciate that a touch surface is not required and that pointers intersecting a two-dimensional plane or within a three-dimensional volume that is viewed by the imaging devices may be detected and tracked. 
         [0039]    Although embodiments have been described above, those of skill in the art will also appreciate that variations and modifications may be made without departing from the spirit and scope thereof as defined by the appended claims.