Abstract:
A method for resolving ambiguities between at least two pointers in a plurality of input regions defining an input area of an interactive input system. The method includes capturing images of the plurality of input regions, the images captured by a plurality of imaging devices having a field of view of at least a portion of the input area, processing image data from the images to identify a plurality of targets for the at least two pointers within the input area, and analyzing the plurality of targets to resolve a real location associated with each pointer.

Description:
FIELD OF THE INVENTION 
       [0001]    The present invention relates generally to input systems and in particular to a multi-touch interactive input system. 
       BACKGROUND OF THE INVENTION 
       [0002]    Interactive input systems that allow users to inject input such as for example digital ink, mouse events etc. into an application program using an active pointer (eg. a pointer that emits light, sound or other signal), a passive pointer (eg. a finger, cylinder or other object) or other suitable input device such as for example, a mouse or trackball, are well known. These interactive input systems include but are not limited to: touch systems comprising touch panels employing analog resistive or machine vision technology to register pointer input such as those disclosed in U.S. Pat. Nos. 5,448,263; 6,141,000; 6,337,681; 6,747,636; 6,803,906; 7,232,986; 7,236,162; and 7,274,356 and in U.S. Patent Application Publication No. 2004/0179001 assigned to SMART Technologies ULC of Calgary, Alberta, Canada, assignee of the subject application, the contents of which are incorporated by reference; touch systems comprising touch panels employing electromagnetic, capacitive, acoustic or other technologies to register pointer input; tablet personal computers (PCs); laptop PCs; personal digital assistants (PDAs); and other similar devices. 
         [0003]    Above-incorporated U.S. Pat. No. 6,803,906 to Morrison et al. discloses a touch system that employs machine vision to detect pointer interaction with a touch surface on which a computer-generated image is presented. A rectangular bezel or frame surrounds the touch surface and supports digital cameras at its four corners. The digital cameras have overlapping fields of view that encompass and look generally across the touch surface. The digital cameras acquire images looking across the touch surface from different vantages and generate image data. Image data acquired by the digital cameras is processed by on-board digital signal processors to determine if a pointer exists in the captured image data. When it is determined that a pointer exists in the captured image data, the digital signal processors convey pointer characteristic data to a master controller, which in turn processes the pointer characteristic data to determine the location of the pointer in (x,y) coordinates relative to the touch surface using triangulation. The pointer coordinates are then conveyed to a computer executing one or more application programs. The computer uses the pointer coordinates to update the computer-generated image that is presented on the touch surface. Pointer contacts on the touch surface can therefore be recorded as writing or drawing or used to control execution of application programs executed by the computer. 
         [0004]    In environments where the touch surface is small, more often than not, users interact with the touch surface one at a time, typically using a single pointer. In situations where the touch surface is large, as described in U.S. patent application Ser. No. 10/750,219 to Hill et al., assigned to SMART Technologies ULC, the content of which is incorporated by reference, multiple users may interact with the touch surface simultaneously. 
         [0005]    As will be appreciated, in machine vision touch systems, when a single pointer is in the fields of view of multiple imaging devices, the position of the pointer in (x,y) coordinates relative to the touch surface typically can be readily computed using triangulation. Difficulties are however encountered when multiple pointers are in the fields of view of multiple imaging devices as a result of pointer ambiguity and occlusion. Ambiguity arises when multiple pointers in the images captured by the imaging devices cannot be differentiated. In such cases, during triangulation a number of possible positions for the pointers can be computed but no information is available to the system to allow the correct pointer positions to be selected. Occlusion occurs when one pointer occludes another pointer in the field of view of an imaging device. In these instances, the image captured by the imaging device includes only one pointer. As a result, the correct positions of the pointers relative to the touch surface cannot be disambiguated from false target positions. Increasing the number of imaging devices allows pointer ambiguity and occlusion to be resolved but this of course results in increased touch system cost and complexity. The placement of the additional cameras with different vantages improves disambiguation as well as difficulties with occlusion as the field of view of each set of cameras looks at the pointers at a different angle. The additional cameras also improve the triangulation of the pointers. 
         [0006]    It is therefore an object of the present invention to provide a novel interactive input system. 
       SUMMARY OF THE INVENTION 
       [0007]    Accordingly, in one aspect there is provided in an interactive input system, a method of resolving ambiguities between at least two pointers in a plurality of input regions defining an input area comprising: 
         [0008]    capturing images of the plurality of input regions, the images captured by a plurality of imaging devices having a field of view of a portion of the input area; 
         [0009]    processing image data from the images to identify a plurality of targets for the at least two pointers within the input area; and 
         [0010]    analyzing the plurality of targets to resolve a real location associated with each pointer. 
         [0011]    According to another aspect there is provided in an interactive input system, a method of resolving ambiguities between at least two pointers in a plurality of input regions defining an input area comprising: 
         [0012]    capturing images of the plurality of input regions, the images captured by a plurality of imaging devices having a field of view of at least a portion of the input area; 
         [0013]    processing image data from the images to identify a plurality of potential targets for the at least two pointers within the input area, the plurality of potential targets comprising real and phantom targets; and 
         [0014]    determining a pointer location for each of the at least two pointers utilizing the plurality of targets. 
         [0015]    According to another aspect there is provided an interactive input system comprising: 
         [0016]    an input surface defining an input area; and 
         [0017]    a plurality of imaging devices having at least partially overlapping fields of view encompassing a plurality of input regions within the input area. 
         [0018]    According to another aspect there is provided an interactive input system comprising: 
         [0019]    at least one imaging device mounted on the periphery of the display surface having a field of view encompassing a region of interest associated with the display surface; 
         [0020]    a bezel disposed around the periphery of a display surface, defining an input area, the bezel having an inwardly facing diffusive surface normal to the display surface, the bezel positioned proximate to the at least one imaging device; 
         [0021]    at least one light source disposed within the bezel to illuminate the region of interest. 
         [0022]    According to another aspect there is provided a bezel for an interactive input system comprising at least one bezel segment to be disposed around the periphery of an input surface, the at least one bezel segment having a front surface, two opposing side surfaces, a top surface, a bottom surface, and a back surface, and the back surface tapering towards the midpoint of the bezel. 
         [0023]    According to another aspect, there is provided an interactive input system comprising: 
         [0024]    an input surface having at least two input areas; 
         [0025]    a plurality of imaging devices having at least partially overlapping fields of view encompassing at least one input regions within the input area; and 
         [0026]    a processing structure for processing image data acquired by the imaging devices to track the position of at least two pointers within the input regions and resolve ambiguities between the pointers. 
         [0027]    According to another aspect there is provided an interactive input system comprising: 
         [0028]    an input surface defining an input area; and 
         [0029]    at least three imaging devices having at least partially overlapping fields of view encompassing at least one input region within the input area; 
         [0030]    a processing structure for processing images acquired by the imaging devices to track the position of at least two pointers within the at least one input region, assigning a weight to each image, and resolve ambiguities between the pointers based on each weighted image. 
         [0031]    According to another aspect there is provided in an interactive input system, a method comprising: 
         [0032]    capturing images of a plurality of input regions, the images captured by a plurality of imaging devices having a field of view of a portion of the plurality of input regions; 
         [0033]    processing image data from the images to identify a plurality of targets for the at least two pointers within the input area; 
         [0034]    determining a state for each image of each target and assigning a weight to the image data of each image based on the state; and 
         [0035]    calculating a pointer location for each of the at least two pointers based on the weighted image data. 
         [0036]    According to another aspect there is provided a computer readable medium embodying a computer program for resolving ambiguities between at least two pointers in a plurality of input regions defining an input area in an interactive input system, the computer program code comprising: 
         [0037]    program code for resolving ambiguities between at least two pointers in a plurality of input regions defining an input area comprising: 
         [0038]    program code for capturing images of the plurality of input regions, the images captured by a plurality of imaging devices having a field of view of at least a portion of the input area; 
         [0039]    program code for processing image data from the images to identify a plurality of targets for the at least two pointers within the input area; and 
         [0040]    program code for analyzing the plurality of targets to resolve a real location associated with each pointer. 
         [0041]    According to another aspect there is provided a computer readable medium embodying a computer program for resolving ambiguities between at least two pointers in a plurality of input regions defining an input area in an interactive input system, the computer program code comprising: 
         [0042]    program code for capturing images of the plurality of input regions, the images captured by a plurality of imaging devices having a field of view of at least a portion of the input area; 
         [0043]    program code for processing image data from the images to identify a plurality of potential targets for the at least two pointers within the input area, the plurality of potential targets comprising real and phantom targets; and 
         [0044]    program code for determining a pointer location for each of the at least two pointers utilizing the plurality of targets. 
         [0045]    According to yet another aspect there is provided a computer readable medium embodying a computer program for resolving ambiguities between at least two pointers in a plurality of input regions defining an input area in an interactive input system, the computer program code comprising: 
         [0046]    program code for capturing images of a plurality of input regions, the images captured by a plurality of imaging devices having a field of view of a portion of the plurality of input regions; 
         [0047]    program code for processing image data from the images to identify a plurality of targets for the at least two pointers within the input area; 
         [0048]    program code for determining a state for each image of each target and assigning a weight to the image data of each image based on the state; and 
         [0049]    program code for calculating a pointer location for each of the at least two pointers based on the weighted image data. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0050]    Embodiments will now be described more fully with reference to the accompanying drawings in which: 
           [0051]      FIG. 1  is a perspective view of an interactive input system; 
           [0052]      FIG. 2  is another perspective view of the interactive input system of  FIG. 1  with the cover removed to expose the imaging devices and illuminated bezel surrounding the input area; 
           [0053]      FIG. 3  is yet another perspective view of the interactive input system of  FIG. 1  with the cover removed; 
           [0054]      FIG. 4  is an enlarged perspective view of a portion of the interactive input system of  FIG. 1  with the cover removed; 
           [0055]      FIG. 5  is a top plan view showing the imaging devices and illuminated bezel that surround the input area; 
           [0056]      FIG. 6  is a side elevational view of a portion of the interactive input system of  FIG. 1  with the cover removed; 
           [0057]      FIG. 7  is a top plan view showing the imaging devices and input regions of the input area; 
           [0058]      FIG. 8  is a schematic block diagram of one of the imaging devices; 
           [0059]      FIG. 9  is a schematic block diagram of a master processor; 
           [0060]      FIGS. 10   a  and  10   b  are perspective and top plan views, respectively, of a bezel segment forming part of the illuminated bezel; 
           [0061]      FIG. 10   c  shows a generic diffuser on the bezel segment of  FIGS. 10   a  and  10   b;    
           [0062]      FIG. 11   a  is a front elevational view of an exemplary illuminated bezel segment showing the dimple pattern of the diffusive front surface; 
           [0063]      FIGS. 11   b  and  11   c  are front elevational views of the illuminated bezel segment showing alternative dimple patterns of the diffusive front surface; 
           [0064]      FIG. 12  is a perspective view of a portion of the illuminated bezel segment showing an alternative diffusive front surface; 
           [0065]      FIG. 13  is a flow chart showing the steps performed by during a candidate generation procedure; 
           [0066]      FIG. 14  is an observation table built by the candidate generation procedure; 
           [0067]      FIG. 15  is a flow chart showing the steps performed during an association procedure; 
           [0068]      FIG. 16  shows an example of multiple target tracking; 
           [0069]      FIGS. 17 and 18  show two targets contacting the input area  62  and the weights assigned to the observations associated with the targets; 
           [0070]      FIGS. 19 to 24  show multiple target scenarios, determined centerlines for each target observation and the weights assigned to the target observations; 
           [0071]      FIG. 25  is a flow chart showing the steps performed during triangulation of real and phantom targets; 
           [0072]      FIGS. 26 to 34  show alternative imaging device configurations for the interactive input system; 
           [0073]      FIGS. 35 to 40  show alternative embodiments of the bezel segments of the interactive input system; and 
           [0074]      FIG. 41  shows exemplary frame captures of the input area showing the three possible states for multiple targets as seen by an imaging device. 
       
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
       [0075]    Turning now to  FIGS. 1 to 6 , an interactive input system is shown and is generally identified by reference numeral  50 . In this embodiment, the interactive input system  50  is in the form of a touch table that is capable of detecting and tracking individually eight (8) different pointers or targets brought into proximity of the touch table. As can be seen touch table  50  comprises a rectangular box-like housing  52  having upright sidewalls  54  and a top wall  56 . A liquid crystal display (LCD) or plasma panel  60  is centrally positioned on the top wall  56  and has a display surface that defines an input area  62 . Alternatively, a projector-based table could be used. Imaging devices  70   a  to  70   f  are mounted on the LCD panel  60  about the input area  62  and look generally across the input area from different vantages. An illuminated bezel  72  surrounds the periphery of the input area  62  and overlies the imaging devices  70   a  to  70   f . The illuminated bezel  72  provides backlight illumination into the input area  62 . A cover  74  overlies the illuminated bezel  72 . 
         [0076]    In this embodiment, each of the imaging devices  70   a  to  70   f  is in the form of a digital camera device that has a field of view of approximately 90 degrees. The imaging devices  70   a  to  70   d  are positioned adjacent the four corners of the input area  62  and look generally across the entire input area  62 . Two laterally spaced imaging devices  70   e  and  70   f  are positioned along one major side of the input area  62  intermediate the imaging devices  70   a  and  70   b . The imaging devices  70   e  and  70   f  are angled in opposite directions and look towards the center of the input area  62  so that each imaging device  70   e  and  70   f  looks generally across two-thirds of the input area  62 . This arrangement of imaging devices divides the input area  62  into three input regions, namely a left input region  62   a , a central input region  62   b  and a right input region  62   c  as shown in  FIGS. 5 and 7 . The left input region  62   a  is within the fields of view of five (5) imaging devices, namely imaging devices  70   a ,  70   b ,  70   c ,  70   d  and  70   f . The right input region  62   c  is also within the fields of view of five (5) imaging devices, namely imaging devices  70   a ,  70   b ,  70   c ,  70   d  and  70   e . The central input region  62   b  is within the fields of view of all six (6) imaging devices  70   a  to  70   f.    
         [0077]      FIG. 8  is a schematic block diagram of one of the imaging devices. As can be seen, the imaging device includes a two-dimensional CMOS camera image sensor  100  having associated lens assembly and a digital signal processor (DSP)  106 . The parallel peripheral port  107  of the DSP  106  is coupled to the CMOS camera image sensor  100  parallel port by a data bus  108 . Serial control bus  110  carries configuration information between the DSP  106  and the CMOS camera image sensor  100 . A boot EPROM  112  and a power supply subsystem  114  are also included. Alternatively, CMOS line scan sensors could be used. 
         [0078]    The CMOS camera image sensor  100  in this embodiment is an Aptina MT9V022 image sensor configured for a 30×752 pixel sub-array that can be operated to capture image frames at high frame rates including those in excess of 960 frames per second. The DSP  106  is manufactured by Analog Devices under part number ADSP-BF524. 
         [0079]    The DSP  106  provides control information to the image sensor  100  via the control bus  110 . The control information allows the DSP  106  to control parameters of the CMOS camera image sensor  100  and lens assembly such as exposure, gain, array configuration, reset and initialization. The DSP  106  also provides clock signals to the CMOS camera image sensor  100  to control the frame rate of the image sensor. The DSP  106  also communicates image information acquired from the image sensor  100  to a master controller  120  via a serial port  116 . 
         [0080]    Each of the imaging devices  70   a  to  70   f  communicates with the master processor  120  which is best shown in  FIG. 9 . Master controller  120  is accommodated by the housing  52  and includes a DSP  122 , a boot EPROM  124 , a serial line driver  126  and a power supply subsystem  128 . The DSP  122  communicates with the DSPs  106  of each of the imaging device  70   a  to  70   f  over a data bus  130  and via a serial port  132 . The DSP  122  also communicates with a processing device  140  accommodated by the housing  52  via a data bus  134 , a serial port  136 , and the serial line driver  126 . In this embodiment, the DSP  122  is also manufactured by Analog Devices under part number ADM222. The serial line driver  138  is manufactured by Analog Devices under part number ADM222. 
         [0081]    The master controller  120  and each imaging device follow a communication protocol that enables bi-directional communications via a common serial cable similar to a universal serial bus (USB). The transmission bandwidth is divided into thirty-two (32) 16-bit channels. Of the thirty-two channels, four (4) channels are assigned to each of the DSPs  106  in the imaging devices  70   a  to  70   f  and to the DSP  122  in the master controller  120 . The remaining channels are unused and may be reserved for further expansion of control and image processing functionality (e.g., use of additional imaging devices). The master controller  120  monitors the channels assigned to the DSPs  106  while the DSP  106  in each of the imaging devices monitors the five (5) channels assigned to the master controller DSP  122 . Communications between the master controller  120  and each of the imaging devices  70   a  to  70   f  are performed as background processes in response to interrupts. 
         [0082]    In this embodiment, the processing device  140  is a general purpose computing device. The computing device comprises for example a processing unit, system memory (volatile and/or non-volatile memory), other removable or non-removable memory (hard drive, RAM, ROM, EEPROM, CD-ROM, DVD, flash memory, etc.), and a system bus coupling various components to the processing unit. The processing unit runs a host software application/operating system and provides display output to the LCD panel  60 . During execution of the host software application/operating system, a graphical user interface is presented on the display surface of the LCD panel  60  allowing one or more users to interact with the graphical user interface via touch input on the display area  62 . 
         [0083]    The illuminated bezel  72  comprises four bezel segments  200   a  to  200   d  with each bezel segment extending substantially the entire length along one side of the input area  62 .  FIGS. 10   a  to  10   c  better illustrate the bezel segment  200   a . As can be seen, the bezel segment is formed of a homogeneous piece of clear, light transmissive material such as for example Lexan®, Plexiglas, acrylic or other suitable material. The bezel segment  200   a  comprises a front surface  212  that extends substantially the entire length along one major side of the input area  62 , a back surface  214 , two side surfaces  216 , a top surface  218  and a bottom surface  220 . The front, back and side surfaces of the bezel segment  200   a  are generally normal to the plane of the input area  62 . Each side surface  216  has a pair of laterally spaced bores formed therein that accommodate light sources. In this particular embodiment, the light sources are infrared (IR) light emitting diodes (LEDs)  222  although LEDs that emit light at different wavelengths may be used. The top, bottom, side and back surfaces of the bezel segment  200   a  are coated with a reflective material to reduce the amount of light that leaks from the bezel segment via these surfaces. The front surface  212  of the bezel segment  200   a  is textured or covered with a diffusive material to produce a diffusive surface that allows light to escape from the bezel segment into the input area  62 . 
         [0084]    The geometry of the bezel segment  200   a  is such that the reflective back surface  214  is v-shaped with the bezel segment being most narrow at its midpoint. As a result, the reflective back surface  214  defines a pair of angled reflective surface panels  214   a  and  214   b  with the ends of the panels that are positioned adjacent the center of the bezel segment  200   a  being closer to the front surface  212  than the opposite ends of the surface panels. Optionally, the front surface  212  can be a diffusive material (using film, surface modifications, lenses, paint, paper, or other type of diffusive element known to those of skill in the art), in order to scatter the light more evenly. This bezel segment configuration compensates for the attenuation of light emitted by the IR LEDs  222  that propagates through the body of the bezel segment  200   a  by tapering towards the midpoint of the bezel segment  200   a . The luminous emittance of the bezel segment  200   a  is maintained generally at a constant across the surface  212  by reducing the volume of the bezel segment  200   a  further away from the IR LEDs  222  where the attenuation has diminished the light flux. By maintaining the luminous emittance generally constant across the bezel, the amount of light leaking out the side  212  is a generally uniform density. This helps to make the bezel segment  200   a  illumination appear uniform to the imaging devices  70   a  to  70   f.    
         [0085]    Shallow notches  224  are provided in the bottom surface  220  of the bezel segment  200   a  to accommodate the imaging devices  70   a ,  70   e ,  70   f  and  70   b . In this manner, the imaging devices are kept low relative to the front surface  212  so that the imaging devices block as little of the light escaping the bezel segment  200   a  via the diffusive front surface  212  as possible while still being able to view across the input surface, and thus, the height of the bezel segment can be reduced. An alternative embodiment may have slots within the bezel segments to accommodate the imaging devices. 
         [0086]      FIG. 11   b  and  11   c  show alternative dimple patterns provided on the front surface  212  of the bezel segment with the density of the dimples  226 ′ and  226 ″ increasing towards the center of the bezel segment to allow more light to escape the center of the bezel segment as compared to the ends of the bezel segment. This type of pattern could be used with the v-shaped bezel or not.  FIG. 12  shows an alternative textured front surface  212 ′ configured to allow more light to escape the center of the bezel segment as compared to the ends of the bezel segment. As can be seen, in this embodiment spaced vertical grooves or slits  228  are formed in the front surface  212 ′ with the density of the grooves or slits  228  increasing towards the center of the bezel segment. Although not shown, rather than texturing the front surface of the bezel segment, a film may be placed on the front surface that is configured to allow more light to escape the center of the bezel segment as compared to the ends of the bezel segment. 
         [0087]    The bezel segment  200   c  extending along the opposite major side of the input area  62  has a similar configuration to that described above with the exception that the number and positioning of the notches  224  (or alternatively slots) is varied to accommodate the imaging devices  70   c  and  70   d  that are covered by the bezel segment. The bezel segments  200   b  and  200   d  extending along the shorter sides of the input area  62  also have a similar configuration to that described above with the exceptions that the side surfaces of the bezel segments only accommodate a single IR LED  222  (as the lighting requirements are reduced due to the decreased length) and the number and the positioning of the notches  224  is varied to accommodate the imaging devices that are covered by the bezel segments. 
         [0088]    During general operation of the interactive input system  50 , the IR LEDs  222  of the bezel segments  200   a  to  200   d  are illuminated resulting in infrared backlighting escaping from the bezel segments via the front surfaces into the input area  62 . As mentioned above, the design of the bezel segments  200   a  to  200   d  is configured such that the illumination escaping each bezel segment is generally even along the length of the bezel segment. Each imaging device which looks across the input area  62  is conditioned by its associated DSP  106  to acquire image frames. When no pointer or target is in the field of view of an imaging device, the imaging device sees the infrared backlighting emitted by the bezel segments and thus, generates a “white” image frame. When a pointer occludes infrared backlighting emitted by at least one of the bezel segments, the target appears in the image frame as a “dark” region on a “white” background. For each imaging device, image data acquired by its image sensor  100  is processed by the DSP  106  to determine if one or more targets (e.g. pointers) is/are believed to exist in each captured image frame. When one or more targets is/are determined to exist in a captured image frame, pointer characteristic data is derived from that captured image frame identifying the target position(s) in the captured image frame. 
         [0089]    The pointer characteristic data derived by each imaging device is then conveyed to the master controller  120 . The DSP  122  of the master controller in turn processes the pointer characteristic data to allow the location(s) of the target(s) in (x,y) coordinates relative to the input area  62  to be calculated. 
         [0090]    The calculated target coordinate data is then reported to the processing device  140 , which in turn records the target coordinate data as writing or drawing if the target contact(s) is/are write events or injects the target coordinate data into the active application program being run by the processing device  140  if the target contact(s) is/are mouse events. As mentioned above, the processing device  140  also updates the image data conveyed to the LCD panel  60  so that the image presented on the display surface of the LCD panel  60  reflects the pointer activity. 
         [0091]    When a single pointer exists in the image frames captured by the imaging devices  70 , the location of the pointer in (x,y) coordinates relative to the input area  62  can be readily computed using triangulation. When multiple pointers exist in the image frames captured by the imaging devices  70 , computing the positions of the pointers in (x,y) coordinates relative to the input area  62  is more challenging as a result of the pointer ambiguity and occlusion issues. 
         [0092]    Pointer ambiguity arises when multiple targets are in contact with the input area  62  at different locations and are within the fields of view of multiple imaging devices. If the targets do not have distinctive markings to allow them to be differentiated, the observations of the targets in each image frame produce real and false target results that cannot be readily differentiated. 
         [0093]    Pointer occlusion arises when a target in the field of view of an imaging device occludes another target in the field of view of the imaging device, resulting in observation merges as will be described. 
         [0094]    Depending on the position of an imaging device relative to the input area  62  and the position of a target within the field of view of the imaging device, an imaging device may or may not see a target brought into its field of view adequately to enable image frames acquired by the imaging device to be used to determine the position of the target relative to the input area  62 . Accordingly, for each imaging device, an active zone with the field of view of the imaging device is defined. The active zone is an area that extends to a distance of radius away from the imaging device. This distance is pre-defined and based on how well a camera can measure an object at a certain distance. When a target appears in the active zone of the imaging device, image frames acquired by the imaging device are deemed to observe the target sufficiently such that the observation of the target within the image frame captured by the imaging device is processed. When a target is within the field of view of an imaging device but is beyond the active zone of the imaging device, the observation of the target is ignored. When a target is within the radius ‘r’ but outside the field of view of the imaging device, it will not be seen and that imaging device is not used in further to process the position of the target. 
         [0095]    When a target appears in the field of view of an imaging device, the observation of the target defined by the area formed between two straight lines is created by the master processor, namely one line that extends from the imaging device to the bezel segment and crosses the left edge of the target, and another line that extends from the imaging device to the bezel segment and crosses the right edge of the target. When two or more imaging devices observe the same target and the observations created by the imaging devices overlap, the overlapping region of the observations is referred to as a candidate. The lines defining the perimeter of a candidate are referred to as a bounding box. If another observation partially overlaps a candidate, the bounding box is updated to include only the area where all observations overlap. At least two observations are needed to create a bounding box. 
         [0096]    When a target is in an input region of the input area  62  and all imaging devices whose fields of view encompass the input region and whose active zones include at least part of the target, create observations that overlap, the resulting candidate is deemed to be a consistent candidate. The consistent candidate may represent a real target or a phantom target. 
         [0097]    The master processor  120  executes a candidate generation procedure to determine if any consistent candidates exist in captured image frames.  FIG. 13  illustrates steps performed during the candidate generation procedure. During the candidate generation procedure, a table is initially generated, or “built”, that lists all imaging device observations so that the observations generated by each imaging device can be cross referenced with all other observations to see if one or more observations result in a candidate (step  300 ). 
         [0098]    As the interactive input system  50  includes six (6) imaging devices  70   a  to  70   f  and is capable of simultaneously tracking eight (8) targets, the maximum number of candidates that is possible is equal to nine-hundred and sixty (960). For ease of illustration,  FIG. 14  shows an exemplary table identifying three imaging devices with each imaging device generating three observations. Cells of the table with an X indicate observations that are not cross-referenced with each other. For example, imaging device observations cannot be cross-referenced with any of their own observations. Cells of the table that are redundant are also not cross-referenced. In  FIG. 14 , cells of the table designated with a “T” are processed. In this example of three imaging devices and three targets, the maximum number of candidates to examine is twenty-seven (27). Once the table has been created at step  300 , the table is examined from left to right and starting on the top row and moving downwards to determine if the table includes a candidate (step  302 ). If the table is determined to be empty (step  304 ), the table therefore does not include any candidates, and the candidate generation procedure ends (step  306 ). 
         [0099]    At step  304 , if the table is not empty and a candidate is located, a flag is set in the table for the candidate and the lines that make up the bounding box for the candidate resulting from the two imaging device observations are defined (step  308 ). A check is then made to determine if the position of the candidate is completely off the input area  62  (step  310 ). If the candidate is determined to be completely clear of the input area  62 , the flag that was set in the table for the candidate is cleared (step  312 ) and the procedure reverts back to step  302  to determine if the table includes another candidate. 
         [0100]    At step  310 , if the candidate is determined to be partially or completely on the input area  62 , a list of the imaging devices that have active zones encompassing at least part of the candidate is created excluding the imaging devices whose observations were used to create the bounding box at step  308  (step  314 ). Once the list of imaging devices has been created, the first imaging device in the list is selected (step  316 ). For the selected imaging device, each observation created for that imaging device is examined to see if it intersects with the bounding box created at step  308  (steps  318  and  320 ). If no observation intersects the bounding box, the candidate generation procedure reverts back to step  312  and the flag that was set in the table for the candidate is cleared. At step  320 , if an observation that intersects the bounding box is located, the bounding box is updated using the lines that make up the observation (step  322 ). A check is then made to determine if another non-selected imaging device exists in the list (step  324 ). If so, the candidate generation procedure reverts back to step  316  and the next imaging device in the list is selected. 
         [0101]    At step  324 , if all of the imaging devices have been selected, the candidate is deemed to be a consistent candidate and is added to a consistent candidate list (step  326 ). Once the candidate has been added to the consistent candidate list, the combinations of observations that relate to the consistent candidate are removed from the table (step  328 ). Following this, the candidate generation procedure reverts back to step  302  to determine if another candidate exists in the table. As it will be appreciated, the candidate generation procedure generates a list of consistent candidates representing targets that are seen by all of the imaging devices whose fields of view encompass the target locations. For example, a consistent candidate resulting from a target in the central input region  62   b  is seen by all six imaging devices  70   a  to  70   f  whereas a consistent candidate resulting from a target in the left or right input region  62   a  or  62   c  is only seen by five imaging devices. 
         [0102]    The master processor  120  also executes an association procedure as best shown in  FIG. 15  to associate candidates with existing targets. During the association procedure, a table is created that contains the coordinates of the predicted target locations generated by the tracking procedure as will be described, and the location of the consistent candidates in the consistent candidate list created during the candidate generation procedure (step  400 ). A check is then made to determine if all of the consistent candidates have been examined (step  402 ). If it is determined that all of the consistent candidates have been examined, any predicted targets that are not associated with a consistent candidate are deemed to be associated with a dead path. As a result, the predicted target location and previous tracks associated with these predicted targets are deleted (step  404 ) and the association procedure is terminated (step  406 ). 
         [0103]    At step  402 , if it is determined that one or more of the consistent candidates have not been examined, the next unexamined consistent candidate in the list is selected and the distance between the consistent candidate and all of the predicted target locations is calculated (step  408 ). A check is then made to determine whether the distance between the consistent candidate and a predicted target location falls within a threshold (step  410 ). If the distance falls within the threshold, the consistent candidate is associated with the predicted target (step  412 ). Alternatively, if the distance is beyond the threshold, the consistent candidate is labeled as a new target (step  414 ). Following either of steps  412  and  414 , the association procedure reverts back to step  402  to determine if all of the consistent candidates in the consistent candidate list have been selected. As a result, the association procedure identifies each candidate as either a new target contacting the input area  62  or an existing target. 
         [0104]      FIG. 16  shows an example of the interactive input system  50  tracking three objects; A, B and C. The locations of four previously triangulated targets for objects A, B and C are represented by an X. From these previously tracked target locations, an estimate (e.g. predicted target location) is made for where the location of the object should appear in the current frame, and is represented by a +. Since a user can manipulate an object on the input area  62  at an approximate maximum velocity of 4 m/s, and if the interactive input system  50  is running at 100 frames per second, then the actual location of the object should appear within [400 cm/s/100 frames/s×1 frame=4 cm] 4 centimeters of the predicted target location. This threshold is represented by a broken circle surrounding the predicted target location. Objects B and C are both located within the threshold of their predicted target locations and are thus associated with those respective previously tracked target location. The threshold around the predicted target location of object A does not contain object A, and is therefore considered a dead track and no longer used in subsequent image processing. Object D is seen at a position outside all of the calculated thresholds and is thus considered a new object and will continue to be tracked in subsequent frames. 
         [0105]    The master processor  120  executes a state estimation procedure to determine the status of each candidate, namely whether each candidate is clear, merged or irrelevant. If a candidate is determined to be merged, a disentanglement process is initiated. During the disentanglement process, the state metrics of the targets are computed to determine the positions of partially and completely occluded targets. Initially, during the state estimation procedure, the consistent candidate list generated by the candidate generation procedure, the candidates that have been associated with existing targets by the association procedure, and the observation table are analyzed to determine whether each imaging device had a clear view of each candidate in its field of view or whether a merged view of candidates within its field of view existed. Candidates that are outside of the active areas of the imaging devices are deemed not relevant and flagged as being irrelevant. 
         [0106]    The target and phantom track identifications from the previous frames are used as a reference to identify true target merges. When a target merge for an imaging device is deemed to exist, the disentanglement process for that imaging device is initiated. The disentanglement process makes use of the Viterbi algorithm. Depending on the number of true merges, the Viterbi algorithm assumes a certain state distinguishing between a merge of only two targets and a merge of more than two targets. In this particular embodiment, the disentanglement process is able to occupy one of the three states shown in  FIG. 41 , which depicts a four-input situation. 
         [0107]    A Viterbi state transition method computes a metric for each of the three states. In this embodiment, the metrics are computed over five (5) image frames including the current image frame and the best estimate on the current state is given to the branch with the lowest level. The metrics are based on the combination of one dimensional predicted positions and widths with one dimensional merged observations. The state with the lowest branch is selected and is used to associate targets within a merge thereby to enable the predictions to disentangle merge observations. 
         [0108]    For states  1  and  2 , the disentanglement process yields the left and right edges for the merged targets. Only the center position for all the merges in state  3  is reported by the disentanglement process. 
         [0109]    Once the disentanglement process has been completed, the states flag indicating a merge is cleared and a copy of the merged status before being cleared is maintained. To reduce triangulation inaccuracies due to disentanglement observations, a weighting scheme is used on the disentangled targets. Targets associated with clean observations are assigned a weighting of one (1). Targets associated with merged observations are assigned a weighting in the range from 0.5 to 0.1 depending on how far apart the state metrics are from each other. The greater the distance between state metrics, the higher the confidence of disentangling observations and hence the higher the weighting selected from the above range. 
         [0110]      FIG. 17  shows an example of two objects, A and B, contacting the input area  62  and being viewed by imaging devices  70   a  to  70   f . Imaging devices  70   a ,  70   e  and  70   c  all have two observations, one of object A and the other of object B. Imaging devices  70   f ,  70   b , and  70   d  all have one observation. Since at least one imaging device shows two observations, the state estimation module determines that there must be two objects contacting the input area  62 . Imaging devices  70   a ,  70   e  and  70   c  each see objects A and B clearly and so each observation is assigned a weight of 1.0. Imaging devices  70   f ,  70   b  and  70   d  observe only one object, determine that these two objects must appear merged, and assign a weight of 0.5 to each observation. 
         [0111]      FIG. 18  shows objects A and B as viewed by imaging devices  70   f  and  70   b . Since these objects appear merged to these imaging devices, the state estimation procedure approximates the actual position of the objects based on earlier data. From previous tracking information, the approximate widths of the objects are known. Since the imaging devices  70   f  and  70   b  are still able to view one edge of each of the objects, the other edge is determined based on the previously stored width of the object. The state estimation module calculates the edges of both objects for both imaging devices  70   f  and  70   b . Once both edges of each object are known, the center line for each object from each imaging device is calculated. 
         [0112]    As mentioned previously, the master processor  120  also executes a tracking procedure to track existing targets. During the tracking procedure, each target seen by each imaging device is examined to determine its center point and a set of radii. The set of radii comprises a radius corresponding to each imaging device that sees the target represented by a line extending from the optical axis of the imaging device to the center point. If a target is associated with an object, a Kalman filter is used to estimate the current state of the target and to predict its next state. This information is then used to backwardly triangulate the location of the target at the next time step which approximates an observation of the target if the target observation overlaps another target observation seen by the imaging device. If the target is not associated with a candidate, the target is considered dead and the target tracks are deleted from the track list. If the target is not associated with a candidate, and the number of targets is less than the maximum number of permitted targets, in this case eight (8), the candidate is considered to be a new target. 
         [0113]      FIG. 19  shows an input situation, similar to that of  FIGS. 16 to 18 . The centerline for each imaging device observation of each target is shown along with the corresponding assigned weight. Note that the centerlines of targets A and C as seen from imaging device  70   a  can be determined, along with the centerline of targets B and C as seen from imaging device  70   f . The centerline of targets A, B and C as seen from imaging device  70   b  could not be determined and as a result, the center of the merged observation is used for the centerline. The value of the weight assigned to these observations is very low. 
         [0114]      FIG. 20  shows the triangulated location of target A from the centerlines of the observations from imaging devices  70   a ,  70   f  and  70   b . Since imaging device  70   f  has a clear view of the target A and has an observation with a high weight, the observation of imaging device  70   a  has a medium weight, and the observation of imaging device  70   b  has a low weight; the triangulated location is located closer to the intersection of the two lines with the higher weight since those observations are more reliable. 
         [0115]    Similar to  FIG. 20 ,  FIG. 21  shows the centerline and triangulated position for target B. The triangulation is dominated by the highly weighted observations from imaging devices  70   a  and  70   e.    
         [0116]      FIG. 22  shows the centerline and triangulated position for target C. It is clearly shown that the triangulated position was insignificantly influenced by the low weighted observation of imaging device  70   b.    
         [0117]      FIG. 23  shows an example of when a low weighted observation becomes important for an input. In this scenario the target is located almost directly between imaging devices  70   a  and  70   c , which both have a clear view of the target and corresponding highly weighted observations. Imaging device  70   b  has a low weighted observation due to an ambiguity such as that situation presented in  FIG. 19 . The triangulation result from two imaging devices, in this case  70   a  and  70   c , triangulating a point directly or nearly directly between the two imaging devices is unreliable. In this case where one observation is lowly weighted, the observation is important because it provides an additional view of the target needed for triangulation. Even though the observation is low weighted, it is still better than no other observation at all. 
         [0118]      FIG. 24  depicts a similar scenario to that of  FIG. 19  but has two imaging devices with low weighted observations (imaging devices  70   b  and  70   d ) and one imaging device with a high weighted observation (imaging device  70   c ). The observations from imaging device  70   b  and  70   d  are averaged to result in a triangulated point between the two observations and along the observation from imaging device  70   c . In this case the triangulated location uses both low weighted observations to better locate the target. 
         [0119]      FIG. 25  shows the steps performed during triangulation of real and phantom targets captured on the input area. During triangulation, the number N of imaging devices being used to triangulate a target (X,Y) coordinate, a vector x of length N containing image x-positions from each imaging device, a 2N x 3 matrix Q containing the projection matrices P for each imaging device, where Q=[P 1 |P 2 | . . . |P N ], and a vector w of length N containing the weights assigned to each image observation position in x are used (step  500 ). If the weight vectors w are not specified, the weight vectors are set to a value of one (1). A binary flag for each parallel line of sight is then set to zero (0) (step  502 ). A tolerance for the parallel lines of sight is set to 2ε, where ε is the difference between 1 and the smallest exactly representable number greater than one. This tolerance gives an upper bound on the relative error due to rounding of floating point numbers and is hardware dependent. A least-squares design matrix A(N×2) and right-hand side vector b are constructed by looping over the N available imaging device views (step  504 ). During this process, a 2×3 camera matrix P is extracted for the current image frame. A row is added to the design matrix containing [P 11 −x·P 21 , P 12 −x·P 22 ]. An element is added to b containing [x·P 23 −P 10 ]. An N×N diagonal matrix W containing the weights w is then created. The determinant (typically constructed using the method outlined in http://mathwold.wolfram.com/determinant.html) of the weighted normal equations is computed and a check is made to determine whether or not it is less than the tolerance for parallelism according to det (W·A) T ·(W·A))≦2·ε (step  506 ). This test determines whether A has linearly dependent rows. If the determinant is less than the tolerance, the parallelism flag is set to one (1) and X and Y are set to empty matrices (step  508 ). Otherwise, the linear least-squares problem for X and Y are solved according to (W A) T (W A)X=(W A) T b (step  510 ) where X=[X,Y] T  and b is also a two-element vector. The errors σ x  and σ y X and Y are computed from the square roots of the diagonal elements Cii of the covariance matrix C defined by C=σ 2 ·((W·A) T ·(W·A)) −1 , where σ 1  is the RMS error of the fit (i.e. the square root of chi-squared). 
         [0120]    If N=2, no errors are computed as the problem is exactly determined. A check is then made to determine if the triangulated point is behind any of the imaging devices (step  512 ). Using the triangulated position, the expected target position for each imaging device is computed according to x cal =P·X, where x contains the image position x and the depth A. The second element of x cal  is the depth A from the imaging device to the triangulated point. If λ=0, the depth test flag is set to one (1) and zero (0) otherwise. If all components of x cal  are negative, the double negative case is ignored. The computed X,Y, error values and test flags are then returned (step  514 ). 
         [0121]    In the embodiment shown and described above, the interactive input system comprises six (6) imaging devices arranged about the input area  62  with four (4) imaging devices being positioned adjacent the corners of the input area and two imaging devices  70   e  and  70   f  positioned at spaced locations along the same side of the input area. Those of skill in the art will appreciate that the configuration and/or number of imaging devices employed in the interactive input system may vary to suit the particular environment in which the interactive input system is to be employed. For example, the imaging devices  70   e  and  70   f  do not need to be positioned along the same side of the input area. Rather, as shown in  FIG. 26 , imaging device  70   e  can be positioned along one side of the input area and imaging device  70   f  can be positioned along the opposite side of the input area. 
         [0122]    Turning now to  FIG. 27 , an alternative imaging device configuration for the interactive input system is shown. In this configuration, the interactive input system employs four (4) imaging devices  70   a ,  70   e ,  70   f , and  70   b  arranged along one side of the input area  62 . Imaging devices  70   a ,  70   b  are positioned adjacent opposite corners of the input area  62  and look generally across the entire input area  62 . The intermediate imaging devices  70   e ,  70   f  are angled in opposite directions towards the center of the input area  62  so that the imaging devices  70   a ,  70   e ,  70   f  and  70   b  look generally across the two-thirds of input area  62 . This arrangement of imaging devices divides the input area  62  into three input regions, namely a left input region  62   a , a central input region  62   b  and a right input region  62   c  as shown. The left input region  62   a  is within the fields of view of three imaging devices, namely imaging devices  70   a ,  70   e , and  70   b . The right input region  62   c  is also within the fields of view of three imaging devices, namely imaging devices  70   a ,  70   f , and  70   b . The central input region  62   b  is within the fields of view of all four imaging devices  70   a ,  70   e ,  70   f  and  70   b.    
         [0123]      FIG. 28  shows another alternative imaging device configuration for the interactive input system. In this configuration, the interactive input system employs four (4) imaging devices  70   a ,  70   b ,  70   c ,  70   d  with each imaging device being positioned adjacent a different corner of the input area  62  and looking generally across the entire input area  62 . With this imaging device arrangement, the entire input area  62  is within the fields of view of all four imaging devices. 
         [0124]      FIG. 29  shows yet another alternative imaging device configuration for the interactive input system. In this configuration, the interactive input system employs three (3) imaging devices  70   a ,  70   b ,  70   c  with each imaging device being positioned adjacent a different corner of the input area  62  and looking generally across the entire input area  62 . With this imaging device arrangement, the entire input area is within the fields of view of all three imaging devices. 
         [0125]    In  FIG. 30 , yet another alternative imaging device configuration for the interactive input system is shown. In this configuration, the interactive input system employs eight (8) imaging devices, with four imaging devices  70   a ,  70   e ,  70   f ,  70   b  being arranged along one major side of the input area  62  and with four imaging devices  70   d ,  70   g ,  70   h ,  70   c  being arranged along the opposite major side of the input area  62 . Imaging devices  70   a ,  70   b ,  70   c ,  70   d  are positioned adjacent the corners of the input area and look generally across the entire input area. The intermediate imaging devices  70   e ,  70   f ,  70   g ,  70   h  along each major side of the input area are angled in opposite directions towards the center of the input area  62 . This arrangement of imaging devices divides the input area into 3 input regions. The number in each input region identifies the number of imaging devices whose fields of view see the input region. 
         [0126]      FIG. 31  shows yet another alternative imaging device configuration for the interactive input system. In this configuration, the interactive input system employs eight (8) imaging devices  70 . Imaging devices  70   a ,  70   b ,  70   c ,  70   d  are positioned adjacent the corners of the input area  62  and look generally across the entire input area. Intermediate imaging devices  70   f ,  70   g  are positioned on opposite major sides of the input area and are angled in opposite directions towards the center of the input area  62 . Intermediate imaging devices  70   i ,  70   j  are positioned on opposite minor sides of the input area  62  and are angled in opposite directions towards the center of the input area  62 . This arrangement of imaging devices divides the input area into nine (9) input regions as shown. The number in each input region identifies the number of imaging devices whose fields of view see the input region. 
         [0127]    In  FIG. 32 , yet another alternative imaging device configuration for the interactive input system is shown. In this configuration, the interactive input system employs twelve (12) imaging devices. Imaging devices  70   a ,  70   b ,  70   c ,  70   d  are positioned adjacent the corners of the input area  62  and look generally across the entire input area  62 . Pairs of intermediate imaging devices  70   e  and  70   f ,  70   g  and  70   h ,  70   i  and  70   k ,  70   j  and  70   l  are positioned along each side of the input area and are angled in opposite directions towards the center of the input area  62 . This arrangement of imaging devices divides the input area into nine (9) input regions as shown. The number in each input region identifies the number of imaging devices whose fields of view see the input region. 
         [0128]      FIG. 33  shows yet another alternative imaging device configuration for the interactive input system. In this configuration, the interactive input system employs sixteen (16) imaging devices  70 . Imaging devices  70   a ,  70   b ,  70   c ,  70   d  are positioned adjacent the corners of the input area and look generally across the entire input area. Pairs of intermediate imaging devices  70   e  and  70   f ,  70   g  and  70   h ,  70   i  and  70   k ,  70   j  and  70   l  are positioned along each side of the input area and are angled in opposite directions towards the center of the input area  62 . Four midpoint imaging devices  70   m ,  70   n ,  70   o ,  70   p  are positioned at the midpoint of each side of the input area  62  and generally look across the center of the input area  62 . This arrangement of imaging devices  70  divides the input area  62  into twenty-seven (27) input regions as shown. The number in each input region identifies the number of imaging devices whose fields of view see the input region. 
         [0129]      FIG. 34  shows yet another alternative imaging device configuration for the interactive input system. In this configuration, the interactive input system employs twenty (20) imaging devices  70 . Imaging devices  70   a ,  70   b ,  70   c ,  70   d  are positioned adjacent the corners of the input area and look generally across the entire input area. Pairs of intermediate imaging devices  70   e  and  70   f ,  70   g  and  70   h ,  70   i  and  70   k ,  70   j  and  70   l  are positioned along each side of the input area and are angled in opposite directions towards the center of the input area  62 . Two further intermediate imaging devices  70   q ,  70   r ,  70   s ,  70   t  are positioned along each major side of the input area  62  and are angled in opposite directions towards the center of the input area  62 . Four midpoint imaging devices  70   m ,  70   n ,  70   o ,  70   p  are positioned at the midpoint of each side of the input area  62  and generally look across the center of the input area  62 . This arrangement of imaging devices divides the input area into thirty-seven (37) input regions as shown. The number in each input region identifies the number of imaging devices whose fields of view see the input region. 
         [0130]    Although particular embodiments of the bezel segments have been described above, those of skill in the art will appreciate that many alternatives are available. For example, more or fewer IR LEDs may be provided in one or more of the bezel surfaces. For example,  FIG. 35  shows an embodiment of the bezel segment generally identified by numeral  600  one surface accommodating a pair of IR LEDs  222   a ,  222   b  and the opposite surface accommodating a single IR LED  222   c . If desired, rather than providing notches in the undersurface of the bezel segments, recesses  602  may be provided in the body of the bezel segments to accommodate the imaging devices as shown in  FIG. 36 . Of course a combination of notches and recesses may be employed. 
         [0131]    In the above embodiments, each bezel segment has a planar front surface and a v-shaped back reflective surface. If desired, the configuration of one or more of the bezel segments can be reversed as shown in  FIG. 37  so that the bezel segment  700  comprises a planar reflective back surface  204  and a v-shaped front surface  702 . Optionally, the v-shaped front surface could be diffusive. Alternatively, the v-shaped back surface could be diffusive and the planar front surface could be transparent. In a further alternative embodiment bezel  800 , instead of using a v-shaped back reflective surface, a parabolic shaped back reflective surface  802  may be used as shown in  FIG. 40  or other similarly shaped back reflective surfaces may be used.  FIG. 38  shows the interactive input system employing an illuminated bezel formed of a combination of bezel segments. In particular, bezel segment  700  is of the type shown in  FIG. 37  while bezel segments  200   b  to  200   d  are of the type shown in  FIGS. 1 to 6 . If desired, for bezel segment, supplementary IR LEDs  222   a ,  222   b  may be accommodated by bores formed in the planar reflective back surface as shown in  FIG. 39 . In this case, the supplementary IR LEDs  222   a ,  222   b  are angled towards the center of the bezel segment. 
         [0132]    In a still further embodiment, the textured front surface  212  has a dimple pattern arranged such that the density of dimples  226  increases towards the center of the bezel segment  200   a  as shown in  FIG. 11   a . This dimple pattern allows more light to escape the center of the bezel segment as compared to the ends of the bezel segment. As will be appreciated, if such a dimple pattern was not provided on the diffusive front surface  212 , the majority of light emitted by the IR LEDs  222  would exit the bezel segment adjacent the ends of the diffusive front surface  212  closest to the IR LEDs  222  and as a result, the bezel segment illumination would not appear uniform to the imaging devices  70   a  to  70   f.    
         [0133]    Although embodiments of the interactive input system have been shown and described above, those of skill in the art will appreciate that further variations and modifications may be made without departing from the spirit and scope thereof as defined by the appended claims.