Interactive input system and tool tray therefor

A tool tray for an interactive input system comprises a housing having an upper surface defining at least one receptacle sized to receive a tool for interacting with an interactive surface, a sensor configured to detect the presence of the tool within the receptacle and at least one display in communication with the sensor. The display is configured to present a pointer attribute selection screen upon removal of the tool from the receptacle.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application relates to U.S. Provisional Application No. 61/294,831 to Bolt et al. filed on Jan. 13, 2010, entitled “INTERACTIVE INPUT SYSTEM AND TOOL TRAY THEREFOR”, and also relates to U.S. patent application Ser. No. 12/709,424 to Bolt et al. filed on Feb. 19, 2010, entitled “INTERACTIVE INPUT SYSTEM AND TOOL TRAY THEREFOR”, the contents of which are incorporated herein by reference in their entirety.

FIELD OF THE INVENTION

The present invention relates generally to interactive input systems, and in particular, to an interactive input system and a tool tray therefor.

BACKGROUND OF THE INVENTION

Interactive input systems that allow users to inject input (e.g. digital ink, mouse events, etc.) into an application program using an active pointer (e.g. a pointer that emits light, sound or other signal), a passive pointer (e.g. a finger, cylinder or other object) or other suitable input device such as for example, a mouse or trackball, are well known. These interactive input systems include but are not limited to: touch systems comprising touch panels employing analog resistive or machine vision technology to register pointer input such as those disclosed in U.S. Pat. Nos. 5,448,263; 6,141,000; 6,337,681; 6,747,636; 6,803,906; 7,232,986; 7,236,162; 7,274,356; and 7,532,206, all assigned to SMART Technologies ULC of Calgary, Alberta, Canada, assignee of the subject application, the contents of which are incorporated herein by reference in their entirety; touch systems comprising touch panels employing electromagnetic, capacitive, acoustic or other technologies to register pointer input; tablet personal computers (PCs); laptop PCs; personal digital assistants (PDAs); and other handheld devices and other similar devices.

Above-incorporated U.S. Pat. No. 6,803,906 to Morrison et al. discloses a touch system that employs machine vision to detect pointer interaction with a touch surface on which a computer-generated image is presented. A rectangular bezel or frame surrounds the touch surface and supports digital imaging devices at its corners. The digital imaging devices have overlapping fields of view that encompass and look generally across the touch surface. The digital imaging devices acquire images looking across the touch surface from different vantages and generate image data. Image data acquired by the digital imaging devices is processed by on-board digital signal processors to determine if a pointer exists in the captured image data. When it is determined that a pointer exists in the captured image data, the digital signal processors convey pointer characteristic data to a master controller, which in turn processes the pointer characteristic data to determine the location of the pointer in (x,y) coordinates relative to the touch surface using triangulation. The pointer coordinates are conveyed to a computer executing one or more application programs. The computer uses the pointer coordinates to update the computer-generated image that is presented on the touch surface. Pointer contacts on the touch surface can therefore be recorded as writing or drawing or used to control execution of application programs executed by the computer.

U.S. Pat. No. 7,532,206 to Morrison et al. discloses a touch system and method that differentiates between passive pointers used to contact a touch surface so that pointer position data generated in response to a pointer contact with the touch surface can be processed in accordance with the type of pointer used to contact the touch surface. The touch system comprises a touch surface to be contacted by a passive pointer and at least one imaging device having a field of view looking generally across the touch surface. At least one processor communicates with the at least one imaging device and analyzes images acquired by the at least one imaging device to determine the type of pointer used to contact the touch surface and the location on the touch surface where pointer contact is made. The determined type of pointer and the location on the touch surface where the pointer contact is made are used by a computer to control execution of an application program executed by the computer.

In order to determine the type of pointer used to contact the touch surface, a curve of growth method is employed to differentiate between different pointers. During this method, a horizontal intensity profile (HIP) is formed by calculating a sum along each row of pixels in each acquired image thereby to produce a one-dimensional profile having a number of points equal to the row dimension of the acquired image. A curve of growth is then generated from the HIP by forming the cumulative sum from the HIP.

Many models of interactive boards sold by SMART Technologies ULC under the name SMART Board™ that employ machine vision technology to register pointer input have a tool tray mounted below the interactive board. The tool tray comprises slots or receptacles for holding pen tools and an eraser tool that can be used to interact with the touch surface of the interaction board. When a pen tool is removed from its receptacle in the tool tray, a sensor in the tool tray detects the removal of that pen tool allowing the interactive board to determine the particular pen tool that has been selected. Subsequently, when a pointer is used to contact the touch surface of the interactive board, software treats the pointer contact with the touch surface as digital ink input in the colour assigned to the selected pen tool, regardless of whether the contact is made using the selected pen tool, or another pointer such as a finger or other object. Similarly, when the eraser tool is removed from its receptacle in the tool tray, the software treats pointer contact with the touch surface as an erasing action, regardless of whether the contact is made using the selected eraser tool, or another pointer such as a finger or other object. Additionally, below the tool tray two buttons are provided. One of the buttons, when pressed, allows the user to use a pointer to execute typical “right click” mouse functions, such as copy, cut, paste, select all, and the like, while the other button, when pressed, displays an onscreen keyboard that allow users to use a pointer to interact with the keyboard to enter text, numbers, and the like. Although such existing tool trays are adequate, improvements are desired.

It is therefore an object of the present invention at least to provide a novel interactive input system and a tool tray therefor.

SUMMARY OF THE INVENTION

Accordingly, in one aspect there is provided a tool tray for an interactive input system comprising a housing having an upper surface defining at least one receptacle sized to receive a tool for interacting with an interactive surface; a sensor configured to detect the presence of the tool within the receptacle; and at least one display in communication with the sensor, the display being configured to present a pointer attribute selection screen upon removal of the tool from the receptacle.

In one embodiment, the pointer attributes selection screen is a pointer colour selection, size selection and/or shape selection screen. The display may be configured to receive touch input. In this case, the pointer attribute selection screen comprises at least one virtual button associated with an attribute of pointer input. Selection of the virtual button assigns a pointer attribute to subsequent pointer interaction with the interactive surface.

The display may be configured to display at least one additional screen. The at least one additional screen may comprise a pointer attribute screen that comprises at least one virtual button selectable for assigning a pointer attribute to subsequent pointer interaction with the interactive surface. The at least one additional screen may be an eraser attribute screen comprising at least one virtual button selectable for assigning an eraser attribute to subsequent pointer interaction with the interactive surface. The at least one additional screen may be an insert object screen comprising at least one virtual button selectable for assigning an object insertion attribute to subsequent pointer interaction with the interactive surface.

In one embodiment, the upper surface of the housing may define a plurality of receptacles sized to receive tools for interacting with the interacting surface. At least one of the receptacles may be configured to receive a pen tool and at least one of the receptacles may be configured to receive an eraser tool. The display may be positioned between receptacles and generally centrally positioned along the housing. In another embodiment, the tool tray may comprise a plurality of displays.

According to another aspect there is provided an interactive input system comprising an interactive surface; and a tool tray adjacent the interactive surface comprising a housing configured to accommodate at least one tool for interacting with the interactive surface; a sensor configured to detect the presence of the tool when accommodated by the housing; and at least one display on the housing, the display being configured to present at least one selectable attribute upon removal of the tool from the housing.

According to yet another aspect there is provided an interactive input system comprising an interactive surface on which an image is presented; and at least one display panel adjacent said interactive surface on which an image is presented, said interactive surface and display panel being individually responsive to input events.

In one embodiment, the image presented on the interactive surface is a computer desktop and the image presented on the display comprises at least one selectable attribute. The at least one selectable attribute may comprise at least one of pointer colour, pointer size and pointer shape. The display may be configured to receive touch input and present at least one virtual button that is selectable to assign a pointer attribute to pointer input made on the interactive surface. The interactive input system may further comprise a tool tray that is mounted on the interactive surface, the tool tray supporting the display panel and accommodating tools for interacting with the interactive surface. Default selectable attributes are presented on the display panel upon removal of a tool from the tool tray.

DETAILED DESCRIPTION OF THE EMBODIMENTS

In the following description, a tool tray for an interactive input system and an interactive input system incorporating the same are discussed. The tool tray has a housing with an upper surface defining at least one receptacle sized to receive a tool for interacting with an interactive surface. A sensor is configured to detect the presence of the tool within the receptacle. At least one display is in communication with the sensor. The display is configured to present a pointer attribute selection screen upon removal of the tool from the receptacle. Further specifics of the tool tray and interactive input system will now be described with particular reference toFIGS. 1 to 13.

Turning now toFIGS. 1 and 2, an interactive input system that allows a user to inject input such as digital ink, mouse events etc. into an application program executed by a computing device is shown and is generally identified by reference numeral20. In this embodiment, interactive input system20comprises an interactive board22mounted on a vertical support surface such as for example, a wall surface or the like. Interactive board22comprises a generally planar, rectangular interactive surface24that is surrounded about its periphery by a bezel26. An ultra-short throw projector (not shown) such as that sold by SMART Technologies ULC under the name SMART UX60 is also mounted on the support surface above the interactive board22and projects an image, such as for example a computer desktop, onto the interactive surface24.

The interactive board22employs machine vision to detect one or more pointers brought into a region of interest in proximity with the interactive surface24. The interactive board22communicates with a general purpose computing device28executing one or more application programs via a universal serial bus (USB) cable30or other suitable wired or wireless connection. General purpose computing device28processes the output of the interactive board22and adjusts image data that is output to the projector, if required, so that the image presented on the interactive surface24reflects pointer activity. In this manner, the interactive board22, general purpose computing device28and projector allow pointer activity proximate to the interactive surface24to be recorded as writing or drawing or used to control execution of one or more application programs executed by the general purpose computing device28.

The bezel26in this embodiment is mechanically fastened to the interactive surface24and comprises four bezel segments40,42,44,46. Bezel segments40and42extend along opposite side edges of the interactive surface24while bezel segments44and46extend along the top and bottom edges of the interactive surface24respectively. In this embodiment, the inwardly facing surface of each bezel segment40,42,44and46comprises a single, longitudinally extending strip or band of retro-reflective material. To take best advantage of the properties of the retro-reflective material, the bezel segments40,42,44and46are oriented so that their inwardly facing surfaces extend in a plane generally normal to the plane of the interactive surface24.

A tool tray48is affixed to the interactive board22adjacent the bezel segment46using suitable fasteners such as for example, screws, clips, adhesive etc. The tool tray48comprises a housing that accommodates a master controller50and that has an upper surface configured to define a plurality of receptacles or slots. In this embodiment, the upper surface of the housing is configured to define two laterally spaced receptacles48aand48bthat are sized to receive pen tools P which can be used to interact with the interactive surface24.

A main power button52is provided adjacent one end of the housing and is user actuable to control the on/off status of the interactive input system20, together with any accessories connected the interactive input system20, such as, for example, the projector (not shown). As can be seen, the power button52is generally positioned at an intuitive, easy-to-find location and therefore allows a user to switch the interactive input system20on and off in a facile manner. Control buttons54are provided adjacent the opposite end of the housing and are user actuable to enable a user to execute additional functions using the interactive input system20. In this embodiment, these additional functions include launching an on-screen keyboard and initiating a calibration routine.

Imaging assemblies60are accommodated by the bezel26, with each imaging assembly60being positioned adjacent a different corner of the bezel. The imaging assemblies60are oriented so that their fields of view overlap and look generally across the entire interactive surface24. In this manner, any pointer such as for example a pen tool, an eraser tool, a user's finger, a cylinder or other suitable object, that is brought into proximity of the interactive surface24appears in the fields of view of the imaging assemblies60. A power adapter62provides the necessary operating power to the interactive board22when connected to a conventional AC mains power supply.

Turning now toFIG. 3, one of the imaging assemblies60is better illustrated. As can be seen, the imaging assembly60comprises an image sensor70such as that manufactured by Aptina (Micron) MT9V034 having an effective resolution of 752×98 pixels. The image sensor70is fitted with a two element, plastic lens (not shown) that provides the image sensor70with a field of view of approximately 104 degrees. In this manner, the other imaging assemblies60are within the field of view of the image sensor70thereby to ensure that the field of view of the image sensor70encompasses the entire interactive surface24.

A digital signal processor (DSP)72such as that manufactured by Analog Devices under part number ADSP-BF522 Blackfin or other suitable processing device, communicates with the image sensor70over an image data bus74via a parallel port interface (PPI). A serial peripheral interface (SPI) flash memory75is connected to the DSP72via an SPI port and stores the firmware required for image assembly operation. Depending on the size of captured image frames as well as the processing requirements of the DSP72, the imaging assembly60may optionally comprise synchronous dynamic random access memory (SDRAM)76to store additional temporary data as shown by the dotted lines. The image sensor70also communicates with the DSP72via a two-wire interface (TWI) and a timer (TMR) interface. The control registers of the image sensor70are written from the DSP72via the TWI in order to configure parameters of the image sensor70such as the integration period for the image sensor70.

In this embodiment, the image sensor70operates in snapshot mode. In the snapshot mode, the image sensor70, in response to an external trigger signal received from the DSP72via the TMR interface that has a duration set by a timer on the DSP72, enters an integration period during which an image frame is captured. Following the integration period after the generation of the trigger signal by the DSP72has ended, the image sensor70enters a readout period during which time the captured image frame is available. With the image sensor in the readout period, the DSP72reads the image frame data acquired by the image sensor70over the image data bus74via the PPI. The frame rate of the image sensor70in this embodiment is between about 900 and about 960 frames per second. The DSP72in turn processes image frames received from the image sensor72and provides pointer information to a master controller50at a reduced rate of approximately 120 points/sec. Those of skill in the art will however appreciate that other frame rates may be employed depending on the desired accuracy of pointer tracking and whether multi-touch and/or active pointer identification is employed.

Three strobe circuits80communicate with the DSP72via the TWI and via a general purpose input/output (GPIQ) interface. The IR strobe circuits80also communicate with the image sensor70and receive power provided on LED power line82via the power adapter62. Each strobe circuit80drives a respective illumination source in the form of an infrared (IR) light emitting diode (LED)84. The IR LEDs84provide infrared backlighting over the interactive surface24. Further specifics concerning the strobe circuits80and their operation are described in U.S. patent application Ser. No. 12/709,451 to Akitt entitled “INTERACTIVE INPUT SYSTEM AND ILLUMINATION SYSTEM THEREFOR” filed on Feb. 19, 2010, the content of which is incorporated herein by reference in its entirety.

The DSP72also communicates with an RS-422 transceiver86via a serial port (SPORT) and a non-maskable interrupt (NMI) port. The transceiver86communicates with the master controller50over a differential synchronous signal (DSS) communications link88and a synch line90. Power for the components of the imaging assembly60is provided on power line92by the power adapter62. DSP72may also optionally be connected to a USB connector94via a USB port as indicated by the dotted lines. The USB connector94can be used to connect the imaging assembly60to diagnostic equipment.

The image sensor70and its associated lens as well as the IR LEDs84are mounted on a housing assembly100that is best illustrated inFIGS. 4aand4b. As can be seen, the housing assembly100comprises a polycarbonate housing body102having a front portion104and a rear portion106extending from the front portion. An imaging aperture108is centrally formed in the housing body102and accommodates an IR-pass/visible light blocking filter110. The filter110has an IR-pass wavelength range of between about 830 nm and about 698 nm. The image sensor70and associated lens are positioned behind the filter110and oriented such that the field of view of the image sensor70looks through the filter110and generally across the interactive surface24. The rear portion106is shaped to surround the image sensor70. Three passages112ato112care formed through the housing body102. Passages112aand112bare positioned on opposite sides of the filter110and are in general horizontal alignment with the image sensor70. Passage112cis centrally positioned above the filter110. Each tubular passage receives a light source socket114that is configured to receive a respective one of the IR LEDs84. Mounting flanges116are provided on opposite sides of the rear portion106to facilitate connection of the housing assembly100to the bezel26via suitable fasteners. A label118formed of retro-reflective material overlies the front surface of the front portion104. Further specifics concerning the housing assembly and its method of manufacture are described in U.S. patent application Ser. No. 12/709,419 to Liu et al. entitled “HOUSING ASSEMBLY FOR INTERACTIVE INPUT SYSTEM AND FABRICATION METHOD” filed on Feb. 19, 2010, the content of which is incorporated herein by reference in its entirety.

The master controller50is better illustrated inFIG. 5. As can be seen, master controller50comprises a DSP200such as that manufactured by Analog Devices under part number ADSP-BF522 Blackfin or other suitable processing device. A serial peripheral interface (SPI) flash memory202is connected to the DSP200via an SPI port and stores the firmware required for master controller operation. A synchronous dynamic random access memory (SDRAM)204that stores temporary data necessary for system operation is connected to the DSP200via an SDRAM port. The DSP200communicates with the general purpose computing device28over the USB cable30via a USB port. The DSP200communicates through its serial port (SPORT) with the imaging assemblies60via an RS-422 transceiver208over the differential synchronous signal (DSS) communications link88. In this embodiment, as more than one imaging assembly60communicates with the master controller DSP200over the DSS communications link88, time division multiplexed (TDM) communications is employed. The DSP200also communicates with the imaging assemblies60via the RS-422 transceiver208over the camera synch line90. DSP200communicates with the tool tray48over communications link210.

As will be appreciated, the architectures of the imaging assemblies60and master controller50are similar. By providing a similar architecture between each imaging assembly60and the master controller50, the same circuit board assembly and common components may be used for both thus reducing the part count and cost of the interactive input system20. Differing components are added to the circuit board assemblies during manufacture dependent upon whether the circuit board assembly is intended for use in an imaging assembly60or in the master controller50. For example, the master controller50may require a SDRAM76whereas the imaging assembly60may not.

The general purpose computing device28in this embodiment is a personal computer or other suitable processing device or structure comprising, for example, a processing unit, system memory (volatile and/or non-volatile memory), other non-removable or removable memory (e.g. a hard disk drive, RAM, ROM, EEPROM, CD-ROM, DVD, flash memory, etc.) and a system bus coupling the various computer components to the processing unit. The general purpose computing device28may also comprise a network connection to access shared or remote drives, one or more networked computing devices, and/or other networked devices.

Turning now toFIGS. 6 and 7, the tool tray48is better illustrated. As can be se seen, the upper surface49aof the housing49has a rear portion49bwith a generally planar surface that is shaped to abut an underside of the interactive board22. The housing49of tool tray48accommodates a plurality of circuit card arrays (CCAs) or boards each supporting circuitry associated with functionality of the tool tray48. A first main controller board (not shown) supports the master controller50and is responsible for controlling the overall functionality of the tool tray48. The main controller board also comprises a USB connector (not shown) and a data connection port (not shown) for enabling connection to the imaging assemblies60. The main controller board additionally has a power connection port (not shown) for enabling connection to the power adapter62, and an audio output port (not shown) for enabling connection to one or more speakers.

The main controller board is connected to an attribute button control board (not shown) comprising tool sensors (not shown) that are associated with the receptacles48aand48b. The tool sensors are described in above-incorporated U.S. patent application Ser. No. 12/709,424 to Bolt et al. In this embodiment, the sensors are grouped into two pairs, with each pair being mounted as a set within a respective receptacle48aor48b. Each pair of sensors comprises an infrared (IR) transmitter and IR receiver for detecting the presence of a tool within its associated receptacle, whereby detection of a tool within the receptacle occurs as a result of the tool interrupting an infrared signal transmitted by the IR transmitter across the slot to the IR receiver. The attribute button control board is linked to a power module board on which the power button52is physically mounted, together with an LED contained within the power button for indicating power on/off status.

The tool tray48also comprises a display98that is centrally positioned on the upper surface of the tool tray housing49between receptacles48aand48b.Display98comprises a colour liquid crystal display (LCD) panel having a resistive touch layer disposed thereon. The resistive touch layer enables the display98to receive touch input, and thereby allows a user to interact with the display98using a pen tool P, a finger or other suitable object.

Display98is connected to the power adaptor62through a display control board (not shown) housed within the tool tray48thereby to allow a user to switch power on and off to selected components of the interactive input system20, such as for example the projector, through touch interaction with the display98. A microcontroller (not shown) having an embedded Linux operating system for controlling the operation of the display98is mounted on the display control board. The microcontroller is also in communication with the master controller50, enabling touch input made on the display98to be reflected in images displayed on the interactive surface24, and also enabling touch input made on the interactive surface24to be reflected in images presented on the display98. As will be understood, since the interactive surface24and display98are each connected to, and controlled by, separate controllers, input events on the display98will not shift focus away from, or interrupt programs running on the general purpose computing device28and displayed on the interactive surface24. In this manner, display98is not merely an extension of any “desktop environment” displayed on the interactive surface24, but rather functions as a separate “environment” simultaneously with any “environment” displayed on the interactive surface24.

The display control board is also in communication with the attribute button control board. As a result, display98is configured to display a pointer attribute selection screen whenever a pen tool P is removed from a receptacle48aor48bas detected by the sensors associated with the receptacle. In this embodiment, the pointer attribute selection screen is a pointer colour selection screen230as shown inFIG. 7. As can be seen, the pointer colour selection screen230comprises a plurality of virtual buttons232ato232h. Each of the buttons232ato232his associated with a different pointer colour, wherein buttons232a,232b,232c,232d,232e,232f,232gand232hare associated with black, red, green, blue, orange, purple, brown and pink pointer colours, respectively. Selection of any of the buttons232ato232hautomatically assigns a pointer mode to subsequent pointer input made on the interactive surface24. For example, selection of virtual button232b, assigns the pointer mode to subsequent pointer input and sets the pointer colour to red. As a result, when a pointer is brought into contact with and moved across the interactive surface24, the resulting pointer data that is provided to the general purpose computing device28by the interactive board22is treated as red digital ink and is used to update the image displayed on the interactive surface24accordingly. Also, when one of the buttons232ato232his selected, the selected button blinks to provide a visual indication that the attributed pointer colour is active. When the button associated with the active colour is selected again, the active status indication is cancelled and the pointer input mode reverts to a default pointer mode so that subsequent pointer contacts on the interactive surface24are treated as mouse events.

The tool tray48is configured to provide functionality for either a single user or multiple users. When sensors detect the removal of a pen tool P from only one of the receptacles48aor48b, the display control board determines that only a single user is interacting with the interactive board22and notifies the master controller50. The master controller50in turn configures the interactive board22to operate in a “single-screen” display mode. In this embodiment, the “single-screen” display mode is the default display mode. As a result, all pointer contacts with the interactive surface24are treated as being made by the same user. When the sensors detect the removal of pen tools P from both receptacles48aand48b, the display control board determines that multiple users are interacting with the interactive board22and notifies the master controller50. The master controller50in turn configures the interactive board22to operate in a “split-screen” mode. In the split-screen mode, the interactive surface24is partitioned into two side-by-side screen portions. Pointer input made on each screen portion of the interactive surface24is treated as pointer input made by different users. In this embodiment, selection of one of the buttons232ato232hwhile the in split-screen mode assigns the associated pointer colour to all subsequent pointer input made on both screen portions of the interactive surface24.

Display98is also configured to display other information and to provide selectability of other pointer input attributes. For example, display98is also configured to display a login screen240, a home screen242, a scheduling screen244, a pointer attribute screen246, an eraser attribute screen248, an insert object screen250, a settings screen252, and a logout screen254, as shown inFIGS. 8ato8h, respectively.FIG. 8abetter illustrates the login screen240, which is displayed on display98when the interactive input system20is powered on. Login screen240comprises an on-screen keyboard240a, and user name and password fields240b. The on-screen keyboard240aallows a user to log into the interactive input system20by entering their user name and password into the fields240b. Once a correct user name/password combination has been entered, settings and files associated with the user may be accessed through interactions with the interactive surface24.

FIG. 8bbetter illustrates the home screen242, which is displayed on display98after a user has logged into the interactive input system20using login screen240. Home screen242comprises a plurality of virtual buttons242ato242f, each of which is selectable through touch input with the display98to display screens244to254.

FIG. 8cshows the scheduling screen244, which is displayed when the scheduling button242don the home screen242is selected. As can be seen, the scheduling screen244comprises information pertaining to status of a current meeting, such as for example a meeting room identifier, a meeting coordinator name, the amount of time remaining in the current meeting, the current time of day, the number of participants (not shown), names of participants (not shown), conflicting meeting notes (not shown), and the time of next booking of the meeting room. Scheduling screen244also comprises a Change Meeting virtual button244a, which may be selected to display a change meeting screen256(not shown) on display98. Change meeting screen256comprises virtual buttons (not shown) each associated with a respective meeting-related function similar to functions provided by Microsoft Office Outlook®, and which may be selected for executing the meeting-related functions. These meeting-related functions may comprise, for example, cancelling the meeting, pausing the meeting, extending the meeting, moving the meeting to a different location or time, and inviting additional participants. As will be appreciated, enabling such meeting-related actions to be executed through touch input with display98, instead of through interaction with interactive surface24, provides both increased privacy and ease-of-use. Scheduling screen244also provides an indication when a meeting is about to end by causing the scheduling screen244to blink red and to display a “meeting end” message. The indication can be stopped by touching the display98. Scheduling screen244also comprises a home button270, which is selectable to return to the home screen242.

FIG. 8dshows the pointer attribute screen246, which is displayed when the pointer button242aon the home screen242is selected. As can be seen, the pointer attribute screen246comprises a plurality of virtual buttons that are each selectable by a user for assigning a pointer attribute to subsequent pointer input. Selection of any of the virtual buttons on pointer attribute screen246automatically assigns the pointer mode to subsequent pointer input made on the interactive surface24. Here, the subsequent pointer input may be from any pointer, such as for example a pen tool P, an eraser tool, a finger or other object, and for example while both pen tools P are present within receptacles48aand48b. In the embodiment shown, the pointer attributes comprise pointer size, pointer shape, and pointer colour. Each of these pointer attributes may be assigned to subsequent pointer input by selecting the virtual button associated with the attribute. In this embodiment, when a virtual button is selected, the thickness of the border surrounding the selected virtual button increases to indicate the active status of the pointer attribute associated with that button. Additionally, a pointer colour selection made using pointer attribute screen246overrides any previous selection made using colour selection screen230. Similarly, any subsequent pointer colour selection made using colour selection screen230overrides any previous selection made using pointer attribute screen246. Pointer attribute screen246also comprises a home button270that may be selected to return to the home screen270.

FIG. 8eshows the eraser attribute screen248, which is displayed when the eraser button242bon the home screen242is selected. As can be seen, the eraser attribute screen248comprises a plurality of virtual buttons that are each selectable by a user for assigning an eraser attribute to subsequent pointer input. Selection of any of the virtual buttons on eraser attribute screen248automatically assigns the eraser mode to subsequent pointer input made on the interactive surface24. Here, the subsequent pointer input may be from any pointer, such as for example a pen tool P, an eraser tool, a finger or other object, and for example while both pen tools P are present within receptacles48aand48b. In the embodiment shown, the eraser attributes comprise eraser size and eraser shape. Each of these eraser attributes may be assigned by selecting the virtual button associated with that attribute. When a virtual button is selected, the thickness of the border surrounding the selected virtual button increases to indicate the active status of the eraser attribute associated with that button. Eraser attribute screen248also comprises a home button270that may be selected to return to the home screen242.

FIG. 8fshows the insert object screen250, which is displayed when the insert object button242aon the home screen242is selected. As can be seen, the insert object screen250comprises a plurality of virtual buttons that are each selectable by a user for assigning an object insertion attribute to subsequent pointer input made on the interactive surface24. Selection of any of the virtual buttons on insert object screen250automatically assigns an object insertion mode to subsequent pointer input. Here, the subsequent pointer input may be from any pointer, such as for example a pen tool P, an eraser tool, a finger or other object, and for example while both pen tools P are present within receptacles48aand48b. In the embodiment shown, the object insertion attributes are animal images. Each of these object insertion attributes may be assigned by selecting the virtual button associated with that attribute. When a virtual button is selected, the thickness of the border surrounding the selected virtual button increases to indicate the active status of the attribute associated with that button. Insert object screen250also comprises a home button270that may be selected to return to the home screen242.

FIG. 8gshows the settings screen252, which is displayed when the settings button242eon the home screen242is selected. As can be seen, the setting screen252comprises a plurality of virtual buttons that are each selectable by a user for accessing various service functions and information pages generally relating to the interactive input system20. In the embodiment shown, the virtual buttons are selectable for adjusting display contrast, for troubleshooting, and for viewing product information. Settings screen252also comprises a home button270that may be selected to return to the home screen242.

FIG. 8hshows the logout screen254, which is displayed when the logout button242fon the home screen242is selected. As can be seen, the logout screen comprises a message prompting a user if he or she would like to logout, and a pair of “yes” and “no” virtual buttons that are each selectable for either logging out or not logging out that user. The logout screen254also comprises a home button270that may be selected to return to the home screen242.

During operation, the DSP200of the master controller50outputs synchronization signals that are applied to the synch line90via the transceiver208. Each synchronization signal applied to the synch line90is received by the DSP72of each imaging assembly60via transceiver86and triggers a non-maskable interrupt (NMI) on the DSP72. In response to the non-maskable interrupt triggered by the synchronization signal, the DSP72of each imaging assembly60ensures that its local timers are within system tolerances and if not, corrects its local timers to match the master controller50. Using one local timer, the DSP72initiates a pulse sequence via the snapshot line that is used to condition the image sensor to the snapshot mode and to control the integration period and frame rate of the image sensor70in the snapshot mode. The DSP72also initiates a second local timer that is used to provide output on the LED control line174so that the IR LEDs84are properly powered during the image frame capture cycle.

In use, the image sensor70of each imaging assembly60acquires image frames at the desired image frame rate in response to the pulse sequence output on the snapshot line. In this manner, image frames captured by the image sensor70of each imaging assembly60can be referenced to the same point of time allowing the position of pointers brought into the fields of view of the image sensors70to be accurately triangulated. Also, by distributing the synchronization signals for the imaging assemblies60, electromagnetic interference is minimized by reducing the need for transmitting a fast clock signal to each image assembly60from a central location. Instead, each imaging assembly60has its own local oscillator (not shown) and a lower frequency signal (e.g. the point rate, 120 Hz) is used to keep the image frame capture synchronized.

During image frame capture, the DSP72of each imaging assembly60also provides output to the strobe circuits80to control the switching of the IR LEDs84so that the IR LEDs are illuminated in a given sequence that is coordinated with the image frame capture sequence of each image sensor70. In particular, in the sequence the first image frame is captured by the image sensor70when the IR LED84accommodated by the socket114in passage112cis fully illuminated in a high current mode and the other IR LEDs are off. The next image frame is captured when all of the IR LEDs84are off. Capturing these successive image frames with the IR LED84on and then off allows ambient light artifacts in captured image frames to be cancelled by generating difference image frames as described in U.S. Application Publication No. 2009/0278794 to McReynolds et al., assigned to SMART Technologies ULC, the content of which is incorporated herein by reference in its entirety. The third image frame is captured by the image sensor70when only the IR LED84accommodated by the socket114in passage112ais on and the fourth image frame is captured by the image sensor70when only the IR LED84accommodated by the socket114in passage112bis on. Capturing these image frames allows pointer edges and pointer shape to be determined as described in International PCT Application Serial No. PCT/CA2011/000036 to SMART Technologies ULC et al. entitled ‘INTERACTIVE INPUT SYSTEM AND ILLUMINATION SYSTEM THEREFOR’ filed on Jan. 14, 2011, the content of which is incorporated herein by reference in its entirety. The strobe circuits80also control the IR LEDs84to inhibit blooming and to reduce the size of dark regions in captured image frames that are caused by the presence of other imaging assemblies60within the field of view of the image sensor70as will now be described.

During the image capture sequence, when each IR LED84is on, the IR LED floods the region of interest over the interactive surface24with infrared illumination. Infrared illumination that impinges on the retro-reflective bands of bezel segments40,42,44and46and on the retro-reflective labels118of the housing assemblies100is returned to the imaging assemblies60. As a result, in the absence of a pointer, the image sensor70of each imaging assembly60sees a bright band having a substantially even intensity over its length together with any ambient light artifacts. When a pointer is brought into proximity with the interactive surface24, the pointer occludes infrared illumination reflected by the retro-reflective bands of bezel segments40,42,44and46and/or the retro-reflective labels118. As a result, the image sensor70of each imaging assembly60sees a dark region that interrupts the bright band in captured image frames. The reflections of the illuminated retro-reflective bands of bezel segments40,42,44and46and the illuminated retro-reflective labels118appearing on the interactive surface24are also visible to the image sensor70.

FIG. 9ashows an exemplary image frame captured by the image sensor70of one of the imaging assemblies60when the IR LEDs84associated with the other imaging assemblies60are off during image frame capture. As can be seen, the IR LEDs84and the filter110of the other imaging assemblies60appear as dark regions that interrupt the bright band160. These dark regions can be problematic as they can be inadvertently recognized as pointers.

To address this problem, when the image sensor70of one of the imaging assemblies60is capturing an image frame, the strobe circuits80of the other imaging assemblies60are conditioned by the DSPs72to a low current mode. In the low current mode, the strobe circuits80control the operating power supplied to the IR LEDs84so that they emit infrared lighting at an intensity level that is substantially equal to the intensity of illumination reflected by the retro-reflective bands on the bezel segments40,42,44and46and by the retro-reflective labels118.FIG. 9bshows an exemplary image frame captured by the image sensor70of one of the imaging assemblies60when the IR LEDs84associated with the other imaging assemblies60are operated in the low current mode. As a result, the size of each dark region is reduced. Operating the IR LEDs84in this manner also inhibits blooming (i.e. saturation of image sensor pixels) which can occur if the IR LEDs84of the other imaging assemblies60are fully on during image frame capture. The required levels of brightness for the IR LEDs84in the low current mode are related to the distance between the image sensor70and the opposing bezel segments40,42,44, and46. Generally, lower levels of brightness are required as the distance between the image sensor70and the opposing bezel segments40,42,44, and46increases due to the light loss within the air as well as inefficient distribution of light from each IR LED towards the bezel segments40,42,44, and46.

The sequence of image frames captured by the image sensor70of each imaging assembly60is processed by the DSP72to identify each pointer in each image frame and to obtain pointer shape and contact information as described in above-incorporated International PCT Application Serial No. PCT/CA2011/000036 to SMART Technologies ULC et al. The DSP72of each imaging assembly60in turn conveys the pointer data to the DSP200of the master controller50. The DSP200uses the pointer data received from the DSPs72to calculate the position of each pointer relative to the interactive surface24in (x,y) coordinates using well known triangulation as described in above-incorporated U.S. Pat. No. 6,803,906 to Morrison. This pointer coordinate data along with pointer shape and pointer contact status data is conveyed to the general purpose computing device28allowing the image data presented on the interactive surface24to be updated to reflect pointer activity and in accordance with pointer attributes assigned to pointer input through interactions with screens presented on the display98.

Interactive input system20is not limited to use with tool tray48described above. If desired, the ends of the housing49of the tool tray48may be configured to receive detachable modules to provide the tool tray48with increased functionality as described in above-incorporated U.S. Application Nos. 61/294,831 and Ser. No. 12/709,424, to Bolt et al. Also, those of skill in the art will appreciate that still other tool tray configurations alternatively be employed.

For example,FIG. 10shows another embodiment of a tool tray for use with interactive input system20, and which is generally indicated by reference numeral348. Tool tray348is generally similar to tool tray48described above with reference toFIGS. 6 to 8. In this embodiment however, tool tray348comprises two (2) displays398aand398bwith each display being positioned adjacent an opposite end of the tool tray348. Tool tray348also comprises two (2) receptacles348aand348beach sized to receive a respective pen tool P1and P2, as shown. Each of the receptacles348aand348bis associated with a respective one of the display398aand398b. Display398ais configured to display a pointer colour selection screen (not shown) that is similar to pointer colour selection screen230described above and with reference toFIG. 7, upon removal of pen tool P1from receptacle348aand display398bis configured to display a similar pointer colour selection screen upon removal of pen tool P2from receptacle348b. As will be appreciated, by providing pointer colour selection screens on separate displays, each associated with an individual receptacle, a different colour may be assigned to pointer input made on the interactive surface24from each of the pen tools P1and P2in an independent manner. Here, the interactive board22differentiates between pen tools P1and P2when it is configured to operate in the split-screen mode. In this mode, each screen portion of the interactive surface24is associated with a respective one of the receptacles348aand348b, and therefore also with a respective pen tool P1or P2associated with each receptacle. The pen tools can also be differentiated if they are active pen tools that are each uniquely identifiable, such as those described in U.S. patent application Ser. No. 12/752,630 to McGibney et al., entitled “Interactive Input System and Information Input Method Therefor”, filed on Apr. 1, 2010, and assigned to SMART Technologies ULC, the content of which is herein incorporated by reference in its entirety. Tool tray348also comprises an eraser tool receptacle350sized to receive a respective eraser tool352.

FIG. 11shows yet another embodiment of a tool tray for use with interactive input system20, and which is generally indicated by reference numeral448. Tool tray448is generally similar to tool tray48described above and with reference toFIGS. 6 to 8. In this embodiment however, tool tray448comprises four (4) receptacles448ato448deach sized to receive a respective pen tool P1to P4. Pen tools P1to P4are active pen tools that are each uniquely identifiable, such as those described in above-incorporated U.S. application Ser. No. 12/752,630 to McGibney et al. Tool tray448also comprises four (4) displays499ato499deach associated with and positioned adjacent a respective receptacle448ato448d. Each display499ato499dcomprises a touch sensitive layer thereon configured to receive touch input. Each display499ato499dis sized to display a single line of text and is configured to display an attribute currently associated with its respective pen tool P1to P4. In this embodiment, the attribute is any of a pointer input mode and a pointer colour assigned to the pen tool. Here, the pointer input mode may be any of a pointer mode, a pointer mode, and an eraser mode. Displays499ato499dare also configured to display a virtual arrow button (not shown) that when selected, scrolls through the available pointer input modes. Each pointer input mode displayed is selectable through touch input with the display499ato499dthereby to allow that pointer input mode to be assigned to subsequent pointer input made on the interactive surface24using the respective pen tool P1to P4. When in the pointer mode, displays499ato499ddisplay the active pointer colour. Here, the active pointer colour assigned to each pen tool P1to P4is selected from a menu displayed on the interactive surface24.

FIG. 12shows still yet another embodiment of a tool tray for use with interactive input system20, and which is generally indicated by reference numeral548. Tool tray548is generally similar to tool tray48described above and with reference toFIGS. 6 to 8. In this embodiment, tool tray548comprises two (2) receptacles548aand548b, each sized to receive a respective pen tool P1and P2. Tool tray548also comprises a display598that is generally similar to display98as well as two (2) displays599aand599b, each of which is associated with and positioned adjacent a respective receptacle548aand548b. Displays599aand599bare generally similar to displays499ato499ddescribed above and with reference toFIG. 11, and are sized to display a single line of text. Displays599ato599dcomprise a touch sensitive layer thereon configured to receive touch input. In this embodiment, display599ais configured to indicate a current pointer input mode of the pen tool P1in the receptacle548aand display599bis configured to indicate a current pointer input mode of the pen tool P2in receptacle548b. When a pen tool P1or P2is removed from its respective receptacle548aor548b, the respective display599aor599bdisplays a set of virtual buttons each of which is associated with a pointer colour. Selection of one of the virtual buttons assigns that colour to subsequent pointer input made on the interactive surface24using pen tool P1or P2or other pointer, and also overrides any colour previously assigned to that pen tool P1or P2. If the interactive board22is configured to operate in the single-screen mode, the pen tool remaining in the other receptacle is also assigned the colour selected for the pen tool that has been removed. Alternatively, if the interactive board22is configured to operate in the split-screen mode, the pen tool remaining in the other receptacle is not assigned the colour selected for the pen tool that has been removed.

FIG. 13shows an alternative display for the tool tray of the interactive input system20and which is generally indicated using reference numeral698. Display698is generally similar to display98described above and with reference toFIGS. 6 to 8. In this embodiment however, display698is not configured to receive touch input. Rather, display698has associated with it a plurality of eight (8) physical buttons632ato632hpositioned along opposite sides of its display surface. Display698, similar to display98, is configured to display a pointer input attribute selection screen when a tool is removed from its receptacle. Each of the buttons632ato632hmay be pressed to assign an associated attribute to subsequent pointer input made on the interactive surface24. In the embodiment shown inFIG. 13, the pointer input attribute selection screen is a pointer colour selection screen630. Pointer colour selection screen630comprises a plurality of pointer colours and associated arrows that point to each of the buttons632ato632h. Here, buttons632a,632b,632c,632d,632e,632f,632gand632hare associated with black, red, green, blue, orange, purple, brown and pink colours, respectively. Selection of any of the buttons632ato632hautomatically assigns a pointer mode to subsequent pointer input made on the interactive surface24. For example, pressing attribute button632bassigns the pointer mode to subsequent pointer input and sets the pointer colour to red. In this embodiment, when a button632ato632his pressed, the pressed button blinks to provide a visual indication that the colour is active. When the button associated with the active colour is pressed again, the active status indication is cancelled and the pointer input mode reverts to a default pointer input mode. In this embodiment, the default pointer input mode is the pointer mode.

Although in embodiments described above, the tool tray comprises selectable virtual or physical buttons, in other embodiments, the tool tray may comprise a combination of virtual and physical buttons and/or other selectable features such as dials to enable a user to provide input to the tool tray.

Although in embodiments described above, the display is described as comprising a colour LCD panel and a resistive touch layer thereon, in other embodiments, the display may alternatively be a black and white or a grayscale LCD display panel. Also, alternative types of display panels may be employed such as for example light emitting diode (LED) display panels, organic light emitting diode (OLED) display panels, organic light emitting transistor (OLET) display panels, surface-conduction electron-emitter display (SED) panels, electroluminescent display (ELD) panels or plasma display panels (PDPs). Also, alternative touch technology may be used with the display such as for example capacitive, projected capacitive, electromagnetic induction, acoustic or camera-based touch technology.

Although in embodiments described above, the display is described as displaying a login screen, scheduling screen, a pointer screen, and an eraser attribute screen, in other embodiments, other screens may alternatively be displayed. For example, the display may alternatively display information related to inserting digital objects, such as for example images, 3D objects, and media files, or formatting toolbars, such as for example those found in Microsoft® Office products. Alternatively, the display may display screens enabling the display to serve as an extended desktop environment for the interactive surface24.

In other embodiments, the tool tray may comprise circuitry for switching between video sources that is controlled by a virtual or physical button which when selected, causes the display to present a list of available video inputs to the projector. For example, these available video inputs may include “VGA”, “HDMI”, “composite video”, “component video”, and so forth, depending on the type of video input. If the projector has more than one of a particular kind of video input, these could be referred to as “VGA1”, “VGA2”, for example. Alternatively, the display may display a list of particular types of devices likely to be connected to the various video ports. For example, one of the video ports input may be designated as “Meeting Room PC”, while another of the ports may be designated as “Guest Laptop”, etc. Selecting a particular video input from the list of available video inputs displayed causes a video switching accessory (not shown) installed in the tool tray to change to that video input. Here, the video switching accessory would have input ports (not shown) corresponding to various formats of video input, such as VGA, HDMI, composite video, component video, and the like, for allowing the connection of laptops, DVD players, VCRs, Bluray players, gaming machines such as Sony Playstation 3, Microsoft Xbox 360 or Nintendo Wii, and/or other video/media devices to the interactive input system20.

Although in embodiments described above, the tool tray is centrally located relative to the interactive surface24, in other embodiments, the tool tray may alternatively be located at another location relative to the interactive surface24, such as towards an edge or on the side of the interactive surface.

Although in embodiments described above, the interactive input system20has one tool tray, in other embodiments, the interactive input system may employ two or more tool trays positioned either on the same or on different sides of the interactive board.

Although in embodiments described above, the tool tray is described as having specific numbers of displays and/or single-line displays, those of skill in the art will appreciate that the specified numbers of displays and single-line displays are for illustrative purposes only and that alternative numbers of displays and/or single-line displays may be employed.

Although in embodiments described above, the interactive input system employs machine vision to detect one or more pointers in proximity with the interactive surface, in other embodiments, the interactive input system may instead employ an interactive board that makes use of other technology to detect pointer interaction with the interactive surface. For example, the interactive input system may employ an interactive board utilizing any of analog resistive, surface acoustic wave, electromagnetic induction, capacitive, and projected capacitive technologies, or other suitable technologies known in the art to detect pointer input.

In the embodiments described above, a short-throw projector is used to project an image onto the interactive surface24. As will be appreciated other front projection devices or alternatively a rear projection device may be used to project the image onto the interactive surface24. Rather than being supported on a wall surface, the interactive board22may be supported on an upstanding frame or other suitable support. Still alternatively, the interactive board22may engage a display device such as for example a plasma television, a liquid crystal display (LCD) device etc. that presents the image data output of the general purpose computing device28.

Although in embodiments described above, the tool tray comprises one or more receptacles for supporting pen or eraser tools, in other embodiments, the tool tray may alternatively not comprise any tool supporting receptacle. In these alternative embodiments, any pointer attribute selected by touch input with the display, or by using physical buttons adjacent the display, is assigned to all subsequent pointer input made on the interactive surface24.

Although a specific processing configuration has been described, those of skill in the art will appreciate that alternative processing configurations may be employed. For example, one of the imaging assemblies may take on the master controller role. Alternatively, the general purpose computing device may take on the master controller role.

Although in embodiments described above, pointer attributes are selectable by pressing virtual attribute buttons displayed on the display or by pressing physical buttons on the display, in other embodiments, pointer attributes may alternatively be selectable from a software toolbar displayed on the interactive surface24. Here, the tool tray may comprise a physical or virtual button which, when pressed, directs the general purpose computing device28to display such a toolbar. In a related embodiment, when the interactive input system20is operating in the split-screen mode, each screen portion of the interactive surface24may have a respective software toolbar displayed thereon. When an attribute is selected from such a toolbar, the selected attribute can be applied to all pointer input made on the interactive surface within the respective screen portion of the interactive surface, and may also be used to override any attribute information previously selected using the display. Here, the selection of an attribute from the software toolbar cancels any status indication provided by the display. If a common attribute (e.g. blue pointer colour) is selected using the respective software toolbar of both screen portions, the blue status indicator on the display is activated.

Although in embodiments described above, the interactive board operates in the split-screen mode in which selection of an attribute button from a single set of attribute buttons displayed on the display assigns that attribute to pointer input on both screen portions, in other embodiments, two sets of attribute buttons may alternatively be displayed on the display for assignment of a respective attribute to pointer input made on each screen portion of the interactive surface24.

Although in embodiments described above, the tool tray comprises control buttons provided on the upper surface of the housing to enable additional functions to be executed using the interactive input system, where the additional functions include launching an on-screen keyboard and initiating a calibration routine, in other embodiments, the additional functions may alternatively comprise other functions.

Although in embodiments described above, each virtual button blinks to indicate that the attribute associated with the virtual button is active, in other embodiments, the virtual button may instead remain in an illuminated state.

Although in embodiments described above, the default pointer input mode is a pointer mode where pointer input is treated as mouse events, in other embodiments, the default pointer input mode may alternatively be the pointer mode having a black pointer colour.

Although embodiments have been described, those of skill in the art will appreciate that other variations and modifications may be made without departing from the spirit and scope thereof as defined by the appended claims.