Source: http://www.google.com/patents/US20070290996?ie=ISO-8859-1&dq=6004266
Timestamp: 2014-10-22 16:17:27
Document Index: 690271028

Matched Legal Cases: ['art 300', 'art 350', 'art 300', 'art 300', 'art 500', 'art 600']

Patent US20070290996 - Image processing method of pointer input system - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsThe present invention relates to an image processing method of a pointer input system. In the image processing method, a color channel image is generated from an image pickup device, so that image processing capability, the image processing efficiency and the data transmitting speed of the pointer input...http://www.google.com/patents/US20070290996?utm_source=gb-gplus-sharePatent US20070290996 - Image processing method of pointer input systemAdvanced Patent SearchPublication numberUS20070290996 A1Publication typeApplicationApplication numberUS 11/754,116Publication dateDec 20, 2007Filing dateMay 25, 2007Priority dateJun 1, 2006Also published asUS7852316Publication number11754116, 754116, US 2007/0290996 A1, US 2007/290996 A1, US 20070290996 A1, US 20070290996A1, US 2007290996 A1, US 2007290996A1, US-A1-20070290996, US-A1-2007290996, US2007/0290996A1, US2007/290996A1, US20070290996 A1, US20070290996A1, US2007290996 A1, US2007290996A1InventorsLi-Wen TingOriginal AssigneeMicro-Nits Co. Ltd.Export CitationBiBTeX, EndNote, RefManReferenced by (5), Classifications (6), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetImage processing method of pointer input systemUS 20070290996 A1Abstract The present invention relates to an image processing method of a pointer input system. In the image processing method, a color channel image is generated from an image pickup device, so that image processing capability, the image processing efficiency and the data transmitting speed of the pointer input system are enhanced.
discriminating the accuracy of said space conversion relationship, wherein said calibrating method is terminated if the accuracy of said space conversion relationship is acceptable; detecting some parameters of said pointer input system if the accuracy of said space conversion relationship is unacceptable; repeating the steps (a)�(e) if said parameters of said pointer input system are correct; and modifying said parameters of said pointer input system if said parameters of said pointer input system are incorrect, and repeating the steps (a)�(e). 6. The calibrating method according to claim 5 wherein the step of discriminating the accuracy of said space conversion relationship is performed by comparing the coordinate values of said aligning data converted by said space conversion relationship with the coordinate value of said reference data, and wherein the step of detecting some parameters of said pointer input system is performed by detecting a number of pixels per frame of an image projecting system and an aspect ratio of said image projecting system.
FIELD OF THE INVENTION The present invention relates to an image processing method, and more particularly to an image processing method of a pointer input system.
BACKGROUND OF THE INVENTION In a conventional pointer input system, a light spot generated from a pointer generating device is displayed on a display screen, and the image of the display screen including the light spot is then captured by an image pickup device so as to obtain an output data relevant to the light-spot position and display a corresponding output pointer on the display screen. Generally, the image processing method of the pointer input system mainly includes a calibrating method and a tracking method. For performing the calibrating method, a aligning pattern is inputted onto the display screen, and the image of the display screen including the aligning pattern is then captured by the image pickup device to obtain a aligning data. After the aligning data is compared with a predetermined reference data, a space conversion relationship between the display screen and the image pickup device is obtained. For performing the tracking method, the image of the display screen including the light spot is captured by the image pickup device to obtain an output data relevant to the light-spot position according to the space conversion relationship. After the output data is processed by a data processing device, related output pointer is projected onto the display screen through an image projecting system.
Typically, a full-color image contains a variety of colors including for example a red color, a blue color and a green color. Since the full-color image is usually processed according to an arithmetic average or a weighted average of a red color value, a blue color value and a green color value, the characteristic properties of having different signal-to-noise ratios (S/N) and gains in each or composite color channel are not utilized to enhance the image processing capability. Under this circumstance, since the data amount of processing the full-color image is very huge, the image processing efficiency and the data transmitting speed fail to be increased and the stability of the pointer input system is impaired.
Therefore, there is a need of providing an image processing method of a pointer input system for increasing the image processing capability, the image processing efficiency and the data transmitting speed of the pointer input system.
SUMMARY OF THE INVENTION It is an object of the present invention to provide an image processing method of a pointer input system for increasing the image processing capability, the image processing efficiency and the data transmitting speed of the pointer input system.
Another object of the present invention provides an image processing method of a pointer input system for use with a cathode ray tube (CRT) monitor, a liquid crystal display (LCD) screen, a plasma display panel or a projection screen.
In accordance with an aspect of the present invention, there is provided a calibrating method of a pointer input system. The calibrating method includes steps of displaying a aligning test pattern on a display screen, capturing the display screen by an image pickup device to obtain a aligning image including the aligning test pattern, generating a color channel image according to the aligning image, scanning a region of interest of the color channel image to obtain a aligning data, and comparing the aligning data with a predetermined reference data to obtain a space conversion relationship between the display screen and the image pickup device.
In accordance with another aspect of the present invention, there is provided a tracking method of a pointer input system. The tracking method includes steps of allowing a pointer generating device to issue a light spot on a display screen, capturing the display screen by an image pickup device to obtain a tracking image including the light spot, generating a color channel image according to the tracking image, eliminating background noise of the color channel image, scanning a region of interest of the color channel image to obtain a scan data, generating an output data associated with the position of the light spot according to a space conversion relationship and the scan data, and displaying a pointer corresponding to the light spot on the display screen through an image projecting system according to the output data.
By utilizing the characteristic properties of having different signal-to-noise ratios (S/N) and gain values in different color channels, the image processing capability of the pointer input system is enhanced. Instead of using the full-color image, a color channel image is processed. Since the data amount of processing the color channel image is largely reduced, the image processing efficiency and the data transmitting speed are both enhanced.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 schematically illustrates three typical aligning test patterns used in an image processing method of a pointer input system according to the present invention;
FIG. 2 is a flowchart illustrating a calibrating method of a pointer input system according to a first embodiment of the present invention;
FIG. 3 is a schematic view of the pointer input system for implementing the calibrating method of FIG. 2;
FIG. 4 schematically illustrates segmental patterns divided by a aligning test pattern to be successively projected onto the display screen;
FIG. 5 schematically illustrates that light spots issued from a pointer generating device are projected onto specified locations of the display screen;
FIG. 6 schematically illustrates that a aligning test pattern is arranged at a corner of the display screen;
FIG. 7 is a flowchart illustrating a calibrating method of a pointer input system according to a second embodiment of the present invention;
FIG. 8 is a flowchart illustrating a calibrating method of a pointer input system according to a third embodiment of the present invention;
FIG. 9 schematically illustrates a aligning test pattern for used in a pre-calibrating method;
FIG. 10 schematically illustrates a aligning test pattern for used in a full-screen calibrating method;
FIG. 11 is a flowchart illustrating a tracking method of a pointer input system according to an embodiment of the present invention;
FIG. 12 is a schematic view of the pointer input system for implementing the tracking method of FIG. 11;
FIG. 13 is a flowchart illustrating a tracking method of a pointer input system according to another embodiment of the present invention;
FIG. 14 is a schematic view illustrating an anticipated extrapolation position constructed from known central positions of the elongated zones by extrapolation algorithm;
FIG. 15 is a schematic view illustrating an interpolation position constructed from known central positions of the elongated zones and the extrapolation position by interpolation algorithm; and
FIG. 16 is a schematic view illustrating some sensing points for discriminating change of the surrounding's light.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT The present invention provides an image processing method of a pointer input system by referring to a color channel image of an image pickup device in order to increase the image processing capability, the image processing efficiency and the data transmitting speed of the pointer input system. The image processing method of the present invention includes a calibrating method and a tracking processing. By adjusting the resolution of the image pickup device, the image obtained during the calibrating method and the image obtained during the tracking method have different resolutions. For example, the resolution of the image obtained during the tracking method is smaller than that obtained during the calibrating method. As a consequence, the aligning accuracy and the tracking processing efficiency and transmitting speed are enhanced.
In FIG. 1, three typical aligning test patterns used in the calibrating method are schematically illustrated. The aligning test patterns of FIG. 1 include a checkerboard pattern 210, a block dot pattern 220 and a cross dot pattern 230, respectively. The aligning test patterns may have arbitrary patterns. In this context, the checkerboard pattern 210 is used for illustration.
FIG. 2 is a flowchart 300 illustrating a calibrating method of a pointer input system according to a first embodiment of the present invention. FIG. 3 is a schematic view of the pointer input system for implementing the calibrating method of FIG. 2. The pointer input system of FIG. 3 principally includes a display screen 402, an image projecting system 404 and an image pickup device 408.
Hereinafter, a calibrating method according to a first embodiment of the present invention will be illustrated with reference to FIG. 2 and FIG. 3.
First of all, in Step 302, a aligning test pattern (e.g. the checkerboard pattern 210) is displayed on the display screen 402 through the image projecting system 404 (e.g. a projector). As shown in FIG. 4, the checkerboard pattern 210 is divided into multiple segmental patterns 440, e.g. 2�2 segmental patterns. These segmental patterns 440 are successively projected onto the display screen 402. Alternatively, as shown in FIG. 5, several light spots 422 issued from a pointer generating device 418 are projected onto specified locations 420 of the display screen 402. Alternatively, as shown in FIG. 6, the checkerboard pattern 210 may be arranged at different sites of the display screen 402. For example, the checkerboard pattern 210 may be arranged at a middle site, a corner or any arbitrary site of the display screen 402. After the calibrating method has been performed on different corners of the display screen 402, the interference from background noise is reduced. An example of the display screen 402 includes but is not limited to a cathode ray tube (CRT) monitor, a liquid crystal display (LCD) screen, a plasma display panel or a projection screen. By using the successive segmental patterns 440 of FIG. 4 or the light spots 422 of FIG. 5 as the aligning test pattern in replace of the whole checkerboard pattern 210, less counts of relevant reference points are sufficient for calibration.
Next, in Step 304, by adjusting the parameters including positions, exposure time, gamma values, gain values or brightness values of the image pickup device 408 (e.g. an array-type photosensing device), the numbers of recognized relevant marks 412 within a preview window 410 are varied and recorded. When the image having the highest number of recognized marks 412 is obtained, a capture field 401 and an optimal parameter of the image pickup device 408 are determined accordingly. In this embodiment, the locations of the marks 412 correspond to intersection points 413 of every four adjacent squares of the checkerboard pattern 210. In an embodiment, the image pickup device 408 may be fastened onto the display screen 402 or fixed in the vicinity of the image projecting system 404, so that the positional parameter of the image pickup device 408 is kept unchanged.
Next, after the display screen 402 is captured by the image pickup device 408 (Step 306), a aligning image 405 including the aligning test pattern is obtained (Step 308).
Next, according to the aligning image 405, a color channel image is generated (Step 310). The color channel image is a single color channel image or a composite color channel image included in the aligning image 405. For example, the color channel image is a red channel image or a composite red-and-blue channel image included in the aligning image 405. Optionally, a polarizer 406 and an optical filter 407 are disposed in front of the image pickup device 408 for filtering off the background noise from the image projecting system 404 or the surrounding light sources.
Next, in Step 312, a region of interest 409 of the color channel image is scanned. For example, the region of interest 409 corresponds to the input field 403 of the image projecting system 404. Since the image of the aligning test pattern or the input light spot is greater than one pixel, the region of interest 409 may be canned in alternate rows or in alternate columns so as to enhance the data transmitting speed and the processing speed. Moreover, since only the region of interest 409 is scanned, the interference from outside area of the region of interest 409 is minimized and the aligning accuracy is increased.
After the region of interest 409 is scanned, a aligning data is obtained (Step 314). Next, the aligning data is compared with a predetermined reference data with respect to the three dimensional coordinate system and the relative locations (Step 315). The reference data includes many spatial coordinate values of the checkerboard pattern 210. According to the aligning data, a space conversion relationship between the display screen 402 and the image pickup device 408 is calculated (Step 316). Meanwhile, the calibrating method of the image processing method of the pointer input system is implemented.
A second embodiment of a calibrating method of the image processing method according to the present invention will be illustrated with reference to a flowchart 350 of FIG. 7. In this flowchart 300, the steps 302, 304, 306, 308, 310, 312, 314, 315 and 316 are identical to those described in the flowchart 300 of FIG. 2, and are not redundantly described herein. In this embodiment, a discriminating procedure 340 is included in the calibrating method of the present invention after Step 316. If a difference between the coordinate value of the aligning data converted by the space conversion relationship (e.g. the coordinate values of the intersection points 413 of the checkerboard pattern 210 converted by the space conversion relationship) and the coordinate value of the reference data is smaller than a threshold value (Step 342), the accuracy of the space conversion relationship is acceptable and the calibrating method is terminated. Otherwise, some important parameters of the pointer input system (e.g. the number of pixels per frame of the image projecting system 404 or the aspect ratio of the image projecting system 404) are automatically detected. If these parameters of the pointer input system are correct (Step 344), the above steps 302�316 are repeated to obtain a new space conversion relationship. Otherwise, after the parameters are modified (Step 346), the above steps 302�316 are repeated.
A third embodiment of a calibrating method of the image processing method according to the present invention will be illustrated with reference to a flowchart 500 of FIG. 8. This calibrating method of FIG. 8 includes a pre-calibrating method 520 and a full-screen calibrating method 530. FIG. 9 and FIG. 10 schematically illustrate two aligning test patterns used in the pre-calibrating method 520 and the full-screen calibrating method 530, respectively. Hereinafter, the calibrating method according to the third embodiment of the present invention will be illustrated with reference to FIGS. 8, 9 and 10. First of all, in Step 502, a first aligning test pattern (e.g. a relatively smaller checkerboard pattern 450) is projected onto the display screen 402 through the image projecting system 404 (e.g. a projector). Then, the steps 302, 304, 306, 308, 310, 312, 314 and 315, which are identical to those described in FIG. 2, are carried out to obtain a first space conversion relationship between the display screen 402 and the image pickup device 408 is calculated (Step 506). Meanwhile, the pre-calibrating method 520 is implemented. Next, in Step 508, a second aligning test pattern (e.g. a relatively larger checkerboard pattern 460) is projected onto the display screen 402 through the image projecting system 404. Then, the steps 306, 308, 310, 312, 314 and 315 are carried out again to obtain a second space conversion relationship between the display screen 402 and the image pickup device 408 is calculated (Step 512). Meanwhile, the full-screen calibrating method 530 is implemented. For enhancing convenience, it is preferred that the full-screen calibrating method 530 is automatically performed. Optionally, the discriminating procedure 340 as described in FIG. 7 is done after the pre-calibrating method 520 or the full-screen calibrating method 530, thereby discriminating whether the accuracy of the first space conversion relationship or the second space conversion relationship is acceptable.
FIG. 11 is a flowchart 600 illustrating a tracking method of the image processing method of the pointer input system according to the present invention. FIG. 12 is a schematic view of the pointer input system for implementing the tracking method. Please refer FIG. 11 and 12. In the step 602, a light spot 702 issued from a pointer generating device 418 is projected onto the display screen 402. After the display screen 402 including the light spot 702 is captured by the image pickup device 408 (Step 604), a tracking image 704 is generated (Step 606). The tracking image 704 is composed of multiple pixels 703. The image pickup device 408 is for example an array-type photosensing device. Since the sensor of the image pickup device 408 has a response time, the image of the light spot 702 on the tracking image 704 is substantially an elongated zone 706. Next, according to the tracking image 704, a color channel image is obtained (Step 608). The color channel image is a single color channel image or a composite color channel image included in the tracking image 704. For example, the color channel image is a red channel image or a composite red-and-blue channel image included in the tracking image 704. Since the background noise resulting from the image projecting system 404 or the surrounding light sources may interfere with the color channel image, the background noise is eliminated (Step 609). For example, a polarizer 406 and an optical filter 407 (or a dual-band filter or a multi-band filter) are disposed in front of the image pickup device 408 for filtering off the background noise. Next, in Step 610, a region of interest 705 of the color channel image is scanned. For example, the region of interest 705 corresponds to the input field 403 of the image projecting system 404. Since the size of the elongated zone 706 is more than one pixel 703, the region of interest 705 may be canned in alternate rows or in alternate columns so as to enhance the data transmitting speed and the processing speed. Moreover, since only the region of interest 705 is scanned, the interference from outside area of the region of interest 705 is minimized and the tracking accuracy is increased. After the region of interest 409 is scanned, a scan data is obtained (Step 612). Next, according to the space conversion relationship obtained in a calibrating method and the scan data, an output data associated with the position of the light spot 702 is generated (Step 614). For example, specified optical attributes (e.g. the gray scales or 8-bit brightness values) for all pixels 703 included in the elongated zone 706 are measured. The characteristic values of respective pixels 703 are substantially weight coefficients. A central position 708 of the elongated zone 706 is obtained by calculating the weighted average of the 8-bit brightness values of all pixels 703 included in the elongated zone 706. That is, the central position 708 is at the center of mass of the elongated zone 706. Alternatively, the central position 708 of the elongated zone 706 is situated at the pixel having the maximum optical attribute or the pixel having the critical optical attribute. In this present invention, the optical attribute includes but is not limited to brightness, color, size or shape. More especially, the output data is generated after the central position 708 of the elongated zone 706 is subject to a space conversion according to the space conversion relationship. In the prior art, the pixels are denoted as either bright spot or non-bright spot by evaluating 2-bit brightness values of the pixels. Whereas, according to the present invention, the weighted average of the 8-bit brightness values of all pixels 703 included in the elongated zone 706 is much more precise for searching the light-spot position. In, the resolution of the image pickup device 408 is largely increased. Next, according to the output data generated in Step 614, a pointer 710 corresponding to the light spot 702 is displayed on the display screen 402 through the image projecting system 404 (Step 616).
FIG. 13 is a schematic view of the pointer input system for implementing the tracking method according to another embodiment of the present invention. The light spot 702 issued from the pointer generating device 418 is projected onto a point A the display screen 402 and moved to the point C through the point B. The display screen 402 including the light spot 702 are continuously captured by the image pickup device 408, thereby successively generating tracking images 712, 714 and 716. By the steps described above, the central positions 724, 726 and 728 of the elongated zones 718, 720 and 722 of the tracking images 712, 714 and 716 are calculated and recorded. As a consequence, continuous moving actions and relative positions of the continuously captured light spots 702 are monitored and tracked. After the central positions 724, 726 and 728 are subject to space conversions according to the space conversion relationship, corresponding output data are generated and displayed on the display screen 402 through the image projecting system 404. Likewise, the central positions 724, 726 and 728 of the elongated zones 718, 720 and 722 are obtained at the center of mass, the pixel having the maximum optical attribute or the pixel having the critical optical attribute. In this present invention, the optical attribute includes but is not limited to brightness, color, size or shape.
Moreover, according to the moving direction of the light spot 702, respective terminal portions 730, 732 and 734 of the elongated zones 718, 720 and 722 are defined. For example, one third of the elongated zone 718, 720 or 722 at the rear along the moving direction may be defined as the terminal portions 730, 732 or 734. Likewise, the central positions 736, 738 and 740 of the terminal portions 730, 732 and 734 are obtained, respectively. After the central positions 736, 738 and 740 of the terminal portions 730, 732 and 734 are subject to space conversions according to the space conversion relationship, the track corresponding to the moving spot 702 are generated and displayed on the display screen 402 through the image projecting system 404. As a consequence, a time delay resulting from data processing and transmission is reduced and the inconsistence between the input data and the output data will be minimized.
As shown in FIG. 14, the central positions 724, 726 and 728 of the elongated zones 718, 720 and 722 are recognized. By extrapolation algorithm, at least an extrapolation position 742 beyond the set of known positions 724, 726 and 728 will be anticipated. Consequently, the problem of causing time delay is reduced.
Moreover, as shown in FIG. 15, at least an interpolation position 744 is obtained from the known central positions 724, 726 and 728 and the extrapolation position 742 by interpolation algorithm (e.g. linear or bilinear interpolation algorithm). As the number of the interpolation position 744 is increase, the movement of the pointer looks smoother.
Alternatively, for further reducing the delay time, at least an extrapolation position beyond the set of known central positions 736, 738 and 740 of the terminal portions 730, 732 and 734 may be anticipated by extrapolation algorithm. Moreover, several interpolation positions may be obtained from the known central positions 736, 738 and 740 and the extrapolation position 742 by interpolation algorithm, so that the movement of the pointer looks much smoother.
In an embodiment, multiple pointer generating devices 418 are employed to project a plurality of light spots 702 onto the display screen 402, so that the purpose of implementing multi-input or opinion exchange will be achieved.
FIG. 16 schematically illustrating a method of detecting change of the surrounding's light by using the pointer input system of the present invention. For example, several sensing points 750, 752, 754 and 756 are disposed within the capture field 401 for the image pickup device 408. If the change of the surrounding's light is over a predetermined signal level, it is meant that the background noise is increased. Under this circumstance, the gain values may be automatically adjusted and thus the influence of the light emitted from the image projecting system 404 is minimized. Alternatively, the predetermined signal levels of the sensing points 750, 752, 754 and 756 are different. Moreover, the number, size and location of the sensing points 750, 752, 754 and 756 may be varied as required.
Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8743212Dec 2, 2009Jun 3, 2014Thomson LicensingOptimizing content calibration for home theatersUS20120293557 *Sep 20, 2011Nov 22, 2012Chip Goal Electronics Corporation, R.O.C.Object tracking apparatus, interactive image display system using object tracking apparatus, and methods thereofUS20140145948 *Mar 8, 2013May 29, 2014Everest Display Inc.Interactive projection system and method for calibrating position of light point thereofEP2332029A1 *Sep 28, 2009Jun 15, 2011Smart Technologies ULCTouch-input system calibrationWO2011068490A1 *Dec 2, 2009Jun 9, 2011Thomson LicensingOptimizing content calibration for home theaters* Cited by examinerClassifications U.S. Classification345/157International ClassificationG09G5/08Cooperative ClassificationG06F3/0418, G06F3/0386European ClassificationG06F3/038L, G06F3/041T2Legal EventsDateCodeEventDescriptionApr 16, 2014FPAYFee paymentYear of fee payment: 4May 25, 2007ASAssignmentOwner name: MICRO-NITS CO., LTD., TAIWANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TING, LI-WEN;REEL/FRAME:019346/0685Effective date: 20070515RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google