Patent Publication Number: US-2011050652-A1

Title: Flexible and portable multiple passive writing instruments detection system

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     Not Applicable 
     STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
     Not Applicable 
     REFERENCE TO SEQUENCE LISTING, A TABLE OR A COMPUTER PROGRAM LISTING COMPACT DISC APPENDIX 
     Not Applicable 
     BACKGROUND OF THE INVENTION 
     Digitize hand writing on surface is a huge effort all the time. Many inventions aim on this function for many years already, but all have similar kind of disadvantages. Most of the inventions are based on technologies of “Inferred light with IR transmitter and receiver”, “Ultrasonic transmitter and receiver”, “CMOS light sensor with IR transmitter”, “touch screen”, “camera” and “electromagnetic” etc. A specially designed electronics pen is necessary in most of the cases. A high cost touch screen or magnetic panel may be used. There are limited number of products can detect “Passive writing instrument” like human finger writing. Quite often, more than one method are used in one system. All the existing products are designed for fixed writing panel format like whiteboard. Some touch panels are installed in front of display unit to form an interactive writing panel. All these methods have some or all typical disadvantages as follows: (1) High cost (2) Specially designed writing instrument (3) High cost touch screen for passive writing instrument, like finger access (4) Cannot use any pen a-like writing instruments (5) Difficult or cannot truly identify multiple writing instruments which are writing at the same time(6) Difficult or cannot works with existing projector screen (7) Fixed installation and NOT portable. (8) Sensors are installed at corner of writing area and reduced the effective writing area (9) Cannot identify true multiple “Passive writing instrument” like multiple fingers writing (10) Do not identify multiple writing instruments but only electronic pens (11) Cannot use true different writing instruments at the same time, for example: finger, chopstick, regular whiteboard pen at the same time. U.S. Pat. No. 6,266,051 is a retrofittable system which can be mounted to existing writing area for easier mounting but not true portable. (12) Linear distortion of wide angle lens as well as pin hole. U.S. Pat. No. 6,917,033 use optical marker to identify “Passive touch” but is a costly, cumbersome and not easy to implement. U.S. Pat. No. 6,803,906 use CMOS cameras to detect “Passive touch” but is not a flexible and portable system as well as use “substantially rectangular frame” around the writing area to help for detection. All these disadvantages limited the function of “multiple passive writing instrument” detection, “flexibilities”, “portability” and “affordability”. These limitations drawn forward my new invention. 
     BRIEF SUMMARY OF THE INVENTION 
     The new invention is using mature and existing image capture technologies to detect writing objects. Image sensors are installed in a box which will be placed at any edge or corner of the writing area which is not necessary to be traditional rectangular and the invented system is flexible to be adjusted to match with any size of writing area to form a true flexible and portable system which prior arts do not provide. Mirrors are installed at corner of the writing area to project images back to the image sensors to fully utilize the writing area which prior arts do not provide as well as other advantages. A special designed, very cost effective “Passive” method is used to identify passive writing instruments in any form which prior arts cannot provide at all. The newly invented system is very cost effective which prior arts are not. The new invention solved all or most of the disadvantages described above in section [ 0005 ] especially “affordability”. 
    
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
         FIG. 1 : depicts an invented system which is including image sensors, minors, light shade, IR filters, batteries, electronics board including micro-controller, rod, rod holder, hooks and mirror holder. 
         FIG. 2 : depicts more detail sample view of the invented system which is including components in  FIG. 1 . Specially designed light shade position and shape which has wide angle of looking toward to monitored writing area. The position of IR filters and the holder of rods. 
         FIG. 3 : depicts details of the special designed minor which is special designed curved minor to cover any angle of view by changing the curved shape of the mirror. The mirror holder is specially designed to have recessed mirror to reduce the effect of ambient light. 
         FIG. 4 : depicts details of the relationship between the invented system and writing area to show how the image sensors look at the monitored area and the relationship between the system and the writing object and the trigonometry relationship for writing instrument location calculation. 
         FIG. 5  and  FIG. 6 : depicts details of the disadvantages of pin hole camera or image sensor and regular camera or image sensor which are installed at the corner or side of writing area. Both camera have minimum distance for taking image as well as other disadvantages shown. 
         FIG. 7 : depicts details of the disadvantages of pin hole camera and regular camera to be installed at the corner of writing area. These disadvantages are: (1) Difficult to have 90 degrees or larger of field of view to cover the whole area. (2) Minimum object distance reduced and limited the effective monitored area. (3) The writing instrument blocked the field of view of camera or image sensor when the object is close to the camera because the pinhole and the lens are small. (4) The object reduced the viewed area very much even not too close to the camera or image sensor because of the size of pinhole and lens are much smaller in compare with the writing instrument as well as a finger. (5) Linear distortion of wide angle lens as well as pin hole. 
         FIG. 8 : depicts details of the difference between the invented system and the camera which is to be installed at the corner of the writing area. The writing object is much smaller than the minor, thus the blocked area (shadow area) created by the writing object is much smaller in compare with the pin hole and regular camera which are installed at the corner. Detection of multiple objects will be much more easier and accurate. Minimum distance is moved away from the corner, thus eliminate the minimum distance limit and fully utilize the monitored area. When the object comes in front of the minor, the object only blocks part of the mirror and thus the camera still can see the area around the writing instrument also. 
         FIG. 9 : depicts the usage of the invented system with curved surface. The figure showed two invented systems are used, however it can be more than one invented system and depends on the shape of the curved area. 
         FIG. 10 : depicts the details of the label/object which are used to identify writing instruments including finger. Strip or strips are on the label/object to help to be identified, hence different writing instruments can be identified as well as different fingers. Any materials can be used as long as the characteristic difference between the strip and label body is good enough for identification by camera or image sensor. 
         FIG. 11 : depicts the details of the invented system to works with projection system, computer system, PDA and other electronics systems via wired and/or wireless communication links. 
         FIG. 12 : depicts the details of the “Laser calibration sub-system” which is for system setup purpose. The laser sub-system help to setup and adjust the mirrors to project a correct image to the image sensors. 
         FIG. 13 : depicts the details of the “LED light sub-system” which includes a LEDs array and a light diffuser which will diffuse the light of LEDs to form a patterned diffused light source to light up the monitored area and writing objects. 
         FIG. 14 : depicts the concept of the “Light reflector” which is using to enhance the laser light beam spot pick up if necessary. The special designed surface helps to reflect more light to sideway, means more reflection light to camera or image sensors. 
     
    
    
     DETAIL DESCRIPTION OF THE INVENTION 
     According to an aspect of an embodiment of the invention, there is provided a system including: image sensors which is installed in a case with no, fixed or special relationship in between sensors. At least one system can be mounted on any writing area or apparatus at one side or more than one side of the monitored area. It can be any one side or corner but not fixed to one particular side or corner. The image sensors monitor the area through two mirrors. The specially designed light shades and the filters reduce the effect of ambient light to improve image detection result. The system is powered by batteries with charging capability by external power as well as using external power for extended time usage, thus can be a portable or permanent system and can be installed on any surface, for example: white board, wall, rough wall, note pad and projector screen etc. There are two rods which are used to hold the mirrors at the ends by minor holders, the mirror holder is adjustable to match with different size of writing area. These rods are inserted into rod holders on the case to form a complete system. These rods are changeable as well as telescopic rods as alternative, thus the system can be a portable system works with different size monitored area but not only fixed size area and is fully flexible. There are mounting hooks to hold the rods, then the system will be hung on the hooks on top of the monitored area or by any other mounting method to be held at any position. The image sensors monitor the writing area through the reflection minors and pass the information to processing unit to calculate the location of the writing instrument which can be any passive writing instruments like finger, pen or a-like, even laser pointer light beam, by simple trigonometry method. The calculated data then can be processed by either the processing unit, other processing unit, personal computer or any internal or external electronics devices to perform any actions. For example: These data can be sent to projector to update the projected image on screen to synchronize the projected image and the writing with or without personal computer system or other electronics systems. Identify passive writing instruments like finger for writing and multiple passive writing instruments writing on the writing area at the same time is always be a huge challenge so far. For example: If the first finger stay on board and the second finger pop in to do zoom in and out function or rotating function are simple and is using by many manufacturers right now, for example: iPhone. But, if the first finger writes some thing at left side of board then left up, then second finger pop in and writes some thing at right side of board, then all the known prior arts cannot identify the second writing is written by first finger or second finger unless by electronics pen method. By using the invented label/object passive method, the said problem vanished and indentify multiple passive writing instruments at low cost are possible. The label/object is specially designed label/object which helps to identify passive writing instruments like finger, pen etc. as well as multiple passive writing instruments by this passive method, thus fingers of different persons can be truly identified as well as different writing instruments. The label/object for identification can be any form, for example: in label form, ring form, sleeve form etc. So, different writing instruments can be identified and write on the area and can be truly identified by the system at much lower cost with this specially designed passive identification method, and is practical. U.S. Pat. No. 6,917,033 with title “Passive touch-sensitive optical marker” is a way to identify passive writing instrument by using optical reflector. However, this method is expensive, difficult to manufacture, cannot identify multiple passive writing instruments and not easy to use in compare with this invention. The image sensors capture images via the reflection mirrors and identify the related angle and then calculate the intersection of two projection lines to locate the writing instrument location by the invented passive identification label/object and further calculate the movement. The image sensors identify the writing object by reading the attached label/object which have different quantity of special shape including strip etc. with different material. The shape or strip give different image in the form of different thermo and/or brightness characteristic to the image sensors because of different temperature coefficient than the label/object body as well as different light reflection factor than the label/object body and writing instrument. These temperature and light reflection differences can be detected easily by image sensing with image sensors. The strips or shapes on the identification label/object can be any type of materials as long as the combination of shape/strip and label/object body have different characteristics to be detected by image sensors. On the other hand, any materials combinations will work as long as can be identified by the image sensors. A subsystem of LEDs array and diffusion filter assembly can be installed with the system to provide additional heat and/or light if necessary. The light source can be any type of light but not limited to LEDs and the filter is used to diffuse the light to form a designed patterned light source as needed. A subsystem of laser pointer and receiver, calibration mirrors with optional image display also mounted with the main system to help to calibrate the system when installation. The invented system corrected disadvantages of prior arts as follows: (1) “High cost”—No more high cost, the invented system is using mature existing image sensing technologies which is very cost effective in these days as well as passive multiple writing instruments detection instead of high cost electronics pen method. (2) “Specially designed pens only”—No more high cost special designed electronics pens. Using image recognition method to identify writing instrument. (3) “Limited usage of finger”—The invented system do not have any limitation of using finger. (4) “Regular pen or electronics pen only”—The invented system can use any pen a-like objects as writing instrument, and do not limit to pen size too. (5) “Difficult to identify multiple writing instruments”—Truly identify multiple writing instruments including fingers by low cost passive identification method. (6) “Difficult to work with projector screen”—Any pen alike writing instruments can work with projector screen. No electronic pen is necessary because of using image recognition method. (7) “Not portable”—The invented system is designed to be adjustable and flexible to match with any size writing area as well as to be removable to act as portable system. (8) “Not fully use of writing area”—The invented system is using mirror to capture image and fully utilize the writing area. (9) “Cannot identify multiple fingers”—The invented system is using specially designed low cost passive identification method to truly identify multiple writing instruments including fingers. (10) “Cannot use multiple different writing instruments at the same time”—The identification method of the invented system is neutral to any writing instruments, thus different writing instruments can be used and identified at same time. For example, a finger, chopstick and a fork can be writing instrument and can be identified and using for writing at the same time. Additional advantage is the specially designed label/object can be “Disposable” to meet hygiene concern. Patent No. 7236162/6803906 with title “Passive touch system and method of detecting user input” are using camera to do image identification and is similar to this invention. However, these 2 patents have similar problems as listed above in section [0005], the systems are not flexible and are not portable as well as cannot identify multiple passive writing instruments. So, this invention is better, lower cost and more practical than the said 4 patents, U.S. Pat. No. 7,236,162, 6,803,906, 6,917,033 and 6,266,051, but not limited to.