Abstract:
A desk top document scanning system in a multi-usage environment executes scanning over a field of interest and forwards results of the scanning selectively to a selected one of a plurality of scan data usage applications. In particular, the usage application is determined by detecting a substantially steady non-pointing first manual gesture by a user which gesture is presented at the field of interest. The system may use biometrical techniques to recognize the user from the dimensions of the hand making the gesture and thereupon further detail the usage application selection. Advantageously, the field of interest may be re-defined by a second manual gesture by a user made in combination with the first manual gesture which second manual gesture is also presented at the field of interest.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]     This nonprovisional application claims priority under 35 U.S.C. § 119(a) on European Patent Application No. 04106116.9, filed on Nov. 26, 2004, the entirety of which is incorporated herein by reference.  
       BACKGROUND OF THE INVENTION  
       [0002]     1. Field of the Invention  
         [0003]     The invention relates to a method of providing a digital document file based on a physical document using a desk top document scanning system. The method includes the steps of scanning over a field of interest and detecting manual gestures made by a user that indicate a usage of scanning results.  
         [0004]     2. Description of Background Art  
         [0005]     U.S. Pat. No. 5,511,148 discloses a feedback feature implemented in a copying environment through effecting a projection on a working plane, whilst effecting a certain manipulation by pointing and tapping with fingers on the working plane. The reference relates to the creating and processing of documents, whereas the present invention is directed to a scanning environment proper that wants to effect selecting among various different fields of usage, and thus selectively forwarding scanned data to a subsystem and/or software application associated to the selected field of usage.  
         [0006]     U.S. Pat. No. 5,732,227 discloses a hand gesture-operated system including a display surface to be used as a desktop, on which document images and so-called “actual objects” may be displayed. Actual objects designate file handling operations, such as file storage, fax processing, keyboard entry, etc. Document images may be dragged to such an object to initiate the associated operation, by an operator using hand gestures over the display surface. However, this background art document does not disclose actual document scanning for obtaining document images. Rather, document images are generated digitally from document files and displayed to facilitate handling of them under hand gesture control. In this respect, the gesture processing is much more similar to the use of a mouse/cursor on a computer screen desktop than to scanner control.  
         [0007]     Furthermore, the present invention recognizes the high worth of intuitive manipulation on an easy-to-understand level that requires little or no critical movements from a user.  
       SUMMARY OF THE INVENTION  
       [0008]     In consequence, amongst other things, it is an object of the present invention to effect such selecting in a straightforward and uncomplicated manner, that would enhance possibilities for using documents and the like presented on a desk top.  
         [0009]     A first aspect of the present invention is directed to a method of providing a digital document file based on a physical document, using a desk top document scanning system. The method comprises the steps of scanning over a field of interest and detecting manual gestures made by a user that indicate a usage of scanning results; detecting in said field of interest a substantially steady non-pointing first manual gesture by a user; determining an intended usage application selection from said gesture; executing a document scanning operation within the field of interest; and forwarding results of said scanning operation to the selected usage application as determined from the gesture.  
         [0010]     In the above method, the gesture is substantially steady, which means that no prescribed motion is necessary to recognize the gesture. The gesture needs not point to a particular spot such as would be the case on a preformatted form, and the operation can thus be used for any document or document-like item, such as text written on an envelope or label. The field of usage does pertain to the usage of the document as such, which may contain text, graphics, images, and other. Generally, the size is such as fitting on a desk top and therefore, rather limited, such as no larger than standard A2, but this particular size is no express limitation.  
         [0011]     In particular, said field of interest may be re-defined by detecting a second manual gesture by a user which second manual gesture is presented at said field of interest. To a certain extent, such detecting can imply both detecting proper and interpreting.  
         [0012]     A second aspect of the present invention relates to a system that is arranged for implementing the method of the first aspect of present invention. The second aspect of the present invention is directed to a desk top document scanning system for operating in combination with a plurality of scan data usage applications. The system comprises a scanning facility for scanning over a field of interest; a detecting facility, connected to said scanning facility, that is arranged for detecting a substantially steady first manual non-pointing gesture by a user, which gesture is presented at said field of interest as representing said usage; a selection determining facility, connected to the detecting facility, for determining a selection of a said usage application, based on said detected gesture; and a forwarding facility for forwarding results of scanning a document placed in the field of interest selectively to a selected one of said usage applications.  
         [0013]     Further scope of applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description. 
     
    
     BRIEF DESCRIPTION OF THE DRAWING  
       [0014]     These and further features, aspects and advantages of the invention will be discussed more in detail hereinafter with reference to the disclosure of preferred embodiments of the invention, and in particular with reference to the appended Figures that illustrate:  
         [0015]      FIG. 1  (including  FIG. 1   a - FIG. 1   d ), a set of gestures to be executed by a user;  
         [0016]      FIG. 2 , a geometrical set-up of a scanning arrangement for use with the invention;  
         [0017]      FIG. 3 , the principal steps for executing the scanning steps without detailing the selection proper;  
         [0018]      FIG. 4 , the system operations as considered on a functional level; and  
         [0019]      FIG. 5 , the system operations as considered from an input perspective. 
     
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS  
       [0020]      FIG. 1  illustrates a set of gestures to be executed by a user.  FIG. 1   a  shows a selecting gesture  10  that is used for selecting a field of interest  11 . In this case, the gesture  10  is made by the extended right hand forefinger  12 , whilst the other fingers are kept folded. As shown, the gesture movement  10  delimits a very rough rectangle. The rectangle is recognized as such in the scanning system, and may then be used for being improved to a “neat” rectangle that circumscribes the finger track, and therewith the area to be used. Alternatively, the rectangle is converted to the most probable area that could be useful for processing, such as a text field as delimited by its white space borders, one or more paragraphs within a text page, a picture as separated from its background, or other. Also, other shapes could be indicated by the gesture, such as a rough circle that could be improved likewise to a “neat” rectangle, or rather, to a “neat” circle or ellipse, depending on the software or the image actually present on the document.  
         [0021]      FIGS. 1   b - 1   d  illustrate action gestures (as opposed to the selecting gesture of  FIG. 1   a ) that are executed by extending one or more selected fingers. In the embodiment, extending the thumb and the first two fingers signals “sending to printer”. Further, extending all fingers signals “sending to email”. Finally, extending only the first two fingers signals “sending to network”. Various other gestures would be feasible, given the number of fingers and their relatively large variations in positioning. In  FIGS. 1   b - 1   d , the gesture is recognized by the software while the hand is substantially steady; it was found that such will generally allow large tolerances in the size, shape and color of the hand.  
         [0022]     As an alternative to the area selection procedure as described in relation with  FIG. 1   a , the selection of the field of interest may also be effected by a hand pose in a substantially steady position. This presents a simple organization, wherein the recognition is effected in two successive tiers. However, it allows less freedom in the selecting of the field of interest, which then could restrict to only a default format, e.g. an upright A4 format. Note that Action gestures are executed within the field of interest, but some extension outside would be well tolerable.  
         [0023]     In a practical implementation, the area is first selected (if appropriate), and thereupon the Action gesture is detected.  
         [0024]     For a multi-page document, the first page is presented, the area is selected, and then a so-called “set” gesture is entered, which is formed by, e.g., four extended fingers. The two hand poses are repeated for every page. Alternatively, the area selection gesture may be left out for the succeeding pages. After the last page has been entered, the Action gesture is presented by the user. In this case, the pages are scanned after the gestures proper. However, a different sequence would well be feasible.  
         [0025]     By itself, recognition of the hand shape proper is well-known to persons skilled in the art. Known methods are, e.g., template matching, contour matching, Eigenface Matching, and neural network application. This aspect is; however, not part of the present invention.  
         [0026]     In a practical embodiment, the camera that is used for the scanning process generates, for instance, 12 images per second. Regarding operating parameters of this embodiment, after selecting the region of interest, at least one image of the following 10 images must be interpretable as an Action command, with a matching score of at least +0.8 in a range from −1 to +1.  
         [0027]     Selecting the region of interest needs to give rise to at least five recognized locations, because such would already be sufficient for interpretation of a rectangular area.  
         [0028]     An action gesture must yield a matching score of 0.8 or up in at least 8 from 10 successive images. The recognition must be relatively secure, because it will start executing a scanning process immediately. This is particularly important in scanning multi-page documents, since additional, erroneous, images in the sequence are annoying. Further, some motion may occur during the detecting. However, the pose itself must remain substantially unchanged. Of course, other parameters would apply to other embodiments, security level wanted, etc.  
         [0029]      FIG. 2  illustrates a preferred geometrical set-up of a scanning arrangement for use with the invention. As shown, the present desk top area scanned is by way of example 48 by 36 centimeters, which will usually be sufficient for most office chores. The scanning facility is realized by a digital camera  28  that is housed in a holder  22 , which may also contain a lighting facility for illuminating the desk top, so that the whole assembly would resemble an office lamp. Furthermore, base element  24  will provide mechanical support, and further contain such power supply, processing facilities, and attachment as necessary to interface to external destinations for the scanned information. The base element also houses a multi-color LED indicator  26  that signals standby (green), scanning (steady red) and transferring (blinking red). Other signalling functions could be useful, but for the invention as envisaged, no full page display was necessary.  
         [0030]     Various alternative camera locations are feasible, such as fixed to or pending from the office ceiling, etc.  
         [0031]      FIG. 3  illustrates the principal steps for executing the scanning steps without detailing the selection proper of the various fields of usage. Here, user  30  presents a document  32  at the scanning area and makes a gesture or a series of gestures, which are detected in step  34 . The system then makes a scan  36 , and by some straightforward processing the image is processed in  38  for transfer to the scan data usage application as indicated in the gestures; namely, applications for e-mail  46 , archiving  44  or printing  42 . For printing  42 , a transform to printable  40  is often necessary.  
         [0032]      FIG. 4  illustrates the system operations as considered on a functional level. After presentation of the document, in block S 50  the system detects the gestures made by the user. In the first place, this leads to a scan command, whereupon the system in block S 52  executes the scanning operation. The scanning produces the scan data which in block S 58  are subjected to automatic pre-processing such as thresholding, edge enhancing, correcting for barrel distortion and contrast enhancement.  
         [0033]     Further to the executing of the gesture, the system finds the location information and calculates the region of interest, S 54 , possibly depending on the manner in which the gesture is executed (cf.  FIG. 1   a , supra). After such determining, in block S 60  as depending on the ROI (Region Of Interest) information, the scan data are subjected to a cropping operation in block S 60 , which will restrict the image to only the region of interest, whilst suppressing margin regions and the like.  
         [0034]     In the third place, if the Action command gesture specifying the selected field of usage is entered by the user after the location information gesture, in block S 56  any necessary post-processing steps dedicated to the selected field of usage are determined. From then on, certain postprocessing steps may follow (cf.  FIG. 3 , supra), which will then be executed in block S 62 . Thereupon, the processed data are delivered to the user (data usage application) in block S 64 .  
         [0035]      FIG. 5  illustrates the system operations as considered from an input perspective. Block  70  represents a streaming camera that repeatedly sends Frames to gesture recognizer block  74 . The latter may in a training phase have received from data base  72  a sequence of Training Gestures. Often, training need only be executed once and for all, so that subsequent other users may immediately start working. Upon subsequently recognizing a gesture, block  74  sends an Event signalization to Input Listener block  82 . The latter has, from further data base  76 , received Event-on-Action-Mapping information and may in consequence signal Actions to be executed to Central Control block  84 . The latter may emit Zoom control signals and Request photo signals to Photo camera  78 . Photo camera  78  may be the same as Camera  70 . Upon being so activated, Camera  78  takes a photo for sending to Scan Pre-processor  80 . In its turn, the Pre-processed Scan information is transferred to Central Control block  84 , which thereupon sends Photos (scan files) to the Action Handler (not shown) selected by the Action signal from block  82 . Further to the above, additional inputs may be given, such as by Pushbutton  86  or Other Facilities  88  such as speech. For clarity, the ultimate handling proper has not been shown in  FIG. 5 .  
         [0036]     In a basic embodiment of the invention, the scanner system is a personal gadget dedicated to one user. In that case, the destinations used by the e-mail and archive applications, and also for printing, may be pre-programmed as the e-mail address of the user and a dedicated directory within the user&#39;s computer system, respectively.  
         [0037]     In a more elaborate embodiment, the scanner system may be a shared appliance in a multi-user environment. In that case, it would be preferable to include a user recognition function in the system. e.g., the scanner might be provided with a reader for, possibly remotely readable, identity cards, such as cards including an RFID tag, or with a device for recognizing biometrical characteristics, such as a fingerprint reader. Such elements could easily be incorporated in the construction of the scanner system, as already mentioned above as the Other Facilities  88 . Also, an identity card may carry a machine-readable code, such as a bar code, and may be presented to the scanner, that can read it and so identify the user.  
         [0038]     Also, and preferably, the system might be able to recognize a user by analysing the biometrical characteristics of the user&#39;s hand as a part the process of analysing the gesture. It is well-known from scientific research that hands of different persons are sufficiently different to enable identification by analysing the dimensions of fingers, phalanges and knuckles, especially in limited groups of people.  
         [0039]     In this embodiment, the system may include a pre-programmed database of users with their identifying data and their preferred scan data destinations, such as an e-mail address and archiving storage locations, or a preferred printer. When a user presents his hand at the scanner field-of-view, or enters his identity data otherwise, he will automatically be recognized and his preferred scan data destination looked-up and applied.  
         [0040]     Of course, a shared scanner may also be connected to a computer standing at its side and implementing a conventional user interface for selecting a destination.  
         [0041]     From the above, it would be clear that the scanning procedures may be executed in various different manners. For example, the scanning proper and the two tiers of gestures may be effected in various different orders, which need not be uniform in a particular application. Furthermore, a single gesture pair may control the processing of a sequence of scans or pages. In such case, the pages are presented after the gestures. The page sequence may be started and terminated by specific gestures. Another specific gesture may be used as an ignore or cancel signal; in particular, the latter may again be a moving gesture. In principle, the number of gestures made by a single hand is relatively large, even while taking into account that various combinations are difficult or impossible for certain persons. Note that in particular the thumb has various distinctive poses possible. The gestures may be made by the right hand alone, or by either left or right hand, both hands then yielding the same or different meanings. In principle, even a two-handed gesture would be feasible, such as a cross. The color of the hand is in particular arbitrary, but some care may have to be taken to distinguish the hand color from the background.  
         [0042]     Now, the present invention has hereabove been disclosed with reference to preferred embodiments thereof. Persons skilled in the art will recognize that numerous modifications and changes may be made thereto without exceeding the scope of the appended Claims. In consequence, the embodiments should be considered as being illustrative, and no restriction should be construed from those embodiments, other than as have been recited in the Claims.