Abstract:
This discloses the art which can significant improve operability by automatically discriminating a plurality of image orientations which are not assured of always being fed in common orientations and reduce possible burdens to operators by eliminating efforts required to arrange the images in a common orientation before feeding or to correct each orientations into a common orientation after feeding.  
     This also discloses the art which can improve the operability also by enabling modes in which orientation discrimination as well as tilt corrections can be performed before operator&#39;s instructions, if the Auto mode has been specified for the orientation recognition function.  
     This also discloses the art which can improve accuracy of processing by determining whether orientations or tilts recognition is proper and providing the result of the judgements to operators.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    The present invention relates to discriminating orientation of an input image.  
           [0003]    2. Related Background Art  
           [0004]    In recent years, document digitizing devices like character recognition devices or facsimile apparatuses have been developed to be in a realization as computer technology has higher performance. With the wide spread of these devices, principally in business applications, demand has been growing for converting vast amounts of documents to image data at a time with a scanner having an auto document feeder (ADF). For that purpose, efficiency is attracting the most attention from industry.  
           [0005]    Typically, ADFs are fed with numerous documents of the same size and orientation. If documents in longitudinal direction and lateral direction are mixed, obtained image data will have different orientations. If some documents have different sizes, such documents may tend to rotate in travelling in an ADF and obtained image data will be more likely to have a wrong orientation.  
           [0006]    Since a conventional character recognition device or a filing device assumes that documents is processed in a common orientation, an operator arranges their orientations to be in a common orientation before the documents are fed into such a device. Moreover, in order to ensure proper operations, after documents have been processed by a device, an operator must visually inspect all document images on a display one after another and if they have wrong orientations, the operator must provide an instruction to arrange it by an appropriate angle (90°, 180° or 270°) accordingly. Therefore, processing vast amounts of documents at a time may significantly degrade practicability of the devices in the point of light efficiency and burden to operators.  
         SUMMARY OF THE INVENTION  
         [0007]    The purpose of the present invention is to reduce operators&#39; burdens and improve efficiency by automatically discriminating orientations of a plurality of images which are not always fed in a common orientation.  
           [0008]    Another purpose of the present invention is to reduce possible burdens to operator and improve efficiency by automatically discriminating orientation of each of a plurality of images which may have different orientations.  
           [0009]    Furthermore, another purpose of the present invention is to improve accuracy of processing by judging if results of automatic discrimination are not normal and informing exteriors of the results. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0010]    [0010]FIG. 1 is a functional block diagram according to the present invention.  
         [0011]    [0011]FIG. 2 is a flow chart for showing cooperation between a processes of a document image orientation correction part  4  and a correction instruction part  6 .  
         [0012]    [0012]FIG. 3 shows an example of an input window for document image orientation correction instructions.  
         [0013]    [0013]FIG. 4 shows a hardware configuration of devices according to the present invention.  
         [0014]    [0014]FIG. 5 is a flow chart for showing a process of a correction instruction part.  
         [0015]    [0015]FIG. 6 is a flow chart for showing a process when an orientation correction is in Auto mode. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0016]    [0016]FIG. 4 shows a hardware configuration of a device according to the present invention.  
         [0017]    In FIG. 4, the numeral  401  shows a central processing unit (CPU), which controls processes according to the present invention by executing control program stored in memory  402 . Steps shown in a flow chart described below are also executed by CPU  401 . Memory  402  is a storage device consisting of RAMs, ROMs, and hard disks, in which control programs for the CPU  401  and various parameters, input image data, and character recognition dictionaries are stored. The numeral  403  shows an external storage medium which can be attached to and removed from the present device, such as optical disks, magnetic disks, magneto-optical disks, and magnetic tapes. Programs and data to be stored in memory  402  are read out from this external storage medium. Any result of processes is provided to external storage medium  403 . The numeral  404  shows a communication I/F for communicating data to other terminals through a network or public circuit and programs and data to be stored in memory  402  and the process result may be provided through this communication I/F. The numeral  405  shows an input means such as a keyboard or pointing device, which is used to transmit the operator&#39;s instructions. Instructions to read images, to start character recognition, and to specify destination where text data resulting from a recognition process is stored are entered through input means  405 . The numeral  406  shows a scanner which optically reads a document and provides a document image as electrical signals to the present device. The numeral  407  shows a display such as a CRT or liquid crystal display, which is used to display text data resulting from process as well as to be an interface for an operator. The numeral  408  shows a laser beam printer (LBP) or an ink jet printer, which is used to print out text data resulting from process in desired fonts.  
         [0018]    [0018]FIG. 1 is a functional block diagram according to the present invention.  
         [0019]    In FIG. 1, reference numeral  2  shows an input part for entering document images, which are fed into the present device from scanner  406  or external storage medium  403  or from other terminals through communication I/F  404 . The numeral  4  shows a document image orientation correction part for correcting the orientation of an input document image,  6  is a correction instruction part,  8  is an area discrimination part for document images, and  10  is a character recognition part for recognizing characters, all of which may be implemented by CPU  401  as shown in the flow chart described below. The numeral  12  shows an output part for providing recognition results to display  407 , printer  408 , or external storage medium  403 , or to other terminals through communication I/F  404 . The numeral  14  shows a document image memory which may be implemented by memory  402 .  
         [0020]    An operation of devices according to the present invention is now described below.  
         [0021]    A document is first converted to a predetermined image through input part  2  and stored in document image memory  14 , regardless of mediums which contains the documents. Acquired document images are provided to document image orientation correction part  4  through document image memory  14  and whether any correction is required or not depends on an instruction from correction instruction part  6 . If correction instruction part  6  does not indicate an instruction to correct the image, document image orientation correction part  4  does not perform any correction and then entered image of orientation is provided to area discrimination part  8 . An operation of document image orientation correction part  4  and correction instruction part  6  will be described later in detail. Results of image correction performed by document image orientation correction part  4  is again accumulated in document image memory  14 . Area discrimination part  8  analyzes images to divide it into subareas, depending its attribute such as character, table, drawing, image, and photograph, and then provides to memory  402  block data for each subarea which consists of the number assigned to area in order of area acquisition or closeness to a reference position in the coordinate system, the attribute, the size, and the position information. The block data is stored in memory  402 . Image data within in a subarea having the attribute “character” which is determined to be a character area by area discrimination part  8  is transmitted to character recognition part  10  for character recognition. Text data resulting from the character recognition is transmitted to output part  12  and then provided as an output through display  407  or printer  408  in fonts corresponding to graphic character codes for the text data. During outputting the text, layout of an original document may be reproduced by arranging appropriate fonts within a character area where the text data was recognized based on the position information for the area contained in the block data. Alternatively, the text data resulting from the character recognition or both text and block data may be stored in external storage medium  403  or provided to other terminals through the communication I/F.  
         [0022]    Now, document image orientation correction part  4  and correction instruction part  6  will be described below in detail.  
         [0023]    [0023]FIG. 2 is a flow chart for showing cooperation between processes of document image orientation correction part  4  and correction instruction part  6 .  
         [0024]    Step S 202  is a routine to input a document image orientation correction instruction, wherein an operator uses input means  405  to specify a desired mode. The mode specified in this step, that is, the instruction as to whether the document image orientation should be automatically corrected is stored in memory  402 . FIG. 3 shows an example of instruction input window actually displayed on display  407 . When the checkbox for the instruction “Auto” ( 300 ) in the group “Rotation” on this window is marked, automatic orientation correction will be performed by document image orientation correction part  4 . When the checkbox for the instruction “None” is marked, the automatic orientation correction will not be performed. It should be appreciated that marking of such an instruction checkbox is a signaling image to allow the operator to visually know whether instruction is selected and that the selection can be effected by placing a mouse cursor on instruction checkboxes and clicking a mouse button on it. For each of the groups “Rotation”, “Black-and-White Inversion”, “Tilt Correction”, “Insert Page”, and “Input Image”, only one checkbox can be marked and if any one of the checkboxes within a group has been marked as described above, the other checkboxes within the group can not be marked, that is, the instructions associated with those checkboxes can not be selected. The selected instruction within each group is stored in memory  402  as flag data associated with the group data. Such flag data will be updated every time the mouse button is clicked. Even if the instruction “Auto” has been selected for the group “Rotation”, the instruction “Auto” should be deselected when any of the other instructions “None”, “90° CW”, “180°” and “90° CCW” is selected. All the instructions selected on the instruction input window shown in FIG. 3 are stored in memory  402  as setting data for image input.  
         [0025]    Step S 204  is a routine to actually input a document, wherein, an instruction to read the document is issued to the driver for a scanner, if used, or document data read from a file is converted to a predetermined image and in either case, document data is stored in document image memory  14  in an expanded form.  
         [0026]    Step S 206  is a routine to decide subsequent process, depending on the instructions selected in step S 202 , wherein if it is determined from the setting data for image input stored in memory  402  that the document image orientation should be automatically corrected, the process proceeds with step S 208 . If it is determined in step S 206  that the instruction “None” has been specified for rotation, the control is passed to step S 212 . If the instruction “90° CW”, “180°” or “90° CCW” has been specified, document image processed in step S 204  is rotated as specified and then control is passed to step S 212 . Rotation process is identical to that for step S 210 .  
         [0027]    Step S 208  is a routine to recognize orientation of document images based on the data stored in memory in step S 204 . In this step, character areas in document image are extracted for each language and then character patterns contained in each character area are rotated in the directions of 0°, 90°, 180°, and 270° so that a predetermined number of character patterns can be recognized by using appropriate dictionaries or in an appropriate manner to that language. From the recognition results, that of highest average similarity among the four directions is determined to be a reference orientation. For this purpose, an algorithm for determining the angle for portrait format orientation (0°, 90°, 180° or 270°) is assumed (see Japanese Patent Application No. 10-147620).  
         [0028]    Step S 209  is a routine wherein from the number of characters detected in the document image during character pattern extraction performed in step S 208  as a pre-process for recognizing document image orientation, it is determined whether orientation recognized in step S 208  is correct. Such a determination is accomplished by comparing the number of characters in the document image with a predetermined value previously stored in memory  402 . If the number of characters in document image exceeds the predetermined value and therefore, it is determined that the orientation is correct, the process proceeds with step S 210 . Otherwise, that is, if the number of characters in the document image is less than the predetermined value, the control is passed to the step S 211 . This routine detects beforehand any document image containing much fewer characters, which may make it difficult to recognize the document image orientation.  
         [0029]    Step S 210  is a routine to rotate the document image stored in document image memory  14  to the reference orientation and to store corrected document image in document image memory  14  again on the basis of a rotation angle obtained at step S 208 .  
         [0030]    Step S 211  is a routine to give an alarm to an operator if it is determined in step S 209  that the orientation discrimination for document image is not normal. As such an alarm, messages or signals is provided through display  407  or printer  408  or through a speaker (not shown), indicating that the document image orientation can not be automatically corrected.  
         [0031]    Alternatively, in step S 211 , data which indicates that document image orientation could not be automatically corrected may be stored in memory  402  in correspondence to document images or data for the document image which could not be automatically corrected can be provided from the output part for further determination in response to the instructions specified by the operator through keyboard  405  during output process of character recognition results.  
         [0032]    Step S 212  is a routine wherein area discrimination part  8  discriminates area of the document image stored in document image memory  14 . This also includes a pre-processing routine for area discrimination. Since area discrimination of this step is performed on image stored in document image memory  14 , that image would have been rotated if an original document image has been rotated in step S 210 .  
         [0033]    Step S 214  is a routine to recognize character patterns contained in any character area of document image data stored in document image memory  14  according to result of area discrimination performed in step S 212  through character recognition part  10 .  
         [0034]    Step S 216  is a routine to convert text data resulting from the character discrimination of the step S 214  to a predetermined form so that the data can be supplied as an output through output part  12 .  
         [0035]    As described above, according to the present invention, any document image can be automatically corrected in correct orientation during document reading by incorporating the novel document orientation recognition technique into a character recognition device through a user-friendly interface. Therefore, efforts required for confirmation before and after reading documents can be greatly reduced and a series of processes from document reading to the recognized character output can be performed at high precision.  
         [0036]    In routine S 209  described above, reliability of any orientation recognition result is determined based on the number of characters contained in the document. It should be appreciated, however, that this determination is not limited to the discrimination described for this embodiment and that if the maximum similarity resulting from the character recognition performed during the orientation recognition is less than a predetermined value (a reference value previously stored in memory  402 ), it may be determined that a result of orientation recognition has a lower reliability and a process may proceed with step S 211  to give an alarm indicating that the correction could not be performed.  
         [0037]    In routine S 211  described above, an alarm is given if it is determined that any result of orientation recognition has a lower reliability. It should be appreciated, however, that the alarm is not limited to that described for this embodiment and that for example, after process ends in step S 216 , an image file containing the original document image stored in document image memory  14 , text data resulting from the character recognition, and block data may be stored together with an identification index to indicate that orientation could not be recognized.  
         [0038]    Now, process for the case where the “Auto” mode is specified for the “Rotation” function will be described below.  
         [0039]    [0039]FIG. 5 shows the process performed by correction instruction part  6 . The term “orientation correction” used hereinafter means automatic correction to be performed when an input document image is tilted at the angle of 90°, 180° or 270°, while the term “tilt correction” means another automatic correction to be performed when an input document image is tilted at a certain angle within ±45°.  
         [0040]    Step S 502  is a routine to access an input window for document image orientation correction instructions and to prompt an operator to input desired instructions. The operator will specify desired modes of operation by marking appropriate checkboxes on the instruction input window as shown in FIG. 3.  
         [0041]    Step S 504  is a routine to determine whether an orientation correction instruction has been specified in the instruction input window invoked in step S 502 , that is, the “Auto” mode has been selected for the “Rotation” function as shown in FIG. 3. If the “Auto” mode has been selected, process proceeds with step S 506 . Otherwise, control is passed to step S 514  labelled “Other Processes”.  
         [0042]    Step S 506  is a routine to turn on orientation correction flags in memory  402  when it is determined that orientations should be corrected.  
         [0043]    Step S 508  is a routine to mark the checkboxes for the instruction “Auto” for the “Rotation” function in the window as shown in FIG. 3 in response to turn-on operation of step S 506 .  
         [0044]    Step S 510  is a routine to turn on also tilt correction flags when it is determined in step S 504  that orientations should be corrected. In other words, if the “Auto” mode has been selected for the “Rotation” function, the “Tilt Correction” function will be automatically turned on without a specific instruction from the operator.  
         [0045]    Step S 512  is a routine to mark the checkboxes for the instruction “ON” for the “Tilt Correction” function in the window as shown in FIG. 3 in response to the turn-on operation of step S 510 .  
         [0046]    Step S 514  is a routine to perform other processes required to finish instruction inputs.  
         [0047]    [0047]FIG. 6 is a flow chart for showing cooperation between processes performed by the correction control part, the document image orientation correction part, and the document image tilt correction part, respectively.  
         [0048]    Step S 602  is a routine to read out orientation correction flags and tilt correction flags from the memory, both of which have been specified through the correction instruction input part.  
         [0049]    Step S 604  is a routine to actually input a document, wherein an instruction to read documents is issued to a driver for a scanner, or document data read from a file is converted to a predetermined image and in either case, the document data of the input document is stored in memory  402  in an expanded form.  
         [0050]    Step S 606  is a routine to determine whether the orientation correction flag read out in step S 602  is ON or OFF. If the flag is ON, control is passed to step S 608  labelled “Document image orientation correction”, and if the flag is OFF, process proceeds with step S 610 .  
         [0051]    Step S 608  is a routine to actually correct document image orientations. In this step, character areas in the document images are extracted for each language and then character patterns contained in each character area are rotated in directions of 0°, 90°, 180° and 270° for character recognition. From the recognition results, highest average similarity among the four directions is determined to be a reference orientation and then direction of reference orientation (0°, 90°, 180° or 270°) is provided to correct the document image orientations accordingly.  
         [0052]    Step S 612  is a routine to determine whether the document orientation has been corrected properly. The control is passed to step S 622  labelled “Alarm unable orientation correction” if the correction was not proper, or to step S 614  labelled “Document image tilt correction” if the correction was proper. In this step, accuracy for angle detection is determined based on the number of characters detected in the documents. That is, it will be determined in step S 612  that correction was proper if the number of document images having a similarity more than a certain threshold is equal to or greater than a predetermined value after the orientation has been corrected in step S 608 . It should be appreciated, however, that the determination as to whether the correction was proper is not limited to that described for this embodiment and that it may be determined proper if maximum average similarity among those obtained for the four angles in step S 608  exceeds a predetermined value. Such predetermined values used for determination have been previously stored in memory  402 .  
         [0053]    Step S 610  is a routine to determine whether tilt correction flags read out in step S 602  is ON or OFF. The control is passed to step S 614  labelled “Document image tilt correction” if the flag is ON, or to the step S 626  labelled “Area discrimination” if the flag is OFF.  
         [0054]    Step S 614  is a routine to actually correct the document image tilt. This correction may be accomplished by using any well-known method.  
         [0055]    Step S 616  is a routine to determine whether the document tilt has been corrected properly. The control is passed to step S 618  labelled “Alarm unable tilt correction” if the correction was not proper, or to step S 626  labelled “Area discrimination” if correction was proper.  
         [0056]    Step S 618  is a routine to activate an alarm window on display  407  so that the operator can know that the tilt correction was not proper.  
         [0057]    Step S 620  is a routine to forcefully terminate the process, for example, by closing the alarm window.  
         [0058]    Step S 622  is a routine to activate an alarm window on display  407  so that an operator can know that orientation corrections were not proper.  
         [0059]    Step S 624  is a routine to forcefully terminate process, for example, by closing the alarm window.  
         [0060]    Step S 626  is a routine to discriminate any document image area of the data stored in the document image memory. This also includes a preprocessing routine for area discrimination.  
         [0061]    Step S 628  is a routine to recognize characters based on the area division result of step S 626  and document image data stored in the memory.  
         [0062]    Step S 630  is a routine to convert character recognition results of step S 628  to a predetermined form.  
         [0063]    It should be appreciated that if it is determined in step S 612  or S 616  that corrections were not proper, an identification index to indicate the determination may be stored together with the image file. In this case, an alarm required in step S 622  or S 618  may be omitted.