Abstract:
There is provided an image processing apparatus which outputs an image suitable for a character recognition process even if an image containing reversed characters or a multilevel image is input. More specifically, there is provided an image processing apparatus for processing a multilevel image ( 401 ), which includes a unit ( 402 ) for creating an edge image ( 404 ) by extracting edges from the multilevel image ( 401 ), a unit ( 403 ) which creates a binary image ( 405 ) from the multilevel image ( 401 ), a partial binarization unit ( 410, 411 ) which creates an output binary image by identifying a character area based on both of the created edge image ( 404 ) and binary image ( 405 ) and binarizing the character area of the multilevel image ( 401 ), and a unit which outputs an output binary image ( 412 ) created by the binarization means ( 410, 411 ), wherein if a character in a character area identified based on the edge image ( 404 ) is a reversed character, the partial binarization unit ( 410, 411 ) creates the output binary image ( 412 ) by reversing and binarizing the character.

Description:
FIELD OF THE INVENTION  
       [0001]     The present invention relates to an image processing technique for extracting a binary image suitable for a character recognition process from an input image.  
       BACKGROUND OF THE INVENTION  
       [0002]     Conventionally, character recognition techniques of identifying characters contained in input images have been put into practice. For example, a PC can convert an image captured by a scanner into text data by using a character recognition process. When a large number of documents are to be scanned in a copying machine using an ADF, the orientation of each document is determined by using this technique, and a copy can be printed out upon correction of the orientation.  
         [0003]     In performing a character recognition process, it is important to convert an input multilevel image into an image suitable for character recognition before the process. In this case, an image suitable for character recognition is a binary image obtained by extracting only characters from the information contained in an input image while maintaining their sizes, layout, thicknesses, fonts, and the like (i.e., information other than character portions is deleted), and expressing a background portion in white and character portions in black. A binarization method of obtaining such a binary image is disclosed in, for example, Japanese Patent Laid-Open Nos. 08-223409 and 09-305754.  
         [0004]     According to the conventional binarization method, if a reversed character is contained in an input image, a binary image is output without changing the reversed character. As a consequence, the reversed character image portion may not be recognized as a character and hence not be regarded as a target for character recognition. It is very difficult to determine whether or not an image after binarization is a reversed character. If, therefore, a reversed character is contained in a given image, the character recognition precision deteriorates.  
         [0005]     Along with increasing color document images owing to improvements in the throughput of computers, increases in memory capacity, advances in scanners, and the like, there have appeared more images which have low contrast between background colors and character colors and more images which contain both images other than characters, e.g., photos, and characters. As a consequence, in some cases, it is impossible to obtain a binary image suitable for character recognition by only binarization threshold adjustment or block size adjustment. When, for example, a single binarization threshold is set for the entire surface of an image, an image unsusceptible to small image unevenness can be generally obtained. In the case of a color image having a plurality of character colors and character portion background colors, however, a deterioration in image quality occurs. Performing binarization while adaptively determining a threshold for each small block can cope with changes in character portion background color for each processing block. On the other hand, when the same processing block contains both a character area and another kind of area such as a photo area, the resultant image tends to include noise. In addition, as the block size is decreased to prevent each processing block from containing a plurality of areas, the influence of noise in each block increases. This rather increases density unevenness and tends to produce noise.  
         [0006]     As described above, no conventional binarization methods could output any binary image which could realize high character recognition precision in the case of an image containing reversed characters, an image having relatively low contrast between a background color and a character color, e.g., a color image, or and an image containing both a character portion and an image portion other than characters, such as a photo portion.  
       SUMMARY OF THE INVENTION  
       [0007]     The present invention has been made in consideration of the above problems, and has as its object to improve character recognition precision by outputting an image suitable for a character recognition process even when an image containing reversed characters or a multilevel image is input.  
         [0008]     In order to achieve the above object, an image processing apparatus according to the present invention has the following arrangement. Specifically, 
        an image processing apparatus which processes an input image, comprising:     edge image creating unit configured to creating an edge image by extracting an edge of the input image;     binary image creating unit configured to creating a binary image from the input image;     partial binarization unit configured to creating an output binary image by identifying a character area based on both of the created edge image and the created binary image and binarizing the identified character area of the input image; and     output unit configured to outputting an output binary image created by the partial binarization means,     wherein the partial binarization means determines whether a character in a character area identified based on the created edge image is a reversed character, and if it is determined that the character is a reversed character, creates an output binary image by reversing and binarizing the character.        
 
         [0015]     According to the present invention, even if an image containing reversed characters or a multilevel image is input, character recognition precision can be improved by outputting an image suitable for a character recognition process.  
         [0016]     Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0017]     The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.  
         [0018]      FIG. 1  is a block diagram showing the arrangement of an image processing apparatus according to the first embodiment of the present invention;  
         [0019]      FIG. 2  is a flowchart showing the flow of processing in the image processing apparatus according to the first embodiment of the present invention;  
         [0020]      FIG. 3  is a flowchart showing the flow of processing in a partial binarization process in the image processing apparatus according to the first embodiment of the present invention;  
         [0021]      FIG. 4  is a data flow diagram showing the flow of data processed in the image processing apparatus according to the first embodiment of the present invention;  
         [0022]      FIG. 5  is a view showing a concrete example of a multilevel image;  
         [0023]      FIG. 6  is a view showing a concrete example of a multilevel image; and  
         [0024]      FIG. 7  is a view showing a concrete example of a multilevel image. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0025]     Preferred embodiments of the present invention will now be described in detail in accordance with the accompanying drawings.  
         [0026]     Each embodiment of the present invention will be described below with reference to the accompanying drawings.  
         [heading-0027]     [First Embodiment] 
         [heading-0028]     &lt;Arrangement of Image Processing Apparatus&gt; 
         [0029]      FIG. 1  is a view showing the arrangement of an image processing apparatus according to this embodiment. Reference numeral  101  denotes a CPU;  102 , a ROM which stores programs and data for the CPU  101 ;  103 , a RAM functioning as a work area for the CPU  101 , which, for example, temporarily stores a document image to be processed;  104 , a storage device which stores document images, files, and the like;  105 , a scanner which scans a target document image;  106 , a printer which prints out a document image;  107 , a network I/F which delivers a created compressed image; and  108  and  109 , a keyboard and display, respectively, which serve as user I/Fs for operation and display.  
         [heading-0030]     &lt;Flow of Processing in Image Processing Apparatus&gt; 
         [0031]      FIG. 2  is a flowchart showing the flow of processing in the image processing apparatus according to this embodiment.  FIG. 4  is a data flow diagram showing the flow of data processed in accordance with this flowchart.  
         [0032]     In step S 201 , the storage device  104  or scanner  105  loads a multilevel document image as a multilevel image  401  into the RAM  103 . Reference numeral  501  in  FIG. 5  denotes a concrete example of the multilevel image  401 .  
         [0033]     In step S 202 , an edge image  404  is created by extracting image edges from the multilevel image  401  input in step S 201  by an edge extraction process ( 402 ). Reference numeral  502  in  FIG. 5  denotes a concrete example of the edge image  404 .  
         [0034]     In step S 203 , a character area  408  is extracted by performing an area identification process ( 406 ) for the edge image  404 .  
         [0035]     In step S 204 , a binary image  405  is created from the multilevel image  401  by a binarization process ( 403 ). Although an arbitrary binarization technique can be used, a binarization technique which provides high image quality for character portions as disclosed in Japanese Patent Laid-Open Nos. 08-223409 and 09-305754 is preferably used. Reference numeral  503  in  FIG. 5  denotes a concrete example of the binary image  405 .  
         [0036]     In step S 205 , a character area  409  is extracted from the binary image  405  by an area identification process ( 407 ).  
         [0037]     Although the area identification processes ( 406  and  407 ) are expressed as different processes in the data flow, the same process can be used. The relatively small characters contained in the edge image  404  are almost identical to boldface characters as known from the edge image  502 , and relatively large characters are identical in shape to outlined characters. A character area can therefore be extracted by the same area identification method.  
         [0038]     In step S 206 , it is checked whether or not a character area can be extracted. If no character area can be extracted in both steps S 203  and  205 , a binary image  407  created in step S 204  is output as an output result (step S 210 ).  
         [0039]     In step S 207 , an output binary image  412  in the memory is initialized. Although edge extraction and binarization processes are performed for the entire area of the multilevel image  401  in steps S 202  and S 204 , since only character area portions are output in the processes in steps S 208  and S 209 , it is necessary to initialize the entire surface with white pixels.  
         [0040]     In step S 208 , a partial binarization process is performed for the multilevel image  401  with respect to the character area obtained on the basis of an area identification result  408  in step S 203 , and the resultant binary image is output to a storage memory. In step S 209 , a partial binarization process is performed for the multilevel image  401  with respect to the character area obtained on the basis of an area identification result  409  in step S 205 , and the resultant binary image is output to the storage memory.  
         [0041]     In step S 210 , the processing results in steps S 208  and S 209  are output to a memory for the output binary image  412 . Reference numeral  504  in  FIG. 5  denotes a concrete example of the output binary image  412 , which is the result obtained by processing the concrete example  501  of the multilevel image using this method. Since only a character portion is binarized and output as the output binary image  412 , the image can be output in a natural form without any conspicuous boundary portion between a reversed character portion and a non-reversed character portion (If a binary image of the entire area of an image is output as in the prior art, this boundary portion is output in an unnatural form, resulting in adverse effects on a process using the output binary image  412 ).  
         [0042]      FIG. 3  is a flowchart showing the flow of processing in the partial binarization processes in steps S 208  and S 209 .  
         [0043]     In step S 301 , an extracted character area is selected. If it is determined in step S 302  that there is a selected character area, the flow advances to step S 303 . Otherwise, the processing is terminated.  
         [0044]     In step S 303 , a binarization threshold computation process is performed to determine a binarization threshold for the selected character area. Although an arbitrary threshold computation method can be used, a computation method which provides high character image quality is preferably used.  
         [0045]     The flow then advances to step S 304  to check whether the currently processed area is the data of the area identification result  408  or the data of the area identification result  409 . If it is determined that this area is a character area extracted from the data of the area identification result  408 , i.e., the edge image, the flow advances to step S 305 . Otherwise, the flow advances to step S 308 . Step S 305  is a reversed/non-reversed character area determination process, in which it is checked whether the target character area is a non-reversed character area with dark characters against a bright background or a reversed character area with bright characters against a dark background.  
         [0046]     If it is determined in step S 306 , on the basis of the determination result in step S 305 , that the currently processed area is a reversed character area, a reversing binarization process is performed in step S 307  with the threshold determined in step S 303 , and only the character area portion is output to the memory for the output binary image  412 . In contrast, if it is determined in step S 306  that the currently processed area is a non-reversed character area, a general binarization process is performed in step S 308  with the threshold determined in step S 303 , and only the character area portion is output to the memory for the output binary image  412 .  
         [0047]     When the processes in steps S 307  and S 308  are complete, the flow returns to step S 301  to shift to the processing for the next character area.  
         [heading-0048]     &lt;Reverse/Non-reverse Determination Method&gt; 
         [0049]     The reversed/non-reversed character area determination method used in step S 305  will be described below by giving a concrete example. Note that as a reversed/non-reversed character area determination method, any one of the following methods may be used, or a different method may be used.  
         [heading-0050]     (First Reversed/Non-Reversed Character Area Determination Method)  
         [0051]     According to a reverse/non-reverse determination method for character areas, the white portion/black portion ratio of an image obtained by binarizing the selected area with the threshold computed in step S 303  is calculated. If the proportion of the white portion is larger, this area is determined as a non-reversed character area. If the portion of the black portion is larger, it is determined that the area is a reversed character area.  
         [heading-0052]     (Second Reversed/Non-Reversed Character Area Determination Method)  
         [0053]     The white portion/black portion ratio of an image obtained by binarizing the boundary portion of a target character area with the threshold computed in step S 303  is calculated. If the proportion of the white portion is larger, this area is determined as a non-reversed character area. If the portion of the black portion is larger, it is determined that the area is a reversed character area.  
         [heading-0054]     (Third Reversed/Non-Reversed Character Area Determination Method)  
         [0055]     In the binarization threshold computation process in step S 303 , reverse/non-reverse determination is performed depending on the sign of the skew of a histogram by using the method disclosed in Japanese Patent Laid-Open No. 08-223409 or 09-305754. If the sign of the skew value is negative, a non-reversed area is determined. If the sign of the skew value is positive, a reversed area is determined. Using a skew value can increase the processing speed because computation for this value is finished when a binarization threshold is determined.  
         [heading-0056]     (Fourth Reversed/Non-Reversed Character Area Determination Method)  
         [0057]     If the difference between the proportions of white and black portions is smaller than a predetermined value in the reverse/non-reverse determination method (first method) and the absolute value of a skew value is smaller than a predetermined value in the reverse/non-reverse determination method (third method), determination is performed by the reverse/non-reverse determination method (second method).  
         [heading-0058]     (Fifth Reversed/Non-Reversed Character Area Determination Method)  
         [0059]     Since non-reversed characters exist in general documents at a high probability, determination criteria in the first to fourth determination methods are so set as to give priority to “non-reverse”.  
         [heading-0060]     &lt;Order of Partial Binarization Processing&gt; 
         [0061]     In a partial binarization process for a character area, after a character area extracted from an edge image is processed, a character area extracted from the binary image is processed. The processing result obtained afterward remains in an overlapping area. If, however, a character area from an edge image is processed first, an overlapping portion is overwritten by a process for a character area of a subsequent binary image. This makes it possible to prevent a reverse/non-reverse determination error.  
         [heading-0062]     &lt;Features of Processes in Image Processing Apparatus&gt; 
         [0063]     The features of the respective processes in the flowchart shown in  FIG. 4  will be described next.  
         [heading-0064]     A. Advantage of Edge Image  404   
         [0065]     A feature of the image processing apparatus according to this embodiment is that an area identification process is also performed with respect to the edge image  404 . The edge image  404  obtained by the edge extraction process  402  is output without any discrimination between non-reversed characters and reversed characters. This makes it possible to perform an area identification process without any discrimination.  
         [0066]     Reference numeral  601  in  FIG. 6  denotes a portion of an input image;  602 , an edge image of the portion; and  603 , a binary image. Assume that a portion of an input image has a dark background like the portion  601 . In this case, if a binarization process is performed only uniformly for the entire input image, an area like the one denoted by reference numeral  603  may be obtained. As a consequence, this area is determined as an image area in the area identification processing  407 , and hence cannot be extracted as a character area. In contrast to this, since the edge image  404  is obtained based on the relative luminance difference (brightness difference) between adjacent pixels in the edge extraction process  402 , even if an image like the image  601  is input, the edge image  602  can be obtained. As a consequence, a character area can be extracted without being influenced by a background. Therefore, the binary image  412  to be output becomes like an image  604 .  
         [heading-0067]     B. Advantage of Binary Image  405   
         [0068]     Another feature of the image processing apparatus according to this embodiment is that the edge extraction  402  and binarization process  403  are used together to complement each other.  
         [0069]     Edge extraction ( 402 ) is only slightly influenced by the density of a background. When, however, the extraction result is viewed as an image, since edges exist on the peripheral portions of characters, the characters increase in size. For this reason, when, for example, edge extraction is performed for the characters in a table shown as an image  701  in  FIG. 7 , since the ruled lines increase in thickness, the characters tend to come into contact with the ruled lines (see “702”). As a result, it is very difficult to separate characters in contact with other portions, character areas cannot be stably extracted in performing an area identification process. In addition, edge extraction is executed by using a filter (e.g. laplacian filter), and hence may react to a local luminance difference to produce noise, thus causing a problem in character extraction. In contrast to this, in the case shown in  FIG. 7 , the character size in the binary image  405  obtained by the binarization process  403  is maintained, and hence characters and ruled lines do not come into contact with each other as indicated by an image  703 . This makes it possible to easily separate characters (see “704”).  
         [heading-0070]     C. Binarization on Character Area Basis ( 410 ,  411 )  
         [0071]     Still another feature of the image processing apparatus according to this embodiment is that a binarization process (partial binarization process) is performed for the character area  409  extracted from the binary image  403  by calculating a binarization threshold for the multilevel image  401 .  
         [0072]     This feature is provided because the following problem arises when the binarization technique in step S 204  is used alone as described above as a problem in the prior art. When a single threshold is used for an entire surface, binarization can be done without any small noise. On the other hand, when a plurality of character colors and character portion background colors exist in an image, a character portion with poor image quality is generated. In addition, when an input image is divided into blocks (processing blocks), each having a predetermined size, in the form of a lattice, and binarization is performed while a threshold is adaptively determined for each processing block, a change in character portion background color can be coped with for each processing block. On the other hand, density unevenness occurs for each processing block, and when a processing block contains both a character area and another kind of area such as a photo area, noise tends to be generated. If the block size is decreased to prevent each processing block from containing a plurality of areas, the influence of noise or the like increases in each block. As a result, density unevenness increases and noise further tends to be generated.  
         [0073]     In contrast to this, partial binarization processes are performed for character areas in steps S 208  and S 209  to determine a threshold for each character area, thereby coping with a change in character background color for each area and stably computing a threshold from a relatively wide area limited to a character area. That is, since this operation is equivalent to adaptively determining an optimal processing block size, an image having high character image quality with less density unevenness and noise can be obtained.  
         [0074]     As is obvious from the above description, according to this embodiment, an image suitable for character recognition can be obtained by performing edge extraction for an input image even if it contains reversed characters. In addition, performing edge extraction together with a binarization process, the drawback caused when an image is processed by edge extraction can be compensated for. Furthermore, performing a binarization process for a character area obtained as a result of edge extraction or a binarization process can obtain a binary image with an optical block size. This makes it possible to output an image suitable for a character recognition process, thus improving the character recognition precision as compared with the prior art.  
         [heading-0075]     [Second Embodiment] 
         [0076]     The above edge image  404  is not required if the area identification result  408  can be obtained. The binary image  405  becomes unnecessary when the area identification result  409  is obtained. Therefore, a memory is shared among the edge image  404 , binary image  405 , and output binary image  412 , memory saving can be realized.  
         [0077]     Since the processes  402 ,  404 ,  406 , and  408  are independent of the processes  403 ,  405 ,  407 , and  409  in the data flow in  FIG. 4 , the processing speed can be increased by assigning independent resources (memories) to the processes and concurrently performing them.  
         [0078]     A character extraction process dedicated to reversed characters may be performed by creating a reversed image by reversing the binary image  405  of the entire input image and performing an area identification process for the reversed image in addition to the processes in steps S 208  and S 209 . In this case, however, the flow branches from step S 304  to step S 307  in  FIG. 3 .  
         [heading-0079]     [Third Embodiment] 
         [0080]     If the difference between the binarization threshold obtained in step S 303  and the binarization threshold in the binarization process  405  falls within a predetermined range, the time for a binarization process can be shortened by copying the corresponding portion of the held binary image  405  to the output binary image  412 .  
         [heading-0081]     [Other Embodiment] 
         [0082]     The present invention may be applied to a system constituted by a plurality of devices (e.g., a host computer, an interface device, a reader, a printer, and the like) or an apparatus comprising a single device (e.g., a copying machine, a facsimile apparatus, or the like).  
         [0083]     The object of the present invention is realized even by supplying a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and causing the computer (or a CPU or an MPU) of the system or apparatus to read out and execute the program codes stored in the storage medium.  
         [0084]     In this case, the program codes read out from the storage medium realize the functions of the above-described embodiments by themselves, and the storage medium storing the program codes constitutes the present invention.  
         [0085]     As a storage medium for supplying the program codes, a floppy disk, a hard disk, an optical disk, a magnetooptical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.  
         [0086]     The functions of the above-described embodiments are realized not only when the readout program codes are executed by the computer but also when the OS (Operating System) running on the computer performs part or all of actual processing on the basis of the instructions of the program codes.  
         [0087]     The functions of the above-described embodiments are also realized when the program codes read out from the storage medium are written in the memory of a function expansion board inserted into the computer or a function expansion unit connected to the computer, and the CPU of the function expansion board or function expansion unit performs part or all of actual processing on the basis of the instructions of the program codes.  
         [0088]     The present invention is not limited to the above embodiments and various changes and modifications can be made within the spirit and scope of the present invention. Therefore, to apprise the public of the scope of the present invention, the following claims are made.  
       CLAIM OF PRIORITY  
       [0089]     This application claims priority from Japanese Patent Application No. 2003-300049 filed on Aug. 25, 2003, which is hereby incorporated by reference herein.