Image processing apparatus, method, and storage medium

A binary image of an input image is generated, and a character region within the binary image and a region surrounding each character are acquired as character segmentation rectangle information. A thinning process is executed on a region within the binary image which is identified based on the character segmentation rectangle information to acquire a thinned image. An edge detected image of the region identified based on the character segmentation rectangle information is acquired. Whether each character identified based on the character segmentation rectangle information is a character to be separated from a background by the binarization process or not is determined based on a result of a logical AND of the thinned image and the edge detected image.

BACKGROUND OF THE INVENTION

Field of the Invention

The present disclosure generally relates to image processing and, more particularly, to an image processing apparatus, an image processing method, and a storage medium storing a program for determining a character within an image.

Description of the Related Art

In recent years, an increased number of colored documents are made available because of wide spreads of color printers and color scanners, for example. There are more chances to capture and store such documents as electronic files by scanning and transmit them to a third party over the Internet, for example. However, because direct storage of full-color data may impose a large load on an apparatus and lines, such data may need to be compressed to reduce its data amount.

In the past, methods for compressing a color image may include, for example, a method including converting a color image to a binary image having a pseudo-gray scale by using an error diffusion method or the like and compressing the binary image, a method including compressing in JPEG, and a method including converting a color image with 8-bit palette colors for ZIP compression or LZW compression.

According to Japanese Patent Laid-Open No. 2002-077633, a character region contained in an input image is detected, and the detected character region is converted to a binary image, is MMR compressed (binary non-reversible compressed) and is stored in a file along with character color information on characters therein. Furthermore, an image having the character region filled with a surrounding color on the input image is JPEG compressed (non-reversible compressed) by reducing its resolution and is stored in the file as a background image. The file compressed by this compression method may provide the character region in high quality and may contribute to a greater compression rate.

According to Japanese Patent Laid-Open No. 2002-077633, in order to detect a character region, whether each set of black pixels in a binary image acquired by binarizing an input image possibly corresponds to a character is determined based on the size (width or height) of the set of black pixels and whether sets of black pixels having an approximately equal size exist closely to each other.

On the other hand, application of such a method for performing the region determination based on a binary image as disclosed in Japanese Patent Laid-Open No. 2002-077633 to an input image in which a character and a background are difficult to be separated by simple binarization may result in difficult identification of pixels included in the character. For example, when simple binarization is performed on a black character over a white background (character image having a larger density difference between the character and the background), the background pixels and the character pixels may be separated easily. On the other hand, when binarization is performed on a black character over a dark background (character image having a small density difference between a character and a background), the separation between the background pixels and the character pixels is difficult. Particularly, performing binarization on a character over a high-density background with a threshold value lower than the density of the background may result in a binary character image with characters degraded in black. In this case, when the size of the high-density background region is approximately equal to the size of the character, the binary image in which the background and the character are degraded in black as a result of the binarization may be wrongly determined as a character pixel part. For example, when a document in which a part of a character string is marked with a thick marker pen is scanned and the scanned image is binarized, the entire part marked with a marker pen may sometimes turn black. When the size of the part marked with a marker pen is close to the character size, the whole pixels of the part marked with the marker pen may have a state degraded in black as a result of binarization and may thus be handled as one character. In other words, all black pixels in a region degraded in black as a result of binarization may possibly be handled as pixels of a character.

SUMMARY OF THE INVENTION

According to an aspect of the present disclosure, an image processing apparatus includes a binarizing unit configured to generate a binary image by executing a binarization process on an input image, a first determining unit configured to determine a character region within the binary image, a character segmenting unit configured to acquire a region surrounding each character contained in the character region as character segmentation rectangle information, a thinning unit configured to acquire a thinned image by executing a thinning process on a region within the binary image, the region being identified based on the character segmentation rectangle information, an edge detecting unit configured to acquire an edge detected image by executing an edge detection process on the region identified based on the character segmentation rectangle information, a logical operation unit configured to take a logical AND of the thinned image and the edge detected image, and a second determining unit configured to determine whether each character identified based on the character segmentation rectangle information is a character to be separated from a background by the binarization process or not based on a result of the logical AND performed by the logical operation unit.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the disclosure will be described in detail below with reference to the drawings.

First Exemplary Embodiment

FIG. 1is a schematic view illustrating a system configuration according to a first exemplary embodiment. Referring toFIG. 1, a multifunction peripheral (MFP)101and a computer (hereinafter, called a PC)102are connected over a network103.

Broken lines104and105represent processing flows. The broken line104represents a processing flow to be performed by a user for reading a paper document by using a scanner of the MFP101. In this case, the user is allowed to set a destination (such as the PC102) to which a scanned image is to be transmitted and define settings regarding the scanning and transmission by using a user interface (203inFIG. 2) of the MFP101, which will be described below. As the settings, a user may designate a resolution, a compression rate, a data format (such as JPEG (Joint Photographic Experts Group), TIFF (Tag Image File Format), PDF (Portable Document Format), high compression PDF, and high compression PDF (with an OCR (Optical Character Recognition) result)). In this exemplary embodiment, it is assumed that high compression PDF (with an OCR result) is designated as a data format. The technical details of high compression PDF will be described below. The broken line105represents a processing flow for generating data by using a software or hardware function of the MFP101and transmitting the data to a designated destination based on the designated settings. Here, because an image having a file format such as PDF is transmitted to the PC102, it may be viewed on a general viewer in the PC102.

FIG. 2illustrates a detail configuration of the MFP101. The MFP101includes a scanner unit201that is an image input device, a printer unit202that is an image output device, a control unit204configured to generally control the MFP, and an operating unit203that is a user interface. The control unit204is a controller connecting to the scanner unit201, printer unit202, and operating unit203and connecting to a local area network (LAN)209on the other hand for input/output of image information and device information. A central processing unit (CPU)205is a processor configured to control the entire system. A read-only memory (RAM)206is a system work memory usable by the CPU205for operations and also functions as an image memory for temporarily storing image data. A random access memory (ROM)210is a boot ROM and stores programs such as a system boot program. A storage unit211is a non-volatile storage medium such as a hard disk drive and stores system control software and image data. As used herein, the term “unit” generally refers to any combination of software, firmware, hardware or other component, such as circuitry, that is used to effectuate a purpose.

An operating unit I/F207is an interface unit to the operating unit (UI)203and is used to output image data to be displayed on the operating unit203to the operating unit203. The operating unit I/F207serves to inform the CPU205of information instructed by a user of the image processing apparatus through the operating unit203. A network I/F208connects the image processing apparatus to the LAN209for data input/output (for example, for transmitting a PDF compressed data to another apparatus and receiving PDF compressed data from another apparatus). These units are provided on a system bus216. An image bus interface212connects the system bus216and an image bus217is a bus bridge configured to transfer image data at a high speed and convert a data structure. The image bus217may be a peripheral component interconnect (PCI) bus, IEEE1394, or the like. The units described above are provided on the image bus217. A raster Image processor (RIP)213performs what-is-called rendering processing including analyzing a PDL (page description language) code and decompressing it to a bitmap image having a designated resolution. A device interface (I/F) unit214connects the scanner unit201being an image input device through a signal line218and connects the printer unit202being an image output device through a signal line219and performs synchronization-based/asynchronization-based conversion on image data. A data processing unit215generates PDF compressed data (515) by performing high compression PDF and OCR processing. The generated compressed data (515) is transmitted to a destination (such as the client PC102) through the network I/F208and over the LAN209. The data processing unit215is capable of decompressing compressed data received through the network I/F208and over the LAN209. The decompressed image is transmitted to the printer unit202through the device I/F214and may be printed.

Description of Data Processing Unit215

Next, a configuration of the image compressing processing unit implemented by the data processing unit215inFIG. 2and a configuration of the image decompression processing unit will be described with reference to the block diagrams inFIGS. 5 and 6. The data processing unit215may be configured such that a processor may execute a computer program to cause the data processing unit215to function as corresponding one of the processing units inFIGS. 5 and 6, or a part or all of the processing units may be configured by hardware such as an ASIC (Application-Specific Integrated Circuit) and an electronic circuit.

A high compression PDF process, as described in Japanese Patent Laid-Open No. 2002-077633, determines the type of a given region for each attribute and compresses the data by adaptively selecting binary reversible compression with MMR (Modified Modified Read) and multi-valued non-reversible compression with JPEG in accordance with the attribute of the region. In other words, MMR compression may be performed on a character region, and JPEG compression may be performed on an image having its character region filled with a surrounding color for a greater compression rate and higher quality of the character region. The high compression PDF processing is an effective compression technology for color or black-and-white multi-valued images. According to this exemplary embodiment, whether a region which may be degraded as a result of binarization is a character region or not may be determined, the details of which will be described below. This allows determination of an actual character region only as a subject of MMR compression.

FIG. 5is a block diagram illustrating a configuration of the image compressing processing unit implemented by the data processing unit215and illustrating processing units for compressing an input image to generate a high compression PDF (with an OCR result).

A binarizing unit502generates a binary image from an input image501that is a multi-valued image. In the binary image, a pixel having a density greater than a threshold value in an input image may be a black pixel, for example, while a pixel having an equal or lower density than the threshold value may be a white pixel, for example. (Apparently, such a binarization result may not be represented in black and white but may be represented in other colors or may be represented by 1, 0 or 0, 1 without color). The binarizing unit502performs its processing for the purpose of distinguishing a pixel having a density greater than a threshold value and a pixel having an equal or lower than the threshold value. Other processes than binarization may be applied as far as the same purpose may be achieved (such as ternarization and quaternarization). However, the following description assumes that binarization is performed by the binarizing unit502. Binarization on an input image701may result in a binary image702. When the input image is a polychrome multi-valued image, the binarization is only performed on a luminance (such as Y of YUV) of the multi-valued image.

A region determining unit503detects a character region and a picture image region from a binary image generated by the binarizing unit502. Thus, character regions704and706and a picture image region705are detected, for example. This processing is performed based on a publicly known region determination method (such as Japanese Patent Laid-Open No. 06-068301). The processing may be summarized as follows, for example.

(1) An outline of 8-connected black pixels is traced on the binary image702to extract a black pixel cluster (black cluster of pixels) that is continuously present in any direction of in 8 directions. The expression “8-connected” refers to continuous presence of pixels having a same color (black in this case) in one of eight directions of upper left, left, lower left, lower, lower right, right, upper right, and upper directions. On the other hand, the expression “4-connected” refers to continuous presence of pixels having a same color in any of four directions of left, lower, right, and upper directions.

(2) If there is a black cluster of pixels larger than a predetermined size (such as a black cluster of pixels surrounding a region larger than a predetermined dimension) among the extracted black clusters of pixels, whether any white cluster of pixels exists within the region or not is determined. In other words, a white cluster of pixels is extracted by tracing an outline of 4-connected white pixels within the region. If the extracted white cluster of pixels is larger than a predetermined size, the outline of the black pixel is traced again similarly to extract a black cluster of pixels. These steps are repeated until the cluster of pixels has a size equal to or smaller than the predetermined size.

(3) The acquired black cluster of pixels is classified into a character or a picture based on at least one of its size, shape, and black pixel density. For example, a black cluster of pixels having an aspect ratio close to 1 (or fits into 1 plus or minus α where α is a fixed threshold value such as 0.1) and having a size within a predetermined range (for example, the number of pixels surrounded by the black cluster of pixels is equal to or lower than 100 pixels) is determined as a black cluster of pixels including a character. The other black clusters of pixels are determined as clusters of pixels included in a picture.

(4) The distance between black clusters of pixels included in one character is equal to or shorter than a predetermined distance (such as 3 pixels), the black clusters of pixels are classified into one group. A bounding rectangle region including both of the black clusters of pixels classified into one group is determined as a character region (704,706). It should be noted that a black cluster of pixels included in a character with no other black cluster of pixels included in the character exists within a predetermined distance, the black cluster of pixels may form one group singly. Therefore, the bounding rectangle region of the single black cluster of pixels is determined as a character region. The same processing as the processing described in (4) is performed on black clusters of pixels included in a picture.

(5) Locations of the regions and attribute identification information (character or picture) of the regions are output as determination results.

The processing in (1) to (5) outputs a determination result that the regions704and706are character regions and the region705is a picture image region. The region determining unit503has been described up to this point.

A character segmenting unit504performs a segmentation process with a character segmentation rectangle on the character regions generated by the region determining unit503.FIG. 7illustrates segmentation results710,711,712, and713. The segmentation process includes the following steps.

(1) One of the character regions is selected (708is selected, for example).

(2) A projection in a horizontal direction is taken for one binary image identified based on the character region. More specifically, the number of black pixels on a horizontally extending line is counted, and the resulting counted number corresponds to its projection.FIG. 7illustrates a taken projection715. In the projection715, a series of lines in the vertical direction having more black pixels than a threshold value is handled as one group. As the result, three groups occur. The three groups include a group of lines having ABCD, a group of lines having EFG, and a group of lines having H.

(3) A projection in the vertical direction is taken for the groups.FIG. 7illustrates the projection716taken for the lines having ABCD.

(4) In the projections for the groups, a series of lines in the horizontal direction having more black pixels than a threshold value is handled as one group. For example, in the projection716, four groups occur. The four groups include a group of lines having A, a group of lines having B, a group of lines having C, a group of lines having D.

(5) The bounding rectangles of the groups of lines acquired in (4) are segmented as character segmentation rectangles. As a result, for example, a bounding rectangle of each character is segmented as a character segmentation rectangle.FIG. 7illustrates results711,712,713, and710of the segmentation.

(6) The processing in (1) to (5) is repeated until no more unselected character region exists.

FIG. 7illustrates images to be processed and image examples of results of the binarization, region determination, and character segmentation processes.FIG. 7illustrates an image701that is an example of the input image501and further illustrates a character image7011on a white background, a character image7012on a background having a lower density, and a character image7013on a higher density background. In other words, the character images7011and7012are character images having a greater density difference between their characters and backgrounds, and the character image7013is a character image having a smaller density difference between its character and background.

FIG. 7further illustrates an example binary image702of a result of binarization performed on the image701in the binarizing unit502and further exemplarily illustrates a character7013is degraded in black as a result of a binarization process with a lower threshold value than the density of the background.

In description of this exemplary embodiment, a character image degraded as a result of a binarization process (an image in which a background and a character are difficult to separate even by performing a binarization process thereon because the density difference between the background and the character is small, such as a character density on a background greater than a threshold value) will be called a “character image difficult to be separated from the background”. A character image not degraded as a result of a binarization process (an image in which a background and a character are easy to separate by performing a binarization process thereon because the density difference between the background and the character is large, such as a black character on a white or lower background than a threshold value) will be called a “character image difficult to be separated from the background”. In other words, the “character image easy to be separated from the background” is an image of a character region having a character image part being black pixels and a background part, excluding the character, being white pixels as a result of a binarization process.

The image703is a result of a region determination process performed on the binary image702in the region determining unit503. As a result of the region determination process, the regions704and706are determined as character regions, and the region705is determined as a picture image region. The character regions707and708are partial images extracted from the binary image703and determined as character regions by the region determining unit503. A schematically illustrated character segmentation rectangle709is a result of a segmentation process performed by the character segmenting unit504. A character segmentation rectangle710is segmented from inside of the character region704. Character segmentation rectangles711,712, and713are segmented from inside of the character region706.

A region determining unit2(505) determines whether the character image within the character segmentation rectangle segmented by the character segmenting unit504is a character degraded by a binarization process (character image difficult to be separated from the background) or not. Details of the determination to be performed by the region determining unit2will be described below. Based on information on the character region determined as a “character image difficult to be separated from the background” by the region determining unit2(505), the character region information generated by the region determining unit503and the character segmentation rectangle information generated by the character segmenting unit504are modified. In other words, information on the character region determined as a “character image difficult to be separated from the background” by the region determining unit2(505) is removed from the character region information generated by the region determining unit503and the character segmentation rectangle information generated by the character segmenting unit504. This may solve a problem that the character region determined as a “character image difficult to be separated from the background” is not determined as a character and does not undergo MMR compression, which will be described below, resulting in invisibility of the character image.

An MMR compression unit506extracts a binary image of the character region based on the character region information modified by the region determining unit2(505) from the binary image generated by the binarizing unit502(or extracts a binary image included in the character segmentation rectangular region determined as a “character image easy to be separated from the background”). Then, the MMR compression unit506performs MMR compression on the binary image of the extracted character region to generate a compression code1(511).

A reducing unit507performs a reduction process (resolution reduction process) on an input image501to generate a reduced multi-valued image (not illustrated).

A representative-color extracting unit508identifies a location of pixels (black pixels) forming characters in the binary image based on the character region information and character segmentation rectangle information modified by the region determining unit2(505). A representative color of characters is calculated for each character segmentation rectangular region based on the identified location of pixels of the characters and with reference to the color of the corresponding location in the reduced multi-valued image so that character color information513of each of the characters is acquired. For example, a representative color may be an average or a weighted average of colors in a multi-valued image of pixels that have turned to black in the binary image in the character segmentation rectangular region or a most frequently occurring color among the pixels. Various methods for acquiring a representative color may be possible, but a color in a multi-valued image of at least one pixel of pixels having turned to black in a binary image in a character segmentation rectangular region is used for the calculation of a representative color.

A character-region filling unit509identifies a location of pixels (black pixels) included in characters in a binary image based on the character region information and character segmentation rectangle information modified in the region determining unit2(505). Then, based on the identified location of pixels, the pixels in the corresponding location in a reduced multi-valued image is filled with color surrounding it (hereinafter, called a surrounding color). The surrounding color may be an average value of pixel values of pixels surrounding a character, and the pixel value of pixels is replaced by the acquired surrounding color. Details of such a fill-up process performed by the character-region filling unit are disclosed in Japanese Patent Laid-Open No. 06-068301.

A JPEG compression unit510performs JPEG compression on an image having undergone the fill-up processing performed by the character-region filling unit509to generate a compression code2(514).

An OCR unit (516) performs publicly known character recognition processing with reference to the character segmentation rectangle information generated in step904for the region determined as a character region by the region determining unit (503). A character code517is a character code acquired by the character recognition processing.

Here, while the MMR compression unit (506) performs MMR compression on a region, determined as a character or a region determined as a “character image easy to be separated from the background” by the region determining unit2(505), the OCR unit (516) performs OCR on a region determined as a character region by the region determining unit (503).

Among such regions, because a “character image easy to be separated from the background” corresponds to a partial region of a region determined as a character region by the region determining unit (503), the region of a “character image easy to be separated from the background” is narrower. In other words, a target region subject to OCR is wider while a target region subject to MMR compression is narrower.

A reason why a target region subject to OCR is wider is that even when a target region subject to OCR contains a region that does not actually correspond to a character, an unnecessary character code may only be acquired, which is not a significant problem (or such a unnecessary character code may be deleted if it is considered so). On the other hand, performing MMR compression on a region that does not actually correspond to a character may cause deterioration in image quality in the region. For that reason, a wider region is subject to OCR while a narrower region is subject to MMR compression.

Thus, compressed data a file in PDF format (515) is generated which contains the compression code1(511) acquired from a component, modified character region information (512), character color information (513), compression code2(514), and character code (517). The generated file in PDF format is transmitted to a destination designated by a user, as described above.

FIG. 6is a block diagram illustrating a configuration of an image decompression processing unit configured to decompress PDF compressed data transmitted from another apparatus. The processing illustrated inFIG. 6may be executed for decompressing and printing a compressed data, for example. Here, an example will be described in which compressed data transmitted from another apparatus is identical to the compressed data file515.

An MMR decompression unit601performs MMR decompression processing on the compression code1(511) contained in the compressed data (515) file to reproduce a binary image. A JPEG decompression unit603performs JPEG decompression processing on the compression code2(514) to reproduce a reduced multi-valued image. An enlarging unit604performs enlarging processing on the reduced multi-valued image decompressed by the JPEG decompression unit (603) to generate a multi-valued image having an equal size to the size of the input image501before it is compressed.

A synthesizing unit602allocates, with reference to the character region information (512), a color corresponding to character color information (hereinafter, called a character color) (513) to black pixels in the binary image decompressed by the MMR decompression unit. The binary image to which the character color is allocated is synthesized over the multi-valued image generated by the enlarging unit604to generate a decompressed image605. For this synthesis, a transparent color is allocated to white pixels in the binary image so that the background multi-valued image may be transparent. In this manner, the image decompression processing unit decompresses compressed data generated by the image compressing processing unit to generate the decompressed image605. The decompressed image605is transmitted to and is printed by the printer unit202through the device I/F214. It should be noted that the image decompression processing unit ignores the character code517. This is because a character code is not necessary for printing decompressed image. A character code is necessary for not the MFP101but an apparatus such as the client PC102which displays the decompressed image605on a display device. Therefore, the MFP101ignores the character code517. More accurately speaking, a character code is necessary for a user who is using the PC102rather than the PC102itself. A character code is used for cutting and paste and editing a character string.

Next, details of processing to be executed by the region determining unit2(505) will be described. The region determining unit2(505) determines whether a character image within a character segmentation rectangle is degraded by binarization based on an binary image generated by the binarizing unit502, a reduced multi-valued image generated by the reducing unit507and character segmentation rectangle information generated by the character segmenting unit504. It should be noted that the input image501may be used instead on the reduced multi-valued image if the input image501is held in the storage unit211.

A detail configuration of the region determining unit2(505) will be described with reference toFIG. 3. For easy understanding of the following description, character image examples inFIG. 4will also be referred.FIG. 4illustrates a character image401having a character on a white background (with a large density difference between the background and the character). When the binarizing unit502binarizes the image401, a binary image402is acquired. A character image406has a character on a dark background (with a small density difference between the background and the character). When the image406is binarized with a lower threshold value than the density of the background407, a black degraded image408is acquired. Because a similar image to the image402is acquired by binarizing a character on a background having a light density with a threshold value between the density of the background and the density of the character, the description will be omitted. Images403to405and409to411will be described below.

The region determining unit2(505) includes a thinning unit301, an edge detecting unit302, a logical operation unit303, an edge counting unit304, and a number-of-edges comparing unit305.

The region determining unit2(505) extracts internal edge pixels of a region having a density greater than a threshold value (or black regions in images402and408) (1). If the number of extracted edge pixels is lower than a threshold value, it is determined as a “character image easy to be separated from the background” (2). If the number of extracted edge pixels is equal to or greater than the threshold value, it is determined as a “character image difficult to be separated from the background” (2).

For example, no edge pixels exist within the black region in the image402. On the other hand, there are edge pixels (edge pixels of the character “H” in the image410) within the black region in the image408. The term “edge pixel” refers to an edge pixel extracted from a multi-valued image (input image) rather than an edge pixel extracted from a binary image.

One configuration will be described below for implementing the steps (1) and (2) though an embodiment of the present disclosure is not limited to the configuration. Other possible configurations will also be described below.

The thinning unit301executes thinning process on a binary image in character segmentation rectangles with reference to character segmentation rectangle information. The thinning process thins a black cluster of pixels by trimming outer 2 pixels of a black cluster of pixels within a binary image (or replacing a black pixel on an outline of a black cluster of pixels by a white pixel). For example, pixels within a binary image contained in a one target character segmentation rectangle may sequentially be handled as a pixel of interest to be scanned by utilize a 5×5 window. If at least one white pixel exists within the 5×5 window, the pixel of interest (at the center of the 5×5 window) may be replaced by a white pixel for implementing the thinning process. Here, performing the thinning process on the binary image402may result in a thinned image403, for example. Performing the thinning process on the binary image408may result in a thinned image409, for example.

The edge detecting unit302performs edge detection on an input reduced multi-valued image in character segmentation rectangles with reference to character segmentation rectangle information. An image in which a pixel determined as an edge is represented by a black pixel and a pixel not determined as an edge is represented by a white pixel will be called an edge detected image. Because the edge detection may be based on a publicly known method, the following processing may be possible though description on details will be omitted. For example, differential filtering is executed on a luminance component of a reduced multi-valued image to acquire edge intensities of pixels. A pixel having an edge intensity equal to or greater than a predetermined threshold value is represented by a black pixel, and a pixel having an edge intensity lower than the predetermined threshold value is represented by a white pixel so that an edge detected image may be generated. However, an edge detection method according to a fourth exemplary embodiment, which will be described below, may be used for achieving edge detection with high precision.

Performing an edge detection process on a reduced multi-valued image, not illustrated, acquired by reducing the input image401may result in an edge detected image404. Performing the edge detection process on a reduced multi-valued image, not illustrated, acquired by reducing the input image406may result in an edge detected image410. If a reduced multi-valued image, not illustrated, acquired by reducing the input image401or406has ½ of the resolutions of an input image, the images404or410may have ½ of resolutions of the corresponding input image. However, those images are illustrated as having equal sizes for simplification. When the storage unit211holds the input image401or406, the edge detection may be performed on the input image401or406instead on a reduced multi-valued image.

The logical operation unit303is configured to take a logical AND of a thinned image generated by the thinning unit301and an edge detected image generated by the edge detecting unit302to generate a logical AND image. More specifically, only when a thinned image generated by the thinning unit301contains a black pixel and a black pixel exists at the corresponding location in the edge detected image generated by the edge detecting unit302, taking a logical AND of them results in a black pixel. When an edge detected image generated by the edge detecting unit302has ½ of resolutions of the thinned image, 0-order interpolation may be performed on the edge detected image to have equal resolutions as those of the thinned image before a logical AND is taken. Alternatively, a thinned image may be thinned out to have equal resolutions to those of the edge detected image before a logical AND is taken. Taking a logical AND of the thinned image403and the edge detected image404, a black pixel does not exist within a logical AND image405fundamentally (though some black pixels may remain due to an influence of noise) because a black pixel in the thinned image403and a black pixel in the edge detected image404do not exist at an identical location. On the other hand, taking a logical AND of the thinned image409and the edge detected image410, a black pixel remains in a part corresponding to a character as in a logical AND image411. From this, there is a characteristic that a fewer black pixels exist within a logical AND image for a character image easy to be separated from the background” while more black pixels exist within a logical AND image for a “character image difficult to be separated from the background”.

An image412is acquired by overlaying the thinned image403and the edge detected image404with each other. The image412has black pixels413corresponding to the thinned image403and has black pixels414corresponding to the edge detected image404. Because the black pixels in the thinned image413and the black pixels in the edge detected image414do not exist at an identical location, taking a logical AND of them does not generate black pixels.

The edge counting unit304is configured to count, as a number of edges, the number of black pixels in a result (logical AND image) of a logical AND taken by the logical operation unit303.

The number-of-edges comparing unit305is configured to compare the number of edges in a given image counted by the edge counting unit304and a predetermined threshold value to determine whether the image is a “character image easy to be separated from the background” or a “character image difficult to be separated from the background”. In other words, if the counted number of edges is lower than the predetermined threshold value, the image is determined as a “character image easy to be separated from the background (character image not degraded by binarization)”. If the counted number of edges is equal to or greater than the predetermined threshold value, the image is determined as a “character image difficult to be separated from the background (character image degraded by binarization)”.

If the pixel width of a black cluster of pixels is smaller than a thinning width of a thinning process, performing the thinning process may result in no black cluster of pixels within the binary image. For example, when a black cluster of pixels in a binary image is a thin line character having a 3-pixel width and a thinning process with a 4-pixel thinning width is performed thereon, no black cluster of pixels remains. In a case where no black cluster of pixels remains as a result, the processing to be performed by the edge detecting unit302, logical operation unit303, edge counting unit304and number-of-edges comparing unit305may be omitted from viewpoint of the improvement of processing speed. This is because counting the number of edges in a logical AND image with a thinned image apparently results in 0 even when if edge detecting unit302detects edge pixels. Because the counting result 0 means that the number of edges is lower than the predetermined threshold value, the image may be determined as a “character image easy to be separated from the background (character image not degraded by binarization)”.

Therefore, in a case where all of black pixels in a subject character segmentation rectangle disappear due to a thinning process performed thereon, the image in the character segmentation rectangle is determined as a “character image easy to be separated from the background (character image not degraded by binarization)” without performing the processing in the edge detecting unit302to the number-of-edges comparing unit305. When the processing to be performed by the units302to305is omitted, the processing in the thinning unit301to the number-of-edges comparing unit305is performed on a next character segmentation rectangular region. The reason for omitting the processing may be explained as follows. That is, such disappearance of black pixels due to a thinning process performed thereon may indicate that width of such black pixels is thin and the black pixels generally corresponds to a character or a line though a width of black pixels in a binary image is significantly narrow. Therefore, it may be explained that a target character segmentation rectangular region may be determined as a “character image easy to be separated from the background (character image not degraded by binarization)” without performing the processing from viewpoint of processing speed.

Alternatively, in a case where all black pixels within a binary image disappear, the number of pixels to be trimmed may be reduced. For example, replacing a pixel of interest (center of a 5×5 window) by a white pixel because one white pixel exists within a 5×5 window may result in replacement of all black clusters of pixels by white pixels. To avoid this, the window size may be reduced for processing with a 3×3 window.

Having described that the number of edges counted by the edge counting unit304is compared with a predetermined threshold value, a value acquired by dividing the number of edges by the number of black pixels in a thinned image may be compared with a predetermined threshold value. This may allow appropriate determination independent of the size of the character segmentation rectangular region. The number of edges may be divided by number of all pixels included in the character segmentation rectangular region or the number of black pixels after the rectangular region is binarized. However, for the highest precision, the number of edges may be divided by the number of black pixels in a thinned image. From this, the proportion of edges present in an internal part of the binary image (internal part within a dark region) may be learned. A greater proportion thereof means that there is a high proportion of edges in the inner part of the binary image and therefore that there is a high possibility that the binary image is not a character.

Next, steps to be executed by the data processing unit215will be described with reference to the flowchart inFIG. 8.FIGS. 2, 3, and 5are further referred in the following description. The region determining unit2(505) executes steps905to911inFIG. 8.

In step901, the binarizing unit502executes the binarization process on the input image501.

In step902, the region determining unit503executes the region determination process on the binary image and identifies regions contained in the binary image and determines whether each of the identified regions is a character region or non-character region.

In step903, the regions having undergone region determination by the region determining unit503are handled sequentially one by one as a region of interest. If the region of interest is a region determined as a character region by the determining unit, the processing moves to step904. If it is a region determined as a non-character region, the processing moves to step913.

In step904, the character segmenting unit504performs character segmentation on an image within the region of interest to generate character segmentation rectangle information.

In step916, the OCR unit516performs a publicly known character recognition process on a region determined as a character region by the region determining unit (503) referring to the character segmentation rectangle information generated in step904.

In step905, the thinning unit301executes a thinning process on each binary image resulting from the binarization in step902within the character segmentation rectangle referring to the character segmentation rectangle information generated in step904.

In step906, the edge detecting unit302executes an edge detection process on each reduced multi-valued image within the character segmentation rectangle (or an input image within the character segmentation rectangle) by using the reduced multi-valued image resulting from the reduction of an input image (or the input image501) and character segmentation rectangle information generated in step904.

In step907, the logical operation unit303takes a logical AND of the thinned image generated by the thinning unit301in step905and the edge image generated in step906.

In step908, the edge counting unit304counts black pixels in an image resulting from the logical AND (AND) performed by the logical operation unit303in step907to acquire the number of edges. Here, the acquired number of edges may be divided by an area of the character segmentation rectangular region (the total number of pixels within the character segmentation rectangular region) to be normalized for acquiring the number of edges per a unit area. This advantageously allows comparison with a threshold value in step909independently from the size of the character segmentation rectangular region.

Next, in step909, the number-of-edges comparing unit305compares the number of edges counted in step908and a threshold value th. Here, if the number of edges is greater than the threshold value th, the character segmentation rectangular region of interest is determined as a “character image difficult to be separated from the background” in step910. If the number of edges is equal to or lower than the threshold value th, the character segmentation rectangular region of interest is determined as a “character image easy to be separated from the background” in step911.

In step912, the character segmenting unit504determines whether all character segmentation rectangles within the character region of interest have been processed. If so, the processing moves to step913. On the other hand, if there still remains an unprocessed character segmentation rectangle, the next character segmentation rectangle is set as a next rectangle of interest in step914, and the processing returns to step905.

If it is determined in step913that the determination on all of the regions has ended, the processing ends. If it is determined that there still remains an unprocessed region, the next unprocessed region is set as a region of interest in step915, and the processing returns to step903.

In this manner, the region determining unit2(505) determines with high precision whether each of character segmentation rectangular regions is a “character image easy to be separated from the background” or a “character image difficult to be separated from the background” based on the number of black pixels (the number of edges) resulting from the logical AND of its thinned image and its edge detected image.

Because a “character image difficult to be separated from the background” (713inFIG. 7, for example) is removed from the character region information, it is not processed by the MMR compression unit506. In other words, a “character image difficult to be separated from the background” is compressed by the JPEG compression unit510along with its background image without binarization.

In this manner, whether a given image is a character image degraded by binarization or not is determined. Thus, degradation of the character image may be prevented.

Having described that according to this exemplary embodiment one character “H” as in images406and408inFIG. 4is determined as a “character image difficult to be separated from the background (character image degraded by binarization)”, for example, the embodiment is not limited thereto. For example, two or more characters may be handled as in an input image1001inFIG. 9. Binarization of the input image1001results in a binary image1002. A character image degraded by binarization is not necessarily rectangular but may be an image having a part of the character image degraded as in an image1003inFIG. 9, for example. Binarization on the input image1003results in a binary image1004.

Next, another configuration of the region determining unit2(505) will be described.

According to another configuration, (A) an image input to the region determining unit2is first divided into a region having a greater density than a threshold value and a region having an equal or lower density than the threshold value (by using binarization, ternarization or other method). As a result, regions402and408may be acquired, for example.

(B) Edge pixels are extracted (by using an extraction method as described above) from a region determined as having a density greater than the threshold value (H region in the image401or an entire region of the image406) in the input image. This edge extraction process is not performed on an end part (such as one or two pixels inside from the end) of a region determined as having a density greater than the threshold value. In other words, according to the configuration in (B), edge pixels at a predetermined distance or longer (inside) from an end part of a region determined as having a density greater than the threshold value are extracted. According to an alternative configuration, edge pixels including in such an end part (pixels not at a predetermined distance or longer) may be extracted, and then edge pixels in such an end part are removed. Thus, the same result as the resulting images405and411may be acquired. While the predetermined distance in this example is 3 pixels, it may be other values.

(C) Subsequently, the number of the resulting edge pixels is counted, and whether the number of the edge pixels is greater than the threshold value th or equal to lower than the threshold value th is determined.

Thus, the same result (or a result of determination whether “character image difficult to be separated from the background” or “character image easy to be separated from the background”) as that of the aforementioned method may be acquired. Edge pixels may be extracted from an entire image input to the region determining unit2, instead of the processing in (B). In this case, an end part of a region determined as having a density greater than the threshold value and a region having an equal to or lower than the threshold value are excluded from edge pixels extracted from an entire input image. Thus, the same result as that of the configuration (B) may be acquired.

The processing in characters on a result of the character segmentation in step904performed by the character segmenting unit504has been described according to this exemplary embodiment. The processing may be performed on sectional regions acquired by segmenting each of characters instead of the processing in characters. For example, a region may be equally divided into four sectional regions for the character segmenting unit504, and the processing may be performed on each of the regions. For example,FIG. 16illustrates images1300to1304which are acquired by equally segmenting the character segmented region406into 4. The processing is performed on each of the images1300to1304. Alternatively, the determination may be performed on a center part of a character segmented region (such as only 60% of a center part of a character-segmented region). For example,FIG. 16further illustrates an image1305which is an extraction of 60% of a center part of the character-segmented region406, and the processing may be performed on the image1305. A combination of the determination on character-segmented regions and the determination on the character-segmented regions and/or a center part thereof may allow determination of whether the image is a “character image easy to be separated from the background” or a “character image difficult to be separated from the background”.

Second Exemplary Embodiment

According to the first exemplary embodiment, a region determined as a “character image difficult to be separated from the background” by the region determining unit2(505) is not subject to the MMR compression process. According to a second exemplary embodiment, the region determining unit2(505) re-executes a binarization process with high precision based on a different algorithm from that of the binarizing unit502on a region determined as a “character image difficult to be separated from the background”, and pixels of a character image part may be separated from a background. In this case, performing the MMR compression process on a character region resulting from the re-binarization process with high precision may contribute to improvement of image quality of the character region. For example, because the region713inFIG. 7is determined as a “character image difficult to be separated from the background”, a region7013in the input image701corresponding to the region713is only binarized with a different threshold value from that for other regions. As a result, a binary image like the image714inFIG. 7may be generated, and the character region may be MMR compressed. An example of the re-binarization process with high precision may be a method for performing a binarization process by using an average value of the density or luminance of a subject region as the threshold value instead of a binarization process by using a fixed threshold value.

Third Exemplary Embodiment

According to the first exemplary embodiment, an input image having a relatively higher character quality like the image401inFIG. 4has been described, for example. However, performing the edge detection process on an image (such as a scan document or a compressed image) having a poor character quality and more noise like the image1101inFIG. 10may cause many edges within the character like an image edge detected image1102inFIG. 12. Such edge appearance may become more significant particularly within a large character.

Here, edges within a character may easily remain in a logical AND image1104acquired from the edge detected image1102and a thinned image1103. Many edges remaining inside of a character may read wrong determination of an actual “character image easy to be separated from the background” as a “character image difficult to be separated from the background”.

According to a third exemplary embodiment, increasing the amount of reduction of the thinning process performed by the thinning unit301may reduce edges remaining inside a character in a case where a subject character-segmented region has a larger size. The processing will be described with reference to images1105to1112inFIG. 10.

An input image1105having poor character quality and more noise contains a small character. An edge detected image1106is a result of an edge detection process executed on the image1105having a small character. A thinned image1107is a result of a thinning process executed on the image1105having a small character. The thinning process may use a 5×5 window and include replacing a pixel of interest (center of a 5×5 window) by a white pixel if the 5×5 window has at least one white pixel.

A logical AND image1108is a result of a logical AND of the edge detected image1106and the thinned image1107. Here, even a small character in an input image having poor character quality and more noise has fewer edges, compared with the logical AND image1104having a large character.

A large character image1109(identical to the character image1101) is in an input image having poor character quality and more noise. An edge detected image1110is a result of the edge detection process executed on the large character image1109. A thinned image1111is a result of the thinning process executed on the large character image1109. The thinning process on a large character image may use a 9×9 window and include replacing a pixel of interest (center of a 9×9 window) by a white pixel if the 9×9 window has at least one white pixel. In other words, changing the size of a window based on the size of a subject character image (size of a subject character-segmented region) may increase the amount of reduction of the thinning process. The size of the window is given for illustration purpose only and is not limited to 5×5 or 9×9.

A logical AND image1112is a result of a logical AND of the edge detected image1110and the thinned image1111. The logical AND image1112has fewer edges than the logical AND image1104. Therefore, even an image having a large character with much noise may be determined as a “character image easy to be separated from the background” by reducing the amount of reduction of the thinning process if the size of the character image is large.

According to the third exemplary embodiment, the control of the amount of reduction of the thinning process to be performed by the thinning unit based on the size of a subject character-segmented region may reduce an influence of noise, for example, and allow the determination with high precision even in a case where an input image is a scan document, as described above.

Fourth Exemplary Embodiment

Next, details of the processing to be performed by the edge detecting unit (302) within the region determining unit2(505) inFIG. 5will be described with reference toFIG. 11. The edge detecting unit (302) includes a variance value detecting unit1001, an edge determination threshold value calculating unit1002, and an edge extracting unit1003. The processing to be performed by the edge detecting unit (302) will be described in more detail further with reference toFIG. 12.FIG. 12illustrates input images1101,1102, and1103segmented in character segmentation rectangles with reference to the character segmentation rectangle information, like the images401and406illustrated inFIG. 4. The input images1101,1102, and1103are image examples having different signal values from each other when they are acquired by the scanner unit201. More specifically, they have signal values represented by a L*a*b* color system where L* represents brightness, and a* and b* represent chromaticity. Notably, while this embodiment applies a L*a*b* color system, an embodiment of the present disclosure is not limited thereto. For example, the same processing may be performed with a signal value in a different color space such as an RGB (red, green, blue) color system. The region1104of the image1101has a signal value of {L*, a*, b*}={1128, −50, +30}. The region1105has a signal value of {L*, a*, b*}={128, +50, −60}.

FIG. 12illustrates an example in which there is a difference in large signal value between the regions1104and1105. On the other hand, the region1106in the image1102has a signal value of {L*, a*, b*}={128, −50, +30}. The region1107has a signal value of {L*, a*, b*}={128, −60, +30}.FIG. 12illustrates an example in which there is a small difference in signal value between the regions1106and1107. The region1108in the image1103has a signal value of {L*, a*, b*}={128, −50, +30}. The region1109has a signal value of {L*, a*, b*}={128, −52, +30}.FIG. 12illustrates an example in which there is substantially no difference in signal value between the regions1108and1109. For example, in a case where the edge detecting unit (302) performs an edge detection based on a result of comparison in signal value simply between adjacent pixels or an edge detection by filtering, instead of the aforementioned configuration may cause the following problems. That is, with some threshold values, an edge may be acquire at a boundary between the regions1104and1105in the image1101while no edge may be acquired at a boundary between the regions1106and1107in the image1102. Furthermore, with a threshold value which allows acquisition of an edge at the boundary between the regions1106and1107in the image1102, an edge may be acquired at the boundary between the regions1108and1109in the image1103. As a result, scanning variations of a scanner and small noise such as Jpeg noise may be detected as edges disadvantageously.

FIG. 11illustrates a configuration for solving those problems. The variance value detecting unit1001corresponds to a calculation unit configured to calculate a variance value of a signal value of an input image segmented in character segmentation rectangles. For example, the variance value may be calculated by the following expression:

Here, n is a number of pixels of a segmented input image, Xi (i=1, 2, . . . , n) is a signal value of each pixel (L*, a*, b* values in this exemplary embodiment), and Xave is an average of signal values of the number of pixels within the region. A variance value with L*, a*, b* values is acquired in this exemplary embodiment, but an embodiment of the present disclosure is not limited thereto. For example, a covariance value may be acquired with a*, b* signal values. In the example input images1101,1102, and1103illustrated inFIG. 12, because the input image1101has a large signal value difference, a greater variance value may be acquired. Because the input images1102and1103have smaller signal value differences, relatively lower variance values may be acquired.

An expression “threshold value for easy edge acquisition” in the following descriptions may refer to a threshold value that allows determination of an edge even when there is a small signal value difference between adjacent pixels. On the other hand, an expression “threshold value for difficult edge acquisition” in the following descriptions refers to a threshold value with which an edge is not determined if a signal value difference is not large and if a signal value difference is small.

The edge determination threshold value calculating unit1002calculates a threshold value for performing edge extraction based on the variance value calculated by the variance value detecting unit1001. For example, a threshold value for difficult edge acquisition is allocated to an image having a large variance value as in the image1101. On the other hand, a threshold value for easy edge acquisition is allocated for the images1102and1103.

The edge extracting unit1003is a processing unit configured to perform an edge extraction process based on the threshold value determined by the edge determination threshold value calculating unit1002. The processing may be performed by a general-purpose method by which, for example, a signal value difference between close pixels is acquired by comparing them and whether the difference is greater than a specific threshold value or not is determined. Alternatively, an amount of edge may be acquired by using a filter for calculating a primary differential value, and whether it is greater than a specific threshold value or not may be determined.

For segmentation based on a condition calculated by the edge determination threshold value calculating unit1002, a threshold value for difficult edge acquisition is allocated to the input image1101for the edge extraction. For example, an example in which the threshold value determined based on a variance value is 5 will be described. When the threshold value is used, an edge between the regions1104and1105may be extracted securely because there is a large signal value difference between the regions1104and1105. This may result in an edge image1110. On the other hand, a threshold value for easy edge acquisition is allocated to the input image1102though there is a small signal value difference between the regions1106and1107so that an edge between the regions1106and1107may be extracted. This may result in an edge image1111. A threshold value for easy edge acquisition is allocated to the input image1103, but the signal value difference between the regions1108and1109is significantly smaller than the signal value difference between the regions1106and1107. Thus, even with a threshold value for easy edge acquisition, an edge present between the regions1108and1109may not be extracted. This may result in an edge image1112.

Next, the edge detecting unit (302) inFIG. 11will be described with reference to the flowchart inFIG. 13.FIG. 11will be further referred in the following description.

First, in step1201, a variance value calculating unit (1001) calculates a variance value of a signal for an input image (501). In this case, variance values for all three channels may be calculated if the image has three channels, or one variance value may be calculated by merging them into one channel.

Next, in step1202, an edge threshold value calculating unit (1002) determines whether the variance values of signals of the image calculated in step1201are equal to or greater than a predetermined value or not. If they are equal to or greater than the predetermined threshold value, a “threshold value for easy edge acquisition” is acquired in step1203. If they are lower than the predetermined threshold value on the other hand, a “threshold value for difficult edge acquisition” is acquired in step1204.

Finally, in step1205, an edge extracting unit (1003) performs an edge extraction process based on the threshold value determined in step1203or1204.

According to this exemplary embodiment, it is configured such that the threshold value is adaptively changed based on variance values of each image segmented for each character segmentation rectangle before an edge extraction process is performed thereon, as described above. This allows s segmentation with high precision between a “character image difficult to be separated from the background” and a “character image easy to be separated from the background”.

Fifth Exemplary Embodiment

According to the fourth exemplary embodiment, a method has been described which changes a threshold value based on a variance value of a signal value for threshold value calculation before an edge extraction process is performed. In a case where an input image is a polychrome image with three channels, for example, an equal number of variance values to the number of channels may be calculated for use in determination of a threshold value with high precision. However, in a case where an input image is a gray-scale image and therefore has one channel, one variance value may be used for the threshold value calculation, making it difficult to calculate a threshold value with high precision.

According to this exemplary embodiment, the edge detecting unit (302) further includes a black pixel density calculating unit1004in addition to the variance value detecting unit1001, edge determination threshold value calculating unit1002, and edge extracting unit1003, as illustrated inFIG. 14. This configuration is also applicable to a binarized image in addition to an input image.

The black pixel density calculating unit1004is configured to calculate a ratio of the number of black pixels to a dimension of a character segmentation rectangle based on an input binarized image. The number of black pixels is counted within an input binarized image, and the count is divided by a dimension of a character segmentation rectangle.

Next, the edge threshold value calculating unit1002calculates an optimum threshold value based on the black pixel density calculated by the black pixel density calculating unit1004. Also in this case, a threshold value for edge extraction is calculated based on the black pixel density similarly to the change of a threshold value for edge extraction based on a variance value according to the first exemplary embodiment. More specifically, if the black pixel density is high, a “threshold value for easy edge acquisition” is set. If the black pixel density is low, a “threshold value for difficult edge acquisition” is set. Setting in this manner may allow edge extraction with a “threshold value for easy edge acquisition” for precise calculation of edges because a “character over a high-density background” has a high black pixel density.

It should be noted that both of a threshold value calculated based on a variance value and a threshold value calculated based on a black pixel density may be used for calculating a threshold value for edge extraction though use of either one is also possible. In this case, though a “threshold value for easy edge acquisition” is desirable from viewpoint of acquisition of more edges, selection of a “threshold value for difficult edge acquisition” is also possible. Priority may be given to a threshold value calculated based on a variance value, for example, by switching the weight on the threshold values.

As illustrated inFIG. 15, the edge detecting unit (302) may further include a number-of-closed-loop calculating unit1005in addition to the variance value detecting unit1001, edge determination threshold value calculating unit1002, edge extracting unit1003, and black pixel density calculating unit1004.

The number-of-closed-loop calculating unit1005is a calculating unit configured to perform a labeling process for calculating the number of closed loops formed by successive pixel in a white part of an input binarized image.

Next, the edge threshold value calculating unit1002calculates an optimum threshold value based on the number of closed loops calculated by the number-of-closed-loop calculating unit1005. Similarly to the first exemplary embodiment again, a threshold value to be used for edge extraction may be calculated based on the number of closed loops. More specifically, for a greater number of closed loops, a “threshold value for difficult edge acquisition” is used. For a lower number of closed loops, a “threshold value for easy edge acquisition” is used.

This processing allows calculation of an optimum threshold for edge extraction even for an image, such as a gray scale image, which has a lower number of channels and for which a threshold value for edge extraction may not be calculated based on a variance of a signal value.

Other Embodiments

This application claims the benefit of priority from Japanese Patent Application No. 2013-262763, filed Dec. 19, 2013, Japanese Patent Application No. 2014-054163, filed Mar. 17, 2014, and Japanese Patent Application No. 2014-139870, filed Jul. 7, 2014, which are hereby incorporated by reference herein in their entirety.