Patent Description:
Conventionally, there are technologies of reading an image with invisible light. Such technologies are used for extracting invisible information embedded in a document with a particular color material or component and improving the legibility of the document and the accuracy of optical character recognition/reader (OCR).

Further, <CIT> discloses an image processing technology for performing a predetermined image correction based on brightness information and color information of each image area. This technology is aimed at obtaining a good-quality monochrome image, without deteriorations in image quality from a color image document, so that the recognition rate of OCR can also improve.

The technology disclosed in <CIT> is to use an image of one color to correct an image of another color to prevent missing of a text (characters) or an image. However, this technology has a drawback that, for example, after a stamp mark imprinted on a text is removed, inevitably a portion of the overlapping text is lost. Further prior art is known from <CIT> and <CIT>.

In view of the above, an object of the present disclosure is to accurately remove only unnecessary information without causing loss of other information or loss of a character.

In order to achieve the above-described object, there is provided an image processing apparatus as described in appended claims. Advantageous embodiments are defined by the dependent claims.

Advantageously, the image processing apparatus includes an invisible light source configured to emit invisible light to a subject, a visible light source configured to emit visible light to the subject, an invisible image reading sensor configured to read invisible reflection light reflected from the subject to acquire an invisible image, a visible image reading sensor configured to read visible reflection light reflected from the subject to acquire a visible image, and an image restoration unit configured to restore information lost in the invisible image acquired by the invisible image reading sensor. The invisible reflection light is a part of the invisible light emitted from the invisible light source (2b). The visible reflection light is a part of the visible light emitted from the visible light source (2a).

Advantageously, there is provided a method for reading an image. The method includes reading invisible reflection light reflected from a subject to acquire an invisible image, reading visible reflection light reflected from the subject to acquire a visible image, and restoring information lost in the invisible image. The invisible reflection light is a part of the invisible light emitted from the invisible light source. The visible reflection light is a part of the visible light emitted from the visible light source.

Accordingly, only unnecessary information can be accurately removed without causing loss of particular information or loss of a character.

Hereinafter, descriptions are given below in detail of an image processing apparatus and an image reading method according to embodiments of the present disclosure, with reference to the accompanying drawings.

A first embodiment is described. <FIG> is a view illustrating an example of a configuration of an image forming apparatus <NUM> common to embodiments according to the present disclosure. In <FIG>, the image forming apparatus <NUM> is an apparatus generally referred to as a multifunction peripheral having at least two of a copy function, a printer function, a scanner function, and a facsimile function.

The image forming apparatus <NUM> includes a scanner <NUM> serving as a reading device as well as an image processing apparatus, an automatic document feeder (ADF) <NUM>, and an image forming device <NUM> below the scanner <NUM>. In order to describe an internal configuration of the image forming device <NUM>, <FIG> illustrates the internal configuration of the image forming device <NUM> from which an external cover is removed.

The ADF <NUM> is a document supporter that positions, at a reading position, a document containing an image to be read. The ADF <NUM> automatically feeds the document placed on a placement table to the reading position. The scanner <NUM> reads the document conveyed by the ADF <NUM> at a predetermined reading position. The scanner <NUM> includes, on the top, an exposure glass that is the document supporter, on which a document is placed, and reads the document on the exposure glass that is the reading position. Specifically, the scanner <NUM> includes a light source, an optical system, and a solid-state image sensing device such as a complementary metal oxide semiconductor (CMOS) image sensor inside. The scanner <NUM> reads, with the solid-state image sensing device through the optical system, light reflected from the document irradiated with light from the light source.

The image forming device <NUM> includes a bypass feeding roller pair <NUM> that feeds a recording sheet, and a recording sheet feeder <NUM> that feeds the recording sheet. The recording sheet feeder <NUM> includes a mechanism for feeding out the recording sheet from multi-stage recording sheet feeding trays 107a. The recording sheet thus fed is sent to a secondary transfer belt <NUM> via a registration roller pair <NUM>.

Onto the recording sheet conveyed on the secondary transfer belt <NUM>, a transfer device <NUM> transfers a toner image from an intermediate transfer belt <NUM>.

The image forming device <NUM> further includes an optical writing device <NUM>, a tandem image forming unit <NUM> for yellow (Y), magenta (M), cyan (C), and black (K), the intermediate transfer belt <NUM>, and the secondary transfer belt <NUM>. Specifically, in an image forming process, the image forming unit <NUM> forms an image written by the optical writing device <NUM> as a toner image on the intermediate transfer belt <NUM>.

Specifically, the image forming unit <NUM> for Y, M, C, and K includes four rotatable photoconductor drums (respectively for Y, M, C, and K) and image forming elements <NUM> including a charging roller, a developing device, a primary transfer roller, a cleaner unit, and a discharger. The image forming elements <NUM> are disposed around the each of the photoconductor drums. As the image forming element <NUM> operates on each photoconductor drum, the image on the photoconductor drum is transferred onto the intermediate transfer belt <NUM> by each primary transfer roller.

The intermediate transfer belt <NUM> is stretched by a drive roller and a driven roller and in a nip between each photoconductor drum and each primary transfer roller. The toner image primarily transferred onto the intermediate transfer belt <NUM> is secondarily transferred onto the recording sheet on the secondary transfer belt <NUM> by a secondary transfer device as the intermediate transfer belt <NUM> runs. The recording sheet is conveyed to a fixing device <NUM> as the secondary transfer belt <NUM> travels, and the toner image is fixed as a color image on the recording sheet. Then, the recording sheet is ejected onto an output tray disposed outside a housing of the image forming apparatus <NUM>. In duplex printing, a reverse assembly <NUM> reverses the recording sheet upside down and sends the reversed recording sheet onto the secondary transfer belt <NUM>.

The image forming device <NUM> is not limited to the one that forms an image by an electrophotographic method as described above. The image forming device <NUM> may be one that forms an image by an inkjet method.

Next, the scanner <NUM> will be described.

<FIG> is a cross-sectional view schematically illustrating a structure of the scanner <NUM>. As illustrated in <FIG>, the scanner <NUM> includes, in a housing <NUM>, a sensor board <NUM> on which an imaging device <NUM> is mounted, a lens unit <NUM>, a first carriage <NUM>, and a second carriage <NUM>. The imaging device <NUM> is a solid-state image sensing device in one example. The first carriage <NUM> includes a light source <NUM>, which is a light emitting diode (LED), and a mirror <NUM>. The second carriage <NUM> includes mirrors <NUM> and <NUM>. Further, the scanner <NUM> includes an exposure glass <NUM> on the top side thereof. A document <NUM> is placed on the exposure glass <NUM>.

The light source <NUM> includes a visible light source 2a and an invisible light source 2b as illustrated in <FIG>. The visible light source 2a emits visible light to a subject and a reference white plate <NUM>. The invisible light source 2b emits invisible light to a subject. Use of an infrared light source as the invisible light source 2b is effective. Generally, the visible light wavelength range is <NUM> to <NUM>, and the wavelength range greater than <NUM> is an infrared wavelength range, which is an invisible wavelength range.

<FIG> is a graph illustrating an example of spectral properties of white plain paper and color toner images. As illustrated in <FIG>, both of a black image formed with black (K) toner (hereinafter "black (K) image") and a black image formed with a composite of cyan, magenta, and yellow (CMY) toners (hereinafter "black (CMY) image") have a low reflectance in the visible light range and are recognized as the same "black" images. However, there is a large difference in reflectance in the infrared light range. In particular, the latter black image is recognized as almost a "white" image and disappears when read as image information.

Since both are black in the visible light wavelength range, it is difficult for human eyes to distinguish a black (K) image and a black (CMY) image before scanned as images. Therefore, whether or not the two image information is lost in infrared light (also "infrared ray") reading is not determinable with human eyes, and unintended loss of information occurs.

Therefore, the present embodiment provides an image processing apparatus that employs an infrared light source as the invisible light source 2b for reading a subject, thereby enabling both of erasing unnecessary information and restoring lost image information.

Further, the scanner <NUM> includes the reference white plate <NUM> on the upper side thereof. More specifically, the reference white plate <NUM> is disposed on the side opposite to the light source <NUM> with respect to the subject, in an image capture range of the imaging device <NUM>.

In the reading operation, the scanner <NUM> emits light from the light source <NUM> upward while moving the first carriage <NUM> and the second carriage <NUM> from the standby positions (home positions) in the sub-scanning direction (indicated by arrow A in <FIG>). The first carriage <NUM> and the second carriage <NUM> cause reflected light from the document <NUM> (i.e., the subject) to be imaged on the imaging device <NUM> via the lens unit <NUM>.

Further, the scanner <NUM> reads the reflected light from the reference white plate <NUM> to set a reference when the power is turned on or the like. That is, the scanner <NUM> moves the first carriage <NUM> directly below the reference white plate <NUM>, turns on the light source <NUM>, and causes the reflected light from the reference white plate <NUM> to be imaged on the imaging device <NUM>, thereby adjusting the gain.

The imaging device <NUM> can capture images in the visible and invisible wavelength ranges. In the imaging device <NUM>, pixels that convert incident light level into electric signals are arranged. The pixels are arranged in a matrix. Each pixel is provided with a color filter that transmits light having a specific wavelength. In the imaging device <NUM> according to the present embodiment, a channel refers to a signal obtained from a group of pixels each of which is provided with the same color filter. In addition, in the present embodiment, a visible image refers to an image captured by the imaging device <NUM> by irradiation of visible light, and an invisible image refers to an image captured by the imaging device <NUM> by irradiation of invisible light such as near-infrared ray.

Although the scanner <NUM> of the present embodiment employs a reduction optical system, the structure of the image reading sensor according to the present disclosure is not limited thereto. For example, a no-magnification optical system, such as a contact optical system or an image sensor (CIS) system may be used.

<FIG> is a block diagram illustrating an electrical connection of components of the scanner <NUM>. As illustrated in <FIG>, the scanner <NUM> includes, in addition to the imaging device <NUM> and the light source <NUM> described above, an image processing unit <NUM>, a controller <NUM>, and a light source driver <NUM>. The controller <NUM>, which may be implemented by a processor such as a central processing unit (CPU) that operates according a program stored in a memory, controls the imaging device <NUM>, the image processing unit <NUM>, and the light source driver <NUM>. The light source driver <NUM> drives the light source <NUM> under control of the controller <NUM>. The imaging device <NUM> transfers the signal to the image processing unit <NUM> on the subsequent stage.

The imaging device <NUM> includes an invisible light image sensor 22b, serving as an invisible image reading sensor, and a visible light image sensor 22a, serving as a visible image reading sensor. The invisible light image sensor 22b reads the invisible reflection light (a part of invisible light) reflected from the subject, thereby acquiring an invisible image (an image in the invisible light wavelength range). The visible light image sensor 22a reads visible reflection light (a part of visible light) reflected from a subject, thereby acquiring a visible image (an image in the visible light wavelength range). The invisible light image sensor 22b and the visible light image sensor 22a are sensors for a reduction optical system, such as a complementary metal oxide semiconductor (CMOS) image sensor.

<FIG> is a graph of spectral sensitivity characteristics of the imaging device <NUM>. As illustrated in <FIG>, the visible light image sensor 22a of the imaging device <NUM> has spectral sensitivity in each visible wavelength range of blue, green, and red. The invisible light image sensor 22b of the imaging device <NUM> has sensitivity only in the infrared (IR) light wavelength range greater than <NUM>. With such a configuration, when the light source <NUM> (the visible light source 2a and the invisible light source 2b) simultaneously emits invisible light and infrared light, the imaging device <NUM> can acquire both images in one-time reading of the subject. That is, such a configuration obviates the need to separate the irradiation light for each image sensor and can simplify the configuration. Note that the infrared light information is mixed in the R, G, and B pixels of the visible light image sensor 22a but can be removed from the respective read data by using the image data of the infrared light pixels.

The visible light image sensor 22a and the invisible light image sensor 22b may be integral with each other. Such a configuration can make the sensor structure compact and the reading positions of visible light and infrared light closer to each other. Accordingly, lost information can be extracted and restored with high accuracy. That is, such a configuration can eliminate deviations of the image, which may occur when the image is read a plurality of times, and correction can be made with high position accuracy.

The image processing unit <NUM> performs various types of image processing on image data according to a purpose of use. Note that the image processing unit <NUM> can be implemented by either hardware or software.

<FIG> is a block diagram illustrating an example of a functional configuration of the image processing unit <NUM>. In <FIG>, a reading unit <NUM> of the scanner <NUM> includes the visible light source 2a, the invisible light source 2b, the visible light image sensor 22a, and the invisible light image sensor 22b. As illustrated in <FIG>, the image processing unit <NUM> includes an image restoration unit <NUM>. For example, in the present embodiment, first, the invisible light image sensor 22b reads an image with the invisible light emitted from the invisible light source 2b. Invisible light is transmitted through information that does not contain a specific component (e.g., color information). Accordingly, based on an invisible image in which only necessary information, such as black (carbon) information, is read as an image, a color background pattern or color ruled lines can be removed, so as to improve legibility and OCR accuracy. However, on the other hand, there are cases where even the information that is to be retained (e.g., color information) is unintentionally removed.

Therefore, in the present embodiment, the visible light image sensor 22a performs image reading by the visible light emitted from the visible light source 2a, and the image restoration unit <NUM> restores the information lost in the invisible image acquired by the invisible light image sensor 22b based on the visible image.

More specifically, as illustrated in <FIG>, the image restoration unit <NUM> includes a first image restoration unit 21a. The first image restoration unit 21a corrects and restores the image information of the invisible image using the visible image read by the visible light image sensor 22a. By performing such an image restoration process, the first image restoration unit 21a generates a first output image.

The image restoration process executed by the image restoration unit <NUM> are described in detail below.

<FIG> are diagrams illustrating an example of the image restoration process. <FIG> illustrates an example of a paper document printed on paper having a color (pale blue, etc.) background pattern. Specifically, characters (text) representing a date and an amount of money are printed with black toner containing a specific component such as carbon, and a receipt mark is stamped with red ink.

<FIG> illustrates a color dropout image of the document illustrated in <FIG>. For scanning a document on order to extract text information by OCR, preferably, the background pattern and the receipt stamp are removed. For example, when such color information is removed by a conventional color dropout process, as illustrated in <FIG>, the background pattern is removed cleanly, but the text is lost in the portion (representing the amount of money) superimposed by the receipt stamp.

<FIG> illustrates an invisible image of the document illustrated in <FIG>. In contrast to the image reading by visible light, in image reading by invisible light emitted from the invisible light source 2b, the invisible light is transmitted through both the background pattern and the receipt mark that are made of materials free of the specific component. As a result, black text information can be read accurately as illustrated in <FIG>.

On the other hand, there are following cases. Although some information is written in black similar to important information such as numbers, the black is made of a material free of the specific component such as carbon, and invisible light is transmitted through such information. As a result, the information (e.g., color information) to be retained is unintentionally deleted.

The document illustrated in <FIG> is an example of a paper document in which a text! representing "important" is added to the receipt printed with black toner. The text TXT1 representing "important" is handwritten with a black material that is different from black toner and does not contain the specific component. The text TXT1 representing "important" is information indicating the importance and priority of the document, and disappearance is not desirable.

<FIG> illustrates an invisible image of the document illustrated in <FIG>. In the image reading with invisible light emitted from the invisible light source 2b, the text TXT1 representing "important", which is the information to be retained, is deleted as illustrated in <FIG>.

<FIG> illustrates a visible image of the document illustrated in <FIG>. In the image reading by the visible light emitted from the visible light source 2a, the text TXT1 representing "important" is not erased but remains as illustrated in <FIG>.

<FIG> illustrates the invisible image plus a restored image of the document illustrated in <FIG>. In the present embodiment, using the visible image (illustrated in <FIG>) acquired by the visible light image sensor 22a by reading with visible light, the first image restoration unit 21a restores the text TXT1 representing "important" in the invisible image acquired by the invisible light image sensor 22b. With such an image restoration process, the first image restoration unit 21a generates the first output image illustrated in <FIG>. This operation can prevent the information to be retained from being unintentionally lost.

As described above, the scanner <NUM> according to the present embodiment deletes unnecessary information in reading by invisible light, and, based on the visible image, supplements read information with image information that has been unintentionally deleted. Accordingly, the scanner <NUM> can accurately remove only unnecessary information without causing loss of other particular information or loss of characters. That is, while unnecessary information such as a background pattern and a stamp mark on a document are removed, information not to be removed can be restored. In addition, readability and OCR character recognition accuracy can be greatly improved.

A description is given of a second embodiment of the present disclosure.

The second embodiment differs from the first embodiment in that a general image sensor made of silicon is used as the invisible light image sensor 22b. In the following description of the second embodiment, descriptions of the same parts as those in the first embodiment are omitted, and differences from the first embodiment are described.

In the first embodiment, the scanner <NUM> includes the invisible light image sensor 22b that acquires an image in the invisible light wavelength range and the visible light image sensor 22a that acquires an image in the visible light wavelength range, respectively. However, the image sensor configuration is not limited thereto. As described above, when an infrared light source is used as the invisible light source 2b for reading the subject, it becomes possible to use an image sensor made of silicon having sensitivity in the infrared wavelength range.

<FIG> is a diagram illustrating the configuration of the reading unit <NUM> of the scanner <NUM> according to the second embodiment. As illustrated in <FIG>, the scanner <NUM> includes a general silicon image sensor 22c of three liens of R, G, and B that serves as both the visible light image sensor 22a and the invisible light image sensor 22b.

<FIG> is a graph illustrating an example of the spectral sensitivity characteristics of the image sensor 22c illustrated in <FIG>. As illustrated in <FIG>, a general silicon image sensor has spectral sensitivity in the infrared wavelength range in addition to the visible wavelength ranges of blue, green, and red. For example, when infrared light is used as invisible light and white light is used as visible light, a general image sensor can be used as the visible light image sensor 22a and the invisible light image sensor 22b. Thus, the sensor configuration can be simplified, while reading of the invisible image and the visible image of the subject can be enabled.

The image sensor illustrated in <FIG> has sensitivity in both the visible light wavelength range and the invisible light wavelength range in each of the R, G, and B pixels. Therefore, in this configuration, when reading a subject, a visible image and an invisible image can be acquired by irradiation by either the visible light source 2a or the invisible light source 2b. At this time, the same image sensor can be used, and the sensor configuration are be simple.

Next, a description is given of a third embodiment.

The third embodiment is different from the first embodiment or the second embodiment in that a visible image is used as a determining factor for a restoration target for the invisible image. In the following description of the third embodiment, descriptions of the same parts as those in the first and second embodiments are omitted, and differences from the first or second embodiment are described.

<FIG> is a block diagram illustrating a functional configuration of the image processing unit <NUM> according to the third embodiment. In the present embodiment, as illustrated in <FIG>, the image restoration unit <NUM> includes a first image information determination unit 21b and a second image restoration unit 21c instead of the first image restoration unit 21a of the first embodiment.

The first image information determination unit 21b determines, based on a visible image, the information that should not be lost as information to be restored (restoration target), and generates a first determination result.

The second image restoration unit 21c generates a second output image in which the information lost in the invisible image is restored based on the first determination result.

Using a visible image as a determination factor for the restoration target to be restored in the invisible image in this way, an image that has unintentionally disappeared can be accurately restored.

<FIG> are diagrams illustrating an example of image restoration processing. <FIG> illustrates an example of a paper document printed on paper having a color (pale blue, etc.) background pattern. Specifically, characters (text) representing a date and an amount of money are printed with black toner containing a specific component such as carbon, and a receipt mark is stamped with red ink.

In addition, the text TXT1 representing "important" is added by handwriting at the upper right corner of the document illustrated in <FIG>, and the text TXT1 is written with the black material that is different from black toner and does not contain the specific component. The text TXT1 representing "important" is information indicating the importance and priority of the document, and disappearance is not desirable.

<FIG> illustrates an invisible image of the document illustrated in <FIG>. In the image reading by the invisible light emitted from the invisible light source 2b, the background pattern, the stamped mark of receipt, and the text TXT1 representing "important" are deleted, as illustrated in <FIG>.

<FIG> illustrates a visible image of the document illustrated in <FIG>. In contrast to the image reading by invisible light, in the image reading by the visible light emitted from the visible light source 2a, the background pattern, the stamped mark of receipt, and the text TXT1 representing "important" are not erased but remain as illustrated in FIG.

<FIG> illustrates determination information. The first image information determination unit 21b determines information that should not be lost from the visible image illustrated in <FIG> and generates the first determination result. Then, the first image information determination unit 21b outputs an image illustrated in <FIG> as the first determination result. <FIG> illustrates an example in which only black information of the image is extracted, for example.

A detailed description is given of the determination, by the first image information determination unit 21b, of whether or not the information is set as the restoration target. The first image information determination unit 21b determines, for example, whether or not at least one of hue and saturation is equal to or greater than a first threshold value, thereby determining the necessity of restoration of an image in an invisible image.

As illustrated in <FIG>, the spectral reflection properties of a cyan (C) toner image, a magenta (M) toner image, and a yellow (Y) toner image formed on white paper are different in the visible wavelength range, and the C, M, and Y toner images are visually recognized as having different colors. Further, the black (CMY) toner produced by the composite of C toner, M toner, and Y toner exhibits low reflectance in the visible wavelength range (wavelength around <NUM> to <NUM>) and is recognized as black. In the near-infrared light range (wavelength <NUM> and greater), the black (CMY) toner exhibits high reflectance and is recognized as white. On the other hand, as illustrated in <FIG>, the black (K) toner exhibits low reflectance over the entire wavelength range, and is recognized as black. That is, even though both colors are visually recognized as black, in the near-infrared wavelength range, the black (K) is read as black texts or images, but the black (CMY) is read as white and disappears as information.

Therefore, the first image information determination unit 21b of the present embodiment determines that the information determined as being "black" is "information that is unintentionally going to disappear" based on information of at least one of hue and saturation contained in the visible image.

The "information that is unintentionally going to disappear" determined in this way is used for restoring information by, for example, adding, correcting, or replacing information to the invisible image, using the corresponding visible image. Even when the "information that is unintentionally going to disappear" is black (K) information, there is no effect because an additional process is performed to the information that has not been lost.

This configuration enables accurate detection of an image that may be unintentionally lost.

<FIG> illustrates the invisible image plus a restored image of the document illustrated in <FIG>. In the present embodiment, the second image restoration unit 21c performs OCR for, for example, each image, using the invisible image illustrated in <FIG> and the determination information illustrated in <FIG>. Accordingly, the scanner <NUM> can acquire the restored image illustrated in <FIG> as a second output image. This configuration enables restoration of an image that may be unintentionally lost while removing unnecessary information.

As described above, according to the present embodiment, using the image information of the visible image as a determination factor for a restoration target to be restored in the invisible image, an image that may be unintentionally lost can be accurately detected and restored.

Note that the first image information determination unit 21b determines whether or not at least one of hue and saturation is equal to or greater than the first threshold value so as to determine the information that should not to be lost based on the visible image, as the basis for determining the necessity of restoration of an image in an invisible image. However, the basis of this determination is not limited thereto. For example, the first image information determination unit 21b may determine whether the information is either black, which has low brightness, or gray, which has low brightness and low saturation, as the basis of this determination.

<FIG> are diagrams illustrating another example of the image restoration process. <FIG> illustrates an example of a paper document printed on paper having a color (pale blue, etc.) background pattern. Specifically, characters (texts) representing a date and an amount of money are printed with black toner containing a specific component such as carbon.

In addition, the text TXT1 representing "important" is added by handwriting at the upper right corner of the document illustrated in <FIG>, and the text TXT1 is written with a black material which is different from black toner and does not contain a specific component. The text TXT1 representing "important" is information indicating the importance and priority of the document, and disappearance is not desirable.

The text TXT1 representing "important", which is printed in black by color ink composition, is erased in the image reading by the invisible light emitted from the invisible light source 2b but is restored by the restoration process according to the present disclosure. The color background pattern of the paper (pale blue, etc.) is not restored because the color background pattern has hue and saturation equal to or higher than the threshold. As a result, as illustrated in <FIG>, only the background pattern is removed, and a restored image thus obtained is a second output image whose recognizability is significantly improved.

Next, a description is given of a fourth embodiment.

The fourth embodiment is different from the first embodiment to the third embodiment in that both a visible image and an invisible image are used as the determining factors for a restoration target for the invisible image. In the following description of the fourth embodiment, descriptions of the same parts as those in the first to third embodiments are omitted, and differences from the first to third embodiment are described.

<FIG> is a block diagram illustrating a functional configuration of the image processing unit <NUM> according to the fourth embodiment.

As illustrated in <FIG>, the image restoration unit <NUM> of the present embodiment includes a second image information determination unit 21d and a third image restoration unit 21e, instead of the first image restoration unit 21a described in the first embodiment.

The second image information determination unit 21d determines, as the restoration target, the information that should not be lost based on a visible image and an invisible image, and generates a second determination result.

The third image restoration unit 21e restores the information lost in the invisible image based on the second determination result generated by the second image information determination unit 21d, and generates a third output image.

Using the visible image and the invisible image as the determination factors for a restoration target to be restored in an invisible image in this way, an image that may be unintentionally lost can be restored accurately. Further, the data amount in the image restoration process can smaller.

<FIG> are diagrams illustrating an example of the image restoration process according to the fourth embodiment. <FIG> illustrates an example of a paper document printed on paper having a color (pale blue, etc.) background pattern. Specifically, characters (text) representing a date and an amount of money are printed with black toner containing a specific component such as carbon.

<FIG> illustrates an invisible image of the document illustrated in <FIG>. In the image reading by the invisible light emitted from the invisible light source 2b, the background pattern and the text TXT1 representing "important" are deleted, as illustrated in <FIG>.

<FIG> illustrates a visible image of the document illustrated in <FIG>. In contrast to the image reading by the invisible light, in the image reading by the visible light emitted from the visible light source 2a, the background pattern and the text TXT1 representing "important" are not erased but remain as illustrated in FIG.

<FIG> illustrates determination information. Using the invisible image illustrated in <FIG> and the visible image illustrated in <FIG>, the second image information determination unit 21d extracts, for example, information that is black and is not common to the two images. Then, the second image information determination unit 21d determines the black and non-common portion as the information that should not be lost, and generates a second determination result. Then, the second image information determination unit 21d outputs an image illustrated in <FIG> as the second determination result. <FIG> illustrates an example in which, for example, the information that is black and not common to the two images is extracted. The image illustrated in <FIG> that is output as the determination information includes a smaller amount of information than the example of <FIG>.

A detailed description is given of the determination, by the second image information determination unit 21d, of whether or not the information is set as the restoration target. The second image information determination unit 21d determines whether or not at least one of hue and saturation is equal to or greater than the first threshold value, so as to determine the information that should not to be lost based on the invisible image and the visible image, for determining the necessity of restoration of an image in the invisible image. Specifically, the second image information determination unit 21d determines that the information determined as being "black" is "information that is unintentionally going to disappear" based on information of at least one of hue and saturation contained in the visible image.

The "information that is unintentionally going to disappear" determined in this way is used for restoring information by, for example, adding information to the invisible image, correcting the invisible image, or replacing information in the invisible image, using the corresponding visible image. Even when the "information that is unintentionally going to disappear" is black (K) information, there is no effect because an additional process is performed to the information that has not been lost.

<FIG> illustrates the invisible image plus a restored image of the document illustrated in <FIG>. In the present embodiment, the third image restoration unit 21e performs OCR for, for example, each image, using the invisible image illustrated in <FIG> and the determination information illustrated in illustrated in <FIG>. Accordingly, the scanner <NUM> can acquire the restored image illustrated in <FIG> as a third output image. This configuration enables restoration of an image that may be unintentionally lost while removing unnecessary information. Further, the determination information can have a smaller data amount.

As described above, according to the present embodiment, using the visible image the invisible image as the determination factors for the restoration target to be restored in the invisible image, an image that may be unintentionally lost can be more accurately detected and restored.

Note that the second image information determination unit 21d determines whether or not at least one of hue and saturation is equal to or greater than the first threshold value so as to determine the necessity of restoration of the image in the invisible image. However, the basis of this determination is not limited thereto. For example, the second image information determination unit 21d may determine whether the information is either black, which has low brightness, or gray, which has low brightness and low saturation, as the basis of this determination.

A description is given of a fifth embodiment of the present disclosure.

The fifth embodiment is different from the first to fourth embodiments in that the color designated by the user is restored. In the following description of the fifth embodiment, descriptions of the same parts as those in the first to fourth embodiments are omitted, and differences from the first to fourth embodiment are described.

<FIG> is a block diagram illustrating a functional configuration of the image processing unit <NUM> according to the fifth embodiment.

As illustrated in <FIG>, the image restoration unit <NUM> of the present embodiment further includes a restoration color designation unit 21f for an invisible image in addition to the configuration illustrated in <FIG>.

The restoration color designation unit 21f enables freely changing information or image to be restored by the image restoration unit <NUM>, thereby preventing the loss of important information of any color (such as a vermilion imprint or a receipt sign with a color pen).

More specifically, the restoration color designation unit 21f enables freely changing the first threshold value when the first image information determination unit 21b determines whether or not at least one of hue and saturation is equal to or higher than the first threshold value. Further, the restoration color designation unit 21f enables freely changing the color between black and gray when the first image information determination unit 21b determines whether the information is either black, which has low brightness, or gray, which has low brightness and low saturation, as the basis of this determination.

The color designated by the restoration color designation unit 21f may be automatically designated by the controller <NUM>, or the restoration color designation unit 21f may be configured so that a user can freely designate a color.

<FIG> are diagrams illustrating an example of the image restoration process. <FIG> illustrates a document in which a date and an amount of money are printed, with black toner containing a specific component such as carbon, on paper having a color background pattern. Further, on the document illustrated in <FIG>, a strikethrough deleting the amount and corrected characters are written with a black material that is different from black toner and does not contain the specific component. In addition, a red correction mark is stamped (e.g., red ink). In this document, the information except the background pattern is necessary information, and losing such information is not desirable.

<FIG> illustrates an invisible image of the document illustrated in <FIG>. In the image reading by the invisible light emitted from the invisible light source 2b, in addition to the background pattern and the stamped mark of receipt, the necessary information, that is, the strikethrough and the corrected characters are deleted as illustrated in <FIG>.

<FIG> illustrates a visible image of the document illustrated in <FIG>. In contrast to the image reading by the invisible light, in the image reading by the visible light emitted from the visible light source 2a, the background pattern, the stamped mark of receipt, and the necessary information (the strikethrough and the corrected characters) are not erased but remain as illustrated in <FIG>.

In this example, it is assumed that the restoration color designation unit 21f designates, for example, black and red as restoration colors. When designating black and red as the restoration colors in the restoration color designation unit 21f, the restoration color designation unit 21f instructs the first image information determination unit 21b to retain or restore black and red information.

<FIG> illustrates determination information DI1. As illustrated in <FIG>, in response to the instruction of designation of black and red as the restoration colors, the first image information determination unit 21b extracts, using the visible image illustrated in <FIG>, black information and red information as the determination information DI1 illustrated in <FIG>. Then, the first image information determination unit 21b outputs the determination information DI1 as the first determination result.

<FIG> illustrates determination information DI2. In response to the instruction of designation of black and red as the restoration colors, the first image information determination unit 21b removes, from the determination information DI1 illustrated in <FIG>, the invisible image illustrated in <FIG>. Then, the first image information determination unit 21b outputs the determination information DI2 illustrated in <FIG> as the first determination result.

<FIG> illustrates an invisible image plus a restored image of the document illustrated in <FIG>. In the present embodiment, the second image restoration unit 21c performs OCR for, for example, each image, using a) the determination information DI1 illustrated in <FIG>) the invisible image illustrated in <FIG> and the determination information DI1 illustrated in <FIG>, or c) the invisible image illustrated in <FIG> and the determination information DI2 illustrated in <FIG>. As a result, the restored image illustrated in <FIG> can be obtained. This configuration enables restoration of a given color image that may be unintentionally lost while removing unnecessary information.

Thus, according to the present embodiment, the controller <NUM> or the user can freely change the color to be restored, and accordingly, the loss of important information of a color (such as a vermilion imprint or a receipt sign with a color pen).

In the present embodiment, the description has been given of an example of adding the restoration color designation unit 21f for the invisible image to the configuration illustrated in <FIG>, but the present disclosure is not limited thereto. <FIG> is a block diagram illustrating a modified example of the functional configuration of the image processing unit <NUM>. As illustrated in <FIG>, the image restoration unit <NUM> may include the restoration color designation unit 21f for an invisible image in addition to the configuration illustrated in <FIG>.

This configuration also enables restoration of a given color image that may be unintentionally lost while removing unnecessary information.

The sixth embodiment is different from the first embodiment to the fifth embodiment in that a visible image is also output to the outside. In the following description of the sixth embodiment, descriptions of the same parts as those in the first to fifth embodiments are omitted, and differences from the first to fifth embodiment are described.

Each of the above-described embodiments concerns the configuration to restore or correct the disappeared information and image in the invisible image and output the corrected image as the first output image or the second output image.

In the present embodiment, a visible image can be output to the outside. With this configuration, for example, a state before restoration or correction according to the present disclosure, that is, a visual recognition image of a received or processed document can be used for storing, confirmation, or evidence.

<FIG> is a block diagram illustrating a functional configuration of the image processing unit <NUM> according to the sixth embodiment. As illustrated in <FIG>, the image restoration unit <NUM> of the present embodiment further outputs a visible image as a fourth output image in the configuration illustrated in <FIG>.

According to the application example described with reference to <FIG>, the restored image illustrated in <FIG> is finally output as the second output image. However, strictly speaking, the background pattern has been removed from the original document illustrated in <FIG>. Accordingly, although the second output image is suitable for OCR and readability improvement, for example, the admissibility as an evidence and legal handling of the document may not be sufficient.

Therefore, the image restoration unit <NUM> of the present embodiment can output the original image which is a visible image of the document of <FIG>, so as to supplement and eliminate insufficiency of such effects.

As described above, according to the present embodiment, in addition to the invisible light corrected image, the visible image can be used for storing, confirmation, evidence, or the like.

In the present embodiment, the description has been given of an example of the configuration to output a visible image as a fourth output image in the configuration illustrated in <FIG>, but the present disclosure is not limited thereto. <FIG> is a block diagram illustrating a modified example of the functional configuration of the image processing unit <NUM>. As illustrated in <FIG>, the image restoration unit <NUM> may further output a visible image as a fourth output image in the configuration illustrated in <FIG>.

Even in such a case, since the original image which is a visible image of the document can be output, the above-mentioned insufficiency of effects can be supplemented and eliminated.

Note that in the embodiments described above, the image processing apparatus is applied to a multifunction peripheral having at least two of copying, printing, scanning, and facsimile functions. Alternatively, the image processing apparatus may be applied to, e.g., a copier, a printer, a scanner, or a facsimile machine.

The above-described embodiments are illustrative and do not limit the present invention. Thus, numerous additional modifications and variations are possible in light of the above teachings. For example, elements and/or features of different illustrative embodiments may be combined with each other and/or substituted for each other within the scope of the present invention.

Claim 1:
An image processing apparatus comprising:
an invisible light source (2b) configured to emit invisible light to a subject;
a visible light source (2a) configured to emit visible light to the subject;
an invisible image reading sensor (22b) configured to read invisible reflection light reflected from the subject to acquire an invisible image, the invisible reflection light being a part of the invisible light emitted from the invisible light source (2b);
a visible image reading sensor (22a) configured to read visible reflection light reflected from the subject to acquire a visible image, the visible reflection light being a part of the visible light emitted from the visible light source (2a), wherein the visible image comprises a specific component including color information that is removed in the invisible image,
characterized by
an image restoration unit (<NUM>) configured to restore the specific component in the invisible image acquired by the invisible image reading sensor (22b).