Image processing apparatus, image processing method, computer-readable medium and computer data signal

An image processing apparatus includes an image receiving unit, an extraction unit, a determination unit, an area conversion unit and an image concealing unit. The image receiving unit receives an image. The extraction unit extracts a portion having been additionally written in the image received by the image receiving unit. The determination unit determines an area to be concealed in the received image based on the additionally written portion extracted by the extraction unit. The area conversion unit converts the area determined by the determination unit. The image concealing unit conceals the area determined by the determination unit in the received image using an area obtained by the conversion of by the area conversion unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2008-180107 filed on Jul. 10, 2008.

BACKGROUND

1. Technical Field

The invention relates to an image processing apparatus, an image processing method, a computer-readable medium and a computer data signal.

2. Related Art

Documents possessed by an administrative organ, etc., have been made public. However, those documents may contain information to be concealed from the viewpoint of protection of personal information, etc. Then, when those document are made to public, parts where the information to be concealed is described are filled in with black (black out).

SUMMARY

According an aspect of the invention, an image processing apparatus includes an image receiving unit, an extraction unit, a determination unit, an area conversion unit and an image concealing unit. The image receiving unit receives an image. The extraction unit extracts a portion having been additionally written in the image received by the image receiving unit. The determination unit determines an area to be concealed in the received image based on the additionally written portion extracted by the extraction unit. The area conversion unit converts the area determined by the determination unit. The image concealing unit conceals the area determined by the determination unit in the received image using an area obtained by the conversion of by the area conversion unit.

DETAILED DESCRIPTION

Now, referring to the accompanying drawings, exemplary embodiments of the invention will be described below.

FIG. 1is a conceptual module block diagram of a configuration example according to an exemplary embodiment of the invention.

A “module” refers to a generally and logically detachable software component (computer program), generally and logically detachable hardware component, and the like. Therefore, the “module” in exemplary embodiments means not only a module in a computer program, but also a module in the hardware configuration. The exemplary embodiments also serve as description of a computer program, a system, and a method. For the convenience of the description, “store”, “cause to store” and its equivalent words are used. If the exemplary embodiment is a computer program, these words are used to mean storing something in storage or controlling so as to store something in storage. Modules have an substantially one-to-one correspondence relation with functions. However, in implementation, one module may be constituted by one program, two or more modules may be constituted by one program, or two or more programs may make up one module. Two or more modules may be executed on one computer or one module may be executed on two or more computers in a distributed or parallel environment. One module may contain any other module. In the following description, the term “connection” is used to mean not only physical connection, but also logical connection (data transfer, command, reference relationship between data, etc.).

Also, a system or an apparatus is not only provided by connecting plural computers, hardware components, devices, etc., via a communication line such as a network (containing peer-to-peer communication connection), etc., but also implemented as one computer, hardware component, device, etc. The words “apparatus” and “system” are used as synonymous with each other. The word “predetermined” refers to a timing, a situation, a state, etc., before a target process; it is used to mean “being determined” in response to the situation, the state, etc., at that time or in response to the situation, the state, etc., so far before processing according to the exemplary embodiments is started or even after the processing according to the exemplary embodiments is started

As shown inFIG. 1, this exemplary embodiment has an image receiving module110, an additionally-written-portion extraction module120, a mask area determination module130, a mask reason determination module140, a mask area conversion module150, a supplemental image generation module160, a combining module170, and a print module180.

The image receiving module110is connected to the additionally-written-portion extraction module120and the combining module170. The image receiving model receives a target image and passes the image (hereinafter, may be refereed to as an “original image”) to the additionally-written-portion extraction module120and the combining module170. The expression “receiving an image” contains reading an image with a scanner, receiving an image by a facsimile machine, reading an image from an image database, etc., and the like. The target image is a document to be made to public, and additional writing is made in a portion to be concealed. The number of images may be one or may be two or more. Additional writing is made with a pen having semitransparent color ink other than black, for example. It is assumed that if the ink is put on black characters, the characters can be seen. The pen may be one called a marker pen, a highlighter or the like. The portion to which additional writing is made with the marker pen, etc., is particularly referred to as a marker image. Additional writing may be made with a red ball-point pen.

The additionally-written-portion extraction module120is connected to the image receiving module110and the mask area determination module130. The additionally-written-portion extraction module120receives the original image from the image receiving module110, extracts a portion that has been additionally written to the original image, and passes the additionally written portion to the mask area determination module130. The extraction process is performed, for example, by extracting a portion having an additionally written color, by making a comparison between the received original image and an image in which additionally writing has not been made (for example, by performing EOR logical operation therefor) to extract the additionally written portion, or the like.

The additionally-written-portion extraction module120may extract, using a first threshold value, a first area including at least a part of a portion having been additionally written in the image received by the image receiving module110. Then, the additionally-written-portion extraction module120may calculate a second threshold value based on a feature in a third area that is arranged along with the first area, which is extracted using the first threshold value. The additionally-written-portion extraction module120may further extract, using a second threshold value, a second area including the portion having been additionally written in the image received by the image receiving module110. Furthermore, in order to deal with the case where black pixels (for example, a line segment) in the image breaks additional writing (particularly, a marker image), the calculation of the second threshold value may use a pixel density and/or a line segment amount in the third area as the feature in the third area. The expression the “third area that is arranged along with the first area, which is extracted using the first threshold value,” is used to mean an area surrounding the first area, which is extracted using the first threshold value. More specifically, this expression is used to mean an area adjacent to the first area, which is extracted using the first threshold value, in a character string direction (a long-side direction of a circumscribed rectangle of the extracted character string), namely, to mean an area in the back and forth direction with respect to an additionally writing direction (a line drawing direction) of the first area, which is extracted using the first threshold value. A specific example thereof will be described later with reference toFIG. 4.

The line segment amount, which is used in calculating the second threshold value, may be the number of line segments in the third area that extend in substantially parallel to a short side of the first area. That is, the line segment amount may be the number of line segments extending parallel to (an inclination of about 30 degrees is permitted) the shorter side of the longitudinal and lateral sides of the circumscribed rectangle of the first area, which include at least a part of the additionally written portion and is extracted using the first threshold value (i.e. line segments perpendicular to the long side thereof (an inclination of about ±30 degrees is permitted)). A specific example thereof will be described later with reference toFIG. 7.

The second threshold value is a value that can be used to extract a larger area than the first area, which includes at least a part of the additionally written portion and is extracted using the first threshold value. Extracting, using the second threshold value, of the second area including the additionally written portion may be performed an certain extraction process using the second threshold value for the third area, which is arranged along with the first area extracted using the first threshold value. There is a possibility that the additionally written portion may be broken and the first area, which is extracted using the first threshold value, may only include a part of the additionally written portion. Therefore, in order to extract the additionally written portion obtained by extending the first area and to reduce a processing area, the extraction process may be performed only for a portion in which the additionally written portion is possibly included.

The mask area determination module130is connected to the additionally-written-portion extraction module120and the mask reason determination module140. The mask area determination module130determines an area to be concealed (which will be hereinafter also referred to as a “concealment area” or a “mask area”) in the image based on the second area, which includes the additionally written portion and is extracted by the additionally-written-portion extraction module120. Since the additionally written portion is often handwritten, it often has a non-rectangle shape and the portion to be concealed may protrude from the additionally written portion. Therefore, if the second area including the additionally written portion is concealed faithfully, a portion that is desired to be concealed may be unable to be concealed. In order to deal with this situation, the mask area determination module130extract rectangles circumscribing respective character image(s) or the like that are made up of black pixels in the second area including the additionally written portion and adopts an area obtained by integrating the circumscribed rectangles, as a mask area.

The mask reason determination module140is connected to the mask area determination module130and the mask area conversion module150. The mask reason determination module140receives the concealment area and the second area including the additionally written portion from the mask area determination module130, and determines a concealing reason based on information concerning manipulation of an image or a feature concerning the second area (the additionally written portion) extracted by the additionally-written-portion extraction module120. The mask reason determination module140passes the concealing reason and the concealment area to the mask area conversion module150. The mask reason determination module140has a count module141, a manipulator detection module142, a feature extraction module143, and a character recognition module144. The mask reason determination module140determines a concealing reason using any one or combination of the detection results of the above modules.

Examples of the concealing (non-disclosing) reason are as follows:

(1) Information that can identify a specific person (personal information)

(2) Information that impairs legitimate interests of a corporation (corporate information)

(3) Information that impairs safety of country, trust relationship with foreign countries, etc., (national security information)

(4) Information that has impact on public security and order (public security information)

(5) Information that relates to deliberation, discussion, etc. and possibly unjustifiedly impairs neutrality of decision making or possibly unjustifiedly causes disorder among people (deliberation/discussion information)

(6) Information that has impact on proper execution of work and business of an administrative organ, an independent administrative agency, etc., (work business information)

Also, information concerning the concealing reason may include a concealing reason or a concealing person (a person who makes additional writing, a person in charge who instruct the concealment or a department name).

The count module141and the manipulator detection module142may use information responsive to manipulator's manipulation or information that identifies a manipulator as the information concerning image manipulation. More specifically, different information are used as the information responsive to manipulator's manipulation, for a manipulation of specifying a reason when the manipulator uses this exemplary embodiment and for a manipulation of preparing, for each reason, plural copies of a document in which additional writing has been made and associating the reason with the number of times the exemplary embodiment is used (for example, associating a “reason1” with the first time). When this exemplary embodiment is used, an ID (Identification Data) card of a manipulator may be read as the information, which identifies the manipulator and an manipulator ID may be extracted therefrom. A responsive person ID may be extracted from a predetermined correspondence table between manipulator IDs and responsive person IDs or the like.

The feature extraction module143extracts color information of the second area (the additionally written portion) as the feature concerning the second area (the additionally written portion). In addition, the feature extraction module143may extract a form (a line width, a line type, a size, etc.) of the second area (the additionally written portion). The character recognition module144recognizes each character image in the concealment area determined by the mask area determination module130.

The mask area conversion module150is connected to the mask reason determination module140and the supplemental image generation module160. The mask area conversion module150receives the concealing reason and the concealment area from the mask reason determination module140, and converts the concealment area determined by the mask area determination module130. The mask area conversion module150passes the concealment reason and an area obtained by the conversion to the supplemental image generation module160. Based on sizes of plural concealment areas determined by the mask area determination module130, the mask area conversion module150may convert the concealment areas. For example, the mask area conversion module150calculates an average value of the sizes of the concealment areas is calculated, and unifies the sizes of the concealment areas into the size of the average value. The conversion of the sizes of the concealment areas will be described later with reference toFIGS. 16A and 16B.

The supplemental image generation module160is connected to the mask area conversion module150and the combining module170. The supplemental image generation module160receives the concealing reason and the concealment area or the area obtained by converting the concealment area from the mask area conversion module150, and generates an image indicating the concealing reason (hereinafter may be referred to as a “concealing reason image” or a “supplemental image”) using the concealing reason determined by the mask reason determination module140. The supplemental image generation module160passes the concealment area and the concealing reason image to the combining module170. The concealing reason image may contain an image indicating information that can identify a person who conceals (e.g., a person who makes additional writing). The supplemental image generation module160may set a different display form to each concealment area, in response to the size of each concealment area. For example, if the size of the concealment area is large enough to describe the concealing reason therein, the concealing reason may be displayed directly in the concealment area; otherwise, a symbol may be used instead and the meaning of the symbol may be displayed on an appendix. This determination is made by calculating a size of a display area based on the number of characters in the concealing reason and the sizes of the respective characters and comparing the size of the display area with the size of the concealment area. The supplemental image generation module160may vary a form (color, shape, etc.) of the concealment area depending on the concealing reason, for example, in such a manner that a reason1is displayed in red and a reason2is displayed in a circular shape. A specific example of the concealing reason image will be described later with reference toFIGS. 10A and 10B.

The combining module170is connected to the image receiving module110, the supplemental image generation module160, and the print module180. The combining module170conceals the concealment area determined by the mask area determination module130within the original image received by the image receiving module110and adds the image indicating the concealing reason generated by the supplemental image generation module160to the original image.

The combining module170may add an image indicating the concealing reason in accordance with a size of a blank area in the original image. That is, if a blank area having such a size as to allow the concealing reason to be displayed therein exists in the original image, the concealing reason is displayed in the original image; if no blank area having such a size as to allow the concealing reason to be displayed therein exists in the original image, the combining module170adds another image like an appendix (e.g., seeFIG. 10B) in which the concealing reasons are displayed (increases the number of pages). For example, the combining module170calculates in advance a size that allows the respective concealing reasons to be displayed. If the calculated size is larger than that of the blank area in the image or if there are plural concealing reasons to be displayed and a sum of the sizes of the plural concealing reasons is larger than the size of the blank area in the image, the combining module170generates another image as an appendix. The size mentioned here may be not only an area, but also longitudinal and lateral lengths that are required for display.

The print module180is connected to the combining module170, and outputs a composite image provided by the combining module170(an image to which a concealment process is performed and to which the concealing reason image is added) (for example, the print module180prints the image by a printer, etc.). This printed image is made to public. The outputting may be not only printing, but also displaying, transmitting to another facsimile machine through a communication line, storing in an image database, or the like.

According to another exemplary embodiment, the mask reason determination module140, the mask area conversion module150, and the supplemental image generation module160may be excluded. In this case, the mask area determination module130and the combining module170are connected. Also, according to further another exemplary embodiment, the mask reason determination module140and the supplemental image generation module160may be excluded. In this case, the mask area determination module130and the mask area conversion module150are connected, and the mask area conversion module150and the combining module170are connected. According to still another exemplary embodiment, the mask area conversion module150may be excluded. In this case, the mask reason determination module140and the supplemental image generation module160are connected.

FIG. 2is a flowchart showing a process example according to the exemplary embodiment of the invention.

At step S202, the image receiving module110receives an image in which additional writing has been made.

At step S204, the additionally-written-portion extraction module120extracts the additionally written portion (e.g., a second area) from the original image. A more detailed process example will be described later with reference toFIGS. 3 and 6.

At step S206, the mask area determination module130determines an area to be masked based on the first area extracted at step S204.

At step S208, the mask reason determination module140determines a mask reason.

At step S210, the mask area conversion module150determines as to whether or not it is necessary to convert the mask area. If the mask area conversion module150determines that it is necessary to convert the mask area, the process goes to S212; otherwise, the process goes to S214. For example, if mask areas have the same size (if mask areas are within a predetermined value range, those mask area may be determined as having the same size), the mask area conversion module150may determine that it is not necessary to convert the mask areas. Mask areas for which this determination is made may only be mask areas contained in the same line.

At step S212, the mask area conversion module150converts the mask area.

At step S214, the supplemental image generation module160determines as to whether or not it is necessary to generate a supplemental image. If the supplemental image generation module160determines that it is necessary to generate a supplemental image, the process goes to S216; otherwise, the process goes to S218. For example, the determination may be made in accordance with specification by a manipulator.

At step S216, the supplemental image generation module160generates a supplemental image.

At step S218, the combining module170combines the original image received at step S202and the mask area determined at step S206(or the mask area provided at step S212) or the supplemental image generated at step S216.

At step S220, the print module180prints the composite image provided at step S218.

FIG. 3is a flowchart showing a first process example of the additionally-written-portion extraction module120. If an image of a target document is read with a scanner and if an additionally written portion is particularly a maker image, it may become difficult to extract the additionally written portion particularly because of settings of the scanner, influence of image compression, and the like. That is, in order to suppress the case where concealment does not comply with intension of a person who makes additional writing, the process example according to the flowchart ofFIG. 3or6is executed.

At step S302, a first area including at least a part of an additionally written portion is extracted using a first threshold value. For example, an area having saturation that is equal to or greater than the first threshold value in an L*a*b* space is extracted. In an example shown inFIG. 4, a shaded area of “F,” “#,” “G,” and “H” portions is the additionally written portion.

At step S304, a pixel block (may be referred to as a “character image”) is clipped from the image. The pixel block contains at least a pixel area that continues in a four or eight-connectivity form. The pixel block also may contain a set of such pixel areas. The expression a “set of pixel areas” is used to mean that there are plural pixel areas each of which continues in the four-connectivity form and the plural pixel areas are close to each other. The pixel areas, which are close to each other, may be pixels areas that are separate in short distance, pixel areas obtained by projecting an image in a longitudinal or lateral direction and dividing the image at blank positions so as to clip characters from one line of sentence one character by one character, or the pixel areas obtained by clipping an image at given intervals.

One pixel block often forms an image of one character. However, it is not necessary that a pixel block is a pixel area that can be actually recognized as a character by a human being. A pixel block may be a part of a character, a pixel area not forming a character or the like, and thus may be any block of pixels.

At step S306, a target character image is determined, and a character image (may be referred to as a “peripheral character image”) that exists around the target character image is determined. For example, an overlap portion where the additionally written portion extracted at step S302overlaps with the character image clipped at step S304is adopted as the target character image. The peripheral character image is a character image that exists within a predetermined distance from the target character image. Further, a rectangle circumscribing the additionally written portion may be obtained, and a character image existing in the longitudinal direction of the circumscribed rectangle (in the example inFIG. 4, the lateral (horizontal) direction) may be adopted as the peripheral character image. In the example inFIG. 4,402to405designate target character images, and401and406designate peripheral character images.

At step S308, a density of each peripheral character image (a ratio of black pixels of each character image to an area of a rectangle circumscribing each character image) is calculated.

At step S310, the first threshold value is changed in accordance with the density calculated at step S308, to obtain a second threshold value. More specifically, if the density is high, the second threshold value is lowered (namely, direction in which a second area including the additionally written portion can be made larger); if the density is low, the second threshold value is set to be close to the original value.

At step S312, a second area including the additionally written portion is extracted from the original image using the second threshold value calculated at step S310. A process similar to that at step S302may be performed. For example, an area having saturation that is equal to or greater than the second threshold value in an L*a*b* space is extracted from the original image. Alternatively, an extraction process may be performed only for the portions of the peripheral character images. For example, an area having saturation that is equal to or greater than the second threshold value in an L*a*b* space is extracted from the peripheral character images, and the extracted area and the first area are combined to obtain the second area. In the example inFIG. 5, a determined mask area500(containing the target character images402to405) is shown.

FIG. 6is a flowchart showing a second process example of the additionally-written-portion extraction module120. For steps that are similar to the steps in the flowchart example shown inFIG. 3, such steps are clearly indicated and will not described again. Particularly, the second process example is an example that deals with the case where there is a line segment that breaks a marker image, a white portion is generated in the surrounding of the line segment due to processing such as character enhancement, and it is difficult to connect divided maker images even if postprocessing such as a process of repeating expansion and shrinkage is performed.

Step S602is similar to step S302.

At step S604, a line direction of the original image (vertical writing or horizontal writing) is determined. For example, the whole original image is projected, and a direction in which high projection exists is the direction of the character string (the expression “high projection exists” means that there appears a peak, that there is a clear difference in level, or that there is a large difference from an average). The line direction may be determined based on an aspect ratio of the additionally written portion.

Step S606is similar to step S304.

Step S608is similar to step S306.

At step S610, a line segment amount of the peripheral character image is extracted. Specifically, the number of line segments in the direction crossing the line direction determined at step S604is extracted. For example, when the peripheral character image is “H” as shown inFIG. 7A, if the line direction is horizontal, the peripheral character image “H” is projected in the longitudinal direction and the fact that the number of line segments is two is extracted as shown inFIG. 7B. Alternatively, the line segment amount may be extracted by performing matching processing with a predetermined direction pattern.

At step S612, the first threshold value is changed in accordance with the line segment amount extracted at step S610to calculate the second threshold value. More specifically, if the line segment amount is large, the threshold value is lowered (namely, direction in which the second area including the additionally written portion can be made larger); if the line segment amount is small, the second threshold value is set to be close to the original value.

Step S614is similar to step S312.

FIGS. 8A to 8Care explanatory views showing an example of an image to be processed by the mask area determination module130.

The mask area determination module130performs a process of shaping the second area (the additionally written portion) extracted by the additionally-written-portion extraction module120to a rectangle. In this processing, it is solved that an additional written portion protrudes into a part of a character image not to be concealed and that a part of a character image is not sufficiently concealed. For example, as shown inFIG. 8A, a target image800contains an additionally written portion801. When a rectangle circumscribing the additionally written portion801is extracted, a mask area811shown inFIG. 8Bis obtained. The mask area811protrudes from the character images “H”, “I”, “J”, “K” and “L” to be concealed, in upper, left, and right directions. Then, the mask area determination module130extracts a rectangle circumscribing only character images each of which is contained in a predetermined ratio (for example, 50%) or more in the mask area811, and determines a mask area821as shown inFIG. 8C.

Examples of a document and a printed document to which the exemplary embodiment is applied will be discussed withFIGS. 9,10A and10B.

FIG. 9is an explanatory view showing an example of an image received by the image receiving module110.

A target image900contains additionally written portions911,912,913, and921. The additionally written portions911to913are areas marked with a marker pen having a certain color, but the additionally written portion921is an area surrounded by a marker pen having a different color from the certain color of the additionally written portions911, etc. The additionally written portions911to913covers personal information, and the additionally written portion921surrounds deliberation/discussion information.

FIGS. 10A and 10Bare explanatory views showing an example of a composite image provided by the combining module170(containing a supplemental image generated by the supplemental image generation module160).

In response to the target image900, an output image1000and an output supplemental image1050are printed. The output image1000contains mask areas1011,1012,1013, and1021. The mask areas1011to1013are images each indicating that the concealing reason is “A”, and the mask area1021is an image indicating that the concealing reason is “E: deliberation/discussion information”. The output supplemental image1050indicates what reason “A” and the like are. Unlike the mask area1021, each of the mask areas1011to1013does not have a sufficient size to show the whole concealing reason. Thus, each of the mask areas1011to1013only shows a symbol, and the output supplemental image1050showing the meanings of the respective symbols is added. Since the output image1000does not have a blank area in which the image shown in the output supplemental image1050can be inserted, the output supplemental image1050is added.

The mask reason determination module140determines the concealing reason. For example, as shown inFIG. 9, a person who makes additionally writing uses additional writing colors properly according to the concealing reasons. The mask reason determination module140uses such information to determine the concealing reasons.

For example, it is assumed that plural copies of a document is prepared, that additional writing is made in one document only for one concealing reason and that additional writing is made in another of the copies only for another concealing reason. It is further assumed that thereafter, the reasons are determined according to the scanning order. For example, a document to which additional writing is made for reason A is read first, and a document to which additional writing is made for reason B is read second. In this case, the count module141detects the number of times the scanning is performed, and determines the reason using a number-of-time/reason correspondence table1100.FIG. 11is an explanatory view showing a data structure example of the number-of-time/reason correspondence table1100. The number-of-time/reason correspondence table1100has a number-of-time column1101and a reason column1102. When using this exemplary embodiment, a manipulator may specify a concealing reason.

As another example, each manipulator may be in charge of a corresponding concealing reason. For example, Jack is in charge of reason A. In such a case, the manipulator detection module142detects a manipulator and determines a reason using a manipulator/reason correspondence table1200.FIG. 12is an explanatory view showing a data structure example of the manipulator/reason correspondence table1200. The manipulator/reason correspondence table1200has a manipulator column1201and a reason column1202.

As further another example, each concealing reason may be associated with a corresponding marker color of additional writing. For example, red is associated with a reason A. In such a case, the feature extraction module143extracts a color of the additionally written portion and determines a reason using a marker color/reason correspondence table1300.FIG. 13is an explanatory view showing a data structure example of the marker color/reason correspondence table1300. The marker color/reason correspondence table1300has a marker color column1301and a reason column1302. In addition to or in place of the marker color, a line width, a line type, a size, etc., of the additional write may be used. Particularly, in order to increase the number of compatible reasons, it is better to use another feature in combination rather than using colors that are different but similar to each other.

As still another example, characters in a mask area may be recognized independently of administration. For example, if a character string of “nation” or the like is contained, the concealing reason is a reason C. In such a case, the character recognition module144performs the character recognition process for the mask area and determines a reason using a character recognition result/reason correspondence table1400.FIG. 14is an explanatory view showing a data structure example of the character recognition result/reason correspondence table1400. The character recognition result/reason correspondence table1400has a character recognition result column1401and a reason column1402.

FIG. 15is a flowchart showing a process example of the mask area conversion module150and the combining module170. There may be a case that concealed characters is able to be inferred based on a size of a mask area (particularly, a name, etc., may be easily inferred). This processing is performed to deal with such a situation.

At step S1502, the mask area conversion module150clips a character string from the original image. Similar processing to that at step S604shown in the example inFIG. 6may be performed or the result of step S604may be used. In an example inFIG. 16A, three character strings are clipped from a target image1600.

At step S1504, the mask area conversion module150clips character images from a character string where a mask area exists. Similar processing to that at step S304shown in the example inFIG. 3may be performed or the result of step S304may be used. In the example inFIG. 16A, character images are clipped from first and second character strings where mask areas1611and1612exist. That is, character images of “A” to “J” are clipped from the first character string, and character images “K” to “S” are clipped from the second character string.

At step S1506, the mask area conversion module150measures a width of each mask area and calculates an average value of the widths. In the example inFIG. 16A, the mask area conversion module150measures a width of the mask area1611(three characters) and a width of the mask area1612(two characters), and calculates an average value of the widths.

At step S1508, the mask area conversion module150converts the width of each mask area into the average value calculated at step S1506. That is, the mask area conversion module150makes the sizes of the mask areas become the same. In the example inFIGS. 16A and 16B, the mask area1611is converted into a mask area1661, and the mask area1612is converted into a mask area1662. The mask areas1661and1662have the same width.

At step S1510, the combining module170adjusts spacing between the character images outside the mask area so as to become a character string having a length equal to the character string of the original image. In an example inFIG. 16B, the spacing between the character images in the first character string is lengthened, and the spacing between the character images of the second character string is shortened. Accordingly, the length of each character string in an output image1650is not changed from that in the original image.

At step S1512, the combining module170moves the character images in accordance with the spacing adjusted at step S1510.

Here, the average value of the widths of the mask areas is calculated. However, a minimum value of the widths of the mask areas, the maximum value of the widths of the mask areas or the like may be used. Mask area conversion is performed for all mask areas, but may be performed for each character string.

A hardware configuration example of the exemplary embodiment of the invention will be discussed with reference toFIG. 17. The configuration shown inFIG. 17is made up of a personal computer (PC), etc., for example, and shows a hardware configuration example including a data reading section1717such as a scanner and a data output section1718such as a printer.

A CPU (Central Processing Unit)1701is a control section for executing processing in accordance with a computer program describing the execution sequences of the various modules described above in the exemplary embodiment, namely, the additionally-written-portion extraction module120, the mask area determination module130, the mask reason determination module140, the mask area conversion module150, the supplemental image generation module160, the combining module170, etc.

ROM (Read-Only Memory)1702stores programs, operation parameters, etc., used by the CPU1701. RAM (Random Access Memory)1703stores programs used in execution of the CPU1701, parameters changing appropriately in the execution of the CPU1701, and the like. They are connected by a host bus1704implemented as a CPU bus, etc.

The host bus1704is connected to an external bus1706such as a PCI (Peripheral Component Interconnect/Interface) bus through a bridge1705.

A keyboard1708and a pointing device1709such as a mouse are input devices operated by the manipulator. A display1710is implemented as a liquid crystal display, a CRT (Cathode Ray Tube), or the like for displaying various pieces of information as text and image information.

An HDD (Hard Disk Drive)1711contains a hard disk and drives the hard disk for recording or playing back a program and information executed by the CPU1701. The images received by the image receiving module110, the composite images provided by the combining module170, etc., are stored on the hard disk. Further, various computer programs, such as various data processing programs, are stored on the hard disk.

A drive1712reads data or a program recorded on a mounted removable recording medium1713such as a magnetic disk, an optical disk, a magneto-optical disk, or semiconductor memory, and supplies the data or the program to the RAM1703connected through the interface1707, the external bus1706, the bridge1705, and the host bus1704. The removable recording medium1713can also be used as a data record area like a hard disk.

A connection port1714is a port for connecting an external connection device1715and has a connection section of a USB, IEEE 1394, etc. The connection port1714is connected to the CPU1701, etc., through the interface1707, the external bus1706, the bridge1705, the host bus1704, etc. A communication section1716is connected to a network for executing data communication processing with an external system. A data reading section1717is a scanner, for example, and executes document read processing. A data output section1718is a printer, for example, and executes document data output processing.

The hardware configuration shown inFIG. 17shows one configuration example and the exemplary embodiment of the invention is not limited to the configuration shown inFIG. 17and may be any if it makes it possible to execute the modules described in the exemplary embodiment. For example, some modules may be implemented as dedicated hardware (for example, an application-specific integrated circuit (ASIC), etc.) and some modules may be included in an external system and may be connected via a communication line and further a plurality of systems shown inFIG. 17may be connected via a communication line so as to operate in cooperation with each other. The hardware configuration (embodiment) may be built in a copier, a fax, a scanner, a printer, a multiple function device (an image processing apparatus having the functions of any two or more of a scanner, a printer, a copier, a fax, etc.), etc.

In the exemplary embodiment described above, images of a document of horizontal writing are illustrated, but the exemplary embodiment may be applied to a document of vertical writing. Character images are illustrated as the target to be concealed, but may be any other image (pattern, photo, etc.).

The described program may be provided as it is stored on a recording medium or the program may be provided through communication line. In this case, for example, the described program may be grasped as the invention of a computer-readable recording medium recording a program.”

The expression “computer-readable recording medium recording a program” is used to mean a recording medium read by a computer recording a program, used to install and execute a program, to distribute a program, etc.

The record media include “DVD-R, DVD-RW, DVD-RAM, etc.,” of digital versatile disk (DVD) and standard laid down in DVD Forum, “DVD+R, DVD+RW, etc.,” of standard laid down in DVD+RW, read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), etc., of compact disk (CD), blue-ray disk, magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and programmable read-only memory (EEPROM), flash memory, random access memory (RAM), etc., for example.

The described program or a part thereof may be recorded in any of the described record media for retention, distribution, etc. The described program or a part thereof may be transmitted by communications using a transmission medium such as a wired network used with a local area network, a metropolitan area network (MAN), a wide area network (WAN), the Internet, an intranet, an extranet, etc., or a wireless communication network or a combination thereof, etc., for example, and may be carried over a carrier wave.

Further, the described program may be a part of another program or may be recorded in a recording medium together with a different program. It may be recorded as it is divided into a plurality of record media. It may be recorded in any mode if it can be restored, such as compression or encryption.

The exemplary embodiment can also be grasped as having the following configuration, which may be combined with the mask area conversion module150:

(A1) An image processing apparatus including:

an image receiving unit that receives an image;

an additionally written portion extraction unit that extracts a portion having been additionally written in the image received by the image receiving unit;

a concealment area determination unit that determines an area to be concealed in the image based on the additionally written portion extracted by the additionally written portion extraction unit;

a concealing reason determination unit that determines a concealing reason based on information concerning manipulation of the image or a feature concerning the additionally written portion extracted by the additionally written portion extraction unit;

an image concealing unit that conceals the area determined by the concealment area determination unit in the image; and

an image addition unit that adds information concerning the concealing reason determined by the concealing reason determination unit to the image.

(A2) The image processing apparatus described in (A1), wherein the concealing reason determination unit uses information responsive to manipulation of a manipulator or information the identifies the manipulator, as the information concerning manipulation of the image.
(A3) The image processing apparatus described in (A1) or (A2), wherein the concealing reason determination unit uses a form of the additionally written portion or a result of a character recognition process performed for the concealment area determined by the concealment area determination unit in combination, in addition to color information of the additionally written portion as the feature concerning the additionally written portion.
(A4) The image processing apparatus described in any one of (A1) to (A3), wherein the image addition unit adds an image showing information concerning the reason in response to a size of a blank area in the image.
(A5) The image processing apparatus described in any one of (A1) to (A4), wherein the image addition unit includes information that identifies a person who has concealed in the information concerning the reason.
(A6) An image processing program for causing a computer to function as:

an image receiving unit that receives an image;

an additionally written portion extraction unit that extracts a portion having been additionally written in the image received by the image receiving unit;

a concealment area determination unit that determines an area to be concealed in the image based on the additionally written portion extracted by the additionally written portion extraction unit;

a concealing reason determination unit that determines a concealing reason based on information concerning manipulation of the image or a feature concerning the additionally written portion extracted by the additionally written portion extraction unit;

an image concealing unit that conceals the area determined by the concealment area determination unit in the image; and

an image addition unit that adds information concerning the concealing reason determined by the concealing reason determination unit to the image.

(C1) An image processing apparatus including:

an image receiving unit that receives an image;

a first extraction unit that extracts, using a first threshold value, a first area including at least a part of a portion having been additionally written in the image received by the image receiving unit;

a threshold value calculation unit that calculates a second threshold value based on a feature in a third area that is arranged along with the first area extracted by the first extraction unit;

a second extraction unit that extracts, using the second threshold value calculated by the threshold value calculation unit, a second area including the portion having been additionally written in the image received by the image receiving unit;

a concealment area determination unit that determines an area to be concealed in the image, based on the second area extracted by the second extraction unit; and

an image concealing unit that conceals the area, determined by the concealment area determination unit, in the image.

(C2) The image processing apparatus described in (C1), wherein the threshold value calculation unit uses a pixel density in the third area as the feature in the third area.

(C3) The image processing apparatus described in (C1), wherein the threshold value calculation unit uses a line segment amount in the third area as the feature in the third area.

(C4) The image processing apparatus described in (C3), wherein the line segment amount used by the threshold value calculation unit is a number of line segments in the third area that extend in substantially parallel to a short side of the first area.

(C5) The image processing apparatus described in (C1), wherein the threshold value calculation unit uses a pixel density and a line segment amount in the third area as the feature in the third area.

(C6) The image processing apparatus described in (C5), wherein the line segment amount used by the threshold value calculation unit is a number of line segments in the third area that extend in substantially parallel to a short side of the first area.

(C7) The image processing apparatus according to any of (C1) to (C6), wherein

the second threshold value calculated by the threshold value calculation unit is a value that can be used to extract a larger area than the first area, which is extracted using the first threshold value, and

the second extraction unit performs a certain process for the third area using the second threshold value to extract the second area.

(C8) An image processing method including:

receiving an image;

extracting, using a first threshold value, a first area including at least a part of a portion having been additionally written in the received image;

calculating a second threshold value based on a feature in a third area that is arranged along with the extracted first area;

extracting, using the calculated second threshold value, a second area including the portion having been additionally written in the received image;

determining an area to be concealed in the image, based on the extracted second area; and

concealing the determined area in the image.

(C9) A computer-readable medium storing a program that causes a computer to execute image processing, the image processing including:

receiving an image;

extracting, using a first threshold value, a first area including at least a part of a portion having been additionally written in the received image;

calculating a second threshold value based on a feature in a third area that is arranged along with the extracted first area;

extracting, using the calculated second threshold value, a second area including the portion having been additionally written in the received image;

determining an area to be concealed in the image, based on the extracted second area; and

concealing the determined area in the image.