Image processing apparatus

Provided is an image processing apparatus including a correction unit that corrects a tilt of each partial image data item which is a result of plural sensors reading a medium on which an image of image data is formed so that at least part of a reading region which is read by each of the plural sensors overlaps a reading region of an adjacent sensor, an extraction unit that extracts the image data at a position and in a size at and in which the image data matches the respective partial image data, and a comparison unit that compares partial image data corrected by the correction unit with extracted image data extracted by the extraction unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2014-192403 filed Sep. 22, 2014.

BACKGROUND

Technical Field

The present invention relates to an image processing apparatus.

SUMMARY

According to an aspect of the invention, there is provided an image processing apparatus including:

a correction unit that corrects a tilt of each partial image data item which is a result of plural sensors reading a medium on which an image of image data is formed so that at least part of a reading region which is read by each of the plural sensors overlaps a reading region of an adjacent sensor;

an extraction unit that extracts the image data at a position and in a size at and in which the image data matches the respective partial image data; and

a comparison unit that compares partial image data corrected by the correction unit with extracted image data extracted by the extraction unit.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments for implementing the invention will be described with reference to the drawings.

FIG. 1is a conceptual module configuration diagram illustrating a configuration example according to a first exemplary embodiment.

In addition, the module indicates components such as software (computer program) and hardware which may be generally and logically divided. Therefore, the module in the present exemplary embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Accordingly, the present exemplary embodiment also includes description of a computer program for causing a computer to function as the module (a program for causing a computer to execute the respective procedures, a program for causing a computer to function as the respective units, or a program for causing a computer to realize the respective functions), a system, and a method. For convenience of the description, although the term “store”, or “make store”, and terms equivalent to these are used, these terms mean that, in a case of a computer program, the program is stored in a storage device or is controlled so as to be stored in the storage device. In addition, the module may correspond to a function in a one-to-one relationship. In a case of mounting, a single module may be configured by a single program, plural modules may be configured by a single program, or conversely a single module may be configured by plural programs. In addition, plural modules may be executed by a single computer, or a single module may be executed by plural computers using computers in distributed or parallel environments. Further, other modules may be included in a single module. Hereinafter, the term “connection” is used not only in a case of physical connection but also in a case of logical connection (data transmission and reception, instruction, reference relationship between data, and the like). The term “predefined” indicates being defined before a process which is a target, and is used to include a meaning of being defined according to circumstances or states at that time or according to circumstances or states hitherto as long as the process which is a target is not performed, even after a process according to the present exemplary embodiment starts including a state where the process according to the present exemplary embodiment does not start. In a case where there are plural “predefined values”, the values may be different, or two or more values (of course, including all the values) may be the same. Further, the phrase having a meaning that “in a case of A, B is performed” is used to mean that “whether or not there is A is determined, if it is determined that there is A, B is performed”. However, a case where determination regarding whether or not there is A is unnecessary is excluded.

In addition, the system or the apparatus includes not only a case where plural computers, plural pieces of hardware, plural apparatuses, and the like are configured to be connected to each other via a communication unit such as a network (including communication connection of one-to-one correspondence) but also a case where it is realized by a single computer, a piece of hardware, a single apparatus, and the like. The “apparatus” and the “system” are used as terms having the same meaning. Of course, the “system” does not include social “structures” (social systems) which are merely artificial arrangements.

In addition, for each process performed by each module, or for each process in a case where plural processes are performed in a module, information which is a target is read from a storage device, the process is performed, and then the process result is written in the storage device. Therefore, there are cases where description of reading from the storage device before the process and writing in the storage device after the process may be omitted. The storage device here may include a hard disk, a random access memory (RAM), an external storage medium, a storage device via a communication line, a register in a central processing unit (CPU), or the like.

An image processing apparatus150according to a first exemplary embodiment reads image from a recording medium125which records the image data thereon and compares the image data with read data, and includes, as illustrated in an example ofFIG. 1, a reading module160, a processing module170, and a comparison/collation module180.

The image processing apparatus150receives a printing image115from an image recording apparatus100, and reads the recording medium125printed by the image recording apparatus100.

The image recording apparatus100includes an image generation module110and a recording module120. The image recording apparatus100and the image processing apparatus150may be provided in separate casings, and may be provided in a single casing. The image recording apparatus100has a function of an output apparatus such as a printer. The image processing apparatus150has a function of an image reading apparatus such as a scanner.

The recording indicates that printing is performed by a printer, for example. The image data is data which is not recorded on the recording medium125and indicates an image having no so-called noise (defect). The read data is data which is obtained by the reading module160reading an image on the recording medium125after the image data is recorded on the recording medium125. Therefore, there is a case where noise may occur therein due to a recording process in the recording module120and a reading process in the reading module160. Here, the noise may include, for example, a case where an image which is not present in image data is recorded on the recording medium125, a case where an image which is present in image data is not recorded on the recording medium125, and a case where enlargement, reduction, rotation, movement, distortion, or the like occurs in image data.

In the present exemplary embodiment, outputting for determining whether or not noise occurs due to the recording module120and the reading module160by comparing image data with read data. In a case where noise occurs, the recording module120and the reading module160are adjusted in order to reduce the noise.

As will be described later, the reading module160does not perform general reading in which a single paper sheet is read with a single sensor, but performs reading in which a single paper sheet is read with plural sensors.

The image generation module110is connected to the recording module120and the processing module170of the image processing apparatus150, and receives printing data105and sends the printing image115to the recording module120and the processing module170. The image generation module110converts the printing data105into the printing image115. Specifically, image conversion processes which are necessary in printing and include a rasterizing process, a conversion process in a color space which may be reproduced by the recording module120, and the like, are performed on the printing data105in which images such as characters, figures, pictures or the like are described, so that printing may be performed by the recording module120. The printing data105may be electronic data which is described in, for example, a portable document format (PDF), and may be created by document creation application software, or the like. The printing image115may be related to any one of binary image data, grayscale image data, and color image data.

The recording module120is connected to the image generation module110and the reading module160of the image processing apparatus150, and receives the printing image115from the image generation module110and sends the recording medium125to the reading module160. The recording module120records the printing image115on a medium such as paper so as to output the recording medium125. Specifically, the recording module120has a function of a printer. In other words, the printing image115is recorded on the recording medium125.

The reading module160is connected to the recording module120of the image recording apparatus100and the processing module170, and receives the recording medium125from the recording module120and sends a read image #1:161 . . . and a read image #n:16n to the processing module170. The reading module160reads the recording medium125(a medium on which an image of the printing image115is recorded) which is output from the recording module120, and generates a read image which is read data. Specifically, the reading module160has a function of a scanner. In other words, the printing image on the recording medium125is converted into a read image through photoelectric conversion or the like in a reading element. However, as illustrated in an example ofFIG. 4, a single paper sheet is read by plural sensors. In addition, images (partial image data items) read by the respective sensors are sent to the processing module170as the read image #1:161 . . . and the read image #n:16n (where n is the number of sensors performing the reading).

FIG. 4is a diagram illustrating a process example according to the first exemplary embodiment (the reading module160).FIG. 4illustrates that plural sensors410read a single paper sheet450. The paper sheet450is a medium on which an image of image data is formed by the recording module120. The sensors410are arranged in parallel. A reading range420ais a range of the paper sheet450read by a sensor410a. A reading range420bis a range of the paper sheet450read by a sensor410b. A reading range420cis a range of the paper sheet450read by a sensor410c. A reading range420dis a range of the paper sheet450read by a sensor410d. A reading range420eis a range of the paper sheet450read by a sensor410e. In each of the reading ranges420, a region including at least one side of the range overlaps a region including at least one side of the adjacent reading range420. For example, the right side of the reading range420aoverlaps the left side of the reading range420b, and the right side of the reading range420boverlaps the left side of the reading range420c. In other words, the plural sensors perform reading so that at least part of the reading region overlaps the reading region of the adjacent sensor.

In order to prevent omission of reading due to alignment deviations of the sensors410, distortions caused by temperature characteristics or the like, or noise such as floating of the paper sheet450, typically, reading is performed in a state in which reading regions overlap each other between the adjacent sensors410.

FIG. 5is a diagram illustrating a process example according to the first exemplary embodiment.FIG. 5illustrates the overlapping of the reading ranges420in the paper sheet450of the example illustrated inFIG. 4from the top (or the bottom). An overlapping region510is a region in which the reading range420aand the reading range420boverlap each other. An overlapping region512is a region in which the reading range420band the reading range420coverlap each other. An overlapping region514is a region in which the reading range420cand the reading range420doverlap each other. An overlapping region516is a region in which the reading range420dand the reading range420eoverlap each other. Here, the term “overlap each other” indicates that two sensors read a portion of the same medium. For example, the overlapping region512is read by both the sensor410band the sensor41c.

The processing module170is connected to the image generation module110of the image recording apparatus100, the reading module160, and the comparison/collation module180, and receives the printing image115from the image generation module110and the read image #1:161 . . . and the read image #n:16n from the reading module160, and sends a printing image172and a read image174to the comparison/collation module180. The processing module170corrects a tilt of partial image data. The correction of a tilt may be performed by using an existing technique. For example, a line segment in the partial image data may be extracted, and may be rotated in a reverse direction after an angle between the line segment and a vertical line or a horizontal line is detected. In addition, a tilt may be detected based on a tilt of a side (paper edge) of the recording medium125. Further, a tilt of partial image data may be corrected based on an angle detected in each partial image data item, and tilts of other partial image data items may be corrected based on an angle detected in any one of partial image data items. The tilt correction may be performed by using, specifically, affine transform.

In addition, the processing module170extracts an image from the printing image115which is image data, so as to be suitable for positions and sizes of the read image #1:161 and the like. Here, the image is extracted at the same tilt as the tilt of the partial image data. Therefore, at least one side of extracted image data which is an extraction result of the printing image115overlaps a region including one side of adjacent extracted image data, in the same manner as in the read image #1:161 and the like.

The processing module170may receive the printing image115not only from the image generation module110, but also from an information processing apparatus (an information processing apparatus which sends the printing data105to the image recording apparatus100) which generates the printing data105, from the recording module120, and from the reading module160in a case where the reading module160receives the printing image115from the image generation module110or the recording module120.

Further, the processing module170may insert a blank space between the partial image data items adjacent to each other of the read image #1:161 and the like so as to correct a tilt, and then may combine the partial image data items with each other so as to generate combined image data. Here, the blank space may have any color (white, black, transparent or the like) and is a region having a fixed value. In addition, the blank space may be a region which is set not to be output as a difference even if the region is compared. In other words, in a case where blank spaces are compared with each other, a region in which it is determined that there is no difference is used as a blank space. In addition, a width of this blank space is equal to or larger than a width of the region in which at least the partial image data items overlap each other. Further, a width of the blank space may be determined in consideration of a tilt angle of the partial image data. In other words, even if a tilt angle is corrected, a width of the blank space is determined so that the overlapping of the partial image data in the longitudinal direction is removed. Consequently, during comparison and collation in the comparison/collation module180which will be described later, it is possible to specify in which read image (sensor) a different portion is generated only by referring to coordinates of a comparison result in the width direction. Still further, a width of the blank space may be a width corresponding to a size of a comparison target (block) in the comparison/collation module180. For example, a multiple of a horizontal width of the comparison target (block) may be used. A blank space may be inserted between extracted image data items which are obtained by dividing the printing image115and are adjacent to each other. In this case, the blank space inserted into the extracted image data may be compared in the same condition as in the combined image data when a width thereof is set to a width of a blank space which is inserted into partial image data and is obtained by correcting a tilt.

The comparison/collation module180is connected to the processing module170, and receives the printing image172and the read image174from the processing module170so as to output a collation result185. The comparison/collation module180compares the partial image data (read image #1:161 and the like) corrected by the processing module170with the extracted image data (extracted image data of the printing image115) extracted by the processing module170. Comparison targets here are the partial image data (the read image #1:161 and the like) of which a tilt is corrected and the extracted image data of the printing image115.

The comparison/collation module180compares the combined image data (combined image data based on the read image #1:161 and the like) generated by the processing module170with the image data in which a blank space is inserted between the extracted image data items of the printing image115. Comparison targets here are a combination of the partial image data items (the read image #1:161 and the like) of which tilts have been corrected, and a combination of extracted image data items of the printing image115.

The comparison is a process of detecting a difference between both images. For example, an exclusive OR (EOR) process of both images may be performed, and a difference between both of the images may be extracted, and a portion having values other than 0 is detected as a different portion. In addition, there is a process of extracting an image which is present in the printing image172but is not present in the read image174, a process of extracting an image which is not present in the printing image172but is present in the read image174, and the like. It is determined that noise occurs in the different portion.

Here, the outputting of the collation result185includes, for example, printing the different portion with a printing apparatus such as a printer, displaying the different portion on a display apparatus such as a display, transmitting an image with an image transmission apparatus such as a facsimile, writing an image in an image storage apparatus such as an image database, storing an image on a recording medium such as a memory card, and sending an image to other information processing apparatuses.

FIG. 2is a diagram illustrating a system configuration example in a case where the first exemplary embodiment is realized.

The image recording apparatus100, the image processing apparatus150, a printing data generation apparatus210, and a collation result storage apparatus220are connected to each other via a communication line290. The communication line290may be a wired line, a wireless line, and a combination thereof, and may be, for example, the Internet as a communication infrastructure. The printing data generation apparatus210generates the printing data105, and sends the printing data105to the image recording apparatus100. The image recording apparatus100prints the printing data105and outputs the recording medium125. The image processing apparatus150reads the recording medium125, and receives the printing data105from the image recording apparatus100(or the printing data generation apparatus210) and sends the collation result185to the collation result storage apparatus220. The collation result storage apparatus220receives and stores the collation result185from the image processing apparatus150. The recording module120of the image recording apparatus100and the reading module160of the image processing apparatus150are adjusted based on the collation result185stored in the collation result storage apparatus220.

FIG. 3is a flowchart illustrating a process example according to the first exemplary embodiment.

In step S302, the image generation module110performs image conversion of the printing data105so as to generate the printing image115.

In step S304, the recording module120records the printing image115on a medium. The recording medium125on which the printing image115is printed is generated.

In step S306, the reading module160reads the recording medium125.

FIG. 6is a diagram illustrating a process example according to the first exemplary embodiment. An image600indicates an image right after reading is performed. The image600is formed by images (read divided images610,612,614,616and618) read by the respective sensors410, and includes overlapping regions510,512,514and516as overlapping portions. In addition, the overlapping region510displays image data items which are respectively read by the sensor410aand the sensor410bin an overlapping manner. Therefore, there is a possibility that there may be a difference between a portion of the overlapping region510read by the sensor410aand a portion of the overlapping region510read by the sensor410b.

In step S308, the processing module170detects skew angles of the read image #1:161 and the like. For example, a skew angle is detected based on a tilt of a paper edge of the read image.

In step S310, the processing module170performs processing on the read image #1:161 . . . and the read image #n:16n, and the printing image115. As the processing here, a blank space is inserted between the divided read image in accordance with, for example, the skew angle, and a size (block size) of a comparison target in collation performed by the comparison/collation module180.

FIG. 7is a diagram illustrating a process example according to the first exemplary embodiment. An image700indicates an image state in which blank spaces are inserted into the image600illustrated in the example ofFIG. 6. In other words, a blank space710is inserted between the read divided image610and the read divided image612; a blank space712is inserted between the read divided image612and the read divided image614; a blank space714is inserted between the read divided image614and the read divided image616; and a blank space716is inserted between the read divided image616and the read divided image618. In addition, the read divided image612includes the overlapping region510and the overlapping region512read by the sensor410b.

FIG. 8is a diagram illustrating a process example according to the first exemplary embodiment. An image800indicates an image state in which skew correction is performed on the image700illustrated in the example ofFIG. 7. In other words, the state is a state in which the tilts of the read divided image612and the like are removed. A tilt of any one of the read divided images610,612,614,616and618may be detected as the tilt here, and the read divided images610,612,614,616and618may be corrected according to the tilt, and a tilt of each of the read divided images610,612,614,616and618may be detected, and the read divided images610,612,614,616and618may be corrected in accordance with each tilt.

FIGS. 9A and 9Bare diagrams illustrating a process example according to the first exemplary embodiment. An image900illustrated in an example ofFIG. 9Ais obtained by performing size correction suitable for a processed output image950, on the image800illustrated in the example ofFIG. 8. The size correction here indicates that a bottom side portion (or a top side portion) of the image800is removed, and a right side portion (or a left side portion) thereof is removed, in order to generate the same size as the size of the processed output image950. The processed output image950illustrated in an example ofFIG. 9Bis obtained by extracting the image from the printing data105and inserting blank spaces thereinto. In addition, in the extraction process, images having the same sizes as those of the read image #1:161 and the like are extracted. However, an overlapping region is made to be included in both divided images adjacent to each other. Further, blank spaces (blank spaces970,972,974and976) having the same sizes (horizontal widths) as the sizes (horizontal widths) after the skew correction is performed, illustrated in the example ofFIG. 7, are inserted between the respective divided images (output divided images960,962,964,966and968).

In step S312, the comparison/collation module180compares and collates the printing image172and the read image174which are converted images, with each other, so as to detect a defect occurrence region.

Here, the defect detection may be performed as follows.

1) At least one of the printing image172and the read image174is divided so that plural block images are extracted, and a difference absolute value of both of the images is calculated for each pixel on an image different from the image from which the block images are extracted while changing a position of the block image in a predefined search range between the printing image172and the read image174.

2) A position where a sum of the difference absolute values is the minimum is searched for in the block, and a difference image is generated.

3) A defect is detected from the difference image. For example, a portion in which a pixel value of the difference image is greater than, or equal to or greater than a predefined threshold value, is detected as a result thereof.

In step S314, the collation result185is output.

The process in step S312will be described in detail by using a second exemplary embodiment. The comparison/collation module180performs processes of a data collection/position estimation module1140, a block movement amount calculation module1150, and a comparison/collation module1160described in the following example. The same portions as in the above-described exemplary embodiment are given the same reference numerals, and repeated description will be omitted. A reading module1130corresponds to the reading module160. In addition, the printing image115corresponds to the printing image172, and a read image1135corresponds to the read image174. In other words, the comparison/collation module180in the first exemplary embodiment performs a process equivalent to the process on the printing image115and the read image1135in the second exemplary embodiment, since each of the printing image172and the read image174is processed as a single image.

The entire image processing system including an image processing apparatus according to the second exemplary embodiment compares image data with read data, and includes the image recording apparatus100and an image processing apparatus1170as illustrated in an example ofFIG. 11. The image recording apparatus100includes the image generation module110and the recording module120, and the image processing apparatus1170includes the reading module1130, the data collection/position estimation module1140, the block movement amount calculation module1150, and the comparison/collation module1160.

The reading module1130is connected to the data collection/position estimation module1140and the comparison/collation module1160. The reading module1130converts the printing image115on the recording medium125into the read image1135using photoelectric conversion or the like. In other words, the read image1135which obtained by reading the recording medium125is generated. This is a so-called scanning process, and the reading module1130is implemented by a scanner which is an image reading device. The read image1135is sent to the data collection/position estimation module1140and the comparison/collation module1160.

The data collection/position estimation module1140is connected to the image generation module110, the reading module1130, the block movement amount calculation module1150, and the comparison/collation module1160. The data collection/position estimation module1140sets a search range which is calculated based on an error, and searches for a position which a selected block image matches in one of the printing image115and the read image1135in relation to a block image extracted from the other. The searched position (an estimated value1145of a displacement amount) is sent to the comparison/collation module1160. The comparison/collation module1160compares and collates the block image with the other image by overlapping the two images at the searched position, so as to detect a defect occurrence region.

A position estimation of a block image in the data collection/position estimation module1140may be performed as follows without first performing position search regarding all block images.

First, a search process will be described. This process shows an example of the process in step S312according to the first exemplary embodiment. Description thereof will be made with reference toFIG. 21.FIG. 21is a diagram illustrating a process example in the comparison/collation module1160. A block image1910aof a reference image1900ais shifted by one pixel in a search range1950bof a read image1900b, so that template matching (for example, an EOR process) is performed. In addition, as a result of the template matching, a relative position to a block image in which the maximum score (for example, as a score, an inverse number of a difference absolute value) indicating the most proximity in the search range1950bis set as a displacement amount2120. In other words, one in the block image1910bcorresponding to the block image1910ais a block image1901b, and in a case where a block image2130is extracted at the maximum score position as a result of performing template matching between the block image in the search range1950band the block image1910a, the displacement amount2120is set. In addition, the comparison/collation module1160receives the estimated value1145of the displacement amount from the data collection/position estimation module1140, and searches for a target block. Further, estimation of a plane is not necessarily performed in the template matching (of course, as will be described later, a plane estimation process may be performed for the template matching).

Hereinafter, a detailed description of a process example according to the second exemplary embodiment will be described.

The data collection/position estimation module1140selects at least one of block images extracted from one of the printing image115and the read image1135, sets a search range calculated based on an error, and searches for a position which the selected block image matches in other image. Then, a corresponding position of an unselected block image in the other image is estimated based on the corresponding position of the selected block image matching in the other image. In addition, the estimated position (the estimated value1145of the displacement amount) is sent to the comparison/collation module1160.

First, with reference toFIGS. 13A to 14, a description will be made of a process of estimating a corresponding position of a block image in the other image.FIGS. 13A to 13Care diagrams illustrating a process example in the data collection/position estimation module1140.

1) As illustrated in examples ofFIGS. 13A and 13B, the printing image115and the read image1135are respectively divided into rectangular shapes (hereinafter, referred to as blocks), so as to generate a printing block image1310and a read block image1320. In other words, images of the printing image115and the read image1135are divided into blocks which are generated based on errors which may occur in the recording module120or the reading module1130, and thus block images are extracted. In addition, as a block size here, a block size1152which is calculated by the block movement amount calculation module1150is used.

More specifically, the following process is performed.

One of the printing image115and the read image1135is divided into a rectangular shape which is generated based on errors which may occur in the recording module120or the reading module1130, and thus a rectangular image is extracted. One image may be the printing image115or the read image1135. Hereinafter, a description will be mainly made assuming that the printing image115is used as one image, and the read image1135is used as the other image. The block division will be described with reference toFIGS. 19A and 19B.FIGS. 19A and 19Bare diagrams illustrating a process example according to the second exemplary embodiment. In the example ofFIG. 19A, the reference image1900ais divided into 9×12 blocks. Of course, a division number differ depending on a block size and a size of the reference image1900a. In addition, the division number is not necessarily an integer number depending on a relationship between the block size and the size of the reference image1900a, and a white image or the like may be added to a portion which does not form a single block. Further, the block division is performed based on the block size1152received from the block movement amount calculation module1150.

FIG. 20is a diagram illustrating a block example, and enlarges and displays the block image1910aillustrated in the example ofFIG. 19A. The block image1910ahas a size of Wp×Hp. In addition, coordinates of a central pixel2010of the block image1910aare set to (Cx,Cy).

2) As illustrated in the example ofFIG. 13C, at least one set of extracted block images is selected, and a difference absolute value of both of the images is calculated for each pixel in blocks (the printing image115and the read image1135) at a corresponding position. For example, an exclusive OR (EOR) process may be performed for each pixel.

3) A position where a sum of the difference absolute values (a sum total of the difference absolute values in the respective pixels in the block) in the block is the minimum is searched for. In addition, a search range A here may or not be the same as the estimated value1145of the displacement amount used in the comparison/collation module1160. The search range may be defined based on a movement amount1154calculated by the block movement amount calculation module1150, and may be a search range which will be described later with reference toFIGS. 16A to 16C.

In the examples ofFIGS. 13A to 13C, the printing block image1310is searched for in the read block image1320. In other words, the printing block image1310and the read block image1320are shift in the search range A, and a sum of the difference absolute values in a positional relationship thereof is calculated. A positional relationship between rectangular images in which the calculated sum of the difference absolute values is the minimum is specified. As illustrated in an example ofFIG. 14, a printing block image1410is searched for regarding a read block image1420in a read image1400. A position1450where a sum of the difference absolute values is the minimum is obtained in the search range A. The position1450corresponds to, for example, differences (Δx,Δy) between central coordinates of the printing block image1410and the read block image1420. Here, an example has been described in which the printing block image1310is searched for in the read block image1320, but the read block image1320may be searched for in the printing block image1310. In the present exemplary embodiment, both the printing image and the read image are cut out as blocks, but only the printing image may be cut out as a block image, a search range may be directly set in the other read image, and a position where a sum of difference absolute values is the minimum may be searched for in a range thereof in the same manner. In addition, only the read image may be cut out as a block image, a search range may be directly set in the other printing image, and a position where a sum of difference absolute values is the minimum may be searched for in a range thereof in the same manner.

In the present exemplary embodiment, the search range A is calculated as follows.

A range of a movement amount for defining the search range A is determined based on errors which may occur in the recording module120which forms an image of the printing image115on the recording medium125or the reading module1130which reads the recording medium125, and an image (which may be either the printing image115or the read image1135). The “errors which may occur in the recording module120or the reading module1130” will be described later in description of the block movement amount calculation module1150. For example, the data collection/position estimation module1140increases a movement amount according to a distance from an origin in the printing image115or the read image1135, and calculates a movement amount at the most distant point from the origin so that a movement amount having undergone affine transform which is an error which may occur in the recording module120or the reading module1130becomes the maximum in a size of the printing image115or the read image1135. The distance from the origin is the same value as a value of a position of the block. The origin of the printing image115or the read image1135is a position where the occurrence of an error in the recording module120or the reading module1130is the minimum, and indicates, for example, coordinates of an upper left corner of the image. The “most distant point from the origin” is a position which is the most distant from the origin. The position which is the most distant from the origin is a position where the occurrence of an error in the recording module120or the reading module1130is the maximum, and indicates, for example, coordinates of a lower right corner of the image. Specifically, the data collection/position estimation module1140increases a size of the search range A in a direction from the coordinates of the upper left corner to the lower right corner. The maximum size thereof is defined based on the maximum value of an “error which may occur in the recording module120or the reading module1130”, and is a movement amount1154calculated by the block movement amount calculation module1150.

A description thereof will be made by using an example ofFIGS. 16A to 16C.FIGS. 16A to 16Care diagrams illustrating an example of a size of the search range A based on a position inside an image.FIG. 16Aillustrates an example of positions of an origin1610(upper left corner) of an image1600and the most distant point1690(lower right corner). In addition,FIG. 16Billustrates an example of a relationship between a block1612and a search range A1614at the origin1610. The search range A1614is larger than the block1612by a predefined value (0 or greater).FIG. 16Cillustrates an example of a relationship between a block1692and a search range A1694at the most distant point1690. The search range A1694has a size defined based on the movement amount1154calculated by the block movement amount calculation module1150. Further, sizes of the block1612and the block1692are the same as each other, and are the block size1152calculated by the block movement amount calculation module1150. Here, an example in which the upper left corner is set as the origin, and the lower right corner is set as the most distant point, but any one of four points (the lower right corner, the upper right corner, the lower left corner, and the upper left corner) may be used depending on properties of errors which may occur in the recording module120or the reading module1130. Here, as the examples, cases have been described in which rotation of the recording module120or the reading module1130occurs centering on the upper left corner, and an effect of enlargement or reduction becomes greater as a point becomes farther from the upper left corner.

Next, the data collection/position estimation module1140searches for a corresponding position of the extracted rectangular image in the other image in the search range A, by using the position1450at which the sum of the difference absolute values is the minimum in the search range A as a reference. For example,FIG. 19Billustrates an example of searching for the block image1910aillustrated inFIG. 19A. As illustrated inFIG. 19B, the block image1910ais searched for in the search range1950bwhich is a “range of a movement amount” inside the read image1900b. In addition, one corresponding to the block image1910ain the reference image1900ais a block image1910bin the read image1900b. Ideally (in a case where errors which may occur in the recording module120and the reading module1130are 0), in order to detect a defect, the block image1910aand the block image1910bhave only to be compared with each other, but, actually, since errors occur in the recording module120and the reading module1130, the search is made in the search range1950b(a search range determined by the data collection/position estimation module1140) which is determined in consideration of the actual errors. Consequently, the entire read image1900bis not required to be searched for, and a position which matches the block image1910ain the read image1900bmay be found. The search range is a region which includes a rectangular shape of the other image at the same position as the rectangular position in one image and is formed by vertical and horizontal movement amounts with respect to the rectangular shape.

The search process is as described with reference toFIG. 21. The example illustrated inFIG. 21displays the search range1950billustrated in the example ofFIG. 19Bin an enlarged manner.

A block in which there may be plural positions where a sum of difference absolute values is the minimum is excluded in advance as a source for estimating positions of unselected other blocks described later.

For example, there are a block (which may include a case where a whole pixel value difference is equal to or smaller than a predefined value even if the block is painted with exactly the same color) which is painted with the same color (for example, white, black, or transparent) as a whole, a block in which only a single straight line is drawn, a block in which only plural parallel lines are drawn, and a block in which plural regular graphics (substantially the same graphics) are drawn. Since it is not possible to uniquely determine a corresponding position in these blocks, it is not possible to estimate a position of an unselected block, or even if a position may be estimated, an error may increase, and thus the blocks are not suitable for estimating a position.

The time at which it is determined that these blocks are excluded may be when a block is selected, and may be when a position of another image is estimated after a block is selected and then a corresponding position is searched, but if the blocks are excluded when a block is selected, it is possible to reduce a calculation amount required for an unnecessary search.

In a case where a distance between rectangular images in which a sum of difference absolute values is the minimum is greater than or equal to a predefined value as a result in which a block is selected and a search is made, a positional relationship between the blocks is excluded as a source for estimating positions of unselected other blocks.

Next, a description will be made of estimation of a corresponding position of a block image which is not selected by the data collection/position estimation module1140in the other image, with reference toFIGS. 15A to 15C.FIGS. 15A to 15Care diagrams illustrating a process example in the data collection/position estimation module1140. As illustrated in an example ofFIG. 15A, a position where a sum of difference absolute values is the minimum in blocks (blocks with the diagonal lines inFIG. 15A) of a printing image1510is calculated. In addition, excluded blocks (white blocks inFIG. 15A) are not used for a position estimation process. The data collection/position estimation module1140calculates a plane (which may be a “plane in a characteristic space”) defined by a position between block images, and estimates positions of unselected block images based on the plane. Specifically, a plane indicating a displacement amount is calculated from the position where the sum of difference absolute values is the minimum, and a displacement amount is calculated according to a position of each block.

In other words, the following process is performed. A position (Δx,Δy) of the block image is plotted in the characteristic space. As the characteristic space, for example, there is a three-dimensional space which has an x axis, a y axis, and Δx or Δy as an axis. Here, the x axis and the y axis are axes representing coordinates of a center of the block. The characteristic space may be a four-dimensional space which has an x axis, a y axis, a Δx axis, and a Δy axis, and may be other spaces as long as an axis representing Δx or Δy is included. A plane formed through the plotting is calculated. For example, the plane is a plane1520representing a displacement amount (Δx) in the example ofFIG. 15A, and is a plane1530representing a displacement amount (Δy) in the example ofFIG. 15B. Of course, all positions are not required to be plotted in a plane, and a plane estimated through the plotting may be calculated by using an existing method.

It is possible to estimate displacement amounts in all blocks in an image by using the plane representing the displacement amounts. Specifically, a position of each block is assigned to an equation (Δx=a1x+b1y+c1, Δy=a2x+b2y+c2) representing the plane, and a displacement amount in each block is estimated.

In relation to estimation of a displacement amount here, at least an unselected block image is required to be targeted. In a case where this process is sequentially performed, when only an unselected block image is targeted, a calculation amount of the displacement amount estimation is further reduced than when all block images are targeted, and thus the process may be performed at a high speed. In a case where the displacement amount estimation is performed in parallel processes by using a multicore CPU, GPU or the like, the process may be performed at a higher speed when the process is uniformly applied to all block images than when a step of determining whether or not a block image is a selection target is added.

Next, the data collection/position estimation module1140searches for a corresponding position in the other image for each block image based on the displacement amount estimated by using the above-described method. In other words, a search range is set by using the estimated displacement amount as a reference position, template matching is performed, and a position indicating the most proximity in the search range is searched for. The template matching here may employ the same method as the above-described template matching.

A search target here is at least an unselected block image. In a case where this process is sequentially performed, when only an unselected block image is targeted, a calculation amount of the displacement amount estimation is further reduced than when all block images are targeted, and thus the process may be performed at a high speed. In a case where the second position search is performed in parallel processes by using a multicore CPU, GPU or the like, the process may be performed at a higher speed when the process is uniformly applied to all block images than when a step of determining whether or not a block image is a selection target is added.

The search range here may be determined in the same manner as in the above-described method of determining a search range, but a position has already been estimated for an unselected block image, and thus a range smaller than that may be set as the search range. A calculation amount required for difference extraction is further reduced than in a case of using the same search range.

The comparison/collation module1160is connected to the image generation module110, the reading module1130, and the data collection/position estimation module1140. The comparison/collation module1160compares and collates the printing image115with the read image1135so as to detect the presence or absence of a defect and a defect occurrence region. The comparison/collation module1160receives the printing image115and the read image1135. The read image1135reads the recording medium125on which an image of the printing image115is formed. In other words, the comparison/collation module1160receives the printing image115which is a source of a print-out target, and the read image1135which is obtained by printing out the printing image115and scanning the printed-out recording medium125. This is because the presence or absence of a defect which occurs in the recording module120or the reading module1130is checked. Here, the defect indicates noise, and, ideally, the printing image115completely matches the read image1135. However, a defect occurs due to maintenance statuses of the recording module120and the reading module1130. In addition, an error may occur in the recording module120and the reading module1130. Here, the “error which may occur” is an error which is allowable in design specifications of the recording module120and the reading module1130. As the error, specifically, there are enlargement or reduction, rotation, and affine transform of movement. In a case where a design specification is realized (a design goal is achieved) in which, for example, rotation of x degrees or larger does not occur, rotation of below x degrees may occur.

Therefore, the comparison/collation module1160extracts a defect (for example, a mixture of three or more black pixels) from which an error which may occur in the recording module120and the reading module1130is excluded. In other words, it is checked whether or not a defect other than an allowable error in the design specification, and a position where the defect occurs is specified to be usefully used for maintenance of the recording module120and the reading module1130.

Next, the comparison/collation module1160receives a search result from the data collection/position estimation module1140, and compares an extracted rectangular image with an image (an image of the search result) of a portion to which the extracted rectangular image corresponds in the other image. In other words, both of the images overlap each other at a position where the rectangular image of the printing image side and the rectangular image of the read image side match each other most, and comparison for extracting a difference therebetween is performed. The “search result” is a result in which a position where both of the images match each other most is extracted by comparing the rectangular image in one image within a search range of the other image. The difference between the position and the position of the rectangular image may be used as a displacement amount. The search result may be used for the “comparison”. In other words, the search result is obtained in a case where the most matching occurs, and thus a matching degree may be used as a defect detection result1165. In addition, the defect detection result1165may include not only the matching degree but also a displacement amount.

A description thereof will be made with reference to FIGS.17A to17C.FIGS. 17A to 17Care diagrams illustrating a defect detection process example in the comparison/collation module1160. A printing block image1710illustrated in an example ofFIG. 17Ais a block image in the printing image115; a read block image1720illustrated in an example ofFIG. 17Bis a block image in the read image1135; and a difference block image1730illustrated in an example of inFIG. 17Cindicates a difference (a result of an EOR process) between the printing block image1710and the read block image1720. The comparison/collation module1160generates a difference image for each block, and detects a defect in the difference image. As the defect, for example, a defect may be detected in which a size of one side of a circumscribed rectangle is a size within a predefined range (for example, three pixels to five pixels).

The block movement amount calculation module1150is connected to the data collection/position estimation module1140. The block movement amount calculation module1150calculates the block size1152and the movement amount1154based on the error which may occur in the recording module120or the reading module1130. The block size1152and the movement amount1154are sent to the data collection/position estimation module1140. The error in the recording module120or the reading module1130includes affine transform in the image forming apparatus or the image reading apparatus. Specific examples of the error include an enlargement or reduction ratio or a rotation angle. The process in the block movement amount calculation module1150may be performed before the data collection/position estimation module1140performs a process, and the block size1152and the movement amount1154may be calculated in advance and be stored in the data collection/position estimation module1140. Further, in order to correspond to the recording module120and the reading module1130used at that time, the block size1152and the movement amount1154may be calculated whenever errors are received from the recording module120and the reading module1130.

The block size1152is defined through affine transform so that a pixel deviation between a rectangular image of one image and a rectangular image of the other image is smaller than one pixel. In addition, calculation of the block size1152will be described later by using an example ofFIG. 23.

The movement amount1154calculated by the block movement amount calculation module1150is defined so that a movement amount obtained by affine transform is the maximum in a size of the printing image115or the read image1135. The movement amount1154is used to form the search range1950b.

FIG. 22is a diagram illustrating a relationship example between the block image1910aand the search range1950b. The movement amount1154includes, specifically, an upward movement amount, a downward movement amount, a left movement amount, and a right movement amount.

A height (block height) of the block image1910ais La+1+Lb, and is Hpillustrated in the example ofFIG. 20.

A width (block width) of the block image1910ais L1+1+Lr, and is Wpillustrated in the example ofFIG. 20.

A height (search range height) of the search range1950bis Hp+La+Lb. Laindicates an upward movement amount. Lbindicates a downward movement amount.

A width (search range width) of the search range1950bis Wp+L1+Lr. L1indicates a left movement amount. Lrindicates a right movement amount.

In addition, calculation of the movement amount1154will be described later by using examples ofFIGS. 24 and 25.

The movement amount1154indicates the maximum value of a search range in the image, and the data collection/position estimation module1140determines the estimated value1145of a displacement amount based on a position of the block in the image by using the movement amount1154.

All process examples according to the second exemplary embodiment will be described.

First, the image generation module110performs image conversion on the printing data105so as to generate the printing image115.

Next, the recording module120records the printing image115on the recording medium125.

Next, the reading module1130reads the recording medium125on which the printing image115is recorded. The read result is the read image1135.

Next, the data collection/position estimation module1140collects data items (specifically, differences between block images) for determining the estimated value1145of a displacement amount.

Next, the data collection/position estimation module1140estimates a displacement amount for each block.

Next, the comparison/collation module1160compares and collates the converted image (printing image115) with the read image1135so as to detect a defect occurrence region.

FIG. 12is a flowchart illustrating a process example according to the second exemplary embodiment.

In step S1202, at least one of a printing data image and a read data image is divided. As a division method here, the image is divided in a size which is calculated by using an error (for example, the size becomes smaller as the error increases).

In step S1204, a search range is determined in some divided images. The determination of the search range here is performed in a size which is calculated by using an error (for example, the size becomes larger as the error increases).

In step S1206, a corresponding position in a partner image is specified. Specifically, the divided image overlaps the other image so that a difference is obtained for each pixel, absolute values of differences are added together while changing a position of the image in the search range, and a position where the difference is smallest is set as a corresponding position.

In step S1208, a corresponding position of another divided image in the partner image is estimated based on specified corresponding position. Specifically, a deviation amount between the corresponding position and an initial position is plotted in a plane, and a deviation amount of another divided image is estimated based on the plane and a position relative to the division source image of the divided image.

In step S1210, a corresponding position of another divided image in the partner image is specified based on the estimated position. A search range here may be the same as the search range in step S1204, but may be a second search range narrower than the search range determined in step S1204. This is because the searched divided image here is searched for from the position estimated in step S1208, and thus a corresponding position is located nearby. The second search range is applied, and thus a calculation amount which requires differences for specifying a position is reduced.

In step S1212, a comparison result at each specified position is output. The difference acquired when the position is specified may be output as a comparison result without change, and an image may be output after the image is further processed (for example, a difference is emphasized).

FIG. 18is a flowchart illustrating a process example in which the block movement amount calculation module1150calculates the block size1152and the movement amount1154.

In step S1802, an enlargement or reduction ratio and a rotation angle (skew angle) which may occur in the recording module120or the reading module1130are acquired.

In step S1804, a block size is calculated.

In step S1806, a movement amount is calculated.

In step S1808, the block size1152and the movement amount1154are sent to the data collection/position estimation module1140and the comparison/collation module1160.

FIG. 23is a diagram illustrating a process example of calculating the block size1152. In a case where rotation, enlargement or reduction is generated in the worst state by the recording module120or the reading module1130, a block size is set so that a pixel deviation is smaller than one pixel at each corner when centers of a block of the printing image115and a block of the read image1135match each other.

An example ofFIG. 23shows that a point2350in the printing image115is how deviated due to an error in the recording module120or the reading module1130.

The point2350is moved to a point2360due to a movement (during the maximum reduction in both scanning directions)2355caused by enlargement or reduction. In addition, the point is moved to a point2370due to a movement (counterclockwise)2365caused by skew. This case corresponds to a case where the maximum reduction occurs (the point becomes smallest), and counterclockwise rotation is the maximum.

The point2350is moved to a point2340due to a movement (during the maximum enlargement in both scanning directions)2345caused by enlargement or reduction. In addition, the point is moved to a point2330due to a movement (clockwise)2335caused by skew. This case corresponds to a case where the maximum enlargement occurs (the point becomes largest), and clockwise rotation is the maximum.

(1) When the maximum reduction (x-axis direction magnification: r1, and y axis direction magnification: r2) and the maximum skew (counterclockwise, rotation angle: α) occur in both of the scanning directions, a movement amount is calculated as in Equation (1) if a movement amount of a point on a diagonal line having the largest movement amount with a center of the block as an origin is set to (1,1).

A difference Δx of x at y=1 is calculated by using a straight line passing through the point2370and the original straight line. In the example ofFIG. 23, the difference Δx is defined by the point2350, the origin, the straight line passing through the point2370, and the straight line of y=1.

(2) On the other hand, when the maximum enlargement (x-axis direction magnification: r3, and y axis direction magnification: r4) and the maximum skew (clockwise, rotation angle: β) occur in both of the scanning directions, a movement amount is calculated as in Equation (2) if a movement amount of a point on a diagonal line having the largest movement amount with a center of the block as an origin is set to (1,1).

A difference Δy of y at x=1 is calculated by using a straight line passing through the point2330and the original straight line. In the example ofFIG. 23, the difference Δy is defined by the point2350, the origin, the straight line passing through the point2330, and the straight line of x=1.

(3) y′ and x′ which cause Δx and Δy to become 1.4 are calculated. In addition, when a width and a height of the block are respectively indicated by W and H, Equation (3) is given.
W=2*ceil(x′)+1, H=2*ceil(y′)+1  Equation (3)

Here, ceil( ) is a function which rounds up a decimal point or less.

Consequently, the block size1152(W and H of Equation (3)) may be calculated. For example, a width and a height of the block are respectively 100 pixels and 110 pixel.

Next, a description will be made of a process example of calculating the movement amount1154(a size of a search range) by using examples ofFIGS. 24 and 25. The movement amount1154calculated here is a movement amount at the most distant point.

In a case where rotation, enlargement, reduction, and movement all occur in the worst case due to the recording module120or the reading module1130, the movement amount1154is set so that a movement amount of a lower right corner in a certain paper size (for example, A3 elongation paper) is taken into consideration. In addition, the following coordinates have an upper left corner as an origin, a right direction and a lower direction as a positive direction, and a clockwise direction as a positive direction of a rotation angle.

Under a condition that a movement amount is the maximum in the upper right direction, with an upper left corner of a certain paper size as an origin, a movement amount of a lower right corner (for example, in the A3 elongation paper, coordinates (3897,5763) at 300 dpi) having the largest movement amount is calculated as in Equation (4).
M=Mskew+Msize+MshiftEquation (4)

Here, Mskewindicates a movement amount caused by skew, Msizeindicates a movement amount caused by enlargement or reduction, and Mshiftindicates a movement amount caused by shift.

Here, γ indicates a rotation angle, r5and r6indicate an enlargement or reduction ratio, and s1and s2indicate a movement amount.

A description will be made by using an example ofFIG. 24.FIG. 24is a diagram illustrating a process example of calculating a movement amount.

A point2450is moved to a point2460due to a movement (the maximum in the upper right direction)2455caused by shift. In addition, the point is moved to a point2470due to a movement (the maximum enlargement in the main scanning direction and the maximum reduction in the sub-scanning direction)2465caused by enlargement and reduction. Further, the point is moved to a point2480due to a movement (counterclockwise)2475caused by skew. In other words, the maximum movement amount in the upper direction and the right direction at the most distant point (the lower right corner) from an origin2410(inFIG. 24, the upper left corner) of an image2400is calculated.

On the other hand, under a condition that a movement amount in the lower left direction is the maximum, with an upper left corner of a certain paper size as an origin, a movement amount of a lower right corner (for example, A3 elongation paper) having the largest movement amount is calculated as in Equations (9), (10) and (11).

Here, θ indicates a rotation angle, r7and r8indicate an enlargement or reduction ratio, and s3and s4indicate a movement amount.

A description will be made by using an example ofFIG. 25.FIG. 25is a diagram illustrating a process example of calculating a movement amount.

The point2450is moved to a point2440due to a movement (the maximum in the lower left direction)2445caused by shift. In addition, the point is moved to a point2430due to a movement (the maximum reduction in the main scanning direction and the maximum enlargement in the sub-scanning direction)2435caused by enlargement and reduction. Further, the point is moved to a point2420due to a movement (counterclockwise)2425caused by skew. In other words, the maximum movement amount in the lower direction and the left direction at the most distant point (the lower right corner) from the origin2410(inFIG. 25, the upper left corner) of the image2400is calculated.

Based on the above description, the upward maximum value, the downward maximum value, the left maximum value, and the right maximum value of a movement amount may be calculated. For example, the upward maximum value, the downward maximum value, the left maximum value, and the right maximum value of a movement amount are respectively 67.41 pixels, 95.99 pixels, 58.73 pixels, and 62.47 pixels, and thus the upward, downward, left and right movement amounts of the block respectively become 68 pixels, 96 pixels, 59 pixels, and 63 pixels at minimum. In addition, actually, a movement amount is out of specifications (over specification) of the recording module120and the reading module1130and may thus be set to be larger by a predefined value. This value is a movement amount at the point2450which is the most distant point from the origin2410. Movement amounts at positions other than the point2450(search range) may be determined by the data collection/position estimation module1140based on the positions and the movement amount at the point2450.

With reference toFIG. 10, a hardware configuration example of the image processing apparatus of the present exemplary embodiment will be described. A configuration shown inFIG. 10is implemented by, for example, a personal computer (PC) or the like, and is a hardware configuration example including a data reading unit1017such as a scanner and a data output unit1018such as a printer.

A central processing unit (CPU)1001is a controller which executes processes according to a computer program which describes execution sequences of the respective modules described in the above exemplary embodiment, that is, the reading module160, the processing module170, and the comparison/collation module180.

A read only memory (ROM)1002stores programs, calculation parameters or the like used by the CPU1001. A random access memory (RAM)1003stores programs used for execution of the CPU1001or parameters which are appropriately varied in the execution and the like. They are connected to each other via a host bus1004constituted by a CPU bus and the like.

The host bus1004is connected to an external bus1006such as a peripheral component interconnect/interface (PCI) bus via a bridge1005.

A keyboard1008and a pointing device1009such as a mouse are input devices operated by an operator. A display1010includes a liquid crystal display, a cathode ray tube (CRT), or the like, and displays a variety of information pieces as text or image information.

An hard disk drive (HDD)1011has a hard disk built therein, drives the hard disk, and records or reproduces a program or information executed by the CPU1001. The hard disk stores the printing image115, the read image #1:161, the collation result185, and the like. In addition, a variety of computer programs such as various other data processing programs are stored therein.

A drive1012reads data or a program recorded on a removable recording medium1013such as a magnetic disk, an optical disc, a magneto-optical disc, or a semiconductor memory which is installed therein, and supplies the data and the program to the RAM1003which is connected thereto via an interface1007, the external bus1006, the bridge1005, and the host bus1004. The removable recording medium1013may be also used as a data recording region in the same manner as the hard disk.

A connection port1014is a port for connection to an external connection apparatus1015and has a connection unit such as a USB, or IEEE1394. The connection port1014is connected to the CPU1001and the like via the interface1007, the external bus1006, the bridge1005, the host bus1004and the like. A communication unit1016is connected to a communication line and executes a data communication process with external apparatuses. The data reading unit1017is, for example, a scanner, and executes a document reading process. The data output unit1018is, for example, a printer, and executes a document data output process.

In addition, the hardware configuration of the image processing apparatus illustrated inFIG. 10shows a configuration example, and the present exemplary embodiment is not limited to the configuration illustrated inFIG. 10and may employ a configuration which may execute the modules described in the present exemplary embodiment. For example, some modules may be constituted by dedicated hardware (for example, an application specific integrated circuit (ASIC)), some modules may have a form where they exist in an external system and are connected via a communication line, and plural systems illustrated inFIG. 10may be connected to each other via a communication line and be operated in cooperation with each other. In addition, the hardware configuration may be incorporated into a copier, a facsimile, a scanner, a printer, a multi-function peripheral (an image processing apparatus having two or more functions of the scanner, the printer, the copier, and the facsimile), or the like.

In addition, the above-described program may be stored on a recording medium, or the program may be provided using a communication unit. In this case, for example, the above-described program may be understood as the invention of a “computer readable recording medium recording the program”.

The “computer readable recording medium recording the program” refers to a recording medium which is used to install and execute the program and distribute the program, records the program thereon and is capable of being read by a computer.

In addition, the recording medium includes, for example, “DVD-R, DVD-RW, DVD-RAM, or the like” which is a digital versatile disc (DVD) and is a standard formulated by the DVD forum, “DVD+R, DVD+RW, or the like” which is a standard formulated by DVD+RW, a read only memory (CD-ROM), a CD-recordable (CD-R), a CD-rewritable (CD-RW), or the like as a compact disc (CD), a Blu-ray disc (registered trademark), a magneto-optical disc (MO), a flexible disc (FD), a magnetic tape, a hard disk, a read only memory (ROM), an electrically erasable and rewritable programmable read-only memory (EEPROM (registered trademark)), a flash memory, a random access memory (RAM), a secure digital (SD) memory card, or the like.

The above-described program or a part thereof may be recorded on the recording medium so as to be reserved or distributed, for example. In addition, the program may be transmitted by communication, for example, wired networks such as a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, an intranet, and an extranet, or wireless communication networks. Further, the program may be transmitted using a combination of the recording media, or may be carried on a carrier.

Further, the above-described program may be a part of another program, or may be recorded on the recording medium along with a separate program. In addition, the program may be recorded on plural recording media so as to be divided. The program may be recorded in any form as long as the program may be recovered through compression, encoding, or the like.