Patent Description:
Staining may occur in print products printed and outputted from printing apparatuses due to ink, toner, and other coloring materials adhering to unintended places. Alternatively, a color omission in which colors end up more faded than an original color in places where an image is formed due to insufficient coloring material adhering in those places may occur. So-called image defect, such as staining and color omissions, degrade the quality of printed products. Therefore, it is necessary to inspect the printed product for such defects and to guarantee the quality of the printed product.

Since visual inspection in which inspectors visually inspect the presence or absence of aforementioned defects requires a lot of time and cost, in recent years inspection systems that perform inspection automatically without relying on visual observation by a human have been proposed. Specifically, such systems determine the image quality of the printed product by detecting the presence or absence of defects by aligning a digital image (reference image), which is used for printing, and a scanned image obtained by scanning a printed product and executing image matching and determination processing.

<CIT> proposes a method of performing high-speed processing in an image matching technique for converting a digital image into a reference image and comparing the reference image and a scanned image. This method realizes high-speed processing by performing reference image generation processing in which rasterization (RIP) processing is performed on image data inputted as a job prior to executing the job. <CIT> also discloses an image inspection device which adjusts the detection threshold depending on color difference so as to avoid erroneous detection of defects.

However, there is a problem that if processing for generating all the reference images from image data inputted as a job is always executed prior to the job being executed, the time required for processing a job increases due to it taking time to generate the reference images.

Embodiments of the present disclosure eliminate the above-mentioned issues with conventional technology.

A feature of embodiments of the present disclosure is to provide a technique that allows efficient generation of a reference image while maintaining high-speed image matching performed by comparison with the reference image.

The present invention in its first aspect provides an inspection apparatus as specified in claims <NUM> to <NUM>.

The present invention in its second aspect provides a method of controlling an inspection apparatus as specified in claim <NUM>.

The present invention in its third aspect provides a computer-readable storage medium as specified in claim <NUM>.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure.

Embodiments of the present disclosure will be described hereinafter in detail, with reference to the accompanying drawings. It is to be understood that the following embodiments are not intended to limit the claims of the present disclosure, and that not all of the combinations of the aspects that are described according to the following embodiments are necessarily required with respect to the means to solve the issues according to the present disclosure.

<FIG> is a diagram illustrating an example of a configuration of a system that includes an inspection apparatus <NUM> according to a first embodiment of the present invention.

An image forming apparatus <NUM> performs print output by processing various kinds of input data. The inspection apparatus <NUM> receives a printed product printed and discharged by the image forming apparatus <NUM> and inspects the contents of the printed product. A finisher <NUM> receives output paper (the printed product) inspected by the inspection apparatus <NUM> and performs post processing, such as book binding. The image forming apparatus <NUM> is connected to an external print server or client PC via a network. The inspection apparatus <NUM> is connected to the image forming apparatus <NUM> on a one-to-one basis via a communication cable. The finisher <NUM> is connected to the image forming apparatus <NUM> on a one-to-one basis via another communication cable. The inspection apparatus <NUM> and the finisher <NUM> are also connected to each other via yet another communication cable. The image forming apparatus <NUM>, the inspection apparatus <NUM>, and the finisher <NUM> can thus communicate with each other. In the first embodiment, an in-line inspection system in which image formation, image inspection, and finishing are performed from start to finish is illustrated.

<FIG> is a block diagram for explaining a hardware configuration of the image forming apparatus <NUM> according to the first embodiment.

The image forming apparatus <NUM> is an example of the image forming apparatus according to the present invention and includes a controller <NUM>, a printer unit <NUM>, and a user interface (UI) unit (operation unit) <NUM>. The UI unit <NUM> includes various switches, a display, and the like for operation.

Image data and document data created by a software application, such as printer driver (not illustrated) on a client PC or a print server on a network, are transmitted to the image forming apparatus <NUM> as page description language (PDL) data via a network (e.g., a local area network). In the image forming apparatus <NUM>, the controller <NUM> receives the transmitted PDL data. The controller <NUM> is connected to the printer unit <NUM> and, upon receiving PDL data from the client PC or the print server, converts that PDL data into print data that can be processed by the printer unit <NUM> and outputs that print data to the printer unit <NUM>.

The printer unit <NUM> prints an image based on the print data outputted from the controller <NUM>. It is assumed that the printer unit <NUM> according to the first embodiment is an electrophotographic printer engine. However, the printing method is not limited thereto and may be, for example, an inkjet (IJ) method.

The UI unit <NUM> is operated by a user and is used by the user to select various functions and perform operation instructions. The UI unit <NUM> includes, for example, a display unit provided with a touch panel on the surface and a keyboard in which various keys, such as a start key, a stop key, and a numeric keypad, and the like are arranged.

Next, the details of the controller <NUM> will be described. The controller <NUM> includes a network interface (I/F) unit <NUM>, a CPU <NUM>, a RAM <NUM>, a ROM <NUM>, an image processing unit <NUM>, an engine interface (I/F) unit <NUM>, and an internal bus <NUM>. The network I/F unit <NUM> is an interface for receiving PDL data transmitted from the client PC or the print server. The CPU <NUM> controls the entire image forming apparatus <NUM> using programs and data stored in the RAM <NUM> or the ROM <NUM> and performs later-described processing performed by the controller <NUM>. The RAM <NUM> provides a work area that the CPU <NUM> uses when executing various kinds of processing. The ROM <NUM> stores computer programs and data for executing various kinds of later-described processing in the CPU <NUM>, setting data of the controller <NUM>, and the like.

The image processing unit <NUM> generates print data that can be processed by the printer unit <NUM> by performing image processing for printing on PDL data received by the network I/F unit <NUM> according to settings from the CPU <NUM>. The image processing unit <NUM> generates image data (RIP data) that has a plurality of color components per pixel, in particular by performing rasterization on the received PDL data. A plurality of color components are independent color components in a color space, such as red, green, and blue (RGB). The image data has a value of, for example, <NUM> bits (<NUM> tones) per color component for each pixel. That is, the image data is multi-value bitmap data that includes multi-value pixel data. In the aforementioned rasterization, attribute data, which indicates the attribute of the pixel of the image data for each pixel is also generated in addition to the image data. This attribute data indicates what type of object the pixel belongs to and is a value that indicates the type of object, such as a character, a line, a graphic, an image, or a background. The image processing unit <NUM> generates print data by performing image processing, such as color conversion from an RGB color space to a cyan, magenta, yellow, and black (CMYK) color space and screen processing using the generated image data and attribute data.

The engine I/F unit <NUM> is an interface for transmitting print data generated by the image processing unit <NUM> to the printer unit <NUM>. The internal bus <NUM> is a system bus for transmitting control signals and the like by connecting the respective above-described units.

<FIG> is a diagram for explaining a mechanism of the image forming apparatus <NUM> according to the first embodiment.

The image forming apparatus <NUM> includes a scanner unit <NUM>, a laser exposure unit <NUM>, photosensitive drums <NUM>, an image forming unit <NUM>, a fixing unit <NUM>, a feed/conveyance unit <NUM>, and a printer control unit <NUM>, which controls these. The scanner unit <NUM> generates image data by optically reading an image of a document placed on a document table by illuminating the document and converting that image into an electric signal. The laser exposure unit <NUM> causes a light beam, such as a laser beam modulated according to the image data, to enter a rotating polygonal mirror <NUM>, which rotates at constant angular speed, and thereby irradiates scanning light reflected off of the rotating polygonal mirror <NUM> onto the photosensitive drums <NUM>. The image forming unit <NUM> drives the photosensitive drums <NUM> to rotate, charges them with chargers, and develops latent images respectively formed on the photosensitive drums by the laser exposure unit <NUM> using toner. It then transfers that toner images to paper and collects the small amount of toner not been transferred at that time and remaining on the photosensitive drums; image formation is realized by having four developing units (developing stations), each for a series of electrophotographic processes as such.

The four developing units arranged in the order of cyan (C), magenta (M), yellow (Y), and black (K) sequentially execute magenta, yellow, and black image forming operations after a predetermined time has elapsed from the start of image formation of the cyan station.

The fixing unit <NUM> includes a roller, a belt, and the like and further contains a heat source, such as a halogen heater, and melts and fixes the toner onto paper to which toner images have been transferred by the image forming unit <NUM> using heat and pressure. When performing printing on thick paper, since the paper is thick and poor in heat conductance, the speed at which the paper passes the fixing unit <NUM> needs to be, for example, half of the usual speed. Accordingly, when performing printing on thick paper, paper conveyance speeds of respective units other than the fixing unit <NUM> are also halved, and thus, the printing speed of the image forming apparatus <NUM> itself is halved.

The feed/conveyance unit <NUM> includes at least one sheet stocker, which is typified by a paper cassette or a paper deck, and separates one sheet of paper (paper) from a plurality of papers stored in the sheet stocker according to an instruction from the printer control unit <NUM> and conveys the paper to the image forming unit <NUM>. The toner images of respective colors are transferred by the aforementioned developing station to the paper thus conveyed, and in the end, a full-color toner image is formed on the paper. When forming an image on both sides of the paper, control is performed such that upon passing through the fixing unit <NUM>, paper is made to pass through a conveyance path for conveyance to the image forming unit <NUM> again.

The printer control unit <NUM> communicates with the controller <NUM>, which controls the entire image forming apparatus <NUM>, and executes control according to its instruction. The printer control unit <NUM> performs instructions, such that the entire apparatus can maintain harmony and operate smoothly, while managing the states of the respective above-described scanner, laser exposure, image forming, fixing, and feed/conveyance units.

<FIG> is a diagram for explaining an overview of an internal configuration diagram of the inspection apparatus <NUM> according to the first embodiment.

Paper (printed product) outputted from the image forming apparatus <NUM> is pulled into the inspection apparatus <NUM> by a feed roller <NUM>. The printed product is then read by an inspection sensor <NUM> above a conveyance belt <NUM> while being conveyed by the conveyance belt <NUM>. The inspection apparatus control unit <NUM> performs inspection processing using the image data (scanned image) obtained by the inspection sensor <NUM> reading the printed product. The inspection apparatus control unit <NUM> also controls the entire inspection apparatus <NUM>. A result of such an inspection is sent to the finisher <NUM>. The inspected printed product is discharged by the discharge roller <NUM>. Although not illustrated here, the inspection sensor <NUM> may also have a structure in which the inspection sensor performs reading from the underside of the conveyance belt <NUM> so as to be capable of handling a double-sided printed product.

<FIG> depicts a top view of the conveyance belt <NUM> viewed from the inspection sensor <NUM> side.

Here, the inspection sensor <NUM> is a line sensor for reading an image of the entire surface of a printed product <NUM>, which has been conveyed as illustrated, for each line. An irradiation device <NUM> irradiates the printed product <NUM> at the time of reading by the inspection sensor <NUM>. A skew feeding detection irradiation device <NUM> is a device for detecting whether the printed product <NUM> is skewed with respect to a conveyance direction when being conveyed on the conveyance belt <NUM>. The skew feeding detection irradiation device <NUM> detects skew feeding of the printed product <NUM> by irradiating light onto the conveyed printed product <NUM> from a diagonal direction, so that the inspection sensor <NUM> reads an image of a shadow of an edge portion of the printed product <NUM>. In the first embodiment, a configuration is such that the inspection sensor <NUM> reads the image of the shadow of the edge portion of the printed product <NUM>; however, a configuration may be such that another reading sensor aside from the inspection sensor <NUM> is used.

<FIG> is a block diagram for explaining a configuration of the inspection apparatus control unit <NUM> of the inspection apparatus <NUM> according to the first embodiment.

A control unit <NUM> performs full control of the inspection apparatus control unit <NUM>. The control unit <NUM> includes a CPU <NUM>, and the CPU <NUM> executes various kinds of later-described processing by deploying programs stored in a memory unit <NUM> in a memory (not illustrated) of the control unit <NUM> and executing them. An image input unit <NUM> receives a scanned image obtained by reading a printed product by the inspection sensor <NUM>. The CPU <NUM> stores the received scanned image in the memory unit <NUM>. A communication unit <NUM> communicates with the controller <NUM> of the image forming apparatus <NUM>. This communication is reception of image data (a reference image), which is used for printing and corresponds to the scanned image, and transmission and reception of inspection control information. The CPU <NUM> also stores the received reference image and the inspection control information in the memory unit <NUM>.

One of the pieces of inspection control information exchanged with the image forming apparatus <NUM> is synchronization information for achieving correspondence between scanned images (inspection images) and reference images, such as print job information, copy count information, and page order information. Others are inspection result information and control information for controlling the operation of the image forming apparatus <NUM> accordingly. The synchronization information is necessary for synchronizing scanned images and reference images for when the order of scanned images received by the inspection apparatus <NUM> and reference images used for printing the scanned images are different due to double-sided printing or printing of a plurality of copies. The synchronization information is also necessary for synchronizing a reference image and scanned images due to there being cases where one reference image corresponds to a plurality of scanned images. The inspection control information to be exchanged between the inspection apparatus <NUM> and the finisher <NUM> is inspection result information and control information for controlling the operation of the finisher <NUM> accordingly.

The operation of an inspection processing module <NUM> is controlled by the CPU <NUM> of the control unit <NUM>. The inspection processing module <NUM> performs inspection processing for sequentially inspecting a corresponding scanned image and reference image pair based on the aforementioned synchronization information, which is one piece of the inspection control information exchanged with the image forming apparatus <NUM>. The details of the inspection processing module <NUM> will be described later. Upon completion of the inspection processing, a determination result thereof is sent to the control unit <NUM> and displayed on an operation/display unit <NUM>. If it is deemed that there is an image defect as a result of this determination, control of the image forming apparatus <NUM> and the finisher <NUM> is switched via the communication unit <NUM> by a method designated in advance by the user via the operation/display unit <NUM>. For example, processing, such as stopping image forming processing by the image forming apparatus <NUM> and switching a discharge tray of the finisher <NUM> to an escape tray, is performed.

Next, a functional configuration of the inspection processing module <NUM> will be described. The functions of the inspection processing module <NUM> are achieved by the CPU <NUM> deploying programs stored in the memory unit <NUM> into the memory of the control unit <NUM> and executing them.

A skew feeding detection module <NUM> is a module for detecting a skew angle of a scanned image. As previously described with reference to <FIG>, a scanned image is scanned such that a shadow is made at an edge portion of a printed product. This is for the inspection sensor <NUM> to scan a shadow made at an edge portion of a printed product when the skew feeding detection irradiation device <NUM> irradiates a printed product pulled into the inspection apparatus <NUM> and conveyed on the conveyance belt <NUM>. A skew angle of a printed product is detected using this shadow. Based on the skew angle thus detected, correction processing is performed in an image deformation module <NUM>, which will be described later.

A color conversion module <NUM> is a module for converting colors of a reference images into moderate colors. The reference image is rasterized in a CMYK color space by the image processing unit <NUM>. Meanwhile, a scanned image to be inspected is rendered in an RGB color space read by the inspection sensor <NUM>. Therefore, the reference image needs to be color-converted from the CMYK color space to the RGB color space; however, RGB values of the scanned image vary greatly depending on the type of paper on which an image to be inspected is printed. Here, conversion to RGB values common to respective types of paper is performed, and then, conversion to RGB values for each type of paper is performed by a paper type supporting module <NUM>, which will be described later. The RGB values common to respective types of paper is, for example, three-layer data in which each RGB signal that corresponds to sRGB, which is an RGB color space that is independent of the printer unit <NUM>, is represented by <NUM> tones. sRGB in the first embodiment refers to an RGB color space standard established by International Electrotechnical Commission (IEC). The color conversion module <NUM> thus converts the reference image into an RGB image. Here, for example, the conversion may be performed using a table (look-up table) for conversion from CMYK to RGB as illustrated in <FIG>. In such a case, a pixel on a grid point is color-converted to RGB with reference to this conversion table; however, for a pixel not on a grid point, an RGB value is obtained by interpolation from neighboring grid points.

A resolution conversion module <NUM> is a module for converting the resolutions of a scanned image and a reference image. The scanned image and the reference image may have different resolutions at the time of input to the inspection apparatus control unit <NUM>. The resolutions of images used in respective modules of the inspection processing module <NUM> and the resolution of an input image may be different. In such a case, resolution conversion is performed in the resolution conversion module <NUM>. For example, it is assumed that the scanned image is <NUM> dpi in main scanning and <NUM> dpi in sub-scanning and the reference image is <NUM> dpi in main scanning and <NUM> dpi in sub-scanning. When the resolution required by the inspection processing module <NUM> is <NUM> dpi for both main scanning and sub-scanning, the respective images are scaled down, and both images are made to be images that are <NUM> dpi for both main scanning and sub-scanning. For the scaling method here, a known method may be used, taking into account the computational load and required accuracy. For example, if scaling in which a SINC function is used is performed, the computational load will be heavy but a high accuracy scaling result can be obtained. In addition, if scaling in which a nearest-neighboring algorithm is used is performed, the computational load will be light but a low accuracy scaling result will be obtained.

An image deformation module <NUM> is a module for performing image deformation of a scanned image and a reference image. There is a geometrical difference between the scanned image and the reference image due to expansion and contraction of paper at the time of printing, skewing of the paper to be printed, skewing of the printed product at the time of scanning, and the like. The image deformation module <NUM> corrects that geometrical difference by performing image deformation based on information obtained in the skew detection module <NUM> or an alignment module <NUM>, which will be described later. For example, geometric differences are linear transformation (rotation, scaling, and shearing) and translation. A geometric difference can be expressed as an affine transform, and by obtaining affine transform parameters from the skew detection module <NUM> and the alignment module <NUM>, the geometric difference can be corrected. Information obtained from the skew detection module <NUM> is only a parameter (skew angle information) related to the rotation of an image.

The alignment module <NUM> is a module for aligning a scanned image and a reference image. It is assumed that a scanned image and a reference image inputted to the alignment module <NUM> are images that have the same resolution. The higher the resolution of the image, the higher the accuracy of alignment; however, the computational load increases as the resolution increases. By correcting the images in the image deformation module <NUM> based on the parameters obtained by alignment, a scanned image and a reference image to be used in a matching module <NUM>, which will be described later, can be obtained. Although various alignment methods are conceivable as the alignment method, in the first embodiment, a method of aligning the entire images using information of partial regions of the images rather than of the entire images is used in order to reduce the computational load. Alignment according to the first embodiment constitutes of three steps: selection of patches for alignment, alignment for each patch, and estimation of affine transform parameters. Each step will be described below.

First, the selection of patches for alignment will be described. Here, "patch" refers to a rectangular region in an image. In the selection of patches for alignment, a plurality of patches suitable for alignment are selected from the reference image. A patch that has a large corner feature in the patch is conceivable as a patch suitable for alignment. A corner feature is a feature such as there being two distinctive edges that have different directions (an intersection of two edges) in a given local neighborhood. A corner feature is a feature that expresses the strength of this edge feature. Various techniques have been devised based on differences in the modeling of "edge features".

There is a known method called Harris Corner Detection as one of the methods for calculating a corner feature. In Harris Corner Detection, a corner feature image is calculated from a horizontal differential image (the edge feature image in a horizontal direction) and a vertical differential image (the edge feature image in a vertical direction). This corner feature image is an image that represents the weaker edge feature of the two edges constituting the corner feature. Since both of the two edges should be strong edges for a corner feature, the magnitude of the corner feature is expressed according to whether an edge has a strong edge feature even if it is the relatively weaker one. The corner feature image is calculated from a reference image, and regions having a large corner feature are selected as patches suitable for alignment. When the regions having a large corner feature are simply selected as patches in order, it may be such that patches are selected only from biased regions. In such a case, the number of regions in which there is no patch in their periphery increases, and information of those regions cannot be used for image deformation, the state thus being unsuitable for performing alignment of the entire images.

Therefore, when selecting the patches, rather than simply considering the magnitude of the corner feature, it is also considered that the patches are arranged so as to be distributed in the image. Specifically, a configuration is taken such that even if a value of the corner feature of a patch candidate region is not large in the entire image, if the value is large in a local region of the image, it is selected as a patch. By doing so, it is possible to arrange the patches so as to be distributed in the reference image. Parameters for when selecting the patches include the size of the patches and the number (or density) of the patches. As the size of the patches and the number of patches increase, the accuracy of alignment increases; however, the computational load increases.

Next, the alignment for each patch will be described. Alignment for each patch is performed such that the patches for alignment in the reference image selected in the previous stage and corresponding patches in the scanned image are aligned.

There are two types of information obtained as results of the alignment: the first is the center coordinates (refpX_i, refpY_i) of an i-th (i = <NUM> to N; N is the number of patches) patch for alignment in the reference image and the second is a position (scanpX_i, scanpY_i) in the scanned image corresponding to the center coordinates (refpX_i, refpY_i). The alignment method may be any method so long as it is a shift amount estimation method by which a relationship between (refpX_i, refpY_i) and (scanpX_i, scanpY_i) is obtained. For example, a method such as that in which Fast Fourier Transform (FFT) is used to estimate a shift amount by bringing patches corresponding to the patches for alignment into a frequency domain and obtaining a correlation therein.

Finally, the estimation of affine transform parameters will be described. An affine transform is a coordinate transformation method expressed by an equation in <FIG>.

In this equation, there are six types of affine transform parameters: a, b, c, d, e, and f. Here, (x, y) corresponds to (refpX_i, refpY_i) and (x', y') corresponds to (scanpX_i, scanpY_i). Using this correspondence obtained from N patches, the affine transform parameters are estimated. For example, the affine transform parameters can be obtained using the least squares method. The reference image or the scanned image is deformed in the image deformation module <NUM> based on the obtained affine transform parameters. An image corrected for alignment is thus created, and a set of the reference image and the scanned image to be used in the matching module <NUM> can be made.

The paper type supporting module <NUM> is a module for performing processing according to the type of paper of the reference image. Specifically, the paper type supporting module <NUM> performs color conversion according to the type of paper and rotation of the image according to the orientation of paper. In this module, image processing is performed on the reference image according to the type and orientation of the paper based on the inspection control information stored in advance in the memory unit <NUM> and the paper information used for printing transmitted from the controller <NUM> of the image forming apparatus <NUM>. Regarding color conversion, the color conversion may be performed in the color conversion module <NUM> according to the type of paper based on the reference image converted into an RGB color space common to all types of paper. Regarding color conversion for each type of paper, the conversion is performed targeting RGB values read by the inspection sensor <NUM> for each type of paper. Regarding color conversion according to the type of paper, for example, the conversion may be performed using a table (look-up table) for color conversion from RGB to RGB as illustrated in <FIG>. A plurality of look-up tables may be held and switched according to the type of paper.

The paper type supporting module <NUM> then rotates the reference image. In this case, the image is rotated by <NUM> degrees according to the notified orientation of the printed product (paper). For example, as illustrated in <FIG>, even if the data is the same, a scanned image in which the image is rotated is obtained depending on the orientation of storage of paper in the sheet stocker. Therefore, it is necessary to rotate the reference image according to the orientation of the scanned image.

The matching module <NUM> is a module for matching the scanned image and the reference image processed by the paper type supporting module <NUM>. It is assumed that the scanned image and the reference image inputted to the matching module <NUM> are images that have the same resolution. It is also assumed that the reference image or the scanned image is corrected by the image deformation module <NUM> based on the information obtained by the alignment module <NUM> such that the images can be compared. The matching module <NUM> performs matching processing using the reference image and the scanned image. At this time, the matching processing is performed based on the parameters notified from the operation/display unit <NUM>.

The operation/display unit <NUM> is a touch screen user interface and receives settings for processing in the inspection processing module <NUM> from the user. For example, the operation/display unit <NUM> displays a settings screen as illustrated in <FIG> and receives settings for the image processing by the inspection processing module <NUM> from the user.

<FIG> depicts a view illustrating an example of an inspection settings screen to be displayed on the operation/display unit <NUM> of the inspection apparatus <NUM> according to the first embodiment.

Here, buttons <NUM> to <NUM> are provided as inspection settings that can be adjusted by the user, and the buttons <NUM> to <NUM> correspond to setting <NUM> to <NUM>. The user can perform the inspection setting corresponding to the selected button by selecting any of these buttons and pressing a "yes" button <NUM>. A "no" button <NUM> is a button for cancelling the settings on this screen.

Here, when the button <NUM> is selected and the "setting <NUM>" is set, the matching module <NUM> detects a defect when a color difference, such as staining and damage, determined by the inspection of the scanned image is "<NUM>" or more. Meanwhile, when the button <NUM> is selected and the "setting <NUM>" is set, the matching module <NUM> detects a defect when the color difference, such as staining and damage, determined by the inspection of the scanned image is "<NUM>" or more. As described above, the smaller the setting number of the inspection setting, the smaller the color difference, such as staining and damage, at which the matching module <NUM> detects a defect. By selecting a button corresponding to any one of the settings <NUM> to <NUM>, the user can thus perform inspection setting in the matching module <NUM>. Here, a color difference parameter is associated in advance with each setting value. Accordingly, the color difference parameter corresponding to the setting value selected by the user on the inspection settings screen is notified to the matching module <NUM>.

In the first embodiment, a color difference, which is staining and damage to be detected, has been adjusted using an inspection setting value; however, the present invention is not limited thereto, and for example, it may be the magnitude (size) of staining and damage to be detected. For example, a configuration may be taken so as to be able to adjust the magnitude (size) in a range of <NUM> to <NUM>, for example.

Inspection processing will be described below with reference to <FIG>.

<FIG> is a flowchart for explaining inspection processing by the inspection apparatus <NUM> according to the first embodiment. The processing described in this flowchart is realized by the CPU <NUM> of the control unit <NUM> executing a program stored in the memory unit <NUM>.

First, in step S601, the CPU <NUM> determines whether data to be printed in the current print job is previously printed data based on the inspection control information. Here, in order to determine whether it is previously printed data, a reprint flag or the like, for example, may be provided in the inspection control information. If it is determined to be previously printed data, the processing proceeds to step S612, and if it is determined not to be previously printed data, the processing proceeds to step S602. In step S602, the CPU <NUM> receives a print job via the communication unit <NUM>, obtains data of the print job, and stores it in the memory unit <NUM>.

The CPU <NUM> then performs preprocessing from step S603 to step S605. The CPU <NUM> performs processing on a reference image received from the image forming apparatus <NUM> and held in the memory unit <NUM> via the communication unit <NUM>. The processing here is performed independently of the paper, and a reference image obtained as a result is stored again in the memory unit <NUM>. First, in step S603, the CPU <NUM> performs color conversion on the reference image. Here, the color conversion module <NUM> converts the reference image into RGB values common to respective types of paper. Next, the processing proceeds to step S604, and the CPU <NUM> performs resolution conversion on the reference image. Here, the resolution conversion module <NUM> converts the resolution of the reference image to the resolution required by the inspection processing module <NUM>. Then, the processing proceeds to step S605, and the CPU <NUM> stores the converted reference image to the memory unit <NUM> with ID (identification information). This ID is assumed to be included in the inspection control information and may be a sequence of numbers, such as "<NUM>" representing the date and time of printing, so long as the reference image is uniquely recognized. For example, the identification information of the reference image may be specified in a print job that uses the reference image.

It is assumed that the reference image inputted here is a CMYK image that is <NUM> dpi in main scanning and <NUM> dpi in sub-scanning. By the above-described preprocessing, it is converted to an RGB image that is <NUM> dpi in both main scanning and sub-scanning and common to the respective types of paper, and the image size thus becomes very small. Specifically, the resolution is <NUM>/<NUM> and the number of channels is reduced by <NUM> channel. Therefore, the capacity of the memory unit <NUM> can be kept small.

Next, from step S606 to step S609, the CPU <NUM> performs immediately preceding processing such that the reference image is an image that can be compared with the scanned image of the image printed in the inputted job. In step S606, the CPU <NUM> selects a pair of the scanned image to be inspected and the reference image using the inspection control information received from the image forming apparatus <NUM> and held in the memory unit <NUM> via the communication unit <NUM>. Here, regarding the reference image, a corresponding reference image is obtained from the memory unit <NUM> based on the ID described in the inspection control information. Then, based on the paper information used for printing the image in the scan transmitted from the controller <NUM>, color conversion is performed according to the type of paper such that the scanned image can be compared with the reference image. Since the type and orientation of paper is not finalized until the print job is executed, the paper information used for printing is notified after the print job is executed.

Next, in step S607, the CPU <NUM> performs color conversion processing by the paper type supporting module <NUM> according to the type of paper of the printed product. Here, by converting the color space of the reference image that has been converted to RGB values common to the types of paper to the color space according to the type of paper, determination for matching with the scanned image can be accurately performed.

Next, the processing proceeds to step S608, and the CPU <NUM> rotates the reference image according to the orientation of paper in the paper type supporting module <NUM>. The orientation of paper is obtained from the inspection control information, and the reference image is rotated such that the orientations of the scanned image and the reference image coincide. Next, the processing proceeds to step S609, and the CPU <NUM> performs alignment using the scanned image and the reference image. Here, first, the CPU <NUM> obtains affine transform parameters by performing processing on the scanned image and the reference image in the alignment module <NUM>. Then, the CPU <NUM> makes the reference image usable for matching by making the coordinate system of the reference image to be the same as that of the scanned image by performing reference image correction processing in the image deformation module <NUM> using the affine transform parameters obtained from the alignment module <NUM>.

Then, the processing proceeds to step S610, and the CPU <NUM> performs matching and determination processing using the scanned image and the reference image obtained in step S609. At this time, the CPU <NUM> performs processing on the scanned image and the reference image in the matching module <NUM>. A reference value for determining whether there is an image defect at this time is based on the setting value set in the above-described inspection settings screen of <FIG>. Next, the processing proceeds to step S611, and the CPU <NUM> displays a result of the inspection processing of step S610 on the operation/display unit <NUM>. At this time, simply displaying an image of the final determination result makes it difficult for the user to recognize the image defect. Therefore, the final determination result image is composited with the scanned image and displayed on the operation/display unit <NUM>. Regarding this compositing, any compositing method may be used so long as the compositing method is that in which the location of the image defect is easily recognized. For example, a configuration may be taken so as to display in in the image of the final determination result the portion of the image defect in red on the scanned image.

Meanwhile, when it is determined in step S601 that the data to be printed in the current print job is previously printed data, the processing proceeds to step S612 and the CPU <NUM> determines whether the reference image of the previously printed data is stored in the memory unit <NUM>. This determination can be made by determining whether the reference image ID described in the inspection control information is stored in the memory unit <NUM>. If there is a coinciding ID, the reference image corresponding to that ID is employed and the processing proceeds to step S606; otherwise, the processing proceeds to step S602 in order to generate a reference image.

Here, a case of previously printed data will be described. When using previously printed data, the preprocessed data stored in the memory unit <NUM> in step S605 is employed, transition is made to step S606, and a reference image is generated. Whether it is previously printed data can be identified by determining in step S612 whether the ID stored with the reference image coincides with the ID described in the inspection control information of the inputted print job. Even if the type and/or orientation of paper is changed from those of the previous print job in the inputted print job, the reference image common to the types of paper generated during previous printing is stored. Accordingly, by using the stored reference image, it is possible to generate a reference image corresponding to the paper of the printed product to be inspected with minimal processing.

The stored reference image common to the types of paper may be provided with a storage expiration date in view of the capacity of the memory unit <NUM> or change in a profile according to the state of the image forming apparatus. A configuration may be taken so as to ensure a storable region in a storage unit by thus deleting the reference images generated before a predetermined period from the storage unit.

As described above, according to the embodiment, a reference image common to the types of paper generated during previous printing is stored such that it can be used in a subsequent corresponding job. Thus, if a subsequent print job corresponds to the stored reference image, even if the type and/or orientation of paper is changed in the subsequent print job, it is possible to perform image matching simply by performing processing for changing the type and/or orientation of paper on that corresponding reference image. Thus, the time required to generate a reference image is reduced, and matching of the scanned image and the reference image can be performed promptly.

Image processing related to a variation of the first embodiment of the present invention will be described below.

In the above-described first embodiment, a concrete example for when reprinting is performed has been described; however, the type and/or orientation of paper change not only according to the job. In the image forming apparatus <NUM>, if paper runs out during printing, a different sheet stocker may be automatically selected. There also are cases where although the type of paper is different, a similar type of paper is automatically fed or cases where paper whose orientation is different is fed.

Accordingly, in this variation, it is determined whether a corresponding reference image is stored not only when a job is inputted but also when the sheet stocker to be used in printing in the image forming apparatus is automatically or manually switched, and if it is stored, processing similar to that of the above-described first embodiment will be performed.

<FIG> is a flowchart for explaining inspection processing by the inspection apparatus <NUM> according to this variation. The processing described in this flowchart is realized by the CPU <NUM> of the control unit <NUM> executing a program stored in the memory unit <NUM>. In <FIG>, the same processing as that in above-described <FIG> is denoted by the same reference numerals, and description thereof will be omitted.

First, in step S1001, the CPU <NUM> determines whether paper has run out, and if paper has run out, the processing proceeds to step S1002, and waits until the paper outage is resolved by paper being supplied or the sheet stocker to be used being switched. When the paper is thus supplied or the sheet stocker to be used is thus switched in step S1002, the processing proceeds to step S1003, and the CPU <NUM> determines whether paper to be newly fed is the same type of paper. If it is determined in step S1003 that it is a different type of paper, the processing proceeds to step S1005; however, if it is determined in step S1003 that it is the same type of paper, the processing proceeds to step S1004, and it is determined whether the orientation of paper is the same as the orientation of paper printed thus far. If it is determined in step S1004 that the orientation of paper is different, the processing proceeds to step S1005; otherwise, the process proceeds to step S601. In step S1005, it is determined whether a reference image corresponding to the print data is stored as in step S612 of <FIG>, and if the reference image is stored, the processing proceeds to step S606 of <FIG> and color conversion according to the new type of paper is performed. Although step S601 is provided in <FIG>, a configuration may be taken so as to omit step S601 and proceed to step S602. In addition, a configuration may be taken such that if it is determined in step S1005 that there is a corresponding reference image and only the type of paper is different, the processing proceeds to step S606, and if only the orientation of paper is different, processing proceeds from step S1005 to step S607.

Thus, even if the type and/or orientation of paper is changed during printing, image matching can be performed by simply performing processing for changing the type and/or orientation of paper on a corresponding reference image. Thus, it is possible to efficiently perform processing for generating a reference image while maintaining high speed for when performing image matching.

In addition, a configuration may be taken so as to not only store the reference image common to the types of paper generated in previous printing in the apparatus itself but also transmit that reference image to another inspection apparatus such that it can be used in a plurality of inspection apparatuses or inspection systems.

In addition, by making it possible to select whether to create and store a common reference image, a configuration may be taken so as to immediately execute resolution conversion (step S604), color conversion according to the type of paper (step S606), and color conversion according to the orientation of paper (step S607) when not storing that reference image.

Embodiments of the present disclosure can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a 'non-transitory computer-readable storage medium') to perform the functions of one or more of the above-described embodiments and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiments, and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiments and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiments. The computer may comprise one or more processors (e.g., central processing unit (the CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The storage medium may include, for example, one or more of a hard disk, a random-access memory (the RAM), a read only memory (the ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.

Claim 1:
An inspection apparatus (<NUM>) operable to inspect an image printed on paper, the inspection apparatus comprising:
first image processing means (S603, S604) for performing first image processing on image data used in printing the image;
storing means (<NUM>) for storing image data for which the first image processing has been performed as a reference image, including identification information of the reference image;
determining means (S612) for determining whether a first reference image corresponding to a reference image that is able to be used in a job to be inspected is stored based on inspection control information and the identification information included in the job;
second image processing means (S607-S609) for, in a case that it is determined by the determining means that the corresponding first reference image is stored, performing second image processing on the first reference image to convert the first reference image into a second reference image corresponding to the job;
obtaining means (<NUM>) for obtaining a scanned image by reading an image of paper on which printing has been performed according to the job; and
matching means (<NUM>, S610) for performing processing for matching the obtained scanned image and the second reference image.