Image processing apparatus, region detection method and computer-readable, non-transitory medium

An image processing apparatus includes a color component image generator for extracting a particular color component from an input image to generate a color component image, an expanded-color-component image generator for expanding particular pixels in the color component image to generate an expanded-color-component image, an edge pixel extractor for extracting edge pixels from the expanded-color-component image, and a document region detector for detecting a document region from the edge pixels.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority of prior Japanese Patent Application No. 2015-58643, filed on Mar. 20, 2015, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

Embodiments discussed in the present specification relate to detect a region from an image.

BACKGROUND

In order to read a document using a scanner or the like and store it as image data, it is necessary to accurately detect a document region in the read image. In a flatbed-type or automatic document feeding-type scanner in general, a document region can be easily distinguished from the other regions using a backing of a monotone color such as white, black, and the like. On the other hand, overhead scanners, which read a document placed on a desk or the like, are currently being used and the capability of accurately distinguishing between a pattern on a desk and a document is required of such scanners.

Overhead scanners are sometimes used for scanning a photograph affixed on a page of an album in which photographs can be affixed on self-adhesive pages. In such a case, it is necessary to accurately distinguish between a region of the page in which a stripe pattern of deteriorated glue appears and the photograph.

An image reading apparatus that obtains a document region from a read image has been disclosed. The image reading apparatus determines a method for cutting out a document region based on degree of coincidence between apex coordinates of a first rectangle region including all subjects and apex coordinates of a second rectangle region including all subjects determined as contents of the document among all of the subjects and having the smallest area (see Japanese Laid-open Patent Publication No. 2009-272676).

An image reading apparatus that generates reference data for detecting document edges using image data in a region in which a baking part appears across the full width of the region and obtains a difference between the reference data and image data to detect edges of a document has been disclosed. The image reading apparatus performs edge detection in given lines in the horizontal and vertical directions to detect the four sides of the document, thereby automatically recognizing the document image (see Japanese Laid-open Patent Publication No. 2007-88654).

An image processing apparatus that detects a boundary of a document in an input image has been disclosed. The image processing apparatus extracts edge pixels from an input image, extracts a plurality of straight lines from the edge pixels, extracts candidate rectangle regions from the extracted straight lines, and selects a rectangle region based on the number of edge pixels that are located within a predetermined distance from each side of each candidate rectangle region and the degree likelihood of being a corner of each possible corner (see Japanese Laid-open Patent Publication No. 2013-106160).

Furthermore, an image processing apparatus that determines a region in which an image of a document is cut out from a read image has been disclosed. The image processing apparatus detects a subject region in an input image and detects a rectangle region in the subject region. When the rectangle region matches the subject region, the image processing apparatus determines the rectangle region as the region to be cut out; when the rectangle region does not match the subject region, the image processing apparatus determines a region including the subject region as a region to be cut out (see Japanese Laid-open Patent Publication No. 2014-195148).

SUMMARY

There is the need for accurate detection of a region in which an image of a document is to be cut out from a read image to an image processing apparatus such as a scanner.

Accordingly, it is an object to provide an image processing apparatus and a region detection method that can accurately detect a region in which an image of a document is to be cut out from a read image and a computer-readable, non-transitory medium storing a computer program for causing a computer to implement such a region detection method.

According to an aspect of the apparatus, there is provided an image processing device. The image processing device includes a color component image generator for extracting a particular color component from an input image to generate a color component image, an expanded-color-component image generator for expanding particular pixels in the color component image to generate an expanded-color-component image, an edge pixel extractor for extracting edge pixels from the expanded-color-component image, and a document region detector for detecting a document region from the edge pixels.

According to an aspect of the method, there is provided a region detection method. The region detection method includes extracting a particular color component from an input image to generate a color component image, expanding particular pixels in the color component image to generate an expanded-color-component image, extracting edge pixels from the expanded-color-component image, and detecting, using a computer, a document region from the edge pixels.

According to an aspect of the computer-readable, non-transitory medium storing a computer program, the computer program causes a computer to execute a process, including extracting a particular color component from an input image to generate a color component image, expanding particular pixels in the color component image to generate an expanded-color-component image, extracting edge pixels from the expanded-color-component image, and detecting a document region from the edge pixels.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an image processing apparatus, a region detection method and a computer program according to an embodiment, will be described with reference to the drawings. However, it should be noted that the technical scope of the invention is not limited to these embodiments, and extends to the inventions described in the claims and their equivalents.

FIG. 1is a diagram schematically illustrating a configuration of the image processing system according to the embodiment. As illustrated inFIG. 1, the image processing system1includes an image reading apparatus10and an information processing apparatus20. The image reading apparatus10may be an image scanner, a digital camera or the like, for example. The information processing apparatus20may be a personal computer or the like, for example, connected and used with the image reading apparatus10.

The image reading apparatus10includes an image input device11, a first image memory12, a first interface device13, a first storage device14, and a first CPU (Control Processing Unit)15. Components of the image reading apparatus10will be described below.

The image input device11includes an image sensor imaging an image of a document or the like that is an imaging subject. The image sensor includes imaging elements and an optical system which forms an image of an imaging subject onto the imaging element. Each of the imaging elements outputs an analog value corresponding to each of RGB colors. The imaging elements are CCD (Charge Coupled Device) elements, CMOS (Complementary Metal Oxide Semiconductor) elements or the like arranged in a one-dimensional or two-dimensional array. The image input device11converts analog values output from the image sensor to digital values to generate pixel data and generates image data (hereinafter referred to as an RGB image) made up of the pieces of generated pixel data. The RGB image is color image data in which each piece of pixel data is made up of a 24-bit RGB value, for example, each of the RGB color components of which is represented by 8 bits.

The image input device11generates an RGB image or an image (hereinafter referred to as a read image) generated by converting the RBG values of the pixels in an RBG image into brightness values and color difference values (YUV values), and stores the read image in the first image memory12. Note that the YUV values can be calculated in accordance with the following expressions, for example.
Yvalue=0.30×Rvalue+0.59×Gvalue+0.11×Bvalue   (1)
Uvalue=0.17×Rvalue−0.33×Gvalue+0.50×Bvalue   (2)
Vvalue=0.50×Rvalue−0.42×Gvalue−0.08×Bvalue   (3)

The first image memory12includes a storage device such as a nonvolatile semiconductor memory, a volatile semiconductor memory, a magnetic disk, or the like. The first image memory12is connected to the image input device11and stores a read image generated by the image input device11.

The first interface device13includes an interface circuit compatible with a serial bus such as USB (Universal Serial Bus) and is electrically connected to the information processing apparatus20to transmit and receive image data and various kinds of information. A flash memory or the like may be connected to the first interface device13, image data stored in the first image memory12may be temporarily stored in the flash memory and then copied to the information processing apparatus20.

The first storage device14includes a memory device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a fixed disk device such as a hard disk or a portable storage device such as a flexible disk, an optical disc, or the like. Computer programs, databases, tables and the like which are used for various kinds of processing by the image reading apparatus10are stored in the first storage device14. Further, the first storage device14stores a computer program, a database, a table, and the like that are used for various processing of the image reading apparatus10. The computer program may be installed on the storage device104from a computer-readable, non-transitory medium such as a compact disk read only memory (CD-ROM), a digital versatile disk read only memory (DVD-ROM), or the like by using a well-known setup program or the like.

The first CPU15is connected to and controls the image input device11, the first image memory12, the first interface device13and the first storage device14. The first CPU15performs control of read image generation by the image input device11, control of the first image memory12, control of data transmission and reception to and from the information processing apparatus20through the first interface device13, control of the first storage device14, and other operations.

The information processing apparatus20includes a second interface device21, a second image memory22, a display device23, an input device24, a second storage device25, a second CPU26, and an image processing circuit27. The components of the information processing apparatus20will be described below in detail.

The second interface device21includes an interface circuit similar to that of the first interface device13of the image reading apparatus10and interconnects the information processing apparatus20and the image reading apparatus10.

The second image memory22includes a storage device similar to that of the first image memory12of the image reading apparatus10. The second image memory22stores a read image received from the image reading apparatus10through the second interface device21and is connected to the image processing circuit27to store various processed images resulting from image processing of read images by the image processing circuit27.

The display device23includes a display formed by a liquid-crystal display, an organic electroluminescence display or the like and an interface circuit which outputs image data to the display, and is connected to the second image memory22to display image data stored in the second image memory22on the display.

The input device24includes an input device such as a keyboard, a mouse or the like and an interface circuit acquiring signals from the input device and outputs a signal responsive to a user's operation to the second CPU26.

The second storage device25includes a memory device, a fixed disk device, a portable storage device or the like that is similar to that of the first storage device14of the image reading apparatus10. The second storage device25stores computer programs, databases, tables and the like used in various kinds of processing by the information processing apparatus20. Further, the second storage device25stores a computer program, a database, a table, and the like that are used for various processing of the information processing apparatus20. The computer program may be installed on the second storage device25from a computer-readable, non-transitory medium such as a CD-ROM, a DVD-ROM, or the like by using a well-known setup program or the like.

The second CPU26is connected to and controls the second interface device21, the second image memory22, the display device23, the input device24, the second storage device25and the image processing circuit27. The second CPU26performs control of data transmission and reception with the image reading apparatus10through the second interface device21, control of the second image memory22, control of display of the display device23, control of input of input device24, control of the second storage device25, control of image processing of the image processing circuit27, and other operations.

The image processing circuit27is connected to the second image memory22and performs document cutting processing. The image processing circuit27is connected to the second CPU26and operates in accordance with a program stored in the second storage device25beforehand under the control of the second CPU26. Note that the image processing circuit27may be formed by an independent integrated circuit, a microprocessor, firmware or the like.

FIG. 2is a diagram illustrating a schematic configuration of the image processing circuit27. As illustrated inFIG. 2, the image processing circuit27includes a first-color-component image generator201, an expanded-color-component image generator202, a first edge pixel extractor203, a second-color-component image generator204, a second edge pixel extractor205, an edge image generator206, a candidate region detector207, an outer edge pixel extractor208, a straight line detector209, a rectangle region detector210, an outer circumference edge pixel extractor211, a non-rectangle region detector212, a mask generator213, a document region detector214, a cutting module215and a storage controller216. Some of these components may be functional modules implemented by software or firmware running on a processor. Note that some of these components may be formed by independent integrated circuits, microprocessors or the like.

FIG. 3is a flowchart illustrating an operation of image reading processing performed by the image reading apparatus10. The operation of the image reading processing will be described below with reference to the flowchart inFIG. 3. Note that the flow of the operation described below is executed primarily by the first CPU15in cooperation with the components of the image reading apparatus10in accordance with a program stored in the first storage device14beforehand.

First, the image input device11images a document, which is an imaging subject, to generate a read image and stores the read image in the first image memory12(step S101).

Then the first CPU15transmits the read image stored in the first image memory12to the information processing apparatus20through the first interface device13(step S102), then ends the sequence of steps.

FIG. 4is a flowchart illustrating an operation of document cutting processing performed by the information processing apparatus20. The operation of the document cutting processing will be described below with reference to the flowchart illustrated inFIG. 4. Note that the flow of the operation described below is executed primarily by the second CPU26in cooperation with the components of the information processing apparatus20in accordance with a program stored in the second storage device25beforehand.

First, the second CPU26acquires a read image from the image reading apparatus10through the second interface device21and stores the read image in the second image memory22(step S201).

Then, the image processing circuit27retrieves the read image stored in the second image memory22, decimates pixels in the read image in the horizontal and vertical directions to generate a first input image, and stores the first input image in the second image memory22(step S202). The first input image is an example of an input image.

In general, the processing speed of the information processing apparatus20significantly varies depending on the number of pixels of images being processed. Therefore the rate of pixel decimation is determined in accordance with the throughput of the second CPU26and other components and the processing speed required of the information processing apparatus20. Note that when processing speed requirements are met without decimating pixels, the read image may be used as the first input image without decimation.

FIG. 5is a schematic diagram for illustrating an image in document cutting processing. The image500illustrated inFIG. 5is an example of a first input image. The first input image500is an image obtained by imaging, as a document, a photograph501affixed on a page of an album502in which photographs can be affixed on self-adhesive pages. The photograph501includes subjects such as people503. The page of the album502has a concavo-convex shape in a stripe pattern. Glue on the page of the album502is reddish due to deterioration and reddish portions and white portions appear as a stripe pattern in the first input image500. Further, random noise504produced by the influence of the imaging element and the optical system of the image input device11of the image reading apparatus10is imaged in the first input image500.

Each unit of the image processing circuit27executes cutting off region determination processing on the first input image (step S203) and then ends a series of steps. In the cutting off region determination processing, each unit of the image processing circuit27determines a cutting off region in which an image of a document is cut out from the first input image. The cutting off region determination processing will be described in detail later.

FIG. 6is a flowchart illustrating an example of an operation of the cutting off region determination processing. The operation flow illustrated inFIG. 6is executed at step S203of the flowchart illustrated inFIG. 4.

First, each unit of the image processing circuit27executes candidate region detection processing on the first input image (step S301). In the candidate region detection processing, the first edge pixel extractor203and the second edge pixel extractor205extract edge pixels from the first input image and the candidate region detector207detects a candidate region surrounded by connected edge pixels among the extracted edge pixels. The candidate region detection processing will be described in detail later.

Then the candidate region detector207determines whether or not a candidate region has been detected in the candidate region detection processing (step S302). When no candidate region has been detected, the candidate region detector207does not perform further processing and ends the sequence of steps; on the other hand, when a candidate region has been detected, the processing proceeds to step S303. Processing at steps S303through S312is performed for each extracted candidate region.

Then the components of the image processing circuit27perform rectangle region detection processing for the candidate region (step S303). In the rectangle region detection processing, the first edge pixel extractor203or the second edge pixel detector205extracts edge pixels and the outer edge pixel extractor208extracts outer edge pixels located at the outer side among the edge pixels. Furthermore, the straight line detector209detects a plurality of straight lines from the outer edge pixels and the rectangle region detector210detects a rectangle region consisting of four straight lines where two pairs of the detected plurality of straight lines intersect at substantially right angles. Details of the rectangle region detection processing will be described later.

Then, the document region detector214determines whether or not a rectangle region has been detected by the rectangle region detector210in the rectangle region detection processing (step S304). When a rectangle region has been detected by the rectangle region detector210, the document region detector214determines the detected rectangle region as the document region and as a cutting off region in which an image of the document is to be cut out from the first input image (step S305). In this way, the document region detector214detects a document region from the edge pixels extracted by the first edge pixel extractor203or the second edge pixel extractor205. Then, the cutting module215cuts out the determined cutting off region, i.e. the detected rectangle region, from the input image (step S306).

Then, the storage controller216stores the cut-out image in the second image memory22(step S307). The storage controller216stores the cut-out image in a storage medium, such as a nonvolatile semiconductor memory, that stores images in the long term basis in the second image memory22. Note that the storage controller216may store the cut-out image in a hard disk or the like of the second storage device25. The image stored in the second image memory22or the second storage device25is displayed on the display device23in response to request from a user using the input device24.

On the other hand, when no rectangle region has been detected by the rectangle region detector210, each unit of the image processing circuit27executes non-rectangle region detection processing on the candidate region (step S308). In the non-rectangle region detection processing, the first edge pixel extractor203and the second edge pixel extractor205extract edge pixels and the outer circumference pixel extractor211extracts outer circumference edge pixels that make up the outer circumference of connected edge pixels among the extracted edge pixels. Further, the non-rectangle region detector212detects a non-rectangle region surrounded by the outer circumference edge pixels and the mask generator213generates, based on the detected non-rectangle region, a mask for separating the background from an image to be cut out. The non-rectangle region detection processing will be described in detail later.

Then, the document region detector214determines a region including the non-rectangle region detected in the non-rectangle region detection processing as a cutting off region in which an image of the document is to be cut out from the first input image (step S309). The document region detector214determines the circumscribed rectangle region of the detected non-rectangle region as a cutting off region. Note that the document region detector214may determine the detected non-rectangle region itself as a cutting off region. In this way, the document region detector214detects a document region from the edge pixels extracted by the first edge pixel extractor203or the second edge pixel extractor205. Then, the cutting module215cuts out a region including the determined cutting off region from the first input image (step S310). Further, the cutting module215uses the mask generated by the mask generator213to separate the background from the cut-out image, thereby extracting a separated image that represents the document alone.

Then, the storage controller216stores the cut-out image and the separated image in the second image memory22(step S311). The storage controller216stores the cut-out image in a storage medium, such as a nonvolatile semiconductor memory, that persistently stores images in the second image memory22and stores the separated image in a storage medium, such as a volatile semiconductor memory, that temporarily stores images. Note that the storage controller216may store the cut-out image in a hard disk or the like of the second storage device25and store the separated image in a RAM. Further, the storage controller216stores the cut-out image as a file and stores the separated image in a clipboard, for example, to make the separated image available to a given application. This allows the user to use the separated image simply by executing a paste command in document editor software, image editor software, spreadsheet software or the like, thereby improving the convenience to the user.

Note that the storage controller216may store a cut-out image (and a separated image) in which a non-rectangle region has been detected in a file format different from the file format of an image in which a rectangle region has been detected. For example, when a rectangle region has been detected (at step S307), the storage controller216stores the cut-out image in the JPEG (Joint Photographic Experts Group) format. On the other hand, when a non-rectangle region has been detected (at step S311), the storage controller216stores the cut-out image in the transparent PNG (Portable Network Graphics) format. Thus, the information processing apparatus20can store an image in a format that is suitable for the shape of the image to improve the convenience to the user.

Then, the image processing circuit27determines whether or not the processing has been performed for all of the candidate regions (step S312). When there is a candidate region for which the processing has not been performed, the image processing circuit27returns to step S303and repeats the processing from step S303through step S312; when the processing has been performed for all of the candidate regions, the image processing circuit27ends a series of the steps.

FIG. 7is a flowchart illustrating an example of an operation of candidate region detection processing. The flow of the operation illustrated inFIG. 7is executed at step S301of the flowchart illustrated inFIG. 6.

First, the image processing circuit27further vertically and horizontally decimates pixels from the first input image to generate a second input image and stores the second input image in the second image memory22(step S401). The second input image is another example of an input image.

FIGS. 8A to 8FandFIGS. 9A and 9Bare schematic diagrams for illustrating each image in the candidate region detection processing. The image800illustrated inFIG. 8Ais an example of a second input image. The decimating rate from the first input image500to the second input image800is previously determined to such a value that random noise504produced by the influence of the imaging element and the optical system of the image input device11of the image reading apparatus10is removed.

Note that the image processing circuit27may remove random noise504by applying a smoothing filter to the first input image instead of generating the second input image. Alternatively, the first edge pixel extractor203may further apply a smoothing filter to the second input image.

Then, the first-color-component image generator201extracts a first color component from the second input image to generate a first-color-component image and stores the first-color-component image in the second image memory22(step S402). A component obtained by subtracting a blue color component from a red color component, for example, is set as the first color component so that a reddish region due to deterioration of glue on a page of an album can be properly distinguished from a white region.

The first-color-component image generator201generates a first-color-component image made up of pieces of pixel data each of which has the value (R value−B value) from the second input image. When each pixel of the second input image is represented by YUV values, the first-color-component image generator201reconverts the YUV values into RGB values and calculates (R value−B value). Note that the RGB values can be calculated in accordance with the following expressions:
Rvalue=1.000×Yvalue+1.402×Vvalue  (4)
Gvalue=1.000×Yvalue−0.344×Uvalue−0.714×Vvalue  (5)
Bvalue=1.000×Yvalue+1.772×Uvalue  (6)

The image810illustrated inFIG. 8Bis an example of the first-color-component image. In the first-color-component image810, a portion811that is reddish due to deterioration of glue on a page of an album and a reddish portion812in a photograph have high pixel values and the rest of the image810has low pixel values.

Note that the first-color-component image generator201may detect a color that is the largest in quantity included in the second input image and may determine the first color component based on the detected color. In that case, the first-color-component image generator201calculates the sum of R values, the sum of G values, and the sum of B values of the pixels in the second input image, detects the color component that has the largest sum, and determines a color component obtained by subtracting the other color components from the detected color component as the first color component. In this way, even when a background pattern tinged with a color other than red appears in the second input image, the first-color-component image generator201can properly extract the pattern to generate a first-color-component image. Note that since white contains red, green and blue components, white is also extracted as a first color component when red, green or blue is used as the first color component. The first-color-component image generator201therefore can inhibit white from being extracted as the first color component by subtracting, from the color component that has the largest sum, the other color components and using the resulting component as the first color component.

Then, the expanded-color-component image generator202determines an expansion coefficient for generating an expanded-color-component image, which will be described later, based on the pixels in the first-color-component image (step S403). The expanded-color-component image is used for submerging the pixels having high values of the first color component by expanding pixels having high pixel values in the first-color-component image to interconnect pixels corresponding to neighboring stripes among the stripes making up the stripe pattern.

For example, the expanded-color-component image generator202detects the cycle of occurrence of a color in the second input image and determines an expansion coefficient based on the detected cycle. In that case, the expanded-color-component image generator202horizontally or vertically scans pixels in the first-color-component image and calculates the number of changes from a pixel that has a pixel value greater than or equal to a predetermined value to a pixel that has a pixel value less than the predetermined value. The expanded-color-component image generator202divides the total number of pixels in the horizontal or vertical direction in the first-color-component image by the calculated number of changes to calculate the divided value as the cycle of occurrence of a color and determines the cycle of occurrence as the expansion coefficient. The expanded-color-component image generator202can use the expansion coefficient to appropriately interconnect pixels that correspond to neighboring stripes among the stripes making up a stripe pattern in an expanded-color-component image.

Note that the expanded-color-component image generator202may determine an expansion coefficient such that the number of first edge pixels extracted from an expanded-color-component image generated based on the expansion coefficient becomes less than or equal to a predetermined number (for example, 10% of the number of all the pixels in the image). In that case, the expanded-color-component image generator202sequentially increments the expansion coefficient, starting from1, and repeats processing at steps S404and S405, which will be described later, until the number of first edge pixels extracted from the expanded-color-component image decreases to a predetermined number or less. As a result, pixels that correspond to neighboring strips in the stripe pattern in the expanded-color-component image can be more reliably interconnected. Alternatively, the expanded-color-component image generator202may use a predetermined number (for example, 5) as the expansion coefficient.

The expanded-color-component image generator202then expands particular pixels in the first-color-component image to generate an expanded-color-component image and stores the expanded-color-component image in the second image memory22(step S404). Pixels that have higher pixel values than pixels surrounding the particular pixels are set as the particular pixels. The surrounding pixels are pixels that are located around a particular pixel and are pixels located within a predetermined distance (for example, within a 3-pixel) from the particular pixel, for example. The expanded-color-component image generator202replaces the pixel values of pixels in the first-color-component image with the highest pixel value among the pixel values of the pixels surrounding each pixel to generate an expanded-color-component image.

Note that expanding pixels having high pixel values shifts the position of an edge formed by an edge of the photograph inward from the original position of the edge of the photograph in the expanded-color-component image. Therefore, the expanded-color-component image generator202replaces the pixel value of each pixel in the expanded-color-component image with the lowest pixel value among the pixel values of the surrounding pixels surrounding each pixel to execute contraction processing on the expanded-color-component image. This can return the position of the edge formed by the edge of the photograph to the original position of the edge of the photograph in the expanded-color-component image.

The image820illustrated inFIG. 8Cis an example of the expanded-color-component image. In the expanded-color-component image820, a portion811of the first-color-component image810inFIG. 8Bthat is reddish due to deterioration of glue on the page of the album expands and the pixel values in the entire portion821that corresponds to the page of the album increase.

Then, the first edge pixel extractor203extracts first edge pixels in the horizontal direction and the vertical direction from the expanded-color-component image (step S405).

The first edge pixel extractor203calculates an absolute value of a difference between the brightness values of both neighboring pixels in a horizontal direction of each pixel of the expanded-color-component image (hereinafter, referred to as a neighboring difference value) and designates the pixel on the image as a vertical edge pixel when the neighboring difference value exceeds a threshold value Th1. It is possible to set this threshold value Th1as a value difference (for example, 20) where, for example, people can visually discriminate a difference between color components on an image. Further, the first edge pixel extractor203performs the same processing on the pixels in the vertical direction and extracts horizontal edge pixels.

Note that the first edge pixel extractor203may label the extracted vertical edge pixels and horizontal edge pixels to group them and may remove the edge pixels included in a group of a size in the horizontal or vertical direction smaller than or equal to a threshold value Th2. The threshold value Th2is set to a value equivalent to the possible size of dirt on the document placement surface (for example a value equivalent to 3 mm). Additionally, when the area of a group or the area of the circumscribed rectangle region of a group is less than or equal to a threshold value, the first edge pixel extractor203may remove the edge pixels included in the group. Using these methods, edge pixels appearing due to dirt on the document placement surface can be removed.

The image830illustrated inFIG. 8Dis an example of first edge pixels extracted by the first edge pixel extractor203. In the image830, a stripe pattern produced due to deterioration of glue are not extracted as first edge pixels and only the edges831of a photograph and the edges832of a reddish portion in the photograph are extracted as first edge pixels.

Then, the second-color-component image generator204extracts a second color component different from the first color component from the second input image to generate a second-color-component image and stores the second-color-component image in the second image memory22(step S406). For example, a red color component is set as the second color component so that both of a region that is reddish due to deterioration of glue on the page of the album and a white region can be extracted.

The second-color-component image generator204generates a second-color-component image made up of pixel data having an R value from the second input image.

The image840illustrated inFIG. 8Eis an example of the second-color-component image. In the second-color-component image840, pixels corresponding to reddish portions and white portions of the second input image have high pixel values and the other portions, i.e. pixels corresponding to the edges841of the photograph and to the other color components in the photograph have low pixel values.

Note that the second-color-component image generator204may detect a color that is the largest in quantity included in the second input image and may determine the second color component based on the detected color. In that case, the second-color-component image generator204calculates the sum of R values, the sum of G values, and the sum of B values of the pixels in the second input image, detects the color component that has the largest sum, and determines the detected color component as a particular color component. In this way, even when a background pattern tinged with a color other than red appears in the second input image, the second-color-component image generator204can generate a second-color-component image in which the pattern is submerged.

Then, the second edge pixel extractor205uses a threshold value Th3to extract second edge pixels in the horizontal direction and the vertical direction from the second-color-component image (step S407).

Like the first edge pixel extractor203, the second edge pixel extractor205calculates a neighboring difference value in the horizontal direction of each pixel of the second-color-component image and designates the pixel on the image as a vertical edge pixel when the neighboring difference value exceeds a threshold value Th3. Furthermore, the second edge pixel extractor205executes the same processing for each image in the vertical direction and extracts horizontal edge pixels. The threshold value Th3is a predetermined value and, it is possible to set the threshold value Th3as a value difference (for example, 20) where, for example, people can visually discriminate a difference between color components on an image.

Note that the second edge pixel extractor205may determine the threshold value Th3based on pixels in the second-color-component image. In that case, the second edge pixel extractor205calculates a neighboring difference value in the horizontal direction and a neighboring difference value in the vertical direction of each pixel in the second-color-component image and generates a histogram of the calculated neighboring difference values. The second edge pixel extractor205determines the threshold value Th3so that the threshold value Th3is equal to a value between neighboring difference values that respectively correspond to local maximum values in the generated histogram by using Otsu's binarization method, for example. By using the threshold Th3to binarize the neighboring difference values of each pixel in the second-color-component image, the reddish region and the white region can be properly separated from the other regions.

Furthermore, like the first edge pixel extractor203, the second edge pixel extractor205may remove the edge pixels included in a group of a size in the horizontal or vertical direction smaller than or equal to a threshold value Th2. Additionally, when the area of a group or the area of the circumscribed rectangle region of a group is less than or equal to a threshold value, the first edge pixel extractor205may remove the edge pixels included in the group. Using these methods, edge pixels appearing due to dirt on the document placement surface can be removed.

The image850illustrated inFIG. 8Fis an example of the second edge pixels extracted by the second edge pixel extractor205. In the image850, a stripe pattern produced due to deterioration of glue is not extracted as second edge pixels and only the edges851of a photograph and the edges852of a portion tinged with a color other than a reddish color in the photograph are extracted as second edge pixels.

The edge image generator206generates, as a first edge image, an image made up of the pixels extracted as the first edge pixels by the first edge pixel extractor203or the pixels extracted as the second edge pixels by the second edge pixel extractor205. The edge image generator206stores the generated first edge image in the second image memory22(step S408). Note that the edge image generator206may execute expansion/contraction processing for the generated first edge image. Even when part of connected portions in the second input image is not extracted as edge pixels, the portions can be connected in the first edge image.

The image900illustrated inFIG. 9Ais an example of the first edge image. In the first edge image900, a stripe pattern produced due to deterioration of glue is not extracted and only the edges901of a photograph and the edges902of people in the photograph are extracted.

Then, the edge image generator206determines whether or not each edge pixels is connected to another edge pixel in the first edge image, and labels connected edge pixels as one group (step S409). The candidate region detector207determines edge pixels adjacent to each other in the horizontal direction, vertical direction or oblique directions (eight neighborhoods) to be connected. Note that the edge image generator206may determine edge pixels adjacent to each other only in the horizontal or vertical direction (four neighborhoods) to be connected.

Then, the edge image generator206determines whether or not both sizes of each group in the horizontal and vertical directions are less than or equal to a threshold value Th4, and removes, from the edge image, edge pixels contained in a group where both sizes of the horizontal and vertical directions are less than or equal to the threshold value Th4(step S410). The threshold value Th4is set as a minimum size (for example, a value equivalent to 1 inch) of a document where the information processing apparatus40ensures detection. Note that the candidate region detector207may remove edge pixels when a size in any one of the horizontal and vertical directions of each group is less than or equal to the threshold value Th4or when an area of each group or an area of a circumscribed rectangle region of each group is less than or equal to the threshold value. The threshold value to be compared with the area of the circumscribed rectangle region may be a value equivalent to 1 square inch, for example.

Then, the edge image generator206determines whether or not each group is included in another group and removes, from the first edge image, edge pixels contained in a group included in another group (step S411).

The image910illustrated inFIG. 9Bis an example of the first edge image resulting from removal of the edge pixels contained in a group included in another group. In the image910, a group of edge pixels based on the content in the photograph, such as the people illustrated inFIG. 9A, are removed.

Then, with respect to each remaining group having not been removed, the candidate region detector207detects a region surrounded by edge pixels contained in the group and detects a region in the first input image corresponding to the detected region as a candidate region. The candidate region detector207stores the circumscribed rectangle region of the candidate region in the first input image in the second image memory22(step S412) and ends a series of steps.

In the image910illustrated inFIG. 9B, the region surrounded by connected edge pixels is a region including the outermost outer circumference911of the group of connected edge pixels and an inside thereof, and includes not only the edge pixels but also all pixels912present inside the outer circumference911.

Note that a region enclosed by connected edge pixels may be a region that does not include the outermost circumference of a group of connected edge pixels. Alternatively, a region enclosed by connected edge pixels may be a region where the outermost circumference of the group of the connected edge pixels and the region inside the outermost circumference are expanded or contracted by a predetermined number of pixels. Furthermore, extraction/contraction processing may be performed for connected edge pixels as described above to include not only pixels adjacent to each other but pixels located within a predetermined distance from each other and pixels between those pixels. Moreover, when edge pixels located at the both ends of connected edge pixels are within a predetermined distance from each other, pixels located between the edge pixels at the both ends may be included in the connected edge pixels. The predetermined distance may be a value equal to the length of the connected edge pixels multiplied by a predetermined factor (for example 80%), for example. This enables edge pixels at the both ends of edge pixels connected in a horseshoe shape, for example, to be interconnected to form an enclosed region.

FIG. 10is a flowchart illustrating an exemplary operation of the rectangle region detection processing. The flow of the operation illustrated inFIG. 10is executed at step S303of the flowchart illustrated inFIG. 6.

At steps S501through S507, the components of the image processing circuit27extract first edge pixels or second edge pixels to generate an image made up of the extracted pixels as a second edge image, as in steps S402through S408ofFIG. 7. The components of the image processing circuit27extract first edge pixels and second edge pixels in a candidate region.

The image processing circuit27may calculate a threshold value based on pixels in the candidate region and may use the calculated threshold value instead of the first threshold value to extract edge pixels and may generate an image made up of the extracted pixels as a second edge image. In that case, the image processing circuit27calculates a neighboring difference value for each of the pixels in the candidate region in the horizontal direction and a neighboring difference value for each pixel in the vertical direction and generates a histogram of the calculated neighboring difference values. The image processing circuit27then uses Otsu's binarization method, for example, to determine a threshold value and uses the determined threshold value to binarize the neighboring difference values of the pixels in the second input image, thereby extracting edge pixels.

Alternatively, the image processing circuit27may use other method to extract vertical edge pixels and horizontal edge pixels. For example, the image processing circuit27decimates the pixels in the first input image with different rates of decimation to generate a plurality of decimated images and extracts vertical edge pixels from each of the first input image and the decimated images to generate edge images made up of vertical edge pixels. The image processing circuit27increases the resolution of an edge image generated from the smallest decimated image to the resolution of an edge image generated from the next smallest decimated image to generate an image made up of vertical edge pixels located in the same positions in the two edge images with the same resolution. The image processing circuit27repeats the same processing on the generated image and an edge image generated from the next smallest decimated image, thereby ultimately extracting vertical edge pixels located in the same positions as the vertical edge pixels in an edge image generated from the first input image. Furthermore, the image processing circuit27performs the same processing to extract horizontal edge pixels. In this way, vertical edge pixels and horizontal edge pixels can be extracted while reducing noise.

Note that the components of the image processing circuit27do not perform expansion/contraction processing on the generated second edge image. This can prevent a pattern on the desk, noise and the like in the second edge image from being erroneously connected.

Then the outer edge pixel extractor208determines whether or not an edge of the candidate region is close to an edge of the first input image (step S508). When any of the left, right, top and bottom edges of the candidate region is located within a predetermined distance from the left, right, top or bottom edge of the first input image, the outer edge pixel extractor208determines that the edges are close to each other; when the edges are not within the predetermined distance, the outer edge pixel extractor209determines that the edges are not close to each other. The predetermined distance is chosen to be a value equal to a maximum distance from an edge of the first input image from which edge pixels is unlikely to be extracted although an object appearing in the candidate region intersects the edge of the first input image, and may be a distance equivalent to 10 mm, for example.

Note that when a candidate region does not fit in a predetermined document reading range in the first input image but instead extends beyond the document reading range, the outer edge pixel extractor208may determine that an edge of the candidate region is close to an edge of the first input image; when the candidate region fits in the document reading range, the outer edge pixel extractor208may determine that none of the edges of the candidate region are close to the edges of the first input image.

When the edges of the candidate region are not close to the edges of the first input image, the outer edge pixel extractor208extracts outer edge pixels located at the outer side among the extracted edge pixels. The outer edge pixel extractor208generates an image made up of the extracted outer edge pixels as a third edge image (step S509).

FIGS. 11A and 11Bare schematic diagrams illustrating the outer edge pixel extraction processing. The image1100illustrated inFIG. 11Ais an example of the second edge image. As illustrated inFIG. 11A, the outer edge pixel extractor208scans each horizontal line in the second edge image1100from left to right as indicated by arrows1101to extract, as an outer edge pixel, the first edge pixel1102detected. Similarly, the outer edge pixel extractor208scans each horizontal line in the second edge image1100from right to left to extracts, as an outer edge pixel, the first edge pixel1103detected. Similarly, the outer edge pixel extractor208scans each vertical line in the second edge image1100from top to bottom and bottom to top to extracts, as outer edge pixels, the first edge pixels1104,1105, detected. In other words, the outer edge pixel extractor208extracts the leftmost and rightmost edge pixels in each horizontal line in the candidate region and the uppermost and lowermost edge pixels in each vertical line in the candidate region as outer edge pixels.

The image1110illustrated inFIG. 11Bis exemplary outer edge pixels extracted. As illustrated inFIG. 11B, outer edge pixels1111are constituted by edge pixels1102-1105detected in the second edge image1100inFIG. 11A.

On the other hand, when an edge of the candidate region is close to an edge of the first input image, the outer edge pixel extractor208generates, as the third edge image, an image made up of all of the edge pixels in the candidate region (step S510). In this case, the rectangle region detector211detects a rectangle region from among all of the edge pixels in the candidate region in rectangle region detection processing, which will be described later.

FIGS. 12A and 12Bare schematic diagrams illustrating an example in which an edge of a candidate region is close to an edge of the first input image. The image1200illustrated inFIG. 12Aillustrates a situation in which a book1201is held by the hands1202and1203of a user. In this example, edge pixels that appear due to the presence of the hands1202and1203of the user do not fit into a document reading range1204but instead extend beyond the document reading range1204to edges of the first input image. The image1210inFIG. 12Billustrates an example of outer edge pixels extracted from the image1200inFIG. 12A. As illustrated inFIG. 12B, edges of the book1205and1206that are located interiorly to the hands1202and1203of the user are not extracted as outer edge pixels1211. In such a case, exclusion of edges of the document from extraction can be prevented by generating an image made up of all of the edge pixels in the candidate region as a third edge image.

This can also prevent exclusion of edges of the document from extraction due to interconnection between edge pixels detected in an edge of the document and edge pixels detected in a pattern on the desk when the edge strength of a pattern or the like on the desk on which the document is placed is high.

Note that when an edge of a candidate region is close to an edge of the first input image, the outer edge pixel extractor208may remove some of the edge pixels in the candidate region from the third edge image. For example, when an edge of the candidate region is close to a horizontal edge of the first input image, the outer edge pixel extractor208detects a vertical straight line in the candidate region; when an edge of the candidate region is close to a vertical edge of the first input image, the outer edge pixel extractor208detects a horizontal straight line in the candidate region. The outer edge pixel extractor208removes edge pixels between two straight lines detected at the outermost locations from the third edge image. This can exclude contents and the like in the document from the document edge detection to improve the accuracy of the document edge detection.

Then the straight line detector209detects a plurality of straight lines in the third edge image (step S511). The straight line detector209uses the Hough transform to detect straight lines. Note that the straight line detector209may use the least-square method to detect straight lines. Alternatively, the straight line detector209may use the least-square method for edge pixels within a predetermined distance from a straight line detected by using the Hough transform to detect a straight line. The predetermined distance is chosen as appropriate in accordance with the environment in which the image processing system1is used or other factors and may be set to a distance equivalent to 2 mm, for example.

The straight line detector209then detects approximate straight lines in the third edge image (step S512).

FIGS. 13A and 13Bare schematic diagrams for illustrating approximate straight lines. In a first input image1300acquired by imaging a book as illustrated inFIG. 13A, an edge1302parallel to the saddle-stitched portion1301of the book is detected as a straight line. However, an edge1303perpendicular to the saddle-stitched portion1301can distort at a portion near the saddle-stitched portion1301and therefore may not be detected as a straight line.

To avoid this, the straight line detector209labels adjacent edge pixels1311to cluster the edge pixels1311together into a group1312as illustrated in the edge image1310inFIG. 13B. The straight line detector209detects, as an approximate straight line, a straight line1315that links an edge pixel1313and an edge pixel1314that are located at the ends of the group1312in the horizontal direction or the vertical direction among the edge pixels included in the group1312.

Then the rectangle region detector210detects a rectangle region composed of the straight lines or approximate straight lines detected by the straight line detector209(step S513) and ends the sequence of the steps. In this way, the rectangle region detector210detects a rectangle region from the first edge pixels or the second edge pixels.

The rectangle region detector210extracts a plurality of candidate rectangle regions each of which is composed of four straight lines or approximate straight lines which consist of two pairs of the plurality of the straight lines or approximate straight lines detected by the straight line detector209intersecting at substantially right angles. The rectangle region detector210first selects a horizontal straight line (hereinafter referred to as a first horizontal line) and extracts a horizontal straight line (hereinafter referred to as a second horizontal line) that is substantially parallel (for example within ±3°) to the selected straight line and at a distance greater than or equal to a threshold value Th4from the selected straight line. The rectangle region detector210then extracts a vertical straight line (hereinafter referred to as a first vertical line) that is substantially orthogonal (for example within ±3° from 90°) to the first horizontal line. The rectangle region detector210then extracts a vertical straight line (hereinafter referred to as a second vertical line) that is substantially orthogonal to the first horizontal line and at a distance greater than or equal to a threshold value Th5from the first vertical line. Note that the threshold values Th4and Th5are predetermined in accordance with the size of a document to be read by the image reading apparatus10and may equal to each other.

The rectangle region detector210extracts all combinations of the first horizontal lines, second horizontal lines, first vertical lines, and second vertical lines that satisfy the conditions described above among all of the straight lines or approximate straight lines detected by the straight line detector209and extracts rectangle regions each of which is formed by each of the extracted combinations as candidate rectangle regions. The rectangle region detector210calculates the areas of the extracted candidate rectangle regions and removes candidate rectangle regions that have areas less than a predetermined value. The rectangle region detector210detects a candidate rectangle region that has the largest area among the remaining candidate rectangle regions as a rectangle region. On the other hand, when no candidate rectangle region has been left, the rectangle region detector210detects no rectangle region.

Note that the rectangle region detector210may calculate the ratio of the number of edge pixels within a predetermined distance (for example a distance equivalent to 2 mm) from each side of each candidate rectangle region to the total number of the pixels within the predetermined distance and may multiply the area of each candidate rectangle region by the ratio to assign a weight to the candidate rectangle region. This enables detection of rectangle regions composed of straight lines that clearly represent the border of the document. Alternatively, the rectangle region detector210may calculate, for each candidate rectangle region, a score for each angle of the candidate rectangle region that represents the likelihood of being a corner and may use the score to assign a weight to the area of the candidate rectangle region.

FIG. 14is a schematic diagram for illustrating the score of each corner of a candidate rectangle region. In the image1400illustrated inFIG. 14, a point1401represents a corner of a candidate rectangle region and straight lines1402,1403represent sides of a candidate rectangle region. When ends of the straight lines1402,1403are in contact with each other, the point1401is likely to be a corner of a rectangle region. When the ends of the straight lines1402,1403are not in contact with each other or the straight lines1402,1403cross each other, the point1401is unlikely to be a corner of a rectangle region.

The base of the scoring is 0. Determination is made as to whether or not there is an edge pixel in each horizontal line in a region1404within a predetermined distance from a side1402near the corner1401. When there is an edge pixel in a horizontal line, 1 is added to the score. The range near each corner and the predetermined distance from each side are chosen as appropriate according to the environment in which the image processing system1is used or other factors and may be a range within a distance equivalent to 5 mm from each corner and a distance equivalent to 2 mm from each side, for example. Similarly, determination is made as to whether or not there is an edge pixel in each vertical line in a region1405within the predetermined distance from a side1403near the corner1401. When there is an edge pixel in a vertical line, 1 is added to the score. Furthermore, determination is made as to whether or not there is an edge pixel in each horizontal line in a region1406within the predetermined distance from a line extended from the side1402near the corner1401. When there is an edge pixel in a horizontal line, 1 is subtracted from the score. Similarly, determination is made as to whether or not there is an edge pixel in each vertical line in a region1407within the predetermined distance from a line extended from the side1403near the corner1401. When there is an edge pixel in a vertical line, 1 is subtracted from the score. The rectangle region detector210normalizes the scores so that the lowest possible score is 0 and the highest possible score is 1 and multiplies the area of each candidate rectangle region by the normalized value to assign a weight, thereby the rectangle region detector210can detects a rectangle region that clearly represents four corners.

FIGS. 15A to 15Care schematic diagrams for illustrating rectangle region detection processing executed on non-rectangle image. The image1500illustrated inFIG. 15Ais an example of a first input image obtained by imaging a round-fan-shaped object1501that has a rectangular pattern1502and is affixed on a page of an album. The image1510illustrated inFIG. 15Bis an example of a third edge image generated from the first input image1500. As illustrated inFIG. 15B, the third edge image1510is made up of outer edge pixels1511and a stripe pattern1503and the rectangular pattern1502are not extracted in the third edge image1510. The image1520illustrated inFIG. 15Cis an example of straight lines extracted from the third edge image1510. As illustrated inFIG. 15C, handle portions1521to1523of the shape of the round fan are detected as straight lines from the third edge image1510. However, a rectangle region formed of four straight lines where two straight lines each are substantially at right angles to each other is not detected in the examples illustrated inFIGS. 15A to 15c.

FIG. 16is a flowchart illustrating an exemplary operation of non-rectangle region detection processing. The flow of the operation illustrated inFIG. 16is executed at step S308of the flowchart illustrated inFIG. 6.

At steps S601through S606, the components of the image processing circuit27extract first edge pixels or second edge pixels as in steps S402through S408ofFIG. 7or steps S501through S507ofFIG. 10. The components of the image processing circuit27generate an image made up of the extracted pixels as a fourth edge image. However, the components of the image processing circuit27extract first edge pixels or second edge pixels in the horizontal, vertical and diagonal directions in a candidate region.

The first edge pixel extractor203and the second edge pixel extractor205calculate neighboring difference values for each of the pixels in expanded-color-component image in the vertical direction from top to bottom, the horizontal direction from left to right, the diagonal direction from bottom left to top right, and the diagonal direction from bottom right to top left. When the absolute value of any of the calculated neighboring difference values exceeds the first threshold value, the first edge pixel extractor203and the second edge pixel extractor205determine pixels on the image as edge pixels.

Note that the first edge pixel extractor203or the second edge pixel extractor205may calculate a threshold value based on the pixels in a candidate region in the second input image, may use the calculated threshold value instead of the first threshold value to extract edge pixels. In that case, the first edge pixel extractor203or the second edge pixel extractor205calculates, for each pixel in the candidate region, a neighboring difference value in the horizontal direction, a neighboring difference value in the vertical direction and neighboring difference values in the diagonal directions and generates a histogram of the calculated neighboring difference values. The first edge pixel extractor203or the second edge pixel extractor205then determines a threshold value by using Otsu's binarization method, for example, and uses the determined threshold value to binarize the neighboring difference values for each pixel in the second input image, thereby extracting edge pixels.

Then the edge image generator206labels the extracted edge pixels to group the edge pixels. The edge image generator206removes the edge pixels included in a group having a size smaller than or equal to the threshold value Th2(step S608). This can remove noise components included in the fourth edge image.

Then the edge image generator206expands edge pixels in the generated fourth edge image to generate a second expansion image (step S609). When part of connected portions in the second input image is not extracted as edge pixels, the expansion can interconnect those portions in the second expansion image.

Then the outer circumference edge pixel extractor211executes labeling of pixels other the edge pixels (non-edge pixels) in the expansion image to extract a group that connects to an edge of the expansion image (step S610).

FIGS. 17A and 17Bare schematic diagrams illustrating images in the non-rectangle region detection processing. As in the image1700illustrated inFIG. 17A, a group1702that connects to the edges of the expansion image is a group of non-edge pixels located outside the outer circumference of a document1701.

Then the outer circumference edge pixel extractor211generates an image in which the pixels included in the extracted group are invalid pixels and the other pixels are valid images (step S611).

The image1710illustrated inFIG. 17Bis an example of the image generated at step S611. In the image1710, the pixels1712included in the extracted group are invalid pixels and the other pixels1711are valid pixels.

Then the outer circumference edge pixel extractor211contracts the valid pixels in the generated image to generate a construction image (step S612). The contraction processing is executed in order to restore the edge pixels expanded at step S609to the original size. Accordingly, the non-rectangle region detector212contracts the non-rectangle region by the same degree as the degree of expansion of the edge pixels expanded at step S609. Note that the non-rectangle region detector212may contract the non-rectangle region by a degree greater than the degree of expansion of the edge pixels expanded at step S609. In that case, edge pixels detected due to a pattern on the desk on which the document is placed or the like can be removed from the valid pixels when the edge strength of the pattern of the desk or the like is high.

The outer circumference edge pixel extractor211then extracts edge pixels making up the outer circumference of the valid pixels in the contracted image as outer circumference edge pixels (step S613).

As illustrated inFIG. 17B, the outer circumference edge pixels1713making up the outer circumference of the valid pixels1711are edge pixels that make up the outer circumference of connected edge pixels.

Then the non-rectangle region detector212detects a non-rectangle region surrounded by the extracted outer circumference edge pixels (step S614). When the non-rectangle region detector212detects a plurality of non-rectangle regions, the non-rectangle region detector212selects only the largest non-rectangle region in area as a detected non-rectangle region. This can remove noise generated due to a pattern on the desk or the like.

Then, based on the detected non-rectangle region, the mask generator213generates a mask for separating a background from an image to be cut out (step S615). The mask includes the same number of pixels as the pixels of the cut-out image in each of the horizontal and vertical directions and is formed by an image including valid pixels which are pixels corresponding to the detected non-rectangle region and invalid pixels which are the other pixels. The information processing apparatus20can extract a separated image by using the mask to extract only the pixels that correspond to the positions of the valid pixels in the mask from the cut-out image.

Note that the image processing circuit27may omit the candidate region detection processing illustrated at step S301ofFIG. 6and may execute the rectangle region detection processing and the non-rectangle region detection processing on the entire first input image. In that case, again, the information processing apparatus20can properly determine a region in which an image of a document is to be cut out.

Furthermore, the image processing circuit27may use edge pixels for detecting a candidate region to detect a rectangle region or a non-rectangle region instead of separately detecting edge pixels for detecting a candidate region, edge pixels for detecting a rectangle region, and edge pixels for detecting a non-rectangle region. In that case, the information processing apparatus20can reduce processing load of the edge pixel detection processing and can speed up the processing.

Furthermore, the image processing circuit27may omit the non-rectangle region detection processing at step S308ofFIG. 6, and may use a candidate region as a non-rectangle region when no rectangle region has been detected. In that case, the outer circumference edge pixel extractor211extracts edge pixels that make up the outer circumference of the candidate region as outer circumference edge pixels and the non-rectangle region detector212detects the candidate region as a non-rectangle region.

Furthermore, the image processing circuit27may omit any one of the processing at step S402through step S405and the processing at S406and S407ofFIG. 7and may generate, as a first edge image, an image made up of only any one of the first edge pixels and the second edge pixels. Similarly, the image processing circuit27may omit any one of the processing at step S501through step S504and the processing at S505and S506ofFIG. 10and may generate, as a second edge image, an image made up of only any one of the first edge pixels and the second edge pixels. Similarly, the image processing circuit27may omit any one of the processing at step S601through S604and the processing at S605and S606and may generate, as a fourth edge image, an image made up of only any one of the first edge pixels and the second edge pixels. In any of these cases, the information processing apparatus20is capable of accurately separating a document from a background.

As has been described above, the information processing apparatus20can accurately detect the region of an image of a document that is to be cut out by operating in accordance with the flowcharts illustrated inFIGS. 4, 6, 7, 10 and 16. In particular, the information processing apparatus20is capable of accurately detecting the region of a photograph with a high-contrast background such as an input image in which a photograph affixed on a page of an album in which photographs can be affixed on self-adhesive pages appears.

Furthermore, the information processing apparatus20can accurately detect the contour of an imaging subject and remove the background even when the imaging subject has a non-rectangular shape, such as a solid, a hand, or a photo clipping.

Furthermore, the information processing apparatus20uses only vertical and horizontal edge pixels as the edge pixels for detecting a rectangle region and uses diagonal edge pixels in addition to the vertical and horizontal edge pixels as the edge pixels for detecting a non-rectangle region. Thus, the information processing apparatus20can reduce the influence of noise due to a background or the like when detecting a rectangle region consisting of straight lines, which can be detected even when a small portion of edges is missing while preventing missing of a portion of edges when detecting a non-rectangle region, which has curves, which are difficult to detect when a portion of edges is missing.

FIG. 18is a diagram schematically illustrating a configuration of another image processing system2. The image processing system2illustrated inFIG. 18differs from the image processing system1illustrated inFIG. 1in that a different device includes an image processing circuit. In other words, in the image processing system2, an image reading apparatus30, instead of the information processing apparatus40, includes an image processing circuit36. The image processing circuit36has a function similar to the function of the image processing circuit27of the information processing apparatus20.

In the image processing system2illustrated inFIG. 18, processing substantially the same as the processing illustrated inFIGS. 3 and 4described previously can be executed. How the image reading processing illustrated in the flowchart ofFIG. 3and the document cutting processing illustrated in the flowchart ofFIG. 4are adapted will be described below. In the image processing system2, processing at step S101and processing at steps S202through S203are executed primarily by a first CPU35in cooperation with elements of the image reading apparatus30in accordance with a program that has been stored in a first storage device34beforehand.

At step S101, an image input device31of the image reading apparatus30takes an image of imaging subject to generate a read image and stores the read image in a first image memory32. Since document cutting process is performed in the image reading apparatus30, transmission and reception of the read image at steps S102and S201are omitted.

Processing at steps S202through S203is executed by the image processing circuit36of the image reading apparatus30. The operation of the processing is the same as the processing executed by the image processing circuit27of the information processing apparatus20described in relation to the image processing system1. The image processing circuit36of the image reading apparatus30transmits a cut-out region and a separated image to the information processing apparatus40through a first interface device33.

In this way, the mode in which the image reading apparatus30includes the image processing circuit36and executes the document cutting processing can achieve the same advantageous effects as the mode in which the information processing apparatus includes the image processing circuit to execute document cutting processing.

While preferred embodiments have been described above, it is not limited to the embodiments. For example, sharing of functions between an image reading apparatus and an information processing apparatus is not limited to the examples of the image processing systems illustrated inFIGS. 2 and 18; the components of the image reading apparatus and the information processing apparatus, including the components in the image processing circuit, can be provided any of the image reading apparatus and the information processing device as appropriate. Alternatively, the image reading apparatus and the information processing apparatus may be configured as a single apparatus.

Furthermore, the first interface device13and the second interface device21in the image processing system1may be interconnected through a network such as the Internet, a telephone network (including a mobile network and a fixed-telephone network) or an intranet. In that case, each of the first interface device13and the second interface device21is equipped with an interface circuit for connecting to the network. Additionally, in that case, a plurality of information processing apparatuses20may be distributed on the network and the information processing apparatuses20may cooperate with one another to perform the straight line detection processing, the rectangle region detection processing and other processing in a distributed manner, so that image processing services can be provided in the form of cloud computing. This enables the document cutting off processing to be efficiently performed by the image processing system1for images read by a plurality of image reading apparatuses10.

Similarly, the first interface device33of the image reading apparatus30and a second interface device41of the information processing apparatus40in the image processing system2illustrated inFIG. 18may be interconnected through a network.

According to the image processing apparatus, the region detection method and the computer-readable, non-transitory medium, it is possible to accurately detect a region in which an image of a document is to be cut out from a read image.