Image processing apparatus

An image processing apparatus includes a document placing table, a scanner configured to read a reading region on the document placing table from below the document placing table, an interface configured to communicate with an image capturing device which is positioned above the document placing table to capture an image of a capturing region on the document placing table from above the document placing table, and a processor configured to generate an image file in which first image data obtained by reading a target placed on the document placing table with the scanner and second image data obtained from the image capturing device via the interface, are associated with each other, wherein the second image data is generated from at least one image of the target captured by the image capturing device from above the document placing table.

FIELD

Embodiments described herein relate generally to an image processing apparatus.

BACKGROUND

In a digital multi-functional machine of the related art, when reading a spread document, such as a book, it is necessary for a user to set a page to be read on a document table of a scanner with the page to be read facing downward. When reading several spread documents consecutively, it is necessary for the user to repeat an operation of instructing the scanning after turning over the page of the book document and replacing the book downward on the document table. In addition, in the related art, there is also a book scanner in which a book document placed at a predetermined position in a state where the page to be read faces upward is read by a camera from above. However, even in such a digital multi-functional machine or the book scanner, the user must set the book document when reading a cover of the document and each page of the document each time. In addition, it is necessary for the user to manually set information that specifies the image data obtained by the scanning in this manner.

DETAILED DESCRIPTION

Embodiments provide an image processing apparatus which can easily generate a file in which an image obtained by reading a reading target from below and an image read from above are associated with each other without a manual operation.

According to an embodiment, an image processing apparatus includes a document placing table, a scanner configured to read a reading region on the document placing table from below the document placing table, an interface configured to communicate with an image capturing device which is positioned above the document placing table to capture an image of a capturing region on the document placing table from above the document placing table, and a processor configured to generate an image file in which first image data obtained by reading a target placed on the document placing table with the scanner and second image data obtained from the image capturing device via the interface, are associated with each other, wherein the second image data is generated from at least one image of the target captured by the image capturing device from above the document placing table.

FIG. 1is a view schematically illustrating an image processing system according to an embodiment.

The image processing system illustrated inFIG. 1includes a digital multi-functional machine (which is an example of an image processing apparatus)1and a portable terminal (which is an example of a capturing device)2. The digital multi-functional machine (also known as MFP, Multi-Functional Peripheral)1is an image processing apparatus including a scanner11that serves as an image reading device. The digital multi-functional machine1includes a printer12that serves as an image forming device, an operation panel13, and the like.

The portable terminal2includes a camera2aand is an image capturing device having a communication function with the digital multi-functional machine1. In the embodiment, the portable terminal2is a portable electronic device (e.g., a digital camera, a smartphone, a portable phone, a tablet PC, and the like) which is attachable to and detachable from the digital multi-functional machine1possessed by a user. However, the portable terminal2may be a device that can capture an image from above a document table11a, and may be an image capturing device which is standard equipment of the digital multi-functional machine1.

First, the details of the digital multi-functional machine1will be described.

As illustrated inFIG. 1, the digital multi-functional machine1includes the scanner11, the printer12, and the operation panel13. Furthermore, the digital multi-functional machine1includes a system processing unit15which is not illustrated inFIG. 1, an image processing unit16, an external I/F17(refer toFIG. 3), and the like. The system processing unit15is connected to the scanner11, the printer12, and the operation panel13, and performs control of each unit, data processing, and the like.

The scanner11is installed on an upper part of a main body of the digital multi-functional machine1. The scanner11is an image reading device which optically reads an image of a document and converts the image into image data. The scanner11includes the document table (which is an example of a placing table)11aand a document table cover11b. The document table11ais made of a transparent member, such as glass. The scanner11reads the document placed on the document table11afrom below the document table11a.

For example, the scanner11is a flatbed type image reading device including a carriage for optically scanning the document from below the document table11a. In addition, the document table cover11bis installed to be freely openable and closable. The document table cover11bcovers a reading region where the document is placed in the document table11ain a closed state. The document table cover11bincludes a mechanism which is stationary in an open state. For example, the document table cover11bincludes a mechanism which is stationary in a state where an angle formed by a document placing surface in the document table11ais a predetermined angle, and a mechanism which is stationary in a full open state where the angle formed by the document placing surface is open by approximately 90°.

The printer12prints the image on a paper sheet that is an image forming medium. For example, the printer12prints the image read by the scanner11or an image captured by the camera2aof the portable terminal2on the paper sheet. In addition, the printer12may print the image on a paper sheet based on image data received via a network or the like. An image forming method of the printer12is not limited to a particular method. For example, the printer12may be an electrophotographic printer, an ink jet type printer, or a thermal transfer type printer.

The operation panel13is a user interface. The operation panel13includes a display unit13aand an operation unit13b. The display unit13adisplays guide and the like. The operation unit13bincludes an operation button or a touch panel for receiving an input to a button (e.g., icon) displayed on the display unit13a. The user instructs the start of scanning on the operation panel13and confirms the result of the scanning. For example, reading of a book document in which the digital multi-functional machine1and the portable terminal2are linked is executed in response to an operation instruction on the operation panel13.

In addition, the digital multi-functional machine1includes a holding unit14for holding the portable terminal2above the document table11a. In the example illustrated inFIG. 1, the holding unit14is installed on the document table cover11bas a recess in which the camera2ais accommodated. In addition, the holding unit14is provided at a position where the camera2aof the portable terminal2can capture an upper surface of the book document placed on the reading region of the document table11a, in the document table cover11b. The holding unit14includes a mechanism (e.g., a pair of movable tabs) for holding the portable terminal2in place within the recess. The holding unit14has a cover attached thereto instead of the portable terminal2when the portable terminal2is not used. When attaching the portable terminal2, the cover is removed from the holding unit14and the portable terminal2is held by the holding mechanism of the holding unit14.

In addition, the holding unit14holds the portable terminal2in a state where the camera2afaces the document table11aand the document table cover11bis open at a predetermined angle. The holding unit14may be a unit that can hold the portable terminal2such that a capturing range of the camera2aincludes the entire reading region on the document table11a. For example, the holding unit14is configured such that the camera2aof the portable terminal2to be fixed can capture the entire surface of the document table11ain a state where the document table cover11bis stationary at a predetermined angle. In addition, the holding unit14may include a sensor for detecting that the portable terminal2is set thereon.

In addition,FIG. 2is a view illustrating another example of the holding unit provided in the digital multi-functional machine1.

FIG. 2illustrates an example in which the holding unit14inFIG. 1is replaced with a holding unit14′. Therefore, a detailed description except the holding unit14′ illustrated inFIG. 2will be omitted.

The holding unit14′ illustrated inFIG. 2includes a pair of supports for supporting the portable terminal2. The holding unit14′ is provided in a side end portion on a near side of the document table cover11b. The holding unit14′ has a first support and a second support which nip the portable terminal2. The first support and the second support hold the portable terminal2such that the camera2afaces the document table11ain the side end portion of the document table cover11bin an open state.

For example, in the holding unit14′ illustrated inFIG. 2, in a state where the document table cover11bis fully open, the portable terminal2is placed such that the camera2afaces downward. Accordingly, the camera2aof the portable terminal2held by the holding unit14′ is substantially parallel to the surface on which the document of the document table11ais placed. In addition, in the first support and the second support which make up the holding unit14′, installation units may be respectively installed to be rotatable around the rotation center in the side end portion of the document table cover11b. Accordingly, each of the supports that serve as the holding unit14′ may be folded and stored so as not to obstruct the opening and closing of the document table cover11bwhen the portable terminal2is not used.

The digital multi-functional machine1including the holding unit14illustrated inFIG. 1is described in the following.

Next, the control system of the digital multi-functional machine1will be described.

FIG. 3is a block diagram for explaining an example of the control system in the digital multi-functional machine1.

The digital multi-functional machine1includes the system processing unit15for controlling the entire apparatus. The system processing unit15is connected to the scanner11, the printer12, the operation panel13, the image processing unit16, and the external interface (I/F)17. In addition, the scanner11is connected to the system processing unit15via a scanner interface18. The printer12is connected to the system processing unit15via a printer interface19.

In addition, in the example illustrated inFIG. 3, the system processing unit15includes a processor21, a RAM22, a ROM23, a page memory24, an HDD25, and the like.

The processor21performs control of each unit, data processing, and the like. The processor21is, for example, a CPU. The processor21is connected to each unit in the system processing unit15via a system bus. In addition, the processor21is also connected to the scanner11, the printer12, the operation panel13, the image processing unit16, the external I/F17and the like, via the system bus.

For example, the processor21outputs operation instructions to each of the units by bidirectional communication with the scanner11, the printer12, the operation panel13, the image processing unit16, and the external I/F17, or obtains various types of information from each of the units. In the embodiment, the processor21obtains a rear surface image scanned by the scanner11via the scan interface. In addition, the processor21obtains a front surface image captured by the camera2afrom the portable terminal2via the external interface.

The RAM22functions as a working memory or a buffer memory. The ROM23is a non-rewritable, nonvolatile memory for storing programs, control data, and the like. The processor21realizes various types of processing by executing a program stored in the ROM23or the HDD25while using the RAM22. For example, by executing the program, the processor21performs processing, such as image cutting-out, skew correction, document size detection, OCR, file conversion, data transfer, and the like.

The page memory24is a memory for storing the image data which is a target of the processing. For example, the page memory24stores image data read by the scanner11or image data obtained from the portable terminal2via the external I/F17. In addition, the page memory24also stores the image data and the like during image processing.

The hard disk drive (HDD)25is a rewritable, nonvolatile memory. The HDD25is an example of a storage device. Another example of the storage device is an SSD. The HDD25stores control programs and control data executed by the processor21. In addition, the HDD25includes an information management table25afor managing information obtained by scanning a book document which will be described later. The information management table25awill be described in detail later.

The image processing unit16includes an input processing unit16aand an image quality improvement unit16b. The input processing unit16aprocesses the input image data. For example, the input processing unit16aperforms resolution conversion, brightness adjustment, contrast adjustment, saturation adjustment, sharpness adjustment, and the like, with respect to the input image data. The image quality improvement unit16bperforms a process of improving the image quality of the image data. In addition, the image processing unit16has a function of converting the image data into image data for printing (for example, image data of YMCK). Further, the image processing unit16may have a compression processing function of compressing the image data and an expansion processing function of expanding the compressed image data.

The external I/F17includes an interface for communicating with the image capturing device. In the embodiment, the external I/F17includes an interface for communicating with the portable terminal2attached to the holding unit14or14′. The external I/F17for communicating with the portable terminal2may be an interface that corresponds to the communication function of the portable terminal2. For example, the external I/F17is an interface that performs wireless communication, such as wireless LAN or short range wireless communication that serves as a wireless communication function of the portable terminal2. In addition, the external I/F17may come into contact with the interface of the portable terminal2to communicate therewith.

In addition, the external I/F17may include a plurality of interfaces. In other words, the external I/F17may include an interface other than the interface for communication with the portable terminal2. For example, the external I/F17may include an interface for network communication, such as a LAN, separately from the interface for communication with the portable terminal2. Further, the external I/F17may include an interface for serial communication with a device, such as an external memory device.

In addition, the system processing unit15functions as a control unit of the scanner11, and the scanner11and the system processing unit15may make up the image reading device. Further, a processor provided in the scanner11may have a corresponding function to the processor21of the system processing unit15which will be described later. In other words, the processing of the embodiment which will be described later may be realized by an image reading device which includes a stand-alone scanner and no printer.

Next, image processing focusing on the image processing unit16and the system processing unit15in the digital multi-functional machine1will be described.

FIG. 4is a block diagram illustrating a flow of each processing block and image data in the digital multi-functional machine1.

As illustrated inFIG. 4, image data is input from the scanner11or the external I/F17in the digital multi-functional machine1. For example, the image read by the scanner11from below the document table11ais supplied to the image processing unit16. In addition, the portable terminal2installed in the holding unit14communicates with the processor21of the system processing unit15via the external I/F17. The processor21supplies a control signal to the portable terminal2, and accordingly, the camera2acaptures the image from above the document table11a. The portable terminal2transmits the image captured by the camera2ato the external I/F17. The image captured by the camera2aof the portable terminal2input via the external I/F17is supplied to the image processing unit16as an input image.

The input processing unit16aof the image processing unit16performs image processing for the input image with respect to the image data input from the scanner11or the external I/F17. For example, the input processing unit16aincludes a LUT processing unit and performs processing for correcting tone reproduction in the input image data. The LUT processing unit converts correction of scanner gradation reproduction skipping a highlight side by LUT data set by the processor21with respect to input image data.

The image quality improvement unit16bof the image processing unit16performs the image quality improvement on the image data from the input processing unit16aor the page memory24. For example, the image quality improvement unit16bhas a function of performing filter processing, scaling processing, density adjustment processing, gradation processing, and the like, with respect to image data. The image quality improvement unit16bexecutes image quality improvement with respect to the image data in accordance with the control from the processor21.

The processor21of the system processing unit15controls each unit and executes various types of image processing. In the example illustrated inFIG. 4, the processor21performs image processing, such as image cutting-out, skew correction, size detection, OCR, file conversion, data transfer, and the like. For example, the processor21executes various types of processing in response to a user operation on the operation panel13.

The image cutting-out is processing of cutting out an image data region surrounded by edges based on the image data stored in the page memory24. In the skew correction, an inclination of the document in the image data cut out by the image cutting-out is corrected. In the skew correction, the processor21analyzes an inclined state of the document region in the image data, and corrects the inclination when there is an inclination. The processor21temporarily stores the image data after the skew correction in the page memory24.

The size detection is processing for specifying the size of the document region in the image data. For example, the processor21specifies the document size based on the image data after the skew correction stored in the page memory24.

The OCR processing is processing for recognizing letters in image data. For example, the processor21specifies the target region of the OCR processing in the image data stored in the page memory24, and recognizes the letters in the target region. The target region of the OCR processing may be a predetermined region in the document region, or may be designated by the user through the operation panel13.

File conversion is processing for generating a file including an image. In a case of executing the reading of the book document, the processor21generates an image file including an image of a cover of the book document to be read by the scanner11(hereinafter, referred to as a cover image) and an image of an internal page of the book document captured by the camera2a(hereinafter, referred to as a page image). The processor21generates an image file related to the reading of one book document by using the cover image, the page image, the processing result of OCR, and the like.

For example, the processor21stores each piece of information related to the reading of the book document in the information management table25a. Based on the information in the information management table25a, the processor21performs merging the cover image and the page image, and generates a file in a predetermined format. In addition, the processor21stores the generated file in the HDD25as an image file to which a file name including information, such as a book name obtained by the OCR processing, is given.

The data transfer is processing of transferring the generated image file. For example, the processor21transfers the image file generated based on designation and the like of the user by the external I/F17to portable terminal2. In addition, the processor21may store the image file in response to the designation of the user by the external I/F17in an external device different from the portable terminal2.

Next, the reading of the book document using the portable terminal2and the digital multi-functional machine1will be described.

In the embodiment, an application program (hereinafter, referred to as an application) for reading the book document is installed in the portable terminal2. The application for reading the book document is stored in the nonvolatile memory in the portable terminal2and executed by the processor in the portable terminal2. The processor of the portable terminal2realizes functions, such as communication with the digital multi-functional machine1, image capturing with the camera2ain accordance with control from the digital multi-functional machine1, transfer of the captured image to the digital multi-functional machine1, by executing the application.

The user sets the portable terminal2in the holding unit14of the digital multi-functional machine1and selects a book document scanning mode with the operation unit13b. When the book document scanning mode is selected, the digital multi-functional machine1communicates with the portable terminal2set in the holding unit14. In response to the scan instruction, the digital multi-functional machine1executes image reading with the scanner11and image capturing with the camera2aof the portable terminal2.

Here, the book document is placed on the document table11ain a state of being open such that the page to be read is facing upward. In other words, the book document is placed such that the cover is in contact with the document table11a. Therefore, the scanner11reads an image including the image of the cover of the book document placed on the document table11a.

FIG. 5is a view illustrating an example of an image to be read by the scanner11in the reading of the book document.

The scanner11reads the image of the reading region in the document table11afrom below the document table11a. In a state where the book document is placed on the document table11a, the scanner11reads an image including the image of the cover of the book document. In addition, in the embodiment, the scanner11reads an image in a state where the document table cover11bis open. Therefore, as illustrated inFIG. 5, a scanned image to be read by the scanner11has the maximum density (e.g., black) in a region where the document does not exist.

In a state where the portable terminal2is held by the holding unit14, the camera2acaptures an image including a front surface of the book document which is open upward, on the document table11afrom above the document table11a. The portable terminal2is remotely controlled in response to the digital multi-functional machine1. The portable terminal2captures the image of each page with the camera2ain response to the instruction from the digital multi-functional machine1. In addition, the portable terminal2transmits the image captured by the camera2ato the digital multi-functional machine1.

FIGS. 6 and 7are views illustrating examples of images captured by the camera2aof the portable terminal2in the reading of the book document.FIG. 6is an example of an image obtained by capturing a capturing region including the document table11aon which the book document in a state where the first page including an image A and an image B is open is placed, by the camera2aof the portable terminal2.FIG. 7is an example of an image obtained by capturing a capturing region including the document table11aon which the book document in a state where the second page, including an image C and an image D is open is placed, by the camera2aof the portable terminal2.

In other words, the camera2aof the portable terminal2captures an image of the capturing region including the entire document table11afrom above the document table11a. The user instructs the reading on the operation panel13of the digital multi-functional machine1in a state where the book document having a desired page open is placed on the document table11a. The digital multi-functional machine1instructs the portable terminal2to capture the image with the camera2ain response to the instruction of the user. The portable terminal2captures an image of a capturing region including an image of the page of the book document with the camera2ain response to the instruction from the digital multi-functional machine1. As illustrated inFIG. 6 or 7, the image captured by the camera2aincludes an image of an opened page in the book document. The portable terminal2transfers the captured image as illustrated inFIG. 6 or 7to the digital multi-functional machine1.

The digital multi-functional machine1generates a book image file in which a cover image to be read by the scanner11and images of each page captured by the camera2aof the portable terminal2are associated with each other. When the camera2aof the portable terminal2captures a plurality of pages of the book document, the book image file is obtained by merging the cover image and the images of the plurality of pages.

FIG. 8is a view schematically illustrating a book image file in which the cover image read by the scanner11and the page image captured by the camera2aare associated with each other.FIG. 8is a view schematically illustrating a book image file in which the cover image included in the scanned image illustrated inFIG. 5and the page images for two pages included in the captured images illustrated inFIGS. 6 and 7are associated with each other.

The digital multi-functional machine1generates a cover image of the book document by performing image processing, such as cutting-out of the document region with respect to the scanned image illustrated inFIG. 5. In addition, the digital multi-functional machine1generates an image of each page by performing the image processing, such as cutting-out of the document region with respect to each of the captured images illustrated inFIGS. 6 and 7. The digital multi-functional machine1generates a book image file by associating the image of the cover obtained from the scanned image of the scanner11with the image of each page obtained from the captured image of the camera2a.

In addition, the digital multi-functional machine1obtains book information, such as a book name or a management number of the book document, by executing the OCR processing with respect to the cover image to be read by the scanner11. The digital multi-functional machine1generates a book document file in which the book information obtained by the OCR processing and the image data on the front and rear sides are associated with each other. In addition, the digital multi-functional machine1may give a file name including a part of the book information obtained by the OCR processing to the generated book document file.

The digital multi-functional machine1stores the book document file generated by associating the book information with the front and rear image data, in the HDD25. In addition, the digital multi-functional machine1also has a function of transferring the book document file to the portable terminal2via the external I/F17in response to the designation of the user. Further, the digital multi-functional machine1may store the generated book document file in a memory device connected to the external I/F17. In addition, the digital multi-functional machine1may transfer the generated book document file to a device, such as a server on a network that can communicate via the external I/F17.

Next, the information management table25aprovided in the HDD25will be described.

FIG. 9is a view illustrating an example of the information management table25a.

The information management table25ais a data structure for storing various types of information obtained in a series of reading using the scanner11and the camera2aof the portable terminal2. In the example illustrated inFIG. 9, the information management table25astores front surface data, rear surface data, and OCR data.

The front surface data is image data captured by the camera2aof the portable terminal2and information related to the image data. The front surface data includes information, such as image data, a document detection position, a detection size, and the like. The front surface data is generated by the number of pages captured by the camera2aof the portable terminal2. The image data in the front surface data is image data obtained from the image captured by the camera2a. The image data of the front surface data may be an image captured by the camera2a, or image data obtained by performing image processing with respect to the captured image.

The document detection position of the front surface data is information indicating the position of the image detected as a document in the captured image of the camera2a. For example, the document detection position is information indicating the position of the document to be cut out by the image cutting-out with respect to the captured image of the camera2a. In addition, the detection size is information indicating the size of a page image detected as a document in the captured image of the camera2a. For example, the detection size is information obtained by the size detection with respect to the captured image.

The rear surface data is image data to be read by the scanner11and information related to the image data. The rear surface data includes information, such as image data, a document detection position, a detection size, and the like of the rear surface. The rear surface data is generated by the number of images scanned by the scanner11. However, when the rear surface is scanned one time in the book document or the like, the rear surface data is generated for the image of the cover scanned by the scanner11.

The image data of the rear surface is obtained by scanning the cover with the scanner11. In addition, the obtained image data may be processed by the image processing unit16. The document detection position is decided based on the obtained image data of the cover. To obtain the document detection position data of the rear surface, in a case of a rectangular document, a vertex out of the four vertices in the image data, which is closest to a corner of the document placing table11a, is used as a reference vertex. For example, the document detection position is information indicating the position of the document to be cut out by the image cutting-out with respect to the scanned image. In addition, the detection size is information indicating the size of the cover detected as a document in the scanned image. For example, the detection size is information obtained by the size detection with respect to the scanned image.

In addition, the OCR data is information obtained by the OCR processing with respect to the image data of the rear surface obtained by a series of reading. In the embodiment, the OCR data is information obtained by the OCR processing with respect to the image data read by the first scanning performed by the scanner11. The OCR data is text information or the like included in the image read by the scanner11. For example, the OCR data inFIG. 9is stored as a book name, an owner name, an administrator and management number as a result of recognizing book information by the OCR processing. The OCR data may be input or corrected by the operation of the user.

The digital multi-functional machine1generates an image file by combining any of the front surface data, the rear surface data, and the OCR data which are stored in the information management table25a. The combination of the front surface data, the rear surface data, and the OCR data may be set to generate an image file according to the type of the document. For example, with respect to the book document as illustrated inFIG. 8, the rear surface scanning is performed once and the front surface scanning is performed a plurality of times. Regarding such a document, it is considered to generate image files with combinations of the following (a), (b), or (c).

(a) image data of cover+image data of plural pages

(b) image data of plural pages+OCR data

(c) image data of cover+image data of plural pages+OCR data

In addition, regarding the document to which double-sided scanning is performed a plurality of times, it is considered to generate the image files (d) to (g) illustrated hereinafter.

(d) image data of plural rear surfaces+image data of plural front surfaces

(e) image data of plural rear surfaces+image data of plural front surfaces+deletion of image data of blank page

(f) image data of plural rear surfaces+image data of plural front surfaces+OCR data

(g) image data of plural rear surfaces+image data of plural front surfaces+OCR data+deletion of image data of blank page

In addition, in (d) and (g), an image file in which the image data of the rear surface and the image data of the front surface are alternately arranged may be generated. It is also considered to set a part of the OCR data as the file name in the image file generated by the above-described combination. In addition, it is also considered to set a part of the OCR data as the file name in the image file generated by the above-described combination.

Here, as an example, the information management table25acreated in the reading of the book document from which image data illustrated inFIG. 8is obtained will be described.

In the reading of the book document, the information management table25astores the image data of the cover as the rear surface data and the image data of each of the pages as the front surface data. In addition, in the reading of the book document, the information management table25astores information obtained by the OCR processing with respect to the image data of the front cover, as the OCR data.

In other words, when the document is a book document, information related to the front cover image read by the scanner11is stored as the rear surface data. In the reading of the book document, the digital multi-functional machine1obtains the image data of one cover by one time of scanning performed by the scanner11. Therefore, the information management table25astores the image data of one cover, the position of the cover and the cover size, as the rear surface data.

In other words, when the document is a book document, information related to each of the page images captured by the camera2aof the portable terminal2is stored as the front surface data. In the reading of the book document, the digital multi-functional machine1obtains the image data of a plurality of pages in response to the operation of the user with the camera2aof the portable terminal2. The information management table25astores the information of each of the pages by the number of pages captured by the camera2aof the portable terminal2, as the front surface data. The information related on each of the page images includes information, such as the image data, the document detection position, and detection size of the page illustrated inFIG. 9.

In addition, when the document is a book document, book information read from the cover of the book is stored as an OCR data. The book information may be obtained by the OCR processing with respect to the image data of the cover obtained by the scanner11. In the example illustrated inFIG. 9, as an example of the book information, a book name, a library name, and the management number are illustrated.

Next, the reading of the book document using the portable terminal2by the digital multi-functional machine1will be described.

FIGS. 10 to 12are flowcharts for explaining an operation example of the reading of the book document using the portable terminal2by the digital multi-functional machine1.

First, the user sets the portable terminal2which activates the application for reading the book document in the holding unit14, and designates the reading of the book document on the operation panel13. In addition, the user designates setting information, such as resolution, a document mode, a file format, file transferor storage destination, and the like, for generating the book image file, as necessary, by the operation panel13. The processor21of the digital multi-functional machine1performs various settings in response to the designation of the user.

When the reading of the book document is designated on the operation panel13, the processor21of the digital multi-functional machine1detects whether or not the portable terminal2is set in the holding unit14(ACT11). When detecting that the portable terminal2is set (ACT11, YES), the processor21sets the operation mode to the reading mode of the book document (ACT12). When the reading mode of the book document is set, the processor21is connected to the portable terminal2to communicate therewith (ACT13).

When the communication connection with the portable terminal2is established, the processor21notifies that the book document is set on the document table11a(ACT14). For example, the processor21controls the display unit13aof the operation panel13to display a message that the reading of the book document is possible and setting guide of the book document. When notifying the setting guide of the book document, the processor21waits for an input of a scan instruction by the operation unit13bof the operation panel13(ACT15). For example, as illustrated inFIG. 1, the user places the book document on the document table11awith the page surface to be scanned of the book document facing upward, and instructs the start of scanning on the operation panel13.

When the start of scanning is instructed (ACT15, YES), the processor21executes image reading by the scanner11(ACT16). By scanning a carriage, the scanner11reads the cover of the book document placed on the document table11afrom below the document table11a. The scanned image including the image of the cover read by the scanner11is stored in the page memory24after being processed by the input processing unit16aof the image processing unit16. The processor21registers the scanned image stored in the page memory24as the cover image data in the information management table25a(ACT17). For example, in the example illustrated inFIG. 9, the scanned image read by the scanner11is registered as cover image data B1in the information of the cover image of the information management table25a.

When the scanned image read by the scanner11is stored, the processor21captures the image with the camera2aby sending an instruction of capturing to the portable terminal (ACT18). For example, the processor21instructs the portable terminal2to capture the image with the camera2aby a function of remotely controlling the portable terminal2that can communicate via the external I/F17. The portable terminal2supplies an image captured by the camera2ain response to the instruction from the digital multi-functional machine1, to the digital multi-functional machine1. When a book document is placed on the document table11ain a state where the page is open upward, the camera2acaptures an image including the image of the page of the book document. The portable terminal2supplies the captured image including the image of the page captured by the camera2a, to the digital multi-functional machine1.

In the digital multi-functional machine1, the image captured by the camera2aby the external I/F17is obtained from the portable terminal2. The captured image of the camera2aobtained by the external I/F17is stored in the page memory24after being processed by the input processing unit16aof the image processing unit16. The processor21registers the captured image stored in the page memory24as the front surface image data of the second page in the information management table25a(ACT19). For example, in the example of the information management table25aillustrated inFIG. 9, the first captured image captured by the camera2ais registered as front surface image data F1in the information of the front surface image of the first page.

In addition, when obtaining the image captured by the camera2aof the portable terminal2, the processor21receives an instruction of scanning an additional page or an instruction of finishing the scanning on the operation panel13(ACTS20and21). For example, the processor21controls the display unit13aon the operation panel13to display an instruction button for scanning an additional page that can be selected by the user and an instruction button for finishing the scanning. Furthermore, when reading another page in the book document, the user turns over the page to be read in the book document placed on the document table11aand instructs scanning of the additional page.

When the scanning of the additional page is instructed (ACT20, YES), the processor21remotely controls the portable terminal2, and accordingly, captures the next image with the camera2aof the portable terminal2(ACT18). The portable terminal2captures the image with the camera2ain response to the instruction from the digital multi-functional machine1. The portable terminal2captures the capturing region including the document table11a, where the book document with the next page open is placed, with the camera2a. Then, the portable terminal2supplies the image captured by the camera2ato the digital multi-functional machine1. The processor21stores the image captured by the camera2aobtained by the external I/F17in the page memory24. The processor21registers the captured image stored in the page memory24as the front surface image information of the second page in the information management table25a(ACT19). For example, in the example of the information management table25aillustrated inFIG. 9, the second captured image captured by the camera2ais registered as front surface image data F2in the information of the front surface image of the second page.

The processor21repeatedly executes the processing of ACTS18and19upon the instruction of the scanning of the additional pages. Accordingly, the processor21can obtain the plurality of captured images of the plurality of pages in the book document with the camera2aof the portable terminal2.

When the finish of the reading of the book document is instructed (ACT21, YES), the processor21performs processing for associating the cover image of the book document with the images of each of the pages to convert images into a file. In the embodiment, the processor21executes each of the image processing with respect to the image data of the rear surface and the image processing with respect to image data of the front surface. The processor21registers the information obtained by the image processing in the information management table25a.

In other words, in order to execute the image processing with respect to the image data of the cover, the processor21reads a scanned image including a cover image (ACT31). When the scanned image is read, the processor21executes the image cutting-out to cut out the image of the cover part with respect to the read scanned image (ACT32). Here, the scanner11scans the reading region of the document table11aon which the book document is placed in a state where the document table cover11bopen. Therefore, the scanned image to be read by the scanner11has the maximum density in the region where the book document does not exist, and the color of the non-existing region is black. In the image cutting-out, the processor21detects an edge of the document of the cover region in order to specify the region where the document does not exist in the scanned image. When detecting the edge of the document region, the processor21cuts out the region of the image data surrounded by the detected edge as the image data of the cover of the book document.

When the image cutting-out is executed, the processor21executes the skew correction with respect to the image data of the cut-out cover (ACT33). The processor21analyzes the inclined state of the image data of the cover to which the image cutting-out is performed. When there is an inclination to be corrected, the processor21corrects the inclination of the image data of the cover.

When the skew correction is executed, the processor21executes size detection for detecting the size of the cover based on the image data of the cover after the skew correction (ACT34). For example, the processor21specifies the size of the cover by temporarily storing the image data of the cover in the page memory24after the skew correction.

When the size of the cover is specified, the processor21registers the cover image information in the information management table25a(ACT35). For example, the processor21stores the image data of the cover to which the skew correction is performed in the cover image information of the information management table25aas the image data B1. In addition, the processor21stores the document detection position and detection size which are obtained by the size detection in the cover image information of the information management table25a.

In addition, after specifying the size of the cover, the processor21performs the OCR processing for recognizing the book information from the image data of the cover to which the skew correction is performed (ACT36). For example, the processor21specifies the target region of the OCR processing in the image data of the cover after the skew correction based on the size of the cover. The processor21recognizes the letters by the OCR processing with respect to the target region in the image data of the cover. The processor21stores the information obtained by the OCR processing in the information management table25aas the book information of the book document (ACT37).

For example, in the image of the cover image, as illustrated inFIG. 5, a region R1and a region R2may be set as the target region of the OCR processing. The region R1illustrated inFIG. 5is assumed as a region of the back cover. The region R1is set as a region near the center of the cover size in a sub-scanning direction (lateral direction inFIG. 5) of the cover image. In addition, the region R2is assumed to be a region with a label pasted at a predetermined position on the cover of the book document. The region R2is set as the region of the label near the lower right part in the image of the cover.

In the example illustrated inFIG. 5, the processor21recognizes the letters “Scan Operation Manual” as the book name of the book document from the region R1. In addition, the processor21recognizes the letters “ABC Library” as the library name that owns the book document from the region R2and the letters “123456” as the management number of the book document. The processor21stores the information obtained by the OCR processing with respect to the image of the cover in the information management table25aas the book information. For example, as illustrated inFIG. 9, the result of the OCR processing with respect to the region R1and the region R2which are illustrated inFIG. 5is stored in the information management table25a.

In addition, the processor21also executes the image processing with respect to the captured image including images of each of the pages captured by the camera2a. When the image data of the plurality of front surfaces are stored in the page memory24, the processor21sequentially reads the image data of the front surface (ACT38). For example, the processor21reads the captured image of the first page. The processor21executes the image cutting-out for cutting out the document region from the read captured image (ACT39). When cutting out the document region, the processor21executes the skew correction with respect to the region of the cut-out page image (ACT40). When the skew correction is executed, the processor21executes the size detection for detecting the size of the document based on the image data of the cover after the skew correction (ACT41).

When the size detection is finished, the processor21stores the page image information of the page in the information management table25a(ACT42). In the example illustrated inFIG. 9, the processor21stores the page image data, the document detection position and the detection size, as the page image information in the information management table25a. For example, the processor21stores the image data of the cover of the first page to which the skew correction is performed in the page image information of the first page of the information management table25aas the image data F1. In addition, the processor21also stores the information, such as the document detection position and the detection size, which is obtained by the size detection in the information management table25a, as the image information of the first page.

Furthermore, when there is a next page (ACT43, NO), the processor21reads the captured image of the next page and executes the processing of the above-described ACTS38to42. For example, the processor21stores the image data of the second page to which the skew correction is performed in the page image information of the second page of the information management table25aas the image data F2. In addition, the processor21also stores the information, such as the document detection position and the detection size, which is obtained by the size detection in the page image information of the second page of the information management table25a.

When the image information for all the pages is stored in the information management table25a, the processor21performs the image quality improvement of each of the images. In other words, the processor21sequentially reads the image data of cover and the image data of each of the pages in the information management table25a(ACT51). The processor21performs the image quality improvement with respect to each image data to be sequentially read by the image quality improvement unit16bof the image processing unit16(ACT52). The image to which the image quality improvement is performed is temporarily stored in the page memory24.

When the image quality improvement with respect to all of the front and rear image data is finished (ACT53, YES), the processor21causes to display the image data after the image processing on the display unit13a(ACT54). For example, the processor21controls the display unit13ato display the image data of the cover and the image data of each of the pages as thumbnail images. The processor21controls the display unit13ato display each of the images and receives correction with respect to the displayed image (ACT55). When the correction with respect to the image is instructed (ACT55, YES), the processor21performs the correction in response to the correction instruction of the user (ACT56).

For example, when the correction with respect to the image of the page is instructed, the processor21may execute the capturing of the page again. In addition, when the correction of the document region in the captured image is instructed, the processor21executes processing, such as image cutting-out, with respect to the captured image again.

In addition, the processor21may receive an instruction of deleting unnecessary images as a correction instruction. When the deletion of the image is instructed, the processor21deletes the image designated by the user and does not store the image in the book image file. For example, when the image of the cover is unnecessary, the processor21does not store the image of the cover in the book image file in response to the instruction of the user.

In addition, the processor21controls the display unit13ato display the book information recognized by the OCR processing (ACT57). The processor21controls the display unit13ato display the book information that serves as the recognition result and receives the instruction of the correction with respect to the displayed book information (ACT58). When the correction with respect to the book information is instructed (ACT58, YES), the processor21performs the correction in response to the correction instruction of the user (ACT59). For example, the processor21receives the correction of the erroneously recognized letters in the book information that serves as the recognition result by the OCR processing. In addition, when the user instructs the target region of the OCR in the image data of the cover, the processor21may execute the OCR processing with respect to the designated region again.

After all of the images and the book information which are displayed on the display unit13aare confirmed, the processor21generates a book image file based on the information stored in the information management table25a(ACT60). For example, the processor21merges the image data of cover and the image data of the plurality of pages and generates a book image file in a predetermined format to which the book information is given.

When generating the book image file, the processor21gives a file name based on the book information to the generated book image file and stores the file name in the HDD25(ACT61). For example, the processor21may be a file name including the book name or a file name including the library name or the management number.

In addition, the processor21determines whether to store the book image file generated according to the setting of the user, or transfer the book image file to the external device (ACT62). When it is determined that transfer or storage of the book image file is instructed (ACT62, YES), the processor21supplies the book image file to the designated transfer destination or storage destination (ACT63). For example, the user may instruct to transfer the book image file generated by the digital multi-functional machine1to the portable terminal2. When the transfer of the book image file to the portable terminal2is instructed, the processor21transfers the generated book image file to the portable terminal2via the external I/F17. Further, the user may instruct the external memory device connected to the external I/F17to store the book image file. In this case, the processor21stores the book document file in the external memory device connected to the external I/F17.

In the above-described embodiment, it is assumed that the target to be read is the book document. However, the target may be anything that can be placed on the document table11a.

For example, when the target is a three-dimensional object having a flat plane, the target can be placed such that the plane is in contact with the document table11a. In this case, the scanner11reads an image of the plane in the three-dimensional object which is in contact with the document table11a. In addition, the camera2aof the portable terminal2set in the holding unit14can obtain an image of the three-dimensional object placed on the document table11ain a state where the plane is at a lower part, from above. Accordingly, the digital multi-functional machine1can generate an image file in which an image obtained by scanning a three-dimensional object from below and an image obtained by capturing the three-dimensional object from above are associated with each other. Furthermore, when the letter information, such as identification information, is described on the plane of the three-dimensional object, the digital multi-functional machine1can also generate an image file to which information described on the plane is attached.

According to the embodiment as described above, the digital multi-functional machine reads the target, such as the book document, from below and above with the scanner and the camera of the portable terminal attached onto the document table. The digital multi-functional machine generates the image file in which the image obtained by reading the target from below with the scanner and the image obtained by capturing the target from above with the camera are associated with each other. Accordingly, even when the user does not perform work of replacing the target or manually performing work of merging the plurality of pieces of image data, it is possible to easily generate the image file in which the image of the lower side of the target and the upper side of the target are associated with each other.

In addition, the portable terminal attached to the digital multi-functional machine can capture the plurality of pages with the camera in response to the operation of the user, in a state where the book document is placed on the document table with the page facing upward. When the camera of the portable terminal captures the images of the plurality of pages, the digital multi-functional machine generates the image file in which the image of the display of the book document read by the scanner and the image of the plurality of pages captured by the camera are associated with each other. In this case, since the book document is placed on the document table in a state where the page is open upward, the user easily turns over the page, and it is also easy to visually confirm the page.

In addition, the digital multi-functional machine recognizes the book information, such as the book name or the management number on the cover, by performing OCR recognition in the region set in the image data of the cover of the book document. The digital multi-functional machine stores the recognized book information in a file by associating the image data of rear surface with the image data of a page. In addition, the digital multi-functional machine can also give the information related to the book information as a file name to a book image file to be generated.

Modification Example

Next, modification examples of the above-described embodiment will be described.

The modification examples which will be described hereinafter can be applied to the digital multi-functional machine1illustrated inFIGS. 1 to 4. Therefore, the following description will be assumed to be realized in the above-described digital multi-functional machine1.

In the above-described embodiment, the digital multi-functional machine1is described assuming that the scanner11scans the cover of the book document one time with respect to one book document. On the other hand, in the modification example, in the digital multi-functional machine1, the scanner11also repeatedly scans the document in response to an additional scan instruction. In addition, the digital multi-functional machine1checks whether the scanned image scanned by the scanner11includes an image to be registered, and registers the image determined to be registered as a rear surface image.

FIGS. 13 to 15are flowcharts for explaining an operation example of the digital multi-functional machine1according to the modification example.

In the modification example, the user sets the portable terminal2which holds the application for capturing the image in cooperation with the digital multi-functional machine1in the holding unit14. The user designates the setting information as required by the operation panel13and designates the reading mode using the portable terminal2. When the reading mode using the portable terminal2is designated, the processor21of the digital multi-functional machine1detects whether or not the portable terminal2is set in the holding unit14(ACT111). When detecting that the portable terminal2is set (ACT111, YES), the processor21sets the operation mode to the reading mode using the portable terminal2(ACT112). When the reading mode is set, the processor21is connected to the portable terminal2to communicate therewith (ACT113).

When the communication connection with the portable terminal2is established, the processor21notifies a message that the document (target) to be read is set on the document table11a(ACT114) and waits for the input of the scan instruction (ACT115). For example, the user presses a start button provided on the operation panel13as the scan instruction.

When the start of scanning is instructed (ACT115, YES), the processor21executes image reading by the scanner11(ACT116). By scanning the carriage, the scanner11reads the scanned image including the rear surface image of the document placed on the document table11afrom below the document table11a. The processor21stores the scanned image processed by the input processing unit16aof the image processing unit16in the page memory24, and registers the scanned image as a first cover (rear surface) image data in the information management table25a(ACT117).

When the scanning by the scanner11is finished, the processor21captures the image with the camera2aby sending the instruction of capturing to the portable terminal2(ACT118). The portable terminal2supplies the image obtained by capturing an image by the camera2ain response to the instruction from the digital multi-functional machine1, to the digital multi-functional machine1.

In addition, the image capturing by the camera2aof the portable terminal2may be performed prior to the scanning by the scanner11. However, the scanning by the scanner11and the capturing by the camera2aof the portable terminal2are performed sequentially (alternately) in order to prevent light interference during the scanning by the scanner11.

The digital multi-functional machine1obtains the captured image captured by the camera2aby the external I/F17from the portable terminal2. The processor21stores the obtained captured image by processing the image by the input processing unit16ain the page memory24, and registers the captured image as a second front surface image data in the information management table25a(ACT119).

In addition, the processor21receives an instruction of additional scanning or an instruction of finishing the scanning on the operation panel13(ACTS120and121). When the additional scanning is instructed (ACT120, YES), the processor21returns to ACT116and performs processing for obtaining the scanned image by the scanner11and the captured image of the camera2a(ACTS116to119).

When the finish of reading is instructed (ACT121, YES), the processor21sequentially reads the obtained scanned image (ACT130). When reading the scanned image, the processor21determines whether or not the scanned image includes an image to be registered as a rear surface image (ACT131). For example, when the processor21determines that the document image included in the scanned image is a white image, the processor21determines that the scanned image is not registered. In addition, when it is determined that the image of the document included in the scanned image is the same image as the registered image of the rear surface, the processor21determines that the scanned image does not include the image to be registered. By these determinations, the processor21can avoid registering of blank images or overlapping images as images of the rear surface.

When it is determined that the scanned image includes an image to be registered (ACT131, YES), the processor21cuts out an image of the document region from the scanned image (ACT132). In addition, the processor21performs the skew correction with respect to the image data cut out from the scanned image (ACT133), and detects the size of the document based on the image data after the skew correction (ACT134). Further, the processor21registers image data of the rear surface to which the processing of ACTS132to134is performed as rear surface image information in the information management table25a(ACT135).

When the rear surface image information is registered or when it is determined that the scanned image does not include an image to be registered (ACT131, NO), the processor21determines that registration of the rear surface image is finished (ACT136). In other words, the processor21performs the processing of ACTS131to135with respect to all of the scanned images read by the scanner11. When the processing of ACTS131to135with respect to all of the scanned images is finished, the processor21determines that registration of rear surface image information is finished (ACT136, YES).

When the registration of the rear surface image information is finished (ACT136, YES), the processor21executes the image processing with respect to each of the captured images captured by the camera2a. Here, all of the captured images captured by the camera2aare image-processed as the front surface image to be registered. When the plurality of captured images captured by the camera2aare stored, the processor21sequentially reads the captured image (ACT138). The processor21performs image cutting-out (ACT139), skew correction (ACT140), and size detection (ACT141) with respect to the read captured image. Further, the processor21registers information related to the image data to which the processing of ACTS139to141are performed as the rear surface image information in the information management table25a(ACT142).

When the registration of the rear surface image information is finished (ACT143, YES), the processor21sequentially reads each of the images (ACT151) and performs image quality improvement with respect to the read image data by the image quality improvement unit16b(ACT152).

When the image quality improvement with respect to all of the image data is finished (ACT153, YES), the processor21controls the display unit13ato display each of the image data after the image processing (ACT154). The processor21controls the display unit13ato display the image data of the rear surface and the image data of the front surface, and receives the correction to the displayed image (ACT155). When the correction with respect to the image is instructed (ACT155, YES), the processor21performs the correction in response to the correction instruction of the user (ACT156).

When the display of each of the images is finished, the processor21generates a book image file based on the information stored in the information management table25a(ACT160). For example, the processor21merges all of the image data of rear surface and the image data of the front surface and generates the book image file in a predetermined format. When generating the book image file, the processor21gives a predetermined file name to the generated book image file and stores the file name in the HDD25(ACT161). Here, the processor21may give a file name including the information obtained by the OCR processing with respect to any rear surface image to the book image file. In addition, when transferring or storing the book image file to the external device (ACT162, YES), the processor21supplies the book image file to the designated transfer destination or storage destination (ACT163).

The digital multi-functional machine of the modification example described above executes not only capturing by the camera of the portable terminal but also scanning of the image by the scanner every time an additional scan instruction is received. The digital multi-functional machine determines whether or not each of the scanned images read by the scanner is an image to be registered, and registers the image determined to be registered.

Accordingly, in the digital multi-functional machine, when a document of which a rear surface is blank is placed on the document table, it is possible to avoid storing of a useless rear surface image without storing a blank rear surface image. In addition, even when the scanning is an operation of reading an image for each scan instruction, the digital multi-functional machine does not store the image when the same image is scanned, and it is possible to avoid storing the same overlapping images. In other words, in the embodiment, only data suitable for specifying the contents of the front surface data as the rear surface data and the like are selectively stored, and further, during the file conversion, the stored data or only necessary text information, such as book information or the like from the stored data is extracted, and the information is converted into files together with the front surface data.