Bending detection device and image processing apparatus

A bending detection device includes a processor configured to receive a read image obtained by reading a document, and notify a user of a bend in the document based on a determination that the read image has been obtained by reading the document in a bent state.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2020-076256 filed Apr. 22, 2020.

BACKGROUND

(i) Technical Field

The present disclosure relates to a bending detection device and an image processing apparatus.

(ii) Related Art

An image processing apparatus that estimates that the size of a document is not correctly detected in the case where a detection result of the size of the document fluctuates a predetermined number of times within a predetermined period of time due to curling or bending of the document when the size of the document is automatically detected and displays a document size setting screen for setting the size of a document in the case where it is estimated that the size of the document is not correctly detected has been known (see, for example, Japanese Unexamined Patent Application Publication No. 2012-222764).

SUMMARY

Documents in a bent state may be read. In such a case, with a configuration in which only a notification indicating that the size of a document is not correctly detected is provided, a user is not able to recognize that the document is bent.

Aspects of non-limiting embodiments of the present disclosure relate to allowing a user to recognize that a document is bent.

According to an aspect of the present disclosure, there is provided a bending detection device including a processor configured to receive a read image obtained by reading a document, and notify a user of a bend in the document based on a determination that the read image has been obtained by reading the document in a bent state.

DETAILED DESCRIPTION

Exemplary embodiments of the present disclosure will be described in detail with reference to attached drawings.

Hardware Configuration of Image Processing Apparatus

FIG. 1is a diagram illustrating an example of the hardware configuration of an image processing apparatus10according to an exemplary embodiment. As illustrated inFIG. 1, the image processing apparatus10includes a central processing unit (CPU)11, a random access memory (RAM)12, a read only memory (ROM)13, a hard disk drive (HDD)14, an operation panel15, an image reading unit16, an image forming unit17, and a communication interface (hereinafter, referred to as a “communication I/F”)18.

The CPU11implements functions described below by loading various programs stored in the ROM13and the like into the RAM12and executing the programs.

The RAM12is a memory that is used as an operation memory or the like for the CPU11. The ROM13is a memory that stores various programs and the like to be executed by the CPU11. The HDD14is, for example, a magnetic disk device that stores image data read by the image reading unit16, image data to be used for image formation by the image forming unit17, and the like.

The operation panel15is, for example, a touch panel that displays various types of information and receives an operation input by a user. The operation panel15includes a display on which various types of information are displayed and a position detection sheet that detects a position indicated by a finger or indicating means such as a stylus pen. The operation panel15may include a display and a keyboard, in place of a touch panel.

The image reading unit16is an example of reading means and reads an image recorded on a recording medium such as paper. The image reading unit16is, for example, a scanner. The image reading unit16may be of a charge coupled devices (CCD) type in which reflection light with respect to light applied to a document from a light source is reduced using a lens and then received by CCD or a contact image sensor (CIS) type in which refection light with respect to light sequentially applied to a document from an LED light source is received by a CIS.

The image forming unit17forms an image on a recording medium such as paper. The image forming unit17is, for example, a printer. The image forming unit17may be of an electrophotographic system that forms an image by transferring toner attached on a photoreceptor to a recording medium or an ink jet system that forms an image by discharging ink onto a recording medium.

The communication I/F18transmits and receives various types of information to and from other apparatuses via a communication line.

Outline of Exemplary Embodiments

In an exemplary embodiment, the image processing apparatus10receives a read image obtained by reading a document. In the case where the read image has been obtained by reading the document in a bent state, the image processing apparatus10notifies a user that the document is bent.

In the case where any one of three conditions described below is satisfied, the image processing apparatus10determines that the read image has been obtained by reading the document in a bent state.

A first condition is that a read image includes a straight line indicating that a document is bent. The straight line indicating that a document is bent may be a straight line that forms an angle equal to or more than a threshold with respect to a horizontal direction and forms an angle equal to or more than a threshold with respect to a vertical direction.

A second condition includes a condition that a read image includes a string of first image elements and a string of second image elements, the string of the first image elements are arranged in a first direction, the string of the second image elements are arranged in a second direction different from the first direction, and the second image elements are of the same type as the first image elements. The second condition may further include a condition that the string of the first image elements ends at a first position and the string of the second image elements starts at a second position that is different from the first position and is within a predetermined distance from the first position.

A third condition is that a read image includes a first string of first image elements and a second string of second image elements, the first string of the first image elements ends at a first position, the second string of the second image elements starts at a second position different from the first position, and the second image elements are of the same type as the first image elements. The first position and the second position may be within a predetermined distance.

Functional Configuration of Bending Detection Device

FIG. 2is a block diagram illustrating an example of the functional configuration of a bending detection device20according to an exemplary embodiment. The bending detection device20may be regarded as a device that is implemented when the CPU11(seeFIG. 1) of the image processing apparatus10reads a program implementing functional units, which will be described below, from, for example, the ROM13(seeFIG. 1) into the RAM12(seeFIG. 1) and executes the program.

As illustrated inFIG. 2, the bending detection device20includes a scanned image acquisition unit21, a bending detection unit22, a notification message generation unit23, a thumbnail image generation unit24, a display control unit25, and an operation receiving unit26.

The scanned image acquisition unit21acquires a read image (scanned image) read by the image reading unit16(seeFIG. 1). In this exemplary embodiment, the scanned image acquisition unit21is provided as an example of means for receiving a read image obtained by reading a document.

The bending detection unit22analyzes the scanned image acquired by the scanned image acquisition unit21to detect whether or not the document is bent. In this exemplary embodiment, the bending detection unit22is provided as an example of means for detecting whether or not a read image has been obtained by reading a document in a bent state.

The notification message generation unit23generates a notification message to be notified to a user. For example, “Bending of the document has been detected. Please check the document.” may be generated as the notification message.

The thumbnail image generation unit24generates a thumbnail image indicating a position in a document at which bending has been detected. The thumbnail image may indicate a straight line indicating bending provided on a blank image. Alternatively, the thumbnail image may indicate a straight line indicating bending provided on a scanned image. In this case, the thumbnail image may indicate a straight line indicating bending provided on an image obtained by removing an image of a straight line indicating the bending from a scanned image. In this exemplary embodiment, a thumbnail image is provided as an example of a notification image including a straight line generated at a position where a document is bent and an example of a notification image further including at least part of a read image.

The display control unit25controls a notification message generated by the notification message generation unit23and a thumbnail image generated by the thumbnail image generation unit24to be displayed on the operation panel15. In this exemplary embodiment, the display control unit25is provided as an example of means for notifying a user that a document is bent and an example of means for notifying, using a notification image, the user that the document is bent, in the case where a read image has been obtained by reading a document in a bent state.

The operation receiving unit26receives an operation performed on the operation panel15by a user.

Operation of Bending Detection Device

FIG. 3is a flowchart illustrating an example of an operation of the bending detection device20according to an exemplary embodiment.

As illustrated inFIG. 3, in the bending detection device20, the scanned image acquisition unit21acquires a scanned image read by the image reading unit16(seeFIG. 1) (step201).

Next, the bending detection unit22analyzes the scanned image acquired in step201to perform a bending detection process for detecting whether or not a document is bent (step202). In the bending detection process in step202, the bending detection unit22determines whether or not bending of the document has been detected (step203).

In the case where it is determined in step203that bending of the document has been detected, the notification message generation unit23generates a notification message to be notified to a user (step204). For example, “Bending of the document has been detected. Please check the document.” may be generated as the notification message. In this case, the thumbnail image generation unit24generates a thumbnail image indicating a position in the document at which bending is generated (step205). As described above, the notification message is generated in step204, and the thumbnail image is generated in step205. Then, the display control unit25controls the notification message and the thumbnail image to be displayed on the operation panel15(seeFIG. 1) (step206).

Accordingly, the notification message and the thumbnail image are displayed on the operation panel15. The user refers to the notification message and the thumbnail image to determine whether or not the document needs to be scanned again. In the case where it is determined that the document needs to be scanned again, the user performs an operation on the operation panel15for issuing an instruction for rescanning. In the bending detection device20, the operation receiving unit26determines whether or not an operation for issuing an instruction for rescanning has been received (step207). In the case where it is determined in step207that an operation for issuing an instruction for rescanning has been received, the display control unit25controls a scan screen for issuing an instruction for scanning to be displayed on the operation panel15(step208).

In the case where it is not determined in step203that bending of the document has been detected, the bending detection device20ends the process.

Furthermore, when referring to the notification message and the thumbnail image displayed on the operation panel15in step206, the user may determine that the document does not need to be scanned again. In this case, it is not determined in step207that an operation for issuing an instruction for rescanning has been received, and the bending detection device20ends the process.

Details of Bending Detection Process

FIG. 4is a diagram illustrating an example of a scanned image acquired by the scanned image acquisition unit21. Hereinafter, a bending detection process by the bending detection unit22will be described on the assumption that a scanned image300is acquired by the scanned image acquisition unit21.

FIG. 5is a diagram for explaining an outline of a first bending detection process. The first bending detection process is a process for determining that, in the case where the first condition mentioned above is satisfied, a scanned image has been obtained by reading a document in a bent state. In this case, the first bending detection process is a process for detecting that a document is bent in the case where a scanned image includes a straight line forming an angle equal to or more than a threshold with respect to each of the horizontal direction and the vertical direction. This is because bending of a document often forms a straight line. Moreover, although a line intentionally drawn on a document generally extends in the horizontal direction or the vertical direction, bending of a document often occurs in a direction different from the horizontal and vertical directions. The straight line may be obtained by, for example, Hough transform.

InFIG. 5, a straight line311forms an angle equal to or more than a threshold with respect to a line312in the horizontal direction and forms an angle equal to or more than a threshold with respect to a line313in the vertical direction. Thus, in the first bending detection process, it is determined that the scanned image300has been obtained by reading a document in a bent state.

FIG. 6is a flowchart illustrating a processing example of the first bending detection process.

As illustrated inFIG. 6, the bending detection unit22substitutes 0 for a variable M for counting the number of straight lines representing bending (step221). The bending detection unit22extracts a plurality of straight lines from the scanned image300(step222). The number of straight lines may be extracted by, for example, Hough transform.

Next, the bending detection unit22pays attention to one of the plurality of straight lines extracted in step222(step223).

Next, the bending detection unit22calculates an angle θ for the straight line to which attention is paid in step223(step224). The bending detection unit22determines whether or not the absolute value of the angle θ calculated in step224is equal to or more than a threshold T1 and less than or equal to a threshold T2 (step225). The threshold T1 represents the threshold for the angle formed between a straight line and a line in the horizontal direction. The threshold T2 represents the threshold for the angle formed between a straight line and the vertical direction.

In the case where it is determined in step225that the absolute value of the angle θ is equal to or more than the threshold T1 and less than or equal to the threshold T2, the bending detection unit22adds one to the variable M for counting the number of straight lines representing bending (step226). The bending detection unit22stores coordinates P1(M) and Q1(M) of end points of the straight line to which attention is paid in step223(step227), and the bending detection unit22causes the process to proceed to step228.

In contrast, in the case where it is not determined in step225that the absolute value of the angle θ is equal to or more than the threshold T1 and less than or equal to the threshold T2, that is, in the case where it is determined that the absolute value of the angle θ is less than the threshold T1 or more than the threshold T2, the bending detection unit22causes the process to proceed to step228.

The bending detection unit22determines whether or not there is any other straight line to which attention has not been paid (step228).

In the case where it is determined in step228that there is a straight line to which attention has not been paid, the bending detection unit22causes the process to return to step223.

In contrast, in the case where it is not determined in step228that there is a straight line to which attention has not been paid, the bending detection unit22determines whether or not the value of the variable M is more than 0 (step229).

In the case where it is determined in step229that the value of the variable M is more than 0, the bending detection unit22outputs a notification indicating that bending of the document has been detected to the notification message generation unit23, outputs the coordinates P1(M) and Q1(M) of the end points of the straight line stored in step227to the thumbnail image generation unit24(step230), and ends the process. The thumbnail image generation unit24renders M straight lines obtained by connecting the corresponding coordinates P1(M) and Q1(M) on a blank image or a scanned image.

In contrast, in the case where it is not determined in step229that the value of the variable M is more than 0, that is, in the case where there is no straight line representing bending, the process ends.

(Examples of Second and Third Bending Detection Processes)

FIG. 7is a diagram for explaining an outline of a second bending detection process. The second bending detection process is an example of a process for determining that, in the case where the second condition and the third condition mentioned above are satisfied, a scanned image has been obtained by reading a document in a bent state.

In the second bending detection process, types of the first image elements and the second image elements in the second condition and the third condition mentioned above are represented by symbols. The symbols include characters, numbers, and symbols in a narrow sense. Here, characters are used as examples of symbols. That is, in the second bending detection process, the first string of the first image elements and the second string of the second image elements are character strings. In this case, the second bending detection process is a process for detecting that a document is bent in the case where directions of two adjacent character strings are different by an angle exceeding a threshold or an ending position of one character string and a starting position of the other character string are different. A direction of a character string may be a direction in which a plurality of coordinates obtained by performing character recognition on a plurality of characters included in the character string are connected. Furthermore, a direction of a character string may be a direction in which coordinates of a plurality of characters included in a word obtained by performing character recognition on the character string and then performing morphological analysis of the character string are connected.

InFIG. 7, directions of adjacent character strings322aand321bare different by an angle exceeding the threshold and an ending position of the character string322aand a starting position of the character string321bare slightly different. Furthermore, directions of adjacent character strings323aand322bare different by an angle exceeding the threshold, and an ending position of the character string323aand a starting position of the character string322bare slightly different. Thus, in the second bending detection process, it is determined that the scanned image300has been obtained by reading the document in the bent state.

FIG. 8is a diagram for explaining an outline of the third bending detection process. The third bending detection process is an example of a process for determining that, in the case where the second condition and the third condition mentioned above are satisfied, a scanned image has been obtained by reading a document in the bent state.

In the third bending detection process, types of the first image elements and the second image elements in the second condition and the third condition mentioned above are represented by straight lines connecting lower limit pixels of images of symbols (hereinafter, referred to as a “lower limit line”). The symbols include characters, numbers, and symbols in a narrow sense. In this example, however, characters are used as symbols. That is, in the third bending detection process, the first string of the first image elements and the second string of the second image elements are represented by lower limit lines of images of character strings. In this case, the third bending detection process is a process for detecting that a document is bent in the case where directions of two adjacent lower limit lines are different by an angle exceeding a threshold or an ending position of one lower limit line and a starting position of the other lower limit line are different.

InFIG. 8, directions of adjacent lower limit lines332aand331bare different by an angle exceeding the threshold, and an ending position of the lower limit line332aand a starting position of the lower limit line331bare slightly different. Furthermore, directions of adjacent lower limit lines333aand332bare different by an angle exceeding the threshold, and an ending position of the lower limit line333aand a starting position of the lower limit line332bare slightly different. Thus, in the third bending detection process, it is determined that the scanned image300has been obtained by reading the document in the bent state.

Furthermore, in the third bending detection process, types of the first image elements and the second image elements in the second condition and the third condition mentioned above may be represented by straight lines such as ruled lines intentionally drawn on a document. In this case, the third bending detection process is a process for detecting that a document is bent in the case where directions of two adjacent ruled lines are different by an angle exceeding the threshold or an ending position of one ruled line and a starting position of the other ruled line are different. The ruled lines may be obtained by, for example, Hough transform.

InFIG. 8, directions of adjacent ruled lines334aand334bare different by an angle exceeding the threshold, and an ending position of the ruled line334aand a starting position of the ruled line334bare slightly different. Furthermore, directions of adjacent ruled lines335aand335bare different by an angle exceeding the threshold, and an ending position of the ruled line335aand a starting position of the ruled line335bare slightly different. Thus, in the third bending detection process, it is determined that the scanned image300has been obtained by reading the document in the bent state.

Furthermore, in the third bending detection process, types of the first image elements and the second image elements in the second condition and the third condition mentioned above may be represented by regions to which color information is added, such as blocks based on color values or brightness values. In this case, the third bending detection process is a process for detecting that a document is bent in the case where directions of two adjacent blocks based on color values or brightness values are different by an angle exceeding the threshold or an ending position of one block based on a color value or a brightness value and a starting position of the other block based on a color value or a brightness value are different.

InFIG. 8, directions of adjacent blocks336aand336bare different by an angle exceeding the threshold, and an ending position of the block336aand a starting position of the block336bare slightly different. Furthermore, directions of adjacent blocks337aand337bare different by an angle exceeding the threshold, and an ending position of the block337aand a starting position of the block337bare slightly different. Thus, in the third bending detection process, it is determined that the scanned image300has been obtained by reading the document in the bent state.

FIG. 9is a flowchart illustrating a processing example of the second and third bending detection processes.

As illustrated inFIG. 9, the bending detection unit22substitutes 0 for a variable N for counting the number of pairs of image elements for detecting whether or not bending occurs (step241). The bending detection unit22extracts a plurality of pairs of image elements from the scanned image300(step242). In the case where the image elements are ruled lines, the plurality of pairs of image elements may be extracted by, for example, Hough transform. Furthermore, in the case where the image elements are character strings, lower limit lines of images of character strings, or blocks based on color values or brightness values, the plurality of pairs of image elements may be extracted by a known image processing method.

Next, the bending detection unit22pays attention to one of the plurality of pairs of image elements extracted in step242(step243).

Next, the bending detection unit22calculates the distance δ between end points of the pair of image elements to which attention is paid in step243(step244). The bending detection unit22determines whether or not the distance δ calculated in step244is less than a threshold T3 (step245). The threshold T3 is a threshold for determining whether or not the distance between end points of image elements is within a predetermined distance.

In the case where it is determined in step245that the distance δ is less than the threshold T3, the bending detection unit22calculates angles α and β of directions of the pair of image elements to which attention is paid in step243(step246). In the case where the image elements are character strings, the directions of the image elements may be obtained by connecting coordinate strings obtained by performing recognition processing on the character strings. In the case where the image elements are lower limit lines or ruled lines, directions of the lower limit lines or the ruled lines may be defined as the directions of the image elements. Furthermore, in the case where the image elements are blocks based on color values or brightness values, the directions of the image elements may be obtained based on directions of border lines of the blocks. The bending detection unit22determines whether or not the absolute value of a difference between the angles α and β calculated in step246exceeds a threshold T4 (step247). The threshold T4 is a threshold for an angle formed between the direction of one of the pair of image elements and the direction of the other one of the pair of image elements.

In the case where it is determined in step247that the absolute value of the difference between the angles α and β exceeds the threshold T4, one is added to the variable N for counting the number of pairs of image elements for detecting whether or not bending occurs (step248). The bending detection unit22calculates coordinates E(N) of an ending position of one of the pair of image elements to which attention is paid in step243and coordinates S(N) of a starting position of the other one of the pair of image elements (step249), and causes the process to proceed to step250.

In contrast, in the case where it is not determined in step245that the distance δ is less than the threshold T3, that is, it is determined that the distance δ is equal to or more than the threshold T3, the bending detection unit22causes the process to proceed to step250. Furthermore, in the case where it is not determined in step247that the absolute value of the difference between the angles α and β exceeds the threshold T4, that is, in the case where it is determined that the absolute value of the difference between the angles α and β is less than or equal to the threshold T4, the bending detection unit22also causes the process to proceed to step250.

Then, the bending detection unit22determines whether or not there is any other pair of image elements to which attention has not been paid (step250).

In the case where it is determined in step250that there is another pair of image elements to which attention has not been paid, the bending detection unit22causes the process to return to step243.

In contrast, in the case where it is not determined in step250that there is another pair of image elements to which attention has not been paid, the bending detection unit22determines whether or not the value of the variable N is two or more (step251). In this example, the determination as to whether or not the value of the variable N is two or more is performed based on the assumption that a straight line representing bending may be rendered in the case where there are at least two pairs of image elements. However, the present disclosure is not limited to this. To render a straight line representing bending more accurately, a determination as to whether or not the value of the variable N is equal to or more than a predetermined value that is greater than two may be performed.

In the case where it is determined in step251that the value of the variable N is two or more, the bending detection unit22obtains a straight line passing through two or more pairs of coordinates E(N) and coordinates S(N) calculated in step249and stores coordinates P2 and Q2 of the end points (step252). Then, the bending detection unit22outputs a notification indicating that bending of the document has been detected to the notification message generation unit23, outputs the coordinates P2 and Q2 of the end points of the straight line stored in step252to the thumbnail image generation unit24(step253), and ends the process. The thumbnail image generation unit24renders the straight line obtained by connecting the coordinates P2 and Q2 on a blank image or a scanned image.

In contrast, in the case where it is not determined in step251that the value of the variable N is two or more, that is, in the case where there is no image element or only one image element for detecting whether or not bending occurs, the process ends.

In this processing example, in the case where the condition that the distance δ between end points of two image elements is less than the threshold T3 and the condition that the absolute value of the difference between the angles α and β of directions of two image elements exceeds the threshold T4 are satisfied, bending of the document is detected. However, the present disclosure is not limited to this. In the case where only the condition that the distance δ between the end points of the two image elements is less than the threshold T3 is satisfied, bending of the document may be detected. Alternatively, in the case where only the condition that the absolute value of the difference between the angles α and β of the directions of the two image elements exceeds the threshold T4 is satisfied, bending of the document may be detected.

For example, a process performed by the bending detection device20according to an exemplary embodiment is prepared as a program such as application software.

That is, a program implementing an exemplary embodiment is regarded as a program for causing a computer to implement a function for receiving a read image obtained by reading a document and a function for notifying a user that the document is bent in the case where the read image has been obtained by reading the document in a bent state.

A program for implementing an exemplary embodiment may not only be provided by communication means but also may be stored in a recording medium such as a compact disc-read only memory (CD-ROM) and provided.