Patent Publication Number: US-2018035007-A1

Title: Image forming apparatus, storage medium, and method for digitizing document

Description:
INCORPORATION BY REFERENCE 
     The present application claims priority under 35 U.S.C. § 119 to Japanese Patent Application No. 2016-149068, tiled on Jul. 28, 2016. The contents of this application are incorporated herein by reference in their entirety. 
     BACKGROUND 
     The present disclosure relates to an image forming apparatus for digitizing a document based on a markup document, which is an original document modified by handwriting. The present disclosure also relates to a storage medium and a method for digitizing a document. 
     An existing document editing device digitizes a document based on a markup document, which is an original document modified by handwriting. 
     SUMMARY 
     An image forming apparatus according to an aspect of the present disclosure includes a central processing unit (CPU), a storage device, and a reading device. The storage device stores therein a document digitization program. The reading device reads an image from an original document. The CPU executes the document digitization program to implement an image acquiring section, an added handwriting extracting section, and a document editing section. The image acquiring section acquires an image of a markup document using the reading device. The markup document is the original document modified by handwriting. The added handwriting extracting section extracts an added handwriting from the image of the markup document acquired by the image acquiring section. The document editing section edits a raw original document in accordance with a modification instruction given through the added handwriting extracted by the added handwriting extracting section to generate a digitized document. The raw original document is the original document without the modification. The raw original document includes one or more characters and one or more graphics. The document editing section alters a position of at least some of the characters and the graphics included in the raw original document to generate the digitized document. 
     A non-transitory computer-readable storage medium according to another aspect of the present disclosure stores thereon a document digitization program. The document digitization program causes an image forming apparatus to implement an image acquiring section, an added handwriting extracting section, and a document editing section. The image forming apparatus includes a reading device. The reading device reads an image from an original document. The image acquiring section acquires an image of a markup document using the reading device. The markup document is the original document modified by handwriting. The added handwriting extracting section extracts an added handwriting from the image of the markup document acquired by the image acquiring section. The document editing section edits a raw original document in accordance with a modification instruction given through the added handwriting extracted by the added handwriting extracting section to generate a digitized document. The raw original document is the original document without the modification. The raw original document includes one or more characters and one or more graphics. The document editing section alters a position of at least some of the characters and the graphics included in the raw original document to generate the digitized document. 
     A method for digitizing a document according to another aspect of the present disclosure is implemented by an image forming apparatus including a reading device. The reading device reads an image from an original document. The method for digitizing a document includes: acquiring an image of a markup document using the reading device, the markup document being the original document modified by handwriting; extracting an added handwriting from the image of the markup document acquired in the acquiring; and altering a position of at least some of one or more characters and one or more graphics included in a raw original document in accordance with a modification instruction given through the extracted added handwriting to generate a digitized document, the raw original document being the original document without the modification. 
    
    
     
       BRIEF DESCRIPTION OF THE GRAPHICS 
         FIG. 1  is a block diagram of a multifunction peripheral (MFP) according to an embodiment of the present disclosure. 
         FIG. 2  is a flowchart illustrating operation of the MFP illustrated in  FIG. 1  for digitizing a document based on a markup document. 
         FIG. 3  is a diagram illustrating an example of an image of the markup document illustrated in  FIG. 2 . 
         FIG. 4  is a diagram illustrating an image of added handwritings in the markup document illustrated in  FIG. 3 . 
         FIG. 5  is a diagram illustrating an image of a raw original document of the markup document illustrated in  FIG. 5 . 
         FIG. 6  is a diagram illustrating the image of the raw original document illustrated in  FIG. 5  divided into a plurality of areas. 
         FIG. 7  is a diagram illustrating original document layout information generated from the image illustrated in  FIG. 6 . 
         FIG. 8A  is a flowchart illustrating a first half of editing illustrated in  FIG. 2 . 
         FIG. 8B  is a flowchart illustrating a last half of the editing illustrated in  FIG. 2 . 
         FIG. 9  is a diagram illustrating part of the original document layout information illustrated in  FIG. 7  in a case where a character area is newly added. 
         FIG. 10A  is a diagram illustrating an example of an area in a case where the MFP illustrated in  FIG. 1  does not recognize a “heading”. 
         FIG. 10B  is a diagram illustrating an example of the area in a case where the MFP illustrated in  FIG. 1  recognizes the “heading”. 
         FIG. 11  is a diagram illustrating a document digitized based on the markup document illustrated in  FIG. 3 . 
         FIG. 12  is a diagram illustrating a layout of the document illustrated in  FIG. 11 . 
     
    
    
     DETAILED DESCRIPTION 
     The following describes an embodiment of the present disclosure with the use of the drawings. 
     First, a configuration of a multifunction peripheral (MFP)  10  serving as an image forming apparatus according to the present embodiment will be described. 
       FIG. 1  is a block diagram of the MFP  10 . 
     As illustrated in  FIG. 1 , the MFP  10  includes an operation section  11 , a display section  12 , a scanner  13 , a printer  14 , a fax communication section  15 , a communication section  16 , a storage section  17 , and a controller  18 . The operation section  11  is an operation device such as a set of buttons for inputting various operations. The display section  12  is a display device such as a liquid crystal display (LCD) that displays various pieces of information. The scanner  13  is a reading device that reads an image from an original document. The printer  14  is a printing device that executes printing on a recording medium such as paper. The fax communication section  15  is a fax device that performs fax communication with an external facsimile machine, not shown, via a communication line such as the public switched telephone network. The communication section  16  is a communication device that performs wired or wireless communication directly with an external device without routing the communication through a network such as a local area network (LAN) or the Internet. Alternatively, the communication section  16  is a communication device that performs communication with an external device via a network. The storage section  17  is a non-volatile storage device that stores therein various types of data, such as semiconductor memory or a hard disk drive (HDD). The controller  18  performs overall control of the MFP  10 . 
     The storage section  17  stores therein a document digitization program  17   a . The document digitization program  17   a  digitizes a document based on an original document modified by handwriting (hereinafter, referred to as “a markup document”) The document digitization program  17   a  may be installed on the MFP  10  during production of the MFP  10 , or may be additionally installed on the MFP  10  from a storage medium such as an SD card and a universal serial bus (USB) memory device, or may be additionally installed on the MFP  10  from a network. 
     The storage section  17  can store therein specific layout information  17   b  indicating a specific layout. The specific layout is for example a header layout, a footer layout, and/or a column layout for text. The storage section  17  may store the specific layout information  17   b  for each of users of the MFP  10  or for each of groups to which users of the MFP  10  belong. The MFP  10  can learn a possible original document in advance and thereby generate the specific layout information  17   b . For example, in a case where a frequency at which a user lays out original documents as two columns is greater than or equal to a specific frequency, the MFP  10  includes, in the specific layout information  17   b  of the user, a layout that shows the text in two columns. 
     The storage section  17  can store therein character attribute information  17   c . The character attribute information  17   c  refers to character attributes such as size, font type, font weight, and distance between characters. The character attribute information  17   c  may refer to character attributes depending on the location of the characters, such as header, footer, and text body. The storage section  17  may store the character attribute information  17   c  for each of users of the MFP  10  or for each of groups to which users of the MFP  10  belong. The MFP  10  can learn a possible original document in advance and thereby generate the character attribute information  17   c.    
     The controller  18  for example includes a central processing unit (CPU), read only memory (ROM), and random access memory (RAM). The ROM stores thereon a program and various types of data. The RAM is used as a work area of the CPU of the controller  18 . The CPU of the controller  18  executes the program stored in the ROM of the controller  18  or the storage section  17 . 
     The controller  18  implements an image acquiring section  18   a , an added handwriting extracting section  18   b , a raw original document reproduction section  18   c , an area extracting section  18   d , a layout plan determination section  18   e , and a document editing section  18   f  by executing the document digitization program  17   a  stored in the storage section  17 . The image acquiring section  18   a  acquires an image of the markup document, which is the original document modified by handwriting, using a scanner  13 . The added handwriting extracting section  18   b  extracts handwritten modification instructions, which in other words is added handwritings, from the image of the markup document acquired by the image acquiring section  18   a . The raw original document reproduction section  18   c  reproduces an original document without the modification by handwriting, which in other words is a raw original document, from the image of the markup document. The area extracting section  18   d  extracts from the raw original document each of character areas and graphic areas in the raw original document. The layout plan determination section  18   e  determines a layout plan of the raw original document based on the areas extracted by the area extracting section  18   d . The document editing section  18   f  edits the raw original document of the markup document in accordance with the modification instructions given through the added handwritings extracted by the added handwriting extracting section  18   b  to generate a digitized document. 
     The following describes operation of the MFP  10  for digitizing a document based on a markup document. 
       FIG. 2  is a flowchart illustrating operation of the MFP  10  for digitizing a document based on a markup document. 
     When an instruction instructing digitization of a document based on a markup document is input via the operation section  11 , the controller  18  performs a process illustrated in  FIG. 2 . 
     As illustrated in  FIG. 2 , the image acquiring section  18   a  uses the scanner  13  to read an image  20  (see for example  FIG. 3 ) from the markup document set in the scanner  13  (S 101 ). 
       FIG. 3  is a diagram illustrating an example of the image  20  of the markup document. 
     The image  20  illustrated in  FIG. 3  has an image  40  of a raw original document and modification instructions  31  to  38  added to the raw original document by way of handwriting using a writing material in a specific color. The specific color is for example red. 
     The instruction  31  is an instruction to add characters “ 1 / 2 ” to the right end of a header. 
     The instruction  32  is an instruction to add characters “of” between characters “Structure” and characters “Document”. The instruction  32  includes a symbol  32   a  for instructing a character insertion. 
     The instruction  33  is an instruction to delete three characters “bbb”. The instruction  33  is made of a symbol  33   a  for instructing a character deletion. 
     The instruction  34  is an instruction to swap a line that reads “ccc” with a line that reads “ddddd”. The instruction  34  is made of a symbol  34   a  for instructing a line swap. 
     The instruction  35  is an instruction to add characters “ttttt” between characters “fff” and characters “fffff”. The instruction  35  includes a symbol  35   a  for instructing a character insertion. 
     The instruction  36  is an instruction to delete a graphic. The instruction  36  is made of a symbol  36   a  for instructing a graphic deletion. 
     The instruction  37  is an instruction to move a graphic. The instruction  37  is made of a symbol  37   a  for instructing a graphic move. 
     The instruction  38  is an instruction to delete characters “ FIG. 3-2 ”. The instruction  38  is made of a symbol  38   a  for instructing a character deletion. 
     As illustrated in  FIG. 2 , after the step S 101 , the added handwriting extracting section  18   b  extracts an image  30  (see for example  FIG. 4 ) of the added handwritings from the image  20 , which is read in S 101 , based on the specific color (S 102 ). 
       FIG. 4  is a diagram illustrating the image  30  of the added handwritings in the markup document illustrated in  FIG. 3 . 
     As illustrated in  FIG. 2 , after the step S 102 , the raw original document reproduction section  18   c  reproduces an image  40  (see for example  FIG. 5 ) of a raw original document (S 103 ). More specifically, the raw original document reproduction section  18   c  removes the image  30 , which is extracted in S 102 , from the image  20 , which is read in S 101 . It should be noted here that with respect to portions of the image  20  where the image  30  of the added handwritings is superimposed on the image  40  of the raw original document (characters and graphics), the raw original document reproduction section  18   c  can reproduce the color of the raw original document based on a change in the color of the added handwritings as a result of the color of the added handwritings being superimposed on the color of the raw original document. Furthermore, with respect to portions of the image  20  where the image  30  of the added handwritings is superimposed on the image  40  of the raw original document, the raw original document reproduction section  18   c  can complement the color of the raw original document with a surrounding color, which in other words is the color of a portion where the image  30  of the added handwritings is not superimposed on the image  40  of the raw original document. 
       FIG. 5  is a diagram illustrating the image  40  of the raw original document of the markup document illustrated in  FIG. 3 . 
     As illustrated in  FIG. 2 , after the step S 103 , the area extracting section  18   d  extracts a character area or a graphic area from the image  40  of the raw original document reproduced in S 103  (S 104 ). It should be noted here that the area extracting section  18   d  extracts a character area from the image  40  in a case where the image  40  has characters therein. The area extracting section  18   d  extracts graphic areas from the image  40  on a graphic-by-graphic basis in a case where the image  40  has graphics therein. The area extracting section  18   d  can extract a plurality of character areas based on a change in distance between characters and placement of a graphic area in the image  40 . 
       FIG. 6  is a diagram illustrating the image  40  of the raw original document divided into a plurality of areas. 
     The image  40  illustrated in  FIG. 6  is divided into character areas  41 - 45  and graphic areas  46 - 47 . The area  42  includes paragraphs  42   a ,  42   b ,  42   c , and  42   d . The area  43  includes a heading  43   a  and paragraphs  43   b  and  43   c.    
     As illustrated in  FIG. 2 , after the step S 104 , the layout plan determination section  18   e  determines whether or not the image  40  of the raw original document has character areas therein (S 105 ). 
     When determining in S 105  that the image  40  has character areas therein, the layout plan determination section  18   e  performs optical character recognition (OCR) on each of the character areas thereby to recognize characters in the character area (S 106 ). 
     When determining in S 105  that the image  40  has no character areas or when the step S 106  is complete, the layout plan determination section  18   e  generates original document layout information (S 107 ). The original document layout information indicates placement of each of the character areas and the graphic areas, which are extracted in S 104 , in the original document layout. 
     For example, the layout plan determination section  18   e  determines, with respect to each of the character areas and the graphic areas, a start position (a left end), a center position, and an end position (a right end) in a left-right direction of the image  40  of the raw original document as well as a start position (an upper end) and an end position (a lower end) in a top-bottom direction of the image  40  of the raw original document. In a case where some of the thus determined positions of an area and some of the thus determined positions of another area in the image  40  of the raw original document coincide, the layout plan determination section  18   e  determines, as the layout plan of the image  40  of the raw original document, that such areas are placed in accordance with such coinciding positions in the layout. This is because it is likely that such positions are made coincide purposely. 
     The layout plan determination section  18   e  also determines distances between areas. In a case where a distance determined for areas is shorter than a specific distance, the layout plan determination section  18   e  determines, as the layout plan of the image  40  of the raw original document, that the distance between the areas is maintained in the layout. The specific distance is for example a distance equivalent to two lines of characters having a specific size. 
       FIG. 7  is a diagram illustrating the original document layout information generated from the image  40 . A distance  54  is a distance between the area  41  and the area  42 . A distance  55  is a distance between the area  42  and the area  43 . A distance  56  is a distance between the area  44  and the area  46 . A distance  57  is a distance between the area  44  and the area  47 . A distance  58  is a distance between the area  45  and the area  47 . 
     The layout plan determination section  18   e  for example determines, as the layout plan of the image  40  of the raw original document, that the start positions of the areas  41  to  43  in the left-right direction are aligned as indicated by a line  51 . For another example, the layout plan determination section  18   e  determines, as the layout plan of the image  40  of the raw original document, that the end positions of the areas  42  and  43  in the left-right direction are aligned as indicated by a line  52 . For another example, the layout plan determination section  18   e  determines, as the layout plan of the image  40  of the raw original document, that the center positions of the areas  44  to  47  in the left-right direction are aligned as indicated by a line  53 . For another example, the layout plan determination section  18   e  determines, as the layout plan of the image  40  of the raw original document, that all of the distances  54 ,  55 ,  56 ,  57 , and  58  are maintained. 
     As illustrated in  FIG. 2 , after the step S 107 , the document editing section  18   f  edits (S 108 ) the image  40  of the raw original document in accordance with the instructions given through the added handwritings, which are extracted in S 102 , and ends the operation illustrated in  FIG. 2 . 
       FIG. 8A  is a flowchart illustrating a first half of the editing in S 108 .  FIG. 8B  is a flowchart illustrating a last half of the editing in S 108 . 
     As illustrated in  FIG. 8A , the document editing section  18   f  makes a copy of the image  40  of the raw original document to generate an image being edited (S 131 ). 
     Next, based on the image  20  read in S 101  and the image  30  of the added handwritings extracted in S 102 , the document editing section  18   f  divides the added handwritings according to the distances between the added handwritings and contents of the added handwritings (S 132 ). For example, in  FIG. 4 , the document editing section  18   f  divides the added handwritings in the image  30  into the instructions  31  to  38 . 
     After the step S 132 , the document editing section  18   f  selects an unselected one of the added handwritings, which are divided in S 132 , as a target (S 133 ). 
     Next, the document editing section  18   f  determines a type of the instruction of the currently-selected target handwriting (S 134 ). 
     As illustrated in  FIG. 8B , when determining in S 134  that the instruction is a “character addition” such as the instruction  31 ,  32 , or  35 , the document editing section  18   f  recognizes all character from the currently-selected target handwriting by OCR (S 135 ). 
     Next, the document editing section  18   f  specifies a position to which the character from the currently-selected target handwriting is to be added (S 136 ). 
     More specifically, in a case where a position to which the character from the currently-selected target handwriting is to be added is appointed in a character area included in the specific layout information  17   b  and the original document layout information, the document editing section  18   f  specifies the appointed position in S 136 . 
     In a case where a position to which the character from the currently-selected target handwriting is to be added is not particularly specified in a character area included in the specific layout information  17   b  and the original document layout information, the document editing section  18   f  specifies an appropriate position in the area based on the specific layout information  17   b , the document layout information, and the position of the currently-selected target handwriting in the markup document in S 136 . For example, in a case where the start position of the currently-selected target handwriting is located close to the start positions of separate areas that are aligned in the left-right direction of the image being edited, the document editing section  18   f  puts the start position of the currently-selected target handwriting in alignment with the start positions of the separate areas. Although starting positions of areas in the left-right direction of the image being edited have been described above, the same is true of center positions and end positions of areas in the left-right direction of the image being edited, and start positions and end positions of areas in the top-bottom direction of the image being edited. The document editing section  18   f  may separate the area of the currently-selected target handwriting from an area adjacent thereto by the same distance as the distance between areas located close to the area of the currently-selected target handwriting. If no regularity is found for the currently-selected target handwriting in terms of the start position, the center position, and the end position of the area thereof in the left-right direction of the image being edited as well as the start position and end position in the top-bottom direction of the image being edited, the document editing section  18   f  may specify the position of the handwriting of the currently-selected target handwriting as the position to which the character from the currently-selected target handwriting is to be added. For example, for adding a new character area  48  to a space under the area  43 , the document editing section  18   f  defines the start position and the end position of the area  48  in the left-right direction using the line  51  and the line  52 , respectively, and positions the area  48  so that a distance  59  between the area  43  and the area  48  is equal to the distance  55  between the area  42  and the area  43  as illustrated in  FIG. 9 . 
     After the step S 136 , the document editing section  18   f  specifies attributes of the character from the currently-selected target handwriting (S 137 ). For example, in a case where the image  40  of the raw original document has an area to which the character from the currently-selected target handwriting is to be added, the document editing section  18   f  acquires attributes of characters located around the position in the area to which the character from the currently-selected target handwriting is to be added. The document editing section  18   f  then specifies the acquired attributes as the attributes of the character from the currently-selected target handwriting. 
     After the step S 137 , the document editing section  18   f  adds the character recognized in S 135  to the position, which is specified in S 136 , in the image being edited with the attributes specified in S 137  or with the attributes indicated by the character attribute information  17   c  (S 138 ). 
     In a case where the position to which the character from the currently-selected target handwriting is to be added is located in the middle of an existing area, for example, the document editing section  18   f  adds the character from the currently-selected target handwriting to the position, and accordingly moves backward characters, among the characters in the existing area, that should follow the character from the currently-selected target handwriting by the number of added characters. The position located in the middle of an existing area is for example a position between characters in a line in a character area included in the specific layout information  17   b  and the original document layout information. In a case where a character is added to a paragraph in an area, and accordingly characters that should follow the added character are moved backward, the document editing section  18   f  maintains the paragraph after moving backward the characters. In such a case, the document editing section  18   f  determines a line indented in the area to be a starting line of the paragraph. Furthermore, the document editing section  18   f  determines a line that ends with some space, a line immediately before a starting line of a following paragraph, or a last line in the area to be an ending line of the paragraph. Furthermore, after adding the character from the currently-selected target handwriting, the document editing section  18   f  moves backward characters following the area including the added character as necessary by an increase in the size of the area as a result of the addition. In a case where a distance between separate areas located downward of the area including the added character is greater than a specific distance, however, a lower area of the separate areas is not moved backward until the distance between the separate areas becomes equal to the specific distance. The specific distance is for example a distance equivalent to two lines of characters having a specific size. 
     The document editing section  18   f  can recognize a “heading” line in a character area by character recognition in S 106 . More specifically, the document editing section  18   f  recognizes a specific style of for example “Chapter . . . ” and recognizes a change in character size as character recognition. In a case where paragraphs in the area that follow the “heading” are indented, therefore, the document editing section  18   f  can be prevented from falsely detecting that each of the lines in the area that follow the “heading” constitutes a paragraph. For example, in a case where the document editing section  18   f  does not recognize a line  61  in an area  60  as a “heading”, the document editing section  18   f  recognizes each of the following lines as a paragraph as illustrated in  FIG. 10A . That is, the document editing section  18   f  falsely recognizes that paragraphs  62  to  67  are present as illustrated in  FIG. 10A . Recognizing that the line  61  in the area  60  is a “heading”, the document editing section  18   f  can correctly recognize paragraphs  68  and  69  as illustrated in  FIG. 10B . 
     When determining in S 134  that the instruction is a “graphic addition”, the document editing section  18   f  specifies a position to which a handwritten graphic from the currently-selected target handwriting is to he added (S 139 ). 
     More specifically, in S 139 , the document editing section  18   f  specifies a new layout of areas based on the specific layout information  17   b , the original document layout information, and the position of the currently-selected target handwriting in the markup document. For example, if the start position of the currently-selected target handwriting is located close to the start positions of separate areas that are aligned in the left-right direction of the image being edited, the document editing section  18   f  brings the start position of the currently-selected target handwriting in alignment with the start positions of the separate areas. Although starting positions of areas in the left-right direction of the image being edited have been described above, the same is true of center positions and end positions of areas in the left-right direction of the image being edited, and start positions and end positions of areas in the top-bottom direction of the image being edited. The document editing section  18   f  may separate the area of the currently-selected target handwriting from an area adjacent thereto by the same distance as the distance between areas located close to the area of the currently-selected target handwriting. In a case where no regularity is found for the currently-selected target handwriting in terms of the start position, the center position, and the end position of the area thereof in the left-right direction of the image being edited as well as the start position and end position in the top-bottom direction of the image being edited, the document editing section  18   f  may specify the position of the handwriting of the currently-selected target handwriting as the position to which the handwritten graphic from the currently-selected target handwriting is to be added. 
     After the step S 139 , the document editing section  18   f  adds the handwritten graphic from the currently-selected target handwriting to the position, which is specified in S 139 , in the image being edited (S 140 ). 
     After adding the handwritten graphic from the currently-selected target handwriting, the document editing section  18   f  for example moves downward characters and/or graphics that are located downward of an area including the added graphic as necessary by a size of the area including the added graphic. 
     When determining in S 134  that the instruction is a “deletion” such as the instruction  33 ,  36 , or  38 , the document editing section  18   f  specifies a character or a graphic instructed to be deleted by the currently-selected target handwriting (S 141 ). 
     Next, the document editing section  18   f  deletes the character or the graphic, which is specified in S 141 , from the image being edited (S 142 ). 
     In a case where a character or a graphic in the middle of an area is to be deleted, for example, the document editing section  18   f  deletes the character or the graphic from the area, and accordingly moves forward characters and/or graphics that are located backward of the deleted character or graphic in the area by the extent of the deleted character or graphic. In a case where a character is deleted from a paragraph in an area, and characters and/or graphics following the deleted character are moved forward, the document editing section  18   f  maintains the paragraph after moving forward the characters and/or graphics. The document editing section  18   f  can recognize a “heading” line in a character area. In a case where paragraphs in the character area that follow the “heading” are indented, therefore, the document editing section  18   f  can be prevented from falsely detecting that each of the lines in the character area that follow the “heading” constitutes a paragraph. Furthermore, after deleting a character or a graphic specified in an area, the document editing section  18   f  moves forward areas that are located downward of the area including the deleted character or graphic as necessary by a decrease in the size of the area as a result of the deletion of the specified character or graphic. 
     When determining in S 134  that the instruction is a “move” such as the instruction  34  or  37 , the document editing section  18   f  specifies a character or a drawing instructed to be moved by the currently-selected target handwriting (S 143 ). 
     Next, the document editing section  18   f  specifies a position of a move destination instructed by the currently-selected target handwriting (S 144 ). 
     Next, the document editing section  18   f  moves the character or the graphic specified in S 143  to the position, which is specified in S 144 , in the image being edited (S 145 ). 
     In a case where the character or the graphic specified in S 143  is moved to the position specified in S 144 , the document editing section  18   f  for example moves downward characters and/or graphics that are located downward of an area including the move destination as necessary by the extent of the character or the graphic specified in S 143 . In a case where a distance between the area including the move destination and an area that is located immediately downward of the area including the move destination is greater than a specific distance, however, the area that is located immediately downward of the area including the move destination is not moved downward until the distance between these areas becomes equal to the specific distance. The specific distance is for example a distance equivalent to two lines of characters having a specific size. In a case where the character or the graphic specified in S 143  is deleted at the move destination, the document editing section  18   f  moves upward areas that are located downward of the area including the deleted character or graphic as necessary by the extent of the deleted character or graphic. In a case where a character is added to a paragraph in an area, and accordingly characters that should follow the added character are moved backward, the document editing section  18   f  maintains the paragraph after moving backward the characters. In a case where a character is deleted from a paragraph in an area, and characters and/or graphics following the deleted character are moved forward, the document editing section  18   f  maintains the paragraph after moving forward the characters and/or graphics. The document editing section  18   f  can recognize a “heading” line in a character area. In a case where paragraphs in the area that follow the “heading” are indented, therefore, the document editing section  18   f  can be prevented from falsely detecting that each of the lines in the area that follow the “heading” constitutes a paragraph. 
     After the step S 138 , S 140 , S 142 , or S 145 , the document editing section  18   f  determines whether or not the added handwritings divided in S 132  include any added handwriting that has not been selected as a target yet (S 146 ). 
     When determining in S 146  that the added handwritings divided in S 132  include an added handwriting that has not been selected as a target yet, the document editing section  18   f  updates the original document layout information (S 147 ) and performs a step S 133  illustrated in  FIG. 8A . 
     When determining in S 146  that the added handwritings divided in S 132  include no more added handwriting that has not been selected as a target yet, the document editing section  18   f  ends the operation illustrated in  FIGS. 8A and 8B . 
     When digitizing a document based on the markup document illustrated in  FIG. 3 , the MFP  10  for example performs the operation illustrated in  FIG. 2  to eventually generate a document illustrated in  FIG. 11  as the image being edited. The MFP  10  can then print the document illustrated in  FIG. 11  using the printer  14  or store the document illustrated in  FIG. 11  in the storage section  17 . 
       FIG. 12  illustrates the layout of the document illustrated in  FIG. 11 . The image illustrated in  FIG. 12  incorporates the following modifications compared to the image  40  of the raw original document illustrated in  FIG. 6 . 
     The characters “of” are added to the area  41  in accordance with the instruction  32 . The start position of the area  41  in the left-right direction, and the start position and the end position of the area  41  in the top-bottom direction are not changed. 
     The three characters “bbb” are deleted from the area  42  in accordance with the instruction  33 . The line including the characters “ccc” and the line including the characters “ddddd” are swapped in the area  42  in accordance with the instruction  34 . The start position and the end position of the area  42  in the left-right direction, and the start position of the area  42  in the top-bottom direction are not changed. The area  42  is reduced by one line, and accordingly the end position of the area  42  in the top-bottom direction is moved upward by one line. 
     The characters “ttttt” are added to the area  43  in accordance with the instruction  35 . The start position and the end position of the area  43  in the left-right direction, and the end position of the area  43  in the top-bottom direction are not changed. As a result of the area  42  being reduced by one line, the start position of the area  43  in the top-bottom direction is moved upward by one line. 
     The area  45  is deleted in accordance with the instruction  38 . 
     The area  46  is deleted in accordance with the instruction  36 . 
     The area  47  is moved in accordance with the instruction  37 . The center position of the area  47  in the left-right direction is not changed. A distance  70  between the end position of the area  47  in the top-bottom direction and the start position of the area  44  in the top-bottom direction is equal to the distance  56  between the area  44  and the area  46  in the image  40  of the raw original document. 
     The characters “ 1 / 2 ” are added to the header in the area  49  in accordance with the instruction  31 . The document editing section  18   f  sets the layout within the header in accordance with the specific layout information  17   b.    
     As described above, the MFP  10  generates the digitized document by altering the position of at least some of the characters and the graphics included in the raw original document of the markup document. Thus, the adequacy of the layout of the digitized document based on the markup document can be improved. 
     The MFP  10  generates the digitized document by editing the raw original document in accordance with the layout plan of the raw original document. Thus, the adequacy of the layout of the digitized document based on the markup document can be improved. 
     When performing at least one of a character addition or a character deletion on a paragraph of the raw original document, the MFP  10  maintains the paragraph after the editing of the raw original document. Thus, the adequacy of the layout of the digitized document based on the markup document can be further improved. 
     The MFP  10  can reproduce the raw original document from the markup document even if the raw original document itself is not available. Thus, usability can be improved. Alternatively, the MFP  10  may store the image of the raw original document in the storage section  17  and use the image of the raw original document stored in the storage section  17  without reproducing the raw original document from the markup document. 
     Some steps of the document digitizing method according to the present disclosure may for example be implemented by a computer such as a personal computer (PC) instead of the MFP  10 . 
     Although the present embodiment has been described using an example in which the image forming apparatus of the present disclosure is an MFP, the image forming apparatus may be any image forming apparatuses other than MFPs.