Abstract:
A non-transitory computer readable medium storing a translation program causes a computer to execute a process. The process includes: displaying image information, text regions, and original text in association with each other, the text regions being obtained by extracting regions including an image of text from the image information, the original text being obtained by performing character recognition on the text included in the text regions; and editing the text regions in accordance with the content of a received operation.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2016-034242 filed Feb. 25, 2016. 
       BACKGROUND 
     Technical Field 
       [0002]    The present invention relates to a non-transitory computer readable medium storing a translation program and an information processing apparatus and method. 
       SUMMARY 
       [0003]    According to an aspect of the invention, there is provided a non-transitory computer readable medium storing a translation program causing a computer to execute a process. The process includes: displaying image information, text regions, and original text in association with each other, the text regions being obtained by extracting regions including an image of text from the image information, the original text being obtained by performing character recognition on the text included in the text regions; and editing the text regions in accordance with the content of a received operation. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0004]    Exemplary embodiments of the present invention will be described in detail based on the following figures, wherein: 
           [0005]      FIG. 1  is a block diagram illustrating an example of the configuration of an information processing apparatus according to an exemplary embodiment; 
           [0006]      FIG. 2  is a schematic view illustrating an example of the configuration of a screen to be displayed when a request to translate a document is received; 
           [0007]      FIG. 3  is a schematic view illustrating an example of the configuration of image information received by a document receiver; 
           [0008]      FIG. 4  is a schematic view illustrating examples of text regions extracted by a text region extractor; 
           [0009]      FIG. 5  is a table indicating an example of the configuration of text region information; 
           [0010]      FIG. 6  is a schematic view illustrating an example of the configuration of a screen displayed by a display controller; 
           [0011]      FIG. 7  is a schematic view illustrating another example of the configuration of a screen displayed by the display controller; 
           [0012]      FIG. 8  is a schematic view illustrating another example of the configuration of a screen displayed by the display controller; 
           [0013]      FIG. 9  is a schematic view illustrating another example of the configuration of a screen displayed by the display controller; 
           [0014]      FIG. 10  is a flowchart illustrating an example of an operation performed by the information processing apparatus; 
           [0015]      FIG. 11  is a flowchart illustrating an example of an operation performed by the information processing apparatus when editing text regions; 
           [0016]      FIG. 12  is a schematic view illustrating an example of the configuration of reliability degree information; 
           [0017]      FIG. 13  is a schematic view illustrating an example of the configuration of a list created by a text region estimator; 
           [0018]      FIG. 14A  is a schematic view illustrating the configuration of a list created by the text region estimator; 
           [0019]      FIG. 14B  is a schematic view illustrating examples of the coordinates of text items in document information; 
           [0020]      FIG. 15  is a schematic view illustrating an example of an area comparing operation performed by the text region estimator; 
           [0021]      FIG. 16  is a schematic view illustrating another example of the area comparing operation performed by the text region estimator; 
           [0022]      FIG. 17  is a schematic view illustrating another example of the area comparing operation performed by the text region estimator; 
           [0023]      FIG. 18  is a schematic view illustrating examples of new text region candidates estimated by the text region estimator; 
           [0024]      FIG. 19  is a schematic view illustrating other examples of new text region candidates estimated by the text region estimator; and 
           [0025]      FIG. 20  is a schematic view illustrating an example of the configuration of a screen displayed by the display controller. 
       
    
    
     DETAILED DESCRIPTION 
     First Exemplary Embodiment 
     (Configuration of Information Processing Apparatus) 
       [0026]      FIG. 1  is a block diagram illustrating an example of the configuration of an information processing apparatus  1  according to a first exemplary embodiment. 
         [0027]    The information processing apparatus  1  includes a control device  10 , a storage device  11 , and a communication device  12 . The control device  10 , which is constituted by, for example, a central processing unit (CPU), controls the individual elements of the information processing apparatus  1  and also executes various programs. The storage device  11  is constituted by a storage medium, such as a flash memory, and stores information therein. The communication device  12  communicates with an external source via a network. 
         [0028]    By executing a translation program  110 , which will be discussed later, the control device  10  is able to serve as a document receiver  100 , a text region extractor  101 , a text recognizer  102 , a translator  103 , a display controller  104 , a text region editor  105 , a reliability degree calculator  106 , and a text region estimator  107 . 
         [0029]    The document receiver  100  receives document information from an external source via the communication device  12 . The document information is image information indicating characters and images or image information indicating images obtained by scanning printed matter including characters and images. 
         [0030]    If text is in included in the image information received by the document receiver  100 , the text region extractor  101  extracts regions where text items are disposed, as text regions. The text region extractor  101  registers the coordinates, height, and width of each of the extracted text regions in text region information  111  of the storage unit  11 . 
         [0031]    The text recognizer  102  recognizes the text included in each of the text regions extracted by the text region extractor  101  by using, for example, an optical character recognition (OCR) technique, so as to generate text information. The text recognizer  102  also registers the generated text information in the text region information  111  as the original text. 
         [0032]    The translator  103  translates the text information generated by the text recognizer  102  as the original text into another language so as to generate translated text. 
         [0033]    The display controller  104  displays an operation screen, and also displays the received image information and the original text in association with each other by referring to the text region information  111 . 
         [0034]    The text region editor  105  merges text regions or divides a text region automatically or in accordance with the content of an operation performed by a user. 
         [0035]    The reliability degree calculator  106  calculates, as an index, the reliability degree representing how much a character string included in a text region is reliable as a character string. 
         [0036]    The text region estimator  107  merges text regions having a low reliability degree or divides a text region having a low reliability degree. Approaches to merging text regions and dividing a text region will be discussed later. 
         [0037]    In the storage device  11 , the translation program  110 , the text region information  111 , and reliability degree information  112  are stored. The translation program  110  causes the control device  10  to operate as the document receiver  100 , the text region extractor  101 , the text recognizer  102 , the translator  103 , the display controller  104 , the text region editor  105 , the reliability degree calculator  106 , and the text region estimator  107 . 
         [0038]    The information processing apparatus  1  is connected to a terminal (not shown) via the communication device  12 , and executes processing in response to a request from the terminal and transmits the processing result to the terminal. The terminal receives the processing result and displays a screen indicating the processing result on a display. The screen displayed on the terminal will be discussed later. 
       (Operation of Information Processing Apparatus) 
       [0039]    The operation of the information processing apparatus of the first exemplary embodiment will now be described below. 
         [0040]    A user first accesses the information processing apparatus  1  by using a terminal (not shown) and requests the information processing apparatus  1  to translate a document. 
         [0041]    The information processing apparatus  1  receives a request from the user and causes a display of the terminal to display the following screen. 
         [0042]      FIG. 2  schematically illustrates an example of the configuration of a screen  104   a  to be displayed on the display of the terminal when a request to translate a document is received from the user. 
         [0043]    The screen  104   a  includes a selection field  104   a   1  for selecting a source (original) language, a selection field  104   a   2  for selecting a target language, a selection field  104   a   3  for selecting a document to be translated, and a selection field  104   a   4  for selecting a page of the document to be translated. The screen  104   a  also includes buttons  104   a   3 , one of which is used for instructing the execution of translation and the other one of which is used for instructing the canceling of a request. 
         [0044]    The user selects one of the source languages in the selection field  104   a   1 , selects one of the target languages in the selection field  104   a   2 , specifies the document to be translated (document information) in the selection field  104   a   3 , selects a page of the document to be translated in the selection field  104   a   4 , and then presses the button  104   a   5  for instructing the execution of translation. 
         [0045]      FIG. 10  is a flowchart illustrating an example of the operation performed by the information processing apparatus  1 . 
         [0046]    In step S 1 , the document receiver  100  receives document information concerning a document specified in the selection field  104   a   3 . The document information is, for example, the following image information. 
         [0047]      FIG. 3  schematically illustrates an example of the configuration of image information  100   a  received by the document receiver  100 . 
         [0048]    The image information  100   a  has a title “5. Correlation of diagram of seller and buyer” and a table constituted by items “Key Issue” and “environment”. 
         [0049]    Then, in step S 2 , the text region extractor  101  specifies the first page of the image information  100   a  received by the receiver  100  as the subject page. In step S 3 , the text region extractor  101  extracts, as text regions, regions where text items are disposed in the first page. The text regions may be extracted by using a technique such as the layout structure recognition in a document image. 
         [0050]      FIG. 4  schematically illustrates examples of the text regions extracted by the text region extractor  101 . 
         [0051]    Text regions  101   a   1  through  101   a   15  are extracted from the image information  100   a  by the text region extractor  101 . A text region would normally be extracted according to the word or the phrase, such as in the text regions  101   a   10  and  101   a   12 . However, there may be some cases in which a text region is extracted according to the character, such as in the text regions  101   a   1  and  101   a   2 , and a text region is extracted as a set of words which does not make sense as a phrase, such as in the text region  101   a   11 , or text regions are extracted as a word or a set of words at a position at which one phrase is split inappropriately, such as in the text regions  101   a   13  through  101   a   15 . 
         [0052]    Then, in step S 4 , the text recognizer  102  recognizes the text included in each of the text regions extracted by the text region extractor  101  by using, for example, an OCR technique, thereby generating the text region information  111 . 
         [0053]      FIG. 5  is a table indicating an example of the configuration of the text region information  111 . 
         [0054]    The text region information  111  indicates the original text obtained as a result of performing character recognition on the images within the text regions, the coordinates, height, and width of each of the extracted text regions, and the image of the text included in each of the text regions. 
         [0055]    Then, in step S 5 , the display controller  104  displays the received image information and the original text in association with each other by referring to the text region information  111 , and receives an editing operation of the image information displayed on the screen from the user. Details of step S 5  are shown in the flowchart of  FIG. 11 . The above-described steps S 3  through S 5  are executed for each page of the document information. More specifically, it is determined in step S 6  whether or not all the pages of the document information have been processed. If the result of step S 6  is NO, the next page is set as the subsequent page in step S 7 . The process then returns to step S 3  and steps S 3  through S 5  are repeated until it is determined in step S 6  that all the pages of the document information have been processed. 
         [0056]      FIG. 11  is a flowchart illustrating an example of the operation performed by the information processing apparatus  1  in step S 5 . 
         [0057]    In step S 50 , the display controller  104  displays the document information (image information) and the original text, as indicated in the following screen shown in  FIG. 6 . 
         [0058]      FIG. 6  schematically illustrates an example of the configuration of a screen  104   b  displayed by the display controller  104 . 
         [0059]    The screen  104   b  includes a document display region  104   b   1  in which the received image information and the extracted text regions are displayed, and a character recognition result display region  104   b   2  in which the original text is displayed. The screen  104   b  also includes a page switching button  104   b   3  for switching the page if there are plural pages in the document information, merge check boxes  104   b   4  for selecting text regions to be merged together, and a selection frame  104   b   5  to be displayed in accordance with the check result of the merge check boxes  104   b   4 . The screen  104   b  also includes a divide button  104   b   6  for dividing a text region by using a cursor, which will be discussed later, a merge button  104   b   7  for merging the text regions for which the merge check boxes  104   b   4  are checked, a translation button  104   b   8  for translating the original text in all the text regions, and a cancel button  104   b   9  for closing the screen  104   b.    
         [0060]    If one of the selection frame  104   b   5  and the merge check boxes  104   b   4  is selected, the other one is automatically selected. That is, if a text region in the document image is selected by using the selection frame  104   b   5 , the corresponding merge check boxes  104   b   4  are checked, so that the user can identify how (as which characters) the text included in the selected text region is recognized. If a certain portion of the original text is selected by using the corresponding merge check box  104   b   4 , the selection frame  104   b   5  appears within the document image, so that the user can identify in which text region the selected portion of the original text is included. 
         [0061]    If the user checks merge check boxes  104   b   4  on the screen  104   b , the text region editor  105  receives the input of the merge check boxes  104   b   4  in step S 51 . 
         [0062]    Then, if the user presses the merge button  104   b   7 , the text region editor  105  receives the operation for pressing the merge button  104   b   7  in step S 52 . Then, in step S 53 , the text region editor  105  merges the text regions corresponding to the selected merge check boxes  104   b   4  and updates the text region information  111 . Then, in step S 54 , the text region editor  105  updates the display content of the screen  104   b , as shown in  FIG. 7 . 
         [0063]      FIG. 7  schematically illustrates another example of the configuration of the screen  104   b  displayed by the display controller  104 . 
         [0064]    In accordance with the above-described operation, the text regions selected by using the selection frame  104   b   5  are merged together, so that the character string within the merged text region is indicated as “Correlation diagram of seller and buyer”. An OCR button  104   b   10  has appeared in a portion corresponding to the merged text region in the character recognition result display region  104   b   2 . If the first character recognition in the merged text region is not correct, the user presses the OCR button  104   b   10  so that the text recognizer  102  can recognize the text in the merged text region again. 
         [0065]    In another example, if the user presses the divide button  104   b   6 , the text region editor  105  receives the operation for pressing the divide button  104   b   6  in step S 52 . Then, in step S 53 , the text region editor  105  divides the corresponding text region and updates the text region information  111 . Then, in step S 54 , the text region editor  105  updates the display content of the screen  104   b , as shown in  FIG. 8 . 
         [0066]      FIG. 8  schematically illustrates another example of the configuration of the screen  104   b  displayed by the display controller  104 .  FIG. 9  schematically illustrates another example of the configuration of the screen  104   b  displayed by the display controller  104 . 
         [0067]    As shown in  FIG. 8 , the user specifies a position at which the text region will be divided by using a cursor  104   b   11 , and presses the divide button  104   b   6 . In accordance with this operation, the character string “Strength Knowledge of Products and Know-how” within the text region is divided into “Strength” and “Knowledge of Products and Know-how”, as shown in  FIG. 9 . 
         [0068]    Then, if the user judges that the text regions are appropriately merged or divided, the user presses the translation button  104   b   8 . 
         [0069]    In step S 8  of  FIG. 10 , the translator  103  sets the text information generated by the text recognizer  102  to be the original text, and translates the original text into another language so as to generate translated text. Then, in step S 9 , the display controller  104  disposes the translated text within the associated text regions of the document information so as to display the translation results on the display of the terminal. When the display controller  104  disposes the translated text, the font size and font type of the translated text are selected by considering the height and width of each text region. 
       Second Exemplary Embodiment 
       [0070]    A second exemplary embodiment of the invention will be described below. The second exemplary embodiment is different from the first exemplary embodiment in that it is determined whether or not it is more suitable if text regions extracted by the text region extractor  101  are merged or divided, and, if it is such a case, a new text region is estimated. The configuration of the second exemplary embodiment is similar to that of the first exemplary embodiment, and an explanation thereof will thus be omitted. 
         [0071]    In a manner similar to the first exemplary embodiment, the information processing apparatus  1  executes steps S 1  through S 4  in  FIG. 10 . Then, the reliability degree calculator  106  calculates the reliability degree of each text item recognized by the text recognizer  102 , and generates reliability degree information  112 , as shown in  FIG. 12 . 
         [0072]      FIG. 12  schematically illustrates an example of the configuration of the reliability degree information  112 . 
         [0073]    The reliability degree calculator  106  calculates the reliability degree of each text item in the original text by making the following determinations. If a text item is a character or a word, it is determined whether or not the character or the word is included in a prepared dictionary. If a text item is a sentence, it is determined whether or not it is grammatically correct. In the reliability degree information  112  shown in  FIG. 12 , for example, “C”, “o”, “r” “r”, “e”, “l”, “a”, “t”, “i”, “o”, and “n”, which are each constituted by one alphabetical character, are not registered in the dictionary as a word. Accordingly, the reliability degree calculator  106  calculates the reliability degree of these text items to be 0. The text item “Strength Knowledge of Products and Know-how” is not grammatically correct since “Strength” is not suitable in this text item. Accordingly, the reliability degree calculator  106  calculates the reliability degree of this text item to be 50. Similarly, the text item “Using its high” is not grammatically correct, and the reliability degree calculator  106  calculates the reliability degree of this text item to be 40. 
         [0074]    The display controller  104  may display the content of the reliability degree information  112  shown in  FIG. 12  for the user. 
         [0075]    Then, the text region estimator  107  creates all possible combinations of thirteen text items for which the reliability degree is calculated to be equal to or lower than a predetermined threshold (for example, 50). The combinations of the text items are formed into a list  107   a , as shown in  FIG. 13 . 
         [0076]      FIG. 13  schematically illustrates an example of the configuration of the list  107   a  created by the text region estimator  107 . 
         [0077]    The number of combinations included in the list  107   a  is 2 13 =8192. 
         [0078]    Then, the text region estimator  107  assumes all the combinations of the text items included in the list  107   a  as new text regions. The text region estimator  107  then adds the coordinates at the top left of each new text region and the width and height of each new text region to the list  107   a  as information, thereby creating a new list  107   b , as shown in  FIG. 14A . 
         [0079]      FIG. 14A  schematically illustrates the configuration of the list  107   b  created by the text region estimator  107 , and  FIG. 14B  schematically illustrates examples of the coordinates of text items in document information. 
         [0080]    As shown in  FIG. 14A , in the list  107   b , the coordinates of the top left of each new text region and the width and height of each new text region are indicated. For example, concerning a text region [C, o, r, r, e, l, a, t, i, o, n], the coordinates at the top left of a text region  107   c   1  including “C” are (60, 10), as shown in  FIG. 14B , and the width and height of the entire text region are (70, 12), as shown in  FIG. 14A . Accordingly, [C, o, r, r, e, l, a, t, i, o, n] are indicated as [C, o, r, r, e, l, a, t, i, o, n: 60, 10, 70, 12] in the list  107   b . Concerning [C, r, using its high], the coordinates at the bottom right of a text region  107   c   11  including “using its high” are (260, 70), and thus, the width and height are (200, 60). Accordingly, [C, r, using its high] is indicated as [C, r, using its high: 60, 10, 200, 60]. 
         [0081]    Then, the text region estimator  107  calculates, from the list  107   b , the area of the new text region and the total area of the original text regions included in the new text region, and compares the two areas with each other. 
         [0082]      FIG. 15  illustrates an example of the area comparing operation performed by the text region estimator  107 . 
         [0083]    The area of a new text region  107   d   1  corresponding to the topmost line of the list  107   b  [C: 60, 10, 8, 12] is 8×12=96. The area of the text region  107   c   1  of the original text “C” included in the new text region  107   d   1  is 8×12=96. The ratio between the two areas is 96/96=1. 
         [0084]    If the ratio between the two areas is equal to or higher than a predetermined threshold (for example, 0.7), the text region estimator  107  determines the new text region  107   d   1  to be a new text region candidate. 
         [0085]      FIG. 16  illustrates another example of the area comparing operation performed by the text region estimator  107 . 
         [0086]    The area of a new text region  107   d   2  corresponding to [C, o, r, r, e, l, a, t, i, o, n: 60, 10, 70, 12] in the list  107   b  is 70×12=840. The total area of the text regions  107   c   1  through  107   c   11  of the original text items “C”, “o”, “r” “r”, “e”, “l”, “a”, “t”, “i”, “o”, and “n” included in the new text region  107   d   2  is 750. The ratio between the two areas is 750/840=0.89. 
         [0087]    If the ratio between the two areas is equal to or higher than the predetermined threshold (for example, 0.7) and is not 1, the text region estimator  107  determines the new text region  107   d   2  to be a new text region candidate and eliminates the new text region  107   d   1  for which the ratio is calculated to be 1 from text region candidates. The reason for this is that the user can more easily identify text in a region of a larger area. 
         [0088]      FIG. 17  illustrates another example of the area comparing operation performed by the text region estimator  107 . 
         [0089]    The area of a new text region  107   d   3  corresponding to [C, r, using its high: 60, 10, 200, 60] in the list  107   b  is 200×60=12000. The total area of the text regions  107   c   1 ,  107   c   3 , and  107   c   11  of the original text items “C”, “r”, “using its high” included in the new text region  107   d   3  is 1200. The ratio between the two areas is 1200/12000=0.10. 
         [0090]    If the ratio between the two areas is lower than the predetermined threshold (for example, 0.7), the text region estimator  107  does not set the new text region  107   d   3  to be a new text region candidate. 
         [0091]    Concerning all the new text regions in the list  107   b , the text region estimator  107  performs the above-described calculations, and, as a result, new text region candidates are determined, as shown in  FIG. 18 . 
         [0092]      FIG. 18  schematically illustrates examples of new text region candidates estimated by the text region estimator  107 . 
         [0093]    The text region estimator  107  estimates new text region candidates  107   e   1  through  107   e   3  by performing the above-described calculations. 
         [0094]    The text region estimator  107  may also merge a text region having a sufficiently high reliability degree with the new text region candidates  107   e   1  through  107   e   3 . 
         [0095]      FIG. 19  schematically illustrates an example of new text region candidates estimated by the text region estimator  107 . 
         [0096]    For example, the y coordinate at the top left of the new text region candidate  107   e   1  in  FIG. 18  and that of the text region “diagram of seller and buyer” are both  10  and are thus equal to each other. Accordingly, the text region estimator  107  merges this text region with the text region candidate  107   e   1 , and then causes the reliability degree calculator  106  to calculate the reliability degree of the merged text region “Correlation diagram of seller and buyer”. If the reliability degree is equal to or higher than the predetermined threshold, the text region estimator  107  determines the merged text region to be a new text region candidate  107   f   1 , as shown in  FIG. 19 . 
         [0097]    The x coordinate at the top left of the new text region candidate  107   e   3  in  FIG. 18  is equal to that of the text regions “Strengthen online sales” and “technology”. Accordingly, the text region estimator  107  merges these two text regions with the new text region candidate  107   e   3 , and then causes the reliability degree calculator  106  to calculate the reliability degree of the merged text region “Strengthen online sales using its high technology”. If the reliability degree is equal to or higher than the predetermined threshold, the text region estimator  107  determines the merged text region to be a new text region candidate  107   f   3 , as shown in  FIG. 19 . 
         [0098]    Then, the display controller  104  causes merge check boxes  104   c   4  on a screen  104   c  shown in  FIG. 20  to be automatically checked so that the original text regions  107   c   1  through  107   c   11  shown in  FIG. 14B  will be changed to the new text regions  107   f   1  through  107   f   3  shown in  FIG. 19  estimated by the text region estimator  107 . 
         [0099]      FIG. 20  schematically illustrates an example of the configuration of the screen  104   c  displayed by the display controller  104 . 
         [0100]    The configuration of the screen  104   c  is similar to that of the screen  104   b  shown in  FIG. 6 , and the merge check boxes  104   c   4  are checked by the text region estimator  107 . The display controller  104  displays the screen  104   c  in this manner and receives an editing operation from the user, as in the first exemplary embodiment. 
         [0101]    The display controller  104  may display the content of the screens shown in  FIGS. 14 through 19 . 
       Other Exemplary Embodiments 
       [0102]    The present invention is not restricted to the above-described exemplary embodiments, and various modifications may be made without departing from the spirit of the invention. 
         [0103]    In the above-described exemplary embodiments, the functions of the document receiver  100 , the text region extractor  101 , the text recognizer  102 , the translator  103 , the display controller  104 , the text region editor  105 , the reliability degree calculator  106 , and the text region estimator  107  of the control device  10  are implemented by a program. Alternatively, all or some of these functions may be implemented by hardware, such as an application-specific integrated circuit (ASIC). The program used in the above-described exemplary embodiments may be stored in a recording medium such as a compact disc-read only memory (CD-ROM) and be provided. Additionally, swapping, deletion, and addition of steps discussed in the above-described exemplary embodiments may be performed without departing from the spirit of the invention. 
         [0104]    The foregoing description of the exemplary embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to understand the invention for various embodiments and with the various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.