Server, server control method, program and recording medium

A request receiver (101) receives a request in which image ID information and a parameter for image processing are specified, from a terminal. An image processor (102) acquires an image based on the ID information that was specified in the received request, and by using the parameter that was specified in the received request to apply the image processing on the acquired image, outputs extracted information that was extracted from inside the image. A response transmitter (103) transmits a response in which the extracted information that was outputted by the image processor (102) is specified, to the terminal. When the extracted information that is associated with the ID information and the parameter that were specified in the received request is cached, the image processor (102), instead of acquiring the image and applying the image processing, obtains the cached extracted information, and outputs the obtained extracted information.

TECHNICAL FIELD

The present invention relates to a server, a server control method, a program and a recording medium, and particularly, relates to the translation of character strings inside an image.

BACKGROUND ART

Conventionally, various translation tools have been known; however, there is a need for translation of WEB pages that have images in which character strings are included using those translation tools. In order to translate character strings that are included in an image, first it is necessary to perform character recognition of characters that are included in an image. Then, after the characters have been identified by character recognition, those characters are translated into the target language. However, the character recognition processing requires a certain amount of time, so that there was a problem in that translation in real-time was difficult. Various methods have been proposed to deal with this kind of problem. For example, Patent Literature 1 discloses a machine translation system that associates and stores images that include characters with character strings for translation, and when translating an image that includes characters, provides a translation result of the character string.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Technical Problem

There is also a desire to translate characters in an image as is without preparing character strings for translation.

The objective of the present invention is to solve a problem such as described above by providing a server, a server control method, a program and a recording medium that are suitable for translating characters of images in a WEB page in real-time.

Solution to Problem

A server according to a first aspect of the present invention, has

a request receiver that receives a request in which image ID information and a parameter for image processing are specified, from a terminal;

an image processor that acquires an image based on the ID information that was specified in the received request, and by using the parameter that was specified in the received request to apply the image processing on the acquired image, outputs extracted information that was extracted from inside the image; and

a response transmitter that transmits a response in which the extracted information that was outputted by the image processor is specified, to the terminal; wherein

after the extracted information has been extracted, the image processor associates and caches the extracted information with the image ID information and the parameter; and

when the extracted information that is associated with the ID information and the parameter that were specified in the received request is cached, the image processor, instead of acquiring the image and applying the image processing, obtains the cached extracted information, and outputs the obtained extracted information.

In the server according to the aspect above,

the parameter that is specified in the request includes a specified area;

the image processing by the image processor includes extraction processing that, by performing character recognition on the specified area inside the image that is included in the parameter, outputs a recognized character string that was recognized from the specified area in the image;

after the recognized character string has been extracted, the image processor associates and caches the recognized character string with the image ID information and the specified area;

when a recognized character string that is associated with the ID information and the specified area included in the parameter that are specified in the received request is cached, the image processor, instead of the character recognition, obtains the cached recognized character string, and outputs that obtained recognized character string; and

the extracted information includes the recognized character string.

Moreover, in the server according to the aspect above,

when the ID information that was associated when the recognized character string was cached matches the ID information that was specified in the received request, and the specified area that was associated when the recognized character string was cached overlaps the specified area that is included in the parameter that was specified in the received request, the image processor determines that the extracted information that is associated with the ID information and the specified area included in the parameter that were specified in the received request is cached.

Furthermore, in the server according to the aspect above,

when the recognized character string is extracted, the image processor caches the recognized character string after correcting the specified area to a recognized area in the specified area where the recognized character string was recognized.

In the server according to the aspect above,

the parameter that is specified in the request includes a language for recognition; and

the character recognition by the image processor is performed using the language for recognition.

Moreover, in the server according to the aspect above,

the parameter that is specified in the request includes conversion information;

the image processing by the image processor includes conversion processing that, by performing a conversion processing on the outputted recognized character string using the conversion information that is included in the parameter that was specified in the received request, outputs a converted character string that is converted from the recognized character string; and

the extracted information includes the converted character string.

Furthermore, in the server according to the aspect above,

when the converted character string is extracted, the image processor associates and caches the converted character string with the image ID information, the specified area, the language for recognition, the recognized character string, and the conversion information; and

when the converted character string is cached for the ID information, the specified area, the language for recognition and the conversion information that were specified in the received request, the image processor, instead of the conversion processing, obtains the cached converted character string, and outputs the obtained converted character string.

In the server according to the aspect above,

the conversion processing obtains the converted character string by translating the recognized character string to a translation target language that is specified in the conversion information.

Moreover, the server according to the aspect above, further has

an inquiry receiver that receives an inquiry in which image ID information is specified, from the terminal; and

an answer transmitter that,when the extracted information that is associated with the ID information that is specified in the received inquiry is not cached, transmits an answer in which a message is specified that prompts transmission of the request in which the image ID information and the parameter for image processing are specified, to the terminal andwhen the extracted information that is associated with the ID information that is specified in the received inquiry is cached, transmits an answer in which the extracted information is specified, to the terminal.

Furthermore, the server according to the aspect above, further has:

an inquiry receiver that receives an inquiry in which the image ID information and a position inside that image are specified, from the terminal; and

an answer transmitter that, when a recognized character string that is associated with the ID information that is specified in the received inquiry and a specified area that includes the position that is specified in the received inquiry is cached by the image processor, transmits an answer in which the cached recognized character string is specified, to the terminal.

The server according to the aspect above, further has:

an inquiry receiver that receives an inquiry in which image ID information, a position inside the image and the conversion information are specified, from the terminal; and

an answer transmitter that, when the converted character string that is associated with the ID information that is specified in the received inquiry, the conversion information, and the specified area that includes the position specified in the received inquiry is cached by the image processor, transmits an answer in which the cached converted character string is specified, to the terminal.

A program according to a second aspect of the present invention is a program that is executed by a computer that has a display that displays an image identified by the ID information on a screen and that is capable of communicating with the server according to the aspect above, and causes the computer to function as:

a position detector that detects a selection of a position inside the image that is displayed on the screen;

an inquiry transmitter that, when the selection of the position is detected, transmits an inquiry in which the ID information and the selected position are specified, to the server; and

an answer receiver that receives an answer from the server;

the program causes the computer to function such that

the display further displays on the screen the extracted information that is specified in the received answer;

the program causes the computer to function as:

an area detector that detects a selection of an area inside the image that is displayed on the screen;

a setter that prompts setting the parameter for image processing;

a request transmitter that, when the selection of the area is detected, transmits a request in which the ID information, the set parameter, and the selected area are specified, to the server; and

a response receiver that receives a response from the server; and

the program causes the computer to function such that

the display further displays on the screen the extracted information that is specified in the received response.

A control method for a server according to a third aspect of the present invention, has:

a request receiving step for receiving a request in which image ID information and a parameter for image processing are specified, from a terminal;

an image processing step for acquiring an image based on the ID information that was specified in the received request, and by using the parameter that was specified in the received request to apply the image processing on the acquired image, outputting extracted information that was extracted from inside the image;

a response transmission step for transmitting a response in which the extracted information that was outputted is specified, to the terminal; wherein

the image processing step includes,

after the extracted information has been extracted, associating and caching the extracted information with the image ID information and the parameter; and

when the extracted information that is associated with the ID information and the parameter that were specified in the received request is cached, instead of acquiring the image and applying the image processing, obtaining the cached extracted information, and outputting the obtained extracted information.

A program according to a fourth aspect of the present invention causes a computer to function as:

a request receiver that receives a request in which image ID information and a parameter for image processing are specified, from a terminal;

an image processor that acquires an image based on the ID information that was specified in the received request, and by using the parameter that was specified in the received request to apply the image processing on the acquired image, outputs extracted information that was extracted from inside the image; and

a response transmitter that transmits a response in which the extracted information that was outputted by the image processor is specified, to the terminal; wherein

after the extracted information has been extracted, the image processor associates and caches the extracted information with the image ID information and the parameter; and

when extracted information that is associated with the ID information and the parameter that were specified in the received request is cached, the image processor, instead of acquiring the image and applying the image processing, obtains the cached extracted information, and outputs the obtained extracted information.

A recording medium according to a fifth aspect of the present invention readable by a computer and records a program that causes the computer to function as:

a request receiver that receives a request in which image ID information and a parameter for image processing are specified, from a terminal;

an image processor that acquires an image based on the ID information that was specified in the received request, and by using the parameter that was specified in the received request to apply the image processing on the acquired image, outputs extracted information that was extracted from inside the image; and

a response transmitter that transmits a response in which the extracted information that was outputted by the image processor is specified, to the terminal; wherein

after the extracted information has been extracted, the image processor associates and caches the extracted information with the image ID information and the parameter; and

when the extracted information that is correlated with the ID information and the parameter that were specified in the received request is cached, the image processor, instead of acquiring the image and applying the image processing, obtains the cached extracted information, and outputs the obtained extracted information.

The program above can be distributed and sold by way of a computer communication network independent from the computer that executes the program. Moreover, the recording medium above can be distributed and sold independent from the computer.

Here, a non-transitory recording medium is a tangible recording medium. A non-transitory recording medium is, for example, a compact disk, a flexible disk, a hard disk, a magneto-optical disk, a digital video disk, a magnetic tape, a semiconductor memory and the like. Moreover, a transitory recording medium is the transmission medium (propagation signal) itself. A transitory recording medium, for example, is an electric signal, an optical signal, an electromagnetic waves and the like. A temporary memory area is an area for temporarily storing data and programs, and is, for example, a volatile memory such as RAM (Random Access Memory).

Advantageous Effects of Invention

With the present invention it is possible to provide a server, a server control method, a program and a recording medium that are suitable for translating characters in an image on a WEB page in real-time.

DESCRIPTION OF EMBODIMENTS

As illustrated inFIG. 1, a server100of embodiments of the present invention is connected to a Internet300. Plural terminals211,212to21n(hereafter, these will collectively be called “terminal200”) that are operated by users and a WEB server400are connected to the Internet300. The WEB server400provides a specified WEB page to the terminal200according to a request from the terminal200. The server100then performs translation of the WEB page according to a request from the terminal200, and provides the translation result to the terminal200. In the embodiments that will be described later, the server100and the WEB server400are connected by way of the Internet300as illustrated inFIG. 1; however, this is only an example. For example, it is possible for the server100and the WEB server400to be achieved by the same device. Alternatively, the server100and the WEB server400can also be connected directly.

In the following, a typical information processing device500by which the server100or the terminal200of the embodiments of the present invention is achieved will be explained.

(1. Basic Construction of an Information Processing Device)

As illustrated inFIG. 2, the information processing device500has a CPU (Central Processing Unit)501, a ROM (Read Only Memory)502, a RAM503, an NIC (Network Interface Card)504, an image processor505, an audio processor506, a DVD-ROM (Digital Versatile Disc ROM) drive507, an interface508, an external memory509, a controller510, a monitor511and a speaker512.

The CPU501controls the overall operation of the information processing device500, and exchanges control signals and data with each of the connected component elements.

The ROM502stores an IPL (Initial Program Loader) that is executed immediately after the power has been turned ON, and by executing this IPL, a specified program is read to the RAM503, and the CPU501starts execution of that program. The ROM502also stores an operating system program and various data that are necessary for controlling the overall operation of the information processing device500.

The RAM503is for temporarily storing data and programs, and stores programs and data that are read from a DVD-ROM, as well as data that is necessary for communication.

The NIC504is for connecting the information processing device500to a computer network such as the Internet300, and has an interface (not illustrated in the figure) for acting as a go-between for devices such as a device according to the 10 BASE-T/100 BASE-T standard that is used when creating a LAN (Local Area Network), an analog modem, an ISDN (Integrated Services Digital Network) modem, an ADSL (Asymmetric Digital Subscriber Line) modem for connecting to the Internet using an telephone line, a cable modem for connecting to the Internet using a cable television line, and the like, and the CPU501.

After processing data that is read from a DVD-ROM or the like by the CPU501or an image computation processor (not illustrated in the figure) of the image processor505, the image processor505stores the result in a frame memory (not illustrated in the figure) of the image processor505. The image information that is stored in the frame memory is converted to a video signal at specified synchronization timing, and outputted to the monitor511. As a result, various pages become possible to be displayed.

The audio processor506converts audio data that is read from a DVD-ROM or the like to an analog audio signal, and causes that audio signal to be outputted from the connected speaker512. Moreover, under the control of the CPU501, the audio processor506creates sounds that are supposed to be generated during processing by the information processing device500, and causes corresponding audio to be outputted from the speaker512.

A program for achieving, for example, the server100of the embodiments, is recorded on a DVD-ROM that is mounted in the DVD-ROM drive507. Under control of the CPU501, the DVD-ROM drive507performs processing to read the mounted DVD-ROM, reads the necessary program and data, and temporarily stores those in the RAM503or the like.

The external memory509, the controller510, the monitor511and the speaker512are connected to the interface508so as to be removable.

The external memory509stores data related to personal information of the user and the like so as to be rewritable.

The controller510receives operation input that is performed such as when performing various settings for the information processing device500. By inputting instructions by way of the controller510, the user of the information processing device500is able to store the data in a suitable external memory509.

The monitor511provides the user of the information processing device500with data that is outputted from the image processor505.

The speaker512provides the user of the information processing device500with audio data that is outputted from the audio processor506.

In addition, the information processing device500is potentially constructed so as to be able to use a large-capacity external storage device such as a hard disk and the like in order to achieve the same functions as the ROM502, the RAM503, the external memory509, a DVD-ROM that is mounted in the DVD-ROM drive507and the like.

In the following, the server100or the terminal200of the embodiments that are achieved by the information processing device500above will be explained with reference toFIG. 1toFIG. 28. By turning ON the power to the information processing device500, the program that causes the information processing device500to function as the server100or terminal200of the embodiments is executed, and the server100or terminal200is achieved.

FIG. 3illustrates an example of a WEB page that is provided to the terminal200by the WEB server400in the embodiments below. The WEB page600includes images601,602and603and texts604,605and606. The images601and603are images that include text, and the image602is an image that does not include text. Moreover, the image URLs of the image601,602and603are taken to be “http://xxx.601.jpg”, “http://xxx.602.jpg” and “http://xxx.603.jpg”.

(2. Basic Construction of a Server of a First Embodiment)

When character recognition has already been performed for an image when translating a character string that is included in the image, a server100of a first embodiment is able to omit the processing for acquiring the image and performing the character recognition.

As illustrated inFIG. 4, the server100of this embodiment has a request receiver101, an image processor102, and a response transmitter103.

The request receiver101receives a request, in which image ID information and a parameter for image processing are specified, from the terminal200.

The image ID information, for example, is an image URL. Moreover, image processing, for example, is a character recognition processing for recognizing characters inside an image. A parameter for image processing, for example, is a language for recognition that indicates which language the characters inside an image are in. Character recognition is performed for the characters inside the image presumed to be in the specified language for recognition.

In the following, an example is explained for the case in which a user requests translation of the image601of the WEB page600from the server100. For example, when a user places a cursor701(mouse over) over the image601that is displayed on the terminal200, a pop up801, requesting that the language for recognition be specified, is displayed as illustrated inFIG. 5A. As illustrated inFIG. 5B, when a user clicks on a button802with the cursor701, a pull-down menu is displayed. The user then selects the language (language for recognition) of the character string inside the image601from the menu. As illustrated inFIG. 5C, when “English” is specified as the language for recognition, a request, in which the image URL “http://xxx.601.jpg” of the image601and the language for recognition “English” are specified, is transmitted to the server100. The request receiver101of the server100receives that request.

In this embodiment, the CPU501and the NIC504work together to function as the request receiver101. The same is true for the following second through sixth embodiments.

The image processor102acquires an image based on the ID information that is specified in the received request, and by applying image processing using the parameter that is specified in the received request, outputs an extracted information that was extracted from inside the image.

Here, the extracted information, for example, is the character recognition result, which is a character string inside the obtained image (in the following, this will be called the “recognized character string”). For example, when a request that specifies the image URL “http://xxx.601.jpg” and the language for recognition “English” is received, the image processor102first acquires the image601from the WEB server400based on that image URL. The image processor102performs character recognition presuming the language of the characters included in the acquired image601to be English. As a result of character recognition, it is determined that the characters inside the image601are “Sale Water Best bang for the buck”, and then the image processor102outputs these character strings as recognized character strings that were extracted from inside the image601.

Moreover, after the extracted information has been extracted, the image processor102associates and caches that extracted information with the image ID information and the parameter.

For example, an extracted information table101asuch as illustrated inFIG. 6is stored in the RAM503. In the extracted information table101a, image ID information (image URLs)101a1, parameters (languages for recognition)101a2, extracted information (recognized character strings)101a3are associated and registered. The image processor102, by using character recognition, obtains the recognized character string “Sale Water Best bang for the buck”, and then associates and registers that recognized character string and the image URL of the image601“http://xxx.601.jpg” with the language for recognition “English” in the extracted information table101a.

In this embodiment, the CPU501and the image processor505work together to function as the image processor102. The same is true in the second through sixth embodiments below as well.

The response transmitter103transmits a response in which the extracted information that was outputted by the image processor102is specified to the terminal200.

For example, the response transmitter103transmits a response in which the recognized character string “Sale Water Best bang for the buck” is specified to the terminal200. Here, that response, for example, also includes a command to add a tag for overlaying and displaying an arbitrary shape to the image in the WEB page that is displayed on the terminal200for which character recognition is completed. For example, as illustrated inFIG. 7, after the terminal200receives that response, a semi-transparent rectangle901is overlaid and displayed on the image601for which character recognition is complete.

In this embodiment, the CPU501and the NIC504work together to function as the response transmitter103. The same is also true for the second through sixth embodiments below.

When the extracted information that is associated with the ID information and parameter that are specified in the received request is cached, the image processor102, instead of acquiring an image and applying image processing, obtains the cached extracted information, and outputs that obtained extracted information.

For example, the request receiver101receives a request, in which the image URL of the image601and the language for recognition “English” are specified, and the image processor102references the extracted information table101ainFIG. 6. The recognized character string “Sale Water Best bang for the buck” that corresponds to the ID information of the image601and the language for recognition “English” is registered, so that the image processor102omits the process for obtaining the image601and performing character recognition, and outputs that recognized character string to the response transmitter103.

(3. Operation of the Server of the First Embodiment)

Next, the flowchart inFIG. 8will be used to explain the operations that are performed by all of the parts of the server100of this embodiment. After the power has been turned ON to the server100, the CPU501starts the control process that is illustrated in the flowchart inFIG. 8.

The request receiver101receives the request, in which the image ID information (image URL) and the parameter (language for recognition) for image processing are specified, from the terminal200(step S101). For example, when the user places the cursor701over the image601and specifies the language for recognition “English” by following the procedure illustrated inFIGS. 5A to 5C, the request receiver101receives the request in which the image URL “http://xxx.601.jpg” and the language for recognition “English” are specified.

The image processor102then determines whether or not the extracted information (recognized character string) that is associated with the ID information and the language for recognition that are specified in the received request are cached (step S102).

When it is determined that the recognized character string is cached (step S102; YES), the image processor102outputs the recognized character string that is cached (step S103). For example, when the extracted information table101aillustrated inFIG. 6is stored in the RAM503, the image processor102outputs the recognized character string “Sale Water Best bang for the buck” that is associated with the image URL “http://xxx.601.jpg” and the language for recognition “English” to the response transmitter103.

On the other hand, when it is determined that the recognized character string is not cached (step S102; NO), the image processor102acquires the image of the image URL that is specified in the received request (step S105). For example, when information about the recognized character string that is associated with the image URL “http://xxx.601.jpg” and the language for recognition “English” is not registered in the extracted information table101a, the image processor102accesses the WEB server400and acquires the image601that corresponds to the image URL “http://xxx.601.jpg”.

After acquiring the image, the image processor102then performs character recognition using the language for recognition that was specified in the request, and extracts a character string (recognized character string) from inside the image (step S106). For example, the image processor102performs character recognition of the image601presuming that the characters inside the image601are “English”, and obtains the recognized character string “Sale Water Best bang for the buck”.

The image processor102associates and caches the recognized character string with the image ID information and the language for recognition (step S107). For example, as illustrated inFIG. 6, the image processor102associates the recognized character string “Sale Water Best bang for the buck” with the image URL “http://xxx.601.jpg” and the language for recognition “English”, and registers these in the extracted information table101a.

The image processor102then outputs the extracted recognized character string (step S108). For example, the image processor102outputs the recognized character string “Sale Water Best bang for the buck” to the response transmitter103as the result of character recognition.

The response transmitter103transmits the response, in which the recognized character string that was outputted in step S103or step S108is specified, to the transmitter200(step S104). For example, the response transmitter103transmits a response, in which the recognized character string “Sale Water Best bang for the buck” is specified, to the terminal200.

With this embodiment, when the result of character recognition for an image is already obtained, it is possible to omit the process of acquiring the image and performing character recognition. Then, by translating the character strings for which the result of character recognition has been obtained, it is possible to shorten the time required for translating the character strings inside the image.

(4. Basic Construction of a Server of a Second Embodiment)

A server100of a second embodiment is able to omit the process of acquiring an image and performing character recognition when the process of character recognition has already been performed for an area specified inside an image when translating character strings that are included in the image.

The server100of this embodiment, as in the first embodiment, has a request receiver101, an image processor102and a response transmitter103(seeFIG. 4); however, the functions and information handled by each part are different from those of the first embodiment. This will be explained in detail below.

The request receiver101receives a request, in which ID information for an image and a specified area inside that image are specified, from the terminal200.

Here, the specified area, for example, is an area inside the image of a WEB page that is specified by a user operating the terminal200. For example, the specified area is taken to be an area that is designated by a rectangular shape and specified by the coordinates of the upper left corner and lower right corner of the rectangle.

For example, as illustrated inFIG. 9A, the user uses the terminal200and selects an area902inside the image601by dragging a cursor701. Here, the coordinates of the upper left corner911of the image601(taken to be the “(vertical coordinate, horizontal coordinate)”) are set as (0, 0), and the coordinates of the lower right corner912are set as (200, 300). Moreover, the coordinates of the upper left corner of area902are taken to be (5, 100), and the coordinates of the lower right corner are (48, 200). After the area902has been specified by the user, the terminal200transmits a request, in which the image URL “http://xxx.601.jpg” of the image601, and the specified area “(5, 100):(48, 200)” are specified, to the server100. The request receiver101of the server100receives that request.

The method for specifying the specified area is not limited to that described above. For example, as illustrated inFIG. 9B, a rectangle903having a specified size with a cursor701in the center can be displayed when the cursor is placed inside the image, and the area of that rectangle903is potentially specified as the specified area. Moreover, the specified area is not limited to a rectangle, and could also be a circle and the like.

The image processor102acquires an image based on the ID information that is specified in the received request, and by performing character recognition of the specified area inside the acquired image, outputs a recognized character string that was recognized from that specified area in the image.

For example, when a request, in which the image URL “http://xxx.601.jpg” and the specified area “(5, 100):(48, 200)” are specified, is received, the image processor102first acquires the image601from the WEB server400based on the image URL. Next, the image processor102determines the language for recognition for performing character recognition. For example, when a domain code indicating a country is included in the URL of the WEB page600, the image processor102determines from that domain code the language of the characters included in the image. For example, when “.UK” is included in the domain code, the image processor102determines “English” as the language for recognition. Alternatively, the image processor102determines whether there is a tag in the html that specifies the language, and when there is such a tag, takes that language to be the language for recognition. For example, when there is a tag “<html lang=“en”>” inside the html, the image processor102takes “English” to be the language for recognition. In this embodiment, it is presumed that the image processor102has determined “English” to be the language for recognition. The image processor102performs character recognition presuming the characters that are included in the specified area “(5, 100):(48, 200)” in the acquired image601to be “English”. When the characters inside that specified area are determined to be “Sale” as a result of character recognition, the image processor102outputs this character string as the recognized character string.

After the recognized character string has been extracted, the image processor102associates the recognized character string with the image ID information and the specified area, and caches that recognized character string.

For example, as illustrated inFIG. 10, the image processor102associates the recognized character string “Sale” with the image URL “http://xxx.601.jpg” of the image601and the language for recognition “English” and the specified area “(5, 100):(48, 200)”, and registers them in the extracted information table101a.

Here, after the recognized character string has been extracted, it is possible for the image processor102to correct the specified area to a recognized area inside the specified area where the recognized character string was recognized, and then cache that recognized character string.

The recognized area, for example, is a rectangle that includes a recognized character string, and is an area that is identified by a rectangle for which lengths in the vertical direction and horizontal direction of the rectangle have been minimized. For example, the recognized area for “Sale” (area921inFIG. 11) is “(8, 110):(45, 170)”, the recognized area for “Water” (area922inFIG. 11) is “(60, 120):(90, 180)”, and the recognized area for “Best bang for the buck” (area923inFIG. 11) is “(110, 25):(190, 270)”.

For example, as illustrated inFIG. 9A, a request is received in which area902is specified as the specified area, and the specified area is specified as “(5, 100):(48, 200)”. In this case, the image processor102performs character recognition for that specified area, and after the character string “Sale” has been extracted, the image processor102obtains the recognized area in which that character string is included. Then, as illustrated inFIG. 12, the image processor102associates the recognized character string “Sale” with the image URL “http://xxx.601.jpg”, the language for recognition “English” and the specified area (recognized area) “(8, 110):(45, 170)”, and registers them in the extracted data table101a.

Moreover, as illustrated inFIG. 9C, for example, the area904that protrudes from the image601is specified as the specified area, and a request in which the specified area “(52, −20):(98, 320) is specified is received. In this case, as illustrated inFIG. 12, the image processor102associates the recognized character string “Water” with the image URL “http;//xxx.601.jpg” of the image601, the language for recognition “English” and the specified area (recognized area) “(60, 120):(90, 180)”, and registers them in the extracted information table101a.

Furthermore, as illustrated inFIG. 9D, for example, an area905the cuts through the character string inside the image601is specified as the specified area, and a request in which the specified area “(85, 15):(210, 285)” is specified is received. In this case, as illustrated inFIG. 12, the image processor102associates the recognized character string “Best bang for the buck” with the image URL “http;//xxx.601.jpg” of the image601, the language for recognition “English” and the specified area (recognized area) “(110, 25):(190, 270)”, and registers them in the extracted information table101a.

The response transmitter103transmits a response, in which the recognized character string that was outputted from the image processor102is specified, to the terminal200.

For example, after receiving a response in which the recognized character string “Sale” is specified, the terminal200, as illustrated inFIG. 13A, displays a semi-transparent rectangle931over the recognized area of the recognized character string “Sale” for which character recognition is finished. Similarly, as illustrated inFIG. 13BtoFIG. 13D, semi-transparent rectangles932,933are displayed over the recognized areas.

When the recognized character string, for which the ID information and the specified area included in the parameter that are specified in the received request are associated, is cached, the image processor102then obtains the cached recognized character string instead of obtaining an image and performing character recognition, and outputs that obtained recognized character string.

Here, when the ID information that was associated with the recognized character string when that recognized character string was cached matches the ID information that is specified in the received request, and the specified area that was associated with recognized character string when that recognized character string was cached overlaps the specified area that is specified in the received request, the image processor102determines that extracted information that is associated with the ID information and specified area that is included in the parameter that are specified in the received request is cached.

For example, as illustrated inFIG. 13A, when a user specifies area906, the request receiver101receives a request that specifies the image URL of the image601and the specified area “(5, 35):(52, 200)”. After that request has been received, the image processor102references the extracted information table101ainFIG. 12. There is a recognized character string registered in the extracted information table101athat the image URL of the image601matches and for which the specified area “(5, 35):(52, 200)” that is specified in the request overlaps the cached specified area “(8, 110):(45, 170)”. Therefore, the image processor102omits the process of acquiring the image and performing character recognition, and outputs the recognized character string “Sale” to the response transmitter103.

Determining whether or not the specified area that is specified in the request overlaps the specified area that is cached is can be appropriately set up.

For example, as illustrated inFIG. 13AandFIG. 13B, when the specified areas (areas906,907) that are specified in the request include all of the cached specified areas (recognized areas, rectangles931,932), it is determined that the areas overlap. On the other hand, as illustrated inFIG. 13C, when only part of the specified area (area908) that is specified in the request overlaps the cached specified area (recognized area, rectangle933), it is determined that the areas do not match. Alternatively, it is also possible to determine that the areas overlap when the recognized area and the specified area, which is specified in the request, overlap by only a specified ratio of the recognized area. Moreover, as illustrated inFIG. 13D, when the cursor701is placed over the recognized area (rectangle933), it is possible to determine that the areas overlap.

(5. Operation of the Server of the Second Embodiment)

Next, the operations that are performed by each of the parts of the server100of this embodiment will be explained using the flowchart inFIG. 14. After the power has been turned ON to the server100, the CPU501starts the control process that is illustrated in the flowchart inFIG. 14.

The request receiver101receives the request from the terminal200in which the image ID information and the specified area are specified (step S201). For example, when the user specifies the area902inFIG. 9A, the request receiver101receives the request in which the image URL “http;//xxx.601.jpg, and the specified area “(5, 100):(48, 200)” are specified.

The image processor102then determines whether or not extracted information (recognized character string), which is associated with the ID information (image URL) and the specified area that are specified in the received request, is cached (step S202).

When it is determined that the recognized character string is cached (step S202; YES), the image processor102outputs the cached recognized character string (step S203). For example, the image processor102references the extracted information table101athat is illustrated inFIG. 12, and determines that the image URL matches and that the specified area “(5, 100):(48, 200)” that is specified in the request and the cached specified area (recognized area) “(8, 110):(45, 170)” overlap. In this case, the image processor102outputs the recognized character string “Sale” that is associated with the image URL and the recognized area to the response transmitter103.

On the other hand, when it is determined that the recognized character string is not cached (step S202; NO), the image processor102acquires the image of the image URL that is specified in the received request (step S205). For example, when the image URL “http://xxx.601.jpg” does not match, or when the specified area that is specified in the request and the cached specified area do not overlap, the image processor102accesses the WEB server400and acquires the image601that corresponds to the image URL “http://xxx.601.jpg”.

After the image has been acquired, the image processor102determines the language for recognition, then performs character recognition using that language for recognition and extracts the recognized character string from inside the image (step S206). For example, presuming the characters in the specified area (area902) of the image601to be English, the image processor102performs character recognition of the area902and obtains the recognized character string “Sale”.

The image processor102associates and caches the recognized character string with the image ID information, the language for recognition and the specified area (step S207). For example, as illustrated inFIG. 10, the image processor102associates the recognized character string “Sale” with the image URL “http://xxx.601.jpg”, the language for recognition “English” and the specified area “(5, 100):(48, 200)”, and registers them in the extracted information table101a. It is also possible to correct the specified area to the recognized area “(8, 110):(45, 170)” before registering, and to register the corrected recognized area as illustrated inFIG. 12.

The image processor102then outputs the extracted recognized character string (step S208). For example, the image processor102outputs the recognized character string “Sale” as the result of character recognition to the response transmitter103.

The response transmitter103transmits the response, in which the recognized character string that was outputted in step S203or step S208is specified, to the terminal200(step S204). For example, the response transmitter103transmits the response, in which the recognized character string “Sale” is specified, to the transmitter200.

With this embodiment, when the result of character recognition has already been obtained for an area inside an image specified by a user, it is possible to omit the process of acquiring the image and performing character recognition. Moreover, by translating a character string that was obtained as the result of character recognition, it is possible to shorten the time required for translating a character string inside an image.

(6. Basic Construction of a Server of a Third Embodiment)

A server100of a third embodiment makes it possible to omit the process of acquiring an image, performing character recognition and performing translation when translating a character string that is included inside an image, in the case where character recognition and translation have already been performed for a specified area inside the image.

The server100of this embodiment, as in the second embodiment, includes a request receiver101, an image processor102, and a response transmitter103(FIG. 4); however, the functions of each of the parts and the information handled differ from those of the second embodiment. This will be explained in detail below.

The request receiver101receives a request, in which image ID information, a language for recognition, a specified area and conversion information are specified, from the terminal200.

The conversion information, for example, is the language (hereafter, referred to as the translation target language) after translation of a character string that is included in an image. A recognized character string that has been extracted from inside an image is translated to the translation target language using the language for recognition.

In the following, an example is explained for the case in which a user requests the server100to perform translation of the specified area (area909having the coordinates “(100, 20):(200, 280)”) inside the image601on the WEB page600. For example, when a user, using the terminal200, drags the cursor701and selects the area909inside the image601, a pop-up803, asking for the language for recognition and the translation target language, is displayed as illustrated inFIG. 15A. Then, as illustrated inFIG. 15B, when the user uses the cursor701and clicks on a button804, a pull-down menu is displayed. The user selects from the menu the language (language for recognition) of a character string inside the area909. Similarly, the user clicks on the button805for the translation target language and selects from the menu the language (translation target language) to which the character string inside the area909is to be translated. As illustrated inFIG. 15C, after the language for recognition “English” and the translation target language “Japanese” have been specified, a request, in which the image URL “http://xxx.601.jpg” of the image601, the specified area “(100, 20):(200, 280)”, the language for recognition “English” and the translation target language “Japanese” have been specified, is transmitted to the server100. The request receiver101of the server100receives that request.

The image processor102acquires an image based on the ID information that was specified in the received request, and by performing character recognition of the specified area inside the acquired image, outputs a recognized character string that was recognized from the specified area inside the image. The image processor102then, by performing conversion of the outputted recognized character string using the conversion information (translation target language) that was specified in the received request, outputs a converted character string that was converted from the recognized character string.

For example, the image processor102acquires the image601that corresponds to the image URL “http://xxx.601.jpg” that was specified in the request from the WEB server400. Next, the image processor102extracts the recognized character string “Best bang for the buck” from the specified area “(100, 20):(200, 280)” using the language for recognition “English”. The image processor102then converts that recognized character string to the translation target language “Japanese”, and outputs the converted character string “(Best bang for the buck)” to the response transmitter103.

After the converted character string has been extracted, the image processor102associates the converted character string with the image ID information, the specified area, the language for recognition, the recognized character string and the conversion information, and caches that converted character string. In this embodiment, the specified area that is cached is the recognized area.

For example, as illustrated inFIG. 16, the image processor102associates the converted character string “” with the image URL “http://xxx.601.jpg” of the image601, the language for recognition “English”, the specified area “(110, 25):(190, 270)”, the translation target language “Japanese” and the recognized character string “Best bang for the buck”, and registers them in the extracted information table101a. The information to be registered is not limited to this combination, and, for example, it is also possible to not register the translation target information and the converted character string.

When the converted character string has been cached for the ID information (image URL), the specified area, the language for recognition and the conversion information (translation target language) that are specified in the received request, the image processor102then obtains the cached converted character string instead of performing the conversion process, and outputs the obtained converted character string.

For example, the user specifies the area910(the coordinates are “(90, 0):(200, 290)”) as illustrated inFIG. 17A, and in a pop-up806, selects the language for recognition “English” and the translation target language “Japanese”. In this case, the request receiver101receives a request in which the image URL of the image601, the specified area “(90, 0):(200, 290)”, the language for recognition “English” and the translation target language “Japanese” are specified. The image processor102references the extracted information table101ainFIG. 16and determines that the image URL, the language for recognition and the translation target language that are specified in the request match the cached information for these, and determines that the specified area “(90, 0):(200, 290)” that was specified in the request, and the specified area “(110, 25):(190, 270)” that is cached overlap. Therefore, the image processor102omits the processes of acquiring the image, performing character recognition and performing conversion, and outputs the converted character string “” to the response transmitter103.

When the image URL and the language for recognition match, and the specified area that is specified in the request and the specified area that is cached overlap, the image processor102can omit only the processes of acquiring the image and performing character recognition, and use the recognized character string that is registered in the extracted information table101ato convert to the translation target language and obtain the converted character string. For example, the request receiver101receives a request in which the image URL of the image601, the specified area “(90, 0):(200, 290)”, the language for recognition “English” and the translation target language “German” are specified. In this case, a corresponding converted character string is not registered in the extracted information table101ainFIG. 16, so that the image processor102performs the conversion processing of translating the recognized character string “Best bang for the buck” to German, and outputs the converted character string.

The response transmitter103transmits a response, in which the converted character string that was outputted from the image processor102is specified, to the terminal200.

For example, when the terminal200receives the response in which the converted character string “” is specified, then as illustrated inFIG. 17C, a pop-up807, in which the converted character string “” is listed, is displayed on the WEB page600. Alternatively, as illustrated inFIG. 17D, it is possible to display a rectangle941, in which the converted character string is listed, over the recognized area.

(7. Operation of the Server of the Third Embodiment)

Next, the operations that are performed by each of the parts of the server100of this embodiment will be explained using the flowchart inFIG. 18. After the power has been turned ON to the server100, the CPU501starts the control process illustrated in the flowchart inFIG. 18.

The request receiver101receives the request from the terminal200in which the ID information (image URL) of the image, the specified area, the language for recognition, and the conversion information (translation target language) are specified (step S301). For example, the user specifies the area909inFIG. 15A, and as illustrated inFIG. 15C, by specifying the language for recognition and the translation target language in the pop-up803, the request receiver101receives a request in which the image URL “http://xxx.601.jpg”, the specified area “(100, 20):(200, 280)”, the language for recognition “English” and the translation target language “Japanese” are specified.

The image processor102then determines whether or not a converted character string, in which the ID information, the specified area, the language for recognition and that translation target language are associated, is cached (step S302).

When the image processor102determines that the converted character string is cached (step S302; YES), the image processor102outputs the converted character string that is cached (step S303). For example, the image processor102references the extracted information table101aillustrated inFIG. 16, and determines that the image URL, the language for recognition and the translation target language match, and determines that the specified area “(100, 20):(200, 280)” that was specified in the request and the specified area “(110, 25):(190, 270)” that is cached overlap. In this case, the image processor102outputs the converted character string “” that is associated with the image URL, the specified area, the language for recognition and the translation target language to the response transmitter103.

On the other hand, when it is determined that the converted character string is not cached, (step S302; NO), the image processor102determines whether or not a recognized character string that is associated with the image URL, the specified area and the language for recognitions is cached (step S305).

When it is determined that the recognized character string is cached (step S305; YES), the image processor102converts the cached recognized character string to the translation target language and obtains the converted character string (step S306). For example, when the translation target language “Japanese” and the converted character string “” are not registered in the extracted information table101ainFIG. 16, the image processor102coverts the recognized character string “Best bang for the buck” to the translation target language in “Japanese” and obtains the converted character string “”.

On the other hand, when the recognized character string is not cached (step S305; NO), the image processor102acquires the image of the image URL that is specified in the received request (step S309). Then, the image processor102performs character recognition for the specified area inside that image and extracts the recognized character string (step S310). For example, when the image URL “http://xxx.601.jpg” do not match, or when the specified area that is specified in the request and the specified area that is cached do not overlap, the image processor102accesses the WEB server400and acquires the image601that corresponds to the image URL “http://xxx.601.jpg”. The image processor102then performs character recognition for the specified area using the language for recognition “English” that is specified in the request, and extracts the recognized character string “Best bang for the buck”.

In step S310, after the recognized character string has been extracted, the image processor102performs conversion on that recognized character string (step S306).

The image processor102associates and caches the converted character string with the image ID information, the language for recognition, the translation target language, the specified area and the recognized character string (step S307). The image processor102then outputs the acquired converted character string to the response transmitter103(step S308). For example, as illustrated inFIG. 16, the image processor102associates the converted character string “” with the image URL “http://xxx.601.jpg”, the language for recognition “English”, the translation target language “Japanese”, the specified area (recognized area) “(110, 25):(190, 270)” and the recognized character string “Best bang for the buck”, and registers them in the extracted information table101a. The image processor102then outputs the converted character string “” to the response transmitter103.

Next, the image processor102transmits the response, in which the converted character string that was outputted in step S303or step S308is specified, to the terminal200(step S304). For example, the response transmitter103transmits the response, in which the converted character string “” is specified, to the terminal200.

With this embodiment, when the result of conversion (translation result) of an area inside an image that was specified by a user has already been obtained, it is possible to omit the processes of acquiring the image, performing character recognition and performing conversion. As a result, the time required for translating a character string inside an image can be shortened.

(8. Basic Construction of a Server of a Fourth Embodiment)

A server100of a fourth embodiment submits to a user whether or not processing for character recognition of an image is completed.

The server100of this embodiment, as illustrated inFIG. 19, includes a request receiver101, an image processor102, a response transmitter103, an inquiry receiver104and an answer transmitter105. The request receiver101, the image processor102and the response transmitter103of this embodiment have the same functions as in the first embodiment. In the following, the inquiry receiver104and answer transmitter105that have different functions will be explained.

The inquiry receiver104receives an inquiry from the terminal200in which image ID information is specified.

For example, as illustrated inFIG. 20A, when the user places the cursor701inside the image601(mouse over), the terminal200transmits an inquiry, in which the image URL of the image601is specified, to the server100. The inquiry receiver104of the server100receives that inquiry.

In this embodiment, the CPU501and the NIC504working together function as the inquiry receiver104. The same is also true in the fifth and sixth embodiments described later.

When the extracted information that is associated with the ID information that was specified in the received inquiry is not cashed, the answer transmitter105transmits an answer, in which a message is specified that prompts transmission of a request, in which the image ID information and the parameter of the image processing are specified, to the terminal200, and when the extracted information that is associated with the ID information that is specified in the received inquiry is cached, the answer transmitter105transmits an answer, in which that extracted information is specified, to the terminal200.

Here, the parameter of image processing is the language for recognition for character recognition, and the extracted information is a recognized character string. The extracted information table101asuch as illustrated inFIG. 6is stored in the server100. The character string “Sale Water Best bang for the buck” that corresponds to the image URL “http://xxx.601.jpg” of the image601that is specified in the request is registered in that extracted information table101a. Therefore, the answer transmitter105transmits an answer, in which the recognized character string is specified, to the terminal200. After the terminal200receives that answer, a pop-up809that includes the recognized character string is displayed on the WEB page that is displayed on the terminal200as illustrated inFIG. 20A.

When the recognized character is cached, it also possible for a message, which indicates that the recognized character string was obtained, to be specified in the answer that is transmitted. After the terminal200receives the answer, a pop-up810that includes the message indicating that the recognized character string was obtained is displayed as illustrated inFIG. 20B.

On the other hand, when the recognized character string that corresponds to the image URL of the image601is not cached in the server100, the answer transmitter105transmits an answer, in which a message is specified that prompts transmission of a request in which the image URL of the image601and the language for recognition are specified, to the terminal200. When the terminal200receives that answer, a pop-up801that asks for input of the language for recognition is displayed on the WEB page that is shown on the terminal200as illustrated inFIG. 5A.

In this embodiment, the CPU501and the NIC504work together to function as the answer transmitter105. The same is also true in the fifth and sixth embodiment that are described later.

(9. Operation of the Server of the Fourth Embodiment)

Next, the operations that are performed by each of the parts of the server100of this embodiment will be explained using the flowchart inFIG. 21. After the power has been turned ON to the server100, the CPU501starts the control process that is illustrated in the flowchart inFIG. 21.

The inquiry receiver104receives the inquiry from the terminal200in which image ID information is specified (step S401). For example, the inquiry receiver104receives an inquiry in which the image URL of the image601is specified.

The answer transmitter105determines whether or not the recognized character string that is associated with the ID information that was specified in the inquiry is cached (step S402).

When it is determined that the recognized character string is cached (step S402; YES), the answer transmitter105transmits an answer, in which the cached recognized character string is specified, to the terminal200(step S403). For example, when the recognized character string “Sale Water Best bang for the buck” that is associated with the image URL of the image601is registered in the extracted information table101a(FIG. 6) that is stored in the server100, the answer transmitter105transmits an answer, in which that character string is specified, to the terminal200.

On the other hand, when it is determined that the recognized character string is not cached (step S402; NO), the answer transmitter105transmits an answer, in which a message is specified prompting for specification of the language for recognition, to the terminal200(step S404). For example, when the recognized character string that corresponds to the image URL of the image601is not registered in the extracted information table101athat is stored in the server100, the answer transmitter105transmits an answer, in which a message is specified prompting transmission of a request in which the image URL of image601and the language for recognition are specified, to the terminal200.

After that, when the request, in which the image URL of the image601and the language for recognition are specified, is received from the terminal200, the server100starts, for example, the control process illustrated in the flowchart inFIG. 8.

With this embodiment, by simply placing a mouse over an image or the like, it is possible to notify a user whether or not processing for character recognition of that image has been completed.

(10. Basic Construction of a Server of a Fifth Embodiment)

A server100of a fifth embodiment provides a user with whether or not processing for character recognition of an area inside an image has been completed.

The server100of this embodiment, as illustrated inFIG. 19, includes a request receiver101, an image processor102, a response transmitter103, an inquiry receiver104and an answer transmitter105. The request receiver101, the image processor102and the response transmitter103have the same functions as in the second embodiment. In the following, the inquiry receiver104and the answer transmitter105that have different functions will be explained.

The inquiry receiver104receives an inquiry, in which image ID information and a position inside that image are specified, from the terminal200.

For example, as illustrated inFIG. 22A, when a user places the cursor701inside the image601(mouse over), the terminal200transmits an inquiry, in which the image URL of the image601and the coordinates (“(75:175)”) of the position of the cursor701are specified, to the server100. The inquiry receiver104of the server100receives that inquiry.

When a recognized character string, which is associated with the ID information that is specified in the received inquiry, and with the specified area that includes the position that is specified in the received inquiry, is cached by the image processor102, the answer transmitter105transmits an answer, in which the cached recognized character string is specified, to the terminal200.

For example, it is presumed that the extracted information table101aillustrated inFIG. 12is stored in the server100. As illustrated inFIG. 22A, the coordinates “(75:175)” of the position that was specified in the request is included in the recognized area922. The recognized character string “Water”, which corresponds to the image URL “http://xxx.601.jpg” of the image601that was specified in the request and the specified area (recognized area922) that includes the coordinates of the position that was specified in the request, is registered in that extracted information table101a. Therefore, the answer transmitter105transmits an answer, in which the recognized character string is specified, to the terminal200. When the terminal200receives that answer, a pop-up811that includes the recognized character string “Water” is displayed on the WEB page that is shown on the terminal200as illustrated inFIG. 22A.

It is also possible to specify a message prompting transmission of a request in which the translation target language is specified in the answer that is transmitted when the recognized character string is cached. After the terminal200receives the answer, a pop-up812requesting that the translation target language be specified is displayed as illustrated inFIG. 22B.

On the other hand, when the recognized character string, which corresponds to the image URL of the image601, and to the specified area in which the specified position is included, is not cached in the server100, the answer transmitter105transmits an answer, in which a message is specified prompting transmission of a request in which the image URL of the image601, the position inside the image and the language for recognition are specified for that specified area, to the terminal200. When the terminal200receives that answer, a pop-up813requesting input of the language for recognition is displayed on the WEB page that is shown on the terminal200as illustrated inFIG. 22C.

(11. Operation of the Server of the Fifth Embodiment)

Next, the operations that are performed by each of the parts of the server100of this embodiment will be explained using the flowchart inFIG. 23. After the power has been turned ON to the server100, the CPU501starts the control process that is illustrated in the flowchart inFIG. 23.

The inquiry receiver104receives an inquiry from the terminal200in which image ID information and a position inside that image are specified (step S501). For example, the inquiry receiver104receives an inquiry in which the image URL of the image601and the position “(75:175)” are specified.

The answer transmitter105determines whether or not a recognized character string, which is associated with the ID information and the specified area that includes the position that were specified in the inquiry, is cached (step S502).

When it is determined that the recognized character string is cached (step S502; YES), the answer transmitter105transmits an answer, in which the cached recognized character string is specified, to the terminal200(step S503). For example, when the recognized character string “Water” that corresponds to the image URL of the image601and the specified area that includes the specified position “(75:175)” is registered in the extracted information table101athat is stored in the server100(FIG. 12), the answer transmitter105transmits an answer, in which that recognized character string is specified, to the terminal200.

On the other hand, when it is determined that the recognized character string is not cached (step S502; NO), the answer transmitter105transmits an answer, in which a message is specified prompting specification of the language for recognition for the specified area of that image, to the terminal200(step S504). For example, when a recognized character string that corresponds to the image URL of the image601and the specified area that includes the specified position is not registered in the extracted information table101athat is stored in the server100, the answer transmitter105transmits an inquiry, in which a message is specified that prompts that a request in which the image URL of the image601, the specified area and the language for recognition are specified be transmitted, to the terminal200.

With this embodiment, by simply placing the mouse over an area inside an image or the like, it is possible to notify a user whether or not character recognition has been completed for that area.

(12. Basic Construction of a Server of a Sixth Embodiment)

A server100of a sixth embodiment provides a user with whether or not a conversion processing has been completed for an area inside an image.

The server100of this embodiment, as illustrated inFIG. 19, includes a request receiver101, and a image processor102, a response transmitter103, an inquiry receiver104and an answer transmitter105. The request receiver101, the image processor102and the response transmitter103have the same functions as in the third embodiment. In the following, the inquiry receiver104and the answer transmitter105that have different functions will be explained.

The inquiry receiver104receives an inquiry from the terminal200in which image ID information, a position inside that image and conversion information are specified.

For example, as illustrated inFIG. 22A, when a user places the cursor701inside the image601, the pop-up812is displayed as illustrated inFIG. 22Basking that the translation target language (conversion information) be specified. After the user selects an arbitrary translation target language (for example, Spanish), the terminal200transmits an inquiry, in which the image URL of the image601, the coordinates “(75:175)” of the cursor701position and the translation target language “Spanish” are specified, to the sever100. The inquiry receiver104of the server100receives that inquiry.

When the converted character string that is associated with the ID information and the conversion information that are specified in the received inquiry, and that is associated with a specified area that includes the position that is specified in the received inquiry is cached by the image processor102, the answer transmitter105transmits an answer in which the cached converted character string is specified.

For example, it is presumed that an extracted information table101aas illustrated inFIG. 24is stored in the server100. The image URL “http://xxx.601.jpg” of the image601that was specified in the request, the specified area (recognized area922) that includes the coordinates of the position that was specified in the request, and the converted character string “Agua” that corresponds to the translation target language “Spanish” are registered in the extracted information table101a. Therefore, the answer transmitter105transmits an answer, in which the converted character string is specified, to the terminal200. When the terminal200receives that answer, a pop-up814that includes the converted character string “Agua” is displayed on the WEB page that is shown on the terminal200as illustrated inFIG. 25A.

On the other hand, when the converted character string, which corresponds to the image URL of the image601, the specified area that includes the specified position, and the translation target language, is not cached in the server100, the answer transmitter105transmits a response, in which a message that indicates that the converted character string is not cached is specified, to the terminal200. When the terminal200receives that answer, a pop-up815that includes the message indicating that the converted character string is not cached is displayed on the WEB page that is shown on the terminal such as illustrated inFIG. 25B.

(13. Operation of the Server of the Sixth Embodiment)

Next, the operations that are performed by each part of the server100of this embodiment will be explained using the flowchart inFIG. 26. After the power has been turned ON to the server100, the CPU501starts the control process that is illustrated in the flowchart inFIG. 26.

The inquiry receiver104receives the inquiry, in which the image ID information, the position inside that image, and the conversion information are specified, from the terminal200(step S601). For example, the inquiry receiver104receives the inquiry in which the image URL of the image601, the position “(75:175)”, and the translation target information “Spanish” are specified.

The answer transmitter105determines whether or not the converted character string, which is associated with the ID information, the specified area that includes the position and the translation target language that were specified in the inquiry, is cached (step S602).

When it is determined that the converted character string is cashed (step S602; YES), the answer transmitter105transmits the answer, in which the cached converted character string is specified, to the terminal200(step S603). For example, when the converted character string “Agua”, which corresponds to the image URL of the image601, the specified area that includes the specified position “(75:175)” and the translation target language “Spanish”, is registered in the extracted information table101athat is stored in the server100(FIG. 24), the answer transmitter105transmits an answer, in which that converted character string is specified, to the terminal200.

On the other hand, when it is determined that the converted character string is not cached (step S602; NO), the answer transmitter105transmits the answer, in which a message is specified indicating that the converted character string is not cached, to the terminal200(step S604). For example, when the converted character string, which corresponds to the image URL of the image601, the specified area that includes the specified position and the translation target language “Spanish”, is not registered in the extracted information table101athat is stored in the server100, the answer transmitter105transmits the answer, in which the message is specified indicating that the converted character string is not cached, to the terminal200.

In this embodiment, it is also possible for the inquiry receiver104to receive an inquiry in which the image URL and the position are specified, and for the answer transmitter105to transmit an answer in which the converted character string, which corresponds to that image URL and the specified area that includes the position, is specified.

For example, when an inquiry is received in which the image URL of the image601and the specified position “(75:175)” are specified, the answer transmitter105references the extracted information table101ainFIG. 24, and acquires the converted character strings “(Water)”, “Agua”, “L'eau” that correspond to the image URL and the specified areas that includes the position. The answer transmitter105then transmits the answer, in which all of these converted character strings are specified, to the terminal200. After the terminal200receives that answer, a pop-up816such as illustrated inFIG. 25Cis displayed on the terminal200. Alternatively, the answer transmitter105can select the converted character string that corresponds to the translation target language that has been specified with the highest frequency up to that point in a request, and transmit an answer in which that converted character string is specified. Moreover, it is also possible for the answer transmitter105to alternatively select at random from the converted character strings that are registered in the extracted information table101a, and transmit an answer in which that converted character string is specified.

With this embodiment, by simply placing the mouse over an area inside an image or the like, it is possible to notify a user whether or not conversion processing (translation process) has been completed for that area.

(14. Basic Construction of a Terminal of a Seventh Embodiment)

A program of a seventh embodiment functions so that results of character recognition or conversion processing that was performed by a server are displayed on a terminal that is capable of communicating with the server.

As illustrated inFIG. 27, the terminal200on which the program of this embodiment operates includes a display201, a position detector202, an inquiry transmitter203, an answer receiver204, an area detector205, a setter206, a request transmitter207and a response receiver208.

The display201displays on a screen an image that was identified according to ID information.

For example, the display201, as illustrated inFIG. 22A, displays the image601that is specified by the image URL “http://xxx.601.jpg”.

In this embodiment, the CPU501and the image processor505work together to function as the display201.

The position detector202detects a selection of a position inside the image that is displayed on the screen.

For example, when the cursor701is placed inside the image, the position detector202, as illustrated inFIG. 22A, detects the coordinates of the position of that cursor701.

In this embodiment, the CPU501and the image processor505work together to function as the position detector202.

After the selection of the position has been detected, the inquiry transmitter203transmits an inquiry, in which the ID information and the selected position are specified, to the server100.

For example, in the case illustrated inFIG. 22A, the inquiry transmitter203transmits an inquiry, in which the image URL of the image601and the coordinates “(75:175)” of the position are specified, to the server100.

In this embodiment, the CPU501and the NIC504work together to function as the inquiry transmitter203.

The response receiver204receives a response from the server100. The display201then further displays the extracted information that is specified in the received response on the screen.

For example, the answer receiver204receives an answer in which the recognized character string “Water” is specified. In this case, as illustrated inFIG. 22A, the display201displays the pop-up811that includes that recognized character string.

In this embodiment, the CPU501and the NIC504work together to function as the answer receiver204.

The area detector205detects a selection of an area inside the image that is displayed on the screen.

For example, as illustrated inFIG. 9A, when the cursor701is dragged inside the image, the area detector205detects the coordinates of the dragged area (area902).

In this embodiment, the CPU501and the image processor505work together to function as the area detector205.

The setter206prompts setting of parameters for image processing.

For example, the setter206prompts setting of the language for recognition that is used in character recognition.

In this embodiment, the CPU501functions as the setter206.

After the selection of the area has been detected, the request transmitter207transmits a request, in which the ID information, the set parameter and the selected area are specified, to the server100.

For example, as illustrated inFIG. 9A, when the area is specified and the coordinates of that area are detected, the request transmitter207transmits a request, in which the image URL “http://xxx.601.jpg”, the language for recognition “English” and the coordinates “(5, 100):(48, 200)” of the selected area are specified, to the server100.

In this embodiment, the CPU501and the NIC504work together to function as the request transmitter207.

The response receiver208receives a response from the server100. The display201then further displays on the screen extracted information that is specified in the received response.

For example, the response receiver208receives a response in which the recognized character string “Sale” is specified. In this case, the display201, as illustrated inFIG. 13Afor example, further displays the semi-transparent rectangle931over the recognized character string “Sale”.

In this embodiment, the CPU501and the NIC504work together to function as the response receiver208.

(15. Operation of the Terminal of the Seventh Embodiment)

Next, the operations of each of the parts of the terminal200of this embodiment will be explained using the flowchart inFIG. 28. After the power has been turned ON to the terminal200, the CPU501starts the control process illustrated inFIG. 28.

The display201displays an image on the screen that was identified from ID information (step S701). The position detector202determines whether or not the selection of the position inside the image that is displayed on the screen has been detected (step S702).

For example, the display201displays the image601for which the image URL “http://xxx.601.jpg” is specified, and when the cursor701is placed over that image601, the position detector202detects the coordinates “(75:175)” of the cursor701(FIG. 22A).

When the position detector202determines that selection of the position has been detected (step S702; YES), the inquiry transmitter203transmits an inquiry, in which ID information and the selected position are specified, to the server100(step S703). The answer receiver204then receives an answer from the server100that corresponds to the inquiry (step S704). The display201further displays on the screen extracted information that is specified by the received answer (step S705).

For example, the inquiry transmitter203transmits an inquiry, in which the image URL of the image601and the coordinates “(75:175)” of the position are specified, to the server100. The answer receiver204then receives an answer in which the recognized character string “Water”, which was obtained by character recognition for the area that includes the coordinates of the position, is specified. After receiving that answer, the display201, as illustrated inFIG. 22A, displays the pop-up811in which the recognized character string is included.

On the other hand, when the position detector202did not determine that the selection of the position was detected (step S702; NO), the area detector205determines whether or not the selection of the area inside the image that is displayed on the screen was detected (step S706).

For example, when the user drags the cursor701inside the image as illustrated inFIG. 9A, the area detector205detects the coordinates “(5, 100):(48, 200)” of the area (area902) where the cursor701was dragged.

When the area detector205determines that the selection of the area was detected (step S706; YES), the setter206prompts setting the parameter for image processing (step S707). The request transmitter207transmits a request, in which the ID information, the set parameter and the selected area are specified, to the server100(step S708). The response receiver208receives a response from the server100that corresponds to the request (step S709). The display201further displays on the screen the extracted information that is specified in the received response (step S710).

For example, the setter206prompts setting of the language for recognition that will be used in character recognition, and after the language for recognition “English” has been set, the request transmitter207transmits a request, in which the image URL of the image601, the language for recognition “English” and the coordinates “(5, 100):(48, 200)” of the area are specified, to the server100. The response receiver208then receives a response in which the recognized character string “Sale”, which was obtained by character recognition for an area that overlaps with the area, is specified. After that response has been received, the display201, as illustrated inFIG. 13Afor example, displays the semi-transparent rectangle931over the recognized character string “Sale”.

On the other hand, when the area detector205did not determine that the selection of the area was detected (step S706; NO), or when processing of steps S705and S710is complete, processing returns to step S701.

In this embodiment, it is also possible to further specify area information, which indicates from which area of an image that extracted information was extracted, in the response that is received by the answer receiver204. Moreover, the program of this embodiment can be such that image ID information, extracted information and area information are associated and cached in the terminal200. In this case, when the user places the mouse over an area that is included in the cached area information for an image having the same ID information as that image, the display201displays the extracted information that corresponds to that area information on the screen.

Moreover, it is possible for the response that is received by the response receiver208to specify area information that indicates from which area of an image that extracted information was extracted. The program of this embodiment can cause the image ID information, the extracted information and the area information to be associated and cached in the terminal200. In this case, when the user selects the area of the image that has the same ID information of that image so as to overlap the area indicated in the cached area information, the display201displays on the screen the extracted information that corresponds to that area information. Determining whether or not the areas overlap can be performed, for example, in the same way as in the second embodiment.

Moreover, the program of this embodiment uses various script languages that are executed by a browser.

With this embodiment, it is possible to have the results of character recognition processing and conversion processing that are performed by a server be displayed on a terminal that is capable of communicating with the server, and it is possible to have characters of an image on a WEB page be displayed in real-time on the terminal.

The embodiments described above are for explanation only and do not limit the range of the present invention. Moreover, the present invention is not limited to being applied to specific technical specifications such as illustrated in the embodiments described above. Therefore, it is possible for a person who is skilled in the art to apply embodiments in which each element or all of the elements described in this specification have been replaced with equivalent elements, and those embodiments are also included within the range of the present invention.

The present invention is based on Japanese Patent Application No. 2011-101534 filed on Apr. 28, 2011. The entire specification, claims and drawings of Japanese Patent Application No. 2011-101534 are incorporated in this specification by reference.

INDUSTRIAL APPLICABILITY

With the present invention, it is possible to provide a server, a server control method, a program and a recording medium that are suitable for translating characters in an image on a WEB page in real-time.

REFERENCE SIGNS LIST