Patent Description:
A technique has been developed to modify and update electronic data by reading or writing on a sheet of paper or the like based on the electronic data output by using an edge device connected to a cloud service.

For example, a user registered in the cloud service sets rules for writing on the paper when printing out electronic data through the cloud service with the printer and scanner connected to the cloud service. As a result, writing on the paper is reflected in the original electronic data by scanning the paper with the scanner.

In addition, since the edge device can be linked to each other through the cloud service, even when the printer and scanner are different devices, the electronic data of content to be printed is available on the cloud service, and the original electronic data is updated.

For example, in <CIT>, a technique is disclosed that reads the written content on the paper and modifies the original electronic data according to a pattern of content written on the paper.

However, with the technique disclosed in <CIT>, the written content may not be reflected on the original electronic data, or in the case the original electronic data is modified, modification may not be reflected to the added content as the user intended.

According to the present disclosure, by using the cloud service linked with the edge device, the writing on the paper is reflected on the original electronic data managed by the cloud service.

<CIT> discloses a document administration system including an administration unit to administer administration information of an image formed on a recording medium in connection with discrimination information of the recording medium and a decoloring device.

<CIT> discloses a transient document management system that can include a transient document identifier placement module for placing a transient document identifier on a transient physical document, a transient document information retrieval module for extracting information from the transient physical document, and a transient document processing module for performing data processing based on the information extracted from the transient physical document.

According to at least one embodiment, by using the cloud linked with the edge device, the writing on the paper is reflected on the original data stored by the user on the cloud.

Hereinafter, a description is given of several embodiments of an information processing apparatus, an information processing system, and an information processing method.

<FIG> is a block diagram illustrating an example of a functional configuration of the information processing system according to the present embodiment. With reference to <FIG>, an example of the functional configuration of the information processing system according to the present embodiment is described.

As illustrated in <FIG>, the information processing system according to the present embodiment includes a cloud <NUM> and an edge device <NUM>. In the present embodiment, the cloud <NUM> may include a cloud gateway. Further, in the present embodiment, the edge device <NUM> may be, for example, a personal computer (PC), a smartphone, a digital camera, a printer, a scanner, or a multifunction peripheral (MFP).

An example of a functional configuration of the edge device <NUM> is described in the following. The edge device <NUM> includes a communication unit <NUM> and a job execution unit <NUM>.

The communication unit <NUM> communicates with an external apparatus such as the cloud <NUM>. The job execution unit <NUM> interprets and executes a job received from the cloud <NUM> by the communication unit <NUM>. A job executed by the job execution unit <NUM> differs for each edge device <NUM>.

An example of the functional configuration of the cloud <NUM> (an example of the information processing apparatus) is described in the following. The cloud <NUM> includes a communication unit <NUM>, an optical character recognition (OCR) processing unit <NUM>, a user interface (UI) unit <NUM>, a writing processing rule registration unit <NUM>, a writing processing unit <NUM>, a data editing unit <NUM>, a data storage unit <NUM>, a data history management unit <NUM>, a user authentication unit <NUM>, an edge device control unit <NUM>, a job reception unit <NUM>, and a job management unit <NUM>.

The communication unit <NUM> is an example of a communication unit that communicates with an external apparatus such as the edge device <NUM>.

The OCR processing unit <NUM> executes OCR processing of a document. Specifically, the OCR processing unit <NUM> is an example of a conversion unit that executes OCR processing on data input from the edge device <NUM> through the communication unit <NUM>. The UI unit <NUM> displays UIs related to various services provided by the cloud <NUM>.

The writing processing rule registration unit <NUM> is an example of a settings reception unit that receives settings of a writing processing rule (an example of a data update rule) for data. Further, the writing processing rule registration unit <NUM> stores the writing processing rule in association with the user, the edge device <NUM>, or the like that has set the writing processing rule. The writing processing unit <NUM> is an example of a calculation unit that detects a modification, a conversion method, and the like from the data for which OCR processing is executed according to the writing processing rule registered by the writing processing rule registration unit <NUM>. Further, the writing processing unit <NUM> detects the modification based on the data history stored by the data history management unit <NUM> described below. Accordingly, the modification in the data is detected easily.

The data editing unit <NUM> reflects the modification and conversion method to the data history stored by the data history management unit <NUM> described below and the target electronic data, which is the original data input by the user, among the data stored in the data storage unit <NUM> described below. Specifically, the data editing unit <NUM> determines the target electronic data, which is the original data input from the edge device <NUM> among the data stored in the data storage unit <NUM> described below, based on the data history stored by the data history management unit <NUM> described below. The data editing unit <NUM> is an example of an update unit that updates the determined target electronic data based on the modification and the conversion method detected by the writing processing unit <NUM>.

By using the data history, the data input from the edge device <NUM> such as the data printed on paper is associated with the target electronic data stored in the data storage unit <NUM>. As a result, by using the cloud <NUM> linked with the edge device <NUM>, the writing on the paper is reflected on the original data stored by the user in the cloud <NUM>.

For example, the data editing unit <NUM> uses Robotic Process Automation (RPA) or the like as a method for updating data. In such a case, the data editing unit <NUM> generates a document correction command based on the detected modification and the target electronic data, and executes the document correction command by the RPA to update the target electronic data.

The data storage unit <NUM> is an example of a storage unit that stores the data output to the edge device <NUM> and the data (target electronic data) updated by the data editing unit <NUM>. The data history management unit <NUM> stores the data history, which is the usage history of the data input to and output from the cloud <NUM>. In the present embodiment, the data history management unit <NUM> stores the data history in association with the user who used the data, the edge device <NUM>, and the like.

The user authentication unit <NUM> is an example of an authentication unit that authenticates a user before transmitting a job to the edge device <NUM>. The edge device control unit <NUM> is an example of a device control unit that controls the edge device <NUM> communicably connected by the communication unit <NUM>.

The job reception unit <NUM> receives data to be output to the edge device <NUM> from the user authenticated by the user authentication unit <NUM>. In the present embodiment, the job reception unit <NUM> receives a job including a user ID, data to be output to the edge device <NUM>, a security level selected by the user, and the like. The job management unit <NUM> manages the jobs received by the job reception unit <NUM>.

<FIG> is a flowchart illustrating an example of an operation process initiated by a user of the information processing system according to the present embodiment. With reference to <FIG>, the example of the operation process initiated by the user of the information processing system according to the present embodiment is described.

In step S211, the user authentication unit <NUM> of the cloud <NUM> executes a login process including authentication of the user of the cloud <NUM>. In step S212, the job reception unit <NUM> of the cloud <NUM> receives a selection of print data to be output to the edge device <NUM> (for example, a printer). The user may select print data from a cloud folder linked with the cloud <NUM>, or may upload the print data to the cloud <NUM>.

In step S213, the writing processing rule registration unit <NUM> of the cloud <NUM> receives settings of the writing processing rule for the print data. In step S214, the UI unit <NUM> receives a selection of a printer for printing the print data among the edge devices <NUM>. In the present embodiment, any of the edge devices <NUM> linked with the cloud <NUM> may be selected. In one example, the UI unit <NUM> receives the selection of the edge device <NUM> from the user. In another example, the UI unit <NUM> automatically selects the edge device <NUM>. Then, the communication unit <NUM> transmits a job including the print data to the selected printer.

In step S215, the job execution unit <NUM> of the edge device <NUM> (printer) prints the print data according to the job received from the cloud <NUM>. In step S216, the user writes on a recording medium such as paper on which print data is printed, and in response to a selection of the edge device <NUM> for reading the paper or the like (for example, a scanner), the selected edge device <NUM> scans the paper with writing or the like in step S217.

Then, the communication unit <NUM> of the edge device <NUM> (scanner) transmits the scanned data to the cloud <NUM>.

In response to input of data from the edge device <NUM> to the communication unit <NUM> of the cloud <NUM>, the data editing unit <NUM> determines whether to update the original data (target electronic data) stored in the data storage unit <NUM> (target electronic data), in step S218. Based on a determination not to update the original data (NO in step S218), the data editing unit <NUM> assigns a storage name designated by the UI unit <NUM> to the received data in step S219 and stores the data in the data storage unit <NUM>.

On the other hand, when updating the original data (YES in step S218), the data editing unit <NUM> selects (determines) the original data (target electronic data) from the data stored in the data storage unit <NUM> in step S220. Then, the data editing unit <NUM> updates the selected original data, assigns the storage name designated by the UI unit <NUM> to the updated original data in step S219, and stores the updated original data in the data storage unit <NUM>.

<FIG> is a flowchart illustrating an example of a determination process of the target electronic data in the information processing system according to the present embodiment. With reference to <FIG>, an example of the determination process of the target electronic data in the information processing system according to the present embodiment is described.

In step S301, the data editing unit <NUM> determines whether a data history associated with the user or the edge device <NUM> is stored in the data history management unit <NUM>. Based on a determination that the data history is not stored (NO in step S301), the data editing unit <NUM> waits for the target electronic data to be designated by the user in step S302. In step S303, the data editing unit <NUM> determines the data designated by the user as the target electronic data (data printed by the edge device <NUM>).

On the other hand, based on a determination that the data history is stored (YES in step S301), the data editing unit <NUM> determines whether a plurality of the data histories is stored in step S304. Based on a determination that the plurality of data histories is not stored (NO in step S304), the data editing unit <NUM> determines the target electronic data from the data stored in the data storage unit <NUM> based on the data history in step S303.

In step S305, based on the determination that the plurality of data histories is stored (YES in step S304), the data editing unit <NUM> displays candidates of the original data (target electronic data) among the data stored in the data storage unit <NUM> based on the plurality of the data histories. In step S306, the data editing unit <NUM> waits for the target electronic data to be designated from the candidates of the original data by the user. In step S303, the data editing unit <NUM> determines the candidate of the original data designated by the user as the target electronic data.

<FIG> is a flowchart illustrating an example of an update process of the target electronic data in the information processing system according to the present embodiment. With reference to <FIG>, the example of the update process of the target electronic data in the information processing system according to the present embodiment is described.

In step S401, the OCR processing unit <NUM> receives data (writing data) input from the edge device <NUM> through the communication unit <NUM> and executes the OCR processing on the received writing data. In step S402, the writing processing unit <NUM> extracts writing from the writing data in which the OCR processing is executed. In step S403, the writing processing unit <NUM> detects a modification and conversion method (data correction method) from the extracted writing according to the writing processing rule.

In step S404, the data editing unit <NUM> determines whether there is the modification in the target electronic data based on the detected modification and conversion method. Based on a determination that there is no modification in the target electronic data (NO in step S404), the data editing unit <NUM> reflects the detected modification and conversion method in the target electronic data in step S405.

On the other hand, based on a determination that there is the modification in the target electronic data (YES in step S404), in other words, in the case there is a competitive situation in which the target electronic data is modified since the target electronic data is printed, the data editing unit <NUM> determines whether there is the modification (correction target) of the target electronic data at the same position as the detected modification in step S406. Based on a determination that there is the modification of the target electronic data at the same position as the detected modification (YES in step S406), the data editing unit <NUM> displays the difference between the detected modification and the modification of the target electronic data in the UI unit <NUM> in step S407, and waits for an instruction from the user in step S408. The UI unit <NUM> is an example of a notification unit that requests the user to instruct the correction method in the case the difference between the detected modification and the target electronic data is found.

In step S405, the data editing unit <NUM> reflects the detected modification and conversion method in the target electronic data according to the instruction from the user. In other words, in the case the difference between the detected modification and the target electronic data is found, the data editing unit <NUM> updates the target electronic data in response to the instruction from the user. Accordingly, the data is updated as intended by the user.

Based on a determination that there is no modification in the target electronic data at the same position as the detected modification (NO in step S406), the data editing unit <NUM> determines whether there is a difference (correction target) from the detected modification in the target electronic data in step S409, and based on a determination that there is a correction target in the target electronic data (YES in step S409), the data editing unit <NUM> displays the correction target in the target electronic data in the UI unit <NUM> in step S410, and waits for an instruction from the user in step S411. In step S405, the data editing unit <NUM> reflects the detected modification and conversion method in the target electronic data according to the instruction from the user.

Based on a determination that there is no correction target in the target electronic data (NO in step S409), the data editing unit <NUM> leaves the detected modification and conversion method in a log as unreflected writing in step S412.

<FIG> are diagrams illustrating examples of an operation screen at the time of printing in the information processing system according to the present embodiment. With reference to <FIG>, the examples of the operation screen displayed at the time of printing in the information processing system according to the present embodiment is described.

In the present embodiment, the operation screen is displayed on an external device such as a smartphone or a PC by using a browser or an application. For example, when printing data on the edge device <NUM>, the writing processing rule registration unit <NUM> displays on the external device, the operation screen for setting a device to be used for printing data (for example, printer A) and the writing processing rule (for example, deletion and addition), as illustrated in <FIG>. Then, the writing processing rule registration unit <NUM> receives the writing processing rule set on the operation screen.

Here, the writing processing rule may be selectable from the writing processing rules provided by the service in advance, or the user may draw a figure or the like on the operation screen and set the writing processing rule. Further, the writing processing rule may include in addition to delete, add, and modification, various writing processing rules such as copy, paste, font size modification, and commands to be operated on the data. Further, a writing processing rule preset by the user may be included as the writing processing rule. Further, as illustrated in <FIG>, one user may set a plurality of writing processing rules, and a name (for example, user X) may be assigned to each writing processing rule to enable use of the writing processing rule.

<FIG> are diagrams illustrating other examples of the operation screen at the time of scanning in the information processing system according to the present embodiment. With reference to <FIG>, the examples of the operation screen displayed at the time of scanning in the information processing system according to the present embodiment is described.

When scanning paper (recording medium) with writing in the edge device <NUM>, the data editing unit <NUM> displays on the external device the operation screen for selecting the device (for example, a scanner B) used for scanning the paper with writing and for selecting whether to update the target electronic data according to the writing on the paper. In response to a selection to update the target electronic data according to the writing, the data editing unit <NUM> may display on the external device, the operation screen for setting the method of storing the target electronic data updated according to the writing. As illustrated in <FIG>, the data editing unit <NUM> displays an operation screen for assigning a file name (for example, ZZ_after. txt) to store the target electronic data updated according to the writing as a new file in response to a determination of the target electronic data based on the user's data history.

The cloud service may store the target electronic data before the update based on the user information, but the cloud <NUM> may not store the usage history of the output data from the viewpoint of security or the like. In this case, the data editing unit <NUM> may display on the external device, an operation screen for selecting the target electronic data corresponding to the data to be scanned by the edge device <NUM> from the data stored in the data storage unit <NUM>, as illustrated in <FIG>. In this case, the file name for the target electronic data is not assigned because the writing is reflected by overwriting the selected target electronic data.

<FIG> are diagrams illustrating examples of the operation screen displayed in response to a reflection of the writing in the information processing system according to the present embodiment. With reference to <FIG>, the examples of the operation screen displayed at the time of reflecting the writing in the information processing system according to the present embodiment is described.

In the case a competitive situation in which the target electronic data is modified after the target electronic data is printed is found after detecting the modification and correction method for the data (scan data) input from the edge device <NUM>, the data editing unit <NUM> displays the operation screen illustrated in <FIG> on the external device. Specifically, as illustrated in <FIG>, the data editing unit <NUM> displays the operation screen indicating that the competitive situation has occurred in the target electronic data and enabling selection whether to reflect the modification in the target electronic data.

In response to a selection by the user to reflect the modification in the target electronic data, the data editing unit <NUM> displays a preview window for identifying the modification in the target electronic data as illustrated in <FIG>. Alternatively, the data editing unit <NUM> may display a preview window capable of comparing the target electronic data and the scanned data. In this case, a marker or the like may be attached to the modification of the target electronic data to identify the modification. Further, as illustrated in <FIG>, the data editing unit <NUM> may include a button on the operation screen for selecting whether to modify the target electronic data.

In response to a selection not to modify the target electronic data and in the case there is no other modification in the target electronic data, the data editing unit <NUM> displays an operation screen indicating that the update of the target electronic data has been completed on the external device as illustrated in <FIG>.

On the other hand, in response to a selection to modify the target electronic data, the data editing unit <NUM> updates the target electronic data according to the detected modification and conversion method. In response to a selection by the user to make manual correction, the data editing unit <NUM> displays on the external device, an operation screen for inputting the modification and conversion method of the target electronic data as illustrated in <FIG>. Then, the data editing unit <NUM> updates the target electronic data according to the modification and conversion method input on the operation screen. Then, the data editing unit <NUM> displays an operation screen indicating that the update of the target electronic data has been completed on the external device as illustrated in <FIG>.

As described above, according to the information processing system of the present embodiment, the data input from the edge device <NUM> such as the data printed on paper and the target electronic data are stored in association with each other in the data storage unit <NUM> by using the data history. As a result, by using the cloud <NUM> linked with the edge device <NUM>, the writing on the paper is reflected on the original data stored by the user in the cloud <NUM>.

The apparatuses or devices described in one or more embodiments are just one example of plural computing environments that implement the one or more embodiments disclosed herein. In some embodiments, the cloud <NUM> includes multiple computing devices such as a server cluster. The plurality of computing devices is configured to communicate with one another through any type of communication link, including a network, shared memory, etc., and perform the processes disclosed herein. Similarly, the edge device <NUM> may include multiple computing devices configured to communicate with one another.

Claim 1:
An information processing apparatus (<NUM>) comprising:
a communication unit (<NUM>) for communicating with an edge device (<NUM>) communicably connected to the information processing apparatus (<NUM>);
a conversion unit (<NUM>) for executing OCR processing on data input from the edge device (<NUM>) through the communication unit (<NUM>);
a device control unit (<NUM>) for controlling the edge device;
an authentication unit (<NUM>) for authenticating a user;
a job reception unit (<NUM>) for receiving selection of data to be output to the edge device;
a storage unit (<NUM>) for storing the data output to the edge device;
a settings reception unit (<NUM>) for receiving setting of data update rule from the user and storing said rule in association with said user;
a calculation unit (<NUM>) for detecting a modification from the data obtained by executing OCR processing based on a modification and conversion method specified in said data update rule;
a data history management unit (<NUM>) for storing a data history that indicates usage history of data input and output; and
an update unit (<NUM>) for determining original data input from the edge device (<NUM>) among the stored data as target electronic data based on the data history and for updating the target electronic data based on said modification and conversion method specified in the data update rule.