Mediation server, terminals and distributed processing method

A highly convenient data processing technique is provided.A mediation server includes: a project information acquiring unit which acquires information on a project from a project server and stores the acquired information in a project database, the project server managing the project in which a computational processing is divided into a plurality of tasks so as to have the plurality of tasks processed by a plurality of terminals; a project information transmitter which presents the information on the project to a terminal and which transmits, to the terminal, information necessary for the terminal to access a project server of a project that the terminal desires to participate in; and an execution status management unit which receives a notification indicating that the terminal having accessed the project server using the information has executed a task assigned from the project server and which stores an execution status in a terminal database.

TECHNICAL FIELD

The present invention relates to a distributed processing technology and more particularly to a mediation server, which mediates between a server for managing a project and a plurality of terminals for distributing and processing tasks contained in the project, the terminals and a distributed processing method.

BACKGROUND TECHNOLOGY

Known is a distributed processing technology where a plurality of computers are linked together via a network so as to virtually construct a high-performance computer. In the conventional practice, a project, such as gene analysis and protein structure analysis, which requires a large-scale computation can only be handled by a very costly and large computer which is yet to be developed. However, the construction of a distributed processing system enables the effective utilization of unused resources and also enables executing the large computation processing at very low cost.

DISCLOSURE OF THE INVENTION

Problems to be Solved by the Invention

However, under present circumstances, there is a small fraction of users who would agree to the content of grid computing and offer their own computer resources. A majority of people are unaware of the presence of the grid computing. In recent years, although a large-scale grid computing is ready for its achievement thanks to a well-built infrastructure of networks and a significantly growing rate of computer ownership, few resources are effectively used in fact.

The present invention has been made under the foregoing circumstances, and a general purpose thereof is to provide a distributed processing technology capable of effectively utilizing resources.

Means for Solving the Problems

One embodiment of the present invention relates to a mediation server. This mediation server comprises: a project information acquiring unit which acquires information on a project from a project server and stores the acquired information in a project database, the project server managing the project in which a computational processing is divided into a plurality of tasks and the plurality of tasks are processed by a plurality of terminals; a project information transmitter which presents the information on the project to a terminal and which transmits, to the terminal, information necessary for the terminal to access a project server of a project that the terminal desires to participate in; and an execution status management unit which receives a notification indicating that the terminal having accessed the project server using the information has executed a task assigned from the project server and which stores an execution status in a terminal database.

Another embodiment of the present invention relates to a terminal. This terminal comprises: a project information acquiring unit which acquires information necessary for accessing a project server that manages a project desired to be participated in, from a mediation server that mediates between the project server and a plurality of terminals, the project server managing the project in which a computational processing is divided into a plurality of tasks and the plurality of tasks are processed by the plurality of terminals; a task acquiring unit which acquires the task by accessing the project server using the information; a task execution unit which executes the task; an execution result transmitter which transmits an execution result of the task to the project server; and an execution status notification unit which conveys to the mediation server that the task has been executed.

Optional combinations of the aforementioned constituting elements, and implementations of the invention in the form of methods, apparatuses, systems and so forth may also be practiced as additional modes of the present invention.

Effect of the Invention

According to the present invention, a distributed processing technology by which resources can be effectively utilized can be provided.

DESCRIPTION OF THE REFERENCE NUMERALS

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 1shows a structure of a distributed processing system10according to an embodiment. The distributed processing system10includes a project server14which manages a project involving a large-scale computation with a certain purpose, a plurality of terminals200which process computations included in the project in a distributed manner, and a mediation server100which mediates between the project server14and the terminals200. These devices are connected to one other via the Internet12which is an example of networks.

In order that the computations included in the project can be processed by a plurality of terminals200in a distributed manner, executable programs and data files storing data or operational parameters used by the programs are prepared in each of processors of the terminals200. Hereinafter, these will be called tasks. As will be described later, if a terminal200includes a plurality of processors, a plurality of tasks may be allocated to the terminal200. In this case, a plurality of tasks will be called a task set. The mediation server100provides support for the mediation server100to efficiently distribute tasks, the execution of which is requested by the project server14, to the terminals200.

FIG. 2is a sequence diagram schematically showing a flow of processing in the distributed processing system10. A general flow is first described here and a detailed description of processing will be given later. The project server14transmits information on a project to the mediation server100(S10). The mediation server100registers the information on the project received from the project server14, in a project database (S12). The project server14further transmits information on a task requested to be executed by the terminals200, to the mediation server100(S14). The mediation server100registers execution request information received from the project server14, in the project database (S16).

The terminals200transmits information on the terminals200to the mediation server100(S18). The information on the terminals200may include, for example, the type, clock frequency, number, or utilization rate of a processor mounted on a terminal200, amount or remaining amount of memory, amount of tasks executable by the terminals200at the time, condition of a project that a user wishes to take part in, and so forth. Based on the information on the terminals received from the terminals200, the mediation server100selects the candidates of projects to be introduced to the terminals200(S20) and transmits them to the terminals (S22). When the user of the terminal200selects a desired project from among the candidates presented (S23) and conveys it to the mediation server100(S24), the mediation server100determines the amount of tasks distributed to the terminals200(S25) and transmits information with which to access the project server14and information on tasks to be executed, to the terminals200(S26). The information with which to access the project server14may be a URL of the project server14, a project ID and the like. The mediation server100informs the project server14of the fact that the tasks have been distributed to the terminals200(S27).

The terminal200accesses the project server14based on the information received from the mediation server100and requests the downloading of a task to be executed (S28). The project server14selects tasks executed by the terminals200and encrypts them so as to be sent to the terminals200(S29). The terminals200decode the tasks acquired from the project server14and executes them (S30). If, among the tasks to be executed by the terminals200, programs are the same as those which the terminals200have already obtained from the project server14and data and/or parameters differ therefrom, the project server14may not transmit the programs to the terminals200but transmit the data and/or parameters only to the terminals200. When the terminals200have completed the execution of tasks, the terminals200appends their signatures to the execution results and transmit them to the project server14(S32) and, conveys the fact that the executions have been completed, to the mediation server100(S34).

The mediation server100registers the execution status of tasks by the terminals200in the terminal databases (S36). The mediation server100gives a compensation to the user according to the amount of tasks executed by the terminal200(S38). For reference used for execution plans of a project in the project server14, statistical information, such as a distribution status of tasks to the terminals200or an execution status of tasks by the terminals200, is conveyed to the project server14(S40).

In this manner, the mediation server100mediates interaction between the project server14and the terminals200and thereby efficiently utilizes resources not in use, so that it can facilitate the project. Since the mediation server100introduces the information on the project to the terminals200, the users of the terminals200can be widely notified of the information on the project and therefore the participation in the project can be facilitated. The mediation server100according to the present embodiment does not mediate the transmission and reception of data such as tasks or execution results, so that traffic increase or congestion can be suppressed.

FIG. 3illustrates a structure of the mediation server100according to the present embodiment. The mediation server100is comprised of a communication unit110, a control unit120, and a storage unit140. The communication unit110controls communications with other apparatuses connected via a network. The control unit120includes a project information acquiring unit121, a task execution request acquiring unit122, a terminal information acquiring unit123, a projection selector124, a project information transmitter125, a task distributor126, a task information transmitter127, a distribution status notification unit128, an execution status management unit129, a compensation provider130, and a statistical information notification unit131. The storage unit140contains a project database142, a terminal database144, and a history database146.

In terms of hardware components, these structures can be realized by a CPU and a memory of an arbitrary computer, a memory-loaded program and the like, but drawn herein are function blocks that are realized in cooperation with those. Thus, it is understood by those skilled in the art that these function blocks can be realized in a variety of forms such as by hardware only, software only or the combination thereof.

The project database142stores information on projects managed by the project server14.FIG. 4shows an example of internal data in the project database142. Provided in the project database142are an project ID column150, a project name column151, a project server URL column152, a project contents column153, a request date/time column154, a time limit column155, a processor column156, a memory column157, a requested task quantity column158, a distribution-completed task quantity column159, and an execution-completed task quantity column160.

The project ID column150stores project IDs managed by the project server14. The project name column151stores the names of projects. The project server URL column152stores URLs used when the terminals200access the project server14. The project contents column153stores information on the project contents. The request date/time column154stores dates and times when the project server14sent task execution requests to the mediation server100. The time limit column155stores time limits within which the execution of the tasks that the project server14has requested of the mediation server100is to be completed. The processor column156stores conditions of processors required for the terminals200executing the tasks included in the project. The memory column157stores conditions on memories of the terminals200. The requested task quantity column158stores the quantity of tasks that the project server14has required of the mediation server100. The distribution-completed task quantity column159stores the quantity of tasks, which the mediation server100has distributed to the terminals200, among those which the project server14has required of the mediation server100. The execution-completed task quantity column160stores the quantity of tasks, the execution of which the terminals200have completed, among those which the mediation server100has distributed to the terminals200. The quantity of tasks may be indicated by the amount of program codes, file size or the like. It may also be indicated by time required for the execution thereof.

The terminal database144stores information on the terminal200that distributes the project and processes it.FIG. 5shows an example of internal data in the terminal database144. Provided in the terminal database144are a terminal ID column165, a user ID column166, a processor column167, a memory column168, a condition column185, an in-distribution project column169, a distribution date/time column170, a time limit column171, an in-distribution task quantity column172, an execution status column173, a total distribution amount column174, a total execution amount column175, and a compensation column176.

The terminal ID column165stores the IDs of the terminals200. The user ID column166stores the user IDs of the terminals200. The processor column167stores information such as the type or the clock frequency of a processor mounted on a terminal200. The memory column168stores information such as an amount of memory mounted on a terminal200. The condition column185stores a condition of a project that the user wishes to take part in. The in-distribution project ID column169stores the project IDs of tasks which are currently distributed to the terminals200. The distribution date/time column170stores dates and times when the tasks were distributed to the terminals200. The time limit column171stores the time limits of tasks distributed to the terminals200. The in-distribution task quantity column172stores the quantity of tasks distributed to the terminals200. The execution status column173stores information on the execution status of tasks distributed to the terminals200. The information on the execution status may be, for example, the quantity of tasks, which have already been completed, and so forth. The total distribution amount column174stores the total quantity of tasks which have been distributed to the terminals200. The total execution amount column175stores the total quantity of tasks executed by the terminals200. The compensation column176stores the compensations provided to the users for the execution of the tasks.

The history database146stores information on the history of tasks distributed to the terminals200.FIG. 6shows an example of internal data in the history database146. Provided in the history database146are a history ID column177, a terminal ID column178, a user ID column179, a project ID column180, a distribution date/time column181, a time limit column182, a completion date/time column, and a task quantity column184. The history ID column177stores IDs of history information. The terminal ID column178stores IDs of terminals200to which the tasks have been distributed. The user ID column179stores the user IDs of the terminals200. The project ID column180stores the project ID of tasks distributed to the terminals200. The distribution date/time column181stores dates and times when the tasks were distributed to the terminals200. The time limit column182stores the time limits of tasks distributed to the terminals200. The completion date/time column183stores dates and times when the terminals200have completed the tasks. The task quantity column184stores the quantity of tasks executed by the terminals200.

The project information acquiring unit121acquires information on projects from the project server14via the Internet20. The information on projects may include project names, URLs, purposes of projects, sentences, images, moving images or the like for explaining the contents thereof, and so forth. When the project information acquiring unit121receives the information on a project from the project server14, it registers the received information in the project database142.

The task execution request acquiring unit122acquires an execution request of tasks from the project server14via the Internet20. At this time, the task execution request acquiring unit122acquires information that includes the quantity and time limit of tasks to be executed, conditions on processors and memories and the like required for the execution, and registers it in the project database142.

The terminal information acquiring unit123acquires information on the terminals200from the terminals200via the Internet20. The information on the terminals200may include the user IDs of the terminals200and information on the processors, memories or the like mounted thereon. When the terminal information acquiring unit43receives the information on the terminals200from the terminals200, it registers the received information in the terminal database144.

When the terminals200participate in a project and are going to execute the tasks, the terminal information acquiring unit123receives conditions on the project that the users of the terminals200desire to participate in and information on resources on the terminals200. The information on resources of the terminals200may include the utilization rates of processors, free space of memory and so forth. When the user of a terminal200is thinking of taking part in a project, the user may access the mediation server100. Or the terminal200may be provided with a structure such that when the resource has an unused space and there is a room for the execution of a task, access is automatically made to the mediation server100. Or the terminal200may be provided with a structure such that information on the resource is automatically sent to the mediation server at a predetermined timing.

Based on the information on the terminals200acquired by the terminal information acquiring unit43, the project selector124selects the candidates of projects to be introduced to the terminals200by referring to the project database142. The project selector124selects a project that agrees with the condition of a project desired to be participated in by referring to the project contents column153of the project database142. The project selector124reads out the conditions on a processor and memory required for the execution of a task from the processor column156and the memory column157of the selected project, and checks if the terminal200meets the condition by referring to the information acquired from the processor column167and the memory column168of the terminal database144and the terminals200.

The project information transmitter125presents information on the candidates of projects selected by the project selector124, to the terminals200and inquires of the terminals200about any of which projects they are to participate in. When the project information transmitter125receives a project that the terminals200desire to participate in, it transmits information necessary for the terminals200to access, such as the URL of the project server14and the project ID, to the terminals200.

When the task distributor126receives the decision about a project to be participated in, it determines the quantity of tasks distributed to the terminals200. The task distributor126first checks the remaining quantity of tasks requested from the project server14by referring to the project database142. Then, based on the information on the unused resources of the terminals200acquired from the terminals200, the task distributor126determines the quantity of tasks distributed to the terminals200within a range that does not exceed the remaining quantity of tasks. The task distributor126checks the execution rate by referring to the total distribution amount column174and the total execution amount column175of the terminal database144, and may restrict the quantity of tasks distributed to a terminal200whose execution rate is low. If the execution of tasks which have already been distributed to a terminal200has not yet been completed, the task distributor126may stop the distribution of tasks to said terminal200. When the task distributor126determines the quantity of tasks distributed to the terminals200, it adds up the distributed quantity of tasks in the distribution-completed task quantity column159of the project database142and, at the same time, registers the information in the in-distribution project ID column169, the distribution date/time column170, the time limit column171and the in-distribution task quantity column172.

The task distributor126may manage the terminals having the similar type of processing circuits or storages, as a group. Thereby, the amount of resources required for the execution of tasks, time taken for the execution of tasks and the like can be estimated easily and accurately, so that the tasks can be distributed efficiently. As will be described later, if all of the terminals200to which the mediation server100distributes the tasks have the same hardware configuration, the time taken for the execution of tasks and the like can further easily and accurately be estimated and therefore the tasks can be distributed with accuracy.

The task information transmitter127transmits the quantity of tasks that the terminals200are to execute, to the terminals200. The quantity of tasks may be expressed as time required to execute tasks or the amount of processing circuits or storages. The distribution status notification unit128conveys to the project server14the fact that the tasks have been distributed to the terminals200. At this time, it may notify the quantity of tasks distributed to the terminals200. If the mediation server100does not distribute the tasks and leaves it to the project server14, the task distributor126and the information transmitter127may not be provided at all.

The execution status management unit129receives the execution status of tasks from the terminals200which have accessed the project server14, using the information that the project information transmitter125has sent to the terminals200. The execution status management unit129checks the execution status of unfinished tasks whose time limit is about to expire, by referring to the terminal database144, and redistributes the tasks to the other terminals200if necessary. When the execution status management unit129receives from the terminals200a request indicating the redistribution of tasks to the other terminals200, it redistributes the tasks, which had been distributed but has not been finished, to the other terminals. When the execution status management unit129receives from a terminal200the notification indicating that a task has been completed, it adds up the quantity of the completed task in the execution-completed task quantity column160of the project database142, and initializes the in-distribution project ID column169, distribution date/time column170, time limit column171, in-distribution task quantity column172and the execution status column173of the terminal database144so as to sum up the quantity of completed tasks in the total execution amount column175. A new record is added to the history database164, and information on the completed tasks is registered.

The compensation provider130acquired the quantity of executed tasks from the terminals200and provides compensation for the execution of the tasks, based on the quantity of tasks. The quantity of tasks may be expressed as time taken for the execution of tasks or the amount of processing circuits or storage units. The compensation provider130calculates the compensations to be given to the users, based on the quantity of tasks executed by the terminals200, and adds them to the compensation column176of the terminal database144.

The statistical information notification unit131conveys the information, on the status of execution by the terminals, which is acquired by the execution status management unit129and stored in the history database146, to the project server14. Thereby, the manager of a project can grasp the distribution status and execution status of tasks, which can help design a project plan and the like.

FIG. 7illustrates a structure of a terminal200according to an embodiment. The terminal200is comprised of a communication unit210and a control unit220. The communication unit210controls communications between the terminal200and the other terminals via a network. The control unit220includes a terminal information transmitter221, a project information acquiring unit222, a task information acquiring unit223, a task acquiring unit224, a task execution unit226, an execution status notification unit227, and an execution result transmitter228. These structures can be realized in a variety of forms such as by hardware only, software only or the combination thereof.

The terminal information transmitter221transmits the information on terminals200to the mediation server100. As described above, the information on terminals200may include the user IDs of the terminals200, information on processors, memories or the like mounted thereon, the utilization rates of processors, free space of memory and so forth. When the user of a terminal200intends to participate in a project, the terminal information transmitter221may transmit the terminal information to the mediation server100. Or the terminal information transmitter221may automatically transmit the terminal information when the resource of its own terminal200has an unused space and there is a room for the execution of a task. Or the terminal information transmitter221may automatically transmit the terminal information at a predetermined timing.

The project information acquiring unit222receives the candidates of projects presented from the mediation server100and acquires information with which to access the project server14of the project that the user has selected from among the candidates.

The task information acquiring unit223acquires information on tasks distributed from the mediation server100. The information on tasks may include the time limits within which the execution of the tasks is to be completed, the quantity of tasks, the amount of processing circuits to be used for the execution of the tasks and so forth.

The task acquiring unit224accesses the project server14using the information acquired by the project information acquiring unit222so as to acquire encrypted tasks. When the project information acquiring unit222acquires a project ID from the mediation server100, the task acquiring unit224may access the project server14by executing a program by which to participate in the project corresponding to the acquired project ID. This program may be downloaded from a not-shown download server. In such a case, the URL of the project server14is contained in the downloaded program. When the project information acquiring unit222acquires the URL of the project server14from the mediation server100, the task acquiring unit224acquires the tasks by accessing the acquired URL. The tasks may be encrypted using a public key encryption or common key encryption. That is, what is required here is that tasks are encrypted so that any third party cannot decrypt them. Unauthorized duplication of a task may be restricted using methods other than the encryption.

The task execution unit226executes the tasks acquired by the task acquiring unit224. The task execution unit226includes a decoder229and a signature unit230. The decoder229decodes the encrypted tasks which have been acquired by the task acquiring unit224. If the tasks are encrypted using a public key, the decoder229will decode them using a secret key stored in secret in the terminals200. If the tasks are encrypted using a common key, the decoder229will decode them using the common key acquired in secrecy from the project server14. As a result, the third parties' attempt to acquire and execute the tasks to obtain information on a project is prevented, so that the interests of the manager of a project can be protected.

The signature unit230attaches the signature of the terminal200to the execution results of the tasks. Electronic signature may be attached to the execution results by encrypting the execution results using the secret key of the terminal200. This can verify the execution results by the terminals200to which the tasks have been distributed. As a result, the transmission of unauthorized execution results to the project server14by a third party pretending to be a terminal200can be prevented and thus the interests of the manager of a project can be protected.

The execution status notification unit227conveys the execution status of tasks to the mediation server100. The execution status notification unit227may notify the mediation server100of the execution status of tasks periodically at predetermined time intervals or it may notify the mediation server100of the execution status when the tasks have been completed up to a predetermined ratio. If it is determined based on the remaining quantity of tasks, memory usage and the like that the tasks cannot be completed by the time limit within which the execution of the tasks is to be completed, the execution status notification unit227may request the mediation server100to redistribute the tasks. When the execution of tasks has been completed, the execution status notification unit227informs the mediation server100of the fact that the execution thereof has been completed. The execution result transmitter228transmits the execution results of tasks to the project server14.

FIG. 8shows a hardware structure of a terminal200. The terminal200includes a microprocessor unit (MPU)22, a graphics processing unit (GPU)40, a main memory42, an auxiliary storage device (HDD)44, and a network control unit46. The respective components are connected via a main bus38. The network control unit46transmits and receives data to and from other devices via a network.

The MPU22is an asymmetric multiprocessor unit and it has an input/output unit24which is an example of a management unit and a plurality of processing units30which are an example of a processing unit. The input/output unit24is a unit which inputs and outputs data from and to other structures, and it includes a processor26and a local memory28. The local memory28is a cache memory, for instance. Each of the processing units30is a unit which executes a task independently, and each of them includes a processor32and a local memory34. A program, data, operation parameters and the like read out from the main memory42are written to the local memory34and are executed by the processor32.

The input/output unit24transmits and receives data to and from the other structural components in the terminal200such as the GPU40, the main memory42, the HDD44and the network control unit46via the main bus38. It also transmits and receives data to and from other devices via the network control unit46. Although in the present embodiment the processing unit30can directly transmit and receive data to and from the other processing units30, the input/output unit24, the GPU40and the main memory42, it cannot directly transmit and receive data to and from the other devices via the network control unit46. The processing unit30transmits and receives data to and from the other devices via the input/output unit24.

In another embodiment, a structure may be such that the processing unit30directly transmits and receives data to and from the other devices. Also, the MPU22may be a symmetric multiprocessor unit. In such a case, any of the processing units30may carry out a function of the input/output unit24. Or all of the processing units30may directly transmit and receive data to and from the other devices.

The tasks distributed to the terminals200are executed by at least part of a plurality of processing units30, under control of a process management function executed in the input/output unit24. The input/output unit24selects processing units30, which are not in use, from among a plurality of processing units30and has them execute the tasks.

The input/output unit24has the processing units30execute the tasks wherein the amount of processing units30is specified to the task information acquiring unit223from the mediation server100. Suppose, for example, that it is specified from the mediation server100that three processing units30are to be used and that the task acquiring unit224has acquired a task set including three tasks which are designed on the assumption that each task can be processed by one processing unit30. Then, the input/output unit24instructs three processing units30to execute the respective tasks. The input/output unit24may select processing units30that execute the tasks, based on the time limit of the tasks that the task information acquiring unit223has acquired from the mediation server100and the usage of a plurality of processing units30. For example, if the task set including three tasks is acquired and there is some time to spare till the time limit, the number of processing units30in use simultaneously may be reduced and a free space may be created in a manner that the three tasks are executed in series by a single processing unit30. If there is not much time left till the time limit, the three tasks may be executed in parallel by three processing units30.

The process management function may be executed by the respective processing units30. In such a case, tasks contained in the task set are added to a task cue provided by the use of part of the main memory42. And when said processing unit30becomes capable of executing other tasks, a process management function in each processing unit30acquires a task awaiting execution from the task cue and then executes it. In this manner, the tasks are executed by the processing units30. Thus the tasks given to the terminals200from the project server14are preferably designed as a program processed by the processing units30.

The task execution unit226may have a function of storing a partial result when the processing unit30suspends the execution of a task and reading out the thus stored partial result when the execution of a task is resumed. For example, even in such a case where another process having a higher degree of priority needs to be executed, the execution of a task is suspended and will be resumed later. Thus the tasks can be switched without wasting the calculation done so far and the resources of the processing units30can be flexibly and effectively utilized.

In this manner, the chance for the terminals200to participate in a project can be increased by the mediation server100informing the terminals200of a project. As a result, the execution of the project can be accelerated.

The present invention has been described based on the exemplary embodiments. These embodiments are merely exemplary, and it is understood by those skilled in the art that various modifications to the combination of each component and each process thereof are possible and that such modifications are also within the scope of the present invention.

INDUSTRIAL APPLICABILITY

The present invention can be used for a distributed processing system.