Patent ID: 7093004

Claim:
In a distributed computing system, a method for operating a broker that manages the execution of active jobs, each with at least one task to be assigned or completed, the method comprising: using the broker to maintain, for each active job, a list of associated: (i) waiting tasks that have not yet been assigned to a processing resource; (ii) pending tasks that have been assigned to at least one processing resource, but have not yet been completed by any processing resource; and (iii) completed tasks that have been completed by at least one processing resource; using the broker to maintain a priority for each active job; in response to an indication from a processing resource that the resource is available to accept tasks, using the broker to (i) select a job and (ii) assign at least one task associated with the selected job to the processing resource; in response to an indication from a processing resource that a task has completed its execution, using the broker to (i) identify the job with which the completed task is associated and (ii) update the identified job's associated task lists; using the broker to periodically compute statistical information regarding waiting, pending, and/or completed tasks, wherein said computed statistical information includes at least (i) a mean time-to-completion for completed tasks associated with each active job and (ii) a standard deviation of time-to-completion for completed tasks associated with each active job; using the broker to identify, based on at least some of said computed statistical information, at least one pending task for redundant assignment to an additional processing resource; and, adding said identified pending task to the waiting task list for the job with which said identified pending task is associated.