Patent ID: 9596298
Date: 2017-03-14
CPC Classifications: G06F,H04L

Claim:
1. A method implemented by data processing apparatus, the method comprising: obtaining, by a distributed processing system that includes a plurality of data processors, data units to be processed by the distributed processing system, each data unit having a corresponding timestamp that indicates a time the data unit was created, and wherein the time the data unit was created is prior to a time that the data unit is processed by the data processors; distributing the data units among the data processors for processing by the data processors; monitoring, for each data processor in the distributed processing system, a rate of cursor progress for the data processor based on the timestamps of the data units that have been processed by the data processor, wherein the rate of cursor progress for each data processor specifies a rate of change of an oldest timestamp of an oldest data unit being processed by the data processor based on the rate of change of the times indicated by the respective timestamps of each oldest data unit processed by the data processor; determining a progress threshold for the distributed processing system based on the respective rates of cursor progress processed by the plurality of data processors; determining, based on a first rate of cursor progress for a first data processor processed by the plurality of data processors, that the first rate of cursor progress for the first data processor does not meet the progress threshold; and performing one or more load balancing operations on the distributed processing system in response to determining that the first rate of cursor progress for the first data processor does not meet the progress threshold.