Parallel Processing on networked heterogeneous computers has been widely adopted.
Load balancing is an issue essential for taking the advantage of parallel processing.
Previously reported load balancing techniques assume that there is only one parallel job running on a given set of computers. When many parallel jobs need to be executed on the same set of computers, a high level central manager is usually involved to enforce the queuing and balancing policy. Such a policy usually allows one parallel job running on the reserved computer to ensure load balancing. These arrangements discourage the sharing of hardware resources among computer owners and can be used only on computers that are dedicated for the parallel processing. In this paper, we introduce a new load balancing method that does not need a high-level central manager.