However, some highly optimization load balancing algorithms do be very complex and take much CPU time. If the parallelable code won't take long, use a sub-optimized algorithm or even give up parallelization will be better. Besides CPU time, parallelization introduces communications. When a segment of code is divided and distributed, every computation node owns only parts of data and results thus it may need to exchange data with other nodes. Communications are needed to distribute data and collect results.
Communications are expensive since they usually take long.
Workstation clusters usually use Ethernet for communication. Even for parallel computers which equip with special high speed communication channels, the communications are still far slower than computing. Parallel code with much communications may run slower than sequential code. Communications also imply synchronizations. Synchronization means idle and wait.