Regular and irregular communication patterns. Ease of programming , particularly for systems in which communication patterns are complex or vary dynamically during execution. Explicit communication forces programmers and compiler writers to pay attention to what is costly and what is not. Shared-memory communication is the much more popular today. Centralized schemes still dominate, but long term trends favor distributing memory.
Any program has a parallel portion and a serial portion.
The parallel portion is the only part that is sped up by having multiple processors. As with uniprocessors, speedup is limited by the fraction of the original program that can be parallelized. For example, suppose we want to achieve a speedup of 80 with 100 processors. The second major challenge involves the large latency of remote memory access . The type of interconnection network.