SIMD refers to a parallel execution model in which all processors execute the same operation at the same time, but each processor is allowed to operate upon its own data. This model naturally fits the concept of performing the same operation on every element of an array, and is thus often associated with vector or array manipulation. Because all operations are inherently synchronized, interactions among SIMD processors tend to be easily and efficiently implemented. MIMD refers to a parallel execution model in which each processor is essentially acting independently. This is a more flexible model than SIMD execution, but it is achieved at the risk of debugging nightmares called race conditions , in which a program may intermittently fail due to timing variations reordering the operations of one processor relative to those of another.
High bandwidth allows large blocks of data to be transferred efficiently between processors.
The latency of a communication system is the minimum time taken to transmit one object, including any send and receive software overhead. Latency is very important in parallel processing because it determines the minimum useful grain size , the minimum run time for a segment of code to yield speed-up through parallel execution. Message Passing Message passing is a model for interactions between processors within a parallel system. In general, a message is constructed by software on one processor and is sent through an interconnection network to another processor, which then must accept and act upon the message contents. Thus, message passing can yield high bandwidth making it a very effective way to transmit a large block of data from one processor to another.