In recent years, as the price of commodity personal computers has fallen, these, special purpose, parallel machines have, for the most part, ceased to be manufactured. As distributed clusters of machines . Here each node is an autonomous computer with its own memory, operating system, and disk space. Furthermore, the nodes may be similar or different from each other. For example, ATS's PC cluster includes 300 MHz Intel II machines as well as 450 MHz Intel II machines.
Parallel programming is performed in distributed clusters of machines via message passing .
Message passing is the process of sending data from a program running on one node to a program running on another of the nodes. SPMD A loosely synchronous parallel programming model in which different nodes execute the same program on different sets of data. An asynchronous parallel programming model in which different nodes execute different programs on different sets of data. Today, machines with 2 or more processors are becoming common. While you can make use of the multiple processors in these machines by running separate programs on each of the processors simultaneously, you can also run in parallel by running a single program across the processors.