Some realistic architectures. Clearly the PRAM is an unrealizable machine, since it does not take communication between each processor and all shared memory locations into account. It can provide useful insight into possible lower bounds. Once we depart from the theoretical PRAM model, we are really in the domain of computer architecture. For a real understanding of this field, it is therefore necessary to consider many architectural issues.
Related questions lead into the fields of design of parallel languages and operating systems.
Some common architectures often used in parallel processing are shown in Figure P1. Typically each processor can pass through the data from the preceding stage or switch the inputs and output them in the opposite order. Clearly each of the architectures pictured can be viewed as a graph. The greater the degree, the faster processors can communicate, but the harder it will be to actually build the machine. This measures reliability and fault tolerance of the network.