The small chunks of data that require similar processing are assigned to different processors. The same set of instructions are executed on each processor but the data sets are different. Each processor can perform the same calculations on a different set of data. This type of decomposition can be implemented using the master-slave model or the hostless programming model. The hostless programming model has each processor doing the same calculations but on different data.
Communication is fundamental to coordinating and integrating the separate pieces of work.
It is used to assign the tasks, transfer and exchange data, and synchronize the program and data. Special libraries of routines and languages have been developed to provide communications. Some of these are PVM, Linda, MPL, MPI, Express, and Craft. Parallel programming is just in its infancy and new and better tools, libraries, and language compilers will be developed as time goes on.