If necessary, restart from step 2 after selecting a new model. If you do not achieve some level of speed-up by using parallel processing, either you did not use the right model or the problem is too sequential to benefit from parallel processing. Decomposing a Serial Program When approaching the task of writing a parallel program, the most common situation is that a serial version of the program already exists. The first step is to determine whether you will use data decomposition or functional decomposition. If the program is complex, you might do both.
Many programs can benefit from data decomposition.
Data decomposition is often easier to perform. Attempting to parallelize such a program using only the distributed paradigm would require a lot of processor idle time as the other nodes wait for the one doing the bulk of the work to complete. A program such as this is more likely to benefit from data decomposition than it is from functional decomposition. When doing data decomposition, strive for large-grain parallelization. For example, suppose most of the work of a program is being performed in a single loop.