This typically refers to the ratio of the number of bytes received by a process to the number of floating point operations it does. Increasing the granularity will speed up the application, but the tradeoff is a reduction in the available parallelism. The total number of messages sent is also a consideration. As a rule of thumb, sending a small number of large messages takes less time than sending a large number of small messages. This is not always the case, though.
Some applications can overlap sending small messages with other computation.
The optimal number of messages is application specific. Some applications are suited to functional parallelism, others to data parallelism. In functional parallelism, different machines do different tasks based on their specific capabilities. For example, a supercomputer may solve part of a problem suited to vectorization, a multiprocessor may solve another part suited to parallelization, and a graphics workstation may be used to visualize the data in real time. In data parallelism, the data is distributed to all of the tasks in the virtual machine.