At the other end you have various machines communicating in all directions among themselves using vast bandwidths. It s not so easy to produce an example of this, because in general it does not happen much. There is also a wide range of choice about the interconnect method used. Not only is there variation in bandwidth but in reliability. This only mentions two dimensions of variability, but is sufficient to make the point that there isn t much reason to think that theory, description, and modeling usefully cover the whole space unchanged.
Reliability applies to computers too, of course, and to their operating systems.
It is evident that if a calculation is done by a vast number of machines the chance that something will fail is thereby increased. So doing a job in a distributed manner makes it intrinsically more likely that undesired events will happen, through failure of communication, operating system, or computer. It is even the case that there are complete impossibilities, such as determining in the case where no response is received to a remote request whether the request was lost on the way out, or whether the response was lost on the way back. One might perhaps wonder why distributed computing is considered a good thing. One reason is that it presents challenges to computer science.