Abstract. The performance of parallel applications running on large clusters is known to degrade due to the interference of kernel and daemon activities on individual nodes, often referred to as noise. In this paper, we focus on an important class of parallel applications, which repeatedly perform computation, followed by a collective operation such as a barrier. We model this theoretically and demonstrate, in a rigorous way, the effect of noise on the scalability of such applications. We study three natural and important classes of noise distributions: The exponential distribution, the heavy-tailed distribution, and the Bernoulli distribution. We show that the systems scale well in the presence of exponential noise, but the performance goes down drastically in the presence of heavy-tailed or Bernoulli noise.
In REF , a research work that models the effect of OS Jitter over the scalability of parallel applications is presented, which shows that in presence of OS Jitter with exponential distribution, the system under study showed the expected scalability.
The Impact of Noise on the Scaling of Collectives: A Theoretical Approach
HiPC
Computer Science