# Performance of Communication- and Computation-Intensive SaaS on the OpenStack Cloud
## 4 Results and Discussion
### Computational Load  
In general, the computational load can be estimated by the number of particles or contacts between neighboring particles. Contacts can rapidly change during computation, while the number of particles remains constant. Thus, the number of particles is a slightly less accurate but more convenient preliminary measure of computational load. Figure 2 shows the number of particles processed by a varying number of parallel processes \(p\). Figure 2a presents the total number of particles, including ghost particles, owned by all processes in particular parallel runs. The dotted columns represent the local particles,always equal to 1,000,188. The red (GL1), blue (GL2), and green (GL3) columns without dots represent ghost particles in ghost layers of thickness = 2\(R\), 4\(R\), and 6\(R\), respectively. Each MPI process handles local particles in its subdomain and ghost particles in relevant ghost layers. Therefore, the total number of processed particles depends on the total number of ghost particles, which increases with the number of parallel processes. In the implemented parallel algorithm, only contact detection and a part of contact force computations are performed on ghost particles, which reduces the computational load of ghost layers. However, the computations performed on ghost particles increase the load and cannot be neglected. The ghost particles of the thinnest layer, GL1, made up 3.1% and 10.0% of the total number of processed particles in the case of 4 and 16 processes, respectively. In the cases of the thicker ghost layers, GL2 and GL3, this percentage increased up to 16.6% and 26.1%, respectively. Moreover, the number of ghost particles defines the amount of data transferred among MPI processes. Thus, the ratio of the number of ghost particles to the total number of particles represents the ratio of communication to computation. Figure 2b shows the variation in the maximum (max) and mean numbers of particles per process, which indicates load imbalance. The RCB method divides particles into nearly equal subsets according to particle coordinates. The maximum number of local particles owned by a processor differed from the mean number of local particles by 3.2% of the mean in the case of the GL1 layer and 16 processes. The number of ghost particles owned by processors can be different due to domain boundaries defined by implicit planes, where ghost layers are not necessary. Thus, the difference between the maximum number of all particles owned by a processor and the mean varied by up to 6.2% (16 processes) of the mean in the case of the GL1 layer. For thicker ghost layers, GL2 and GL3, the difference increased by up to 8.9% and 11.4% of the mean, respectively, indicating the growing load imbalance. Load balance, minimizing the idle time of processes, can be critical to the parallel performance of the computationally intensive SaaS. Load imbalance can be estimated by using the percentage imbalance measure, which evaluates how unevenly the computational load is distributed. The percentage imbalance \(\lambda\) was computed using the following formula:  
\[\lambda=\left(\frac{L_{\text{max}}}{L_{\text{avg}}}-1\right)\cdot 100\% \tag{2}\]  
where \(L_{avg}\) is the averaged load over all processes and \(L_{max}\) is the load of the process that has the largest computational load. Figure 2: The number of particles owned by a varying number of parallel processes: (**a**) total number of particles and (**b**) varying number of particles per process. The time consumed by computational procedures is almost the exact measure of the computational load. Therefore, it was considered the load in this study. The computing time is measured by timers in the computational procedures of the DEM code. The communication time is not included in the computing time. Figure 3 shows the computing time and load imbalance for benchmarks with ghost layers of different thicknesses solved by a varying number of MPI processes.