# Performance of Communication- and Computation-Intensive SaaS on the OpenStack Cloud
## 2 Discrete Element Method Software
### Parallel Implementation  
The employed DEM software was developed using the C++ programming language. The GNU compiler collection (GCC) was used with the second-level optimization option for compiling the code. In this study, CPU-time-consuming computational procedures, such as contact detection, contact force computation, and time integration, were implemented using standard algorithms, widely available in open-source codes [45], to increase the usability of obtained results. Contact detection was based on the simple and fast implementation of a cell-based algorithm [46]. The explicit velocity Verlet algorithm [46] was used for time integration.  
The long computational time of DEM simulations limits the analysis of industrial-scale applications. The selection of an efficient parallel solution algorithm depends on the specific characteristics of the considered problem and the numerical method used [44; 45; 46; 47]. The parallel DEM algorithms differ from the analogous parallel processing in the continuum approach. Moving particles dynamically change the workload configuration, making parallelization of DEM software much more difficult and challenging. Domain decomposition is considered one of the most efficient coarse-grain strategies for scientific and engineering computations; therefore, it was implemented in the developed DEM code [16; 44]. The recursive coordinate bisection (RCB) method from the Zoltan library [48] was used for domain partitioning because it is highly effective for particle simulations [16; 48]. The RCB method recursively divides the computational domain into nearly equal subdomains by cutting planes orthogonal to the coordinate axes, according to particle coordinates and workload weights. This method is attractive as a dynamic load-balancing algorithm because it implicitly produces incremental partitions and reduces data transfer between processors caused by repartitioning. Interprocessor communication was implemented in the DEM code by subroutines of the message passing library MPI.  
The main CPU-time-consuming computational procedures of the DEM code are contact detection, contact force computation, and time integration. Each processor computes the forces and updates the positions of particles only in its subdomain. To perform their computations, the processors need to share information about particles that are near the division boundaries in ghost layers. A small portion of communications is performed when processors exchange particles as the particles move from one subdomain to another. This communication is optional and is performed only in the case of a non-zero number of exchanging particles. The main portion of communications is performed prior to performing contact detection and contact force computation. In the present implementation, particle data from the ghost layers are exchanged between neighboring subdomains. The exchange of positions and velocities of particles between MPI processes is a common strategy often used in DEM codes [45], but an alternative based on transferring computed forces also exists. Despite its local character, interprocessor particle data transfer requires a significant amount of time and reduces the parallel efficiency of computations. The size of ghost layers can depend on the particle size, particle flow, and implemented algorithms. Therefore, in this study, ghost layers of different sizes were considered in order to study communication issues.