# Adaptation of Parallel SaaS to Heterogeneous Co-Located Cloud Resources
## 3 Cloud Infrastructure and Developed SaaS
### Parallel DEM SaaS
```
1:Input
2:W Weights of partitions, initial value 1.0
3:whiletime_loop do
4:\(W^{\text{old}}\gets W\)
5:\(\mathbf{L}\leftarrow\text{Monitor\_load}(0)\)
6:ifCondition(\(\mathbf{L}\)) then
7:\(\mathbf{W}\leftarrow\text{Weight\_calculation}(\mathbf{W}^{\text{old}}\mathbf{L})\)
8:\(\text{Transfer\_Particles\_To\_Host}()\)
9:\(\text{Zoltan\_Partitioning}(\mathbf{W})\)
10:\(\text{Redistribute\_Particles}()\)
11:\(\text{Registration\_Ghost\_Particles}()\)
12:\(\text{Exchange\_Ghost\_Information}()\)
13:\(\text{Transfer\_Particles\_To\_Device}()\)
14:endif
15:\(\text{Transfer\_Ghost\_Particles\_To\_Host}()\)
16:\(\text{Exchange\_Ghost\_Particles}()\)
17:\(\text{Transfer\_Ghost\_Particles\_To\_Device}()\)
18:\(\text{Contact\_search}()\)
19:\(\text{Contact\_history}()\)
20:\(\text{Calculation\_forces}()\)
21:\(\text{Boundary\_conditions}()\)
22:\(\text{External\_forces}()\)
23:\(\text{Integrator}()\)
24:ifwrite_result then
25:\(\text{Transfer\_Particles\_To\_Host}()\)
26:\(\text{Write\_results}()\)
27:endif
28:endwhile
```  
**Algorithm 1** The pseudocode of hybrid MPI/OpenCL parallelization In lines 6-14 of the pseudocode, domain decomposition or partitioning of particles into subsets is outlined. In line 6, the subroutine Condition(0) makes the decision if the repartitioning procedure should be performed. Commonly, repartitioning is necessary if load imbalance measure (1) exceeds the predefined value. In the performed research, a more sophisticated approach is developed according to the specific needs of DEM computations. The dynamic nature of granular flows leads to rapid particle movement through the computational domain, large changes in contact topology, and notable variations in the load configuration. When particles move from one partition to another, there is a need for some communication between MPI processes. The particle data transfer is optional, but information exchange is necessary in each time step. Despite its local character, information exchange and internode particle data transfer consume a significant amount of time, which decreases the performance of parallel computations. Therefore, this particle exchange is performed during the repartitioning procedure. The skinning technique is employed to avoid frequent repartitioning. In the considered time step, particles of one subset can contact particles of another subset only if they are in the ghost layer. In order to make contact with a particle from another subset, the internal particle needs time to cross the ghost layer. This time can be estimated [55], which is often used to reduce the frequency of contact search. Thus, repartitioning should be performed with frequency defined by granular flow physics despite low load imbalance. Finally, the subroutine Condition(0) combines the load imbalance condition with repartitioning frequency defined by application physics. In most benchmarks of the presented research, repartitioning is regularly performed with the frequency defined by application physics to smooth load oscillations and save execution time extensively consumed by repartitioning. In the case of highly variable loads, the percentage load imbalance threshold equal to 10% is also used to trigger repartitioning, which helps accurately capture the load variations. Lower threshold values approach the accuracy limit of the repartitioning method and can lead to insignificant improvements sensitive to load oscillations. Higher threshold values are close to the load imbalance of the lowest heterogeneity cases considered in the research. In line 7, the heterogeneity of resources is evaluated by using different values of weights that result in subsets of different sizes after repartitioning. The new weight \(W_{i}^{\text{new}}\) is computed according to the runtime measured computational load \(L_{i}\) of the parallel process \(i\):  
\[W_{i}^{\text{new}}=W_{i}^{\text{old}}\cdot\left(2-\frac{L_{i}}{L_{\text{avg} }}\right), \tag{2}\]  
where \(W_{i}^{\text{old}}\) is the previous weight of process \(i\).