## 4 Experimental Evaluation
### Evaluating the Proposed Approach
#### 4.4.2 Assessing the Execution Times of the Proposed Approach
For example, the simulation time of a single time step, detailed in Section 4.1, and the in-transit co-processing time can vary by orders of magnitude depending on the simulation, the specified simulation parameters and which types of analysis and visualization tasks are performed during the co-processing stage. Furthermore, co-processing could be performed for each simulated time step or for a small fraction of time steps. As such, we chose to focus the evaluation exclusively on the actual performance of the three main stages of the proposed approach, i.e., the co-processing stage only consists of data decompression and data reconstruction. The average execution times of each stage are shown in Figure 11. The importance calculation of the RLE, HOMO, and LZ77 tests used for comparison consisted of a single Mean probe. Figure 10: Size of the transferred compressed data using the MF, MY, and MZ data sets on (**a**) cluster A and (**b**) cluster B. The results of the RLE and Pipeline 1 tests overlap. Out of the lossless compression methods used for comparison, RLE achieved the best performance. For the total execution time, the RLE compression method achieved a speedup of 1.26\(\times\) (cluster A) and 1.3\(\times\) (cluster B) compared to using no compression. The lossless Pipeline 1 achieved an even more significant speedup; 1.62\(\times\) on cluster A and 1.55\(\times\) on cluster B. Compared to RLE, Pipeline 1 could speed up the execution time by up to 1.29\(\times\) and 1.19\(\times\) on the two respective clusters. Similarly, Pipeline 2, which reduces unimportant data, achieved total speedups of 1.69\(\times\) and 1.52\(\times\) compared to the RLE method. Table 4 summarizes the average speedups of the total execution time for both pipelines compared to the other evaluated methods. In more detail, both Pipeline 1 and Pipeline 2 achieved significantly better performance for the data distribution and co-processing stages than the NONE, RLE, and LZ77 methods. The execution time of the co-processing stage for Pipeline 1 was similar to the RLE test when using static load balancing. The uneven load caused this; although multiple transit processes achieved significantly lower computation time than when using only RLE compression, the total execution time is dependent on the slowest process. Using a k-d load balancing technique solved the uneven load on the transit nodes. On cluster A, Pipelines 1 and 2 achieved speedups of 1.91\(\times\) and 1.94\(\times\) of the co-processing stage as compared to using RLE compression, and speedups of 1.32\(\times\) and 1.33\(\times\) compared to using no compression. Similarly, on cluster B the two pipelines achieved speedups of 2.00\(\times\) and 1.99\(\times\) as compared to an RLE compression. Compared to using no compression, the speedup of the co-processing stage was 1.05\(\times\) and 1.04\(\times\) for the two respective pipelines. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Cluster A**} & \multicolumn{4}{c}{**Cluster B**} \\ \cline{2-9}  & **None** & **RLE** & **HOMO** & **LZ77** & **None** & **RLE** & **HOMO** & **LZ77** \\ \hline Pipeline 1 & 1.62 & 1.29 & 0.55 & 3.51 & 1.55 & 1.19 & 0.26 & 2.98 \\ Pipeline 2 & 2.13 & 1.69 & 0.72 & 4.60 & 1.98 & 1.52 & 0.33 & 3.81 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Relative speedups of the total execution time for Pipeline 1 and Pipeline 2 compared to the other evaluated methods. Figure 11: Average time step execution time for the three main stages using all three data sets on (**a**) cluster A and (**b**) cluster B. The error bars display the standard deviation of the computation times. Utilizing dynamic load balancing significantly improved the execution times of the co-processing stage. However, the load balancing only determines where data is sent. The compression time and the amount of data that needs to be transferred from each simulation process do not change.