# A SqueezeSAR Spatially Adaptive Filtering Algorithm Based on Hadoop Distributed Cluster Environment
## 3 Experimental Results and Analysis
### Comparative Experiments and Analysis of Results
#### 3.3.2 Performance Evaluation
Therefore, when evaluating the performance of the parallel Spark-based filtering algorithms for YARN cluster data processing, we used 480 tasks for data partitioning as a reference. \begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Configuration** \\ \hline operating system & Linux RedHat 4.8.5 \\ Hadoop & Version 3.2.2 \\ Spark & Version 3.0.2 \\ Java & Version 1.8 \\ Python & Version 2.7.5 \\ Doris (Master node installation only) & Version 5.0.3 \\ StaMPS (Master node installation only) & Version 4.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: YARN cluster software configuration information. \begin{table}
\begin{tabular}{c c} \hline \hline
**Task Number** & **Run Time/s** \\ \hline
240 & 99 \\
320 & 95 \\
400 & 90 \\
480 & 86 \\
560 & 89 \\
640 & 93 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Processing time of a single PATCH for different tasks with 8 executors. To effectively evaluate the performance of our proposed algorithm, we parallelized the entire dataset with executors assigned different processing units separately and evaluated the parallelized processing performance with executors assigned single-core and 2-core processing units. The speedup and time comparison plots for the spatially adaptive filtering processing of a single PATCH file and the entire dataset are depicted in Figures 12-15 for the assigned single and dual cores, respectively, in a YARN cluster environment. The entire dataset had 72 PATCH files, and the total time spent filtering the interferogram using the conventional filtering algorithm was 3.68 h. It is clear from Figures 13 and 15 that the parallelized processing time is lower than that of the conventional filtering algorithm when the number of cores in the cluster reaches two. However, the results of the two parallelization strategies were significantly different when the number of allocated cores was the same. With a single core allocated per executor, the performance of the overall data processing decreased when the number of executors exceeded 16. Compared with the speedup ratio of the computation of the parallelization strategy with two cores, as shown in Figure 12, the parallelization strategy with two cores increased slowly with an increase in the number of executors, and the speedup ratio of the parallelization strategy with a single core was lower with the same number of cores. The main reason for this difference is the frequent data communication between the different nodes, which gradually dominates the process of broadcasting large datasets to the executors via the broadcast algorithm as the number of executors increases. Despite the addition of the processing units, the process of broadcasting the variable serialization consumes more time. Figure 12: Number of executors versus speedup ratio for the case of three nodes: the \(x\)-axis represents the number of executors, where the first value indicates that one executor is started in the case of a single core, and the remaining values start 2 cores. The \(y\)-axis represents the speedup. Figure 14: The relationship between the number of executor-allocated single-core processing units and the speedup in the case of three nodes: the \(x\)-axis represents the number of executors. The \(y\)-axis represents the speedup as the number of executors increases. Figure 13: Comparison of filtering processing time with increasing number of executors for the entire study area (all PATCHs): the \(x\)-axis represents the number of executors, where the first value indicates that one executor is started in the case of a single core and the remaining values all start 2 cores. the \(y\)-axis represents the filtering processing. Although the entire dataset can achieve a performance boost in a clustered environment, the effect is not significant, and the reasons for this can be explored in various ways.