# A SqueezeSAR Spatially Adaptive Filtering Algorithm Based on Hadoop Distributed Cluster Environment
## 4 Conclusions  
In this study, we proposed a parallelization solution for the SqueezeSAR spatially adaptive filtering algorithm based on the Spark computing engine in a Hadoop cluster environment. Through this solution, not only is the processing time of the filtering algorithm reduced without affecting the accuracy of the conventional algorithm, but idle computer resources are also reasonably utilized, and the advantages of distributed cluster computing and storage are fully utilized. This can provide a feasible and valuable data processing solution for personnel and research units engaged in InSAR research. In addition, the parallelization solution is implemented by Spark, a popular distributed computing framework, which is highly portable, suitable for the effective development of distributed computing cloud platforms, easier to implement, and more applicable than the traditional multi-point interface (MPI) architecture. From the perspective of performance optimization, we determined the impact of different parameters on the performance of the filtering algorithm in the performance evaluation process. Using different resource-allocation mechanisms for different datasets can effectively balance the load of the cluster and lead to the reasonable use of the resources of the distributed cluster.  
The parallelized processing scheme was combined with the current, more mature Hadoop distributed cluster framework to process the SqueezeSAR spatial adaptive filtering  
Figure 15: Comparison of filtering processing time with increasing number of executors (single cores) for the entire study area (all PATCHs): \(x\)-axis represents the number of executors. \(y\)-axis represents the filtering processing time with increasing number of executors.  
algorithm. A total of 496 interferograms (with a total data size of 114 GB) generated from 32 scenes of Sentine-1A satellite data, containing the main urban area of Kunming, were used for testing the performance and for accuracy comparison. The spatial adaptive filtering algorithm can compress the processing time of the entire study area from 3.68 h to 1.84 h when the entire study area is divided into 72 PATCHs, and when the study area is larger, hundreds of PATCHs can result in a more significant speedup. It is worth mentioning that, owing to the limited hardware conditions, the scalability of the cluster was not highlighted in this study. With respect to resource utilization, when the amount of data is too large, the advantage of dynamically scalable nodes can be used in the face of huge data flows by dynamically adding and deleting nodes for reasonable computation and storage resources, which greatly reduces the idleness of the computational resources. For future work, we will primarily focus on the cloud environment, solving the problem of Spark not being able to serialize broadcast data over 2 GB and provide more powerful support for InSAR applications in the Big Data environment by taking advantage of the cloud environment.  
Conceptualization, Y.L. ([PERSON]), W.S. and B.J.; software, X.Z. and Y.L. ([PERSON]); validation, K.C.; resources, W.S. and Y.L. ([PERSON]); writing, Y.L. ([PERSON]) and W.S.; supervision, B.J.; project administration, X.Z.; funding acquisition, X.Z. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the National Natural Science Foundation of China (Grant No. 42161067) and the Yunnan Province Key Research and Development Program (No. 202202 AD080010).  
Not applicable.  
The authors would like to thank the Copernicus program for free access to Sentinel-1 images processed in this analysis. The authors would also like to thank the open-source projects Hadoop and Spark provided by the Apache Foundation for their contributions to distributed computing and storage and Doris and StaMPS developers for their contributions to SAR data processing.  
The authors declare no conflict of interest.