# Surface Reconstruction Planning with High-Quality Satellite Stereo Pairs Searching
## 2 Motivation and Method Overview
### Framework of Quality-Driven Stereo Planning Algorithm  
As previously discussed, the estimation of stereo reconstruction quality in advance is an essential premise for quality-driven planning. In this paper, we propose to learn a regression model to quantify this value from historical data by correlating 3D reconstruction results with the attributes of time windows, as shown in Figure 1. The possible opportunities of image shooting are calculated according to satellite orbital dynamics and are illustrated as slices in the orbits. While historical data contain images which can be used to determine the final reconstruction quality for estimation model learning, the future tasks to be planned can only have the time windows with their attributes as input. The images to be planned corresponding to the time slices in orbits are depicted with the same color of blue or orange, representing the images to be captured with different satellites in the constellation.  
As shown in Figure 1, the proposed algorithm consists of three main components including 3D surface reconstruction, quality prediction, and stereo pair searching. The role of surface reconstruction is to accurately recover a realistic 3D Earth surface using satellite photogrammetric images as input. Though progress has continuously been made in 3D remote sensing by employing state-of-the-art computer vision technologies, the reconstruction outcome highly depends on the original quality and attributes of the satellite image pair, which needs to be carefully planned, for which the estimation of quality is essential and can be learned from historical DSM reconstruction results. For historical images in Figure 1, pair-wise image combinations are connected, and the DSM accuracy of each pair can be compared to the ground-truth 3D model to provide quality values which are depicted as the edge weights of pair-wise links in historical images. The detailed attributes of each image's time window are then mapped to the calculated quality to learn the regression model for further predictions of future tasks whose quality of 3D reconstruction is unknown beforehand. In this procedure, a CatBoost algorithm is employed with customized feature enhancement and selection to fulfill accurate quality prediction. The planning model can then be modeled with the overall quality as one objective, and the optimal combinations of imaging opportunities can be found by searching for high objective fitness, so as to achieve a quality-driven planning method. As shown in Figure 1, given the constraints such as satellite maneuvering time, the searching of stereo pairs can be solved as a constrained combinatorial optimization problem which is NP-hard. In the proposed planning, multi-objectives including reconstruction quality as well as task quantity and timeliness are optimized to generate a feasible imaging schedule for satellites to execute. In this way, high-precision reconstruction with appropriate image pairs can be obtained, thus significantly improving the quality and efficiency of 3D reconstruction.