# UV3D: Underwater Video Stream 3D Reconstruction Based on Efficient Global SFM
###### Abstract  
With the increasing demand for underwater resource exploration, three-dimensional (3D) reconstruction technology is important for searching for lost underwater civilizations, underwater shipwrecks, or seabed structures. However, faced with the limitations of underwater unmanned systems in terms of energy, bandwidth, and transmission delay, 3D reconstruction technology based on video streams as direct data will not work well. We propose a terminal image processing strategy to save data transmission time and cost and to obtain 3D scene information as soon as possible. Firstly, we propose an adaptive threshold key frame extraction algorithm based on clustering, which extracts key frames from the video stream as structure from motion (SFM) image sequences. On this basis, we enhance the underwater images with sufficient and insufficient illumination to improve the image quality and obtain a better visual effect in the 3D reconstruction step. Additionally, we choose global SFM to construct the scene and propose a faster rotation averaging method, the least trimmed square rotation averaging (LTS-RA) method, based on the least trimmed squares (LTS) and L1 RA methods. It is proven to reduce 19.97% of the time through experiments. Finally, our experiments demonstrate that the dense point cloud saves about 70% of the transmission cost compared to video streaming. key frame extraction; clustering; image enhancement; LTS rotation averaging; global SFM-PMVS +
Footnote †: journal: Applied sciences  
0000-0002-0002-3899  
optical images are colorful and full of detailed information compared to acoustic sensors. They are used on land more commonly, such as with drone navigation or self-driving technology. The limitation of optical camera applications underwater mainly comes from the poor visibility underwater due to the much stronger attenuation and scattering of light in water than that in the atmosphere, meaning optical sensors are more suitable for exploring close-range targets [5]. In general, research on underwater 3D reconstruction is mainly based on the SFM (structure from motion) system--a method used for calculating both camera poses and structures from a set of images [6], which can be divided into two categories, incremental SFM and global SFM. Incremental SFM [7; 8] starts from a two-view 3D reconstruction as the initial structure and add views one-by-one with bundle adjustment [9] every time. Reconstructing the model of the scene sequentially can be robust and accurate; however, with the repeated registration and triangulation process, the cumulative error becomes larger and larger, which may result in scene drift [10]. Additionally, repetitively solving nonlinear bundle adjustments results in poor run-time efficiency. Regarding global SFM [11], the pipeline usually solves the problem in 3 steps. The first step solves all pairwise relative rotations through the epipolar geometry, and constructs a view graph whose vertexes represent cameras and whose edges represent epipolar geometry constraints. The second step involves rotation averaging [12] and translation averaging [13], which separately solve camera orientation and motion issues. The last step is the bundle adjustment, which aims to minimize the reprojection error and refine both the scene structure and camera poses. Compared to incremental SFM, the global method avoids cumulative error and is more efficient. Regarding the disadvantages of global SFM, it is not robust enough to outliers. According to a large number of studies, most underwater stereo vision systems are based on incremental SFM [14; 15]. However, the incremental approach is strongly dependent on the choice of the initial image pair and the selection of the next frame. Due to the complexity of the underwater environment coupled with the surge in water, the instability of the device will increase and the image may be intermittent, meaning that the selection of the next frame will be more difficult.