# Deepwater 3D Measurements with a Novel Sensor System
## 2 Materials and Methods
### Data Recording
#### 2.2.3 Motion Compensation  
For the planned application in inspection tasks, average ROV speeds of between 0.1 m/s and 1.0 m/s are assumed. Despite the high image recording frequency of up to 900 Hz and the associated low observed object movement from image to image, the length of an image sequence of ten subsequent images results in non-negligible shifts in the image  
Figure 4: ArUco markerboards for calibration.  
content during a sequence. This shift is approximately twelve pixels at a speed of 1 m/s over a sequence of ten subsequent images at 900 Hz. This causes non-negligible image disturbances, particularly in the vertical image direction, which can generate errors in the 3D reconstruction or render the measurements unworkable.  
To reduce these errors, a motion compensation algorithm was developed. To simplify the calculation, the constant linear movement of the sensor system and a constant known measurement distance were assumed for the period of image acquisition for a 3D scan. The movement direction and speed were estimated from the image data and the IMU data. A detailed description of this process is documented in [30]. Figure 5 shows the effect of motion compensation using the example of measuring a pipe section.  
The standard deviation of the 3D points on the surfaces is significantly reduced, namely from 0.39 mm to 0.33 mm at 0.1 m/s and from 0.52 mm to 0.32 mm at a 0.7 m/s velocity for the sphere measurement and from 0.39 mm to 0.34 mm at a 0.7 m/s velocity for the cylinder measurement. At a 0.1 m/s velocity, motion compensation for the cylinder measurement did not provide an improvement in the standard deviation value. Additionally, more valid 3D points were obtained, namely by 5% at 0.1 m/s and 88% at a 0.7 m/s velocity for the sphere measurement and by 4% at a 0.7 m/s velocity for the cylinder measurement.  
The developed motion compensation was aimed, in this special case, at the underwater context. To this end, the position of the sensor at the current time is extrapolated with low latency using visual odometry (see next section). The 3D reconstruction process interpolates the expected current mean horizontal and vertical pixel offset on the measurement cameras for all images of the current 3D measurement sequence. This interpolation utilizes time stamps from both past and current sensor poses. For this purpose, an object point is assumed for each camera pixel on a plane positioned orthogonally to the sensor viewing axis and at a nominal measurement distance.  
For the actual motion compensation, the rectified 2D output images of a 3D reconstruction can now be shifted by n\(\cdot\)A(x, y) pixels, where n is the number of images in the sequence, and \(\Delta\)(x, y) is the shift in the horizontal and vertical direction in each image of the sequence. As a result (under the assumptions explained above), locations on the measurement object recorded at different times are mapped to the same image pixel with a known movement. This means that all further calculation steps (correlation, filtering, triangulation, etc., see [24]) can be applied as usual. However, in order to avoid the additional effort of this shift, we expanded the correlation module to directly take into account the offset \(\Delta\)(x, y) during the correlation. This means that motion compensation can be carried out within the framework of the temporal correlation implemented in OpenCL without any loss of performance.