# Towards underwater docking through teaching-by-showing

Nesrine Karboua, Olivier Kermorgant

<figure><img src="image_1.png" /></figure>

## To cite this version:

Nesrine Karboua, Olivier Kermorgant. Towards underwater docking through teaching-by-showing. Oceans, IEEE, Jun 2025, Brest, France. hal-05026269

HAL Id: hal-05026269

https://hal.science/hal-05026269v1

Submitted on 9 Apr 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Towards underwater docking through teaching-by-showing

Nesrine Karboua and Olivier Kermorgant

Nantes Université, École Centrale Nantes,
CNRS, LS2N, UMR 6004, F-44000 Nantes, France

***Abstract***— We propose an underwater docking approach based on teaching-by-showing. A visual trajectory is obtained from a manual docking, then used as a reference during autonomous mode. Using a full trajectory instead of a single reference at the docking pose allows for more variation in the initial pose. It is also compatible with a moving docking station, as the vehicle will adapt its motion to track the trajectory in the image. A proof-of-concept is illustrated in a simulation scenario.

***Index Terms***— Underwater docking, visual memory, visual servoing

## I. INTRODUCTION

The global push for sustainable energy solutions has led to substantial progress in marine renewable energy (MRE). Offshore wind turbines, and more recently floating offshore wind structures (FOWs), have emerged as pivotal technologies for capturing clean energy from deeper waters [1]. These FOWs rely on intricate anchoring systems comprised of cables, buoyant mechanisms, and ballast components that must endure extreme marine conditions, including high currents, waves, and corrosive seawater. Regular inspection and maintenance are therefore critical to ensure operational efficiency, safety, and extended service life. Yet, the harsh offshore environment constrains direct human intervention and calls for the deployment of underwater robotic systems such as remotely operated vehicles (ROVs) and autonomous underwater vehicles (AUVs).

Despite their indispensable role, ROVs and AUVs come with inherent limitations. Subsea-resident AUVs, intended for prolonged autonomous missions, frequently encounter restrictions related to battery capacity and data storage, necessitating periodic retrieval for recharging and data offloading. Analogously, ROVs—primarily used for brief operations from a surface vessel—must be recovered at the conclusion of each task. These challenges have brought attention to the need for effective recuperation solutions, including docking stations that allow for retrieval, battery charging, and data transfer of underwater vehicles.

Autonomous docking of underwater vehicles has been extensively investigated in scenarios involving seabed-fixed docking stations, predominantly in the context of resident AUVs. For example, [2] presented a docking system for torpedo-style AUVs using acoustic and visual sensors, whereas [3] proposed a binocular-vision positioning method for a similar class of vehicles. Additionally, [4] explored a combined acoustic and vision-based positioning technique tailored to hovering-type AUVs. Although these solutions

have yielded promising outcomes, they are generally designed for static resident docking stations.

Seabed-mounted docking stations also face numerous drawbacks. They typically require high-bandwidth and low-latency communication [5], which is often infeasible to implement across the expansive domains occupied by offshore wind farms. Furthermore, deploying and sustaining these stations in deep waters is logistically demanding and costly. These limitations have stimulated research into dynamic docking alternatives better suited to some specific tasks and environments.

Nevertheless, dynamic docking platforms introduce a fresh set of obstacles due to the station's continuous motion. Several innovative strategies have been proposed, [6] used a three-dimensional imaging sonar for AUV docking into a ROV-based station, [7] converted one AUV into a funnel-like docking station to recover another AUV via a vision-based approach. In [8] the authors engineered a launch and recovery system (LARS) featuring a claw-like mechanism on an AUV to grasp a second AUV, while remaining tethered to a surface vessel. Though such designs illustrate remarkable innovation, they largely remain at the research and development stage.

In industrial practice, the predominant option is a LARS secured to a support vessel, which deploys a docking station (DS) through a tether management system (TMS) (see Fig.1). However, the DS is continuously in motion because of tether forces and the effects of waves and currents, making

<figure><img src="image_3.png" /><figcaption>Fig. 1. Dynamic docking station setup</figcaption></figure>
the docking procedure especially challenging. This process is still commonly carried out by skilled ROV operators, as fully autonomous docking in these circumstances is not yet widespread. To date, [5] have conducted the only study that successfully demonstrates autonomous docking under these conditions, employing a commercial hovering-type ROV and a standard cage-type TMS with a vision-based method for pose estimation. One drawback of that approach is that camera calibration inaccuracies could degrade performance, and the underwater environment can further hamper pose calculations owing to reduced image clarity.

To address these issues, our work presents an image-based visual servoing (IBVS) framework, in which the control signals rely directly on visual features rather than on pose estimates as in position-based visual servoing (PBVS). By integrating a concept of visual memory with a moving LARS/TMS setup, we propose a novel visual reference trajectory that can be pursued autonomously without requiring explicit pose estimation. In this strategy, the path is characterized by a set of visual features, and the IBVS automatically accommodates TMS motions that can be estimated and compensated for in real time.

## II. VISUAL MEMORY

Robot navigation driven by visual memory (VM), for both indoor [9] and outdoor [10] contexts, mimics human behavior by storing crucial visual snapshots during an initial exploratory phase to guide subsequent navigation. In such an approach, a VM builds a topological map of key images, expressed as a directed graph whose nodes represent these stored images and whose edges depict the relationships among them. This graph-based representation supports robot localization, path planning, and autonomous navigation, extending beyond standard visual servo-control paradigms by allowing robots to scale up their working domain.

However, these methods generally presuppose feature-rich surroundings so that a sufficient array of key images can be captured. In a high-seas docking scenario with an otherwise featureless environment, the only significant visual cue is typically the docking station itself. Nonetheless, a streamlined method inspired by the VM concept can still be exploited, given that it has yet to be thoroughly explored in this vision-dominated domain and that skilled ROV pilots are readily available in industrial contexts. Specifically, a pilot could undertake a teleoperated docking in relatively calm conditions, recording a sequence $s_i^*$ of reference visual features that define a viable docking trajectory that can be used later as a reference for our IBVS controller.

### A. High-level control with visual servoing

Let $s$ represent the current set of visual features (e.g., the image coordinates of a set of interest points, alongside any other known data regarding the target). During autonomous docking, the visual error $e = s - s^*$ is driven to zero via IBVS [11]. The time derivative of these features in relation

<figure><img src="image_4.png" /><figcaption>Fig. 2. Using a visual memory from offline teaching</figcaption></figure>

to camera velocity $v_c$ is given by:

$$ \dot{s} = L_s v_c + \frac{\partial s}{\partial t} \quad (1) $$

where $L_s$ denotes the interaction matrix that ties changes in the visual features to the robot's motion, and $\frac{\partial s}{\partial t}$ captures any intrinsic motion of the observed object (in this case, the TMS). A classical proportional control is used for the visual servoing, that is:

$$ \dot{e} = \dot{s} - \dot{s}^* = -\lambda e \quad (2) $$

Solving (2) and (1) for $v_c$ lead to the well-known camera frame velocity control in visual servoing:

$$ v_c = -\lambda L_s^+ \left( e - \frac{\hat{\partial s}}{\partial t} + \dot{s}^* \right), \quad (3) $$

where $\hat{\partial s}/\partial t$ is an estimation of the intrinsic motion of the observed object. A classical way to estimate this motion is to filter the difference between the expected visual feature evolution $L_s \hat{v}_c$ to the observed one $\dot{s}$ [12]:

$$ \frac{\partial \hat{s}}{\partial t} = \frac{s(t) - s(t - \Delta t)}{\Delta t} - L_s \hat{v}_c \quad (4) $$

where $\hat{v}_c$ is an estimation of the actual camera velocity. In practice, for an underwater vehicle this may captures additional uncertainties such as ocean current or coarse estimation of vehicle state.

Finally, the $\dot{s}^*$ term in (3) is a feedforward term to take into account the future evolution of the visual reference. While this is well-suited for visual trajectory tracking, it forces the system to track the docking path with a predefined velocity. In our approach, a single set of features $s_i^*$ is tracked and a threshold is used to move towards the next feature $s_{i+1}^*$.

### B. Low-level thruster control

As depicted in Fig.2, the visual servoing layer outputs a reference velocity $v_c$ for the camera. In this section, we detail the low-level controller mapping this velocity to the actual thruster inputs.

The velocity setpoint is first changed to the robot control frame, leading to a reference velocity $v_r^*$. This is compared to the current velocity $v$, obtained by a state estimator of the vehicle. We rely on a simple model-based PID [13], where 6 independent PIDs run on each axis of the robot, generating a desired acceleration $a^*$. The acceleration is then
<figure><img src="image_5.png" /><figcaption>Fig. 3. Experiment #1, sequence followed by the AUV. Visual trajectory consists in dots from blue (start) to red (end).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 4. Experiment #1, task information during docking. While visual features (a) are discontinuous due to change in the tracked points, the corresponding twist setpoint (b) is smooth. When the robot deviates from its ideal trajectory the forward velocity (blue) decreases, inducing slight plateaus in the x-component of the pose error (c). This pose error is shown to assess the behavior but is not actually used in the control.</figcaption></figure>

fed to the dynamic model of the vehicle in order to get the corresponding reference wrench $\tau^*$:

$$ \tau^* = Ma^* + M_a \dot{v}_w + C(v) v + C_a(v_w) v_w + d(v_w) v_w + \tau_g $$

where M and C are the inertia and Coriolis matrices, $M_a$ and $C_a$ are the added ones due to the water, d is the damping function and $\tau_g$ combines the gravity and buoyancy forces. The relative velocity $v_w$ between the vehicle and the fluid is currently assumed to be equal to the vehicle twist, but an additional observer could be used to estimate this value.

Finally, as our vehicle has vectored thrusters with a dead-zone, we rely on the approach proposed in [14] to map the reference wrench to thruster inputs.

### III. SIMULATION RESULTS

We evaluate the proposed method in a simulation environment integrating a BlueROV2 platform and a cage-like TMS using Gazebo [15]. Instead of manual docking, an ideal trajectory (Fig.3 a) is used to generate the corresponding visual features (see Fig.3 b) that the robot should track. Such reference is similar to the one that would be obtained from teaching-by-showing and is robust to calibration errors. The visual markers are simple points, in practice they would require lights on the TMS. The same approach could be used with other appearance-based features [10].

Three scenarios with increasing difficulty are considered. In the first one, no external disturbances exist and only approximations in the robot dynamic model lead to an

imperfect tracking. In the second one, ocean current is added. In the last one, the TMS is also oscillating. For all scenarios, the 3D trajectory that was used to generate the visual trajectory is displayed. We highlight the fact that this 3D trajectory is only displayed for information, and that this data is neither used nor available for the controller.

#### A. No external disturbances

The behavior during a simple docking is illustrated in Fig.4. The visual error (Fig.4a) slightly oscillates due to approximations in the low-level control (hydrodynamic model and PIDs tuning). The same error also shows discontinuity when the desired features are changed to the next in the sequence, as detailed in Section II. In order to reduce the impact on the actuators, the reference velocity is thus filtered (Fig.4b). As the main error is on the x-axis of the vehicle (blue curve), most of the motion is surge but minor sway, heave and yaw corrections appear regularly when the vehicle is not on the trajectory. This imperfect control is due to uncertainties in the dynamic model.

While tracking a discrete visual trajectory may introduce control discontinuities in a classical robot, these discontinuities are effectively smoothed out for an ROV. In practice, any potentially non-smooth velocity input merely serves as a reference for the low-level thruster control, which ensures continuous motion.

The case of lateral ocean current is depicted in Fig.5.
<figure><img src="image_7.png" /><figcaption>Fig. 5. Experiment #2, Sequence followed by the AUV. Visual trajectory consists in dots from blue (start) to red (end).</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6. Experiment #2, task information during docking.</figcaption></figure>

## B. Ocean current

Figures 5 and 6 highlight a similar experiment, with a starting position further away and a lateral ocean current. Thus, a higher lateral velocity (orange in Fig.6b) is induced in order to cope with the current even if the TMS does not move by itself.

Compared to the first simulation, the ROV has more distance to travel and higher velocities are visible in Fig.6b). Due to current uncertainties, the visual feature error (Fig.6a) is also higher than in the previous experiment. As can be seen in Fig.5, the ROV quickly manages to align with the ideal (but unknown) trajectory and eventually enters the TMS by following the visual trajectory.

## C. TMS motion

The third illustration of the proposed approach now considers a moving TMS. This motion may come from surface waves and has the following characteristics in the simulation:

<table><caption>TABLE I
TMS PER-AXIS MOTION</caption><thead><tr><td>Axis</td><td>Amplitude</td><td>Period</td></tr></thead><tbody><tr><td>surge</td><td>±10 cm</td><td>18 s</td></tr><tr><td>sway</td><td>±10 cm</td><td>22 s</td></tr><tr><td>heave</td><td>±20 cm</td><td>20 s</td></tr><tr><td>roll</td><td>±3°</td><td>23 s</td></tr><tr><td>pitch</td><td>±3°</td><td>25 s</td></tr></tbody></table>

The overall oscillation frequency is clearly visible in Fig.8a where the visual feature error is smoother, yet higher, than in previous experiments. Notably, at the beginning, when the ROV is the farthest from the TMS, a slight motion in the TMS does not induce so much changes in the visual features. As a consequence, it is difficult in (4) to clearly identify the motion of the observed object. As the ROV approaches the TMS (around t = 300 s in Fig.8) the estimation of the intrinsic motion gets better and the ROV stays close to the ideal trajectory. As the TMS is moving, the visual error is never exactly reduced to 0 and the control continues even when the vehicle has reached the end of the trajectory. In practice, the vehicle would of course be physically docked at this point.

## D. Discussion

The same controller was used in all three experiments, showing that the proposed approach is robust enough to handle ocean currents and the moving TMS. While overall performances decrease as the setup gets more challenging, the ROV was able to follow an implicit trajectory, only defined by the visual features. Few parameters are needed to tune the behavior, namely the control gain $\lambda$ (2) and the filter coefficient in (4). The most sensitive parameter is the threshold to move forward to the next key feature $s_i^*$, as it is likely to depend on the visual features $s$ that are used to define the trajectory.
<figure><img src="image_9.png" /><figcaption>Fig. 7. Experiment #3, sequence followed by the AUV. Because of the TMS motion, the ROV is quickly far from the ideal trajectory at the beginning. The impact of the motion is less and less important as the ROV approaches the TMS.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 8. Experiment #3, task information during docking.</figcaption></figure>

## IV. CONCLUSION

We propose a visual memory approach to underwater docking. The tracked trajectory does not require any pose computation and is entirely defined in the image space. This makes it robust to calibration uncertainties, docking station motion, and allows for complex trajectories depending on the starting pose. While we rely on image points in this study, our approach could be applied with other visual features. Future work will validate this approach on a real setup. Other visual navigation schemes will also be considered, such as visual MPPI [16] which is well-suited for visual path tracking. Finally, a frequency-based estimation of the TMS motion will also be investigated to better compensate for the expected disturbances.

## REFERENCES

[1] W. Europe, “Floating offshore wind vision statement,” *Wind Europe: Brussels, Belgium*, 2017.

[2] G. Vallicrosa, J. Bosch, N. Palomeras, P. Ridao, M. Carreras, and N. Gracias, “Autonomous homing and docking for auvs using range-only localization and light beacons,” *IFAC Conf. on Control Applications in Marine Systems*, 2016.

[3] Y. Li, Y. Jiang, J. Cao, B. Wang, and Y. Li, “Auv docking experiments based on vision positioning using two cameras,” *Ocean Engineering*, 2015.

[4] Y. Sato, T. Maki, K. Masuda, T. Matsuda, and T. Sakamaki, “Autonomous docking of hovering type auv to seafloor charging station based on acoustic and visual sensing,” in *IEEE Underwater Technology, 2017*.

[5] P. Trslic, M. Rossi, L. Robinson, C. W. O'Donnel, A. Weir, J. Coleman, J. Riordan, E. Omerdic, G. Dooly, and D. Toal, “Vision based autonomous docking for work class rovs,” *Ocean Engineering*, 2020.

[6] H. Uchihori, I. Yamamoto, and A. Morinaga, “Concept of autonomous underwater vehicle docking using 3d imaging sonar,” *Sensors and Materials*, 2019.

[7] S. Liu, H. Xu, Y. Lin, and L. Gao, “Visual navigation for recovering an auv by another auv in shallow water,” *Sensors*, 2019.

[8] N. Dahn, C. Gaudig, T. Lehr, N. Pech, P. Kampmann, L. Christensen, and F. Kirchner, “Autonomous docking between a mobile subsea docking station and an auv while in motion,” in *OCEANS 2024 - Halifax*, 2024.

[9] J. Delfin, H. M. Becerra, and G. Arechavaleta, “Humanoid localization and navigation using a visual memory,” in *2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids)*, 2016.

[10] J. Courbon, Y. Mezouar, and P. Martinet, “Autonomous navigation of vehicles from a visual memory using a generic camera model,” *IEEE Trans. on Intelligent Transportation Systems*, 2009.

[11] F. Chaumette and S. Hutchinson, “Visual servo control. I. Basic approaches,” *IEEE Robotics and Automation Mag.*, 2006.

[12] F. Chaumette and S. Hutchinson, “Visual servo control. II. Advanced approaches,” *IEEE Robotics and Automation Mag.*, mar 2007.

[13] G. Antonelli, “Modeling of underwater robots, underwater robots, 3rd edn., edited by b. siciliano and o. khatib,” 2013.

[14] B. Ropars, A. Lasbouygues, L. Lapierre, and D. Andreu, “Thruster’s dead-zones compensation for the actuation system of an underwater vehicle,” in *European Control Conference*, 2015.

[15] N. Koenig and A. Howard, “Design and use paradigms for gazebo, an open-source multi-robot simulator,” in *IEEE/RSJ Int. Conf. on Intelligent Robots and Systems*, 2004.

[16] I. S. Mohamed, G. Allibert, and P. Martinet, “Sampling-based mpc for constrained vision based control,” in *IEEE/RSJ Int. Conf. on Intelligent Robots and Systems*, 2021.