Embedded Sonar & Video Processing for AUV Applications
Authors: Lane D.M. , Trucco E. 
Published: October 2008

Abstract
We present recent work developing real-time sector scan sonar and video data processing algorithms for embedded use in AUVs and their topside data processors. Forward looking sonar algorithms have been designed and evaluated for object tracking, mapping, navigation and obstacle avoidance. Real time video algorithms have been evaluated for stabilization and docking on the vehicle, and for image stabilization, view synthesis, panoramic mosaicing and 3D shape reconstruction in post processing of data. Uncalibrated, or coarsely calibrated cameras are used throughout. The paper overviews key activities and developments, with references for more detailed descriptions.

Introduction
This paper reports ongoing work developing embedded sonar and video processing systems suitable for mounting on underwater systems such as AUVs. Applications are for vehicle navigation and obstacle avoidance, sea bed mapping and classification, visual servoing/docking, 3D shape reconstruction, and video image mosaicing. For underwater vehicles to navigate safely and accurately in a possibly unknown environment, forward looking sonars can provide useful mission specific and navigation information. Examples are:tracking of underwater objectsvehicle motion estimationmap building in a world reference frameobstacle avoidance.
Similarly, real-time video processing can enhance vehicle utility, and be used to post process video data products for improved visualization and measurement. We have focused on the following:Development of a robust tracker, which targets optimal features for video tracking and implements a completely automatic mechanism for rejecting unstable image featuresA module recovering the camera/platform motion from the acquired video sequence (the homography).Use of the motion data to stabilize the image viewpoint as the AUV swims adjacent to a target, or to synthesize alternative views of objects.Real time panoramic mosaicing using clusters of such tracked points.A module computing a 3D reconstruction of the viewed scene from the set of features declared reliable by the tracker in 2D imagesUse of motion data to stabilize the vehicle relative to a structure in the field of view.
The former are useful tools for post processing AUV gathered video data for qualitative and quantitative analysis of subsea structures and seabed objects. The latter is relevant for AUV docking and accurate relative position keeping during inspection.
We consider sonar and video processing in turn.

Sonar Tracking and Navigation
We propose a system that builds a map of features in the environment, localises it-self with respect to these features, and avoids obstacles. The system uses standard deadreckoning sensors and a mechanical forward looking sonar. Our previous research [1-3] presented different approaches for tracking returns from a forward looking sonar, for the purpose of classification and obstacle avoidance. Our current research tracks the position of objects in the environment and that of the vehicle itself, with respect to the world reference frame, to build a stochastic map [4] and allow the vehicle to navigate in this map whilst avoiding obstacles.