Edge Feature and Optical Flow Terrain Aid for GNSS Denied Airborne Visual Navigation

Conventional autonomous aircraft navigation relies on an inertial measurement unit and Global Navigation Satellite System (GNSS) to correct for time accrued dead reckoned drift. Inertial measurements are used to accurately track fast changes, whereas GNSS signals prevent the localisation solution from deteriorating over time due to inertial drift. This reliance on GNSS risks disorientation of the autonomous vehicle due to GNSS failure from atmospheric events, jamming or satellite destruction. This would result in an inability to navigate through GNSS denied environments. Navigation methods which do not require GNSS predominantly rely on active sensors, such as radar or laser range finders, which impose critical platform restrictions due to weight, volume and power draw. Furthermore, reliance on active sensors will prevent any stealthy operations of an aerial vehicle. For a human pilot, vision plays a major role in the methods by which they use to navigate an aerial vehicle. These navigation techniques, such as Visual Flight Rules enable a pilot to localise an aircraft without reliance on GNSS or any active sensors by recognising visually distinct features. These features, or landmarks, may be associated with a feature map to determine location, orientation, altitude and speed. Visually distinct features may include point landmarks such as buildings, however will more generally consist of curve features, such as lake or river edges, forest boundaries or roads. Furthermore, a pilot may also use the shape of underlying terrain features such as hills and valleys as localisation aids. This thesis outlines the development and implementation of a computer vision based autonomous navigation system. This system employs the same visual navigation techniques a human pilot would, using curve and surface based landmarks. Visual information is fused with inertial information in a probabilistic state filter in order to minimise the dead reckoning drift from pure inertial integration. This visual system is separated into two main localisation research sections; curve feature tracking and terrain surface matching. Curve feature tracking allows curves such as edges to be estimated, tracked and associated with known features for position updates. Terrain surface matching involves the estimation of the local terrain profile. This estimate may then be associated with a known terrain profile, such as a digital terrain elevation map (DTEM) for position updates. The Simultaneous Localisation and Mapping (SLAM) process is a method by which newly detected features may be used for navigation, even through unknown environments. The first research area of this thesis involves the application of the SLAM algorithms to spline features in the full six degrees of freedom aerial navigation case. Image processing techniques are outlined which allow regions of differing ground type to be determined, in order to detect edge curve features. The methods by which splines are used to characterise features, and the mechanics of how they are updated through multiple measurements, are outlined in detail. Furthermore, this thesis shows how the range to detected features may be estimated. Methods which help to improve the robustness of the feature detection algorithms, as well as the estimated SLAM spline feature map and data associations, are also outlined. i Although navigation in unknown environments is an interesting topic of research, there are few real world environments which are truly completely unknown. A number of freely available databases, such as Google Earth, provide convenient sources of information which may be used to localise an aircraft. This thesis demonstrates the segmentation of aerial imagery retrieved from Google Earth into edge features. These features are then assembled into a data compact database of spline features, which is used for Visual Terrain Aided Navigation (VTAN) allowing accurate localisation of the aerial vehicle. The second research area of this thesis involves the use of Digital Terrain Elevation Maps (DTEM) Some of these maps describe the shape of most of the Earth's landmass, such as that produced by the Shuttle Radar Topography Mission. These describe the terrain profile, such as hills, cliffs and valleys. For human pilots, this information is often provided in the form of a topography map. A pilot can estimate the shape of surrounding terrain by observing changes in the apparent relative speed of terrain as it moves by. Comparisons may then be made between this and the topography map, helping to localise the vehicle. This thesis presents a method by which movement rates of imagery in a camera frame (optical flow) can be used to estimate the shape of underlying terrain. The estimated terrain contour shape is then compared to a freely available DTEM, allowing localisation of the vehicle. Optical flow is simultaneously used to provide visual odometry information, estimating the vehicle velocity. Some visually distinct features may change over time, such as water body boundaries changing with varying water height. This can cause navigation problems, as localising using these features may give biased position estimates. Considering river or lake edge changes can be expected to follow the shape of the terrain, a Digital Terrain Map may be used to predict this change. Should changes in water level be known (such as through water storage reports) changes in edge feature location can be accounted for, as demonstrated in this thesis. This improves the reliability of the presented visual navigation system. Conversely, should the water level change not be known, a method of using the terrain map to estimate this offset is established. This allows the possibility of aerial monitoring of remote river or lake ecosystems. The outlined system is demonstrated to greatly improve navigation accuracy in unknown environments using real flight test data. Terrain assistance associations with a known curve feature map are shown to provide visual localisation accuracies of under 20 metres. Finally, optical flow based terrain contour matching techniques result in localisation accuracies approaching those of edge feature terrain assistance, without requiring the use of a pre processed feature map.