# Netting and Background Distinction with In-Flight Optic Flow

Xavier Daïni, Romain Raffin, Thibaut Raharijaona, Franck Ruffier

► To cite this version:

Xavier Daïni, Romain Raffin, Thibaut Raharijaona, Franck Ruffier. Netting and Background Distinction with In-Flight Optic Flow. Journal of Bionic Engineering, 2025, 22 (3), pp.1143-1166. 10.1007/s42235-025-00677-w. http://hal-04966013v3

HAL Id: hal-04966013

https://hal.science/hal-04966013v3

Submitted on 19 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Netting and Background Distinction with
In-Flight Optic Flow

Xavier Daïni¹*, Romain Raffin², Thibaut Raharijaona³,
Franck Ruffier¹

¹ISM, Aix-Marseille Université, CNRS, Marseille, 13009, France.

²LIB, EA 7534, University of Burgundy, Dijon, 21078, France.

³University of Lorraine, LCFC, Arts et Métiers Institute of Technology,
HESAM Université, F-57000 Metz, France.

*Corresponding author(s). E-mail(s): xavier.daini@univ-amu.fr;
Contributing authors: romain.raffin@u-bourgogne.fr;

thibaut.raharijaona@univ-lorraine.fr; franck.ruffier@univ-amu.fr;

Abstract

Web-like obstacles, such as safety nets, represent a unique hazard for drones, and especially UAVs (Unmanned Aerial Vehicles). Fencing and netting are particularly difficult to distinguish from the background using either computer vision, lidar and sonar. In contrast, animals such as flying insects may detect these web-like obstacles using Optic Flow (OF), and more precisely motion parallax. A netting-avoidance solution was proposed using a OF-based detection method. The netting-detection method was based on a signature defined by the shape of the OF magnitude across the visual field. We established that the OF shape depends on the orientation of the netting in relation to the hexarotor's movement. This paper demonstrates netting detection in real-world experiments, according to any direction flight made by the UAV along the net. The proposed NOWA method (which stands for Netting Optical-flow-based distinction Algorithm) separates the OF signatures belonging to these different surfaces -netting or background- whatever their orientations. By extracting the OF signatures of these different surfaces and separating them, the proposed visual method can estimate their relative locations and orientations. In a robotic simulations, the multirotor explores and navigates automatically using this netting detection method, using saccades to avoid obstacles. In the simulations, these saccades are also used to simplify netting detection by orienting itself systematically parallel to these planes, a behavior reminiscent of flying insects.
Introduction

The autonomous exploration of an unknown environment remains one of the main
challenges facing today's Unmanned Aerial Vehicles (UAVs). During exploration, an
UAV may encounter numerous obstacles, some of which are difficult to detect using
conventional collision avoidance methods. Whether indoors or outdoors, a safety-net
remains very difficult to detect with precision using the most advanced methods, such
as computer vision or Lidar:

* Conventional computer vision methods are based on solid, opaque surfaces: therefore pattern recognition algorithms can be used to detect windows frames [1].

* Some recent studies tackle the problem of semi-transparent surfaces using Lidar [2–5].

Other approaches rely on the use of sensor fusion to detect glass [6]. During operating
flight, UAVs may have to fly in an environment where semi-open fences or open grids
are present, or in the presence of netting (such as on a construction site). Animals use
Optic Flow (OF) – visual angular speed– for both navigation and obstacle avoidance
[7–9], including insects, to avoid spider webs [10].

Based on numerous observations, particularly in insects, several authors have pro-
posed OF-based strategies for autonomous exploration [11], terrain following [12, 13],
landing [12–14] and, very recently, visual stabilization without an accelerometer in
a flapping-wings UAV [15]. Previously, a UAV’s control strategy relying on OF that
includes insect-like saccadic behavior, has already been simulated on a hovercraft (a
2D vehicle) using a limited number of OF sensors and cues [16]. However, in [16], the
general shape of the OF set across the visual field was neither used as a cue to detect
nor follow surfaces. These studies have always been based on a single-layer hypothesis,
where no semi-transparent obstacles were present between the UAV and the opaque
plane it followed [17].

To detect several surfaces, previous studies have relied on piece-wise planar approx-
imations of the environment. The approach used in this study supposes en environment
made of a series of oriented planes, as seen in [18].

In computer vision, common OF extraction methods are based on a limited number
of 2D-points denoted as *corners* or *fiducial points* [19, 20], while other authors focus
on a dense vector field where the OF vectors are estimated at each pixel [21, 22].

As OF can be defined as the perceived angular speed in a visual scene [23],
the motion parallax can be defined by the existence of a discontinued or superposed
collections of OF profile in the visual scene, as for example, the foreground and the
background.

To the best of our knowledge, we proposed here the first generic netting detection
method. The generic method takes advantage of the motion parallax to detect several
surfaces and to assess orientations and locations of multi-layer surfaces and we tested
this generic method on a UAV.

In a preliminary paper presented at a conference, we demonstrated OF-based net
detection when a UAV flies parallel to surfaces, with no incidence angle [24]. The axis
of the camera measuring the OF was always oriented perpendicular to the surface –
netting and wall– [24].

This present paper extends this OF-based netting detection for any orientation
between the camera axis and the surfaces. A new improved generic netting-
distinguishing method, called NOWA (which stands for Netting Optical flowW-based
distinction Algorithm) is detailed in section 4. The NOWA method separates the OF
vectors of one surface from another and is robust to various incidence angles. The the-
oretical framework was developed to assess the localization and orientation (section
5) of the perceived wall and netting.

Section 1 describes the properties of the OF equation, its 3D expression in the UAV frame of reference, and identifies the OF cues used to define the signatures of planar surfaces.

Section 2 focuses on the specific signatures of planar surfaces, emphasizing the differ-
ence in amplitude between the netting and background OF shapes.

Section 3 details the material used in the experiments (both real and simulated), from
the Hexarotor and image recording, to the OF extraction.

In section 4 tests show that surfaces, including safety-nets, can be distinguished,
in several configurations. Three possibles surfaces configurations were distinguished:
single surface, netting in front of a surface, netting in front of any type of background.
Section 5 shows the relative angle and distance estimation of both the netting and the
surface seen through this same netting.

Finally, section 6 presents an exploration autopilot relying on the OF-based detection of surfaces and their orientation, and distance estimation. This OF-based autopilot enables an UAV to explore an unknown environment made up of planar surfaces, netting or both.

# 1 OF Theory

This section describes the equations, the visual principles, and the material used in
all the real-world experiments we carried out (Fig. 1). To simplify the study and add
some generalities, typical cases were determined : *Case 1. Flight direction parallel to surfaces, Case 2. Any flight direction with parallel surfaces and Case 3. Any flight direction with any two surfaces.*

## 1.1 OF Definition and Extraction

OF is an optic property based on movement vectors created by a moving observer
facing a static environment (Fig. 1a). The theoretical definition of the OF vector
$\omega(\theta, \phi)$ can be expressed with equation (1).

To use the definition (1), we decided to use the discrete vectors extraction type
with a Shi-Tomasi corner extraction algorithm [20] and with a Lucas-Kanade OF
computation [19], the parameters of which can be founded in Supplemental Information section S1 Table S1.

We based the computation of our algorithm on the python-opencv version of the
Lucas-Kanade<fnref n="1" /> in conjunction with a Shi-Tomasi corner-detector.

The extraction of a 3D vectorial OF vector from a pin-hole camera is not straight-
forward (Supplemental Fig. S1). As a retina is a planar object, the 2D expression of
the OF (Eq.(1)) is back-projected onto a 3D space (Supplemental Information section
S2). In order to obtain a 3D expression, the camera's parameters must be taken into
account [25], as well as the sensor's properties [26]. The computations in the frame
used in the present paper are detailed in Supplemental Information section S2 (is
described in [27]).

The back-projection of the 2D OF in pixels/s on the sphere produces a third
component along the *Y* – *axis*, the depth axis. From this back-projection, only the
projection of OF vectors along the two axes (*X*,*Y*) will be used to define the signature
of the planar surfaces to be detected.

1.2 OF Theoretical Expression

<figure><img src="image_2.png" /><figcaption>**Fig. 1:** (a) Simplified representation of the UAV and the Optic Flow acquired in front of an opaque surface.(b) In the present work, the magnitude $\tilde{\omega}$ of the OF is represented on the *Y* – *axis* and the azimuthal angle of observation *θ* on the *X* – *axis*. (c) shows the important variables of the Koenderinck and Van Doorn OF expression (Eq.(2)) in a Top-Down view.</figcaption></figure>

In the present work, $\omega(\theta, \phi)$ represents the OF vector at $(\theta, \phi)$. The OF vector is determined by the Movement vector (Translation **T** and Rotation **R**), the distance to the point considered $D(\theta, \phi)$ and the direction vector $\mathbf{d}(\theta, \phi)$, by the following equation (using the frames and definitions of Fig. 1c):

<footnote n="1">¹https://docs.opencv.org/3.4/d4/dee/tutorial_optical_flow.html</footnote>
$$ \omega(\theta, \phi) = -\frac{1}{D(\theta, \phi)} \cdot (\mathbf{T} - (\mathbf{T} \cdot \mathbf{d}(\theta, \phi)) \cdot \mathbf{d}(\theta, \phi)) - \mathbf{R} \times \mathbf{d}(\theta, \phi) \quad (1) $$

## Translational flight hypothesis

The **R** component is null when the UAV flies in pure translation, between two rotations: these rotations are called saccades in insects flight. Therefore the translational OF is defined as follows:

$$ \omega(\theta, \phi) = -\frac{1}{D(\theta, \phi)} \cdot (\mathbf{T} - (\mathbf{T} \cdot \mathbf{d}(\theta, \phi)) \cdot \mathbf{d}(\theta, \phi)) \quad \text{with } \mathbf{R} = 0 \quad (2) $$

In this hypothesis, the translational OF depends solely on the translation vector **T** of the UAV, the orientation and distance of the environment.

At each time step ($\delta t = 100$ms) a new set of OF measurement is obtained. Each $\omega_i$, expressed in $[\text{rad.s}^{-1}]$, reference the element $i$ of the set, corresponding to angles $(\theta, \phi)$ from Eq. (2):

$$ \omega_i = -\frac{1}{D_i} \cdot (\mathbf{V} - (\mathbf{V} \cdot \mathbf{d}_i) \cdot \mathbf{d}_i) \quad \text{with} \quad \mathbf{V} = \frac{\mathbf{T}}{\delta t} \quad (3) $$

We introduced the expression of the linear speed vectors **V** (projection of the vector **T**) and direction of observation $d_i$ in the case of a $(X, Y)$ 2D movement (Fig. 1c). Therefore $\omega_i$ can be expressed as follows:

$$ \omega_i = -\frac{1}{D_i} \cdot \left( \begin{bmatrix} V_X \\ V_Y \end{bmatrix} - (V_X d_{iX} + V_Y d_{iY}) \cdot \begin{bmatrix} d_{iX} \\ d_{iY} \end{bmatrix} \right) \quad (4) $$

Finally, the list of OF measurements norm $\tilde{\omega}_i$ and its X-axis component $\omega_{iX}$ given by the Lucas-Kanade algorithm (Fig. 1a,b) depend on **V** and $D_i$, as follows:

$$ \omega_{iX} = -\frac{1}{D_i} \cdot (V_X - (V_X d_{iX} + V_Y d_{iY}) d_{iX}) \quad (5) $$

$$ \tilde{\omega}_i = \frac{1}{D_i} \cdot \sqrt{[V_X - (V_X d_{iX} + V_Y d_{iY}) d_{iX}]^2 + [V_Y - (V_X d_{iX} + V_Y d_{iY}) d_{iY}]^2} \quad (6) $$

In section 4, the OF norm measurements were fitted with a Gaussian function, in order to cluster the OF, thus avoiding the complexity of Eq. (6). In section 5, the projection of OF measurements along the X axis was fitted using the expression $\omega_{iX}$, in order to directly estimate the relative orientation $\alpha$ on which $D_i$ in (5) depends (see section 5.1.1 for the specific equation). The magnitude of the OF measurements in the center of the FOV is used to estimate the relative distance $D$ (see section 5.1.2 for the specific equation).
## 2 Difference between the OF Generated by the Netting and the Planar Surface

<figure><img src="image_3.png" /><figcaption>Fig. 2: Theoretical OF set obtained from different scenarios.</figcaption></figure>

a.i) A single distribution can be obtained, from a.ii) only a single wall.

b.i) A very close distribution can also be obtained from b.ii) a netting (square- or diamond-shaped) tautened between two structures.

c.i) Two distributions can be obtained from c.ii) a netting and a planar surface behind it.

d.i) A single distribution with an addition of several, un ordened OF vectors, can be observed from d.ii) a single netting with an non-planar background.

See Supplemental Videos S1 & S2 for in experiment OF shapes.

### 2.1 Single planar Surface

#### 2.1.1 Single Wall surface

The OF norm expression, when created by a movement along a single surface Fig. 2a is shown in Appendix A and is recalled here:

$$ \tilde{\omega}(\theta) = K \cdot \cos(\alpha + \theta) \cdot \cos(\theta - \psi_v) \quad (7) $$

Expression (7) is very powerful, however it has some drawbacks that can be corrected by the usage of a Gaussian approximation :

$$ f(\theta) = A \cdot e^{-\left(\frac{\theta - m}{\sqrt{2}\sigma}\right)^2} \quad (8) $$

A Gaussian approximation Eq.(8) has several advantages:

* With a single maximum and extremities tending towards zero, the Gaussian function naturally defines a range in which the data has the greatest amplitude. This characteristic of the Gaussian function is particularly well suited to our approach to find several planar surfaces in a given FOV.
* The norm $\tilde{\omega}$ and projections $(\omega_{iX}$ and $\omega_{iY})$ of the OF perceived along a planar surface can be efficiently approximated by a Gaussian function

* The number of elementary operations is also lower with a Gaussian function than with a product of trigonometric functions. For instance, fitting<fnref n="2" /> $\omega_{iX}$ with its formal mathematical expression (given in section 5) takes twice as long as the same fit made with the Gaussian function for 1000 data.

We make the assumption that, in the case of a single planar surface, the OF set only come from this surface (with the possible addition of noise) and therefore be gathered into a single F set:

$$ \forall i, \tilde{\omega}_i \in F \qquad (9) $$

This single set F (Fig. 2b) can be fitted by a single function f defined Eq.(8).

### 2.1.2 Single Netting Surface against the sky

The presence of a single netting against the sky – tautened between two structures with no background – creates the same OF signature as that of a single planar surface (Fig. 2a.i & b.i) with no background OF.

This scenario was successfully treated in a dedicated simulation (as our lab is not authorized to fly UAV outdoors) and shown in Supplemental Information section S3.4.2, Figure S3.

## 2.2 Netting in Front of a Planar Surface

We have defined the motion parallax as a collection of spatially discontinuous or overlapping OF profiles in the visual scene. In the case of several planar surfaces or netting (Fig. 2c), this collection of OF profiles corresponds to equations Eq.(3) on local spatial regions and/or magnitude intervals.

As a netting allows OF cues from the background behind to be extracted, two juxtaposed OF distributions will appear: the foreground (fg) and the background (bg) (Fig. 2d).

This OF juxtaposition can then be written as:

$$ \forall i, \tilde{\omega}_i \in F_{fg} \Delta F_{bg} \quad (F_{fg} \cap F_{bg} = \emptyset) \qquad (10) $$

Where $F_{fg}$ and $F_{bg}$ are OF sets, and $\Delta$ the symbol for the symmetric difference function<fnref n="3" />.

Therefore, Eq. (10) defines to 2 OF sets $F_{fg}$ and $F_{bg}$, each with a specific Gaussian functions (Fig. 2d foreground $f_{fg}$ or background $f_{bg}$) with:

* $f_{fg}(\theta) = A_{fg} \cdot e^{-(\frac{\theta-m_{fg}}{\sqrt{2}\cdot\sigma_{fg}})^2}$

* $f_{bg}(\theta) = A_{bg} \cdot e^{-(\frac{\theta-m_{bg}}{\sqrt{2}\cdot\sigma_{bg}})^2}$

<footnote n="2">²https://docs.scipy.org/doc/scipy/reference/optimize.html</footnote>

<footnote n="3">³https://en.wikipedia.org/wiki/SymmetricDifference</footnote>
Although OF profiles can be very different depending on experimental conditions (speed, distance, inter-distances, relative orientations, flight direction, etc.), Eq. (7) allows to define just three typical cases in section 2.5.

## 2.3 Netting in Front of Non-Planar Background

In the case of a single planar netting (Fig. 2e), OF come from both the netting and the unstructured non-planar background. We made the assumption that, in the case of a netting, the OF is a juxtaposition of the OF signatures from the netting ($F_{fg}$), and the background behind ($F_{bg}$).

Eq. (10) defines only one region of the OF field behaving as a Gaussian function, the *foreground* $F_{fg}$, with:

$$f_{fg}(\theta) = A_{fg} \cdot e^{-(\frac{\theta - m_{fg}}{\sqrt{2} \cdot \sigma_{fg}})^2} \quad (11)$$

When the background is not planar, the OF set $F_{bg}$ has no specific shape, and therefore no signature (Fig. 2f).

In the scenario of a poorly contrasted background, the background would create a lot of OF noise, rendering distinction impossible (see Supplemental section S3.4).

## 2.4 Spatial Frequency Property

Another important feature of netting is their repetitive pattern. Because of this repetition, one might think that the spatial distribution of measured OF vectors would tend to follow a regular pattern Fig. (2c,d). However, this spatial property is mesh-type dependent: for example, in the case of diamond-shaped mesh, there is no spatial frequency signature (Fig. 2 b.ii).

Despite this, a discrete Fourier transform (DFT) was performed on the histogram of the $\theta$ angles at which the OF vectors were calculated (Supp Info Fig S5) to detect a square-shaped net where this specific spatial distribution is present. Supplemental Information Tables S2 & S3 show mesh-specific (not generic) results of netting detection by spatial frequency analysis, solely valid for the square-shaped mesh example.

Therefore, in the rest of this study, we focused on shape-based analysis to qualify the presence of a netting and the type of background.

## 2.5 Three Cases of Surface Orientations Characterizing the Signature Shape of the OF Set

In order to simplify the study, some typical 2D cases have been defined. Fig. 3 shows how the orientation of the surfaces shapes the set of the OF norm thus creating different OF signature.

We can list three main cases of surface(s) orientation(s):
**Case 1:** *Flight direction parallel to surfaces:* all the shapes are centered around 0°, when $\alpha \approx 0$, $\psi_v \approx 0$ and $\alpha \approx \alpha_2$, defining a parallel flight direction to the surfaces plane.

**Case 2:** *Any flight direction with parallel surfaces:* all shapes are shifted to the same side, when only $\alpha \approx \alpha_2$, defining an oblique direction of movement w.r.t. the surfaces plane.

**Case 3:** *Any flight direction with any orientation surfaces:* any two shapes ($\alpha \neq \alpha_2$) each defining an oblique flight direction.

<figure><img src="image_4.png" /><figcaption>**Fig. 3:** For each Case, (i) is an example of surfaces relative orientation and UAV's movement generating an OF shapes signature (ii) typical of each cases. As the movement is always mainly along the X-axis of the camera, the evolution of $\omega_X(\theta)$ and $\tilde{\omega}(\theta)$ are similar.</figcaption></figure>

Furthermore, *Cases 1 and 2* are defined with and with no netting, while *Case 3* with no netting corresponds to *Case 2* as it can be seen on Fig. 3.

# 3 Material & Methods

## 3.1 The Hexarotor

We designed the experimental environment to receive a hexarotor, developed with Hexadrone™. The UAV was controlled by a PX4 low-level flight controller [28]. The
<figure><img src="image_5.png" /><figcaption>**Fig. 4:** (a) Photograph of the hexarotor with its fixed, onboard, camera and Field-Of-View. (b) Zoom on the camera's optic. (c) Image of the onboard camera during a flight in the flying arena and (d) sample image captured by the camera with camera-frame definition as seen from the camera. (e) Image of the environment used in *Case 3*. (f) Environment used for all the single-wall scenarios.</figcaption></figure>

hexarotor's trajectory was in a closed loop recorded by a motion capture system based on 19 cameras.

Fig. 4a shows a top-down view of the UAV, with its camera and Field-Of-View. Fig. 4c shows the UAV flying in the arena.

The UAV was equipped with a Jetson Xavier NX, as onboard computer. It was connected via USB to a low-cost global-shutter camera<fnref n="4" />, visible in Fig. 4b. This camera

<footnote n="4">⁴ELP USBGS720P02-L36 with a 3.6 mm objective</footnote>
was used to store images at a frequency of $f_{cam} = 10Hz$, as in Fig. 4d, due to hardware limitations. While low, such frame rate can be sufficient to extract OF [15].

## 3.2 Camera and Images

The chosen global-shutter camera produced images (Fig. 4d) of size 1280 × 720 on a planar retina, corresponding to a Field-Of-View (FOV) of 60°.

The camera's optic deformations must be taken into account. The larger the FOV, the greater the impact on the final image. A first step of camera calibration has to be performed before any images can be taken<fnref n="5" /> and a scaling factor $k$ will be introduced in section 5 for orientation estimation.

## 3.3 Camera Field-Of-View (FOV) and surface angle estimation

A minimal FOV is needed for the camera to determine the orientation and distance of the environment based on OF. In order to estimate this minimal FOV, the following formula is used (see Appendix B and [13] for detailed expression):

$$ \alpha = -2\theta_1 + \psi_v \qquad (12) $$

Where:

* $\alpha$ is the relative orientation of the plane with respect to the retina of the observer,

* $\psi_v$ is the angle made by the speed vector with respect to the retina of the observer,

* $\theta_1$ is the angle $\theta$ at which the OF magnitude $\tilde{\omega}$ of set 1 is maximum, and $\theta_2$ for set 2 in Fig. 3 when facing several sets.

With equation 12, the limits of the suggested plane detection method can be assessed. **Case 1** (Fig. 3) is the most favorable case to detect surfaces, based on OF as will be shown in section 4. In **Case 1**, $\psi_v = 0$, thus Eq. (12) becomes $\theta_1 = \alpha/2$.

Such expression imposes a limit on the relative surface orientation ($\alpha$), which depends on the FOV. Specifically $\theta_1$ needs to be in the FOV to ensure accurate estimation.

In addition, we have constrained the OF extraction within a region of interest (called here central band of interest) in order to obtain a series of OF vectors, located as close as possible to the horizontal line of the image. The norm $\tilde{\omega}$ of the OF vector was calculated from the X, Y components, excluding the Z component, to remain within the framework described in section 1.2, where the theoretical OF results from an infinitesimal horizontal band at the center of the image.

## 3.4 Real-world Experiment with Hexarotor in the Flying Arena

The flights were realized in a Flying Arena of 6 × 8 × 6 cubic meters, equipped with 19 motion capture cameras (Vicon) used to measure a ground-truth position and orientation of a defined object inside. For the exploration, the UAV was controlled by a trajectory tracking algorithm relying on the motion capture system 6. The walls of

<footnote n="5">⁵https://docs.opencv.org/4.x/dc/dbb/tutorialPYCalibration.html</footnote>

<footnote n="6">⁶https://github.com/gipsa-lab-uav/trajectory_control</footnote>
the Flying Arena were lined with textured panels (Fig. 4c-f) to facilitate the extraction of OF.

## 3.5 Computer Simulation with Gazebo

In order to handle a wide range of parameters and situations, we constructed a dedicated visual environment using Gazebo. We used a simulation made by linking Simulink and Gazebo-9 via ROS. The camera moved along a surface, oriented at 90° from the forward direction. Two simulations were carried out:

* In sections 4 and 5, the camera moved at a linear speed V = 0.25m/s.

* In section 6, the camera was attached to a fully actuated hexarotor that had its dynamics simulated via a set of non linear equations with a sliding surface control algorithm [29].

<table><caption>**Table 1:** Parameters of the simulated camera and the global shutter camera used in real experiments</caption><thead><tr><td>Specifics</td><td>Simulated camera</td><td>Real camera</td></tr></thead><tbody><tr><td>Pixel Length</td><td>0.78µm</td><td>3.6µm</td></tr><tr><td>Number of Pixels</td><td>1280 × 720</td><td>1280 × 720</td></tr><tr><td>FOV</td><td>120°</td><td>60°</td></tr><tr><td>Focal length</td><td>0.28mm</td><td>3.6mm</td></tr><tr><td>Framerate</td><td>10 Hz</td><td>10 Hz</td></tr></tbody></table>

In the simulation, we used a 120° FOV (Table 1) with the same image resolution as the real-world flight (1280 × 720 pixels).

The parameters used in both simulations and real-world tests (such as the Lucas-Kanade parameters, the band size, etc) can be found in Supplemental Information section S1.

## 3.6 Rejection of OF vector field extraction

### Translational hypothesis

The input of the generic NOWA method (Figs. 5 and 6) contains all the OF vectors extracted, as long as the hexarotor carried out only a pure translation, Eq.(2). To be precise, the OF vectors extracted are processed if the rotational speed, between two images, on any axis, was of less than 3°/s. Fig. 7d shows that this threshold is usually respected during a translation phase.

### Minimum number of OF vectors extracted

Another point for data rejection appears when the data are not numerous enough to allow for a fit. The proposed NOWA method needs at least 5 values to be conclusive.
## Fitting convergence

As the algorithms often makes use of fitting functions, a maximum number of computation steps prevents endless computations. The limited number of steps causes some vector fields not to complete their process, resulting in their rejection, as shown Fig. 7d (in red).

In the experiments, the maximum number of steps was $N = 10000$.

# 4 Generic Netting and Surface Distinction Algorithm, called NOWA

## 4.1 NOWA: Netting Optical flow-based distinction Algorithm

We made the assumption Eq. (10), which states that the OF signature coming from a *multi-layered* environment are a juxtaposition of the OF signatures sets coming from each layer (Fig. 2b,d,f). For convenience, we call the measured juxtaposition of OF distributions *OF set(s)*, and *OF cluster(s)* the groups of OF vectors created as part of the algorithm.

The generic NOWA steps, working on any flight direction, are detailed below in Fig. 5 as well as in Fig. 6 and in Supplemental Video S3:

*Step i. Extracting, Fitting, First outlier removing.* Acquisition of the OF norm $\tilde{\omega}_i$ and fitting to a theoretical shape. For the sake of computation, a single Gaussian function was first used: $f(\theta) = A \cdot e^{\left(\frac{\theta-m}{2\sqrt{\sigma}}\right)^2}$. A first outlier removal was then carried out to keep only relevant OF values (parameters given in Supplemental Information section S1).

*Step ii. Computing the fitting error.* In the case of a *single-surface* shape (Fig. 6d) the mean fit-error can be estimated as less than *n*% of the amplitude of the set. However, in the case of several layers, a larger mean error is expected (see Figs. 6a.ii and b.ii). Thus, if a small mean fit error is computed, an early conclusion of *single-layer* can be given.

*Step iii. Removing secondary outliers.* This step filters out the remaining outliers in the OF cluster that may appear in *multi-layer* sets, with the remaining data noted $\omega_i^{filt}$.

*Step iv. Data clustering.* Using a simple sign of fit-error, each vector norm can be assigned to a cluster noted $F_{fg}$ (the *foreground* region) and $F_{bg}$ (the *background* region) for each remaining $\tilde{\omega}_i$ with a positive and negative fit-error respectively Fig. 6a.iii.

*Step v. Secondary fitting.* As the norms are distributed in two groups, a mirror of step i, where each cluster ($F_{fg}$ and $F_{bg}$) is to be approximated by a Gaussian function ($f_{fg}$ and $f_{bg}$ respectively) is realized.

*Step vi. Re-clustering.* As an optional step, each vector can be reassigned with respect to the new functions obtained. To prevent endless loops, a fixed number of re-clustering is defined (usually 3).
*Step vii. Determining the presence of netting or wall surfaces.* Using two final sub-steps:

* Checking fitted OF clusters difference: If the two clusters are close to one another, it means that they both come from a single plane with some noise added. This is evaluated by using a threshold of n-% of difference in amplitude. As two sets coming from two parallel planes will have the same $m_i$ and $\sigma_i$ fitted, only the amplitude of the Gaussian $A_i$ is used in this step.

* Checking fg OF clusters signature: A first test is realized on $F_{fg}$ only to test its planarity. As two clusters were detected, if $F_{fg}$ does not behave like a netting, then the data are rejected. To check planarity, a new threshold $n_i$ was used.

* Checking bg OF clusters signatures: To conclude on the planar aspect of $F_{bg}$ a new fit error is computed. If this fit error is below the threshold (of $n_i$ %) then it is concluded that each set represents a true planar layer, thus the data represent a *multi-layer surface*. If not, then only $F_{fg}$ has a planar behavior.

The NOWA method has 3 possible outputs: “single surface”, “netting & non-planar surface” and “netting & planar surface”.

To ensure greater consistency of results, a moving filter with a span of 5 provides the most frequent surface detection in this window<fnref n="7" />.

## 4.2 Netting & Background Distinction during Flight Experiments

Several scenarios taken from Fig. 3 are studied in this section, using the algorithm Fig. 5. The UAV was given a trajectory that it had to follow, while its camera was recording images and the MoCap system was tracking its position in the case of the real experiments.

The generic NOWA method proposed here does not need the UAV's position and speed.

Experiments were realized with different lighting conditions (500 lux, 50 lux, <5 lux) and with different distances netting-wall (30 cm, 50 cm, 80 cm, 150cm, and single netting with no background wall) resulting in 6 different scenarios with 3 lighting conditions. Results for 3 scenarios (single wall, single netting and an inter-distance of 80 cm) are shown<fnref n="8" /> in Tables 2 for *Case 1* and 3 for *Case 2*. The full results for 6 scenarios are available in Supplemental Tables S2 & S3.

Tables 4 illustrates experimental results for *Case 3: Any flight direction with any two planar surfaces*, in two different lighting conditions, with an inter-angle of 25°.

Two main results appear:

<footnote n="7">⁷In cases of a tie between two of the three outputs, the most conservative results was prioritized, favoring a singular plane over other possibilities and an non-planar background over a planar one.</footnote>

<footnote n="8">⁸Note that in the case of a single wall the detection of a single surface means both no-netting and planar background, hence only one percentage in the tables</footnote>
<figure><img src="image_6.png" /><figcaption>**Fig. 5:** (a) Generic *Netting Detection Method* via the NOWA method. Details of each step are in the text, images from Fig. 6 are given as examples of OF shape for some key steps as well as a Representation of the Surfaces segmentation in case of two planar surfaces. During the experiments shown in this paper, the parameters were chosen as follows: $n = 5\%$, $n^{-} = 10\%$, $n_{i} = 15\%$, $p^{+} = 1.75$ and $p^{-} = 0.25$.</figcaption></figure>

A video showing each step, applied to two images acquired during a real-world experiment, is provided in Supplemental Video S3.
<figure><img src="image_7.png" /><figcaption>**Fig. 6:** Example of OF processing for netting and surface detection. Two cases are presented, one (a) where the UAV moves along a *multi-layer* (2-layers) surface with a netting and a wall, and another (b) where a single plane is viewed. The OF band considered is shown in red. The OF magnitudes are presented in a normalized unit, to facilitate comparison between the two cases. (c) and (d) are the fitting error for either the two clusters (cluster ① and ②) in (c) or the single cluster (cluster ③) in (d). This way of representing the results (2 boxplots representing, on the left the 2 layers, and on the right, a single wall detected) will be used in the paper.</figcaption></figure>
Case 1

<figure><img src="image_8.png" /></figure>

Sample of experimental shapes of OF magnitude in FOV

Raw True Netting detection % based on OF shapes (background detection %)

**Filtered True Netting Detection %** (background detection %)

[Nb of processed frame]

<caption>**Table 2:** Accurate detection results for different scenarios of *Case 1*. Each OF data insert represents a different scenario: the rows are the setups and the columns, the different lighting conditions. The setups images are views from the UAV at 50 lux. For each inserts, a sample of experimental normalized OF norm shape is presented with OF segmentation given by the NOWA method: • foreground vectors, • planar background vectors, • non-planar background vectors. Below each inserts are the percentages of accurate raw instantaneous detection of foreground and in parenthesis the accurate detection of both foreground and background</caption>

The filtered detection are given in **bold** and are the final outputs of the algorithm.
We also show in brackets the number of vector fields on which the algorithm was
performed to obtain accuracy estimation.

Due to the flight arena geometric constraints (Fig 7c), in the scenarios with a “single wall”, the distance UAV-plane was between 1.7 and 1.8m. In the last 2 row of scenarios (“Netting and planar background” and “Netting and unstructured background”, the distance UAV-netting was between 1.8 and 2.2m.

Supplemental Fig. S3 shows the small inserts with higher resolution.
## Case 2

Sample of experimental shapes of OF magnitude in FOV

Raw True Netting detection % based on OF shapes (background detection %)

Filtered True Netting Detection % (background detection %)

[Nb of processed frame]

### 3 Scenarios

<figure><img src="image_9.png" /><figcaption>**Table 3:** Accurate detection results for different scenarios of *Case 2* with the same definition as Table 2.</figcaption></figure>

For each insert, a sample of experimental normalized OF norm shape is presented with OF segmentation given by the NOWA method: • foreground vectors, • planar background vectors, • non-planar background vectors.

Experimental configuration: $\alpha = 0$ and $\psi_v \approx -45^\circ$. OF shapes have been acquired with the UAV at distances $\approx 2.5$ m from the foreground surface.

* First, the NOWA method works well even under challenging lighting conditions. In complex cases, accurate background characterization appears highly dependent on lighting, whereas foreground netting detection remains robust, achieving at least 80% accuracy even under the very low lighting conditions (< 5 lux).

* Second, performances remain particularly consistent from an inter-distance of at least 50 cm as demonstrated in Supplemental Tables S2 & S3 with an accuracy > 84% at 500 lux. Below an inter-distance of 50 cm, the two OF shapes tend to merge, making it more difficult to distinguish them.
## Case 3

Sample of experimental shapes of OF magnitude in FOV

Raw True Netting detection % based on OF shapes (planar bg - non-planar bg %)

Filtered True Netting Detection % (planar bg - non-planar bg %)

[Nb of processed frame]

### 2 Scenarios

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

[49]

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

[38]

**Table 4:** Accurate detection results for 2 relevant scenarios of *Case 3* with the same definition as Table 2 & 3.

Experimental configuration: $\alpha_1 = 0$, $\alpha_2 = 25^\circ$ and $\psi_v \approx 0^\circ$. OF shapes were acquired with the UAV at distances $\approx 2.5$ m from the foreground surface and inter-distance ranging from 50 cm to 150 cm.

## Netting Detection in Simulation

A simulated environment was also designed and will be used for navigation in section 6. For comparison, the Generic NOWA method was also used on simulated flight in the environment. The simulation results Table S4 can be founded in Supplemental Information section S3.

In summary, simulated images result in very accurate netting detection performances with at least 88% accuracy for all three cases.

## Netting Detection over a flight

Figs. 7c shows a representation of trajectory, corresponding for some parts to *Cases 1 and 2*, in a top-down view with colored dots as netting detection results (the data acquired are available in Supplemental Information and [30]). Furthermore, netting detection during the simulated exploration, which gathered all three cases, can be seen in section 6.

Supplemental Videos S1 and S2 show the view from the onboard camera, the OF acquired and the shape clustering, adding an external bird-view during its flight. Furthermore, supplemental video S1 shows that the netting detection method is robust against "OF gaps".
# 5 Surface Localization

An extension of this work consists in estimating both the distance to and the relative orientation of the netting and/or wall.

In the previous section only the norm of the OF $\tilde{\omega}_i$ was used. While the norm usually gives better results than a specific component, this section will also make use of one component $\omega_{iX}$. As the angle $\alpha$ is defined in a specific plane $(X, Y)$ (Fig. 1c) it reduces estimation error to use the OF from this plane $(X, Y)$.

Furthermore, the largest component of the OF in this plane shows less relative noise than any other when estimating the angle $\alpha$.

Thus, in this section, as the movement will be mainly along the $X$-axis ($\psi_v < 45^{\circ}$), $\omega_{iX}$ will be the less noisy OF cue to fit (Fig. S.1a) to obtain $\alpha$.

## 5.1 OF based Estimation of $\alpha$ and $D_0$

Fig. 1c gives the definitions of the orientation and distance parameters that will be used in the following equations. The distance UAV-Environment and the relative orientation of the UAV with respect to the environment, needed for spatial reconstruction, are: $D_0$ and $\alpha$. These are the only parameters needed during a 2D experiment (i.e. no vertical movement if following a wall horizontally).

Considering only $\phi = 0$ along the central band of the image (Fig. 1c) results in a simplification of the vectorial expression Eq.2. The computations are given in Appendix A.

### 5.1.1 Relative Orientation $\alpha$ Estimation by Fitting $\omega_{iX}$

It has already been shown that the study of the $X$-axis component of the OF vector is sufficient to get information about the distance from the wall [24]. Yet this study dealt with only a trivial case with no incidence angle ($\psi_v = 0$) nor orientation error ($\alpha = 0$). In order to generalize the fitting function, it is necessary to take these new parameters into account and thus to take another look at the continuous definition of the OF as detailed in Appendix A.

By taking only the component of the OF on the $X$-axis the following expression is obtained:

$$ \omega_X(\theta, 0) = A \cdot \cos(\alpha + \theta) \cdot \cos(\theta) \cdot \cos(\theta - \psi_v) \quad (13) $$

Where:

* $\omega_X(\theta, 0)$ is the $X$-axis component of the OF vector (noted $\omega_{iX}$ for the discretized OF).

* $A$ is a fitting coefficient. Its expression can be found in appendices A,B.

* $\psi_v$ is the direction of the speed vector $V$ with respect to the camera's retina plane.

* $\alpha$ is the relative orientation of the faced wall with respect to the camera's retina.

* $\theta$ is the 2D angle at which the OF is obtained.
To find the value of $\alpha$, a least square adjustment<fnref n="9" /> has been used, knowing the parameters $\psi_v$, and outputting an Amplitude coefficient $A$ and an estimation of $\alpha$.

To reduce the impact of the optical error measurement due to the camera, it is possible to replace $\theta$ by $k \cdot \theta$. The correction $k$ will also need to be fitted.

By requiring a new function to be fitted, more data loss can be expected as this fit will also have a limited number of iterations allowed. The possibility to use other OF components to address this data loss is discussed in Supplemental Information section S4.

### 5.1.2 Calculating Distance $D_0$ Estimation from $\tilde{\omega}_0$ and Vector V

In order to estimate $D_0$, as defined in Fig. 1, the expression of the OF Eq.(2) at a given angle $(\theta, \phi)$ is used.

Studying the problem at $(\theta, \phi) = (0, 0)$ simplifies the computation as it is $\alpha$-free (cf Supplemental Fig. S5, $D_\theta$ depends on $\alpha$ but not $D_0$). We have made the computations available in Appendix C so anyone can obtain the expression of $D_0$:

$$D_0 = \frac{\tilde{V}}{\tilde{\omega}_0} \cdot \sqrt{\cos(\gamma)^2 \cdot \cos(\psi_v)^2 + \sin(\gamma)^2} \quad (14)$$

Where:

* $D_0$ is the distance between the camera and the observed point at $(0, 0)$.

* $\tilde{\omega}_0$ is the norm of the OF vector at $(0, 0)$.

* $\tilde{V}$ is the norm of the speed vector of the UAV.

* $\psi_v$ is the orientation of the speed vector with respect to the camera's retina.

* $\gamma$ is the orientation of the generalized 3D speed vector with respect to the Z-axis (as defined in Supplemental Fig. S5.a).

In order to limit the computation costs, the estimation of $D_0$ relies on only one computation of the Eq.(14). Using only one computation implies a higher risk of errors, but does not rely on other estimations such as $A$ in Eq.(13) that relies also on $\alpha$ (Appendix A).

In order to obtain $\tilde{\omega}_0$, a mean was done on all $\tilde{\omega}_i$ with an $abs(\theta_i) < 10^\circ$. In both simulations and experiments, the threshold is a valid approximation (see Supplemental Information section S5). In addition, the data-set was not used for a computation of $D_0$ if less than $n > 10$ values were present in the area studied.

Some minimum errors were also expected in the estimation of $D_0$. By using a non-null central band on the image (Fig. 6 a.i,b.i) some errors are related to the width choice. Indeed, equations (13) and (14) were based on the OF seen at $\phi = 0$. However, along the Z-axis of the image the norm of the OF also follows a Gaussian shape (Fig.2). Thus, when $\phi \neq 0$, the OF vector norm will be less significant than when $\psi = 0$. By using a larger band more points could be used, but it would also raise estimation errors on $D_0$. It is possible to greatly reduce this band in the simulation (as all the points are easily detectable and there are no perturbations in flight) but in the

<footnote n="9">⁹https://docs.scipy.org/doc/scipy/reference/optimize.html</footnote>
real-world experiments a wider band is needed to get enough points with sufficient
quality to allow for a meaningful fit.

## 5.2 Estimation Results of $\alpha$ and $D_0$ in Case 1 Flight
*Direction Parallel to Surfaces*

*Case 1* represents the movements and environments that creates an OF shape centered around 0. Usually, a *Case 1* is obtained when the UAV moves parallel to a wall, with the observing camera being perfectly orthogonal to the wall (thus $\alpha = 0$ and $\psi_v = 0$). Obviously as the set of Eq. (13,14) used are in 2D, no movement outside the plane is allowed. As a compromise, the inter-distance used between netting and wall was 80 cm (as shown in Fig. 4 and in Tables 2 & 3), with 500 lux lighting.

In real-world flight, the requirements for *Case 1* are not fully respected because *Case 1* accepts small movements on the *Z - axis* (usually $|\gamma| < 10^\circ$) and some small $\psi_v$ (created by the flying errors). To reduce the impact of small variations between two measurements, a median-filter on two consecutive value smooths the $\alpha$ and $D_0$ measurements. The filter reduces the experimental noise, while not changing the overall behavior of the system.

In order to represent the results, a boxplot<sup>10</sup> is used. It represents the dispersion of the estimation errors (both in angle and distance).

The notations for the different values are, for the ground truth: $\alpha_{gt}$ and $D_{gt}$ for $\alpha$ and $D_0$, respectively. For the measured values (which are estimated) we use the notations $\hat{\alpha}$ and $\hat{D}_0$. For the error between the measurement and the ground truth, the notations are $err_{\alpha} = \hat{\alpha} - \alpha_{gt}$ and $err_D = \hat{D}_0 - D_{gt}$.

<footnote n="10"><sup>10</sup>In the boxplot representation used here, the box corresponds to the 1st and 3rd quartile while the whiskers are respectively Q1 - 1.5*IQR and Q3 + 1.5*IQR, they represent the dispersion of the data when the outliers are removed.</footnote>
<figure><img src="image_15.png" /><figcaption>**Fig. 7:** Results of 2-data smoothed estimations of the environment's orientation and distance in **Case 1**. (a) was obtained with simulation data while (b) was obtained after some real-world experiments with a.i and b.i being examples of input images. a.ii and b.ii are representations of the dispersion of error in angular estimation (with Eq. (13)). a.iii and b.iii are the same type of representations for the distance error Eq. (14). Real world results are given in Table 5. In simulations, 63 OF fields were used to estimate the geometry of a fg netting, 62 for orientation of back plane, and 60 for distance. Finally, with only a planar surface, 61 OF fields were used for both orientation and distance.</figcaption></figure>

In both experiments, the ground truth for the closest distance was $D_0 = 2m$.
(c) shows a top-down view of the whole trajectory inside the flying arena with the NOWA method output. (d) represents the evolution of the angles $\psi_v$, $\gamma$ and the speed of pitch $\Delta\Theta$. On average, the definitions of the **Case 1** (Fig. 3) were respected during the flight however (except in the yellow bands), with exceptions represented by a red-star in (d). In the flight shown in (d), $mean(|\psi_v|) = 6.6^\circ$ and $\sigma(|\psi_v|) = 3.3^\circ$ while $mean(|\gamma|) = 4.3^\circ$ and $\sigma(|\gamma|) = 3.4^\circ$.

The study of **Case 1** shows the impact of the estimation by one-measure of $D_0$ (Fig. 7a.ii, b.ii). In the case of perfect measurements (from simulation) there were almost no errors on the estimation for both $\alpha$ and $D_0$. However, in a real-world arena (Fig. 7b.ii) a non-null dispersion was observed. Another fact is that $D_0$ is (in simulation) underestimated (< 5cm of error). This error can be explained by several factors:
* Some factors of error have already been described (estimation of $\omega_0$, size of the image band etc).

* As a netting is a recurring pattern, the Lucas-Kanade can make errors by tracking a point according to its vicinity, resulting in greater scatter on the measured OFs (Fig. 6).

The same is valid for the relative angular estimation, but this time, the errors are quite small (< 5°) which are of the order of magnitude of the incertitude (as the planes were hand placed, and the camera may have not been at a perfect 90° with the gyroscope direction), so a few degrees are an acceptable error. It is interesting to note that as $\alpha$ needs several OF measures (it is produced by an OF shape optimization) the impact of the width of the band is reduced as well as the local following error that may occur.

It should also be noted that during a real-world flight, it is hard to fly in a true *Case 1* with $\psi_v = 0, \gamma = 0$. While equations (13) and (14) do take into account this possibility, optical issues start to appear. In flight, there are two main sources of uncertainty:

* The Global Shutter camera's exposure time causes slight motion blur on the image.

* Hypothesis of equation (2) is not always respected because of the movements of the under-actuated hexarotor.

While it could be anticipated that they will have more errors than *Case 1*, it is interesting to see how both the accuracy of the estimations of $\alpha$ and $D_0$ evolve in those cases.

## 5.3 Estimation Results of $\alpha$ and $D_0$ in Cases 2 and 3: Any Parallel Surfaces and Any Two Surfaces Orientation

### 5.3.1 Simulated Experiments

*Case 2* represents the set of movements and environments that creates a non-centered OF shape. Several conditions can create such a shape. For example, the camera not being perpendicular ($\alpha \neq 0$) to the observed plane, or a non null incidence angle ($\psi_v \neq 0$) or some combination of both, are possibilities for *Case 2*. In this study we used both a non parallel movement and an inclination angle. This was done by having the UAV moving toward the wall, and having an angle between its camera and wall.

*Case 3* is defined by the dissimilarity in the planes' orientations.
<figure><img src="image_16.png" /><figcaption>Fig. 8: Orientation and distance estimation performances in simulated Case 2 and 3. Case 2 (i) represents the environment in which the surfaces are parallels, thus $\alpha = \alpha_2$. Case 3 (i) represent any two surfaces with $\alpha \neq \alpha_2$. In (ii) boxplot representation of the estimation error on $\alpha$, and similarly (iii) estimation errors on $D_0$ for each cases are presented.</figcaption></figure>

Not all data acquired may be useful as discussed in 3.6.

(a) For Case 2, from 80 input data, 75 estimations of $\alpha_{fg\ net}$ and $\alpha_{bg\ wall}$ (94%) as well as 71 estimations of $\alpha_{single\ wall}$ (89%) were conclusive, same for $\hat{D}$.

(b) For Case 3, as the case is complex, from 80 data, at least 53 estimations of $\alpha_{fg\ net}$ and $\alpha_{bg\ wall}$ (66%) were conclusive and 73 $\alpha_{single\ wall}$ (91%), the same for $\hat{D}$. Note that a Case 3 with no netting is a Case 2.
*Cases 2 and 3* were studied in simulation (Fig. 8), to confirm that the system of equation is sufficiently generic, and provide a reference that can be compared to the real-world experiments. The simulation results were obtained under ideal conditions: minimum image noise, no motion blur and perfect translational movement. In particular, the UAV's speed was stable throughout the acquisition, the images were acquired instantly and the hypothesis (2) of no rotation was respected.

The simulation shows that *Case 2* is close to *Case 1* in terms of accuracy, although its estimation errors are more dispersed, particularly for the background wall distance estimate. *Case 3* is also close to *Case 1*, mainly in terms of median distance estimation errors. For the angular estimation, *Case 2* seems coherent with *Case 1* with a small increase in dispersion. However, *Case 3* poses a number of challenges. Correct detection of the angles of both layers is difficult if they are not aligned with their maxima.

### 5.3.2 UAV Experiments in Flying Arena

<figure><img src="image_17.png" /><figcaption>**Fig. 9:** Images of environments used in real-world experiments for *Case 2* (a) with $\psi_v = 15^\circ$ and $\alpha = \alpha_2 = 30^\circ$ and *Case 3* and 80cm between netting and background (b) with $\psi_v = 0^\circ$, $\alpha = 0^\circ$ and $\alpha_2 = 25^\circ$. Results of the estimations can be seen in Table 5.</figcaption></figure>

Real-world experiments show detection results over an entire flight (Fig. 7c). Clearly, in most cases, the algorithm has no problem detecting the presence of two planar surfaces (red dots and Table 2) or single opaque one (blue dots). Some measures are grayed out as they reflect situations where the UAV was not sufficiently stable (self-rotation, high acceleration etc) or the generic NOWA method failed to converge to either one planar surface or two planar ones.

The study carried out in the Flight Arena (with environment Fig. 9) presents some interesting aspects compared to the simulation (from Table 5 and Fig. 8):

* The median value of $\alpha_{est}$ does not vary much between *Cases 1 and 2*, the same observation is valid for data dispersion (STD). (Table 5).
(a) Case 1

<table><thead><tr><th>Estimation error</th><th>median</th><th>STD</th><th>nbr of OF fields</th></tr></thead><tbody><tr><td>D<sub>fgnet</sub>(m)</td><td>0.15</td><td>0.36</td><td>47</td></tr><tr><td>D<sub>bg wall</sub>(m)</td><td>-0.13</td><td>0.51</td><td>47</td></tr><tr><td>D<sub>single wall</sub>(m)</td><td>-0.07</td><td>0.26</td><td>41</td></tr><tr><td>α<sub>fg net</sub>(°)</td><td>-1.9</td><td>3.9</td><td>46</td></tr><tr><td>α<sub>bg wall</sub>(°)</td><td>-6.8</td><td>5.6</td><td>47</td></tr><tr><td>α<sub>single wall</sub>(°)</td><td>4.2</td><td>1.5</td><td>41</td></tr></tbody></table>

(b) Case 2

<table><thead><tr><th>Estimation error</th><th>median</th><th>STD</th><th>nbr of OF fields</th></tr></thead><tbody><tr><td>D<sub>fg net</sub>(m)</td><td>-0.13</td><td>0.73</td><td>84</td></tr><tr><td>D<sub>bg wall</sub>(m)</td><td>-0.35</td><td>1.03</td><td>84</td></tr><tr><td>D<sub>single wall</sub>(m)</td><td>-0.31</td><td>0.62</td><td>45</td></tr><tr><td>α<sub>fg net</sub>(°)</td><td>0.5</td><td>5.5</td><td>84</td></tr><tr><td>α<sub>bg wall</sub>(°)</td><td>-3.5</td><td>6.8</td><td>84</td></tr><tr><td>α<sub>single wall</sub>(°)</td><td>-1.7</td><td>3.8</td><td>45</td></tr></tbody></table>

(c) Case 3

<table><thead><tr><th>Estimation error</th><th>median</th><th>STD</th><th>nbr of OF fields</th></tr></thead><tbody><tr><td>D<sub>fg net</sub>(m)</td><td>0.07</td><td>0.46</td><td>24</td></tr><tr><td>D<sub>bg wall</sub>(m)</td><td>-0.18</td><td>1.1</td><td>24</td></tr><tr><td>α<sub>fg net</sub>(°)</td><td>-4.5</td><td>6.5</td><td>24</td></tr><tr><td>α<sub>bg wall</sub>(°)</td><td>-6.3</td><td>10.7</td><td>21</td></tr></tbody></table>

**Table 5:** (a) **Case 1** estimation results in real-world experiments. With $\psi_v = 0°$ and $\alpha = \alpha_2 = 0°$.

(b) **Case 2** estimation results in real-world experiments. With $\psi_v = 15°$ and $\alpha = \alpha_2 = 30°$. Note that the two data sets do not have the same length, due to flight arena geometric constraints.

(c) **Case 3** estimation results in real-world experiments. With $\alpha = 0°$ and $\alpha_2 = 25°$.

In all experiments, the real-world environment is shown in Fig. 9.

* Although the errors in **Case 3** are significant in real-world experiments, the simulation shows that it is indeed difficult to estimate the orientation of the background plane accurately.

* The presence of a netting in front of a wall increases the dispersion of the estimate of $\alpha$ and $D_0$ for the wall behind the netting. This increase in error also depends on the case, with **Cases 2 and 3** being more impacted than **Case 1**.

In the end, experimental measurements are too dispersed to be reliably used. It can be explained by low frame rate and instability during flight. In simulations however, **Case 1** is shown to be the case with the best estimates. Successive reorientations based on orientation estimation place the UAV in the **Case 1** configuration, making it softer to follow the planes of the environment.
# 6 Simulation of Automatic UAV Exploration based on Optical Flow in the Presence of Netting

## 6.1 Exploration Strategy

In the previous section, it was shown that accurate estimations of the orientation and distance of the environment are better when the UAV's camera remains in *Case 1*. A suitable flight strategy could be designed which, as defined in Supplemental Information Fig. S6 using parameters in section S7, is based on a state machine focused on staying and returning to *Case 1*.

The state machine separates translational flight from rotational flight, as inspired by the saccadic flight behavior based on OF known from insects [7] and birds [31]. The saccadic behavior allowed to remove the rotational part of the OF, and thus stay in the conditions of equation (2).

The two modes of the navigation strategy are:

Mode 0: During **Translational flight**, the UAV uses the OF generated by its movement along a wall to adjust its relative distance and orientation. The multirotor's exploration controller requires measurement of its own speed vector $V$ and heading $\phi$.

Mode 1: During **Rotation**, the UAV corrects the measured heading error $err_{\alpha} > 10^{\circ}$ without reading the OF. As the UAV's speed is controlled, it may move in **Rotation** mode (Fig. 10d) due to its inertia.

The automatic navigation passes through the following stages:

* Defining a type of movement (via a state machine) from smoothed regulation errors $err_{\alpha}$ and $err_{D}$ (with a time based window of 5-data as median filter). The controller defines the input for the thrusters of the UAV using a sliding surface controller (based on [29]).

* Simulating the fully actuated UAV's dynamics with a model on Simulink, during a time-step between two OF measurements, with $f_{sim} = 1kHz$.

* Updating in Gazebo, the position of the UAV $(x, y, z, \psi)$ at $f_{cam}$.

* Frame grabbing and keeping the previous frame in memory, in order to extract the OF between the two frames $\omega$.

* Determining the presence of netting by the generic NOWA method (Fig. 5) that clusters optical flow $\omega_i$ based on the outlier-free optical flow $\omega_i^{filt}$.

* Computing both $D_0$ and $\alpha$ from the optical flow set with the largest optical flow amplitude $F_{fg}$.

* Updating the regulation errors $err_{\alpha}$ and $err_{D}$. These regulation errors allow for a new mode to be selected (if triggered) and a new time-step of simulation.

During the exploration, $\alpha_{ref} = 0$, $D_{ref} = 2.4m$. These references mean that the UAV has to try to follow and remain parallel with the walls, while maintaining a distance of 2.4m from them. This distance is relative to the onboard camera, which is not the closest part of the UAV to the walls (according to the model, Supplemental Table S5, the UAV is treated as a point, but its dynamic corresponds to an arm length of 37cm). This means that the closest distance is, in fact, shorter than that which the UAV measures. For simplicity, we have decided to increase the reference, and always stay at a minimum of 40cm from the walls (Fig. 10b). After the simulation, it can
be shown that the closest distance $D_0$ is $D_{0min} > 1.12m$, which is coherent with the
UAV's dimensions.
<figure><img src="image_18.png" /><figcaption>**Fig. 10:** Result of the simulation of a UAV navigating inside an unknown room with threshold $n = 10\%$ and Shi-tomasi global and local quality threshold at $5\%$. Each angle was of $45^\circ$, the threshold to start a rotation was $\alpha_{min} = 10^\circ$. It also made use of a median filter for the input data, where a 5-elements sliding window was used to smooth the input, both in Orientation ($\alpha$) and Distance ($D_0$).</figcaption></figure>

(a) is an image of the simulated flight arena with a larger netting than in the simulation experiments, for resolution purposes. (b) is the trajectory of the UAV in the simulated Arena, with colored dots corresponding to its raw instantaneous local detection of a net. The simulated UAV made 5 full laps of the arena. (d) is a focus on one corner of the arena, with the representation of heading at 4 different positions (named I, II, III, IV). (c) is the continuous evolution of the heading during the successive saccadic re-orientations appearing in one corner as shown in (d). In this example when several layers generate OF, we can observe 3 successive re-orientations: (i) the first reorientation is triggered at the corner when OF measurement coming from two differently oriented surfaces (*Case 3*) are treated as a single layer surface, resulting in an inaccurate estimate of orientation, (ii) as the UAV' speed vector is not yet properly oriented parallel to the surface (*Case 2*), the second reorientation reduces this error on the basis of optical flow measured mainly from a single surface, and (iii) a final reorientation is performed with OF measurements coming from only one surface, thus retrieving the *Case 1*.

A video of the firsts turns of exploration in (b) with the corresponding heading and ground truth (c) is provided in Supplemental Video S4. The parameters of the simulation can be found in Supplemental Information section S7, Tables S7 and S8).

In (b), (c) and (d), the color codes the output of the generic NOWA method. Data were almost always conclusive in mode 0 (net detected or single surface detected), and always rejected in mode 1, as Eq. 2 is not respected during the successive re-orientations.
## 6.2 Results

The results in the Simulated Flight Arena show the effectiveness for a behavior inspired by the saccadic nature of insect flight (Fig. 10d). The first estimation of angle is computed when the UAV is at the corner (between position (I) and (II)), and is therefore expected to contain some errors, as the two sets will overlap. However, once the first rotation is made (Fig. 10d position (II)), the UAV makes one (or more) secondary rotation(s) (Fig. 10d positions (III),(IV)) to adjust its heading with respect to the single wall it is now following. The subsequent rotations each bring the UAV closer to a *Case 1*, resulting in better measurements. Since rotations are limited by the trigger $err_α > 10°$, the oscillatory behavior of small rotations is avoided.

The simulation shows that an exploration strategy can be designed on the basis of the OF. Using the motion parameters ($V$ and $\psi_v$) and the OF cues the simulated UAV can explore an unknown room, avoiding and detecting nets by following the surfaces (wall or netting). Results also show that a single netting without background would be detected and considered as a single surface to be followed.
A video of the full experiment with the UAV's heading is presented in Supplemental Videos S4.

# 7 Discussion

## 7.1 OF Extraction

### 7.1.1 Extraction Quality

For OF-based exploration and geometry estimation, the quality of the OF-extraction is a critical issue. The quality of the measured OF depends on the number of vectors required. Too few vectors would prevent OF shapes from being clustered, while too many would make fitting extremely time-consuming.

This range in the number of OF vectors, combined with a limited number of steps for convergence of the fit, explains the data rejections observed.

The results showed that there is a link between cases and accuracy, both in real experiments and in simulations. According to the results, it seems that *Case 1* is less noisy than the others and results in a better estimation of distance and orientation of the environment. This results in a bio-plausible exploration strategy while detecting and avoiding netting-like obstacles, focused on keeping in and returning to a *Case 1* (flight direction parallel to surfaces).

To tackle the issue of data quality and data quantity, some improvements could be considered. Supplemental Information Table S6 shows that other OF cues give marginally better geometry estimation, while requiring more computations.
### 7.1.2 Luminosity

Lighting is generally an important requirement for computer vision, as can also be seen in the present work on Tables 2,3 & 4 (and supplementary video S5). Although the surface were classified accurately at low light levels: the NOWA's overall performances were still light-dependent. Supplementary Tables S2 & S3 show that in more complex cases (mainly *Case 2*), the influence of brightness on performances is more pronounced.

Finally, OF sets were more dispersed in low light conditions. This increased the difficulty in detecting single planes, as a very noisy single plane can be seen as two low-noise surfaces.

### 7.1.3 UAV Velocity and framerate

The camera movement speed is an important aspect of OF. However we were unable to test our netting detection method at high camera movement speeds, because of the UAV setup limitations. The same limitations applied to the frame rate. As the same processor controls the UAV's position and acquires the images, a high frame rate would make the flight unstable (image recording would take priority over position control). Both these problems could be solved by using a more powerful on-board computer, or by optimizing the images processing. For example CUDA, developed by NVIDIA, is a parallel computing platform that accelerates applications by leveraging GPU power. Another option is Cython<fnref n="11" />, which is a programming language that combines Python and C, enabling developers to write Python code with C-level performance for faster execution.

## 7.2 Background impact

### 7.2.1 Distance from Observed Obstacles

As it can be seen on Tables 2 & 3, shape separation is case and scenarios dependent. For instance with an inter-distance of 80 cm at 500 lux, the shapes were distinguishable by using the ratio of their amplitudes in *Case 1* whereas in the same conditions, such shape separation was not obvious in *Case 2*.

Despite this, in both cases *Case 1 & 2*, the ratio of the shape separation was above the threshold $n^{-} = 10\%$. For instance in examples at 500 lux shown in Tables 2 & 3, the ratio of the shape amplitude was of 39% in *Case 2* and of only 16% in *Case 2*.

This difference has several explanations:

* In the case of any flight direction with two parallel surfaces (*Case 2*) the maximum of each shape is not centered in the visual field.

* the distances to the surfaces tend to be greater which decreases the amplitude of the OF shape.

For these two reasons, in the presence of noise, both OF sets tend to overlap.

As shown in Fig. 7c, the robustness of detection decreases as the distance between the UAV and the wall/netting increases. This issue is similar to the findings of biological studies, where insects' visual resolution and ability to detect obstacles such as spider

<footnote n="11">¹¹https://cython.org/</footnote>
webs decrease with increasing distance from the obstacles [9].

### 7.2.2 Relative inter-distance in Multi-Layered Environment

The relative inter-distance between surfaces is also an important aspect when exploring a *multi-layered* environment. Small inter-distances between netting and background make separation of OF vectors sets more difficult, while larger inter-distances facilitate distinction and therefore detection.

Another point highlighted in Supplementary Tables S2 & S3 is the link between inter-distance and lighting condition. Detection accuracy at <5 lux seems to peak for 80 cm and decreases for 150 cm. This phenomenon can be explained by the greater impact of low-luminosity noise on a more distant background, endowed with a smaller OF amplitude.

### 7.2.3 Presence of Non-Planar Background

Finally, the presence of a non-planar background is also possible. Since it's difficult to obtain an environment without local planarity indoors, a large number of OF sets were classified as planar surfaces, when in fact the background had only partial planarity. However, the presence of a netting was still very well detected in any cases.

### 7.2.4 Poorly contrasted background

Shi-Tomasi algorithm rank fiducial points with respect to their quality. In the case of a low-textured, poorly contrasted environment, this can either increase the number of noisy fiducial points or result in very few well-defined fiducial points (as shown in Supplemental Information section S3). However, the observation of the OF sets on Fig. S2 shows the impact of the presence of a netting. The study of this specific scenario in front of a poorly contrasted background is the subject of future work.

### 7.2.5 Netting with no Background, Against the Sky

A netting with a very distant background or no background at all is also a scenario that can exist, especially when only the sky is in the background. In this instance, the background is not poorly contrasted, it is simply absent, thus the background will generate no OF noise.

Although real experiments with no background visual cues could not be conducted, simulations have produced promising results (see Supp Info Fig S3). Since the NOWA method aims to distinguish netting from a background, the absence of a background enabled the netting to be interpreted as a simple planar surface. Furthermore, the OF generated by a tautened netting is shaped similarly to that of a planar surface, enabling detection with accurate estimates of both distance and orientation. This is illustrated in Fig. 10, where the lower part of the octagonal arena consisted solely of netting, which was consistently detected, avoided, and followed by the simulated UAV.
## 7.3 Obstacle Avoidance & Navigation

### 7.3.1 Geometry Estimation Limitations

The OF based geometry is well estimated in simulation (Fig 8). To prove robustness, a flight simulation was realized where the simulated hexarotor had to keep its relative distance “foreground-plane camera” constant and to keep and to stay parallel (Fig. 10).

However, real flight experiments in the flying arena reveal results in orientation and distance estimation with a wide standard deviation.

This dispersion is due to several factors, but mainly to two of them:

* UAV small vibrations

* Frame rate of images acquisition

As the camera acquires images and UAV position at 10 Hz (due to some material constraints), high frequency vibrations disturbed images acquisition (motion blurr). To reduce the impact of such vibrations, two solutions can be explored: either (i) increase the camera's frame rate (itch is not possible with our current material) or (ii) use a more stable multirotor (such as a fully actuated hexarotor).

### 7.3.2 Simulated Saccadic Exploration to Fly Parallel Along Surfaces

By incorporating saccadic movements, the simulated UAV can efficiently follow surfaces (flying parallel to them), avoid crashes, and explore a closed environment. Interestingly, experimental studies presented here, also show better performances for netting detection when the multirotor is moving parallel to surfaces. Simulation shows an extension of these findings when in case of a complex environment. By successive reorientation to keep its flight direction parallel to surfaces as much as possible, netting detection, as well as distance and orientation estimation can be achieved no matter the type of surface followed, in the simulation. To remain within the framework of the Eq. 2 hypothesis, the simulated UAV maneuvered by alternating forward motion and rotation.

## 8 Conclusion

To the best of our knowledge, we present the first study that successfully detects the presence of netting in-flight. In addition, our study consider all configuration of surface incidences, various flight directions and various lighting condition. Our experimental result demonstrated the ability of a flying UAV to detect (including in real-world) a netting and to estimate its orientation and its distance in front of different background resulting in a *multi-layered* environment. The experimental tests showed high detection accuracy (in average 80% at 500 lux) and low error of estimation by identifying the OF signatures of surfaces, including when the surfaces create *multi-layered* environment. The detection and estimation quality depended on the angles between the surfaces and the flight direction that we have carefully defined in 3 cases and those independently on which surfaces are present on the visual field (defined as scenarios).
By using a dedicated exploration strategy based on successive re-orientations, a simulated UAV can retrieve the most favorable situation, where the UAV fly parallel to the surface(s), **Case 1**. In this strategy, the simulated UAV explored a netting-equipped unknown octagonal environment by following walls, detecting and avoiding netting by considering them as true obstacles (section 6). This behavior mimics the saccadic behavior of Dipteran [7].

This specific strategy will pave the way for autonomous UAV exploration of hazardous spaces containing barely visible obstacles such as netting, by only requiring a low-cost camera.

## 8.1 Future works

As an extension of this work, OF segmentation could be used to create a collection of local planar surface including netting, as proposed in [18]. In addition, the exploration of more complex environments could be performed by searching for OF signatures of several planes on the foreground netting surfaces and on the background surfaces. As a result, the mapping of the environment by an approximation of a series of oriented plans would then possible as our method based on shapes estimate the distance and orientation of each local plane. As a start, the 3D generalization of the OF equations are given in Supplemental Information S8.

The improvement of the distance and orientation estimation can also be subject of further studies, mainly by using the divergence of the OF combined with an EKF inspired by [32, 33]. The introduction of frontal OF divergence could also make it possible to use OFs from the frontal region, even if the optical flow is particularly low there.

In addition, the navigation strategy could be improved by relying less on external measures such as speed and heading. For speed control, we could consider translational OF regulation [34–36].

## 9 Acknowledgment

We thank J.M. Ingargiola and Amaury Negre (from Gipsa-Lab, Grenoble) for their kind help with the UAV hardware and software respectively, as well as D. Wood for revising the English manuscript.

### Declaration

### Competing interests

On behalf of all authors, the corresponding author states that there is no conflict of interest.
## Fundings

The participation of X.D. in this research was made possible by the joint PhD grant from the Agence Innovation Défense (AID) and Aix-Marseille University.

Financial support for the running costs was provided via a ProxiLearn project grant (ANR-19-ASTR-0009) to F.R. and via SpotReturn project grant (ANR-21-ASRO-0001-02) to T.R. and F.R. from the ANR (Astrid Program).

X.D. and F.R. were also supported by Aix Marseille University and the CNRS (Life Science, Information Science, and Engineering and Science & technology Institutes).

The facilities for the experimental tests has been mainly provided by ROBOTEX 2.0 (Grants ROBOTEX ANR-10-EQPX-44-01 and TIRREX ANR-21-ESRE-0015).

## Data Availability

Link to the data acquired during the experiments and simulations in Supplemental Information or at doi: 10.57745/DZIM2L.

## Video Accessibility

Links to supplemental videos are provided in the supplementary information and can also be found at: https://amubox.univ-amu.fr/s/k83RfCkk7JW7Dyw
Appendix A Demonstration of OF Expression in
general cases

A 3D movement V and for any direction of observation dᵢ defined with the reference
frame given in Supplemental Fig. S5 is:

$$
\begin{bmatrix} V_X \\ V_Y \\ V_Z \end{bmatrix} = \tilde{V} \cdot \begin{bmatrix} \cos(\gamma) \cos(\psi_v) \\ \cos(\gamma) \sin(\psi_v) \\ \sin(\gamma) \end{bmatrix} \qquad (A1)
$$

$$
\begin{bmatrix} d_{iX} \\ d_{iY} \\ d_{iZ} \end{bmatrix} = \begin{bmatrix} \cos(\phi) \sin(\theta) \\ \cos(\phi) \cos(\theta) \\ \sin(\phi) \end{bmatrix} \qquad (A2)
$$

It is easier to start from Eq.(4) than from the specific expressions Eq.(5) and
Eq.(6). In the case of $\omega_i = \omega(\theta, \phi)$ the expressions are thus:

$$
\omega(\theta, \phi) = \frac{1}{D(\theta, \phi)} \left[ \tilde{V} \begin{bmatrix} \cos(\gamma) \cos(\psi_v) \\ \cos(\gamma) \sin(\psi_v) \\ \sin(\gamma) \end{bmatrix} - \left( \tilde{V} \begin{bmatrix} \cos(\gamma) \cos(\psi_v) \\ \cos(\gamma) \sin(\psi_v) \\ \sin(\gamma) \end{bmatrix} \cdot \begin{bmatrix} \cos(\phi) \sin(\theta) \\ \cos(\phi) \cos(\theta) \\ \sin(\phi) \end{bmatrix} \right) \begin{bmatrix} \cos(\phi) \sin(\theta) \\ \cos(\phi) \cos(\theta) \\ \sin(\phi) \end{bmatrix} \right] \quad (A3)
$$

By only studying the *central band* of the image, it is only possible to acquire $\omega(\theta, 0)$:

$$
\omega(\theta, 0) = \frac{\tilde{V}}{D(\theta, 0)} \left[ \begin{bmatrix} \cos(\gamma) \cos(\psi_v) \\ \cos(\gamma) \sin(\psi_v) \\ \sin(\gamma) \end{bmatrix} - (\cos(\theta) \cos(\gamma) \sin(\psi_v) - \sin(\theta) \cos(\gamma) \cos(\psi_v)) \cdot \begin{bmatrix} \sin(\theta) \\ \cos(\theta) \\ 0 \end{bmatrix} \right] \quad (A4)
$$

$$
\omega(\theta, 0) = \frac{\tilde{V}}{D(\theta, 0)} \begin{bmatrix} \cos(\gamma) \cos(\psi_v) - (\sin(\theta)^2 \cos(\gamma) \cos(\psi_v) - \cos(\theta) \sin(\theta) \cos(\gamma) \sin(\psi_v)) \\ \cos(\gamma) \sin(\psi_v) - (\cos(\theta)^2 \cos(\gamma) \sin(\psi_v) - \sin(\theta) \cos(\theta) \cos(\gamma) \cos(\psi_v)) \\ \sin(\gamma) \end{bmatrix} \quad (A5)
$$

$$
\omega(\theta, 0) = \frac{\tilde{V}}{D(\theta, 0)} \begin{bmatrix} \cos(\gamma) [\cos(\theta) \sin(\theta) \sin(\psi_v) + \cos(\psi_v)(1 - \sin(\theta)^2)] \\ \cos(\gamma) [\cos(\theta) \sin(\theta) \cos(\psi_v) + \sin(\psi_v)(1 - \cos(\theta)^2)] \\ \sin(\gamma) \end{bmatrix} \quad (A6)
$$

By focusing on the component of each row in the vector, we have the following set
of equations (we remove the Z component as it was close to 0 during the major part
of the movement, and gave weak information).
$$ \left\{ \begin{aligned} \omega(\theta, 0)_X &= \frac{\tilde{V}}{D(\theta, 0)} (\cos(\gamma) \cos(\psi_v) \cos(\theta)^2 + \cos(\gamma) \sin(\psi_v) \cos(\theta) \sin(\theta)) \\ \omega(\theta, 0)_Y &= \frac{\tilde{V}}{D(\theta, 0)} (\cos(\gamma) \sin(\psi_v) \sin(\theta)^2 + \cos(\gamma) \cos(\psi_v) \sin(\theta) \cos(\theta)) \\ \tilde{\omega}(\theta, 0) &= \frac{\tilde{V}}{D(\theta, 0)} \cos(\gamma) \cos(\theta - \psi_v) \end{aligned} \right. \quad (A7) $$

$$ \left\{ \begin{aligned} \omega(\theta, 0)_X &= \frac{\tilde{V}}{D(\theta, 0)} \cos(\gamma) \cos(\theta) (\cos(\psi_v) \cos(\theta) + \sin(\psi_v) \sin(\theta)) \\ \omega(\theta, 0)_Y &= \frac{\tilde{V}}{D(\theta, 0)} \cos(\gamma) \sin(\theta) (\sin(\psi_v) \sin(\theta) + \cos(\psi_v) \cos(\theta)) \\ \tilde{\omega}(\theta, 0) &= \frac{\tilde{V}}{D(\theta, 0)} \cos(\gamma) \cos(\theta - \psi_v) \end{aligned} \right. \quad (A8) $$

$$ \left\{ \begin{aligned} \omega(\theta, 0)_X &= \frac{\tilde{V}}{D(\theta, 0)} \cos(\gamma) \cos(\theta) \cos(\theta - \psi_v) \\ \omega(\theta, 0)_Y &= \frac{\tilde{V}}{D(\theta, 0)} \cos(\gamma) \sin(\theta) \cos(\theta - \psi_v) \\ \tilde{\omega}(\theta, 0) &= \frac{\tilde{V}}{D(\theta, 0)} \cos(\gamma) \cos(\theta - \psi_v) \end{aligned} \right. \quad (A9) $$

By using the definition of $D(\theta, 0)$ given in B21 (as we are indeed in a 2D case when dealing with distance computation), we have:

$$ \left\{ \begin{aligned} \omega(\theta, 0)_X &= \frac{\tilde{V}}{D_0 \cos(\alpha)} \cos(\gamma) \cos(\alpha + \theta) \cos(\theta) \cos(\theta - \psi_v) \\ \omega(\theta, 0)_Y &= \frac{\tilde{V}}{D_0 \cos(\alpha)} \cos(\gamma) \cos(\alpha + \theta) \sin(\theta) \cos(\theta - \psi_v) \\ \tilde{\omega}(\theta, 0) &= \frac{\tilde{V}}{D_0 \cos(\alpha)} \cos(\gamma) \cos(\alpha + \theta) \cos(\theta - \psi_v) \end{aligned} \right. \quad (A10) $$

Which gives us the equation: 13.

## Appendix B Demonstration of the relationship between slope angle $\alpha$ and maximum OF $\theta_1$

By first using the definition of the cos and sin function inside the FOV of the camera (Supplemental Fig. S5c), and using as reference $D_0$, we obtain the following equation B11:

$$ D_{\theta} - D_{err} = \frac{D_{0}}{\cos(\theta)} \quad (B11) $$

The same can be done using tan and the small triangle made by the slope and the axis of the camera, giving B12:

$$ \tan(\alpha) \cdot (l_1 + l_2) = h \quad (B12) $$

$$ D_{err} = \frac{h}{\cos(\theta)} = \frac{l_1}{\sin(\theta)} ; \tan(\theta) = \frac{l_1}{h} \quad \text{and} \quad l_2 = D_0 \cdot \tan(\theta) \quad (B13) $$

By using B13 into B12 we obtain:
$$ \tan(\alpha) \cdot (D_{err} \cdot \sin(\theta) + D_0 \cdot \tan(\alpha)) = \cos(\theta) \cdot D_{err} \quad (B14) $$

$$ D_{err} (\cos(\theta) - \tan(\alpha) \sin(\theta)) = \tan(\alpha) \cdot \tan(\theta) \cdot D_0 \quad (B15) $$

Using the definition of B11 we get:

$$ \tan(\alpha) \cdot \tan(\theta) \cdot D_0 = D_{\theta} \cdot (\cos(\theta) - \tan(\alpha) \cdot \sin(\theta)) - \frac{D_0}{\cos(\theta)} (\cos(\theta) - \tan(\alpha) \cdot \sin(\theta)) \quad (B16) $$

$$ D_{\theta} \cdot (\cos(\theta) - \tan(\alpha) \cdot \sin(\theta)) = D_{0}(\tan(\alpha) \cdot \tan(\theta) + 1 - \tan(\alpha) \cdot \tan(\theta)) \quad (B17) $$

$$ D_0 = D_{\theta} \cdot (\cos(\theta) - \tan(\alpha) \cdot \sin(\theta)) \quad (B18) $$

$$ D_{\theta} = D_{0} \cdot \frac{1}{\cos(\theta) - \tan(\alpha) \cdot \sin(\theta)} \quad (B19) $$

$$ D_{\theta} = \frac{D_{0}}{\cos(\theta)} \cdot \frac{1}{1 - \tan(\alpha) \cdot \tan(\theta)} \quad (B20) $$

$$ D_{\theta} = D_{0} \cdot \frac{\cos(\alpha)}{\cos(\alpha + \theta)} \quad (B21) $$

In another hand Eq.(4) can be expressed with a dedicated reference frame, for an easier computation. As the direction of observation vector $d_i$ is a unitary vector, $d_i$ can be a first construction vector of the frame. Another vector $d_{iorth}$ can then be constructed by staying in the plane $(X, Y)$, being orthogonal to $d_i$, unitary and using the trigonometrical orientation.

In this dedicated frame ($d_i, d_{iorth}$), as $d_i$ is the frame vector $X$, the projection of $d_i$ on the $X$-axis is itself thus: $d_i = d_{iX}$ ($d_{iY} = 0$) by construction.

Expressing $V$ in this frame allows the use of the angle $\theta - \psi_v$ (Fig. 1c) and Eq.(4) becomes:

$$ \omega(\theta) = -\frac{1}{D(\theta)} \cdot \left( \tilde{V} \begin{bmatrix} \sin(\theta - \psi_v) \\ \cos(\theta - \psi_v) \end{bmatrix} - \tilde{V} \begin{bmatrix} \sin(\theta - \psi_v) \\ \cos(\theta - \psi_v) \end{bmatrix} \right) \cdot \begin{bmatrix} 1 \\ 0 \end{bmatrix} \quad (B22) $$

$$ \tilde{\omega}(\theta) = -\frac{1}{D(\theta)} \cdot \tilde{V} \cdot \cos(\theta - \psi_v) \quad (B23) $$

By combining B21 and B23 we have:

$$ \tilde{\omega}(\theta) = -\frac{\tilde{V}}{D_0 \cdot \cos(\alpha)} \cdot \cos(\alpha + \theta) \cdot \cos(\theta - \psi_v) \quad (B24) $$

We then derive B24 with respect to $\theta$ in order to study the evolution of the value (or norm) of the OF along the observation angle. We want to see if a relation exists
between the angle at which the flow is maximum (null derivative) and other parameters of our system. This gives us:

$$ \frac{d\tilde{\omega}(\theta)}{d\theta} = K(\sin(\theta - \psi_v) \cos(\alpha + \theta) + \sin(\alpha + \theta) \cos(\theta - \psi_v)) \quad (B25) $$

$$ \frac{d\tilde{\omega}(\theta)}{d\theta} = K \sin(\alpha + 2\theta - \psi_v) \quad (B26) $$

Finally, equation Eq.B26 is annulled for $\theta_1$, the argmax of $\tilde{\omega}(\theta)$, thus:

$$ \alpha + 2\theta_1 - \psi_v = 0[\pi] \quad (B27) $$

And B27 gives us Eq. (12).

## Appendix C Computation of $D_0$ from $\tilde{\omega}_0$

For the sake of clarity, we here provide trigonometry functions $\cos(\theta)$ with $c_\theta$ and same with $\sin(\theta)$ and $s_\theta$.

$$ \omega(0) = - \frac{1}{D(0)} \cdot \left( \tilde{V} \cdot \begin{bmatrix} c_{\gamma}c_{\psi_v} \\ c_{\gamma}s_{\psi_v} \\ s_{\gamma} \end{bmatrix} - \left( \tilde{V} \cdot \begin{bmatrix} c_{\gamma}c_{\psi_v} \\ c_{\gamma}s_{\psi_v} \\ s_{\gamma} \end{bmatrix} \cdot \begin{bmatrix} c_{\phi}s_{\theta} \\ c_{\phi}c_{\theta} \\ s_{\phi} \end{bmatrix} \right) \cdot \begin{bmatrix} c_{\phi}s_{\theta} \\ c_{\phi}c_{\theta} \\ s_{\phi} \end{bmatrix} \right) \quad (C28) $$

$$ \text{gives } \omega(0) = - \frac{\tilde{V}}{D(0)} \cdot \left( \begin{bmatrix} c_{\gamma}c_{\psi_v} \\ c_{\gamma}s_{\psi_v} \\ s_{\gamma} \end{bmatrix} - \begin{bmatrix} 0 \\ c_{\gamma}s_{\psi_v} \\ 0 \end{bmatrix} \right) \quad (C29) $$

$$ \text{gives } \omega(0) = - \frac{\tilde{V}}{D(0)} \cdot \begin{bmatrix} c_{\gamma}c_{\psi_v} \\ 0 \\ s_{\gamma} \end{bmatrix} \quad (C30) $$

$$ \text{thus } \tilde{\omega}_0 = \frac{\tilde{V}}{D(0)} \cdot \sqrt{\cos(\gamma)^2 \cdot \cos(\psi_v)^2 + \sin(\gamma)^2} \quad (C31) $$

$$ D(0) = \frac{\tilde{V}}{\tilde{\omega}_0} \cdot \sqrt{\cos(\gamma)^2 \cdot \cos(\psi_v)^2 + \sin(\gamma)^2} \quad (C32) $$

$$ D_0 = \frac{\tilde{V}}{\tilde{\omega}_0} \cdot \sqrt{\cos(\gamma)^2 \cdot \cos(\psi_v)^2 + \sin(\gamma)^2} \quad (C33) $$

Thus, equation 14 is obtained.
## References

[1] Ali, H., Seifert, C., Jindal, N., Paletta, L., Paar, G. (2007). Window detection in facades. In: 14th International Conference on Image Analysis and Processing (ICIAP 2007), Beijing, China, pp. 837–842. https://doi.org/10.1109/WRC-SARA.2018.8584213

[2] Tibebu, H., Roche, J., De Silva, V., Kondoz, A. (2021). Lidar-based glass detection for improved occupancy grid mapping. *Sensors* 21(7). https://doi.org/10.3390/s21072263

[3] Zhao, X., Yang, Z., Schwertfeger, S. (2020). Mapping with reflection-detection and utilization of reflection in 3D LiDAR scans. In: 2020 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Abu Dhabi, United Arab Emirates, pp. 27–33. https://doi.org/10.1109/SSRR50563.2020.9292595

[4] Weerakoon, L., Herr, G.S., Blunt, J., Yu, M., Chopra, N. (2022). Cartographer_glass: 2D graph SLAM framework using LiDAR for glass environments. *arXiv preprint arXiv:2212.08633*. https://doi.org/10.48550/arXiv.2212.08633

[5] Wang, R., Bach, J., Ferrie, F.P. (2011). Window detection from mobile LiDAR data. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), Kona, HI, USA, pp. 58–65. https://doi.org/10.1109/WACV.2011.5711484

[6] Wei, H., Li, X.-e., Shi, Y., You, B., Xu, Y. (2018). Multi-sensor fusion glass detection for robot navigation and mapping. In: 2018 WRC Symposium on Advanced Robotics and Automation (WRC SARA), Beijing, China, pp. 184–188. https://doi.org/10.1109/WRC-SARA.2018.8584213

[7] Kern, R., Boeddeker, N., Dittmar, L., Egelhaaf, M. (2012). Blowfly flight characteristics are shaped by environmental features and controlled by optic flow information. *Journal of Experimental Biology* **215**(14), 2501–2514. https://journals.biologists.com/jeb/article-pdf/215/14/2501/1280780/2501.pdf. https://doi.org/10.1242/jeb.061713

[8] Sarmiento T. A., M.R.R. (2018). Insights on obstacle avoidance for small unmanned aerial systems from a study of flying animal behavior. *Robotics and Autonomous Systems* **99**, 17–29. https://doi.org/10.1016/j.robot.2017.09.000

[9] Sathyakumar, V. (2022). How do stingless bees navigate around spider webs? https://doi.org/10.25949/21539952.v1

[10] Bruce, M., Heiling, A., Herberstein, M. (2005). Spider signals: Are web decorations visible to birds and bees? *Biology letters* **1**, 299–302. https://doi.org/10.1098/rsbl.2005.0307

[11] Franceschini, N., Pichon, J.M., Blanes, C., Brady, J.M. (1992). From insect vision
to robot vision [and discussion]. *Philosophical Transactions: Biological Sciences* **337**(1281), 283–294. https://doi.org/10.1098/rstb.1992.0106. Accessed 2024-03-07

[12] Ruffier, F., Franceschini, N. (2005). Optic flow regulation: the key to aircraft automatic guidance. *Robotics and Autonomous Systems* **50**(4), 177–194. https://doi.org/10.1016/j.robot.2004.09.016

[13] Expert, F., Ruffier, F. (2015). Flying over uneven moving terrain based on optic-flow cues without any need for reference frames or accelerometers. *Bioinspiration & Biomimetics* **10**(2), 026003. https://doi.org/10.1088/1748-3182/10/2/026003

[14] Ho, H.W., de Croon, G.C., Van Kampen, E., Chu, Q., Mulder, M. (2018). Adaptive gain control strategy for constant optical flow divergence landing. *IEEE Transactions on Robotics* **34**(2), 508–516. https://doi.org/10.1109/TRO.2018.2817418

[15] De Croon, G.C., Dupeyroux, J.J., De Wagter, C., Chatterjee, A., Olejnik, D.A., Ruffier, F. (2022). Accommodating unobservability to control flight attitude with optic flow. *Nature* **610**(7932), 485–490. https://doi.org/10.1038/s41586-022-05182-2

[16] Serres, J.R., Ruffier, F. (2017). Optic flow-based collision-free strategies: From insects to robots. *Arthropod structure & development* **46**(5), 703–717. https://doi.org/10.1016/j.asd.2017.06.003

[17] Hérissé, B., Hamel, T., Mahony, R., Russotto, F.-X. (2010). A terrain-following control approach for a vtol unmanned aerial vehicle using average optical flow. *Autonomous robots* **29**, 381–399. https://doi.org/10.1007/s10514-010-9208-x

[18] Vogel, C., Roth, S., Schindler, K. (2013). Piecewise rigid scene flow. In: 2013 International Conference on Computer Vision. Institute of Electrical and Electronics Engineers, Sydney, NSW, Australia. https://doi.org/10.1109/ICCV.2013.174

[19] Lucas, B.D., Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In: IJCAI'81: 7th International Joint Conference on Artificial Intelligence 1981, vol. 2. Vancouver, Canada, pp. 674–679

[20] Jianbo, S., Tomasi, C. (1994). Good features to track. In: 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 593–600. https://doi.org/10.1109/CVPR.1994.323794

[21] Farnebäck, G. (2003). Two-frame motion estimation based on polynomial expansion. In: Image Analysis: 13th Scandinavian Conference, SCIA 2003 Halmstad, Sweden, June 29–July 2, 2003 Proceedings 13, Berlin, Germany, pp. 363–370. https://doi.org/10.1007/3-540-45103-X_50
[22] Adarve, J.D., Mahony, R. (2016). A filter formulation for computing real time optical flow. *IEEE Robotics and Automation Letters* 1(2), 1192–1199. https://doi.org/10.1109/LRA.2016.2532928

[23] Koenderink, J.J., van Doorn, A.J. (1987). Facts on optic flow. *Biological cybernetics* 56(4), 247–254. https://doi.org/10.1007/BF00365219

[24] Daini, X., Coquet, C., Raffin, R., Raharijaona, T., Ruffier, F. (2023). Safety net detection by optic flow processing. In: 2023 International Conference on Unmanned Aircraft Systems (ICUAS), Warsaw, Poland, pp. 32–39. https://doi.org/10.1109/ICUAS57906.2023.10156597

[25] Corke, P.: *Robotics, Vision and Control*. Springer (2017). https://doi.org/10.1007/978-3-319-54413-7

[26] Vassallo, R.F., Santos-Victor, J., Schneebeli, H.J. (2002). A general approach for egomotion estimation with omnidirectional images. In: Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in Conjunction with ECCV’02, Seoul, Korea (South), pp. 97–103. https://doi.org/10.1109/OMNVIS.2002.1044502

[27] Gluckman, J., Nayar, S.K. (1998). Ego-motion and omnidirectional cameras. In: Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271) 1998, Bombay, India, pp. 999–1005. https://doi.org/10.1109/ICCV.1998.710838

[28] Vassallo, R.F., Santos-Victor, J., Schneebeli, H.J. (2002). A general approach for egomotion estimation with omnidirectional images. In: Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in Conjunction with ECCV’02, Seoul, Korea (South), pp. 97–103. https://doi.org/10.1109/OMNVIS.2002.1044502

[29] Castillo-Zamora, J.J., Escareno, J., Boussaada, I., Labbani, O., Camarillo, K. (2019). Modeling and control of an aerial multi-cargo system: Robust acquiring and transport operations. In: 2019 18th European Control Conference (ECC), Naples, Italy, pp. 1708–1713. https://doi.org/10.23919/ECC.2019.8796045

[30] Daini, X.: Camera images from embedded Camera on a drone flying in front of walls and nettings in simulation and experimental data. https://entrepot.recherche.data.gouv.fr/dataset.xhtml?persistentId=doi:10.57745/DZIM2L (2024).

[31] Eckmeier, D., Geurten, B.R., Kress, D., Mertes, M., Kern, R., Egelhaaf, M., Bischof, H.-J. (2008). Gaze strategy in the free flying zebra finch (taeniopygia guttata). *PLoS One* 3(12), 3956

[32] Ho, H.W., de Croon, G.C., Chu, Q. (2017). Distance and velocity estimation using optical flow from a monocular camera. *International Journal of Micro Air*
Vehicles **9**(3), 198-208. https://doi.org/10.1177/1756829317695566

[33] Bergantin, L., Coquet, C., Dumon, J., Negre, A., Raharijaona, T., Marchand, N., Ruffier, F. (2023). Indoor and outdoor in-flight odometry based solely on optic flows with oscillatory trajectories. *International Journal of Micro Air Vehicles* **15**, 17568293221148380. https://doi.org/10.1177/17568293221148380

[34] Roubieu, F.L., Serres, J.R., Colonnier, F., Franceschini, N., Viollet, S., Ruffier, F. (2014). A biomimetic vision-based hovercraft accounts for bees' complex behaviour in various corridors. *Bioinspiration & biomimetics* **9**(3), 036003. https://doi.org/10.1088/1748-3182/9/3/036003

[35] Serres, J.R., Ruffier, F. (2015). Biomimetic autopilot based on minimalistic motion vision for navigating along corridors comprising u-shaped and s-shaped turns. *Journal of Bionic Engineering* **12**(1), 47-60. https://doi.org/10.1016/S1672-6529(14)60099-8

[36] Castillo-Zamora, J.J., Bergantin, L., Ruffier, F. (2022). Corridor 3d navigation of a fully-actuated multirotor by means of bee-inspired optic flow regulation. In: 2022 26th International Conference on System Theory, Control and Computing (ICSTCC), Sinaia, Romania, pp. 318-324. https://doi.org/10.1109/ICSTCC55426.2022.9931809