# A stand-alone polarimetric acquisition system for producing a long-term skylight dataset

Léo Poughon, Vincent Aubry, Jocelyn Monnoyer, Stéphane Viollet, Julien Serres

► To cite this version:

Léo Poughon, Vincent Aubry, Jocelyn Monnoyer, Stéphane Viollet, Julien Serres. A stand-alone polarimetric acquisition system for producing a long-term skylight dataset. IEEE Sensors 2023, IEEE, Oct 2023, Vienna, Austria. 10.1109/SENSORS56945.2023.10325176. [https://doi.org/10.1109/SENSORS56945.2023.10325176](https://doi.org/10.1109/SENSORS56945.2023.10325176)

HAL Id: hal-04267766

https://hal.science/hal-04267766v1

Submitted on 2 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# A stand-alone polarimetric acquisition system for producing a long-term skylight dataset

1st Léo Poughon

<figure><img src="image_2.png" /></figure>

Aix Marseille Univ, CNRS, ISM

Marseille, France

leo.poughon@univ-amu.fr

2nd Vincent Aubry

Stellantis

Velizy-Villacoublay, France

vincent.aubry@stellantis.com

3rd Jocelyn Monnoyer

Stellantis

Velizy-Villacoublay, France

jocelyn.monnoyer@stellantis.com

4th Stéphane Viollet

<figure><img src="image_3.png" /></figure>

Aix Marseille Univ, CNRS, ISM

Marseille, France

stephane.viollet@univ-amu.fr

5th Julien R Serres

<figure><img src="image_4.png" /></figure>

Institut Universitaire de France (IUF)

Paris, France

Aix Marseille Univ, CNRS, ISM

Marseille, France

julien.serres@univ-amu.fr

**Abstract**—Skylight polarization phenomenon is at the origin of a recent growing interest for bio-inspired navigation. Skylight-based orientation sensors can be simulated on the basis of physical models. In parallel, machine learning algorithms require a large amount of data to be trained. However, while some simulated databases already exists in the literature, a public database composed of real-world color polarimetric images of the sky in various weather conditions does not. In this study, a long-term experimental device is presented, designed to be left in a distant roof to acquire data over several months, using a Division-of-Focal-Plane polarization imager with a fisheye lens mounted on a rotative telescope mount. An open-source mechanical and electrical design is proposed for easy replication at other locations, with an algorithm to get the sensor's orientation and geometrical calibration in the East-North-Up frame. A sample one-month long dataset is provided on a public archive.

**Keywords**—Skylight Polarization, Navigation, Deep Learning, Heading, Celestial Compass, Polarized Vision.

## I. INTRODUCTION

Skylight polarization has recently been receiving growing interest for several fields, such as cloud detection [1], realistic rendering of outdoor scenes [2], or for bio-inspired navigation [3]. Sunlight is scattered by the atmosphere. By measuring the polarization pattern, we can estimate the sun's position in the sky dome, and use it to estimate a heading with respect to north.

In Nature, many insects are sensitive to skylight polarization and use it for navigational purposes. For example, the desert ant *Cataglyphis* [4] is able to leave its nest and forage widely in search for food, and then to head directly back to the entrance without using pheromones. Desert ants use a navigation strategy named path integration (PI), relying on odometry and heading estimation, which the ant achieves by

means of an optical compass based on its sensitivity to the sky's polarized light.

There are several models of this phenomenon. The simplest is Rayleigh scattering where there is simple scattering by particles in the atmosphere. A more complex model that takes into account the presence of singularities in the sky called "neutral points" has been proposed by Berry *et al.* [6]. These models allow us to simulate the pattern in a given place at a specific time with a uniform sky. However, the real world presents varying weather conditions (such as the presence of clouds, fog, airplanes tracks, the moon ...) which lead to more complex situations. Previous studies have thoroughly analysed skylight polarization [7]. However for more complex and random events (like clouds), models solely based on Rayleigh scattering cannot account for the real pattern of skylight. Even if many studies on polarized skylight for navigation purposes have been developed [3], very few of them are sufficiently robust to deal with cloudy or polluted skies.

As suggested by a recent study based on deep neural networks [8], deep learning algorithms seem to be very promising for improving the robustness of polarimetric image processing. As lots of data are needed to train AI models, one can either build a large simulated dataset as expressed in previous studies [9] but to our knowledge no real-world data as yet been issued. This is the need this paper tries to fill with the build of an experimental skylight polarization dataset taken from a laboratory rooftop in Marseille, France (43.286990365824785°N, 5.403361407820939°E).

## II. METHODS

### A. Mechanical Design

The polarimetric camera (Sony IMX250MYR) was mounted on a rotative telescope mount in order to be able to align it in azimuth and elevation. As the mount only has two axes of rotation, a rotational degree of freedom was lost, and we could't freely orientate our camera in space. However, we developed a calibration method (see section II D) to accurately

L.P. was supported by a CIFRE doctoral fellowship from the ANRT and Stellantis (agreement #2019/0658). J.R.S has received funding from the Excellence Initiative of Aix-Marseille Université - A*Midex, a French "Investissements d'Avenir" programme AMX-21-ERC-02 and AMX-20-TRA-043. This research work was also supported by the SUD Provence-Alpes-Côte d'Azur Region (PACA) (Grant #2021/08135).
<figure><img src="image_5.png" /><figcaption>Fig. 1. The color-polarization macro-pixel pattern of our polarimetric camera.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 2. Global view and mechanical layout of our system (closeup).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 3. The hardware architecture of the setup.</figcaption></figure>

measure the orientation of the camera by comparing the sun's position in spherical coordinates obtained with ephemerides with the position of the sun measured by the camera.

We used an aluminum profile-based structure to achieve solid connection between the mount and the camera. It was also useful to maintain a poly(methyl methacrylate) dome that protected the camera and mount from rain via a 3d printed support. A laser-cut piece also allowed the fixture of a tarpaulin at the bottom (see figure 2) to make the mount able to rotate freely while providing protection from rain.

## B. Electrical Design

Our system had to be left on the rooftop for long periods without requiring any intervention. As no power source was easily available, we added a 160W solar panel and a 120Ah lead battery to maintain power during the nights and cloudy days. It was sufficient to supply enough power to our experiment without interruption over months.

For monitoring purposes, an analog-to-digital converter (ADC) allowed us to track hourly the battery voltage via a resistor ladder to adapt the input scale to the ADC's requirements. As the battery voltage fluctuated, we added a DC-DC 11-15V to 12V regulator to stabilize it, helping us to feed our camera and rotative mount with a single stable voltage.

## C. Fisheye Distorsion

The polarimetric camera was composed of a SONY IMX250MYR imager and a Fujinon FE185C057HA-1 fisheye lens with a 185° field of view and a selectable aperture in the range $f/1.4 - f/22$. If the aperture was too great, the polarization was modified due to large angles of incidence of the polarized light [11] so we chose to use a $f/4$ aperture. Theoretical focal length and distortion function are given by the manufacturer: here, $f = 1.8mm$ and the distortion function is computed as follows: $y = f \cdot \Theta$, where y is the distance between the sensor center and the projected point, and $\Theta$ is the incidence angle on the fisheye lens.

## D. Orientation Calibration

Once the system was powered up and the camera roughly oriented toward the sky, we needed to know the orientation of the camera frame in the local East-North-Up (ENU) frame. To do this, we chose to compare the sun's course measured on the sensor with Sun ephemeris provided by the Astropy community-developed core Python package. (http://www.astropy.org, connected 30 June 2023)

<figure><img src="image_8.png" /><figcaption>Fig. 4. Rotations between camera frame and ENU frame.</figcaption></figure>

During a sunny day, when capturing images with the lowest time-of-exposure our camera could provide, we got an image with the projection of the sun through the fisheye lens, this showed a bright spot (the sun) surrounded by a dark background (the sky). A point detection algorithm (using the Scipy package [10]) allowed us to evaluate the sun's projection on the sensor. By taking several pictures during the day, we plotted the course of sun's travel on the sensor. Simultaneously, we had a set of known orientations for the sun at the same time in the ENU frame. We then needed to find a rotation matrix between the two frames,
<figure><img src="image_9.png" /><figcaption>RGB Image</figcaption></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>Measured AOP</figcaption></figure>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /><figcaption>Measured DOP</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 5. An image from the dataset with AOP and DOP visualization (bottom) compared to Rayleigh simulation (top).</figcaption></figure>

which is obtained with a least-squares method. Code and
mechanical plans are available here : https://github.com/mol-1/
Long-Term-Skylight-Polarization-Measurement-Device

III. ACQUISITION

Several thousand of images were taken between 2022-07-29
and 2022-08-25. One image was acquired every 10 minutes,
with several fixed and auto-adaptive exposure times. Camera
orientation was assumed to be constant over time. Database is
available here: https://doi.org/10.57745/610QTS.

#### IV. DATA VISUALIZATION

Once a database of images had been compiled, we applied
a post-processing program that was able to process Angle
(AOP) and Degree of Polarization (DOP) of each image. If
an image was saturated over 20%, it was deleted from the
dataset. Images were cropped to reduce their size, and then a
folder was created for each image containing raw polarization
image, exposure time, timestamp, processed AOP and DOP,
and a figure showing sun's course over the day, RGB image,
processed AOP and DOP, and reconstructed AOP and DOP
from a Rayleigh simulation of the sky at the same time with
the same optical system.

V. CONCLUSION

We have depicted a stand-alone system based on a polari-
metric camera for building a database of images. We have
also developed an open source tool in Python to visualize and
compare the real images with simulated images obtained with
the Rayleigh model. In a future development, the hardware
of the system could be simplified without the need for a
telescope mount in order to take panoramic images in a unique
orientation of the camera. Our database could be useful for
testing and developing neural network-based processing and
also classical image processing. Finally, our database could
be used to validate models of the polarized skylight patterns
in various outdoor conditions.

ACKNOWLEDGEMENT

"Centre de Calcul Intensif d'Aix-Marseille" is acknowl-
edged for granting access to its high performance comput-
ing resources to compute quickly over the whole database
(Mésocentre d'Aix Marseille Université: https://mesocentre.
univ-amu.fr/). We thank L. Perrinet and G. Masson for autho-
rizing and facilitating the installation of the camera on the
rooftop of the "Institut des Neurosciences de la Timone".
The authors thank David Wood (English at your Service,
http://www.eays.eu/) for revising the English of this study.
REFERENCES

[1] W. Li, Y. Cao, W. Zhang, Y. Ning, and X. Xu, Cloud Detection Method Based on All-Sky Polarization Imaging. Sensors, 22(16), 6162, 2022

[2] A. Wilkie, C. Ulbricht, R. F. Tobler, G. Zotti, and W. Purgathofer, An analytical model for skylight polarisation. In Rendering Techniques (pp. 387-398), June 2004

[3] F. Kong, Y. Guo, J. Zhang, X. Fan, and X. Guo, Review on bio-inspired polarized skylight navigation. Chinese Journal of Aeronautics, 2023

[4] G. Horváth, and D. Varjú, Polarized light in animal vision: polarization patterns in nature. Springer Science & Business Media, 2004

[5] J. W. Strutt, XV. On the light from the sky, its polarization and colour. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 41(271), 107-120, 1871

[6] M. V. Berry, M. R. Dennis, and R. L. Lee, Polarization singularities in the clear sky. New Journal of Physics, 6(1), 162., 2004

[7] M. L. Brines, and J. L. Gould, Skylight polarization patterns and animal orientation. Journal of Experimental Biology, 96(1), 69-91., 1982

[8] T. Yang, H. Bo, X. Yang, J. Gao, and Z. Shi, Conditional LS-GAN Based Skylight Polarization Image Restoration and Application in Meridian Localization. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1-5). IEEE, June 2023

[9] H. Liang, and H. Bai, Polarized skylight navigation simulation (PSNS) dataset. arXiv preprint arXiv:2007.13081, 2020

[10] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, ... and P. Van Mulbregt, SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature methods, 17(3), 261-272, 2020

[11] C. Lane, D. Rode, and T. Rösgen, Calibration of a polarization image sensor and investigation of influencing factors. Applied Optics, 61(6), C37-C45, 2022