# Advancements in estimating differential pistons for the Extremely Large Telescope using deep learning

Pierre Janin-Potiron, Morgan Gray, Benoît Neichel, Maxime Dumont, Jean-François Sauvage, Romain Fetick, Thierry Fusco

► To cite this version:

Pierre Janin-Potiron, Morgan Gray, Benoît Neichel, Maxime Dumont, Jean-François Sauvage, et al. Advancements in estimating differential pistons for the Extremely Large Telescope using deep learning. SPIE Astronomical Telescopes + Instrumentation 2024, Jun 2024, Yokohama, Japan. pp.130974Y, 10.1117/12.3018666. hal-04879563

HAL Id: hal-04879563

https://hal.science/hal-04879563v1

Submitted on 25 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Advancements in estimating differential pistons for the Extremely Large Telescope using Deep Learning

Janin-Potiron, P.ª, Gray, M.ª, Neichel, B.ª, Dumont, M.ᵇ,ᶜ,ª, Sauvage, J.-F.ᵇ,ª, Fetick, R.ᵇ,ª, and Fusco, T.ᵈ,ª

ªAix Marseille Univ, CNRS, CNES, LAM, Marseille, France

ᵇDOTA, ONERA, F-13661 Salon Cedex Air - France

ᶜFaculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias, s/n, 4200-465 Porto, Portugal

ᵈDOTA, ONERA, Université Paris Saclay, 91123 Palaiseau, France

## ABSTRACT

As the Extremely Large Telescope (ELT) nears operational status, the focus is made on maximizing its capabilities to produce the best possible images. Among other influential factors, the differential pistons – caused by the joint presence of spiders breaking the spatial continuity of the wavefront and low-order aberrations from various origins – significantly impact the overall image quality and must therefore be addressed. Our study is centered on utilizing Neural Networks (NN) to accurately estimate the ELT’s six differential pistons in the presence of turbulence residuals using a 2x2 Shack-Hartmann wavefront sensor (SHWFS). The results of this work could be applied to instruments like HARMONI or MORFEO, which will be equipped with 2x2 SHWFS for their natural guide star wavefront sensing. In our earlier work, the use of ResNet networks fed with data covering all pupil/SHWFS rotation angles demonstrated success in piston estimation. In that study, a simplified atmospheric turbulence model was employed, consisting solely of turbulence residuals, implying perfect correction for frequencies below the deformable mirror’s cutoff frequency. In this paper, we propose to evaluate the network’s performance under more realistic atmospheric turbulence conditions relevant to the ELT. We show that the network can still extract differential piston information from single-frame images, while exhibiting increased uncertainty in its estimation. However, using 10-frame averaged images leads to a significant improvement in root mean square error (RMSE) performance. We conclude this work by addressing remaining open questions and outlining potential future research directions. These findings contribute to refining the NN models usage for cophasing applications, addressing alignment challenges, and enhancing ELT instrumentation performance.

**Keywords:** Petaling, Differential piston, ELT, PSF, Deep Learning

## 1. INTRODUCTION

The Extremely Large Telescope¹,² (ELT), currently finalizing the construction of its dome structure, will become the world’s largest optical telescope with expected first light in 2028. With its 39-meter aperture, the ELT will surpass all existing optical telescopes. This unprecedented capability will open new possibilities for astronomical observation, pushing our understanding of the Universe to new heights. This innovative telescope, by its novel architecture, will present unique control challenges, requiring new solutions to achieve its full potential.

Amongst these challenges, the so-called petaling effect³ is of primary importance. Due to the large secondary mirror structure of the ELT (with foreseen 25 cm spiders), the entrance pupil will be split into six areas called petals. These petals, however, can exhibit differential piston errors, also known as petaling, under certain observing conditions, thereby degrading the overall image quality of the telescope. To achieve its 4-mas angular resolution goal in the visible wavelength, the ELT’s phasing strategy must address and correct for these differential piston errors.

<footnote>Further author information: (Send correspondence to Pierre Janin-Potiron)</footnote>

<footnote>Pierre Janin-Potiron: E-mail: pierre.janin-potiron@lam.fr</footnote>
First generation instrument like HARMONI⁴ or MORFEO⁵ will come equipped with a 2x2 Shack-Hartmann wavefront sensor (SH-WFS) for low-order wavefront sensing using an off-axis natural guide stars (NGS). These sensors will produce four focal images which will be used to measure tip-tilt and focus aberrations that are not sensed by the Laser Guide Star (LGS) systems. Taking advantage of this configuration, the four focal images can be used to measure the differential piston as they inherently contain the necessary informations in their point spread function (PSF).

Deep learning (DL) techniques have revolutionized the field of image analysis. Their ability to learn complex patterns from a large amount of data makes them particularly suited for 2D images application. The effectiveness of DL algorithms for differential piston sensing is well-established in prior publications.⁶,⁷ In our previous work,⁸ we successfully demonstrated that our network, built using the DEEPLOOP⁹ deep learning environment, can retrieve differential piston information from 2 × 2 SH-WFS images under simplified atmospheric conditions, assuming perfect adaptive optics (AO) correction for frequencies below the deformable mirror's cutoff frequency. Building upon these initial findings, we opted to investigate the performance of our algorithm under more realistic atmospheric turbulence conditions. We created five representative turbulence profiles using the Python TIPTOP library¹⁰ and we present in this paper the results we obtained under these conditions.

The plan of the paper is as follows. Section 2 details the instrumental setup and key experimental parameters. The variables and notations are defined in this section. Section 3 focuses on the simulation environment for image generation. In particular, we dedicate one section to the selection of turbulence profiles and one section to the choice of differential piston distributions, which represent major parameters of our work. Section 4 presents the deep learning framework employed from the dataset structure to the network's architecture. Finally, Sec. 5 reports the findings obtained for the tested configurations and Sec. 6 ends this paper with an open questions section where we discuss the main actual interrogation we are currently working on.

## 2. INSTRUMENTAL SETUP

This section details the instrumental setup and notations used in our simulations. As introduced in Sec. 1, our algorithm targets the estimation of differential piston values from a PSF acquired using a 2x2 Shack-Hartmann wavefront sensor (SH-WFS). To illustrate the method's core principles, a simplified schematic of the optical setup is presented in Fig. 1.

Figure 1a presents the entrance pupil of the ELT in its nominal configuration, segmented into six subsections by the spiders supporting the secondary mirror. These subsections are referred to as petals and, following ESO notation, numbered from 1 to 6. In addition, the SH-WFS axes, depicted as white straight lines, subdivide the entrance pupil into four subpupils. We designate these subpupils as UL (Upper Left), UR (Upper Right), LL (Lower Left), and LR (Lower Right) for clarity and simplicity.

In addition, the telescope's tracking motion induces a relative rotation between the pupil and the SH-WFS axes, as illustrated in Fig. 1b. We define this rotation angle, denoted by $\theta$, as the angle formed between the SH-WFS x-axis and the axis separating petals 1 and 2 of the pupil. In this specific case, for consistency with our previous work,⁸ we define rotation angles to increase as they rotate in the clockwise direction. Due to symmetry, analyzing only subpupils UR and UL for a rotation angle $\theta \in [0^{\circ}, 30^{\circ}]$ is sufficient to characterize the differential piston effects. This approach reduces the number of differential piston values to be considered to one for the UR sub-pupil ($\Delta_{p_{4,3}}$) and two for the UL sub-pupil ($\Delta_{p_{4,3}}$ and $\Delta_{p_{5,4}}$).

Each subpupil generates a corresponding PSF, referred to as a sub-PSF, as shown in Fig. 1c. The PSFs exhibit characteristic diffraction patterns that arise due to two primary factors: (1) the presence of the telescope's spider and (2) the square cropping of the entrance pupil. Figure 1c illustrates that when the pupil is rotated relative to the SH-WFS axes, the PSFs of subpupils containing three petals (such as UL and LR in Fig. 1b) exhibit more intricate diffraction patterns.

The six petals of the entrance pupil are susceptible to piston errors, which manifest as a uniform displacement of an entire petal along the optical axis. These errors can arise from various sources, including atmospheric turbulence, adaptive optics (AO) system itself, and mechanical or thermal effects. While a global piston shift across the entire pupil does not degrade image quality, differential piston, also known as petaling, occurs when individual petals exhibit different piston errors. This differential piston significantly impacts the final image
<figure><img src="image_2.png" /><figcaption>Figure 1: (a) Initial pupil configuration. The petals are numbered using the ESO convention. (b) Pupil configuration for a pupil rotation angle $\theta = 15^{\circ}$. (c) and (d) Respective resulting sub-PSFs as measured by the 2 × 2 SH-WFS.</figcaption></figure>

quality. We define here the differential piston between petals i and j as the difference in their respective piston values $p_i$ and $p_j$ thus yielding

$$ \Delta_{p_{j,i}} = p_j - p_i \text{ with } i,j \in [1;6], i \neq j. \qquad (1) $$

# 3. SIMULATION ENVIRONMENT

In this section, we briefly describe the process used to create the sub-PSFs images as well as the parameters that are used in our simulations.

## 3.1 Image generation

The simulation code, written in Python, relies on monochromatic Fourier optics for optical propagation. The tool of aoSystems from the P3 library¹¹ are employed to generate the petals of the ELT primary mirror. Assembled together, these segments form the complete entrance pupil, as depicted in Fig. 2a. Each petal is then assigned a random piston drawn from a predefined distribution resulting in the pupil shown in Fig. 2b. Subsequently, atmospheric turbulence is simulated and applied to the pupil which gives the image presented in Fig. 2c. To mimic the action of the SH-WFS, the pupil is subdivided into four subpupils. These subpupils are then zero-padded to reach the desired final sampling as depicted in Fig. 2d. Finally, the sub-PSFs images are obtained by computing the squared modulus of the Fourier transform of each subpupil's complex amplitude. Figure 2e shows the sub-PSF corresponding to the UR sub-pupil.
<figure><img src="image_3.png" /><figcaption>Figure 2: (a) Entrance pupil function. The value is 1 inside the pupil and 0 outside. (b) Phase distribution for piston only case. (c) Phase distribution for piston + atmospheric turbulence case. (d) Phase distribution for UR subpupil. (e) Resulting UR sub-PSF.</figcaption></figure>

## 3.2 Parameters

In this section, we detail the list of the important parameters used in the simulation.

* **array_size**: specifies the final dimensions of the sub-PSF arrays. It is consistently set to 96 throughout the paper, resulting in sub-PSF images of size 96 × 96 px. The sub-PSF being Nyquist-sampled, this correspond to 48 × 48 λ/D.

* **rotation_angle**: the pupil rotation angle $θ$, given in degree. It verifies $θ ∈ [0°, 30°]$, due to symmetry reasons discussed in Sec 2.

* **piston_distribution**: statistical distribution for individual pistons (currently supports uniform or normal distributions). It is consistently set to the uniform distribution $U(-π/2, π/2)$ throughout the paper, and this choice is discussed in Sec. 3.4.

* **psd**: a PSD FITS file create with TIPTOP¹⁰ and containing: (1) an array representing the power spectral density of the atmospheric turbulence, (2) pupil's physical diameter, (3) wavelengths of the LGS, NGS and science target and (4) the pixel scale.

## 3.3 Turbulence conditions

To simulate atmospheric turbulence, we use a power spectral density (PSD) profile obtained from the TIPTOP library.¹⁰ Random samples are then drawn from this profile to generate the turbulence phase screens. Five turbulence profiles have been generated for our simulations by varying three key TIPTOP parameters: (1) seeing (i.e., turbulence strength), (2) zenith angle of the NGS (the further the NGS is from the science target the higher the anisoplanetism effects), and (3) photon flux (lower signal-to-noise ratio for fainter sources). Table 1 summarizes the five turbulence profiles along with their corresponding Strehl ratios. The JQ0 and JQ00 profiles represent unrealistic scenarios included for algorithmic validation and comparative purposes only. The JQ1, JQ2, and JQ3 profiles correspond to scenarios representing good, intermediate, and bad atmospheric conditions, respectively. Figure 3 presents the modulation transfer functions (MTF) for all the cases.
<figure><img src="image_4.png" /><figcaption>Figure 3: Modulation Transfer Function (MTF) of the full pupil for the five turbulence profiles (solid lines) and for the diffraction limited case (in dashed line).</figcaption></figure>

<table><caption>Table 1: List and main characteristics of the datasets used in our simulations.</caption><thead><tr><td>Name</td><td>Description</td><td>Seeing (in ”)</td><td>NGS offset (in ”)</td><td>Photon flux (in ph/frame/subap.)</td><td>Strehl</td></tr></thead><tbody><tr><td>JQ00</td><td>Unrealistic condition for comparative purpose only</td><td>0.400</td><td>0</td><td>7000</td><td>~ 90%</td></tr><tr><td>JQ0</td><td>Unrealistic condition for comparative purpose only</td><td>0.432</td><td>30</td><td>1500</td><td>~ 75%</td></tr><tr><td>JQ1</td><td>Good conditions scenario</td><td>0.432</td><td>70</td><td>1500</td><td>~ 50%</td></tr><tr><td>JQ2</td><td>Intermediate conditions scenario</td><td>0.568</td><td>100</td><td>250</td><td>~ 25%</td></tr><tr><td>JQ3</td><td>Bad conditions scenario</td><td>1.042</td><td>60</td><td>50</td><td>~ 10%</td></tr></tbody></table>

## 3.4 Piston distribution

This section focuses on the importance of a careful selection of the piston distribution and its consequences on image formation. Like any WFS that directly estimate phase (i.e., not its derivative like a SH-WFS or a Pyramid WFS), the $2\pi$ ambiguity represents a physical limitation to the amplitude of piston measurement. This inherent limitation indeed restricts the sensor's capture range to $2\pi$. Several techniques exist to mitigate the ambiguity$^{12-16}$ and a detailed discussion is beyond the scope of this paper. Therefore, we restrict the differential piston values to $\Delta_p \in [-\pi, \pi]$ for this study.

To achieve this limitation on differential piston values, we must find a single piston distribution (i.e. $p_i$, $i \in [1, 6]$) that produces the desired range when calculating the differential piston (i.e. the difference of two single pistons, see Eq. 1). We tested two different distributions: (1) a uniform distribution and (2) a Gaussian distribution. We know that the distribution of the sum of two continuous random variables is the convolution of their respective probability density function (PDF). Because both distributions (1) and (2) are symmetrical, the difference of two random variables drawn from them also follows the convolution of their respective PDFs.

We present on Fig 4a the single piston PDF comprised of one uniform distribution $U(-\pi/2, \pi/2)$ and several
<figure><img src="image_5.png" /><figcaption>Figure 4: (a) Probability density functions (PDFs) of various single piston distributions. (b) Resulting PDFs for the differential pistons. The color scheme in both panels corresponds for ease of comparison.</figcaption></figure>

variation of zero-meaned Gaussian distributions $N(0, \sigma^2)$. Figure 4b presents the respective PDFs for the differential pistons. When single pistons follow a uniform distribution, the resulting differential pistons $\Delta_p$ exhibit a triangular distribution, denoted as $T(-\pi, \pi)$. For the Gaussian distribution case, the resulting PDF remains Gaussian but with a variance broadened by a factor of 2.

Our evaluation revealed imperceptible performance differences between the uniform distribution and the Gaussian distribution with $\sigma = 8\pi/32$. Consequently, we opted for the uniform distribution due to the more appealing shape of its sampling of the differential piston values. Despite careful control of the piston distribution generation within the desired range, introducing turbulence to the entrance pupil may occasionally lead to values exceeding these boundaries. We will address this scenario and its implications in Section 5.5.

The standard deviation of $T(-\pi, \pi)$, giving the initial RMSE of our simulation, is given by

$$ \sigma_{p,0} = \frac{\pi\sqrt{6}}{6} \text{ (rad)} = \frac{\lambda\sqrt{6}}{12} \text{ (m)}. \qquad (2) $$

This set the baseline level for our estimations. Using a wavelength of $\lambda = 2.2\mu m$ in Sec. 5, the zero-point standard deviation for piston thus verify

$$ \sigma_{p,0} \approx 449 \text{ nm}. \qquad (3) $$

### 3.5 Limit cases

In our previous publication,⁸ we demonstrated that the UR differential piston is easier to retrieve compared to the two UL differential pistons. This can be attributed to two factors being: (1) the UL sub-pupil has two differential values ($\Delta_{p4,3}$ and $\Delta_{p5,4}$), making the disentanglement of their contributions to the sub-PSF more complex and (2) when the pupil rotation angle approaches 30° (respectively 0°) for UL1 (respectively UL2), the surface area of one of the two petal contributing to the differential piston becomes negligible, leading to potential estimation errors. Consequently, we restrict our investigation to pupil rotation angles between 0°-26° and 4°-30° for the UL1 and UL2 cases, respectively.

## 4. DEEP LEARNING

We detail in this section the composition of our datasets and the neural network architecture. Our simulations use the DEEPLOOP⁹ deep learning environment to build and train our neural networks.
<figure><img src="image_6.png" /><figcaption>Figure 5: Network architecture used to measure differential piston from a sub-PSF. The elements comprising the network are, from left to right: (0) Input image. (1)(2)(3) Residual units. (4) Convolutional layer. (5) Flaten layer. (6) Dense layers. (7) Differential piston value. (A) Convolution cut layer. (B) Identity cut layer.</figcaption></figure>

## 4.1 Data

The datasets are generated following the steps presented in Sec. 3 where the PSF sub-images are Nyquist-sampled. These datasets contain images with random pupil rotation values between 0° and 30° (see Sec. 2). Following common practice in machine learning, we divided the data into an 80/20 training/validation split. This resulted in a training set of 300000 samples and a validation set of 60000 samples. In addition, a separate test set containing 90000 samples was established for evaluating model performance after the training phase is completed. These samples are binned into groups of 1° width to later assess performance with respect to the pupil's rotation angle. This consist in 3000 sub-PSFs per 1° bin.

## 4.2 Neural networks

Our goal is to recover differential piston information from sub-PSF images. Due to the two-dimensional structure of the input data, Convolutional Neural Networks (CNN) are particularly well-suited for this task. Building upon our prior work,⁸ which investigated the application of both Visual Geometry Group Networks (VGGNets) and Residual Neural Networks (ResNets) architectures, we opted for ResNets in this study. Their deeper architecture, facilitated by skip connections and residual learning, allows them to learn more complex image features compared to shallower networks like VGGNets. To limit network's complexity, we opt for a lighter and simpler ResNet architecture compared to our previous work.⁸ Future investigations will explore the potential benefits of more expansive network architectures.

The architecture of the network is illustrated in Fig. 5. The entrance image is a 96 × 96 pixels sub-PSF produced as described in Sec. 3.2. The first stage of the network consist of five residual units. Each residual unit is composed of two consecutive convolutional layers with a 3 × 3 kernel size. A bypass connection connecting the input of the unit directly to its output allows the unit to learn from both the original input and the modified features learned by the convolutional layers. This bypass connection can be of two types: either the identity (i.e. the input is added directly to the output of the unit) or a convolutional layer with a 1 × 1 kernel size, called a cut layer. At the end of the unit, a 2 × 2 maxpooling operation (not represented on Fig. 5) reduce the dimension of the input by a factor 2. The shape of the data are, in order of the residual units:

0. input - (96, 96, 1)

1. after layers 1a and 1b - (96, 96, 32)

2. after layers 2a and 2b - (48, 48, 64)

3. after layer 3 - (24, 24, 128)

Following the five residual units, a convolutional layer with a 3 × 3 kernel size and a following 4 × 4 max pooling layer are applied. This configuration reduces the data dimensionality to:
4. after convolutional layer 4 - (6, 6, 128)

A flatten layer and two dense layers complete the network. Adding the pupil rotation angle as input to the network is critical for achieving robustness to its variations. While this information can be integrated at various places within the model architecture, we decided to include it after the first dense layers's output. This positioning allows the rotation angle value to considerably influence network training, resulting in robustness against variations in this parameter.

5. after flattening layer 5 - (4608,)

6. after dense layer 6a - (16 ∼ θ = 17, ) and after dense layer 6b - (16,)

7. output - (1,)

In summary, the network architecture has 610913 trainable parameters.

To obtain the best possible performances, we train one network per differential piston/turbulence profile combination (e.g. Δp4,3 at JQ2, see Sec. 2 and Tab. 1). This approach results in 15 distinct cases to be investigated (3 differential pistons values × 5 turbulence profiles). An hyper-parameterer search on the scheduler defining the learning rate values is performed for each combination.

5. RESULTS

This section presents the results obtained by training our networks on realistic, AO-corrected turbulence conditions. We recall in Sec 5.1 the metric we use to assess the performance of our algorithm and comment the results in the following sections.

We begin with the study of the UR sub-pupil case. Section 5.2 presents the results obtained when training the network with single-frame images as we did it in our previous work.⁸ These results latter indicated the potential benefit of frame averaging to mitigate the turbulence effects and enhance the differential piston signal. We therefore explores the gain in performance using averaged frames in Section 5.3.

5.1 Metrics

To estimate the performances of our networks, we use the root mean square error (RMSE), a widely used metric for evaluating the performance of regression models. It is defined as

$$
\text{RMSE} = \sqrt{\sum_{k=0}^{N} \left( \widehat{\Delta}_{p_{j,i}}^{k} - \Delta_{p_{j,i}}^{k} \right)^{2}}, \quad (4)
$$

where $\widehat{\Delta}_{p_{j,i}}^k$ is the estimated value of the k-th realization of differential piston $\Delta_{p_{j,i}}^k$, and N stands for the number of differential piston values in the considered set. This set can be composed of the differential piston measured for either (1) all possible pupil rotation angles or (2) a specific subset of pupil rotation angles. In case (1), the network's global performance is measured while in case (2) the specific performance for a given angle is assessed.

A pupil plane metric, such as the Strehl Ratio (SR), could also provide complementary insights. SR can be calculated to quantify the loss in image quality induced by a given differential piston distribution. It is expressed in Ref. 3 as

$$
SR = \frac{1 + (n - 1)e^{\sigma_p^2}}{n}, \qquad (5)
$$

where n = 6, the number of petals in our case and σ²ₚ is the differential piston variance over the pupil. This formula provides an indication on the differential piston effect in the pupil plane in the case where no other aberration is present. Table 2 compile the correspondence between SR decrease in percentage point (p.p.) and differential piston variance for a couple of indicative values.
<table><caption>Table 2: Indicative correspondence between differential piston RMSE and Strehl ratio.</caption><thead><tr><th>Strehl decrease<br>(in percentage point)</th><th>Differential piston standard deviation<br>(in rad)</th><th>Differential piston standard deviation<br>(in nm at 2.2µm)</th></tr></thead><tbody><tr><td>1</td><td>0.11</td><td>38</td></tr><tr><td>5</td><td>0.25</td><td>87</td></tr><tr><td>10</td><td>0.36</td><td>125</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>Figure 6: (a) Scatter plot of estimated versus true differential piston for the five turbulence profiles. (b) Differential piston RMSE (given in nm at $\lambda = 2.2\mu m$) as a function of pupil rotation angle for single-exposure UR sub-PSF.</figcaption></figure>

## 5.2 Result on single frame exposures for UR sub-pupils

Thanks to our previous work,⁸ we demonstrated that the network presented in Sec. 4.2 is able to retrieve differential piston at a nanometer-level RMSE in presence of turbulence residuals (i.e. considering an ideal AO system that corrects perfectly until the cutoff frequency). Following the same logic, we trained the network with the new sub-PSF obtained using realistic turbulence phase screens from TIPTOP (see Sec. 3.3).

Figure 6 presents the results obtained in this configuration. Figure 6a shows the scatter plot of estimated differential piston versus introduced differential piston while Fig. 6b shows the RMSE as a function of pupil rotation angle. The five pre-defined turbulence profiles are assigned their respective colours from Fig. 3.

All curves from Fig. 6b exhibit a similar profile, characterized by a slight curvature at their extremities. This curvature is due to the unbalanced distribution of the two petals area for angles close to 0° or 30°. As anticipated, the RMSE increases with stronger turbulence across all rotation angles. This behavior can be attributed to the increasing speckle counts going along with worsening turbulence conditions. The speckles can be seen as unwanted “noise” diluting the differential piston information. Across the three realistic turbulence profiles (JQ1, JQ2, and JQ3), the mean RMSE ranges from approximately 50 nm to 150 nm, corresponding to a SR decrease of roughly 2 p.p. for JQ1, 4 p.p. for JQ2 and 14 p.p. for JQ3. These SR variations are presented assuming a system only subject to differential piston. The inclusion of additional turbulent phase errors is expected to further modify these values. Moreover, working with realistic turbulence profiles means that differential piston mode is inherently present within the phasescreens that are applied to the pupil. This has a significant impact on the estimation of the telescope’s true differential piston and will necessitate careful and more in depth consideration.
<figure><img src="image_8.png" /><figcaption>Figure 7: (a) Scatter plot of estimated versus true differential piston for the five turbulence profiles for 10-frames averaged UR sub-PSFs. (b) Differential piston RMSE (given in nm at $\lambda = 2.2\mu m$) as a function of pupil rotation angle for 10-frames averaged UR sub-PSFs. The y-axis differs from Figure 6b.</figcaption></figure>

Our previous study demonstrated a RMSE of approximately 2 nm in the presence of residual turbulence alone for the UR sub-pupil. Introducing AO residuals into the system therefore significantly degrades performance. This degradation likely arises because AO residuals primarily affect low spatial frequencies, which are crucial for differential piston retrieval and because the atmosphere inherently contains the differential piston mode that is not yet taken into account in our simulations. A detailed discussion of this aspects is beyond the scope of this work and will be presented in a future publication.

Following the evaluation of the network's performance using single-frame inputs, we explore the potential benefits of frame averaging to mitigate turbulence effects. This approach is investigated in the next section.

## 5.3 Results on averaged frames for UR sub-pupils

This section investigates the influence of frame averaging on network performance. We chose to create 10-frame averaged images, balancing the averaging quality with manageable simulation times.

Figure 7 presents the results obtained in this averaging configuration. Figure 7a shows the scatter plot of estimated differential piston versus introduced differential piston while Fig. 7b shows the RMSE as a function of pupil rotation angle. The five pre-defined turbulence profiles are assigned their respective colours from Fig. 3. The scale in Fig. 7b differs from Fig. 6b. The cases are now labelled JQxxA10, with $xx \in [00, 0, 1, 2, 3]$ to denote the 10-frames averaging.

As already observed in Section 5.2, the curves from Fig. 7b exhibit a slight curvature at their extremity due to the unbalanced petal areas in these edge cases. The major information given by Fig. 7 is the significant improvement in RMSE achieved by averaging only 10 uncorrelated frames for all the turbulence conditions. For instance, the mean RMSE across all rotation angles for the JQ3 case reduces from ~150 nm to ~50 nm (2 p.p. decrease in SR) when using 10-frame averaging. This reduction represents a significant improvement achieved with only averaging images.

As anticipated, this behavior arises because averaging multiple images enhances the differential piston signal relative to the background speckle noise generated by the turbulence and cancels out the potential differential piston that can be contained into the turbulent phase screens. This process effectively increases the signal-to-noise ratio (SNR) for differential piston retrieval.
<figure><img src="image_9.png" /><figcaption>Figure 8: (a) Scatter plot of estimated versus true differential piston for the five turbulence profiles for 10-frames averaged UL1 (top) and UL2 (bottom) sub-PSFs. (b) Differential piston RMSE (given in nm at $\lambda = 2.2\mu m$) as a function of pupil rotation angle for 10-frames averaged UL1 (solid lines) and UL2 (dashed lines) sub-PSFs. The y-axis scale is adopted from Fig. 7b to facilitate direct comparison.</figcaption></figure>

Building upon these findings, a comprehensive investigation is ongoing to better understand the sub-PSF behavior as a function of observing parameters, such as differential piston strength, turbulence profile or the number of frames used for averaging. This investigation has the potential to reveal new insights into the petaling phenomenon, potentially leading to novel solutions for its measurement and correction.

## 5.4 Results on averaged frames for UL sub-pupils

This section presents the results obtained for the UL sub-pupil case. As detailed in Sections 2 and 3.5, the UL case is generally more challenging due to the requirement of estimating two differential piston values from a single sub-PSF.

Figure 8 presents the results obtained in this configuration. Figure 8a shows the scatter plot of estimated differential piston versus introduced differential piston for the UL1 case (top, $\Delta_{p_{4,3}}$) and for the UL2 case (bottom, $\Delta_{p_{5,4}}$). Figure 8b depicts the RMSE values for the UL1 case, represented by solid lines and the UL2 case, represented by dashed lines. As usual, the five pre-defined turbulence profiles are assigned their respective colours from Fig. 3.

As discussed in Section 3.5, the accuracy of differential piston estimation by the networks lowers as the pupil rotation angles approach the UL1 and UL2 limits (30° and 0°, respectively). This trend is present in Figure 8b, where the RMSE curves exhibit a visible increase as the rotation angle approaches their limit. The results for UL sub-pupil reveal an unexpected asymmetry between the UL1 and UL2 cases. Contrary to our initial expectation of similar network performance, the RMSE values for the UL2 case are consistently lower than those for UL1. One potential explanation for this asymmetry could lie in the power distribution of the atmosphere PSDs. As explained in Ref. 17, tomographic systems yield anisotropic PSDs, therefore creating an anisotropy in the speckle field intensity in the focal plane. Consequently, the network exhibits a position-dependent sensitivity to differential piston signal.* Considering the differential piston signature as signal and speckles as noise, regions with higher speckle intensity present a lower signal-to-noise ratio, making it more challenging for the network

<footnote>*At the time, the PSD does not rotate along with the pupil, which does not reproduce an on-sky behaviour. We believe that correcting this effect will remove the slopes that can be observed on Fig. 8b curves, resulting in flat curves at distinct RMSE levels for UL1 and UL2.</footnote>
<figure><img src="image_10.png" /><figcaption>Figure 9: PSD radial mean of turbulence profiles (solid lines) presented in Sec. 3.3 and differential piston aberration (dashed line). For differential piston, the mean PSD for $\Delta_p \in [-\pi; \pi]$ is represented along with a $\pm 1\sigma$ interval.</figcaption></figure>

to retrieve accurate differential piston values. Conversely, darker regions offer a higher SNR, leading to more precise piston estimations.

## 5.5 Data selection

Figures 6b, 7b and 8b have been slightly modified to ensure clarity and avoid potential confusion of the reader. The modifications concern edge cases related to introduced differential piston values close to $\pm\pi$. The JQ3 case on Fig. 6a illustrates this phenomenon where a differential piston value close to $\pm\pi$ can be misinterpreted by the network to a value that ranges between $-\pi$ and $+\pi$. This occurs because phase errors in monochromatic light are $2\pi$-periodic, and differential piston values close to $\pm\pi$ can be indistinguishable for the network. This effect is more pronounced under high turbulence conditions, where the network is exposed to stronger turbulence residuals, rendering it less robust to this phenomenon. Fortunately, this misinterpretation affects a negligible fraction of the image data (estimated probability $\sim 3 \cdot 10^{-5}$). As discussed in Section 3.4, some strategies exist to correct this effect. All the results have been corrected from this effect except the JQ3 UR case presented on Fig. 6a where the number of outliers is too important to be reasonably rectified.

# 6. OPEN QUESTIONS

This section contains a series of open questions that, while deserving further investigation, earn inclusion in this publication. Their exploration stimulate critical thinking about the underlying principle governing the differential piston phenomenon.

* **Where should we look for the differential piston signal?**

Following the approach used in Ref. 18, we can compare the turbulence and differential piston PSDs to identify locations in the focal plane where the signal-to-noise ratio (SNR, with the signal is defined as the speckles created by differential piston and the noise is defined as the speckles originating from the turbulence) is favorable for retrieving differential piston information within the turbulence speckles. Figure 9 presents the PSD radial mean of turbulence (solid lines) and differential piston (dashed line). The differential piston contribution is calculated as the mean PSD across all piston values (uniformly sampled $\Delta_p$ from $-\pi$ to $+\pi$). The light blue shaded area depicts a $\pm 1\sigma$ interval, which visually represents the range of variations of the differential piston PSD. Its value is minimal for differential pistons close to zero
<figure><img src="image_11.png" /><figcaption>Figure 10: Ratio of differential piston to residual turbulence PSDs for the five turbulence profiles. The colorscale is consistent across all plots.</figcaption></figure>

and reaches its maximum near $\pm\pi$. Figure 9 shows that the differential piston PSD consistently surpasses the turbulence PSD at lower spatial frequencies. This observation suggests that in most cases, differential piston can be estimated at this location with a favorable SNR.

Building upon this, we can extend the reasoning from the 1D radial mean to a 2D map. Figure 10 presents this map where each pixel corresponds to the ratio between the differential piston and the turbulence PSDs. The presented maps has been computed for the UR case at $\theta = 0^{\circ}$. We see that for lower frequency, the ratio is always greater than 1 for all the turbulence conditions, indicating that the estimation of differential piston in this region should always be possible. Figure 10 additionally reveals regions along vertical and $-30^{\circ}$ orientations that exhibit favorable ratios for piston estimation. However, these regions are far from the PSF's center ($> 20\lambda/D$) and while theoretically promising, their practical utility is limited by the significantly lower light intensity at these locations, which would likely be exceeded by detector noise.

* **What is the differential piston statistical distribution?**

The statistical distribution of differential piston across the pupil plane of the ELT remains an open question in the current literature. As differential piston can originate from various sources, it is difficult to quantify the level of aberrations that will be present in real observing conditions. While we decided to restrain the piston values to $[-\pi, \pi]$ in this paper, it would be important to know even a rough estimate of its amplitude to put our study into perspective.

* **How does the system performs when noise is added to the images?**

This study is based on noise-free simulations which is an ideal case. We plan to add detector noise in the future to test the robustness of the network. As established in the first bullet point, the differential piston signal is mainly located near the center of the PSF. Consequently, the photon count at this location will be maximized. Anyway, an in-depth analysis is required to assess the performances of our network regarding detector noise and is dedicated to a forthcoming publication.
* How does the system performs in polychromatic light ?

We have assumed during this paper a monochromatic light, which is an ideal case. In a real-world scenario, the light arriving at the SH-WFS will be polychromatic, potentially affecting the network's performances as the PSF will be blurred depending on the coherence length of the spectrum. While potentially lowering the differential piston estimation performance, the broadening of the spectral band will benefit to the system by increasing the number of photons collected by the SH-WFS detector.

7. CONCLUSION

Building upon our initial work,⁸ we defined in Sec. 2 and 3 a precise and controlled simulation environment for differential piston estimation using images from a 2 × 2 SH-WFS. To enhance the realism of our simulations and better understand the actual phenomenon, we implemented the generation of realistic turbulence profiles using the TIPTOP¹⁰ library. This transition resulted in a measured increase in the differential piston RMSE, presented in Sec. 5, which reflects the added complexity of real-world scenarios. Despite this increase, the DL approach still demonstrated promising performances, suggesting its potential for practical applications. We evaluated performance under two scenarios: (1) using single-frame images, which achieved a RMSE lower than λ/15, and (2) employing 10-frame averaged images, which yielded a RMSE lower than λ/50 for the more challenging JQ3 turbulence condition. The error level reach with the latter result is promising. However, further testing with uncorrelated turbulent phase screens is necessary to determine the number of frames needed to achieve comparable performance on-sky.

In addition to evaluating performances, this study has raised, in Sec. 6, valuable insights into the differential piston phenomenon. By analyzing the PSDs of turbulence and differential piston, we were able to identify favorable locations within the pupil plane for piston estimation. These findings need to be refined but already contribute to a deeper understanding of the interaction between turbulence and differential piston.

ACKNOWLEDGMENTS

This work benefited from the support of the BPI with France2030 SSA-DOSSA project, the French National Research Agency (ANR) with APPLY (ANR-19-CE31-0011) and LabEx FOCUS (ANR-11-LABX-0013), the Programme Investissement Avenir F-CELT (ANR-21-ESRE-0008), the Action Spécifique Haute Résolution Anguilaire (ASHRA) of CNRS/INSU co-funded by CNES, the ECOS-CONYCIT France-Chile cooperation (C20E02), the ORP-H2020 Framework Programme of the European Commission's (Grant number 101004719), STIC AmSud (21-STIC-09), the french government under the France 2030 investment plan, the Initiative d'Excellence d'Aix-Marseille Université A*MIDEX, program number AMX-22-RE-AB-151. This research has made use of computing facilities operated by CeSAM data center at LAM, Marseille, France.

REFERENCES

[1] Tamai, R., Cirasuolo, M., González, J., Koehler, B., and Tuti, M., “The e-elt program status,” 99060W (07 2016).

[2] Ramsay, S., D'Odorico, S., Casali, M., González, J. C., Hubin, N., Kasper, M., Käufl, H. U., Kissler-Patig, M., Marchetti, E., Paufique, J., Pasquini, L., Siebenmorgen, R., Richichi, A., Vernet, J., and Zerbi, F. M., “An overview of the E-ELT instrumentation programme,” in [Ground-based and Airborne Instrumentation for Astronomy III], McLean, I. S., Ramsay, S. K., and Takami, H., eds., **7735**, 773524, International Society for Optics and Photonics, SPIE (2010).

[3] Bertrou-Cantou, A., *Analyse de front d'onde et commande en optique adaptative pour la pupille fragmentée de l'ELT*, PhD thesis (2021). Thèse de doctorat dirigée par Rousset, Gérard et Gendron, Eric Astronomie et astrophysique Université Paris Cité 2021 2021UNIP7191.

[4] Thatte, N., Tecza, M., Schnetler, H., Neichel, B., Melotte, D., Fusco, T., Ferraro-Wood, V., Clarke, F., Bryson, I., O'Brien, K., Mateo, M., Lorenzo, B. G., Evans, C., Bouché, N., Arribas, S., and Consortium, T. H., “Harmoni: the elt’s first-light near-infrared and visible integral field spectrograph,” *The Messen-ger* **182**, 7–12 (March 2021).
[5] Baruffolo, A., Baronchelli, I., Savarese, S., Lampitelli, S., Foppiani, I., Capasso, G., Schipani, P., Petrella, A., Selvestrel, D., Busoni, L., Agapito, G., Plantet, C., Valles, M. S., Oberti, S., Pettazzi, L., Hagenauer, P., Biasi, R., Manetti, M., Gratadour, D., Rigaut, F., Véran, J.-P., Kerley, D., Smith, M., Dunn, J., Balestra, A., Giro, E., Sordo, R., Chinellato, S., and Ciliegi, P., “MORFEO at ELT: preliminary design of the real-time computer,” in [*Adaptive Optics Systems VIII*], Schreiber, L., Schmidt, D., and Vernet, E., eds., **12185**, 121855K, International Society for Optics and Photonics, SPIE (2022).

[6] Rossi, F., Plantet, C., Cheffot, A.-L., Agapito, G., Pinna, E., and Esposito, S., “Machine learning techniques for piston sensing,” in [*Adaptive Optics Systems VIII*], Schreiber, L., Schmidt, D., and Vernet, E., eds., **12185**, 121855D, International Society for Optics and Photonics, SPIE (2022).

[7] Dumont, M., Correia, C. M., Sauvage, J.-F., Schwartz, N., Gray, M., and Cardoso, J., “Phasing segmented telescopes via deep learning methods: application to a deployable cubesat,” *J. Opt. Soc. Am. A* **41**, 489–499 (Mar 2024).

[8] Chauvet, C., Gray, M., Neichel, B., Dumont, M., Bardou, L., Sauvage, J.-F., and Beltramo-Martin, O., “Estimating the elt’s differential pistons with deep learning,” in [*Adaptive Optics for Extremely Large Telescopes 7th Edition*], ONERA, Avignon, France (2023).

[9] Gray, M., Dumont, M., Beltramo-Martin, O., Lambert, J.-C., Neichel, B., and Fusco, T., “DEEPLOOP: DEEP Learning for an Optimized adaptive Optics Psf estimation,” in [*Proceedings SPIE Adaptive Optics Systems VIII*], 117, SPIE, Montréal, Canada (July 2022).

[10] Neichel, B., Beltramo-Martin, O., Plantet, C., Rossi, F., Agapito, G., Fusco, T., Carolo, E., Carla, G., Cirasuolo, M., and van der Burg, R., “Tiptop: a new tool to efficiently predict your favorite ao psf,” (2021).

[11] Beltramo, Olivier, “P3 - fourier psf,” (2020) - https://github.com/astro-tiptop/P3.

[12] Chanan, G., Troy, M., Dekens, F., Michaels, S., Nelson, J., Mast, T., and Kirkman, D., “Phasing the mirror segments of the keck telescopes: the broadbandphasing algorithm,” *Appl. Opt.* **37**, 140–155 (Jan 1998).

[13] Pinna, E., Esposito, S., Puglisi, A., Pieralli, F., Myers, R. M., Busoni, L., Tozzi, A., and Stefanini, P., “Phase ambiguity solution with the Pyramid Phasing Sensor,” in [*Ground-based and Airborne Telescopes*], Stepp, L. M., ed., **6267**, 62672Y, International Society for Optics and Photonics, SPIE (2006).

[14] Surdej, I., *Co-phasing segmented mirrors: theory, laboratory experiments and measurements on sky*, PhD thesis (October 2011).

[15] Vigan, A., Dohlen, K., and Mazzanti, S., “On-sky multiwavelength phasing of segmented telescopes with the zernike phase contrast sensor,” *Appl. Opt.* **50**, 2708–2718 (Jun 2011).

[16] Martinez, P. and Janin-Potiron, P., “Laser-guide-stars used for cophasing broad capture ranges,” *Astronomy and Astrophysics* **593**, L1 (Aug. 2016).

[17] Neichel, B., Fusco, T., and Conan, J.-M., “Tomographic reconstruction for wide-field adaptive optics systems: Fourier domain analysis and fundamental limitations,” *J. Opt. Soc. Am. A* **26**, 219–235 (Jan 2009).

[18] Levraud, N., Chambouleyron, V., Sauvage, J., Neichel, B., Cisse, M., Fauvarque, O., Agapito, G., Plantet, C., Cheffot, A., Pinna, E., Esposito, S., and Fusco, T., “A strategy for sensing the petal mode in the presence of ao residual turbulence with the pyramid wavefront sensor,” *Astronomy & Astrophysics* **682**, A84 (02 2024).