# Performance comparison of the Shack-Hartmann and pyramid wavefront sensors with a laser guide star for 40 m telescopes

Francisco Oyarzún, Cedric Taissir Heritier, Vincent Chambouleyron, Thierry Fusco, Paul Rouquette, Benoit Neichel

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Francisco Oyarzún, Cedric Taissir Heritier, Vincent Chambouleyron, Thierry Fusco, Paul Rouquette, et al.. Performance comparison of the Shack-Hartmann and pyramid wavefront sensors with a laser guide star for 40 m telescopes. Astronomy & Astrophysics - A&A, 2024, 691, pp.A205. 10.1051/0004-6361/202451670. hal-04783246

HAL Id: hal-04783246

https://hal.science/hal-04783246v1

Submitted on 14 Nov 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Performance comparison of the Shack–Hartmann and pyramid wavefront sensors with a laser guide star for 40 m telescopes

F. Oyarzún<sup>1,*</sup> ◦, C. Heritier<sup>3,1</sup>, V. Chambouleyron<sup>2</sup>, T. Fusco<sup>3,1</sup>, P. Rouquette<sup>1</sup>, and B. Neichel<sup>1</sup> ◦

Received 26 July 2024 / Accepted 19 September 2024

## ABSTRACT

**Context.** Upcoming giant segmented mirror telescopes will use laser guide stars (LGS) for their adaptive optics (AO) systems. Two options of wavefront sensors (WFSs) are the Shack–Hartmann wavefront sensor (SHWFS) and the pyramid wavefront sensor (PWFS).

**Aims.** In this paper, we compare the noise performance of the PWFS and the SHWFS. We aim to identify which of the two is best to use in the context of a single or tomographic configuration.

**Methods.** To compute the noise performance, we extended a noise model developed for the PWFS to be used with the SHWFS. To do this, we expressed the centroiding algorithm of the SHWFS as a matrix-vector multiplication, which allowed us to use the statistics of noise to compute its propagation through the AO loop. We validated the noise model with end-to-end simulations for telescopes of 8 and 16 m in diameter.

**Results.** For an AO system with only one WFS, we found that given the same number of subapertures, the PWFS outperforms the SHWFS. For a 40 m telescope, the limiting magnitude of the PWFS is around one magnitude higher than the SHWFS. When using multiple WFS and a generalized least-squares estimator to combine the signal, our model predicts that in a tomographic system, the SHWFS performs better than the PWFS (with a limiting magnitude that is higher by a 0.3 magnitude. When using sub-electron RON detectors for the PWFS, the performance quality is almost identical for the two WFSs.

**Conclusions.** We find that when using a single WFS with LGS, PWFS is a better alternative than the SH. For a tomographic system, both sensors would give roughly the same performance.

**Key words.** instrumentation: adaptive optics – methods: numerical

## 1. Introduction

New giant segmented mirror telescopes (GSMTs) will use adaptive optics (AO) systems to deal with the degradation in angular resolution given by atmospheric effects. With AO, it would be possible to obtain angular resolutions close to the theoretical limit, giving exciting opportunities to study small, distant objects. With the help of coronagraphic instruments, it would offer the capabilities of obtaining direct images of more exoplanets than ever before, and be able to study their atmospheric composition (Davies et al. 2016). As the GSMTs gather up to ten times more light than their predecessors, they will allow us to study fainter objects, unlocking new insights into the early universe and its formation (Gilmozzi & Spyromilio 2007).

The AO systems require the use of a guide object. The GSMTs will use laser guide stars (LGS; Davies et al. 2016; Thatte et al. 2016; Ciliegi et al. 2022) to compensate for the lack of natural guide stars (NGSs) that are bright enough to have good sky coverage (Primmerman et al. 1991). These LGS are generated by exciting sodium atoms present on a layer around 90 km above sea level (Foy & Labeyrie 1985). This layer is around 20 km thick and has an evolving sodium density profile (Pfrommer & Hickson 2014). To generate the LGS, a sodium laser is used. For this, dedicated laser launch telescopes are mounted at the side of the telescope that shoots beams of light calibrated at 589 nm. Due to beam divergence and atmospheric

distortions, the beam has a width of around 1 arcsec when going through the sodium layer. Due to this width, and the thickness of the layer, the LGS has a cylinder shape on the sky, which generates a 3D focal volume after the telescope.

When using a Shack–Hartmann wavefront sensor (SHWFS), each subaperture observes the LGS from a different perspective. The subapertures closest to the laser launch telescope observe an object whose size is limited by the width of the laser beam, and the subapertures further away see an elongated shape, as they see this cylinder from the side. For a 40 m telescope, the geometry implies that the elongation on the furthest subaperture can reach up to 20 arcsec. The measurements of the SHWFS rely on finding the displacement of the images of the source each subaperture generates. The extension of the source decreases the precision of the measurements, as it increases the observed variance of the position of the source. The extension of the source also means that large detectors have to be used to correctly sample the spot. An 80 × 80 subaperture SHWFS for a 40 m telescope would need a detector of around 1600 × 1600 pixels (Fusco et al. 2019).

The pyramid wavefront sensor (PWFS; Ragazzoni 1996) showed an interesting alternative to the SHWFS. It is from the family of the Fourier Filtering Wavefront Sensor (FFWFS), which uses the camera on the pupil plane. For this reason, an equivalent 80 × 80 subaperture PWFS would need a detector no bigger than 240 × 240 pixels. The PWFS also has a higher sensitivity than the SHWFS when using an NGS in closed loop

<footnote n="1">¹ Aix Marseille Univ, CNRS, CNES, LAM, Marseille, France</footnote>

<footnote n="2">² University of California Santa Cruz, 1156 High St, Santa Cruz, USA</footnote>

<footnote n="3">³ DOTA, ONERA, Université Paris Saclay, 91123 Palaiseau, France</footnote>

<footnote>* Corresponding author; francisco.oyarzun@lam.fr</footnote>
operation; however, as we showed in Oyarzún et al. (2024), the extension of the LGS (both due to the width of the laser beam and the thickness of the sodium layer) lowered the sensitivity to a point where the performance of the AO loop was significantly degraded. This decrease in sensitivity comes from the fact that the extension of the source acts similarly to modulation.

In this paper, using the end-to-end physical optics models from OOMAO (Conan & Correia 2014), we compare the performance with respect to noise of the SHWFS and the PWFS to determine which one is the best when using LGS. To this end, in Sect. 2, we extended a sensitivity formalism from the PWFS to the SHWFS, to be able to compare the same metrics for both. Then, in Sect. 3 we show end-to-end simulations to validate the predictions of the noise model and then use it to predict the noise propagation for 40 m telescopes. We then extended the analysis to compute the noise propagation for laser tomography adaptive optics (LTAO) systems with multiple WFSs.

## 2. Noise and sensitivity

### 2.1. Pyramid wavefront sensor

In this work, we use the reduced intensity approach presented in Chambouleymon et al. (2023) to compute the signal of the PWFS and the noise propagation. This reduced intensity is obtained by normalizing the intensity $I(\phi)$ recorded in the detector given an input phase $\phi$, by the number of photons in a frame $N_{ph}$, and then subtracting a reference intensity, $I_0 = \frac{I(\phi=0)}{N_{ph}}$, that has been chosen to represent the intensity for a flat wavefront. Thus, the reduced intensity is:

$$ \Delta I(\phi) = \frac{I(\phi)}{N_{ph}} - I_0. \qquad (1) $$

To reconstruct the phase from a given WFS measurement we use an interaction matrix $\mathcal{D} = [\delta(\phi_1), \dots, \delta(\phi_N)]$, calibrated using a push-pull procedure inputting orthogonal modes (KL modes) in the phase space $[\phi_1, \dots, \phi_N]$,

$$ \phi' = \mathcal{D}^{\dagger} \Delta I(\phi). \qquad (2) $$

As we record intensity information for given known phases, we have to invert the interaction matrix to recover the phase from the intensity measurements. For this, a pseudo-inverse approach is used, such that $\mathcal{D}^{\dagger} = (\mathcal{D}'\mathcal{D})^{-1}\mathcal{D}$.

Overall, PWFSs can operate with small, subelectron read-out noise (RON) detectors thanks to the limited number of pixel required. Nevertheless, to have the same conditions to compare with the SHWFS, we use a RON value of $\Sigma_{RON} = 2e^-/pix/frame$. To avoid confusion in this work, we denote $\Sigma_{RON}$ as the standard deviation of the electronic noise of the detector, measured in $e^-/pix/frame$, and $\sigma_{RON}^2$ the phase variance introduced due to RON.

Following Chambouleymon et al. (2023), it is possible to define the sensitivity to RON, $s(\phi_i)$, and to photon noise, $s_{\gamma}(\phi_i)$, for each corrected mode, $\phi_i$, such that the total phase variance introduced by noise, $\sigma_{RON+\gamma}^2$, at each measurement is:

$$ \begin{aligned} \sigma_{RON}^2 &= \sum_N \frac{N_{sap} \Sigma_{RON}^2}{N_{ph}^2 s^2(\phi_i)}, \\ \sigma_{\gamma}^2 &= \sum_N \frac{1}{N_{ph} s_{\gamma}^2(\phi_i)}, \\ \sigma_{RON+\gamma}^2 &= \sigma_{RON}^2 + \sigma_{\gamma}^2, \end{aligned} \qquad (3) $$

with $N_{sap}$ the number of subapertures. As a remark, the name sensitivity to RON (or photon noise) can be misleading; thus, sensitivity “against” RON (or photon noise) might be a better name in this regard. For consistency with the work developed by Chambouleymon et al. (2023), we use the former. For the control loop, we use the same one described in Oyarzún et al. (2024); thus, the noise propagated through the loop is:

$$ \hat{\sigma}_{RON+\gamma}^2 = \delta \sigma_{RON+\gamma}^2, \qquad (4) $$

with $\delta = 0.33$. In this work, we use the $\hat{\cdot}$ symbol to denote the noise is propagated via the loop.

### 2.2. Extending the sensitivity analysis to the SHWFS

Several works have derived theoretical formulas to predict the centroiding variance due to read-out and photon noise for a SHWFS for different centroiding algorithms (Rigaut & Gendron 1992; Rousset 1999; Thomas et al. 2006). Some have specifically focused on the prediction when using a laser guide star (Robert et al. 2010). In these papers, it is generally assumed that the source is diffraction-limited or it has a Gaussian profile. For both of these cases, analytical formulas have been derived considering the statistics of the noise.

These formulas are useful to observe the impact of the different design options of the AO system, but actual spot shapes might differ from the theoretical expectations. For example, in a real subaperture, a Gaussian object would not necessarily produce a Gaussian spot, as the image would also be convolved by the diffraction-limited spot. Elongated images of a laser guide star pose substantial difficulties, as each subaperture has a different image of the source. Also, LGS spots themselves might differ from Gaussian spots, given the density profile of the sodium layer.

With these issues in hand, we extended the sensitivity analysis developed for Fourier filtering WFS, presented in Chambouleymon et al. (2023), to the SHWFS. The signal from the SHWFS will be processed using the center of gravity (CoG) or the weighted CoG (WCoG, Nicolle et al. 2004) algorithm. We are interested in obtaining the same noise formalism as with the PWFS for the SHWFS, to be able to compare both using the same framework. We go on to consider the following operation to reconstruct the phase using SHWFS measurements:

$$ \phi' + \xi = \mathcal{D}_{SH}^{\dagger} \mathrm{CoG}(I(\phi) + b(\phi)), \qquad (5) $$

where $\phi'$ is the phase estimation, $\xi$ the error in the phase reconstruction due to noise, $\mathcal{D}_{SH}$ is the interaction matrix, $\mathrm{CoG}(I(\phi))$ is the CoG algorithm, $I(\phi)$ the noiseless intensity pattern in the detector, and $b(\phi)$ the intensity due to RON and photon noise. If we assume that the proportion of light each subaperture receives is constant, it is possible to build a matrix, $M$, that performs the CoG computation for all subapertures simultaneously if we normalize the intensity in the detector by the total number of photons in the frame

$$ \mathrm{CoG}(I(\phi) + b(\phi)) = M \left( \frac{I(\phi) + b(\phi)}{N_{ph}} \right). \qquad (6) $$

Note that for WCoG we can construct the same matrix, but take into account the weighting mask, which can be different for each subaperture if needed. Also, when using WCoG, optical gains are needed to ensure a unitary response for all subapertures (Thomas et al. 2006). For this, we performed a calibration
step where optical gains for the X and Y axis were computed for each subaperture, as this value changes depending on the elongation of the source. With the M matrix calibrated, we can express the phase reconstruction of the SHWFS as a series of matrix multiplications:

$$ \phi + \xi = \mathcal{D}_{SH}^{\dagger} M \frac{I(\phi) + b(\phi)}{N_{ph}}. \qquad (7) $$

If we define $n(\phi) = b(\phi)/N_{ph}$ and $G^\dagger = \mathcal{D}_{SH}^\dagger M$, the expression for the error in the phase reconstruction due to noise is

$$ \xi = G^{\dagger} n(\phi). \qquad (8) $$

As we arrive at the same equation as Eq. (8) in Chambouleyron et al. (2023), we can therefore follow the same steps to define both RON and photon noise sensitivities. For a given mode $\phi_i$, the RON and photon noise sensitivities can be computed as:

$$ s(\phi_i) = \sqrt{\frac{N_{sap}}{(G^\dagger G^{\dagger t})_{i,i}}}, \qquad (9) $$

$$ s_{\gamma}(\phi_i) = \frac{1}{\sqrt{(G^{\dagger} \mathbf{diag}(I_0) G^{\dagger t})_{i,i}}} \qquad (10) $$

with $N_{sap}$ as the number of subapertures and $I_0 = \frac{I(\phi=0)}{N_{ph}}$ as the raw reference intensity of the detector of the SHWFS. Here, we did not use the approximation $G^\dagger G^{\dagger t} = (G^t G)^\dagger$; therefore our expressions are slightly different, but carry out the same meaning. With the sensitivities, we can then use Eq. (3) to compute the total residual phase variance $\sigma_{RON+\gamma}^2$, introduced by RON and photon noise at each measurement.

It is important to remark that $\sigma_{RON+\gamma}^2$ corresponds to the residual phase variance introduced at each measurement of the SHWFS and not the centroiding error at each subaperture. Nevertheless, with this formalism, it is also possible to obtain the centroiding variance at each subaperture. This is done by doing the same statistical analysis, but without taking into account the interaction matrix, so that it is possible to obtain the raw centroids. We refer to Appendix A for the mathematical formalism and examples comparing the theoretical formulas with the predictions using this model. This could be useful, for example, if dealing with large simulations where the SHWFS model used is geometrical (i.e., computing directly the signal from the gradient of the incoming phase) instead of diffractive. In this case, the contributions of RON and photon noise to the centroiding variance can be precomputed by using a single noiseless image on the detector of the SHWFS.

To compare the sensitivity model with end-to-end simulations, we simulated 200 realizations with no atmosphere (i.e. a flat wavefront) of an open loop AO system with no controller for an SH with LGS for a 16 m telescope. We used the WCoG algorithm with a noiseless image of the detector as the weighting mask. For these simulations, we set the standard deviation of the read-out noise to be $2e^{-/pix/frame}$. Figure 1 shows with blue markers the result of the end-to-end simulations and with solid lines the expected residual variances predicted using the sensitivity model, where we found a good agreement between them. It is possible to observe that for low magnitudes the dominant term is photon noise (in green) and that at lower photon counts RON is the limiting factor (in red). We also repeated this for the regular CoG algorithm and found good agreement between end-to-end simulations and the predictions from the model.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Evolution with respect to the magnitude of the guide star of the residual phase due to noise in open loop for a SHWFS with LGS for a 16 m telescope. The solid lines correspond to the residual phase variance due to RON (red line), photon noise (green line), and the sum of both (blue line), as predicted by the model. The markers correspond to the mean of 200 end-to-end iterations, with the errorbar the standard deviation of the residual variance. For reference, at 1 kHz and magnitude 10, $N_{ph} = 1.8 \times 10^5$ photons on the full pupil.</figcaption></figure>

## 2.3. Comparing the sensitivities of both WFSs

A previous work by Plantet et al. (2015) explored the possibility of comparing both WFS with the same formalism, by using the Fisher information matrix. In it, the authors develop the relationship between the Fisher coefficients and the noise propagation coefficients, making it possible to compare different wavefront sensors with the same metric. The main difference with this work is that for the SHWFS, they use the centroids as signal and, for the centroiding variances, they computed the theoretical values given the desired centroiding algorithm and the shape of the spot. In this work, we use as signal the raw image from the SHWFS; therefore, we are not limited to the spot shapes the theoretical formulas have closed solutions for. With our formalism, the centroiding variance is a direct result (refer to Appendix A.7) that is computed in parallel for all subapertures simultaneously. As we do not make any assumptions on the shape of the source, the sensitivity approach presented here is a more general formalism.

To compute these sensitivities we need access to the interaction matrix. For the PWFS, we used the same approach as in Oyarzún et al. (2024), where we computed the interaction matrix for the LGS by updating the optical gains of an interaction matrix built using a point source. By doing this the interaction matrix for the 40 m telescope with 81 × 81 actuators can be computed in a couple of hours.

The interaction matrix for the SHWFS for a 40 m telescope with 81 × 81 actuators can also take a long time to compute. For this reason, we took a shortcut on how to compute it. We assume that each actuator in the deformable mirror behaves identically, and the SHWFS has the same response to each actuator, just shifted in position. We can simulate a single push-pull operation on the central actuator, and then shift the signal of the WFS, according the position of each actuator. Once we have this, we can gain access to the zonal interaction matrix. To get a modal interaction matrix, we can use a zonal-to-modal matrix,
<figure><img src="image_4.png" /><figcaption>Fig. 2. Sensitivity to RON (top plot) and photon noise (bottom plot) for an SHWFS (solid line) and a PWFS (dashed line), both using an LGS. The sensitivity was computed for telescopes of 8, 16, and 40 m in diameter (red, green, and blue, respectively).</figcaption></figure>

such that

$$D_{\text{modal}} = D_{\text{zonal}} M2C. \qquad (11)$$

The M2C matrix can be obtained by computing the modes of the modal basis using the influence functions of the deformable mirror. Partially illuminated subapertures may not be perfectly modeled in this approach, but they still manage to recover the wavefront accurately. With this approach, computing the modal interaction matrix can take minutes instead of weeks. Once we have the modal interaction matrix, we can compute the sensitivities to RON and photon noise for WCoG and compare them with the ones for the PWFS. This is presented in Fig. 2, where we compute the sensitivities for telescopes of 8, 16, and 40 m (red, green and blue curves, respectively). For both WFSs, we used the same number of subapertures, with a constant projected size of 50 cm. This means that for the 8 m telescope, we used 16 × 16 subapertures, while for the 40 m, we used 80 × 80 subapertures. The pixel size on the SHWFS was 0.5 arcsec (Nyquist sampling as the non-elongated axis has 1 arcsec of FWHM). The weighting mask of each subaperture had a specific weighting function that corresponded to the convolution of a noiseless image from the detector with a Gaussian with 1 arcsec of FWHM. This gives

a good balance of retaining sensitivity and giving robustness to the system.

Regarding the sensitivity to RON (top figure), it is possible to observe that both the SHWFS (solid lines) and the PWFS (dashed lines) are more sensitive to higher order modes than to lower-order ones. This is because both sensors are acting as gradient sensors: the SH is, by nature, a gradient sensor, and the PWFS (due to the size of the LGS) is acting in the gradient-sensing regime (Vérinaud 2004). The first three modes used for this work are tip, tilt, and focus, which allow us to filter them out in the closed loop that is then to be corrected by an external loop. The rest of the modes correspond to KL modes built to be orthogonal to these three Zernike modes. When comparing the sensitivity to RON of the SH and the PWFS, the latter is consistently more sensitive, being (on average) three times more sensitive for the 8 m telescope and up to five times more for the 40 m one. The sensitivity itself is not the whole picture, as the detectors needed for the PWFS are small (only 240 × 240 pixels for an 80 × 80 subaperture PWFS), it is possible to use subelectron RON detectors (Gach et al. 2011). This means that the value for $\Sigma_{\text{RON}}^2$ (and therefore $\sigma_{\text{RON}}^2$) in Eq. (3) would be much lower than for the SH.

Looking at the evolution of the sensitivity to RON with respect to the size of the telescope, it is possible to observe that both the SH and the PWFS have lower sensitivities as the diameter of the telescope increases. This can happen for at least two reasons: (1) for both WFS, to maintain the same equivalent diameter of the subapertures, bigger telescopes need more subapertures, which use more pixels. Increasing the number of pixels in the detector then decreases the sensitivity, as there is an increase in noisy measurements; (2) the elongation of the LGS increases with the diameter of the telescope. For the SH, this means that the field of view of each subaperture has to increase, which increases the impact of noise on the measurement. For the PWFS, the elongation acts in a similar way to modulation, which reduces the sensitivity.

The sensitivity to photon noise (bottom plot in Fig. 3) is almost the same for both the SH and the PWFS for the 8 m telescope, and as the diameter increases the difference grows, as for the 16 m telescope the PWFS is around 1.2 times more sensitive and for the 40 m approximately 1.7 times more sensitive. For the 8 m telescope, the main structure of the laser beam that is limiting the sensitivity for both the PWFS and the SHWFS is the width of around 1 arcsec, as the largest elongation is only around 2–3 arcsec. As the telescope increases in diameter, the elongation of the spot of the laser is more pronounced in the SHWFS than in the PWFS given the depth of field: the subapertures of the SHWFS have the depth of field of a 50 cm diameter telescope, which is able to have almost all of the image of the LGS in focus. On the other hand, the PWFS has the depth of field of the full aperture, meaning that the image of the LGS quickly gets defocused, leaving the central focused spot as the limiting structure. For the 40 m telescope, this is around 2.5 arcsec (Oyarzún et al. 2024).

The use of the WCoG algorithm is a trade-off between RON and photon noise, as increasing the size of the weighting mask reduces the propagation of photon noise, but it increases that of the RON (Nicolle et al. 2004). If the RON value of a detector is known, it is possible to use this sensitivity analysis to optimize the size of the weighing mask. For example, if using a detector with low RON, it could be useful to use a weighing mask substantially larger than the noiseless spots. Not only it would help reduce the overall noise propagation, but also it could increase the linearity of the sensor.
<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Comparison of the sensitivities of a PWFS (red) and a SHWFS (blue) with 80 × 80 subapertures, and a single SHWFS designed for HARMONI (green).</figcaption></figure>

By observing both sensitivities, the advantage of the PWFS is mainly in the higher sensitivity to RON, as both have similar sensitivities to photon noise. The higher sensitivity to RON allows the PWFS to use more subapertures to measure the wavefront to mitigate aliasing, without sacrificing performance (as it would be limited by photon noise).

In the case of the High Angular Resolution Monolithic Optical and Near-infrared Integral field spectrograph (HARMONI; Thatte et al. 2016), technological constraints forced the designers to lower the amount of subapertures from 80 × 80 to 68 × 68, and reduce the sampling from 0.5 arcsec/pix (i.e., Nyquist assuming a 1 arcsec spot) to 1.15 arcsec/pix. Cropping of the most elongated laser spots could also have an impact on the noise propagated, but it cannot be represented by the model; therefore we have not taken it into account in the present study. We used the sensitivity method to compare how noise behaves in a single SHWFS of HARMONI against an 80 × 80 subaperture SHWFS with 0.5 arcsec pixels. Further analyses are presented in Sect. 3.2 for the LTAO system; however, as a first step, we discuss the effects on a single WFS in a single conjugated adaptive optics (SCAO) configuration. Figure 3 shows the sensitivities to RON and photon noise for a 40 m telescope using LGS with an 80 × 80 subaperture PWFS, an 80 × 80 subaperture SHWFS, and a single 68 × 68 SHWFS designed for HARMONI in the SCAO

configuration. As expected, the one designed for HARMONI has a higher sensitivity to RON, as it uses fewer subapertures and fewer pixels to make its measurements. In contrast, it has a lower sensitivity to photon noise. This could be due to the use of larger pixels, therefore leading to undersampling, as well as subsequent non-linearities and a loss in terms of sensitivity (Thomas et al. 2006).

# 3. Closed loop systems

## 3.1. *End-to-end simulations*

To observe the predictive capabilities of the sensitivity method, we performed closed-loop simulations with 8 and 16 m telescopes for both a SHWFS and a PWFS. We then compared the results with those predicted by the model. The simulation parameters are found in Table 1. Here, we chose a static atmosphere, such that the temporal errors due to wind did not affect the measurement of the contribution of RON and photon noise to the wavefront error. For the same reason, we chose the atmosphere to have a single ground layer, such that the cone effect would not affect the simulations, given that it is not correlated with noise. The cone effect will have a significant impact on the performance of the ELT, but to deal with this effect LTAO is planned to be used, but tomographic reconstruction of the wavefront is outside the scope of this work. We used the same sodium density profile for both WFS that was presented in Fig. 2 of Pfrommer & Hickson (2014). We used a simple integrator in the feedback path with two frames of delay for the correction and a loop gain of 0.3. We also chose to include a simulation of a PWFS with an NGS, modulated at $4\lambda/D$, to have a comparison for the LGS. The modulation radius was chosen to have a good trade-off between sensitivity and dynamic range. As the LGS can't measure global tip or tilt and has difficulties measuring focus, a separate, noiseless AO loop was used to correct for these modes. For the science wavelength, we chose the same as the one used for the sensing, as the difference between the performance curves and, therefore, the comparison between the WFS is independent of wavelength.

One disadvantage that both WFSs would have in common when using an LGS is the optical gain tracking (assuming a WCoG for the SHWFS), as the evolving structure of the sodium layer and changing seeing conditions will impact these values. Extra complexity both in software and hardware has to be included to compute the optical gains in real time. As LGS wavefront sensing is (mostly) blind to tip-tilt, a SHWFS system might introduce a tip-tilt mirror to modulate the LGS spots with a known displacement (Saddlemeyer et al. 2004), such that it would be possible to estimate the optical gains by taking the ratio of the measured and known displacements. For the PWFS, the introduction of a gain scheduling camera as proposed in Chambouleyron et al. (2021) could allow for the online estimation of the optical gains.

We tested several magnitudes for the guide stars and plotted the Strehl ratio obtained with the closed loop for each case in Fig. 4. For the residual phase due to fitting and aliasing, we simulated 20 phase realizations of a noiseless PWFS with NGS and averaged the variance of the residual phase. The low performance is due to the low number of KL modes used to correct the wavefront. The fitting error alone limits the performance to around 20%, while the aliasing and reconstruction errors further reduce it to the 10–15% range.

Regarding aliasing, it has been shown that when using an NGS, the PWFS has lower residual phase variance due to aliasing than the SH (assuming the same number of subapertures),
<table><caption>**Table 1.** Simulation parameters</caption><tbody><tr><td>Telescopes</td><td></td></tr><tr><td>Diameter</td><td>8.0, 16.0</td></tr><tr><td>Throughput</td><td>100 %</td></tr><tr><td>Central obstruction</td><td>None</td></tr><tr><td>Natural guide star</td><td></td></tr><tr><td>Zenith angle</td><td>0°</td></tr><tr><td>Magnitudes</td><td>5–16</td></tr><tr><td>Zero point</td><td>8.96 × 10<sup>9</sup> photons/s/m<sup>2</sup></td></tr><tr><td>Modulation</td><td>4 λ/D</td></tr><tr><td>Laser guide star</td><td></td></tr><tr><td>Launch geometry</td><td>Side launch</td></tr><tr><td>Zenith angle</td><td>0°</td></tr><tr><td>Magnitudes</td><td>5–14</td></tr><tr><td>Zero point</td><td>8.96 × 10<sup>9</sup> photons/s/m<sup>2</sup></td></tr><tr><td>Number of samples</td><td>10 000</td></tr><tr><td>Sodium profile</td><td>TopHatPeak</td></tr><tr><td>Atmosphere</td><td></td></tr><tr><td>r<sub>0</sub></td><td>15 cm</td></tr><tr><td>L<sub>0</sub></td><td>25 m</td></tr><tr><td>Layers</td><td>1</td></tr><tr><td>Altitudes</td><td>0 m</td></tr><tr><td>Wind speed</td><td>0 m/s</td></tr><tr><td>WFS</td><td></td></tr><tr><td>Order for 8 m telescope</td><td>16 × 16 subapertures</td></tr><tr><td>Order for 16 m telescope</td><td>32 × 32 subapertures</td></tr><tr><td>Frequency</td><td>1 KHz</td></tr><tr><td>λ<sub>sens</sub></td><td>589 nm</td></tr><tr><td>SHWFS specific parameters</td><td></td></tr><tr><td>Pixel scale</td><td>0.5 arcsec/pix</td></tr><tr><td>Subaperture FoV for 8 m telescope</td><td>5 arcsec</td></tr><tr><td>Subaperture FoV for 16 m telescope</td><td>10 arcsec</td></tr><tr><td>DM</td><td></td></tr><tr><td>Order for 8 m telescope</td><td>17 × 17 actuators (200 KL modes)</td></tr><tr><td>Order for 16 m telescope</td><td>33 × 33 actuators (700 KL modes)</td></tr><tr><td>AO loop</td><td></td></tr><tr><td>Delay</td><td>2 frames</td></tr><tr><td>Gain</td><td>0.3</td></tr><tr><td>Science</td><td></td></tr><tr><td>λ<sub>sci</sub></td><td>589 nm</td></tr></tbody></table>

given its flattening sensitivity curve (Vérinaud 2004). Nevertheless, for extended objects, the size of the guide star has the same effect as modulation, which makes the PWFS operate as a gradient sensor. Therefore, we would expect to see a similar behavior here as we would for the SH.

To obtain the curves shown in Fig. 4, we first had to obtain the interaction matrices for each case. For the PWFS, we took

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** Performance of the AO loop for a SHWFS (green curves) and a PWFS (red curves) using LGS for an 8 (top) and 16 m telescopes (bottom). A PWFS using NGS (blue curve) was added as a reference for the performance.</figcaption></figure>

the convolutional approach as in Oyarzún et al. (2024), where we computed the interaction matrix for an NGS and then used a convolutional model (Fauvarque et al. 2019) to compute the optical gains (Korkiakoski et al. 2008; Deo et al. 2018; Chambouleyron et al. 2021) to optimize the interaction matrix for the LGS. With the interaction matrix, we could directly compute the sensitivities to RON and photon noise. For the SH, we computed the interaction matrix assuming that each actuator has the same signal footprint in the detector, just shifted depending on the position of the actuator. Then, we computed the optical gains to ensure a unitary response across all subapertures and finally build the matrix that performs the centroiding, which took the optical gains into account. With the interaction matrix and the centroiding matrix we could use Eqs. (9) and (10) to compute the sensitivities to RON and photon noise, respectively.

Let us start analyzing only the theoretical performance expectations, represented as the curves in the figures and not the E2E results, represented as the markers. Figure 4 shows two curves for each tested case: the performance limited only by photon noise with a dashed curve and the performance limited by both RON (2 e⁻/pix/frame) and photon noise as a solid line. This distinction was made to understand which noise source is
limiting the system. If the solid curve is close to the dashed, then the system would mainly be photon noise limited. On the other hand, if the solid curve deviates from the dashed, then the system would be limited mainly by RON. It is important to keep in mind that the evolution of noise propagated for RON and photon noise is different. For RON, residual phase variance goes as $N_{ph}^{-2}$ and for photon noise it goes as $N_{ph}^{-1}$. This means that once the drop in performance starts (the “knee” in the plot), the fall is much steeper for RON than for photon noise.

Being limited by one or the other noise has different implications. If the system is limited by photon noise, it means that it is possible to increase the number of pixels in the detector to improve aliasing or linearity, without degrading the noise performance; however, improving the detector technology would not have a big impact on the noise propagated. On the other hand, if the system is limited by RON, then improving the technology (i.e., lowering the RON) would have a positive impact on the performance.

Specifically, for the SH, if RON were dominating, then narrower weighting functions could be used at each subperture to increase the performance. On the other hand, if it was limited by photon noise (e.g., using a detector with less RON), increasing the size of the weighting mask could improve the performance and linearity.

For the 8 m telescope, the PWFS with NGS (represented with the color blue) has the best noise performance. It can operate without a significant loss in performance up to magnitude 12. In this scenario, the solid line deviates from the dashed, meaning that RON is the limiting factor. In a real scenario, this would not be the case, because the low number of pixels allows for the use of subelectron RON detectors. Therefore, by improving the technology the performance can be similar to the dashed blue curve.

For the LGS, both the PWFS (in red) and the SH (in green) have a similar performance with respect to photon noise, as both dashed curves are nearly on top of each other. Nevertheless, even though both have the same RON, the higher sensitivity to RON of the PWFS makes it perform better than the SH. Using a detector with lower RON would not directly improve the noise performance of the PWFS with LGS, but it would allow us to use more pixels (and, thus, more subapertures) to better sample the wavefront and help mitigate aliasing effects. The limiting source of noise for the SH is not as straightforward as with the PWFS. In this case, at high flux, the performance is limited by photon noise, as it is possible to observe that the beginning of the drop in performance follows the dashed curve. Then, as the photon flux diminishes, RON, with its steeper decrease dominates. This means that improving the technology of the detector would not improve the limiting magnitude, it would just improve the performance once it has already been affected by the return flux.

For the 16 m telescope, the performance of the PWFS with NGS has almost the same evolution with magnitude as with the 8 m telescope. Having a bigger telescope gives a higher collection of photons, which reduces the overall noise propagated. Nevertheless, the bigger telescope needs more subapertures to reach the same pitch as the smaller one. These two effects cancel out each other, making the overall performance approximately independent on the telescope size.

For the LGS, the expected performance of both the PWFS and the SH are similar to that of the 8 m telescope, but the curves are shifted to the left around 1 magnitude. Given the same arguments as for the NGS, bigger telescopes gather more light, but need more pixels and subapertures to maintain the same

<figure><img src="image_7.png" /><figcaption>**Fig. 5.** Expected performance of the AO loop for a SHWFS and a PWFS using LGS for a 40 m telescope. A PWFS using NGS was added as a reference for the performance.</figcaption></figure>

pitch. On top of that, the elongation of the LGS is increased by the size of the telescope: the larger the elongation, the lower the sensitivity; therefore, this leads to a worse performance. Looking only at the PWFS, we can observe that the solid curve is almost the same as the dashed one; therefore, photon noise is evidently the limiting factor. This means that increasing the number of pixels and therefore subapertures is a possible options to help mitigate aliasing. For the SH, again the limiting magnitude is given by photon noise, as the “knee” in the plot is the same as the dashed curve and, later, RON starts to dominate at lower photon fluxes.

Now observing the markers, for the PWFS with NGS it is possible to observe a good agreement between the theoretical expectations and the E2E simulations. The drop in performance is around 0.3 magnitudes less for the E2E simulations than what was expected with the model. This can be explained as the sensitivity model does not take into account the non-linearities of the PWFS, which are known to decrease the sensitivity (Chambouleyron et al. 2020). For the extended objects, both the PWFS and SH show good agreement between the E2E simulations and the theoretical curves.

## 3.2. Prediction of the performance of the 40 m telescope

Having shown that the sensitivity model accurately predicts the performance of the AO loop, we can now use it for the 40 m telescope for both WFS. These correspond to 80 × 80 subapertures, with an 81 × 81 deformable mirror with 5100 actuators. The sampling of the SHWFS is 0.5 arcsec/pix with a field of view of 25 arcsec. A detector of this size and noise characteristics is not available, but (as we show in Figure 3) we would expect a similar performance as if using the detector designed for HARMONI. The weighting mask used was a noiseless image of the detector, convolved with a 0.5 arcsec FWHM Gaussian for purposes of robustness and linearity. Figure 5 shows the expected performance for both the SHWFS and the PWFS for a 40 m telescope. A yellow vertical stripe was included in the plot as a reference for the expected return of the LGS.

Considering the chosen design parameters, the PWFS is photon noise limited, and the SH is mainly photon noise limited, but RON has an impact at lower fluxes. The PWFS has a limiting magnitude that is around one magnitude higher than the SH, and
<table><caption>**Table 2.** Average variance of the signal coming from the SHWFS and PWFS for a 40 m telescope with six LGS using the generalized least squares estimator.</caption><thead><tr><th>Wavefront sensor</th><th>Normalized average signal variance due to RON</th><th>Normalized average signal variance due to Photon noise</th></tr></thead><tbody><tr><td>6 SH + 6 LGS</td><td>0.81</td><td>0.24</td></tr><tr><td>6 PWFS + 6 LGS</td><td>0.70</td><td>0.50</td></tr></tbody></table>

**Notes.** The variances are normalized by the variance obtained by using a 1-arcsec spot as a guide source.

the fact that it is photon noise limited means that increasing the number of pixels could increase the performance with respect to aliasing, without sacrificing performance with respect to noise. Nevertheless, at the expected return fluxes for the LGS (magnitudes of between 7 and 9), both the PWFS and the SH are not only in a low-performance region, but small changes in the return flux would also result in sizeable variations in performance. Considering the SH designed for HARMONI, as it has a higher sensitivity to RON and a slightly lower sensitivity to photon noise than the SH tested here (refer to Fig. 3), we expect it to behave in a similar way to the well-sampled, 80 × 80 subaperture SH.

### 3.3. Expected noise performance in a tomographic configuration for a 40 m telescope

In general, the use of LGS for wavefront sensing is not done with a single guide source, but several of these are implemented to, for example, use wide-field adaptive optics, or mitigate cone effect. For this, laser tomography adaptive optics (LTAO) or multi-conjugate adaptive optics (MCAO) systems are used.

In the following discussion, we estimate the noise performance of an LTAO system. Tomographic reconstructors pose substantial difficulties in estimating the propagation of noise; therefore, we chose to use only a simplified version. We will assume each WFS observes the same atmosphere and has uncorrelated RON and photon noise. This approach is not useful for the tomographic reconstruction of the atmosphere, but it can give insights into the expected performance with respect to noise. Using LTAO also allows for the use of super-resolution (Oberti et al. 2022), which could help mitigate aliasing. This will not be taken into account for the discussion; therefore, the value of the Strehl ratio at high fluxes is not representative of a full LTAO reconstruction, however, the evolution with respect to noise is.

Let us assume an LTAO system with six LGS and six WFS. For both wavefront sensors, each subaperture sees a different perspective of the LGS, elongated and oriented given the position of the subaperture in the pupil and the location of the laser launch telescope. This implies that each subaperture will produce a measurement (centroiding for the SH and slopes for the PWFS) with a variance that is dependent on its position in the pupil. Similar to what has been done in Tallon et al. (2010) and Béchet et al. (2012), in Appendix C we show that by using a Generalized least square estimator (GLSE) to combine the measurements at each subaperture produced by the LGSs, the overall variance of the signal at each subaperture is better than using a single WFS with a non-elongated 1-arcsec spot. Table 2 shows the average signal variances for both wavefront sensors using 6 LGS, combining the signal using the GLSE. The variances are normalized by the signal variance of a non-elongated 1 arcsec spot. We used the results from the GLSE for the noise propagation as an approximation of the multiple LGS system.

Figure 6 shows the expected performance of these approximated systems. The increase in performance of the SHWFS is

<figure><img src="image_8.png" /><figcaption>**Fig. 6.** Performance of the AO loop for the best case LTAO using PWFS (in black) and SHWFS (in cyan). In red and green is the performance of a single PWFS and SHWFS, respectively.</figcaption></figure>

drastic, as the limiting magnitude of the best-case scenario (cyan curve) is 4.4 magnitudes greater than for the single WFS case. For the PWFS, the increase in performance is of 3.2 magnitudes (black curve). By combining the measurements, the SHWFS could be capable of outperforming the PWFS by 0.3 magnitudes, requiring about 25 % less photons to achieve the same performance. Nevertheless, both WFS would be able to operate without any significant loss in performance at the expected return fluxes of the LGS. As the SH is affected more by RON than the PWFS, we increased the value and found that the PWFS system (in a multiple WFS configuration) can perform better than the SH system if RON is higher than $7e^{-}/\text{pix/frame}$. However, as the pixel requirements of the PWFS are low, it can use a sub-electron RON detector; in that case, the performances would be almost identical (dashed black curve in Figure 6).

A pyramid-like WFS could be used with the geometry of the LGS into consideration, as, for example, in the Ingot WFS proposed by Ragazzoni et al. (2024). Since it follows the height extension of the LGS, it could operate as if it were a regular PWFS with a 1 arcsec spot. In this ideal scenario, the equivalent 6 Ingot + 6 LGS with the generalized least square estimator could outperform the SH system by around one magnitude.

## 4. Conclusion

In this work, we compute the expected performance of the AO loop for the SHWFS and the PWFS, using an analytical model. To do so, we extended a sensitivity analysis developed for the PWFS to the SHWFS by expressing the centroiding algorithm as a matrix multiplication. This allowed us to use the same
formalism for both wavefront sensors and also use the same metrics for a fair comparison between the two.

We found that the main difference between the two wavefront sensors is in the sensitivity to RON, where the PWFS has an advantage. Sensitivity to photon noise is the same for both in an 8 m telescope and 1.7 times higher for the PWFS at 40 m.

As photon noise is evidently limiting the performance of the PWFS, it could be possible to increase the number of subapertures to mitigate aliasing. For the SHWFS, both RON and photon noise are limiting its performance; therefore, it is not advisable to increase the number of subapertures.

Approximating the noise propagation of the LTAO system by using a generalized least-squares estimator, we found that the multiple SH system has a limiting magnitude up to 4.4 magnitudes higher than the single WFS system. For the PWFS, it would increase the limiting magnitude by 3.2. In this scenario, the SH system has a limiting magnitude 0.3 higher than the PWFS system. If using a sub-electron RON detector for the PWFS, both systems would perform almost identically. Nevertheless, if a WFS designed specifically for the LGS geometry would be capable of achieving the performance of a PWFS with a non-elongated 1 arcsec spot, it could outperform the SH system by one magnitude. If the LTAO reconstruction were capable of performing as well as the approximated model, either WFS would display almost the same performance with respect to noise.

**Acknowledgements.** This work benefited from the support of the French National Research Agency (ANR) with WOLF (ANR-18-CE31-0018), APPLY (ANR-19-CE31-0011) and LabEx FOCUS (ANR-11-LABX-0013); the Programme Investissement Avenir F-CELT (ANR-21-ESRE-0008), the Action Spécifique Haute Résolution Angulaire (ASHRA) of CNRS/INSU co-funded by CNES, the ECOS-CONYCIT France-Chile cooperation (C20E02), the ORP-H2020 Framework Programme of the European Commission's (Grant number 101004719), STIC AmSud (21-STIC-09), the french government under the France 2030 investment plan, as part of the Initiative d'Excellence d'Aix-Marseille Université A*MIDEX, program number AMX-22-RE-AB-151, the Conseil régional Provence-Alpes-Côte d'Azur with the emplois jeune doctorant program, co-funded by First Light Imaging, and the Millennium Science Initiative Program (ACIP, NCN19 161).

## References

Béchet, C., Tallon, M., & Thiébaut, É. 2012, SPIE Conf. Ser., 8447, 84472C

Chambouleyron, V., Fauvarque, O., Janin-Potiron, P., et al. 2020, A&A, 644, A6

Chambouleyron, V., Fauvarque, O., Sauvage, J. F., Neichel, B., & Fusco, T. 2021, A&A, 649, A70

Chambouleyron, V., Fauvarque, O., Plantet, C., et al. 2023, A&A, 670, A153

Ciliegi, P., Agapito, G., Aliverti, M., et al. 2022, SPIE Conf. Ser., 12185, 1218514

Conan, R., & Correia, C. 2014, SPIE Conf. Ser., 9148, 91486C

Davies, R., Schubert, J., Hartl, M., et al. 2016, SPIE Conf. Ser., 9908, 99081Z

Deo, V., Gendron, É., Rousset, G., Vidal, F., & Buey, T. 2018, SPIE Conf. Ser., 10703, 1070320

Fauvarque, O., Janin-Potiron, P., Correia, C., et al. 2019, J. Opt. Soc. Am. A, 36, 1241

Foy, R., & Labeyrie, A. 1985, A&A, 152, L29

Fusco, T., Neichel, B., Correia, C., et al. 2019, in AO4ELT6

Gach, J.-L., Balard, P., Stadler, E., Guillaume, C., & Feautrier, P. 2011, in Second International Conference on Adaptive Optics for Extremely Large Telescopes, Online at http://ao4elt2.lesia.obspm.fr 44

Gilmozzi, R., & Spyromilio, J. 2007, The Messenger, 127, 11

Korkiakoski, V., Vérinaud, C., & Louarn, M. L. 2008, Appl. Opt., 47, 79

Nicolle, M., Fusco, T., Rousset, G., & Michau, V. 2004, Opt. Lett., 29, 2743

Oberti, S., Correia, C., Fusco, T., Neichel, B., & Guiraud, P. 2022, A&A, 667, A48

Oyarzún, F., Chambouleyron, V., Neichel, B., Fusco, T., & Guesalaga, A. 2024, A&A, 686, A1

Pfrommer, T., & Hickson, P. 2014, A&A, 565, A102

Plantet, C., Meimon, S., Conan, J. M., & Fusco, T. 2015, Opt. Express, 23, 28619

Primmerman, C., Murphy, D., Page, D., Zollars, B., & Barclay, H. 1991, Nature, 353, 141

Ragazzoni, R. 1996, J. Mod. Opt., 43, 289

Ragazzoni, R., Portaluri, E., Greggio, D., et al. 2024, A&A, 688, A21

Rigaut, F., & Gendron, E. 1992, A&A, 261, 677

Robert, C., Conan, J.-M., Gratadour, D., Schreiber, L., & Fusco, T. 2010, J. Opt. Soc. Am. A, 27, A201

Rousset, G. 1999, in Adaptive Optics in Astronomy, ed. F. Roddier, 91

Saddlemyer, L. K., Herriot, G., Vrran, J.-P., Smith, M., & Dunn, J. 2004, SPIE Conf. Ser., 5490, 1384

Tallon, M., Tallon-Bosc, I., Béchet, C., et al. 2010, SPIE Conf. Ser., 7736, 77360X

Thatte, N. A., Clarke, F., Bryson, I., et al. 2016, SPIE Conf. Ser., 9908, 99081X

Thomas, S., Fusco, T., Tokovinin, A., et al. 2006, MNRAS, 371, 323

Vérinaud, C. 2004, Opt. Commun., 233, 27
## Appendix A: Analytical method of computing the centroid variance

With the matrix formalism to express the CoG algorithm, it is possible to compute the expected centroiding variances given RON and photon noise. We consider Eq. (7), but expressed to compute the centroid X and the centroiding error Ξ as follows:

$$X + \Xi = M \frac{I(\phi) + b(\phi)}{N_{ph}}, \qquad (A.1)$$

defining $n(\phi) = b(\phi)/N_{ph}$ we can express the centroiding error as

$$\Xi = M n(\phi). \qquad (A.2)$$

With this we can now compute the covariance matrix of the CoG:

$$\langle \Xi \Xi' \rangle = M \langle n(\phi)n(\phi') \rangle M', \qquad (A.3)$$

By using the statistics of the noise, we can compute the centroiding variance due to RON and photon noise. For RON, denoted as $\Sigma_{RON}$ to avoid confusion, we have that

$$\langle n(\phi)n(\phi') \rangle = \frac{\Sigma_{RON}^2}{N_{ph}^2} I, \qquad (A.4)$$

with $I$ the identity matrix. We can replace this in the centroiding covariance matrix to obtain

$$\langle \Xi \Xi' \rangle = \frac{\Sigma_{RON}^2}{N_{ph}^2} M M'. \qquad (A.5)$$

The diagonal of the centroiding covariance matrix would then correspond to the centroiding variance due to read-out noise. For photon noise, we have that the statistics of it follow a Poisson distribution given the light distribution on the detector. For an NGS, the spot shape changes considerably in operation, but for an LGS, its shape remains approximately constant. If we consider $I_0$ to be the noiseless raw image on the detector of the SHWFS, normalized by the number of photons for the measurement to be independent of the intensity, the statistics of the noise is

$$\langle n(\phi)n(\phi') \rangle = \frac{\mathrm{diag}(I_0)}{N_{ph}} I, \qquad (A.6)$$

The centroiding covariance matrix for photon noise can then be expressed as

$$\langle \Xi \Xi' \rangle = \frac{1}{N_{ph}} M \mathrm{diag}(I_0) M'. \qquad (A.7)$$

When computing the theoretical expectations for the centroiding variance, it is generally assumed that the source is diffraction-limited or it has a Gaussian profile. For both of these cases, analytical formulas have been derived considering the statistics of photon noise. For a diffraction-limited spot, Thomas et al. (2006) showed that the centroiding error due to photon

<figure><img src="image_9.png" /><figcaption>Fig. A.1. Centroiding error for CoG due to photon noise for a diffraction-limited source with 100 photons per subaperture per frame. The markers correspond to the E2E simulations, the green line to the theoretical expectation and the blue line to the prediction using the extended sensitivity model.</figcaption></figure>

noise is dependent on the field of view (FoV) of the subapertures, given by

$$\sigma_{\gamma}^{2} \approx 2 \frac{W}{n_{ph}}, \qquad (A.8)$$

where $W$ is the FoV of the subaperture expressed in $\lambda/d$ units, and $n_{ph}$ is the number of photons per subaperture per frame. The approximation becomes better for bigger FoV of several $\lambda/d$. For a well-sampled Gaussian profile (i.e., over 2 pixels per FWHM), with its FWHM sampled using $N_T$ pixels, and $N_{ samp}$ the number of pixels used to sample a diffraction-limited spot, the formula for the centroiding error due to photon noise is

$$\sigma_{\gamma}^{2} = \frac{\pi^{2}}{2 \ln 2} \frac{1}{n_{ph}} \left( \frac{N_{T}}{N_{ samp}} \right)^{2}. \qquad (A.9)$$

Figure A.1 shows the evolution of centroiding variance due to photon noise for a diffraction-limited spot with varying FoV, which is accurately predicted by both the theoretical and the new model. The markers correspond to the centroiding variance simulation of $10^5$ independent iterations, the solid red line corresponds to the theoretical expectation using Eq. (A.8), considering 100 photons per subaperture and per frame, and the yellow line corresponds to the new method corresponding to the extension of the sensitivity analysis.

For the Gaussian spots, we set the FoV of the subaperture at 3 arcsecs and then varied the FWHM of the spot and did the same number of simulations as with the diffraction-limited case. Figure A.2 shows the results from the E2E simulations, theory and the new model, again with good agreement between them. We intentionally set the FoV of the subaperture comparable to the biggest FWHM tested, to emphasize the fact that the theoretical formula assumes that the FoV is much bigger than the spot, but in reality, this spot gets cropped, an effect that is not taken into account by the theoretical formula, but is better predicted by this new model.

Theoretical predictions of the centroiding variance cannot reach much further than these two cases: there are no closed formulas for when the light distribution is not one of these cases.
<figure><img src="image_10.png" /><figcaption>Fig. A.2. Centroiding error for CoG due to photon noise for a Gaussian source with 100 photons per subaperture per frame. The markers correspond to the E2E simulations, the red line to the theoretical expectation and the yellow line to the prediction using the extended sensitivity model.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. A.3. Centroiding error for CoG due to photon noise for the convolution of a Gaussian source with the diffraction-limited spot with 100 photons per subaperture per frame, a field of view of 15 λ/D and a pixel scale of 0.12 arcsec/pix. The markers correspond to the E2E simulations, the red line to the theoretical expectation and the yellow line to the prediction using the extended sensitivity model.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. A.4. Centroiding error for WCoG due to RON and photon noise for an LGS with an 8 m telescope and a 16 × 16 subaperture SH with 100 photons per subaperture per frame. The black curve corresponds to the E2E simulations, the red line to the predicted centroiding variance due to RON, the blue line to the predicted variance due to photon noise and the green line the sum of the two. Both X and Y centroiding variances are shown in the same plot. The left part of the plot is for the variance along the X axis and the right part of the plot is the variance along the Y axis</figcaption></figure>

In Appendix B, we show that it is possible to express the CoG variance of the convolution of two sources (e.g. the convolution of the diffraction-limited spot and a Gaussian object), as the sum of the centroiding variances of each separately. To test this, we simulated using E2E propagation the spot for a Gaussian object, that was affected by the size of the optics, meaning that it was convolved with the diffraction-limited spot. We then varied the FWHM of the Gaussian object and recorded the evolution of the centroiding error due to photon noise, as can be observed in Fig. A.3. Both the theoretical predictions and the new model can predict the behavior of the simulation.

We have shown that the new method using Eq. (A.7) can replicate the results predicted by the theory. The benefit of this new method is that it can predict the centroiding variance for any shape of the spot. This is particularly useful for its use with

LGS, as each subapertures has a different spot shape, and depending on the sodium density profile, the spots themselves can differ from Gaussian objects. Also, as the orientation of the spot depends on the physical position of the subaperture, we don't need to assume that the object's elongation is along one of the axes. Finally, this model allows to predict the centroiding variance when using WCoG, a task that can be too complicated to do in theory. Figure A.4 shows a simulation of 10, 000 independent iterations of centroiding for a 16 × 16 subaperture SH with an LGS for an 8 m telescope. We added RON of 2 e-/pix/frame and computed the expected centroiding variance for both RON and photon noise. The total expected centroiding variance accurately predicted the one obtained in the independent simulation. We plotted both the centroiding in the X and Y direction in the plot, and there is a clear difference in behavior between the two, given that the elongation of the LGS is along the Y axis.

## Appendix B: Centroiding variance for convolved images

An image $P(x, y)$ in the detector can be expressed as the convolution of two images $f(x, y)$ and $g(x, y)$, for instance, diffraction-limited and Gaussian spots, as follows

$$P(x, y) = n_{ph} f(x, y) * g(x, y), \qquad (B.1)$$

with $n_{ph}$ the number of photons in the image. Here the images have to be normalized such that

$$\iint f(x, y)dxdy = \iint g(x, y)dxdy = 1. \qquad (B.2)$$
Consider the following expressions to compute the CoG and the centroiding variance due to photon noise:

$$
\begin{align}
\bar{x}_P &= \frac{\iint x P(x, y) dx dy}{\iint P(x, y) dx dy} = \iint x f(x, y) * g(x, y) dx dy, \nonumber \\
\sigma_{x_P}^2 &= \frac{\iint (x - \bar{x}_P)^2 P(x, y) dx dy}{\iint P(x, y) dx dy}, \tag{B.3} \\
&= \iint (x - \bar{x}_P)^2 f(x, y) * g(x, y) dx dy. \nonumber
\end{align}
$$

Let us consider both *f* and *g* as probability density functions (PDF). If we sum the two PDF, the resulting PDF is obtained by the convolution of *f* and *g*, which is what we are interested in. The resulting PDF will have a mean (or CoG) equal to the sum of the means of each independent PDF, and the variance of the new PDF will be equal to the sum of the variances of *f* and *g*. Therefore, considering the expressions for the CoG and the centroiding variances given in B.3, for the convolution of two images we have

$$
\begin{align}
\bar{x}_P &= \bar{x}_f + \bar{x}_g, \nonumber \\
\sigma_{x_P}^2 &= \sigma_{x_f}^2 + \sigma_{x_g}^2. \tag{B.4}
\end{align}
$$

This derivation is not possible is using WCoG, as then cross-
correlation terms appear in the expression that do not allow us
to separate the contribution to the centroiding variance of each
image that is convolved.

**Appendix C: Estimating the centroiding variance**
**for multiple LGS systems**

If we have knowledge about the centroiding covariance matrix, it
is possible to combine the measurements of multiple LGS with
a Generalized least squares estimator. If we have *N* LGSs, for
each subaperture we have *N* measurements *x*'<sub>1</sub>, *x*'<sub>2</sub>, ..., *x*'<sub>*N*</sub> and *N*
measurements *y*'<sub>1</sub>, *y*'<sub>2</sub>, ..., *y*'<sub>*N*</sub>. Given the direction of the elonga-
tion of the LGS, *x*'<sub>*i*</sub> and *y*'<sub>*i*</sub> might be correlated, and instead of
measuring along the regular *x/y* axis, a rotated axis might be
used to align the measurements with the elongation direction. If
we combine all the measurements into a single linear system, in
matrix notation we have

$$
\begin{pmatrix} x'_{1} \\ x'_{2} \\ \vdots \\ x'_{N} \\ y'_{1} \\ y'_{2} \\ \vdots \\ y'_{N} \end{pmatrix} = \begin{pmatrix} \cos(\theta_1) & \sin(\theta_1) \\ \cos(\theta_2) & \sin(\theta_2) \\ \vdots & \vdots \\ \cos(\theta_N) & \sin(\theta_N) \\ -\sin(\theta_1) & \cos(\theta_1) \\ -\sin(\theta_2) & \cos(\theta_2) \\ \vdots & \vdots \\ -\sin(\theta_N) & \cos(\theta_N) \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}, \quad (C.1)
$$

with $\theta_i$ the rotated angle of the axis in which the measure-
ments are made, and $x, y$ are the unknown centroid coordinates.
In matrix form, the equation can be written as

$$
\mathbf{x}' = B\mathbf{x}. \tag{C.2}
$$

<figure><img src="image_13.png" /><figcaption>Fig. C.1. SH centroiding variances in X and Y direction for a 40 m tele-scope resulting from the combination of 6 LGSs with the Generalized least square. The variances are normalized by the variance of a 1 arcsec spot.</figcaption></figure>

If we have the centroiding covariance matrix Σ of the N *x*'<sub>*i*</sub>
*y*'<sub>*i*</sub> measurements, the solution to the Generalized least squares
problem is

$$
\mathbf{x} = (B^T \Sigma^{-1} B)^{-1} B^T \Sigma^{-1} \mathbf{x}' \quad (C.3)
$$

from which the covariance of the estimates **x** is:

$$
\mathrm{Cov}(\mathbf{x}) = (B^T \Sigma^{-1} B)^{-1}. \qquad (C.4)
$$

In this work, we use six LGSs around the primary mirror.
Figure C.1 shows the centroiding variance in the X and Y direc-
tion at each subaperture, resulting from the combination of the
measurements from each LGS. The variances are normalized by
the variance of a 1 arcsec spot. For RON, the average centroiding
variance in both X and Y directions is 0.81 times the centroiding
variance of a 1 arcsecond spot, meanwhile, for photon noise it
corresponds to 0.24 times the variance of a 1 arcsecond spot.

For the PWFS, it is possible to follow the same formalism as
the SH shown in App. A.7 to obtain the X-slopes and Y-slopes
covariance matrix for RON and photon noise. By using the gen-
eralized least square, it is possible to obtain the variance of the
combined X-slopes and Y-slopes of the six LGSs, as shown in
figure C.2. The variances are normalized by the variance of a 1
arcsecond spot. For RON, the average variance is 0.70 times that
of the 1 arcsecond spot and for photon noise is 0.50 times that of
the 1 arcsecond spot.
<figure><img src="image_14.png" /><figcaption>Fig. C.2. PWFS variances in X and Y direction for a 40 m telescope resulting from the combination of 6 LGSs with the Generalized least square. The variances are normalized by the variance of a 1 arcsec spot. 16 x 16 subapertures are used here for visualization purposes.</figcaption></figure>