# A new calibration strategy for adaptive telescopes with pyramid WFS

E Heritier, E. Esposito, E Fusco, E Neichel, E Oberti, E Briguglio, E Agapito, P Puglisi, P Pinna, P Madec

► To cite this version:

E Heritier, E. Esposito, E Fusco, E Neichel, E Oberti, et al.. A new calibration strategy for adaptive telescopes with pyramid WFS. Monthly Notices of the Royal Astronomical Society, 2018, 10.1093/mn-ras/sty2485. hal-01936862

HAL Id: hal-01936862

https://hal.science/hal-01936862

Submitted on 14 Jan 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# A new calibration strategy for adaptive telescopes with pyramid WFS

C. T. Heritier,<fnref n="1" />,<fnref n="2" />,<fnref n="3" />,<fnref n="4" />★ S. Esposito,<fnref n="3" /> T. Fusco,<fnref n="1" />,<fnref n="2" /> B. Neichel,<fnref n="1" /> S. Oberti,<fnref n="4" /> R. Briguglio,<fnref n="3" /> G. Agapito,<fnref n="3" /> A. Puglisi,<fnref n="3" /> E. Pinna<fnref n="3" /> and P.-Y. Madec<fnref n="4" />

Accepted 2018 September 6. Received 2018 September 6; in original form 2018 August 27

## ABSTRACT

Several telescopes include large deformable mirrors (DM) located directly inside the telescope. These adaptive telescopes trigger new constraints for the calibration of the adaptive optics (AO) systems as they usually offer no access to an artificial calibration source for the interaction matrix measurement. Moreover, the optical propagation between the DM and the wavefront sensor (WFS) may evolve during the operation, resulting in misregistrations that highly affect the AO performance and thus the scientific observation. They have to be measured and compensated, for instance by updating the calibration. A new strategy consists of estimating the misregistrations and injecting them into synthetic models to generate noise-free interaction matrices. This pseudo-synthetic approach is the baseline for the adaptive optics facility working with a Shack-Hartmann WFS and seems particularly suited for the future Extremely Large Telescope as the calibration will have to be regularly updated, for a large numbers of actuators. In this paper, the feasibility of a pseudo-synthetic calibration with pyramid WFS at the Large Binocular Telescope (LBT) is investigated. A synthetic model of the LBT AO systems is developed, and the procedure to adjust the misregistrations parameters is introduced, extracting them from an experimental interaction matrix. We successfully tested an interaction matrix generated from the model on the real system in high-order AO mode. We recorded a slightly better performance with respect to the experimental one. This work demonstrates that a high-accuracy calibration can be obtained using the pseudo-synthetic approach with pyramid WFS.

**Key words:** instrumentation: adaptive optics – telescopes.

## 1 INTRODUCTION

Adaptive optics (AO) is now commonly spread on large aperture optical telescope facilities to compensate in real time the variations of optical index in the atmosphere and retrieve the full angular resolution of the telescope (Babcock 1953). The principle of a classical AO system is the following: a wavefront sensor (WFS) measures a signal relative to the phase and sends it to a real-time computer (RTC) that computes the corresponding commands to apply on a deformable mirror (DM). This system is usually operated in a feedback loop at a higher frequency than the temporal evolution of the turbulence (typically, a few hundred Hz). However, to provide a good correction (e.g. to be able to apply the correct shape on the DM at the right time), the AO loop has to be properly calibrated before the operations. This is achieved by measuring the interaction

matrix of the system which consists of recording the WFS signals corresponding to a specific command of the DM actuators.

Recent developments in telescope designs and DM technology have led to consider DM located directly into the telescope (turning them from active-to-adaptive telescopes) to reduce the number of optics and increase the numbers of photons available for the instruments. This concept was first validated on the MMT (Wildi et al. 2003) and is now used on two of the largest ground-based optical telescopes, the Large Binocular Telescope (LBT), with its two adaptive secondary mirrors (ASMs) (Riccardi et al. 2003; Esposito et al. 2010a) of 672 actuators, and the Unit Telescope 4 of the Very Large Telescope, recently upgraded to become the adaptive optics facility (AOF) (Stroebele et al. 2006; Arsenault et al. 2008) with its deformable secondary mirror (DSM) of 1170 actuators. The next generation telescopes, the Extremely Large Telescope (ELT) (Gilmozzi & Spyromilio 2007) and Giant Magellan Telescope (GMT) (Johns 2006), will also include large adaptive mirrors in their design with, respectively, 4356 and 4702 actuators (Hinz et al. 2010; Vernet et al. 2012). The Thirty Meter Telescope (TMT)

<footnote n="1">¹Aix Marseille Univ, CNRS, CNES, LAM, Marseille, France</footnote>

<footnote n="2">²ONERA, DOTA, Unité HRA, 29 avenue de la division Leclerc, F-92322 Chatillon, France</footnote>

<footnote n="3">³INAF – Osservatorio Astrofisico di Arcetri Largo E. Fermi 5, I-50125 Firenze, Italy</footnote>

<footnote n="4">⁴European Southern Observatory, Karl-Schwarzschild-str-2, D-85748 Garching, Germany</footnote>

<footnote>* E-mail: cedric.heritier@lam.fr</footnote>
is also considering this design and will thoroughly study this option in the coming months (Boyer 2018). These numbers put light on the very first challenge for the AO calibration of this new generation of adaptive telescopes: how much of precious telescope time will be required to provide a satisfying calibration of the AO systems?

This problematic was already addressed in Oberti et al. (2006) in which a summary of the main technical challenge is provided: large number of actuators and regular update of the calibration with often no external source illuminating the DM. Indeed, such telescopes usually don't provide any access to an intermediate focal plane ahead of the DM, and thus to any artificial calibration source. This specificity requires to completely rethink the way to acquire the interaction matrix of the system. However, even in presence of such an artificial calibration source, the large distance between the DM and the WFS may frequently affect the registration between both systems (due to gravity or flexure). These so-called misregistrations,<fnref n="1" /> affect the performance of the AO correction, especially for high-order systems which are extremely sensitive to these misalignments and require then a high-accuracy calibration. Such calibration errors impact the performance of the scientific instruments and can lead to instability of the AO loop, perturbing the telescope operation.

The ELT will take these constraints to another level. The first challenge in terms of telescope operation will be to calibrate a large number of actuators with no access to a calibration source. Moreover, the calibration will require regular updates during the operations due to the unprecedented distance between the DM and the WFS with moving optics between them.

There is then a crucial need to optimize and develop new calibration strategies to overcome these constraints and minimize the telescope time required. Some methods have already been identified to achieve the measurements. The first approach consists of measuring the interaction matrix on-sky (Wildi & Brusa 2004; Esposito et al. 2006; Pieralli et al. 2008; Kellerer et al. 2012; Pinna et al. 2012) while the second idea is to generate it synthetically using an AO simulator, injecting experimental inputs (Oberti et al. 2006; Kolb et al. 2012a,b). A summary of the classical procedure and of some of these new strategies is developed in Section 2.

The community seems to be converging towards the second approach, working on synthetic models of the AO systems in which are injected some misregistrations parameters to fit with the alignment of the real system. This approach is especially suited for systems that require a frequent update of the calibration with a large number of actuators. The pseudo-synthetic interaction matrix (PSIM) is currently the baseline for the AOF instruments in ground layer adaptive optics (GLAO) and laser tomography adaptive optics (LTAO) modes working with a Shack-Hartmann WFS (Hartmann 1900; Shack 1971).

It is however still to be investigated in the case of the LBT that operates in single conjugate adaptive optics (SCAO) mode, working with a pyramid WFS (Ragazzoni 1996) and natural guide stars (NGS). The pyramid WFS specificities could add constraints for the calibration. This WFS provides a gain in sensitivity with respect to the Shack-Hartmann (Ragazzoni & Farinato 1999) but can be complex to model with a modal linearity and sensitivity (Esposito & Riccardi 2001; Vérinaud 2004; Fauvarque et al. 2017). The response of the sensor is also seeing dependent (Ragazzoni & Farinato 1999) as the PSF shape on the top of the pyramid will depend on the seeing conditions. Some work has also shown that tracking the

modal gains during the operations will improve the closed loop performance (Korkiakoski, Vérinaud & Le Louarn 2008; Esposito et al. 2012; Esposito et al. 2015; Bond et al. 2018; Deo et al. 2018). Considering that all of the first light instruments of the ELT will include a pyramid WFS in their design (Brandl et al. 2016; Clénet et al. 2016; Neichel et al. 2016) it is necessary to identify the key elements and the accuracy requirements to reproduce the behaviour of an AO system with pyramid WFS with the overall goal to generate calibration data that can be used on a real system.

After a short introduction of the classical and new calibration methods in the context of the adaptive telescopes (Section 2), this paper will introduce the development of a synthetic model, reproducing the FLAO-LBT systems (Esposito et al. 2010b), focusing on the model definition and sensitivity (Section 3). Section 4 details the adjustment procedure for the misregistrations parameters that have been thoroughly verified in simulation. Section 5 presents the results of day-time validation at the LBT.

## 2 AO CALIBRATION OF AN ADAPTIVE TELESCOPE

This section aims to present the calibration procedure for a classical AO system and provides a short summary of the different calibration strategies in the frame of the future ELT.

### 2.1 General case

Mathematically, the interaction matrix of an AO system is the transfer matrix between the DM and the WFS space. Following the notations introduced in Meimon, Petit & Fusco (2015), the interaction matrix $D$ of an AO system is:

$$D = M_{\text{WFS}} \cdot M_{\text{DM}}, \quad (1)$$

where $M_{\text{DM}}$ is the conversion matrix between the DM commands and the optical phase deformations and $M_{\text{WFS}}$ the corresponding WFS measurement matrix. This matrix is then inverted to provide the reconstructor $R$ that is used in closed loop.

$$R = D^{\dagger}. \quad (2)$$

The most common inversion method consists of using a truncated singular values decomposition (SVD), filtering the modes badly seen by the WFS for stability (Boyer, Michau & Rousset 1990 but using a generalized SVD adding priors on the noise and turbulence statistics provides a gain in the reconstruction (Wallner 1983; Fusco et al. 2001; Gilles 2005).

The measurement of the interaction matrix is achieved by recording the WFS signals $Y$ corresponding to a given set of actuation pattern $U$. These measurements are impacted by two sources of noise, the WFS detector noise $W$ and the local turbulence $\Phi$ during the measurements.

$$Y = DU + W + M_{\text{WFS}} \cdot \Phi. \quad (3)$$

Multiplying (3) by the pseudo-inverse of the calibration pattern $U^{\dagger}$ provides the estimation of the interaction matrix $\hat{D}$.

$$\hat{D} = Y \cdot U^{\dagger}. \quad (4)$$

The calibration pattern is usually chosen to be full rank (for instance a zonal actuation or a modal basis such as DM stiffness modes or Kharunen-Loève modes) to calibrate all the degrees of freedom of the DM, providing $U \cdot U^{\dagger} = \text{Id}$. That way, we can define the calibration error $\Delta D$:

$$\Delta D = D - \hat{D} = -(W + M_{\text{WFS}} \Phi) \cdot U^{\dagger}. \quad (5)$$

<footnote n="1">¹Relative shifts, rotation, magnification, or higher order pupil distortion between the DM actuators and WFS subapertures.</footnote>
This equation consists of two independent terms, W the noise related to the WFS detector and $M_{\text{WFS}} \Phi$ the noise related to the turbulence, both multiplied by the pseudo-inverse of the calibration pattern U.

In the optimal case, both noise contribution can be minimized using a bright artificial calibration source and reducing $\Phi$ to the local turbulence of the optical bench. In that case, the choice of calibration patterns becomes arbitrary as a good SNR is easily achievable. However, if these contributions become non-negligible, and if the number of actuators becomes important, it is necessary to optimize the choice of calibration pattern to maximize the SNR and speed up the measurement. Such optimization have already been investigated by Oberti et al. (2004), Kasper et al. (2004), and Meimon et al. (2015) showing that using system modes or Hadamard patterns to maximize the signal provides a consistent gain in calibration time.

In any case, quantifying the quality of an interaction matrix is still an open question. Some metrics have been investigated in Meimon et al. (2015) and Oberti et al. (2004) but no clear criteria have been identified yet. The only solution to validate the accuracy of an interaction matrix is then to try using it to close the loop of the AO system.

## 2.2 On-sky calibration

With no calibration source available, an alternative method consists of measuring the interaction matrix on-sky. This strategy faces one major challenge: the impact of the turbulence on the WFS measurements. In equation (3), the term $M_{\text{WFS}} \Phi$ becomes indeed dominant. Actuating a calibration pattern using a classical push-pull would require such a large amplitude to extract it from the turbulence noise that it will completely saturate the WFS.

The problem becomes then pretty classical: one wants to retrieve a low signal in a noisy (turbulence noise) and variable environment (seeing variation), with a constraint of not saturating the WFS nor the DM stroke.

Taking the two extreme solutions, one could either average the turbulence using a long integration, or freeze it using a fast push-pull measurement. An alternative approach inspired from the telecommunication domain, consists of modulating signals with a low amplitude on the DM (to minimize the impact on science) but large enough to be detectable by the WFS. Using an orthogonal basis, in both temporal and spatial domain, of multiplexed signals, one can retrieve their signatures in the WFS space using a simple demodulation process (Esposito et al. 2006). This approach has been validated on several systems, at the VLT with a curvature sensor by Oberti et al. (2006), with a high-order pyramid WFS at the LBT by Pinna et al. (2012) (operating the demodulation in the Fourier space) and with a Shack-Hartmann on the CANARY facility (Myers et al. 2008) by Kellerer et al. (2012) (demodulating in the direct space).

The analysis of the results has shown that an accurate knowledge of the temporal behaviour of the system is required (rejection transfer function of the AO loop, temporal delay of the loop) and a trade-off for the modulation parameters (frequency, amplitude, sampling, multiplexing) has to be done to perform an efficient calibration. If these methods are applied during the observation, the impact on science is still to be investigated. In addition, to overcome the constraint of WFS saturation, the signals modulation has to be operated in closed loop (at least partial) using a first interaction matrix, that could be synthetic.

This demonstrated the feasibility of the on-sky approach as a potential alternative strategy but to our knowledge no operating

system is currently using it as its baseline for its AO calibration. Such a calibration requires indeed a longer time than a classical measurement with fibre (with a lower SNR). Moreover, in the case of a Pyramid WFS, if the on-sky calibration takes too long, the WFS response may evolve during the measurements, as it depends on the seeing conditions, and bias the interaction matrix measurement. Therefore, a full on-sky calibration does not seem suited for a system with a large number of degrees of freedom, especially with pyramid WFS. In the case of adaptive telescopes, potential nonlinearities and complex internal behaviour (evolution or instability of the DM/WFS registration) may also appear during the operation. These techniques could however be used to retrieve only a few signals on-sky to identify key misregistrations parameters that are then injected in a synthetic model. This pseudo-synthetic approach seems then to be a better strategy to overcome such constraints.

## 2.3 New pseudo-synthetic calibration

The pseudo-synthetic calibration has been identified as the most promising calibration strategy for the future ELT: misregistrations parameters extracted from experimental inputs<fnref n="2" /> are injected into a synthetic model of the AO systems. The principle is summarized in Fig. 1.

This approach provides a noise-free interaction matrix, fast to compute and thus, easy to update. The good quality of the calibration relies only on two key ingredients: the ability to model accurately the WFS and the DM and the accuracy of the misregistrations parameters. Theoretically, if all the experimental effects are perfectly reproduced, a pseudo-synthetic interaction matrix should then provide a better calibration than a noisy experimental one.

Concerning the first ingredient, the state of the art in terms of AO modelling has shown on various systems (e.g. LBT or VLT) that an accurate calibration can be generated from a synthetic model (Oberti et al. 2006; Kolb et al. 2012a; Pinna et al. 2012). The modelling of a Shack-Hartmann seems to be one step ahead with respect to the pyramid WFS case as extensive studies related to the sensitivity are available (see Kolb et al. 2012a). In the case of a Pyramid WFS, the feasibility was demonstrated in Pinna et al. (2012) but no analysis of sensitivity was achieved so far. In this paper, we will try to address this point.

The second ingredient is related to the estimation of the misregistrations parameters. The nature of the reference signals required to align the model is still open to discussion. The first approach is non-invasive and consists of using telemetry data (WFS slopes and DM commands) to retrieve the corresponding misregistrations parameters. At the AOF for instance, the idea consists of using closed-loop data to estimate a noisy interaction matrix and project it on sensitivity matrices to identify these parameters (see Béchet et al. 2011; Kolb et al. 2012a).

The second approach is invasive and consists of dithering signals on the DM to retrieve their signature in the WFS space (Chiuso, Muradore & Marchetti 2010). In Neichel et al. (2012), a strategy based on a Levemberg-Marquardt (Marquardt 1963) type algorithm is presented in the frame of tomographic AO systems but requires an experimental interaction matrix as a reference, that could for instance be measured on-sky. The difficulty here is to find the minimum number of signals that would be characteristic of each type of misregistrations, independently from the others. Since the param-

<footnote n="2">²That we will call reference signals in this paper.</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Pseudo-synthetic calibration: Experimental inputs (solid black lines) are injected into the synthetic model (dashed blue lines) to reproduce the registration of the real system. The pseudo-synthetic interaction matrix (dotted red lines) is then regularly updated during the operation tracking the misregistrations parameters.</figcaption></figure>

eters are strongly correlated, the solution consists here in using an iterative procedure to identify each parameter.

Optimizing the accuracy of a pseudo-synthetic interaction matrix is another question. The matrix generated is indeed noise free and equation (5) takes another form as the only source of error comes from the model: both $M_{DM}$ and $M_{WFS}$ become $\bar{M}_{DM}$ and $\bar{M}_{WFS}$. So far, the only way consists of validating the model against experimental inputs.

# 3 MODELLING OF THE LBT AO SYSTEMS FOR A PSEUDO-SYNTHETIC INTERACTION MATRIX CALIBRATION

## 3.1 Model definition

We reproduce the FLAO systems (Esposito et al. 2010b) at the LBT in the end to end simulator OOMAO (Conan et al. 2014) with the overall goal to generate an interaction matrix that can be used on the real system. It required a fine tuning of the two key elements of the model, ASM and Pyramid WFS, using experimental inputs from the telescope to take in consideration all the features of the existing systems. A summary of the model definition is given in Fig. 2.

The ASM model is composed of 663 valid actuators arranged in circular concentric rings with a 30 cm radial pitch projected on-sky. The ASM influence functions measured with interferometer are input to the model. The modal commands matrix from the telescope to produce the 594 modes on the ASM is used in the simulator. The modal basis consists of Kharunen–Loève modes, computed by diagonalizing the covariance matrix of the turbulence and re-orthogonalised in the DM space (Esposito et al. 2010a). The amplitude of the modes used for the interaction matrix computation is low enough to prevent non-linearity effects. As in the real system, the ASM is the stop aperture of the optical system. The arm of the telescope is not taken into consideration as it is done on-site for the interaction matrix measurement procedure, achieved using a retroreflector during daytime (Esposito et al. 2010a). The four spiders of the retroreflector are not considered either.

The *Pyramid WFS* diffractive model is based on the FLAO pyramid WFS with 30 by 30 subapertures (1 pixel per subaperture) and 36 pixels separating each pupil image centres. The WFS model is a

perfect single pyramid<fnref n="3" /> with no scratches or faces misalignment and operates at 750 nm, the central wavelength of the FLAO WFS. The FLAO valid subapertures mask is input to the model, selecting its position to maximize the amount of light in the WFS model pupils. The WFS model is using the optimal Tip/Tilt modulation<fnref n="4" /> of 3 $\lambda/D$ considered at the telescope during the observations in high-order mode. To reproduce the exact LBT configuration for the interaction matrix measurement, the WFS signals are normalized with a factor 2 to be in line with the double pass procedure using a retroreflector (Esposito et al. 2010a).

The computation of the synthetic interaction matrix is achieved in a noise-free environment. This feature is well illustrated on the interaction matrix eigen values spectra (Fig. 3). In the synthetic case, the distribution of eigen values is flatter and the knee of the curve occurs for a larger eigen mode number. Here, note that the eigen modes will be slightly different as we still have residual differences between the two interaction matrices. However, the synthetic interaction matrix is better conditioned suggesting that it would be easier to control more modes using a synthetic interaction matrix than an experimental one, taking profit from the infinite SNR of the synthetic WFS signals. These considerations are however only valid if the registration of the real system is well reproduced in the simulator.

The pyramid sensitivity to the modal basis depends on the tip/tilt modulation used for the WFS. A rough estimation of the WFS sensitivity is obtained by considering the WFS slopes root mean square (RMS). The comparison between the model and the experimental WFS is provided in Fig. 4. Both sensitivity plots follow the same tendency which show that the model reproduces well the real system with the same parameter values.

## 3.2 Sensitivity to a misregistration

The accuracy of the model misregistrations parameters is a key ingredient for high-order AO systems. In practice, with a Shack Hartmann WFS, the common rule is not to exceed an error of

<footnote n="3">³The FLAO pyramid WFS is a double pyramid to avoid chromatic dispersion (Esposito et al. 2010a) but the WFS model is achromatic as it operates at a single wavelength.</footnote>

<footnote n="4">⁴No tuning of the tip/tilt modulation was achieved so far.</footnote>
<figure><img src="image_4.png" /><figcaption>**Figure 2.** Summary of the model definition. Experimental inputs (solid black lines) are injected into the synthetic model of the AO systems (dashed blue lines). The model misregistrations parameters are first calibrated so that the experimental interaction matrix can be used to close the loop in the simulator. The synthetic interaction matrix can then be tested on the real system.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Figure 3.** Comparison of the interaction matrices eigen values distributions between the experimental and synthetic cases. The synthetic interaction matrix has a lower conditioning number (8.1 versus 8.85).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Figure 4.** Comparison of slopes RMS plots. Both WFS have the same sensitivity to the KL modes.</figcaption></figure>
10 per cent of a subaperture (Béchet et al. 2011). This result has to be taken carefully as the sensitivity to a misregistration will depend on the DM geometry and on the number and type of modes controlled in the reconstructor.

Using our model of pyramid WFS and ASM with circular geometry, we could simulate the impact of each type of misregistrations on the performance for a seeing of 1 arcsec in the visible. The corresponding wavefront error (WFE) as a function of each type of misregistrations, controlling 400 and 594 modes in the reconstructors is given Fig. 5. In the case of the LBT, a rotation of 1° corresponds to a shift of around 25 per cent of a subaperture on the border of the pupil.

More details are provided Fig. 6 displaying the modal PSD corresponding to the first misregistration value after the drop of performance when controlling 594 modes (0.8° for the rotation and 40 per cent for the shift). It is clearly visible that the high-order modes are the most impacted by the misregistrations and get amplified even to higher values than the incoming turbulence, confirming that high-order AO systems are more sensitive to misregistrations.

These results show that the most critical misregistrations parameters, the shifts and the rotation, should be accurately identified (with less than 10 per cent of a subaperture for the shifts and less than 0.1° for the rotation which corresponds to a shift of 2.5 per cent on the border of the pupil) while the impact of a magnification becomes significant when controlling a high number of modes only.

## 4 ASM/WFS MISREGISTRATIONS ADJUSTMENT PROCEDURE

This section describes the procedures to finely tune the model misregistrations parameters and provide a functional interaction matrix for the telescope. We consider here only the fine tuning and rely on a first rough estimation of the parameters (±1 subaperture shift and ±2° of rotation) that could be done using flux considerations for instance (Kolb et al. 2012b).

The tuning of the model requires to define a metric to quantify the error between the model and the real system (Section 4.2). In our case, we use reference WFS signals that correspond to the measurement of 400 KL modes (the current interaction matrix used at the telescope). We then play on the model misregistrations parameters to generate the corresponding synthetic WFS measurements and use an iterative procedure to minimize the error with the reference.

### 4.1 Misregistrations parameters

The model has four misregistrations parameters, the shifts $\alpha_x$ and $\alpha_y$, the rotation $\alpha_{rot}$ and the radial magnification $\alpha_{magn}$ as it appears to be symmetric. We define $\alpha$ as the corresponding model misregistrations parameter vector:

$$ \alpha \triangleq \{\alpha_x, \alpha_y, \alpha_{rot}, \alpha_{magn}\}. \qquad (6) $$

In practice, the relative shift between WFS and ASM is applied by shifting the WFS pupils on the WFS detector pixel grid by tilting the pyramid model (providing a sub pixel sensitivity). Both rotation and magnification are applied on the ASM model, interpolating the influence functions.

### 4.2 Minimization criteria

We define $\sigma_j$ as the root mean square error (RMSE) between the synthetic WFS measurement $Y^*(\alpha)$ in the configuration $\alpha$ and the

<figure><img src="image_7.png" /><figcaption>Figure 5. Impact of a misregistration on the performance (wavefront RMS) for a rotation (Top), a shift in X (Middle) and a magnification (Bottom). Results are given for two configurations, controlling 400 and 594 modes.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>**Figure 6.** Modal PSD for the nominal and a misregistrated case in the case of a rotation (top) and a shift in X (bottom).</figcaption></figure>

reference WFS measurement Y from a real interaction matrix of the mode j:

$$ \sigma_j = \text{RMSE}(Y_j, Y_j^*(\alpha)) = \sqrt{\frac{1}{N_S} \sum_{n=1}^{N_S} |Y_j - Y_j^*(\alpha)|^2}, \quad (7) $$

where $N_S$ is the number of WFS slopes. The criteria $\chi_N$ to minimize is the quadratic norm of $\sigma_N = \{\sigma_i\}_{i=1,2,...,N}$ adjusting $\alpha$ and eventually considering different numbers of reference signals N:

$$ \chi_N(\alpha) = \arg_{\min} ||\sigma_N||^2. \quad (8) $$

The optimal value chosen for $\alpha$ is the convergence value of the iterative minimization of $\chi_{400}$ using all the reference signals available.

## 4.3 Model adjustment procedure

The misregistrations parameters of the model, especially the shifts and rotation, are strongly correlated in the WFS space. It is therefore necessary to achieve an iterative procedure to adjust correctly the misregistrations parameters. The parameters are estimated one by one and the procedure is summarized in Table 1. A first estimation of each shift and rotation is achieved using a large step and then a second estimation using a smaller step. Applying a third estimation does not change the value of the estimated parameters therefore two steps for the iterative procedure are sufficient. The magnification is identified as a last step as its effect is less crucial (see Fig. 5).

The starting point $\alpha_0$ is defined using the tip and tilt WFS signals to roughly estimate the starting rotation value as these modes do

<table><caption>**Table 1.** Model alignment iterative procedure. The values are in fraction of a subaperture (shift), in fraction of pupil diameter (magnification) and in degrees (rotation).</caption><thead><tr><th>Step</th><th>α<sub>i</sub></th><th>δα<sub>i</sub></th><th>Value</th></tr></thead><tbody><tr><td>1</td><td>α<sub>rot</sub></td><td>0.125°</td><td>65.33°</td></tr><tr><td>2</td><td>α<sub>x</sub></td><td>25 %</td><td>-100 %</td></tr><tr><td>3</td><td>α<sub>y</sub></td><td>25 %</td><td>25 %</td></tr><tr><td>4</td><td>α<sub>rot</sub></td><td>0.1°</td><td>65.08°</td></tr><tr><td>5</td><td>α<sub>x</sub></td><td>2 %</td><td>-92 %</td></tr><tr><td>6</td><td>α<sub>y</sub></td><td>2 %</td><td>35 %</td></tr><tr><td>7</td><td>α<sub>rot</sub></td><td>0.025°</td><td>65.08°</td></tr><tr><td>8</td><td>α<sub>x</sub></td><td>2 %</td><td>-92 %</td></tr><tr><td>9</td><td>α<sub>y</sub></td><td>2 %</td><td>35 %</td></tr><tr><td>10</td><td>α<sub>magn</sub></td><td>0.45 %</td><td>98 %</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>**Figure 7.** Last step of the minimization criteria $\chi_{400}$ for the shifts, rotation, and magnification parameters using all the reference signals available. The solid line is a second-order polynomial fit.</figcaption></figure>

not have a circular symmetry.

$$ \begin{aligned} \alpha_0 &= \{\alpha_{\text{rot}} = 64.5^{\circ}, \alpha_x = 0 \text{ per cent}, \\ \alpha_y &= 0 \text{ per cent}, \alpha_{\text{magn}} = 100 \text{ per cent}\}. \end{aligned} \quad (9) $$

Fig. 7 gives the second step of the parameters estimation, showing a quadratic behaviour around each optimal value. The final value taken for the misregistrations parameter $\alpha_f$ is obtained from a second-order polynomial fit of $\chi_{400}(\alpha)$ and is illustrated in Fig. 8.

$$ \begin{aligned} \alpha_f &= \{\alpha_{\text{rot}} = 65.08^{\circ}, \alpha_x = -92 \text{ per cent}, \\ \alpha_y &= 35 \text{ per cent}, \alpha_{\text{magn}} = 98 \text{ per cent}\}. \end{aligned} \quad (10) $$

As a last step, we also tuned the modal gains of the pyramid model by playing on the amplitude of the KL modes during the interaction matrix computation and minimize the residual slopes RMS with the reference. The corresponding plots are given in Fig. 4.
<figure><img src="image_10.png" /><figcaption>**Figure 8.** Final configuration for the registration between the ASM actuators (diamonds) and the WFS subapertures (crosses) in normalized units.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>**Figure 9.** Comparison of pseudo-synthetic (left), LBT (centre), and residual (right) normalized slopes maps after adjustment of the model. These slopes maps correspond to KL modes 1, 5, 10, 50, 100, 200, 300, 350, and 400.</figcaption></figure>

## 4.4 Validation of the misregistrations parameters adjustments in simulation

The adjustment of the model was intensively validated in simulation. A comparison of a few slopes maps from both interaction matrices is given in Fig. 9 and shows only negligible differences. It also shows that some features of the real measurements are missing from the model especially close to the central obscuration. These features seem to be purely experimental as evolving between different experimental interaction matrices (see Fig. 10). We did not

<figure><img src="image_12.png" /><figcaption>**Figure 10.** Slopes variance map [Sx Sy] corresponding to the 400 KL modes contained in the Interaction matrices of the LBT from 2016 (top), 2017 (middle) and from the simulator (bottom). It is visible that some parts of the pupils gathered less signals than others, the bottom left part, the central obscuration and behind the spiders of the retroreflector. These features were not considered in the model.</figcaption></figure>

consider these features in the model and their impact is still to be investigated.

To provide a meaningful comparison, the first milestone was to use the interaction matrix measured at the telescope to close the loop of the simulated AO system. The comparison of simulated closed loop performance of both synthetic and measured interaction matrices provides an indication of the model accuracy. This is given in Fig. 11 and as expected, the synthetic calibration provides a better correction (the corresponding calibration is the optimal calibration for the model) but overall, both reconstructors provide an equivalent correction with a residual variance of 128 nm in the synthetic case and 147 nm using the experimental reconstructor. The differences in residual variance come from the remaining model errors and/or calibration errors from the experimental data. To get rid of these residual differences in performance, some eventual upgrade of the model could be to take in consideration the imperfect illumination identified on the experimental interaction matrix to improve the accuracy of the model.

Moreover, Fig. 12 gives the simulated closed-loop performance using the experimental reconstructor and around the optimal misregistration parameters value identified in Section 4. For each parameter, the optimal value provides the best AO performance, confirming the high accuracy of the misregistrations parameters identification. We also retrieve the same sensitivity to the misregistrations as in the synthetic case (see Fig. 5).
<figure><img src="image_13.png" /><figcaption>**Figure 11.** Comparison of simulated closed-loop modal power spectral density (PSD) using both reconstructors, synthetic and measured at the telescope.</figcaption></figure>

## 5 EXPERIMENTAL VALIDATION

The model was validated using daytime remote tests at the LBT. In that configuration, a star of magnitude 7 is simulated using an external fibre and a retroreflector. A disturbance signal is then applied on the ASM to simulate a turbulence with a seeing of 1 arcsec in the visible. A summary of the calibration procedure of the LBT is developed in Esposito et al. (2010a).

Before closing the loop, a tuning of the integrator gain is applied for the tip/tilt, the following modes up to 100 and for the remaining modes up to 400. The tuning of the gains consists of applying a ramp of gain values and select the one minimizing the RMSE of the residual slopes. That way, a different gain value will be used for each group of modes during the closed-loop operation. For our experimental tests, we applied this procedure to the experimental interaction matrix and used the corresponding values for our synthetic interaction matrix with no further tuning.

Fig. 13 provides the comparison of performance between the pseudo-synthetic and experimental interaction matrix using the misregistration values identified from the iterative procedure defined in Section 4. Both reconstructors provide an equivalent correction and the details of the corresponding residuals are summarized in Table 2, showing that the pseudo-synthetic interaction matrix provided a slightly better correction than the experimental one. We can notice on the modal PSD that some low-order modes (mostly tip/tilt and focus) have a significantly higher variance compared to what we could expect with a simulated turbulence and a bright star. This effect takes origin from the estimation of the residuals. These are computed from the DM positions which include the correction for the vibrations of the telescope. These modes should then not be considered for the performance comparison.

In addition, we managed to push the synthetic interaction matrix to correct up to 500 modes and get stable closed loop with high-order correction visible in Fig. 14. This validates the high accuracy of the misregistrations parameters estimation while having WFS reference measurements for only 400 KL modes. The corresponding PSF from

<figure><img src="image_14.png" /><figcaption>**Figure 12.** Simulated closed loop performance using the experimental reconstructor in the same conditions as Fig. 5, playing around the optimal value of rotation (top), both shifts (middle) and magnification (bottom) identified during the model adjustment procedure.</figcaption></figure>
<figure><img src="image_15.png" /><figcaption>**Figure 13.** Comparison of modal PSD using both interaction matrices, synthetic and reference, for a same disturbance applied on the ASM.</figcaption></figure>

<table><caption>**Table 2.** Summary of the experimental closed-loop performance for both pseudo-synthetic and experimental interaction matrices.</caption><thead><tr><th>Int. Mat.</th><th>WFE</th><th>TT rem.</th><th>50 KL rem.</th></tr></thead><tbody><tr><td>PSIM 400 modes</td><td>195 nm</td><td>41.7 nm</td><td>30.1 nm</td></tr><tr><td>LBT 400 modes</td><td>216 nm</td><td>42.1 nm</td><td>30.8 nm</td></tr><tr><td>PSIM 500 modes</td><td>228.3 nm</td><td>42.5 nm</td><td>27.39 nm</td></tr></tbody></table>

the instrument LUCI (Heidt et al. 2018) for 400 and 500 modes using the synthetic interaction matrix are also given in Fig. 14.

## 6 CONCLUSIONS

We developed a synthetic model of the FLAO pyramid WFS and ASM using experimental inputs from the telescope. We considered a perfect pyramid WFS and used measured influence functions for the ASM model. To identify the misregistrations parameters, we defined an iterative procedure using an experimental interaction matrix as a reference to adjust the model misregistrations parameters. The accuracy of these parameters has been thoroughly verified in simulation.

The interaction matrix generated from the model has been validated experimentally using daytime at the LBT. This demonstrates the feasibility of the pseudo-synthetic calibration for high-order AO systems with pyramid WFS. Using the misregistrations parameters identified from the experimental reference, no further tuning on-site was required to efficiently close the loop of the real system. It provided slightly better performance than the experimental one and we could control up to 500 modes with the synthetic reconstructor.

Through this experimental validation, we have now a tool to achieve meaningful analysis by simulations. As a first result, we could study the sensitivity of the AO system to the different misregistrations, showing that the most critical parameters are the shifts and rotation.

This work will be very relevant for the future ELT as all the SCAO modules of the first light instruments will most likely include a pyramid WFS in their design. The next step will be to

<figure><img src="image_16.png" /><figcaption>**Figure 14.** PSF in H band (log scale) controlling 400 and 500 modes with the synthetic and experimental interaction matrix. The field of view of the instrument is 30 arcsec.</figcaption></figure>
consider the constraints of this telescope for the AO calibration. For instance, with no calibration source, a whole reference interaction matrix may not be available as it would most probably have to be acquired on-sky. In that case, acquiring only a few WFS signals (selected to maximize the sensitivity to each type of misregistrations) might be sufficient to provide an accurate estimation of the misregistrations parameters. There is a need to study the number and type of signals required to adjust the model misregistrations parameters. If these signals are acquired on-sky, the impact of the noise on the parameters estimation will also have to be investigated.

In this paper, we highlighted that the key ingredients to generate an accurate pseudo-synthetic interaction matrix is the estimation of the misregistrations parameters. There is a need to optimize the identification and especially the tracking of these parameters during the operation, if possible without injecting any perturbation. This had to be done taking into account the complexity of the pyramid WFS (modal sensitivity and linearity dependent on the seeing conditions and on the WFS modulation). Many concepts have already been proposed to this purpose. They still have to be carefully evaluated.

## ACKNOWLEDGEMENTS

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement no. 730890. This material reflects only the authors views and the Commission is not liable for any use that may be made of the information contained therein.

## REFERENCES

*   Arsenault R. et al., 2008, Proc. SPIE, 7015, 701524

*   Babcock H. W., 1953, PASP, 65, 229

*   Béchet C., Kolb J., Madec P.-Y., Tallon M., Thiébaut E., 2011, 2nd AO4ELT Conference - Adaptive Optics for Extremely Large Telescopes

*   Bond C., Correia C., Sauvage J., Hadi K., Abautret Y., Neichel B., Fusco T., 2018, 5th AO4ELT Conference - Adaptive Optics for Extremely Large Telescopes

*   Boyer C., 2018, in Close L. M., Schreiber L., Schmidt D., eds, Proc. SPIE Conf. Ser. Vol. 10703, Adaptive Optics Systems VI. SPIE, Bellingham, p. 107030Y

*   Boyer C., Michau V., Rousset G., 1990, Adaptive Optics and Optical Structures. SPIE, Bellingham, p. 63

*   Brandl B. R. et al. 2016, in Evans C. J., Simard L., Takami H., eds, Proc. SPIE Conf. Ser. Vol. 9908, Ground-based and Airborne Instrumentation for Astronomy VI. SPIE, Bellingham, p. 990820

*   Chiuso A., Muradore R., Marchetti E., 2010, IEEE Transact. Control Syst. Technol., 18, 705

*   Clénet Y. et al., 2016, Marchetti E., Close L. M., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 9909, Adaptive Optics Systems VI. SPIE, Bellingham, p. 99090A

*   Conan R., Correia C., 2014, in Marchetti E., Close L. M., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 9148, Object-Oriented Matlab Adaptive Optics Toolbox. SPIE, Bellingham, p. 91486C

*   Deo V., Gendron É., Rousset G., Vidal F., Buey T., 2018, in Close L. M., Schreiber L., Schmidt D., eds, Proc. SPIE Conf. Ser. Vol. 10703, Adaptive Optics Systems VI. SPIE, Bellingham, p. 1070320

*   Esposito S., Riccardi A., 2001, A&A, 369, L9

*   Esposito S., Tubbs R., Puglisi A., Oberti S., Tozzi A., Kompero M., Zanotti D., 2006, in Ellerbroek B. L., Calia D. B., eds, Proc. SPIE Conf. Ser. Vol. 6272, Advances in Adaptive Optics II. SPIE, Bellingham, p. 62721C

*   Esposito S. et al., 2010a, Appl. Opt., 49, G174

*   Esposito S. et al., 2010b, in Ellerbroek B. L., Hart M., Hubin N., Wizinowich P. L., eds, Proc. SPIE Conf. Ser. Vol. 7736, Adaptive Optics Systems II. SPIE, Bellingham, p. 773609

*   Esposito S. et al., 2012, in Ellerbroek B. L., Marchetti E., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 8447, Adaptive Optics Systems III. SPIE, Bellingham, p. 84470U

*   Esposito S., Pinna E., Puglisi A., Agapito G., Veran J. P., Herriot G., 2015, 3rd AO4ELT Conference - Adaptive Optics for Extremely Large Telescopes

*   Fauvarque O., Neichel B., Fusco T., Sauvage J.-F., Girault O., 2017, J. Astron. Telescopes Instrum. Syst., 3, 019001

*   Fusco T., Conan J.-M., Rousset G., Mugnier L. M., Michau V., 2001, J. Opt. Soc. Am. A, 18, 2527

*   Gilles L., 2005, Appl. Opt., 44, 993

*   Gilmozzi R., Spyromilio J., 2007, The Messenger, 127, 11

*   Hartmann J., 1900, Zeitschrift für Instrumentenkunde, 20, 47

*   Heidt J. et al., 2018, in Evans C. J., Simard L., Takami H., eds, Proc. SPIE Conf. Ser. Vol. 10702, Ground-based and Airborne Instrumentation for Astronomy VII. SPIE, Beelingham, p. 10702

*   Hinz P. M., Bouchez A., Johns M., Shectman S., Hart M., McLeod B., McGregor P., 2010, in Ellerbroek B. L., Hart M., Hubin N., Wizinowich P. L., eds, Proc. SPIE Conf. Ser. Vol. 7736, Adaptive Optics Systems II. SPIE, Bellingham, p. 77360C

*   Johns M., 2006, in Stepp L. M., eds, Proc. SPIE Conf. Ser. Vol. 8447, Ground-based and Airborne Telescopes. SPIE, Bellingham, p. 626729

*   Kasper M., Fedrigo E., Looze D. P., Bonnet H., Ivanescu L., Oberti S., 2004, J. Opt. Soc. Am. A, 21, 1004

*   Kellerer A., Vidal F., Gendron E., Hubert Z., Perret D., Rousset G., 2012, in Ellerbroek B. L., Marchetti E., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 8447, Adaptive Optics Systems III. SPIE, Bellingham, p. 844765

*   Kolb J., Madec P.-Y., Louarn M. L., Muller N., Béchet C., 2012a, in Ellerbroek B. L., Marchetti E., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 8447, Adaptive Optics Systems III. SPIE, Bellingham, p. 84472D

*   Kolb J., Muller N., Aller-Carpentier E., Andrade P., Girard J., 2012b, in Ellerbroek B. L., Marchetti E., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 8447, Adaptive Optics Systems III. SPIE, Bellingham, p. 84475U

*   Korkiakoski V., Vérinaud C., Le Louarn M., 2008, Appl. Opt., 47, 79

*   Marquardt D. W., 1963, J. Soc. Ind. Appl. Math., 11, 431

*   Meimon S., Petit C., Fusco T., 2015, Opt. Express, 23, 27134

*   Myers R. M. et al., 2008, in Hubin N., Max C. E., Wizinowich P. L., eds, Proc. SPIE Conf. Ser. Vol. 8447, Adaptive Optics Systems. SPIE, Bellingham, p. 70150E

*   Neichel B., Parisot A., Petit C., Fusco T., Rigaut F., 2012, in Ellerbroek B. L., Marchetti E., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 8447, Adaptive Optics Systems III. SPIE, Bellingham, p. 84475N

*   Neichel B. et al., 2016, in Marchetti E., Close L. M., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 9909, Adaptive Optics Systems V. SPIE, Bellingham, p. 990909

*   Oberti S., Bonnet H., Fedrigo E., Ivanescu L., Kasper M. E., Paufigue J., 2004, in Calia D. B., Ellerbroek B. L., Ragazzoni R., eds, Proc. SPIE Conf. Ser. Vol. 5490, Advancements in Adaptive Optics. SPIE, Bellingham, p. 139

*   Oberti S. et al., 2006, in Ellerbroek B. L., Calia D. B., eds, Proc. SPIE Conf. Ser. Vol. 6272, Advances in Adaptive Optics II. SPIE, Bellingham, p. 627220

*   Pieralli F., Puglisi A., Quirós-Pacheco F., Esposito S., 2008, in Hubin N., Claire E. M., Wizinowich P. L., eds, Proc. SPIE Conf. Ser. Vol. 7015, Adaptive Optics Systems. SPIE, Bellingham, p. 70153A

*   Pinna E. et al., 2012, in Ellerbroek B. L., Marchetti E., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 8447, Adaptive Optics Systems III. SPIE, Bellingham, p. 84472B

*   Ragazzoni R., 1996, J. Mod. Opt., 43, 289

*   Ragazzoni R., Farinato J., 1999, A&A, 350, L23

*   Riccardi A., Brusa G., Salinari P., Gallieni D., Biasi R., Andrighettoni M., Martin H. M., 2003, in Tyson R. K., Bonaccini D., Roggemann M. C., eds, Proc. SPIE Conf. Ser. Vol. 4494, Adaptive Optical System Technologies II. SPIE, Bellingham, p. 721

*   Shack R. V., 1971, J. Opt. Soc. Am., 61, 656

*   Stroebele S. et al., 2006, in Ellerbroek B. L., Calia D. B., eds, Proc. SPIE Conf. Ser. Vol. 6272, Advances in Adaptive Optics II. SPIE, Bellingham, p. 62720B
Vérinaud C., 2004, *Optics Commun.*, 233, 27

Vernet E., Cayrel M., Hubin N., Mueller M., Biasi R., Gallieni D., Tintori M., 2012, in Ellerbroek B. L., Marchetti E., Véran J.-P., eds, Proc. SPIE Conf. Ser. Vol. 8447, Adaptive Optics Systems III. SPIE, Bellingham, p. 844761

Wallner E. P., 1983, J. Opt. Soc. Am., 73, 1771

Wildi F. P., Brusa G. 2004, in Calia D. B., Ellerbroek B. L., Ragazzoni R., eds, Proc. SPIE Conf. Ser. Vol. 5490, Advancements in Adaptive Optics. SPIE, Bellingham, p. 164

Wildi F. P., Brusa G., Lloyd-Hart M., Close L. M., Riccardi A., 2003, in Tyson R. K., Lloyd-Hart M., eds, Proc. SPIE Conf. Ser. Vol. 5169, Astronomical Adaptive Optics Systems and Applications. SPIE, Bellingham, 5169

This paper has been typeset from a *TEX/LATex* file prepared by the author.