# Laboratory comparison of coronagraphic concepts under dynamical seeing and high-order adaptive optics correction

P. Martinez, E. Aller-Carpentier, M. Kasper, A. Boccaletti, C. Dorrer, J. Baudrand

<figure><img src="image_1.png" /></figure>

## To cite this version:

P. Martinez, E. Aller-Carpentier, M. Kasper, A. Boccaletti, C. Dorrer, et al., Laboratory comparison of coronagraphic concepts under dynamical seeing and high-order adaptive optics correction. Monthly Notices of the Royal Astronomical Society, 2011, 414, pp.2112-2124. 10.1111/j.1365-2966.2011.18529.x. insu-03624338

HAL Id: insu-03624338

https://insu.hal.science/insu-03624338

Submitted on 30 Mar 2022

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Laboratory comparison of coronagraphic concepts under dynamical seeing and high-order adaptive optics correction

P. Martinez,<fnref n="1" />,<fnref n="2" />* E. Aller-Carpentier,<fnref n="1" /> M. Kasper,<fnref n="1" /> A. Boccaletti,<fnref n="3" /> C. Dorrer<fnref n="4" /> and J. Baudrand<fnref n="3" />

Accepted 2011 February 11. Received 2011 January 31; in original form 2010 December 14

## ABSTRACT

The exoplanetary science through direct imaging and spectroscopy will largely expand with the forthcoming development of new instruments at the Very Large Telescope (VLT; SPHERE), Gemini (GPI), Subaru (HiCIAO) and Palomar (Project 1640) observatories. All these ground-based adaptive optics instruments combine extremely high performance adaptive optics systems correcting for the atmospheric turbulence with advanced starlight-cancellation techniques such as coronagraphy to deliver contrast ratios of about $10^{-6}$ to $10^{-7}$. While the past 15 yr have seen intensive research and the development of high-contrast coronagraph concepts, very few concepts have been tested under dynamical seeing conditions (either during sky observation or in a realistic laboratory environment). In this paper, we discuss the results obtained with four different coronagraphs – phase and amplitude types – on the High-Order Testbench, the adaptive optics facility developed at ESO. This facility emphasizes realistic conditions encountered at a telescope (e.g. VLT), including a turbulence generator and a high-order adaptive optics system. It enables to evaluate the performance of high-contrast coronagraphs in the near-infrared operating with an adaptive optics corrected point spread function of 90 per cent Strehl ratio under 0.5 arcsec dynamical seeing.

**Key words:** instrumentation: adaptive optics – instrumentation: high angular resolution – methods: laboratory.

## 1 INTRODUCTION

The imagery and spectroscopy of extrasolar planets are among the most exciting and ambitious goals of contemporary observational programmes. Direct detection and characterization of faint objects around bright astrophysical sources is highly challenging due to the large flux ratio and small angular separation. For instance, self-luminous giant planets are typically $10^6$ times fainter than their parent star in the near-infrared. In this context, the worldwide emergence of high-contrast imaging instruments e.g. SPHERE (Beuzit et al. 2008), GPI (Macintosh et al. 2008), HiCIAO (Hodapp et al. 2008) or Project 1640 (Oppenheimer et al. 2004; Hinkley et al. 2011) will have the potential to dramatically enlarge the actual discovery space of exoplanets. These instruments will use extreme adaptive optics (XAO) systems in association with coronagraphy to overcome the contrast issue.

A coronagraph used in association with an AO system can improve the sensitivity of an imaging system to faint structures surrounding a bright source. These devices block the core of the image of an on-axis source and suppress the bright diffraction rings and halo that would otherwise reduce the dynamic range. The state-of-the-art of coronagraphy has impressively expanded during the past 15 yr, motivated by the detection and imaging of exoplanets, ideally down to Earth-like planets. An extensive review of the different families of coronagraph was carried out by Guyon et al. (2006), where optimal solutions were proposed in the context of space-based observations. Likewise, for ground-based instruments several concepts have been compared (Martinez et al. 2008).

Coronagraphs can now provide a very large on-axis extinction as demonstrated in laboratory conditions (e.g. Abe et al. 2003; Riaud et al. 2003; Crepp et al. 2006; Mawet et al. 2006, 2009; Enya et al. 2007, 2008; Martinez et al. 2009a, 2010a; Guyon et al. 2010; N'diaye et al. 2010), while very few have been tested on-sky (e.g. Baudoz et al. 2000; Boccaletti et al. 2004a; Swartzlander et al. 2008; Mawet et al. 2010). Their performance are impacted by the large

<footnote n="1">¹European Southern Observatory, Karl-Schwarzschild-Strasse 2, D-85748 Garching, Germany</footnote>

<footnote n="2">²UJF-Grenoble 1/CNRS-INSU, Institut de Planétologie et d'Astrophysique de Grenoble (IPAG) UMR 5274, Grenoble F-38041, France</footnote>

<footnote n="3">³LESIA, Observatoire de Paris Meudon, 5 pl. J. Janssen, 92195 Meudon, France</footnote>

<footnote n="4">⁴Aktiwave, 241 Ashley Drive, Rochester, NY 14620, USA</footnote>

<footnote>*E-mail: patrice.martinez@obs.ujf-grenoble.fr</footnote>
amount of residual phase aberrations that are left uncorrected by the AO system. Although coronagraphy is a mandatory technique, this critical subsystem can only reduce the contribution of the coherent part of the on-axis starlight. As a result, coronagraphic on-sky capabilities strongly depend on the AO system performance.

In this context and in the framework of the SPHERE and EPICS (Kasper et al. 2010, for the future European extremely large telescope) instruments, we have developed and tested four different coronagraphic concepts: a four-quadrants phase mask (Rouan et al. 2000), several Lyot coronagraphs (LCs; Lyot 1939), an apodized pupil Lyot coronagraph (APLC; Aime, Soummer & Ferrari 2002; Soummer et al. 2003) and a band-limited coronagraph (Kuchner & Traub 2002). The objective is to compare their respective behaviour under realistic conditions and to analyse their responses to the XAO system. The selection of an optimal concept is a by-product of this study as initiated in a previous study by means of simulations (Martinez et al. 2008). These four prototypes were tested on the High-Order Testbench (HOT; Vernet et al. 2006; Aller Carpentier et al. 2008), which reproduces realistic conditions at a telescope [e.g. Very Large Telescope (VLT)], including a turbulence generator and a high-order adaptive optics system. HOT provides a practical realistic environment (e.g. Strehl ratio of 90 per cent in *H* band under 0.5 arcsec dynamical seeing) to assess coronagraphic performance in the realm of high-contrast imaging instruments. Therefore, this study intends to provide comparative insights of coronagraphic devices but cannot address performance estimation of a particular concept for a real instrument. Although HOT provides a realistic environment for evaluating high-contrast techniques, it cannot deliver similar design optimization and stability, nor operational conditions (i.e. observational and data reduction strategies, instrumental speckle suppression techniques etc.) provided by real instruments currently in use (Project 1640) or being commissioned (SPHERE, GPI or HiCIAO).

In Section 2 we present the optical set-up and experimental conditions, while in Section 3 all coronagraphic concepts are described and details of their performance characterization are provided. In

Section 4 we discuss the laboratory results obtained under dynamical seeing and XAO, and finally in Section 5 we draw conclusions.

## 2 OPTICAL SET-UPS

In the following we present two optical set-ups. The first one, the HOT, is used to test coronagraphs under realistic conditions with dynamical seeing and XAO correction, while the second one, the coronagraphic testbench, is a dedicated optical set-up for the characterization of the coronagraph prototypes before their implementation and use on HOT (i.e. to assess their intrinsic limitations). The latter corresponds to the near-infrared (IR) arm of HOT assuming minor modifications.

### 2.1 High-Order Testbench

The HOT is a high-contrast imaging adaptive optics bench installed at the ESO headquarters. It implements an XAO system and a star and turbulence generator to create realistic conditions encountered at a telescope. It provides ideal conditions to study XAO and coronagraphy. HOT gathers several critical components as shown in Fig. 1: a turbulence generator with phase screens to simulate real seeing conditions (A), a VLT pupil mask installed on a tip-tilt mount and a 60-bimorph large stroke deformable mirror (B), a 32 × 32 microdeformable mirror (D) – DM, electrostatic MEMS device – a beam splitter transmitting the visible light to wavefront sensing either with a pyramid concept (PWFS, E), or a Shack–Hartmann wavefront sensor (SHWFS, F), while the IR light is directed towards a coronagraph (G) and an IR camera (1k × 1k HAWAII detector, H) and the ESO SPARTA real-time computer (RTC). All the optics are set on a table with air suspension in a dark room and are fully covered with protection panels forming a nearly closed box. After the generation of the dynamical aberrations, the output *f*/16.8 beam is transformed into *an* *f*/51.8 beam by a spherical on-axis mirror (C), and directed towards the pupil plane located about 1010 mm above

<figure><img src="image_3.png" /><figcaption>**Figure 1.** The HOT optical set-up: turbulence generator (A), VLT pupil, tip-tilt mount and bimorph-DM (B), spherical on-axis mirror (C), deformable mirror (D), pyramid wavefront sensor (E), Shack–Hartmann wavefront sensor (F), science channel (near-IR path; G) and infrared camera (H).</figcaption></figure>
the table level, with its axis tilted at 13.26 as in the VLT Coude train. After that, relay optics prior to the beam splitter use flat mirrors and spherical mirrors to produce an f/50 telecentric beam. All relayed optics in the IR path are made with IR achromatic doublets.

The SHWFS (F), developed by the University of Durham, provides a plate scale of 0.5 arcsec pixel⁻¹ with 31 × 31 subapertures, each ones sampled by 4 × 4 pixels of a 24 µm pixel size L3-CCD (Andor camera, read-out noise <1e⁻). The SHWFS RTC is an all-CPU architecture.

While not used during the experiment, HOT contains a pyramid wavefront sensor built by Arcetri (Pinna et al. 2008), which is formally equivalent to the LBT wavefront sensor optical design, and consists in a double refractive pyramid modulated by a tip-tilt mirror, combined with an L3-CCD camera. The PWFS uses a dedicated all-CPU RTC.

The input source for HOT is a white-light source combined either with an IR narrow-band filter of Δλ/λ = 1.4 per cent, central wavelength of 1.64 µm, or a broad-band *H* filter centred on 1.6 µm, Δλ/λ = 24 per cent, fed in by an 8-µm fibre.

In the science arm of HOT (IR path), a pupil imager system is implemented for coronagraphic components alignment purpose. All pupil plane coronagraphic components are placed into optomechanical mounts that allow *x*, *y*, focus and rotation adjustment, while focal plane masks can be adjusted in *x*, *y*, focus and tip-tilt.

## 2.2 Coronagraphic testbench

It is fundamental to evaluate the intrinsic capabilities and limitations of each coronagraphic concept on a dedicated optical testbench before testing the coronagraphs on HOT under dynamical seeing and XAO correction. This is essential to appropriately interpret experimental contrast results obtained on HOT afterwards, i.e. to distinguish the impact of the aberrations left uncorrected by the XAO system from the limitations introduced by the coronagraph prototypes themselves. In practice this is possible by using the science arm of HOT (near-IR optical path) as an independent coronagraphic optical set-up. The relevant modifications made on the science arm of HOT are described in Fig. 2 where one can compare the standard configuration (top) to that of the coronagraphic testbench configuration (bottom). The main difference comes from the fact that a flat mirror used to fold the beam after the DM is replaced by an entrance fibre, which is used as a star source for the optical set-up (6.6-µm diameter). The coronagraphic testbench uses a VLT pupil mask made in a laser-cut, stainless-steel sheet. The dimensions and positioning of the VLT pupil in both the coronagraphic testbench and HOT are alike.

The Strehl ratio of the coronagraphic testbench was evaluated in the *H* band to be 93 ± 1 per cent by measuring the peak-intensity ratio of the experimental point spread function (PSF) to that of the theoretical PSF normalized to the total intensity. The theoretical PSF is created through two different methods, both leading to the same Strehl ratio: (1) by performing the forward fast Fourier transform (FFT) of the autocorrelation of an oversampled and uniformly illuminated entrance pupil image from our telescope pupil mask; (2) by performing the FFT of a simulated aperture function with radius determined from the experimental PSF on the basis of photometric criteria.

As high-frequency wavefront components impose an important contrast limitation, we determined the wavefront error of the optical components prior to the pupil stop in the coronagraphic testbench to an overall total amount of ~λ/67 rms at 1.6 µm, i.e. 24 nm rms (Martinez et al. 2009c).

<figure><img src="image_4.png" /><figcaption>**Figure 2.** From the top to the bottom: details of the near-IR arm of HOT (science channel), and its configuration when used as an independent coronagraphic testbench (a folding mirror is replaced by an entrance fibre).</figcaption></figure>

All the parameters described below are identical when operating with the HOT or the coronagraphic testbench: the VLT pupil in the first pupil plane of the IR arm has 3-mm diameter, a central obscuration of 0.47 mm and four spider vanes with 15 µm thickness. The coronagraphic focal masks are installed at an *F*/48.4 beam. The pupil is re-imaged in the Lyot plane where the pupil stop is installed, and appears with similar dimensions as in the first pupil plane of the IR arm. All re-imaging optics are made with λ/10 achromatic IR doublets. Therefore, all coronagraphs (Fig. 3) can be tested similarly on HOT and on the coronagraphic testbench.

## 2.3 Definition of metrics

Several metrics can be used to quantify the capability of a coronagraph to suppress the on-axis starlight.

(i) The *rejection rate* (τ): ratio of total intensity of the direct image to that of the coronagraphic image.

(ii) The *peak rejection rate* (τ₀): ratio of the maximum intensity of the direct image to that of the coronagraphic image.

(iii) The *contrast* (C): ratio of the coronagraphic image at a given angular separation to that of the maximum of the direct image to the intensity, azimuthally averaged.

In addition, as contrast evaluation is no longer suitable on post-processed coronagraphic images, the azimuthal standard
<figure><img src="image_5.png" /><figcaption>**Figure 3.** Various coronagraphic devices manufactured and tested on HOT: APLC (1), BLCs (2), LCs (3) and FQPM (4).</figcaption></figure>

deviation – quantifying the ability to pick out a companion at a given angular distance – will be applied in such situation. The azimuthal standard deviation is measured in a ring of width $\lambda/D$ on the post-processed image, i.e. it includes all pixels within one resolution element. Although the detectability is commonly estimated at the level of $5\sigma$ on real data (e.g. Hinkley et al. 2007) and since we are not sensitive to confidence level threshold with our data, we quantify the detectability at the level of $1\sigma$. We adopt the azimuthal contrast and standard deviation as primary metrics although we note they are conservative estimates, and that other criteria adapted to the case of high-contrast images have been discussed by Marois et al. (2008).

## 2.4 Pupil stops optimization

The design of the coronagraph pupil stops (Fig. 4) emphasizes the minimization of diffraction effects (concentration of light around the edges, spider vane structures and central obscuration) while preserving the throughput. All pupil stops have been optimized following the metric presented in Boccaletti et al. (2004b), which attempts to maximize the ratio of the off-axis PSF throughput to the stellar diffraction residuals with the use of homothetic stop patterns (same shape as the VLT pupil but under/oversized). This optimization is performed at the operating wavelength in a wavefront error-free condition (i.e. Strehl equal to 100 per cent). The performance of the coronagraph combined with the optimized pupil stop is afterwards compared with intrinsic or external limitations affecting the coronagraph efficiency (e.g. stellar size, chromaticism...). Based on the analysis of this budget error and accounting for alignment con-

siderations the optimized pupil stop is retained, relaxed or further optimized.

## 2.5 Chromaticity considerations

A smooth and flat chromaticity response of the coronagraph is paramount for exoplanet detection as broad-band observations are required for spectroscopy. For instance, most of the next generation of high-contrast imaging instruments will use integral field spectroscopy for data collection. Analysing coronagraphic performance in a large range of small bandwidths would therefore be extremely relevant. However, it requires a dedicated optimization of each coronagraph that can only be carried out in correlation with specific observation and data reduction strategies, which is beyond the scope of this study and beyond the capabilities of our experimental set-up. Therefore, the chromatic propagation (contrast versus wavelength) of each coronagraph cannot be addressed in this study, although it is certainly a relevant comparative criterion.

# 3 CORONAGRAPH PROTOTYPES

In this section, we provide details of our four coronagraph prototypes (Fig. 3) and their performance. We first examine the impact of several error sources on their respective efficiency by probing the effect of manufacturing, wavefront error, chromaticity and the source diameter. A performance characterization realized on the coronagraphic testbench will be presented and discussed afterwards. This first study is a prerequisite for proper analysis and interpretation of the results obtained on HOT under dynamical seeing and XAO correction, i.e. to identify the limitations occurring in the coronagraphic images.

## 3.1 Four-quadrants phase mask

The four-quadrant phase mask (FQPM) is a phase coronagraph proposed by Rouan et al. (2000). The mask divides the focal plane into four areas, two of which being $\pi$ phase shifted. As a result, a destructive interference occurs in the relayed pupil, where the on-axis starlight is rejected and filtered by an appropriate pupil stop. This concept has been studied from a theoretical point of view (Rouan et al. 2000; Riaud et al. 2001), in the laboratory (Riaud et al. 2003), and has provided on-sky results (e.g. Boccaletti et al. 2004a, 2008; Gratadour et al. 2005; Riaud et al. 2006).

Our prototype was manufactured by GEPI in collaboration with LESIA both from the Paris Observatory. The prototype is a monochromatic device, i.e. the $\pi$ phase shift is obtained at a single wavelength. Achromatic FQPMs can be made using half-waves plates (SPHERE coronagraph; Mawet et al. 2006), annular groove

<figure><img src="image_6.png" /><figcaption>**Figure 4.** From the left to the right: as specified pupil stops (black) superimposed to the VLT pupil (red) for the FQPM, LC, APLC and BLC.</figcaption></figure>
phase mask (AGPM; Mawet et al. 2005) or optical vortex coronagraph (OVC; Mawet et al. 2009). We also note that achromatic implementation of the FQPM exist (Baudoz et al. 2007).

The FQPM is manufactured by engraving of two opposite quadrants on an optical medium. The substrate is made in INFRASIL 301 with 16-mm diameter and 3-mm thickness (±0.1 mm). The optical quality for the two faces is $λ/20$ peak-to-valley at 633 nm. The thickness (e) and optical index (n) of the layer have been defined to provide a π phase shift at the operating wavelength $λ_0 = 1.65$ µm (H band) following:

$$ \Phi = \frac{2\pi(n-1)e}{\lambda_0} \quad (1) $$

As the thickness of the FQPM step (e) directly defines the optimal wavelength at which the attenuation is best, an error between specified and manufactured step reduces the FQPM efficiency. Our specification was $e = 1.89$ µm (±3 per cent). A dedicated visible spectroscopic bench was used at LESIA to measure the thickness of the FQPM step (Riaud et al. 2003; Boccaletti et al. 2004a) by measuring the wavelength corresponding to the optimal nulling in the near-IR. In practice this facility enables to record low-resolution spectra with a source centred on the FQPM (coronagraphic spectrum) and with the source out of the FQPM (direct spectrum). The ratio of these two spectra allows one to identify different coronagraphic minima that correspond to a phase difference between the quadrant of $ΔΦ = kπ$ (with $k = 1, 3, 5, 7$ and so on). From results of a given identified order in the visible (odd value of k), it is straightforward to derive from these data and from the known optical index of the material, the operating wavelength of the FQPM ($λ_0$) at $k = 1$. Equation (1) can then be used to extract the thickness (e). While a precision of less than 3 per cent was required, a depth accuracy of 0.2 per cent was finally obtained after several runs.

Ideally the transition between the four quadrants must be infinitely sharp. Departure from this ideal case decreases the capability of the real device. Microscopic inspection of the manufactured FQPM has shown that the transition quality is faster than 1 µm (2 µm peak-to-peak transition). The accuracy on the quadrant orthogonality has been measured to be ≤0.8 arcmin.

The manufactured FQPM is sensitive to several parameters as discussed previously (Table 1). Specific care has been taken when defining the pupil stop (Fig. 4, left) in order not to allow higher rejection rate than the limit imposed by inherent parameters from the prototype. It is indeed useless to define an aggressive pupil stop in diffraction-limited regime but rather convenient to save throughput through a relaxed optimization when external error sources, e.g. the stellar leakage, chromaticity, will first set the limitation in the efficiency of the coronagraph. Table 1 quantifies all manufacturing defects and error sources that impact the FQPM efficiency, and will be discussed in the next subsection. These data were used during

<table><caption>**Table 1.** Manufacturing defects and error sources impact on the total rejection rate of the FQPM.</caption><thead><tr><th>Parameter</th><th>Achievable rejection (τ)</th></tr></thead><tbody><tr><td>Step thickness (0.2 per cent)</td><td>120 668</td></tr><tr><td>Quadrant transition (1 µm)</td><td>1890</td></tr><tr><td>Chromaticity (R = 1.4 per cent)</td><td>23 830</td></tr><tr><td>Chromaticity (R = 24 per cent)</td><td>121</td></tr><tr><td>Wavefront error (λ/67 rms at 1.64 µm)</td><td>428</td></tr><tr><td>Source diameter (6.6 µm, i.e. 0.08λ/D)</td><td>1350</td></tr><tr><td>Central obscuration (15 per cent)</td><td>680</td></tr></tbody></table>

the optimization process. The FQPM pupil stop mimics the VLT pupil mask (see Fig. 4, first pattern from the left) with a spider vane thickness larger by a factor of 5.5 (82.5 ± 4 µm), an outer diameter smaller by a factor of 0.90Φ (2.70 ± 0.002 mm) and a central obscuration equal to ~0.3Φ (0.90 ± 0.002 mm). The pupil stop transmission is 70 per cent.

### 3.1.1 Coronagraphic capabilities

Table 1 presents the manufacturing defects of the prototype and several error sources that reduced the FQPM efficiency. The formulae quantifying the impact of these particular error sources are not detailed here since they are available in the literature (e.g. Riaud et al. 2001, 2003). Several aspects are considered such as manufacturing defects, chromaticity (depending on the filter bandpass we use in the experiment), stellar leakage (the FQPM is very efficient to observe at close angular separations of the star but highly sensitive to the stellar angular size), high-frequency wavefront errors of the coronagraphic testbench and the central obscuration of the pupil (VLT like, performance of the FQPM degrades with the central obscuration size). As all these aspects can be quantified it is straightforward to estimate the best performance that can be reached with our prototype. Assuming that all these independent errors are added quadratically, the expected global rejection factor ($τ$) is found to be 334 and 114 for 1.4 and 24 per cent spectral bandwidth, respectively.

The FQPM coronagraphic characterization is presented in Table 2. Both narrow- and broad-band runs have led to results compatible with the limitations previously described ($τ = 278$ and 77 for $Δλ/λ = 1.4$ and 24 per cent, respectively, while the expected nulling for similar bandwidth were 334 and 114). The difference between experimental and expected values may come from the uncertainty of the filter bandpass (more likely in the case of the polychromatic test, i.e. the actual filter spectral transmission is not taken into account), or typical errors that arise in a coronagraphic system and neglected here (offset pointing, alignment issue, pupil shear and rotation...). With these considerations in mind, the agreement between experiment and expectation is fairly good.

The FQPM contrast evolves from ~$10^{-4}$ at short angular separations ($θ < 3λ/D$) to $3 × 10^{-6}$ in the halo ($θ = 20λ/D$). Being a monochromatic device, the FQPM is sensitive to the filter bandpass, where short angular distances are mostly affected. At inner working angle (IWA; ~$1λ/D$), the contrast delivered is $4 × 10^{-3}$ and $9.5 × 10^{-4}$ in monochromatic and polychromatic images, respectively.

## 3.2 Lyot coronagraphs

A set of three LCs with different mask diameters has been manufactured using wet-etch lithography on BK7 glass by GEPI. They are made using Cr deposition (+Au) to reach an optical density (OD) of 6.0 at 1.65 µm. The diameters are 360, 390 and 600 µm [i.e. 4.5, 4.9 and $7.5λ/D$, respectively, defined hereafter as LC(1), LC(2) and LC(3), respectively]. The diameter accuracy is ~1 µm. Inspection with a microscope has been carried out and confirmed the perfectly circular shape.

The LC pupil stop throughput is ~60 per cent and has been optimized for the smallest Lyot mask diameter ($4.5λ/D$), being usable for the two others, avoiding multiple pupil stops for the LCs. As the Lyot mask diameter increases, the diffracted light in the pupil stop is more localized and concentrated. The LC pupil stop mimics the VLT pupil mask (see Fig. 4, second pattern from the left) with a
<table><caption>**Table 2.** Performance of each coronagraph concepts obtained when tested on the coronagraphic testbench (turbulence free).</caption><thead><tr><th></th><th>BW (per cent)</th><th>τ</th><th>τ<sub>0</sub></th><th>C<sub>3λ/D</sub></th><th>C<sub>12λ/D</sub></th><th>C<sub>20λ/D</sub></th></tr></thead><tbody><tr><td colspan="7" style="text-align: center;">Raw images</td></tr><tr><td rowspan="2">FQPM</td><td>1.4</td><td>278</td><td>760</td><td>7.9 × 10<sup>-5</sup></td><td>9.4 × 10<sup>-6</sup></td><td>2.9 × 10<sup>-6</sup></td></tr><tr><td>24</td><td>77</td><td>156</td><td>1.9 × 10<sup>-4</sup></td><td>1.2 × 10<sup>-5</sup></td><td>3.9 × 10<sup>-6</sup></td></tr><tr><td rowspan="2">LC(1)</td><td>1.4</td><td>239</td><td>586</td><td>1.0 × 10<sup>-4</sup></td><td>9.9 × 10<sup>-6</sup></td><td>4.9 × 10<sup>-6</sup></td></tr><tr><td>24</td><td>247</td><td>640</td><td>1.2 × 10<sup>-4</sup></td><td>1.2 × 10<sup>-5</sup></td><td>5.7 × 10<sup>-6</sup></td></tr><tr><td rowspan="2">LC(2)</td><td>1.4</td><td>253</td><td>606</td><td>9.5 × 10<sup>-5</sup></td><td>9.0 × 10<sup>-6</sup></td><td>3.8 × 10<sup>-6</sup></td></tr><tr><td>24</td><td>231</td><td>593</td><td>1.7 × 10<sup>-4</sup></td><td>1.3 × 10<sup>-5</sup></td><td>5.3 × 10<sup>-6</sup></td></tr><tr><td rowspan="2">LC(3)</td><td>1.4</td><td>586</td><td>1477</td><td>–</td><td>8.5 × 10<sup>-6</sup></td><td>3.8 × 10<sup>-6</sup></td></tr><tr><td>24</td><td>408</td><td>1120</td><td>–</td><td>1.2 × 10<sup>-5</sup></td><td>6.0 × 10<sup>-6</sup></td></tr><tr><td rowspan="2">APLC</td><td>1.4</td><td>489</td><td>627</td><td>5.0 × 10<sup>-5</sup></td><td>2.3 × 10<sup>-6</sup></td><td>1.2 × 10<sup>-6</sup></td></tr><tr><td>24</td><td>355</td><td>674</td><td>1.5 × 10<sup>-4</sup></td><td>3.5 × 10<sup>-6</sup></td><td>1.8 × 10<sup>-6</sup></td></tr><tr><td rowspan="2">BLC</td><td>1.4</td><td>–</td><td>–</td><td>–</td><td>–</td><td>–</td></tr><tr><td>24</td><td>2410</td><td>2554</td><td>–</td><td>5.6 × 10<sup>-7</sup></td><td>2.6 × 10<sup>-7</sup></td></tr></tbody></table>

spider vane thickness larger by a factor of 4 (60 ± 4 µm), an outer diameter smaller by a factor of 0.78Φ (2.36 ± 0.002 mm) and a central obscuration equal to ~0.17Φ (0.50 ± 0.002 mm).

### 3.2.1 Coronagraphic capabilities

As the manufacturing aspects of the LCs do not present any particular issues or difficulties (i.e. the LC is the easiest concept to manufacture in practice) no error budget has been set. By contrast to the FQPM, stellar leakage, central obscuration, manufacturing defects or chromaticity are not strongly limiting factors. With the as manufactured pupil stop and focal masks, simulations assuming ideal conditions have shown that the expected total rejection rate ($\tau$) in a monochromatic case is 250, 260 and 630 for the 4.5, 4.9 and 7.5$\lambda/D$ [LC(1) to LC(3)] masks, respectively.

The characterization results are presented in Table 2. The monochromatic run ($\Delta\lambda/\lambda = 1.4$ per cent) nicely fulfilled expectations (total rejection rate predicated from simulations). The measured contrast evolves from ~$10^{-4}$ at close angular separations ($\theta \sim 3\lambda/D$) to 4–6 × $10^{-6}$ at best at farther separations ($\theta = 20\lambda/D$). Apart from the peak attenuation, no real gain is observable by using larger mask configurations [e.g. LC(3)].

All LCs demonstrate almost similar performance from $\Delta\lambda/\lambda = 1.4$ to 24 per cent, and therefore do not present strong chromaticity dependence.

## 3.3 Apodized pupil Lyot coronagraph

### 3.3.1 Prototype details

The APLC (Aime et al. 2002; Soummer et al. 2003) combines pupil apodization with hard-edge focal plane mask. Our APLC is a $4.5\lambda/D$ mask diameter configuration coupled with its corresponding optimized bagel-shaped pupil apodizer. This APLC configuration is specifically adapted for obstructed entrance apertures (Soummer 2005; Martinez et al. 2007; Soummer et al. 2009). The prototype has been detailed in a previous paper (Martinez et al. 2009a). Briefly, the $4.5\lambda/D$ hard-edge opaque Lyot mask was fabricated by GEPI at the Paris Observatory [360 ± 1 µm in diameter, OD = 6.0 at 1.65 µm using two metallic layers of chrome (20 nm) and gold

(200 nm)], while the apodizer has been manufactured by Precision Optical Imaging in Rochester, New York, and is made with binary metal pixels (Martinez et al. 2009a,b, 2010b). The profile accuracy is of about 3 per cent and the transmission of the apodizer is ~50 per cent.

The APLC pupil stop (Fig. 4, third pattern from the left) mimics the VLT pupil mask with a spider vane thickness larger by a factor of 4 (60 ± 4 µm), an outer diameter smaller by a factor of 0.96Φ (2.88 ± 0.002 mm) and a central obscuration equal to 0.16Φ (0.49 ± 0.002 mm). The pupil stop throughput is about 92 per cent.

### 3.3.2 Coronagraphic capabilities

In the coronagraphic testbench we installed the entrance pupil mask (VLT-like pupil) and the apodizer mask in the same collimated beam because of the lack of space. Therefore, the apodizer cannot be in practice in the same plane as the VLT pupil mask. To minimize this imaging issue, the apodizer was placed inside a rotating adjustable-length lens tube that allows translation motion along the collimated beam, and was adjusted at ~3.5 mm from the pupil mask. The APLC performance is presented in Table 2, where an order of magnitude discrepancy (mostly in the halo) was found between theory and measured data (Martinez et al. 2009a). Some error sources impact the APLC performance, such as the apodizer profile error (~3 per cent), or the defocus between the apodizer and the pupil plane (~3.5 mm) although the depth of focus at the pupil is ~7 mm. Because the quality of the optical set-up also plays a role it is difficult to estimate the respective impact of each aspects. However, it is known that the apodizer positioning error in focus mainly affects the halo level, while the apodizer profile error affects both peak level and contrast in the halo. In addition, typical defects that occur at the level of the focal mask (pointing error, defocus) and at the pupil stop (shear and rotation) impact the APLC performance as well. All these aspects acting together may explain the discrepancy between data and theory. The contrast evolves from ~$10^{-4}$ at close angular separations ($\theta > 3\lambda/D$) to ~$10^{-6}$ at best at farther separations ($\theta = 20\lambda/D$). The impact of chromatism is slightly observable at small angular separations (less than $4\lambda/D$), but the halo is found to be achromatic. In addition, we note that dedicated studies aimed to
mitigate the chromatic response of the APLC (e.g. Martinez et al. 2007; Soummer et al. 2011).

## 3.4 Band-limited coronagraph

### 3.4.1 Prototype details

The band-limited coronagraph (BLC; Kuchner & Traub 2002) is an improvement of the LC concept through the use of a specific design of the amplitude focal plane mask. The mask has a power spectrum with power in a limited range of frequencies insuring that the mask is designed to both remove starlight and the diffraction effects caused by the removal of the light. In principle the BLC is achromatic. Our prototype is based on a band-limited function proposed by Kuchner & Traub (2002):

$$M(r) = N \left( 1 - \text{sinc} \left( \frac{\epsilon r D}{\lambda} \right) \right), \quad (2)$$

where $\lambda$ is the wavelength of the application, $r$ the radial coordinates in the image plane, $D$ the telescope primary diameter, $\epsilon$ the bandwidth which rules the IWA of the coronagraph and finally $N$ is a constant of normalization insuring that $0 \le M(r) \le 1$.

Our prototype corresponds to $\epsilon = 0.17$ (i.e. IWA = $5\lambda/D$) and was manufactured by Precision Optical imaging (Rochester, New York) with binary metal pixels (Martinez et al. 2009c). The mask was designed for 1.64 µm, and fabricated using wet-etch contact lithography of an aluminium layer (OD = 8 +, $e = 2000$ Å) deposited on a BK7 substrate ($\lambda/10$ peak-to-valley, 0.5-inch diameter). Antireflection coating in H band has been applied on each face. The BLC uses 5 µm pixels. Profile accuracy is of about 5 per cent of the specification, where the error is mostly localized in the outer part of the mask (high-transmission part), as the centre part (for the low transmissions) is highly accurate (Martinez et al. 2009c). The error in the outer part originates in a calibration issue of the process that was later corrected with new prototypes demonstrating a profile error below 1 per cent.

The BLC pupil stop optimized for HOT (i.e. designed for the VLT pupil, Fig. 4, right) has been manufactured with a spider vane thickness larger than the entrance pupil by a factor of 27 (0.4 ± 4 µm), an outer diameter that is smaller by a factor of 0.80Φ (2.40 ± 0.002 mm) and a central obscuration that is equal to 0.35Φ (1.05 ± 0.002 mm). The pupil stop throughput is about 42 per cent. In principle, the BLC can accommodate arbitrary telescope apertures with proper pupil stop optimization (Kuchner & Traub 2002), although this can usually impose aggressive reduction of the pupil when dealing with sophisticated telescope apertures (e.g. ELTs; Martinez et al. 2008), as in our situation.

The BLC characterization on the coronagraphic testbench is presented in Table 2 and has been obtained with a clear aperture and a 70 per cent throughput pupil stop as the pupil stop optimized for the VLT pupil was not available during the run carried out on the coronagraphic testbench. However, a new prototype of this BLC was recently tested on the coronagraphic testbench with the VLT pupil and its corresponding pupil stop yielding to similar contrast level in the halo as the one discussed in the next subsection, while the peak and total rejection rate were improved by more than a factor of 2.

### 3.4.2 Coronagraphic capabilities

The BLC has demonstrated impressive performance on the coronagraphic testbench (Table 2), contrasts evolve from ~3 × 10⁻⁵ at IWA to ~3 × 10⁻⁸ at $20\lambda/D$, while the peak rejection is 2554

<table><caption>**Table 3.** Coronagraphs main characteristics.</caption><thead><tr><th>Concept</th><th>IWA (λ/D)</th><th>Throughput (per cent)</th></tr></thead><tbody><tr><td>FQPM</td><td>1.0</td><td>70</td></tr><tr><td>LC(1)</td><td>2.3</td><td>60</td></tr><tr><td>LC(2)</td><td>2.5</td><td>60</td></tr><tr><td>LC(3)</td><td>3.8</td><td>60</td></tr><tr><td>APLC</td><td>2.3</td><td>45</td></tr><tr><td>BLC</td><td>5.0</td><td>42</td></tr></tbody></table>

(i.e. more than four times the peak rejection of most of the other concepts). More details on the BLC manufacturing and characterization can be found in Martinez et al. (2009c), where achromaticity of the concept has been confirmed.

## 4 RESULTS UNDER DYNAMICAL SEEING AND XAO

While in the previous section we discussed the characterization of our prototypes on the coronagraphic testbench, the present section is dedicated to the test of all the coronagraphs on HOT under dynamical seeing and XAO correction.

In Table 3, we recall the IWA and throughput for each concept. In addition, Fig. 5 (right) compares the radial transmission of each coronagraph, which provides the transmission of an off-axis companion at close and farther radii. Fig. 5 (right) has been obtained by simulation given the shape of the entrance VLT pupil and pupil stop, and each coronagraphic mask used in the experiment. One should note that contrast evaluation presented in this paper does not account for the radial transmission of each coronagraphic mask, which is especially important at close radii from the axis. Indeed from Fig. 5 (right) one can observe that the FQPM allows access to very short angular separations that cannot be observed by others.

### 4.1 Experimental conditions

During the experiment, the XAO system was operating with the SHWFS. The dynamical turbulence is generated using two-phase screens in reflection rotating independently from each other, and specified to reproduce a turbulence of 0.5 arcsec seeing. Measurements with an HASO 64 Shack-Hartmann sensor have been carried out to verify the phase screens parameters. The power spectrum of the reconstructed wavefront was compared to the theoretical and specified Von Karman spectrum ($L_0 = 25$ m and $r_0 = 12$ cm) with good agreement. The phase screens are low-order aberration reduced to avoid saturation of the DM (limited actuator stroke of about 2 µm).

The static aberrations, corresponding to common-path wavefront errors, have been reduced from ~200 to 50 nm rms using a bimorph DM in open loop, i.e. by applying a corrective voltage pattern to the initial voltage pattern that set-up the mirror. The SHWFS non-common path wavefront errors have been measured using a reference fibre at the entrance of the SHWFS, and slope offsets have been applied to the DM not to take into account these aberrations in the correction. Additionally, some slope offsets have been applied to correct for non-common path errors from the near-IR optical path on the basis of the PSF image quality. Further details can be found in Vernet et al. (2006) and Aller Carpentier et al. (2008).

The SHWFS closed-loop runs at 80 Hz using 600 modes for the modal reconstruction on an 8-m pupil. In 0.5 arcsec seeing with 1.3 m s⁻¹ wind speed, the system delivers a 90 per cent Strehl ratio
<figure><img src="image_7.png" /><figcaption>**Figure 5.** Left: raw coronagraphic contrast profiles azimuthally averaged ($\Delta\lambda/\lambda = 24$ per cent). Right: radial transmission of each coronagraph.</figcaption></figure>

in *H* band at high flux (star magnitude of 5). The DM actuator pitch being equal to 340 µm, the AO cut-off frequency is localized in the field at an angular separation of $15\lambda/D$, i.e. 0.6 arcsec for an 8-m telescope in *H* band.

The experiment was carried out with a series of 3-s short exposure images averaged over 3 min, and neutral density filters were applied on non-coronagraphic images only. Dark frames were obtained by switching off the artificial star source. The data reduction corrects bad pixels, background and scales images by the exposure time and OD.

While the contrast profiles are evaluated on raw coronagraphic images, all coronagraphic images were post-processed with a high-pass filter (HPF) to remove smooth structures (atmospheric speckle halo), leaving the small-scale high-frequency components (e.g. planets) unaffected. These HPF images allow us to identify the limit imposed by the speckle background when the dynamic atmospheric wavefront has been reduced by the XAO system. Detectability estimates for profile evaluation are applied on HPF images, where contrasts evaluation is no longer suitable.

## 4.2 Laboratory results

### 4.2.1 General trend

All raw images (Fig. 6) present similar features: the coronagraphic images demonstrate starlight attenuation, and exhibit higher energy level at close angular separations (from the centre of the image to basically the IWA) due to diffraction residuals and pinned speckles. A radial trend in speckle intensity is observable in the image: while speckles closer to the centre are brighter, the speckle intensity decreases at larger angular separations until it increases to reach a local maximum at the AO cut-off frequency ($15\lambda/D$). The AO cut-off frequency is clearly observable in all images as its position in the field is identified by the slope of intensity in the speckle field at 0.6 arcsec from the images centre (as expected).

Diffraction resulting from the spider vanes structures of the VLT pupil is apparent in the images. The shape and extent of the central pattern of each coronagraphic image are influenced by the nature of the coronagraph (we remind the reader that IWA are different pending on the coronagraphic concept considered, see Table 3). For instance, the FQPM image exhibits the expected FQPM butterfly-like pattern with the four blind zones introduced by the quadrant transitions on the mask. While the FQPM IWA is the sharpest one, more

energy is observable close to the star with this coronagraph due to the impact of the central obscuration. The HPF images (Fig. 7) correspond to coronagraphic images removed from smooth structures (i.e. atmospheric speckle halo), while small-scale high-frequency components remain alike.

### 4.2.2 Performance comparison

All coronagraphs approximately deliver similar contrast levels from $5\lambda/D$ to $15\lambda/D$ (AO cut-off frequency, see Fig. 8 and Table 4). Likewise, detectability obtained on HPF images is comparable (Fig. 9). Performance essentially differs from the centre of the image to the IWA (different from a concept to another). Small IWA concepts (e.g. FQPM) deliver similar performance in the AO control domain as others, but the peak rejection rate is lower. While large IWA concepts [LC(3) or BLC] deliver very high peak rejection rate, contrast levels are not improved in the AO control domain. More standard IWA concepts [LC(1), LC(2) or APLC] stand in-between (improved peak rejection rate with respect to small IWA concepts, with similar contrast levels in the halo).

From the total rejection obtained with the FQPM in narrow band and the budget error defined in Table 1 we can derive an estimation of the wavefront error after XAO (conservative assessment, i.e. all other error sources apart from the ones described in Table 1 are neglected). In the case of HOT the size on the source is 8 µm ($0.1\lambda/D$) instead of 6.6 µm (coronagraphic testbench), then the total rejection imposed by the stellar diameter changes from 1380 to 930. We obtain an estimate of the wavefront error after XAO to be of the order of $\lambda/18$ rms at 1.64 µm, i.e. ~90 nm rms. This value is consistent with the 90 per cent Strehl estimation at the same wavelength, i.e. ~$\lambda$/20 rms, and reads realistically considering the 50 nm rms static aberrations measurement of the common path (left uncorrected by the 60-bimorph DM), and the 24 nm rms static aberrations estimation of the non-common path (IR path, high-frequency wavefront error prior to the pupil stop). A rough estimation of the AO residual wavefront error would therefore be at the level of ~16 nm rms.

When the atmospheric residual halo is removed by post-processing (HPF images, Fig. 9), the detectability levels obtained either almost achieved the intrinsic limitation of the coronagraphs (a factor of 2 to 6 for the FQPM, or the APLC), or roughly attained the coronagraph limitation (LCs). The BLC case is particular as more than an order of magnitude difference remains between HPF contrasts and the actual limits imposed by the BLC
<figure><img src="image_8.png" /><figcaption>**Figure 6.** XAO-corrected coronagraphic images under 0.5 arcsec seeing (Δλ/λ = 1.4 per cent). Top row (from left to right): FQPM, APLC and LC(1) images. Bottom row (from left to right): LC(2), LC(3) and BLC images. The arbitrary colour distribution and image dynamic aim at enhancing the contrast for the sake of clarity.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>**Figure 7.** High-pass filtered coronagraphic images (corresponding to Fig. 6). Top row (from left to right): FQPM, APLC and LC(1) images. Bottom row (from left to right): LC(2), LC(3) and BLC images. The arbitrary colour distribution and image dynamic aim at enhancing the contrast for the sake of clarity.</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>**Figure 8.** Raw coronagraphic contrast profiles azimuthally averaged under 0.5 arcsec dynamical seeing corrected by XAO (Strehl ~90 per cent). Left: $\Delta\lambda/\lambda = 1.4$ per cent. Right: $\Delta\lambda/\lambda = 24$ per cent.</figcaption></figure>

<table><caption>**Table 4.** Performance of each coronagraph concepts obtained when tested on HOT with turbulence and XAO.</caption><thead><tr><th></th><th>BW (per cent)</th><th>τ</th><th>τ<sub>0</sub></th><th>C<sub>3λ/D</sub></th><th>C<sub>12λ/D</sub></th><th>C<sub>20λ/D</sub></th></tr></thead><tbody><tr><td colspan="7" style="text-align: center;">Raw images</td></tr><tr><td>FQPM</td><td>1.4</td><td>29</td><td>62</td><td>1.5 × 10<sup>-3</sup></td><td>9.7 × 10<sup>-5</sup></td><td>3.8 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>16</td><td>41</td><td>2.2 × 10<sup>-3</sup></td><td>1.6 × 10<sup>-4</sup></td><td>5.8 × 10<sup>-5</sup></td></tr><tr><td>LC(1)</td><td>1.4</td><td>65</td><td>244</td><td>1.0 × 10<sup>-3</sup></td><td>1.6 × 10<sup>-4</sup></td><td>6.1 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>57</td><td>191</td><td>1.4 × 10<sup>-3</sup></td><td>2.3 × 10<sup>-4</sup></td><td>1.0 × 10<sup>-4</sup></td></tr><tr><td>LC(2)</td><td>1.4</td><td>55</td><td>183</td><td>8.9 × 10<sup>-4</sup></td><td>1.2 × 10<sup>-4</sup></td><td>4.6 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>71</td><td>268</td><td>1.2 × 10<sup>-3</sup></td><td>2.0 × 10<sup>-4</sup></td><td>8.7 × 10<sup>-4</sup></td></tr><tr><td>LC(3)</td><td>1.4</td><td>184</td><td>1719</td><td>—</td><td>1.3 × 10<sup>-4</sup></td><td>6.2 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>103</td><td>767</td><td>—</td><td>2.1 × 10<sup>-4</sup></td><td>9.2 × 10<sup>-5</sup></td></tr><tr><td>APLC</td><td>1.4</td><td>69</td><td>174</td><td>6.6 × 10<sup>-4</sup></td><td>1.1 × 10<sup>-4</sup></td><td>4.6 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>46</td><td>118</td><td>1.4 × 10<sup>-3</sup></td><td>1.3 × 10<sup>-4</sup></td><td>5.4 × 10<sup>-5</sup></td></tr><tr><td>BLC</td><td>1.4</td><td>136</td><td>373</td><td>—</td><td>1.4 × 10<sup>-4</sup></td><td>6.5 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>140</td><td>373</td><td>—</td><td>1.6 × 10<sup>-4</sup></td><td>6.5 × 10<sup>-5</sup></td></tr><tr><td colspan="7" style="text-align: center;">High-pass filtered images</td></tr><tr><td>FQPM</td><td>1.4</td><td>—</td><td>—</td><td>4.6 × 10<sup>-4</sup></td><td>2.3 × 10<sup>-5</sup></td><td>1.1 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>—</td><td>—</td><td>5.6 × 10<sup>-4</sup></td><td>2.8 × 10<sup>-5</sup></td><td>1.3 × 10<sup>-5</sup></td></tr><tr><td>LC(1)</td><td>1.4</td><td>—</td><td>—</td><td>2.1 × 10<sup>-4</sup></td><td>2.3 × 10<sup>-5</sup></td><td>1.1 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>—</td><td>—</td><td>1.3 × 10<sup>-4</sup></td><td>1.4 × 10<sup>-5</sup></td><td>9.1 × 10<sup>-6</sup></td></tr><tr><td>LC(2)</td><td>1.4</td><td>—</td><td>—</td><td>2.2 × 10<sup>-4</sup></td><td>1.1 × 10<sup>-5</sup></td><td>1.1 × 10<sup>-5</sup></td></tr><tr><td></td><td>24</td><td>—</td><td>—</td><td>1.5 × 10<sup>-4</sup></td><td>1.3 × 10<sup>-5</sup></td><td>8.5 × 10<sup>-6</sup></td></tr><tr><td>LC(3)</td><td>1.4</td><td>—</td><td>—</td><td>—</td><td>1.1 × 10<sup>-5</sup></td><td>8.3 × 10<sup>-6</sup></td></tr><tr><td></td><td>24</td><td>—</td><td>—</td><td>—</td><td>1.2 × 10<sup>-5</sup></td><td>8.3 × 10<sup>-6</sup></td></tr><tr><td>APLC</td><td>1.4</td><td>—</td><td>—</td><td>1.1 × 10<sup>-4</sup></td><td>1.2 × 10<sup>-5</sup></td><td>8.5 × 10<sup>-6</sup></td></tr><tr><td></td><td>24</td><td>—</td><td>—</td><td>1.5 × 10<sup>-4</sup></td><td>1.1 × 10<sup>-5</sup></td><td>5.1 × 10<sup>-6</sup></td></tr><tr><td>BLC</td><td>1.4</td><td>—</td><td>—</td><td>—</td><td>9.0 × 10<sup>-6</sup></td><td>4.2 × 10<sup>-6</sup></td></tr><tr><td></td><td>24</td><td>—</td><td>—</td><td>—</td><td>8.9 × 10<sup>-6</sup></td><td>4.8 × 10<sup>-6</sup></td></tr></tbody></table>

prototype. These results indicate that an improvement in the wave-front error control would be advantageous for the BLC, while only a slight improvement can be obtained with the FQPM and APLC and none with the LCs.

As the azimuthal standard deviation (detectability) is a conservative estimate and the HPF images do not exhibit azimuthally uniform speckle noise distribution (i.e. spider vane residual structures are observable) more favourable detectabilities than the one presented in Table 4 are locally evident. Therefore, by analysing the HPF images in favourable areas that exclude remnant spider vane patterns (Fig. 11, right) we can derive uppermost detectability levels reachable in each HPF image. Results are gathered in Fig. 10 and demonstrate an improvement of the detectability that is a function of the angular separation and the coronagraph. Basically, the
<figure><img src="image_11.png" /><figcaption>**Figure 9.** High-pass filtered detectability profiles (1σ). Left: Δλ/λ = 1.4 per cent. Right: Δλ/λ = 24 per cent.</figcaption></figure>

improvement evolves from a factor 2 to 4.5 (from 3 to 12λ/D), and
can nearly differ by a factor of 1.5 from a concept to another. Finally,
it brings the agreement of performance between all coronagraphs
even more closer than previously presented.

Additionally, because we obtained lower performance than the
one presented in Table 4 with the BLC when combined with its
pupil stop (Fig. 4, right), the results presented here correspond to
the BLC used with the LC pupil stop (less aggressive, Fig. 4, second
pattern from the left). In such situation, a contrast improvement by
a factor varying from 1.3 to 2.1 was obtained in the halo, while the
gain in throughput is about 18 per cent. This indicates that in real
situation the use of an aggressive pupil stop with a BLC can be
avoided even with a sophisticated entrance pupil (at least in a Strehl
ratio ≤90 per cent regime). This demonstrates the importance of a
well-balanced error budget when designing complex systems.

The chromaticity impact is found negligible in the AO control
domain, influencing contrast levels at minimum by a factor of below
2 to a maximum of 5. Nearly all the impact is found in the area
confined between the central core of the PSF and the IWA (e.g. the
peak rejection rate, see Table 4).

4.2.3 Planet PSFs

Four ghosts originating from reflections in the optical system before
the coronagraph (indeed upstream of the IR path of HOT, and poten-
tially taking origin from the protecting window of the deformable

mirror, or the beam splitter) are observable as bright PSFs in all
images. None of them is observable in the images recorded on the
coronagraphic testbench (Section 3). Fig. 11 (left) presents a close
view of the APLC HPF image where the four ghosts (quoted A,
B, C and D) are observable. These ghosts are virtually similar in
intensity in all coronagraphic images, although few differences are
measurable due to the fact that coronagraph pupil stops are not sim-
ilar and modify the width of the PSFs. In addition the non-constant
radial transmission of the BLC mask at large radii also introduce
some differences. Ghosts A, B, C and D are localized in the field at
0.4, 0.7, 0.5 and 0.9 arcsec, with peak intensity (1σ) of ~5 × 10⁻⁴,
~5 × 10⁻⁴, ~2 × 10⁻⁴ and ~3 × 10⁻⁵, respectively.

5 CONCLUSION

This paper studies the efficiency of several coronagraphic concepts
under realistic conditions and provides insights on their use in the
context of forthcoming planet finder instruments. We selected and
developed prototypes of various concepts (FQPM, LC, APLC and
BLC) and evaluated their performance under 0.5 arcsec dynamical
seeing corrected by an XAO system to a 90 per cent Strehl ratio.

The delivered coronagraphic performances have been analysed
at two levels of contrast:

(1) considering the residuals of the XAO system;

<figure><img src="image_12.png" /><figcaption>**Figure 10.** High-pass filtered detectability profiles (1σ) evaluated in area free of spider diffraction pattern and ghosts (see Fig. 11). Left: Δλ/λ = 1.4 per cent. Right: Δλ/λ = 24 per cent.</figcaption></figure>
<figure><img src="image_13.png" /><figcaption>**Figure 11.** Left: bottom left quarter of the APLC HPF image presented in Fig. 7, where the four ghosts in the image are identified (A, B, C and D). Right: areas free of the spider quasi-static pattern and ghosts considered in the HPF images for the evaluation of the detectability presented in Fig. 10.</figcaption></figure>

(2) when this residual halo has been removed by post-processing.

In this context, similar conclusions are derived for both contrast levels regardless of the coronagraphic concept.

(i) At Strehl ratio ~90 per cent all coronagraphs deliver similar contrast levels from IWA to the AO cut-off frequency and are therefore suitable for such XAO performance.

(ii) When the residual AO wavefront error is removed by post-processing, similar detectability is delivered by all coronagraphs.

(iii) As contrast levels are similar at larger angular separations than the IWA for all concepts, performance estimates for signal-to-noise ratio predictions for real observations (exposure time calculation) with multicoronagraph instruments (e.g. SPHERE) can in principle be achieved regardless of the coronagraphic concept.

(iv) Increasing the IWA of a coronagraph (e.g. LCs or BLC) does not provide better contrast, but increases the peak rejection. The peak rejection improvement varies from a factor of ~4 to ~30 when the IWA increases from a factor of 2 to 4, respectively (by comparing the FQPM to the LCs for instance). This aspect is important as wavefront errors downstream of the coronagraph produce quasi-static speckles that are proportional to the residual peak intensity.

(v) As small IWA is critical to probe the innermost region of astrophysical objects (basically only the FQPM offers the possibility to observe between 1 and $2\lambda/D$), efforts should be made to mitigate this level of residual energy in the centre of the image by using e.g. a FQPM combined with a small Lyot mask at its centre, or by implementing multi-FQPM designs (Baudoz et al. 2007).

(vi) At similar IWA the selection of a BLC over a LC should be restricted to situations where the XAO performance delivers better Strehl ratio than 90 per cent. This confirms the predictions from Crepp et al. (2007), where similar Strehl ratio regime was identified by means of simulations.

To take full advantage of a coronagraph the most demanding parameter is definitely the level of the wavefront control. The comparison of the coronagraphic contrast levels obtained on the two testbenches enables a differentiation for different levels of speckle noise, i.e. when the speckle limitation is set by the instrumental non-common path aberrations of HOT (coronagraphic testbench); and by either a combination of AO residual wavefront errors and instrumental common/non-common path aberrations (HOT, raw images), or common/non-common path aberrations only (HOT, HPF images). A straightforward distinction between coronagraphs occurs only on the coronagraphic testbench where the uppermost concepts

revealed are the APLC and BLC. This demonstrates the importance of a well-balanced error budget when designing complex systems. The conclusions of this experimental study support results from a previous analysis (Martinez et al. 2008) comparing several coronagraphic concepts (including the FQPM, LC, APLC and the BLC) in the context of ELTs by means of numerical simulations.

Speckle noise calibration/correction strategies are fundamental for direct detection of exoplanets with high-contrast instruments currently in use (Project 1640) or being commissioned (SPHERE, GPI or HiCIAO), and were not addressed in this study. These instruments are foreseen to deliver better contrast levels (e.g. $10^{-7}$ to $10^{-8}$) than the one presented in this paper. To tackle the speckle noise limitation several solutions have been proposed based on spectral characteristics (Marois et al. 2000), polarization states (Baba et al. 2003), image differential rotation (Marois et al. 2006) or coherence-based approaches (Codona et al. 2004; Baudoz et al. 2006; Galicher et al. 2008). The dynamic range can be improved by orders of magnitude (e.g. Marois et al. 2006; Hinkley et al. 2007; Oppenheimer et al. 2008; Thalmann et al. 2009; Crepp et al. 2011), while several authors have also proposed speckle subtraction methods through image post-processing (e.g. Sparks et al. 2002; Marois et al. 2006; Lafrenière et al. 2007). In addition, most of these instruments will take advantage of post-coronagraph wavefront calibration systems which take aim at reconstructing the wavefront error at the critical location of the coronagraph (e.g. Sauvage et al. 2007; Sivaramakrishnan et al. 2008; Wallace et al. 2010; Hinkley et al. 2011).

## ACKNOWLEDGMENTS

The activity outlined in this paper has been partially funded as part of the European Commission, Sixth Framework Programme (FP6), ELT Design Study, contract no. 011863, Seventh Framework Programme (FP7), Capacities Specific Programme, Research Infrastructures, contract no. INFRA-2007-2.2.1.28, and Opticon joint research activity (JRA) 1, contract no. RII3-CT-2004-001566.

## REFERENCES

* Abe L., Domiciano de Souza A., Jr, Vakili F., Gay J., 2003, A&A 400, 385

* Aime, Soummer, Ferrari, 2002, A&A, 389, 334

* Aller Carpentier E. et al., 2008, Proc. SPIE, 7015

* Baba N., Murakami N., 2003, PASP, 115, 1363

* Baudoz P. et al., 2000, A&AS, 145, 341

* Baudoz P., Boccaletti A., Baudrand J., Rouan D., 2006, in Aime C., Vakili F., eds, Proc. IAU Colloq. Vol. 200, Direct Imaging of Exoplanets: Science & Techniques. Cambridge Univ. Press, Cambridge, p. 553

* Baudoz P., Boccaletti A., Rouan D., 2007, in Kalas P., ed., Proc. of the Conference In the Spirit of Bernard Lyot: The Direct Detection of Planets and Circumstellar Disks in the 21st Century. June 04–08, 2007. Univ. California, Berkeley

* Beuzit J.-L. et al., 2008, in McLean I. S., Casali M. M., eds, Proc. SPIE Vol. 7014, Ground-based and Airborne Instrumentation for Astronomy II. SPIE, Bellingham, p. 701418

* Boccaletti A., Riaud P., Baudoz P., Baudrand J., Rouan D., Gratadour D., Lacombe F., Lagrange A.-M., 2004a, PASP, 116, 1061

* Boccaletti A., 2004b, in Aime C., Soummer R., eds, Astronomy with High Contrast Imaging. EAS Publications Series, Vol. 12. EDP Sciences, Les Ulis, p. 165

* Boccaletti A., Chauvin G., Baudoz P., Beuzit J.-L., 2008, A&A, 482, 939

* Codona J. L., Angel R., 2004, ApJ, 604, L117

* Crepp J. R., Ge J., Vanden Heuvel A. D., Miller S. P., Kuchner M. J., 2006, ApJ, 646, 1252

* Crepp J. R., Vanden Heuvel A. D., 2007, ApJ, 661, 1323

* Crepp et al., 2011, ApJ, 729, 132
Enya K., Tanaka S., Abe L., Nakagawa T., 2007, A&A, 461, 783

Enya K., Abe L., Tanaka S., Nakagawa T., Haze K., Sato T., Wakayama T., 2008, A&A, 480, 899

Galicher R., Baudoz P., Rousset G., 2008, A&A, 488, L9

Gratadour D., Rouan D., Boccaletti Riaud P., Clénet Y., 2005, A&A, 429, 433

Guyon O., Pluzhnik E. A., Kuchner M. J., Collins B., Ridgway S. T., 2006, ApJS, 167, 81

Guyon O., Pluzhnik E., Martinache F., Totems J., Tanaka S., Matsuo T., Blain C., Belikov R., 2010, PASP, 122, 71

Hinkley S. et al., 2007, ApJ, 654, 633

Hinkley S. et al., 2011, PASP, 123, 74

Hodapp K. W. et al., 2008, in McLean I. S., Casali M. M., eds, Proc. SPIE Vol. 7014, Ground-based and Airborne Instrumentation for Astronomy II. SPIE, Bellingham, p. 701419

Kasper M. et al., 2010, in McLean I. S., Ramsay S. K., Takami H., eds, Proc. SPIE Vol. 7735, Ground-based and Airborne Instrumentation for Astronomy III. SPIE, Bellingham, p. 77352E

Kuchner M. J., Traub W., 2002, ApJ, 570, 900

Lafrenière D., Doyon R., Nadeau D., Artigau E., Marois C., Beaulieu M., 2007, ApJ, 661, 1208

Lyot B., 1939, MNRAS, 99, 580

Macintosh B. A. et al., 2008, in Hubin N., Max C. E., Wizinowich P. L., eds, Proc. SPIE Vol. 7015, The Gemini Planet Imager: from science to design to construction. SPIE, Bellingham, p. 701518

Marois C., Doyon R., Racine R., Nadeau D., 2000, PASP, 112, 91

Marois C., Lafrenière D., Doyon R., Macintosh B., Nadeau D., 2006, ApJ, 641, 556

Marois C., Lafrenière D., Macintosh B., Doyon R., 2008, ApJ, 673, 647

Martinez P., Boccaletti A., Kasper M., Baudoz P., Cavarroc C., 2007, A&A, 474, 671

Martinez P., Boccaletti A., Kasper M., Cavarroc C., Yaitskova N., Fusco T., Vêrinaud C., 2008, A&A, 492, 289

Martinez P., Dorrer C., Aller Carpentier E., Kasper M., Boccaletti A., Dohlen K., Yaitskova N., 2009a, A&A, 495, 363

Martinez P., Dorrer C., Kasper M., Boccaletti A., Dohlen K., 2009b, A&A, 500, 1281

Martinez P., Dorrer C., Kasper M., 2009c, ApJ, 705, 1637

Martinez P., Aller Carpentier E., Kasper M., 2010a, PASP, 122, 916

Martinez P., Dorrer C., Kasper M., Boccaletti A., Dohlen K., 2010b, A&A, 520, A110

Mawet D., Riaud P., Absil O., Surdej J., 2005, ApJ, 633, 1191

Mawet D., Riaud P., Baudrand J., Baudoz P., Boccaletti A., Dupuis O., Rouan D., 2006, A&A, 448, 801

Mawet D., Serabyn E., Liewer K., Hanot C., McEldowney S., Shemo D., O'Brien N., 2009, Opt. Express, 17, 1902

Mawet D., Serabyn E., Liewer K., Buruss R., Hickey J., Shemo D., 2010, ApJ, 709, 53

N'diaye M., Dohlen K., Cuevas S., Lanzoni P., Chemia F., Chaumont C., Soummer R., Griffiths E. T., 2010, A&A, 509, A8

Oppenheimer B. R. et al., 2004, in Calia D. B., Ellerbroek B. L., Ragazzoni R., eds, Proc. SPIE Vol. 5490, Advancements in Adaptive Optics. SPIE, Bellingham, p. 433

Oppenheimer B. R. et al., 2008, ApJ, 679, 1574

Pinna E., Puglisi A. T., Quiros-Pacheco F., Busoni L., Tozzi A., Esposito S., Aller Carpentier E., Kasper M., 2008, in Hubin N., Max C. E., Wizinowich P. L., eds, Proc. SPIE Vol. 7015, The Gemini Planet Imager: from science to design to construction. SPIE, Bellingham, p. 701559

Riaud P., Boccaletti A., Rouan D., Lemarquis F., Labeyrie A., 2001, PASP, 113, 1145

Riaud P., Boccaletti A., Baudrand J., Rouan D., 2003, PASP, 115, 712

Riaud P., Boccaletti A., Baudrand J., Rouan D., 2006, A&A, 458, 317

Rouan P., Mawet D., Absil O., Boccaletti A., Baudoz P., Herwats E., Surdej J., 2000, PASP, 112, 1479

Sauvage D., Riaud P., Boccaletti A., Clénet Y., Labeyrie A., 2007, J. Opt. Soc. America, 24, 2334

Sivaramakrishnan J.-F., Fusco T., Rousset G., Petit C., 2008, ApJ, 688, 701

Soummer R., 2005, ApJ, 618, L161

Soummer A., Soummer R., Pueyo L., Wallace J. K., Shao M., 2003, A&A, 397, 1161

Soummer R., Aime C., Falloon P.-E., 2009, ApJ, 695, 695

Soummer R., Pueyo L., Ferrari A., Aime C., Sivaramakrishnan A., Yaitskova N. R., Sivaramakrishnan A., Pueyo L., Macintosh B., Oppenheimer B. R., 2011, ApJ, 729, 144

Sparks W. B., Ford H. C., 2002, ApJ, 578, 543

Swartzlander G. A., Jr, Ford E. L., Abdul-Malik R. S., Close L. M., Peters M. A., Palacios D. M., Wilson D. W., 2008, Opt. Express, 16, 10200

Thalmann C. et al., 2009, ApJ, 707, L123

Vernet E. et al., 2006, in Ellerbroek B. L., Calia D. B., eds, Proc. SPIE, Vol. 6272, Advancements in Adaptive Optics II. SPIE, Bellingham, p. 62722K

Wallace J. K., Burruss R. S., Bartos R. D., Trinh T. Q., Pueyo L. A., Fregoso S. F., Angione J. R., Shelton J. C., 2010, in Ellerbroek B. L., Hart M., Hubin N., Wizivovich P. L., eds, Proc. SPIE Vol. 6272, Advancements in Adaptive Optics II. SPIE, Bellingham, p. 77365D

This paper has been typeset from a *TEX/LATEX* file prepared by the author.