# I2C@2024: An interferometry imaging contest in 2024

Florentin Millour, Mathis Houllé, Jeremy Perdigon, Julien Drevon, Ryan

Norris, Rebecca Proni, Alexis Matter, Ferréol Soulez, Eric Thiebaut

► To cite this version:

Florentin Millour, Mathis Houllé, Jeremy Perdigon, Julien Drevon, Ryan Norris, et al., I2C@2024: An interferometry imaging contest in 2024. Optical and Infrared Interferometry and Imaging IX, Jun 2024, Yokohama, France. pp.39, 10.1117/12.3019243. [hal-04932243]

HAL Id: hal-04932243

https://hal.science/hal-04932243v1

Submitted on 6 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
I2C@2024: An interferometry imaging contest in 2024

Florentin Millour<sup>a</sup>, Mathis Houllé<sup>a</sup>, Jérémy Perdigon<sup>a</sup>, Julien Drevon<sup>b</sup>, Ryan Norris<sup>c</sup>, Rebecca Proni<sup>c</sup>, Alexis Matter<sup>a</sup>, Ferreol Soulez<sup>d</sup>, and Eric Thiebaut<sup>d</sup>

<sup>a</sup>Laboratoire Lagrange, UniCA, CNRS, OCA, Nice, France

<sup>b</sup>European Southern Observatory, Alonso de Córdova, 3107 Vitacura, Santiago, Chile

<sup>c</sup>Department of Physics, New Mexico Institute of Mining and Technology, 801 Leroy Place, Socorro, NM 87801, USA

<sup>d</sup>Univ Lyon 1, ENS de Lyon, CNRS, Centre de Recherche Astrophysique de Lyon,, UMR5574, F-69230, Saint-Genis-Laval, France

ABSTRACT

Images in optical long-baseline interferometry have seen a boost in the recent years thanks to new techniques and recipes invented by the community. These images are more and more used for science interpretation and not only illustration, and their fidelity has improved significantly, thanks mainly to the increase in the number of telescopes used in interferometers. The focus today is to improve their reliability and dynamic range. With this contest, we follow up the quest introduced in 2004 of comparing the state of the art image reconstruction software for long-baseline interferometry. This is done in a festive way in the form of an imaging contest, where the organizers propose simulated datasets of targets, whose brightness distributions are meant to be blindly retrieved using various algorithms by the contestants. A prize is offered to the winner of the contest. This year is not different from previous ones and we proposed to the contestants tools to compare their reconstructed images with original images. These tools are now distributed, together with example datasets and images, enabling further tests at home of any image reconstruction tool.

**Keywords:** Images, contest, interferometry, spiral, disk, exoplanets

1. INTRODUCTION

Long baseline interferometry contests have been organized since 2004<sup>1-9</sup> and have enabled a comparison of algorithms' performances along their many editions. A total of 20 algorithms were used in the 18 years of previous contests. We provide in Fig. 1 a digest of the algorithms running for the imaging contest, showing the prominent ones as well as the ones that may deserve a special attention. Among the winner algorithms (marked with an asterisk), we can find Maximum Entropy Methods, implemented in BSMEM,<sup>10</sup> Bayesian approaches to image reconstruction, like in MIRA,<sup>11</sup> or IRBIS,<sup>12</sup> and statistical methods like the one implemented in MACIM.<sup>5</sup> Alternate methods include attempts for 3D image reconstructions (including an implementation of self-cal in use with MIRA<sup>13</sup>), mix of model-fitting and image reconstruction, with a special mention to SPARCO<sup>14</sup> and RHAPSODY,<sup>15</sup> as well as machine learning-based algorithms.<sup>16</sup>

In the 2024 contest, we wished the contestants to address wavelength-dependent datasets on objects representative of typical imaging cases, namely a spiral nebula, that can be found in e.g. interacting binaries, and a multi-gapped disk, with planets embedded inside of different fluxes and spectral shape.

2. THE CONTEST DATA

To produce the contest data, we tried as much as possible to use the JMMC tools to simulate objects and datasets.

Further author information: (Send correspondence to F. Millour)
F. Millour: E-mail: fmillour@oca.eu, Telephone: +33 489 15 03 59
<table><caption>Figure 1. All the participating algorithms used in previous interferometry imaging contests. Winning algorithms are marked with an asterisk, and we tried as much as possible to group algorithms by family.</caption><thead><tr><th>Year</th><th>Maximum Entropy</th><th>CLEAN-like</th><th>Bayesian</th><th>*3D"</th><th>MCMC</th><th>Model-based</th><th>Al-based</th></tr></thead><tbody><tr><td>2004</td><td>BSMEM*</td><td>VLBMEM</td><td>MIRA</td><td>WISARD</td><td></td><td></td><td></td></tr><tr><td>2006</td><td>BSMEM*</td><td>BBM</td><td>RPR</td><td>MIRA</td><td></td><td>MACIM</td><td></td></tr><tr><td>2008</td><td>BSMEM</td><td>BBM</td><td>RPR</td><td>MIRA*</td><td></td><td></td><td></td></tr><tr><td>2010</td><td>BSMEM*</td><td></td><td>RPR</td><td>WISARD</td><td></td><td>SQUEEZE</td><td></td></tr><tr><td>2012</td><td>BSMEM</td><td>CASA</td><td>RPR?</td><td>MIRA IRS</td><td>MIROIRS</td><td>MACIM*</td><td></td></tr><tr><td>2014</td><td>BSMEM*</td><td>Pearl/CLEAN</td><td>MIRA IRBis</td><td>WISARD</td><td>MIRA3D</td><td>MACIM SQUEEZE SPARCO</td><td></td></tr><tr><td>2016</td><td>BSMEM</td><td></td><td>IRBis*</td><td></td><td>PAINTER</td><td>Self-Cal</td><td>Chips Imaging LITpro</td></tr><tr><td>2018</td><td>BSMEM*</td><td></td><td></td><td></td><td></td><td>SQUEEZE SPARCO</td><td></td></tr><tr><td>2022</td><td>BSMEM*</td><td></td><td>MIRA</td><td></td><td></td><td>SPARCO</td><td>RHAPSODY ORGANIC NEBIRA</td></tr></tbody></table>

<table><caption>Table 1. Spiral model parameters as put into the AMHRA service.</caption><tr><td>Type of spiral</td><td>rings</td><td>Dust temperature</td><td>1500 K</td></tr><tr><td>Filling factor</td><td>20</td><td>q</td><td>0.3</td></tr><tr><td>Rounds</td><td>3</td><td>Gap factor</td><td>1.5</td></tr><tr><td>Opening angle</td><td>40°</td><td>WR temperature</td><td>70,000 K</td></tr><tr><td>Distance to first turn (size)</td><td>10 mas</td><td>OB temperature</td><td>40,000 K</td></tr><tr><td>Inclination</td><td>24°</td><td>Luminosity ratio of stars</td><td>3</td></tr><tr><td>Rotation in spiral plane</td><td>75°</td><td>Fraction of flux from the binary</td><td>5</td></tr><tr><td>Distance to shocked material (hollow middle)</td><td>4 mas</td><td>Scale factor for pinwheel (x flux)</td><td>10<sup>-16</sup></td></tr><tr><td>Binary separation</td><td>1 mas</td><td></td><td></td></tr></table>

## 2.1 Spiral model

We used the AMHRA* model of a spiral (derived from Millour et al. 2008¹⁷) to produce an image cube in the H, K, L and N bands. The size was selected so that the first spiral turn is about 10 mas in size, and has a total of 3 turns. All the parameters of the spiral are shown in Table 1, and the corresponding model images, convolved with the beam of the interferometer is shown in Figure 2.

The AMHRA services allowed us to generate the spiral model as a cube of images for each set of wavelengths of interest. We fed these image cubes to ASPRO to get OIFITS files for each instrument, producing a realistic VLTI (u,v) coverage. We checked the "Compute OIFITS data", "Add error noise to the data", as well as the "Use instrumental and calibration error biases" to produce the OIFITS files of the contest. We verified using the Oimaging tool†, especially using the MIRA¹¹ and SPARCO¹⁸ implementations, that we could indeed reconstruct images that visually looked like the input models from the contest OIFITS files.

## 2.2 Disk with planets model

We simulated a disk of dust around a young A-type star ($T_{\star} = 10\,000$ K, $R_{\star} = 2.2\,R_{\odot}$, $M_{\star} = 2.8\,M_{\odot}$), using the ray tracing code from Perdigon et al. (in prep), to which we superimposed point sources representing planets. We aimed to produce a model that is similar to the circumstellar environment around a young Herbig Ae/Be star.¹⁹

The ray tracing code computes synthetic images from the integration along rays of the circumstellar source function. In order to achieve the integration, the code requires the optical properties of the medium, namely the source function and the extinction coefficient, at each point in the circumstellar envelope. In the following, we describe the properties of the dust we used to compute the synthetic images of the disc. We then explain how we consequently injected the planets in the image.

<footnote>*https://amhra.oca.eu/AMHRA/index.htm</footnote>

<footnote>†https://www.jmmc.fr/english/tools/data-analysis/oimaging/</footnote>
<figure><img src="image_2.png" /><figcaption>Figure 2. Spiral model at selected wavelengths, showing the main structures we expect to see reconstructed by the contestants. Intensity scale is normalized to its maximum and displayed as the cubic root of the image intensity to make faint structures more visible.</figcaption></figure>

The disk is made of four concentric rings separated by three gaps (Fig. 3-Right). The smallest dust inner rim is set to $R_{in} = 1$ A.U while the largest outer radius is $R_{out} = 150$ A.U. The disc is situated at a distance $d = 184$ pc. These values insure all components remain resolved by the VLTI instruments. The first gap ranges 2 to 3 A.U, the second ones spans from 7 to 11 A.U and the last one extends from 30 to 40 A.U. The object is seen at an inclination of $i = 70$ deg with respect to the pole.

The dust extinction coefficient $\kappa_{\nu}^{\text{ext}}$ is given by

$$ \kappa_{\nu}^{\text{ext}} = C_{\nu}^{\text{ext}} \rho \quad (1) $$

where $C_{\nu}^{\text{ext}}$ are the dust extinction cross sections and $\rho$ is the dust density. The composition of the dust varies from one ring to another. We included this feature to reflect the decrease in dust crystallinity with increasing radius, due to associated changes in temperature, pressure and radiation. The composition of the four rings are (from the inner radius to outer radius): (i) 100 % forsterite, (ii) 50 % forsterite, 50 % olivine, (iii) 50 % olivine, 50 % ISM (silicate and graphite) and (iv) 100 % ISM. The extinction cross-sections, for each dust species, are shown in Fig. 3-Left.

The density law is given by that of a disk hydrostatically supported in the vertical direction$^{20}$

$$ \rho = \rho_{\text{in}} \left( \frac{\omega}{R_{\text{in}}} \right)^{-5/2} \exp \left\{ -\frac{1}{2} \left( \frac{z}{H} \right)^2 \right\} \quad (2) $$

where $\omega$ is the equatorial radius, in the disk mid-plane and $z$ is the height with respect to the equatorial plane. $\rho_{\text{in}}$ is the dust density at the base of the disk and is arbitrarily set to $\rho_{\text{in}} = 10^{-7}$ kg.m$^{-3}$. This value insures that the disk remains optically-thick to the observer. Finally, $H$ is the scale height of the disk and is set to be a power-law

$$ H = H_0 \left( \frac{\omega}{R_*} \right)^{13/10} \quad (3) $$

with $H_0 = (k_B T_* R_*^3 / (m_{H_2} G M_\star))^{1/2}$. The exponent 13/10 insures the disk does not flare too much so that the inner parts (such as the star and the inner radius) are not obscured by the outer rings.

For the source function $S_{\nu}$, we omit scattering and assume the dust emits like a black-body spectrum $S_{\nu} = B_{\nu}(T)$. The dust temperature is also chosen to be a power law, on the form

$$ T = T_{\text{in}} \left( \frac{\omega}{R_{\text{in}}} \right)^{-1/2} . \quad (4) $$
<figure><img src="image_3.png" /><figcaption>Figure 3. **Left:** Extinction cross sections for the three different dust species used in the disk model. **Right:** Density profile in the disk equatorial plane, colored by dust composition.</figcaption></figure>

We set the inner dust temperature $T_{in}$ = 1500 K to be consistent with the usual dust evaporation temperature of silicate-based dust species. The temperature radial exponent −1/2 is that of a grey optically-tick and spatially flat disk.²¹

For the substellar companions, we injected point sources with spectra extracted from the ExoREM model grid.²² The atmospheric parameters and the relative astrometry of each companion are reported in Table 2. Before injection, the spectra are convolved at the spectral resolutions of the instruments by a Gaussian filter*, and interpolated on the instruments' wavelength grids†.

<figure><img src="image_4.png" /><figcaption>Figure 4. Disk model at selected wavelengths. The arrows mark the position of the three planets to be recovered. Notice the large scale of the image. The inset shows a zoom into the central region. Intensity scale is the cubic root of the image intensity to make faint structures more visible.</figcaption></figure>

<footnote>*implemented by the `scipy.ndimage.gaussian_filter` function</footnote>

<footnote>†with `numpy.interp`</footnote>
<figure><img src="image_5.png" /><figcaption>Figure 5. Disk model spectrum (top) as well as injected planets spectra (bottom).</figcaption></figure>

<table><caption>Table 2. Injected substellar companions parameters.</caption><thead><tr><th>Companion</th><th>Pos. X [mas]</th><th>Pos. Y [mas]</th><th>T<sub>eff</sub> [K]</th><th>log g</th><th>[M/H]</th><th>C/O</th><th>R [R<sub>Jup</sub>]</th></tr></thead><tbody><tr><td>Planet 1</td><td>-6.5</td><td>-7.5</td><td>1500</td><td>3.5</td><td>1.0</td><td>0.55</td><td>2.0</td></tr><tr><td>Planet 2</td><td>20.5</td><td>-0.5</td><td>1000</td><td>4.5</td><td>0.32</td><td>0.35</td><td>2.0</td></tr><tr><td>Brown dwarf</td><td>101.5</td><td>171.5</td><td>2000</td><td>5.0</td><td>1.0</td><td>0.55</td><td>2.0</td></tr></tbody></table>

In the same way as for the spiral model, we fed the model image cubes to ASPRO to get OIFITS files for each instrument, producing a realistic VLTI (u,v) coverage. We checked the "Compute OIFITS data", "Add error noise to the data", as well as the "Use instrumental and calibration error biases" to produce the OIFITS files of the contest. We verified using the Oimaging tool that we could indeed reconstruct images that visually looked like the input models from the contest OIFITS files.

The resulting dataset is one OIFITS file per instrument and infrared band. The datasets are chromatic, and have typical spectral resolutions, observables, noises and biases produced by each instrument. We posted the datasets on the OiDB database to make it available to anyone willing to participate to the contest this year*, set up a contest page providing general information about the contest†, and we provided and distributed largely a set of instructions‡ that we detail in the next section

## 3. INSTRUCTIONS TO THE CONTESTANTS

On the general webpage presenting the contest, we provided some context to the targets procided in the datasets: "we have two targets observed with all the VLTI instruments (PIONIER, GRAVITY, and the two L- and N-band arms of MATISSE). – The first one is a hot star with an environment, – The second one is a young star where there is a suspicion of a companion. One of them is an easy target, so we will expect to see details of

<footnote>*http://oidb.jmmc.fr/collection.html?id=6df579f8-752b-4536-b579-f8752bd536d0</footnote>

<footnote>†http://fmillour.com/index.php/2024/03/25/2024-interferometry-imaging-contest/</footnote>

<footnote>‡https://docs.google.com/document/d/1F6mkPixTRgC6VTxyzoGaWKPQaFroEqFRY79NnWsLaFc/edit?usp=sharing</footnote>
the structures in the images. The other one on the contrary is a rather difficult target, so we expect to see the
prominent features and structures in it.”

We publicised the calendar of the contest in the distributed set of instructions in the following: 1st of March 2024: contest announcement, take contact with interested teams; mid-March, datasets distribution; May 31: the teams send their results to the organizers, comparison of results with input images, start of proceeding writing. At about the same time or earlier: distribution of the codes to establish the score; presentation of results at SPIE in June.

The contestants obtained oifits files for several objects that were produced using simulations of observations on the Very Large Telescope Interferometer, with all available instruments (i.e. one oifits file per instrument / band). The datasets were chromatic, and had typical spectral resolutions, observables, noises and biases produced by each instrument.

We expected from the contestants image cubes that contained 3D images: 2D images plus one additional wavelength dimension. Images scales were supposed to be provided using the CRPIX1, CRVAL1, CDELT1, CUNIT1, CRPIX2, CRVAL2, CDELT2, CUNIT2 (spatial scales) and CRPIX3, CRVAL3, CDELT3, CUNIT3 (wavelength scale)*.

We also provided general rules as: if no image was provided by the contestant for a given wavelength/band, it would have been replaced by a uniform image. We warned the contestants that such a procedure would have heavily penalized the score, this was to strongly encourage contestants to provide images at all wavelengths.

We finally provided a basic description of the metrics that would be used for the image comparison, and
provided some FAQ treating the questions from the contestants that arrived along the way.

4. CONTEST ENTRIES

We contacted the whole interferometry community for the contest through the OLBIN† mailing list and obtained 15 answers from various teams, that expressed interest in the contest. Out of these, we received two entries that both contained chromatic images of the objects of interest. We present them here and after we compare their entries with the input images.

The few contributions this year may be a symptom of a fading interest of the experts in the development of new imaging algorithms, or the fact that this year's contest was focused on producing realistic instrument datasets, including calibration biases, that are indeed usually much harder to tackle than simpler simulated data that can be found in many former imaging contests. Another possibility is that simply most of the available image reconstruction tools (including the most popular ones) are still not ready for real interferometry datasets (heavily multispectral, with tons of systematics), a situation that calls for new developments and adaptations of algorithms.

Nevertheless, the two entries received were of very high quality and both neatly managed to reproduce the main structures of the input objects, meaning that future contests may focus more on dynamic range exploration, as we tried to experiment for the second object.

As a side note, none of the participants followed closely the provided guidelines for images submission, especially for the wavelength scale, meaning that a more thorough explanation together with an example script may be provided for future editions of the contest. For this contest, we adapted the wavelengths scales for one participant, and asked one of the participants to submit a new contribution with a sorted out wavelength scale.

4.1 J. Drevon entry using Mira

J. Drevon sent a set of input images cubes for both objects, one input file for each band. Figures 6 and 7 show on the top one single wavelength image for each band, while the bottom set of images shows the median of all wavelengths, that has a neat tendency to wipe out single-wavelength non-stationary artifacts in the images.

These entry images were interpreted by J. Drevon in the following way:

<footnote>*see here for instructions how to set it up: https://github.com/JMMC-OpenDev/aspro-doc/blob/main/aspro-doc.
md#user-defined-model</footnote>

<footnote>†olbin@univ-grenoble-alpes.fr</footnote>
4.1.1 Spiral

<figure><img src="image_6.png" /><figcaption>Figure 6. Spiral entry from J. Drevon. **Top:** images at one selected wavelength. **Bottom:** the median image of all entry wavelengths. Image scale is the cubic root of the intensity to make faint structures more visible.</figcaption></figure>

**PIONIER H-band:** The image reconstructed with MiRA using the data from the PIONIER instrument, reveal a beginning of spiral formation close from the central source. This spiral is induced by a companion that reshape the circumstellar environment. The shape of the spiral arms seems to be wavelength dependent by comparison with the data from the following instruments.

**GRAVITY K-band:** The image reconstructions done using the data from GRAVITY reveal and confirm the previous image reconstruction, we observe an elongated structure with a beginning of a deformation in the circumstellar environment. We should take care about possible artifacts in the image reconstruction especially far from the central source.

**MATISSE L-band:** This time the image reconstruction seems to reconstruct centro-symmetric spiral arms. It is hard to say if the central source is resolved or if it is the product of artifacts in the image reconstruction process. Looking at the different images reconstructed for each spectral channel, we mainly resolve what might be a central source with a spiral arm. The spiral arm seem to have once again strong non-homogeneity in the spiral arms.
**MATISSE N-band:** For the N-band data of the VLTI/MATISSE instrument, it is hard to resolve more than a close spherical-like Gaussian shape.

### 4.1.2 Disk

<figure><img src="image_7.png" /><figcaption>Figure 7. Disk entry from J. Drevon. **Top:** a selected wavelength in the image cube. **Bottom:** the median of all entry wavelengths. Image scale is the cubic root of the intensity to make structures more visible.</figcaption></figure>

**PIONIER H-band:** Image reconstruction procedure using the VLTI/PIONIER data in H-band only show an elongated structure filed by artifacts with a resolved central source.

**GRAVITY:** The image reconstruction procedure using the VLTI/GRAVITY data does not allow us to describe more the geometry of the object than what have been done using PIONIER. We only detect an elongated structure filled by artifacts.

**MATISSE L-band:** With MATISSE we clearly observe a central source with very small elongation. It is not clear if the faint arms present around the central source are real or not. To be conservative on the conclusion, I decided to rank those structures as mostly artifacts.

**MATISSE N-band:** In N-band it was hard to reconstruct good images, however we can observe an elongated central source with some deviation from centro-symmetry. The central source seems to be embedded inside a thermal background. As previously mentioned in LM-band, it is hard to tell if the structures and the overdensity close to the central source in the thermal background are rather artifacts or real structures.
## 4.2 R. Norris using SQUEEZE + OITOOLS.jl

R. Norris used a combination of SQUEEZE and OITOOLS.jl to reconstruct wavelength-dependent images of the contest objects. He sent as well image cubes, one for each wavelength. He provided the following description prior to the evaluation process:

### 4.2.1 Spiral

<figure><img src="image_8.png" /><figcaption>Figure 8. Spiral entry from R. Norris. Top is a selected wavelength, bottom is the median of all entry wavelengths. Image scale is the cubic root of the intensity to make structures more visible.</figcaption></figure>

Object 1 has the appearance of a Wolf Rayet system similar to WR104. The winds between the two stars in the system collide, compressing gas in a shock that cools to become dust. As seen in the PIONIER and MATISSE data, this forms a pinwheel due to binary motion. In the GRAVITY image we see evidence of successive dust shells from past wind collision events. This also appears within the longer wavelength MATISSE images, which have a prominent feature from a denser region, similar to what was recently observed by JWST in WR104.

### 4.2.2 Disk

Object 2 is a region around a young star with a possible planet. In the PIONIER data, we see the central region as well as some denser areas in the outer disk. The image from the GRAVITY data shows some more complex structures within the disk, though this is visible using a square root intensity scale in the image. The lower wavelength MATISSE image has a spiral appearance, which has been interpreted as evidence of a planet in some systems. The longer wavelength MATISSE image has the central bright central region once more and a feature to the NW.
<figure><img src="image_9.png" /><figcaption>Figure 9. Disk entry from R. Norris. Top is a single selected wavelength, bottom is the median of all entry wavelengths. Image scale is the cubic root of the intensity to make structures more visible.</figcaption></figure>

## 5. HOW DID WE COMPUTE THE CONTEST SCORES

Image comparison is a sensitive topic, especially for interferometric image reconstruction! To provide a fair comparison of images, it is important that the computed score does not depend on irrelevant changes. For instance, the image metric should be insensitive to some geometrical transform such as translation or to scaling: a multiplication of the brightness by some positive factor which does not affect the shape of the object. In addition, as we did not gave to the contestant any guidance about the size of the pixels in the reconstruction, the effective resolution achievable by the instrument must be taken into account when comparing a true image $z$ (with potentially an infinitely high resolution) to a restored image $x$. Otherwise, on pixel-wise comparisons, the slightest displacement of sharp features would lead to large loss of quality (according to the metric) whereas the images may look very similar at a lower and more realistic resolution.

This problem of assessing the quality of restored images has already been extensively studied in Gomez et al.²³ that based the evaluation of the best merit function on experts’ poll and comparison of methods. Following its prescription, we use the L1 norm to compare the images as it seems to be the one that best match both the global minimum relative to the input image, and the human assessment. This L1 norm is normalized by the reference image total flux. The resulting score between two images $x$ and $y$ can be described as follows:

$$L(x, y) = \frac{\sum_{k \in \text{pixels}} |x_k - y_k|}{\sum_{k \in \text{pixels}} |y_k|} \qquad (5)$$

where the reference image $y$ is the true image $z$ blurred to a prescribed spatial resolution. To keep this metric insensitive to translation and scaling, it is minimized by shifting the input image by $\delta = (\delta_x, \delta_y)$ and scaling its
flux by $\alpha$ for each wavelength. The overall multispectral score then writes:

$$S(\boldsymbol{x}) = 1 - \frac{1}{N_{\lambda}} \sum_{\lambda} \min_{\alpha, \delta} L(\alpha \mathbf{T}_{\delta} \cdot \boldsymbol{x}_{\lambda}, \boldsymbol{y}_{\lambda}), \quad (6)$$

where $\mathbf{T}_{\delta}$ is an operator translating the image by $\delta = (\delta_x, \delta_y)$ and $N_{\lambda}$ is the number of spectral channels in $\boldsymbol{x}_{\lambda}$ and $\boldsymbol{y}_{\lambda}$, the reconstructed and reference images in spectral channel $\lambda$. Since there are no closed-form expression of the best scaling factors $\alpha$, and best translation $\delta$ the score is numerically optimized using a bisection method.

In addition, the contestant submission does not necessarily match the model image in terms of field of view, pixel scale and wavelength scale. To alleviate these issues, we do the following assumptions and prepossessings:

* during translation, we assumed the pixels outside of the field of view to be equal to zero,

* we generate the model image with a high spatial resolution and spatially resample the contestant image using bilinear interpolation to the resolution of the model image,

* similarly we resample the image spectrally to match the wavelength scale of both the model and the contestant's ones.

We produced scripts to implement this score computation, both in Julia and python, with examples snippets*. This allows anyone to run them on their side to test their own datasets as well as the 2016 interferometry imaging contest datasets.

## 6. RESULTS

<figure><img src="image_10.png" /><figcaption>Figure 10. Score calculated with the recipe described above for the J. Drevon entries. The first 4 panels are for the spiral object, while the 4 later panels are for the disk object.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure 11. Score calculated with the recipe described above for the R. Norris entries. The first 4 panels are for the spiral object, while the 4 later panels are for the disk object.</figcaption></figure>

All images of the input model and the entry images shown in figures 2, 4, 6, 7, 8, and 9, are convolved to $\frac{\lambda}{4B_{\max}}$ and are plot at the same scale, easing a visual comparison.

Wavelength-histograms of scores were computed for all participants and each image (see Figures 10 and 11). They all show peaked distributions, making us confident that the mean scores are indeed comparing relevant

<footnote>*https://github.com/JMMC-OpenDev/ImageMetrics</footnote>
<table><thead><tr><th>Participant</th><th>Spiral H</th><th>Spiral K</th><th>Spiral L</th><th>Spiral N</th><th>Disk H</th><th>Disk K</th><th>Disk L</th><th>Disk N</th></tr></thead><tbody><tr><td>J. Drevon</td><td>0.14</td><td>0.13</td><td>0.27</td><td>0.78</td><td>0.65</td><td>0.52</td><td>0.31</td><td>0.38</td></tr><tr><td>R. Norris</td><td>0.15</td><td>0.09</td><td>0.65</td><td>0.84</td><td>0.27</td><td>0.52</td><td>0.22</td><td>0.24</td></tr></tbody></table>

<table><caption>Table 3. Computed scores as in eq. 5</caption><thead><tr><th>Participant</th><th>Spiral score</th><th>Disk score</th><th>Total score</th></tr></thead><tbody><tr><td>J. Drevon</td><td>1.19</td><td>1.86</td><td>3.05</td></tr><tr><td>R. Norris</td><td>1.64</td><td>1.25</td><td>2.89</td></tr></tbody></table>

features in the (different for each wavelength) images, and not only tracing wavelengths-by-wavelength shifts or wavelength-dependent flux variations.

We compile the mean scores for each band here in Table 3. The total score is just the sum of all scores, except the K-band spiral one, where both participants provided images that clearly do not look like the input image (keeping the K-band spiral would not change the final result).

As a consequence, the contest organizers award the 2024 imaging contest prize to Julien Drevon for his nice images of both the disk and the spiral. We also wanted to make a special mention to Ryan Norris, that provided neat images of both objects and had a score close-enough to the best score to have it mentioned.

## 7. CONCLUSION

This year's contest showed nice entries with chromatic contributions that both showed very interesting images of the model images. We introduced example scripts and used a method to compare the images irrelevant of the input data, a method that provides a better way to evaluate whether the images match the input ones, thus not relying on the underlying datasets noises or particularities.

The small participation this year may be a sign that imaging softwares are not yet ready to handle real datasets, i.e. ones that contain large systematic errors (not necessarily accounted for in the ofits files) and massively multispectral information.

We may for the next editions of the imaging contest to change the agenda so that participants may have more time to produce their entries. Another focus may be on high-dynamic range imaging as was attempted this year for the disk model.

## ACKNOWLEDGMENTS

We warmly thank the SPIE interferometry session organizers, S. Sallum and J. Sanchez-Bermudez, for their support in the organization of the imaging contest this year.

This contest has been organized using funding of several projects and the community contribution of several tools: we warmly thank the JMMC for providing observation preparation and post-processing tools that were used for the preparation of this contest. We are also grateful for two French's Agence Nationale de la Recherche projects: the project MASSIF (ANR-21-CE31-0018), and the project ExoVLTI (ANR-21-CE31-0017) that contributed to the preparation of the contest.

## REFERENCES

[1] Lawson, P. R., Cotton, W. D., Hummel, C. A., Monnier, J. D., Zhao, M., Young, J. S., Thorsteinsson, H., Meimon, S. C., Mugnier, L. M., Le Besnerais, G., Thiebaut, E. M., and Tuthill, P. G., "An interferometry imaging beauty contest," in [*New Frontiers in Stellar Interferometry*], Traub, W. A., ed., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series* 5491, 886 (Oct. 2004).

[2] Lawson, P. R., Cotton, W. D., Hummel, C. A., Baron, F., Young, J. S., Kraus, S., Hofmann, K.-H., Weigelt, G. P., Ireland, M., Monnier, J. D., Thiébaut, E., Rengaswamy, S., and Chesneau, O., "2006 interferometry imaging beauty contest," in [*Advances in Stellar Interferometry*], Monnier, J. D., Schöller, M., and Danchi, W. C., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series* 6268, 62681U (June 2006).
[3] Cotton, W., Monnier, J., Baron, F., Hofmann, K.-H., Kraus, S., Weigelt, G., Rengaswamy, S., Thiébaut, E., Lawson, P., Jaffe, W., Hummel, C., Pauls, T., Schmitt, H., Tuthill, P., and Young, J., “2008 imaging beauty contest,” in [*Optical and Infrared Interferometry*], Schöller, M., Danchi, W. C., and Delplancke, F., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 7013*, 70131N (July 2008).

[4] Malbet, F., Cotton, W., Duvert, G., Lawson, P., Chiavassa, A., Young, J., Baron, F., Buscher, D., Rengaswamy, S., Kloppenborg, B., Vannier, M., and Mugnier, L., “The 2010 interferometric imaging beauty contest,” in [*Optical and Infrared Interferometry II*], Danchi, W. C., Delplancke, F., and Rajagopal, J. K., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 7734*, 77342N (July 2010).

[5] Baron, F., Cotton, W. D., Lawson, P. R., Ridgway, S. T., Aarnio, A., Monnier, J. D., Hofmann, K.-H., Schertl, D., Weigelt, G., Thiébaut, E., Soulez, F., Mary, D., Millour, F., Vannier, M., Young, J., Elias, N. M., Schmitt, H. R., and Rengaswamy, S., “The 2012 interferometric imaging beauty contest,” in [*Optical and Infrared Interferometry III*], Delplancke, F., Rajagopal, J. K., and Malbet, F., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 8445*, 84451E (July 2012).

[6] Monnier, J. D., Berger, J.-P., Le Bouquin, J.-B., Tuthill, P. G., Wittkowski, M., Grellmann, R., Müller, A., Renganswany, S., Hummel, C., Hofmann, K.-H., Schertl, D., Weigelt, G., Young, J., Buscher, D., Sanchez-Bermudez, J., Alberdi, A., Schoedel, R., Köhler, R., Soulez, F., Thiébaut, É., Kluska, J., Malbet, F., Duvert, G., Kraus, S., Kloppenborg, B. K., Baron, F., de Wit, W.-J., Rivinius, T., and Merand, A., “The 2014 interferometric imaging beauty contest,” in [*Optical and Infrared Interferometry IV*], Rajagopal, J. K., Creech-Eakman, M. J., and Malbet, F., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 9146*, 91461Q (July 2014).

[7] Sanchez-Bermudez, J., Thiébaut, E., Hofmann, K. H., Heininger, M., Schertl, D., Weigelt, G., Millour, F., Schutz, A., Ferrari, A., Vannier, M., Mary, D., and Young, J., “The 2016 interferometric imaging beauty contest,” in [*Optical and Infrared Interferometry and Imaging V*], Malbet, F., Creech-Eakman, M. J., and Tuthill, P. G., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 9907*, 99071D (Aug. 2016).

[8] Mérand, A., Alberdid, A., Kluska, J., Tallon-Bosc, I., Tallon, M., Thiébaut, É., Sánchez-Bermudez, J., Schödel, R., Soulez, F., and Young, J., “Optical interferometry image reconstruction contest VIII,” in [*Optical and Infrared Interferometry and Imaging VI*], Creech-Eakman, M. J., Tuthill, P. G., and Mérand, A., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 10701*, 107011U (July 2018).

[9] Sanchez-Bermudez, J., Mérand, A., Sallum, S., Drevon, J., Kluska, J., Millour, F., Vermot, P., and Young, J., “Optical interferometry imaging contest IX,” in [*Optical and Infrared Interferometry and Imaging VIII*], Mérand, A., Sallum, S., and Sanchez-Bermudez, J., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 12183*, 121831G (Aug. 2022).

[10] Baron, F. and Young, J. S., “Image reconstruction at Cambridge University,” in [*Optical and Infrared Interferometry*], Schöller, M., Danchi, W. C., and Delplancke, F., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 7013*, 70133X (July 2008).

[11] Thiébaut, E., “MIRA: an effective imaging algorithm for optical interferometry,” in [*Optical and Infrared Interferometry*], Schöller, M., Danchi, W. C., and Delplancke, F., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 7013*, 70131I (July 2008).

[12] Hofmann, K. H., Weigelt, G., and Schertl, D., “An image reconstruction method (IRBis) for optical/infrared interferometry,” *A&A* **565**, A48 (May 2014).

[13] Millour, F., Meilland, A., Chesneau, O., Stee, P., Kanaan, S., Petrov, R., Mourard, D., and Kraus, S., “Imaging the spinning gas and dust in the disc around the supergiant A[e] star HD 62623,” *A&A* **526**, A107 (Feb. 2011).

[14] Kluska, J., Malbet, F., Berger, J. P., Baron, F., Lazareff, B., Le Bouquin, J. B., Monnier, J. D., Soulez, F., and Thiébaut, E., “SPARCO : a semi-parametric approach for image reconstruction of chromatic objects. Application to young stellar objects,” *A&A* **564**, A80 (Apr. 2014).
[15] Drevon, J., Millour, F., Cruzalebes, P., Scicluna, P., and Paladini, C., “RHAPSODY: Reconstructing Hankel rAdial Profiles in centro-Symmetric Objects with Discrete rings for astrophYSics,” in [Optical and Infrared Interferometry and Imaging VIII], Mérand, A., Sallum, S., and Sanchez-Bermudez, J., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 12183*, 121831O (Aug. 2022).

[16] Claes, R., Kluska, J., Van Winckel, H., and Min, M., “Neural network based image reconstruction with astrophysical priors,” in [*Optical and Infrared Interferometry and Imaging VII*], Tuthill, P. G., Mérand, A., and Sallum, S., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 11446*, 114461U (Dec. 2020).

[17] Millour, F., Chesneau, O., Driebe, T., Petrov, R. G., Bonneau, D., Dessart, L., Hofmann, K.-H., and Weigelt, G., “Wolf-Rayet stars probed by AMBER/VLTI,” in [*Optical and Infrared Interferometry*], Schöller, M., Danchi, W. C., and Delplancke, F., eds., *Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 7013*, 70130T (July 2008).

[18] Kluska, J., Malbet, F., Berger, J. P., Baron, F., Lazareff, B., Le Bouquin, J. B., Monnier, J. D., Soulez, F., and Thiébaut, E., “SPARCO : a semi-parametric approach for image reconstruction of chromatic objects. Application to young stellar objects,” *A&A* **564**, A80 (Apr. 2014).

[19] Ginski, C., Stolker, T., Pinilla, P., Dominik, C., Boccaletti, A., de Boer, J., Benisty, M., Biller, B., Feldt, M., Garufi, A., Keller, C. U., Kenworthy, M., Maire, A. L., Ménard, F., Mesa, D., Milli, J., Min, M., Pinte, C., Quanz, S. P., van Boekel, R., Bonnefoy, M., Chauvin, G., Desidera, S., Gratton, R., Girard, J. H. V., Keppler, M., Kopytova, T., Lagrange, A. M., Langlois, M., Rouan, D., and Vigan, A., “Direct detection of scattered light gaps in the transitional disk around HD 97048 with VLT/SPHERE,” *A&A* **595**, A112 (Nov. 2016).

[20] Carciofi, A. C. and Bjorkman, J. E., “Non-LTE Monte Carlo Radiative Transfer. I. The Thermal Properties of Keplerian Disks around Classical Be Stars,” *ApJ* **639**, 1081–1094 (Mar. 2006).

[21] Beckwith, S. V. W., “Circumstellar Disks,” in [*Star Formation 1999*], Nakamoto, T., ed., 211–211 (Dec. 1999).

[22] Charnay, B., Bézard, B., Baudino, J. L., Bonnefoy, M., Boccaletti, A., and Galicher, R., “A Self-consistent Cloud Model for Brown Dwarfs and Young Giant Exoplanets: Comparison with Photometric and Spectroscopic Observations,” *ApJ* **854**, 172 (Feb. 2018).

[23] Gomes, N., Garcia, P. J. V., and Thiébaut, É., “Assessing the quality of restored images in optical long-baseline interferometry,” *MNRAS* **465**, 3823–3839 (Mar. 2017).