# Lessons Learned from the Two Largest Galaxy Morphological Classification Catalogues built by Convolutional Neural Networks

Ting-Yun Cheng, H. Domínguez Sánchez, J. Vega-Ferrero, C.J. Conselice, M.

Siudek, A. Aragón-Salamanca, M. Bernardi, R. Cooke, L. Ferreira, M.

Huertas-Company, et al.

► To cite this version:

Ting-Yun Cheng, H. Domínguez Sánchez, J. Vega-Ferrero, C.J. Conselice, M. Siudek, et al. Lessons Learned from the Two Largest Galaxy Morphological Classification Catalogues built by Convolutional Neural Networks. Mon.Not.Roy.Astron.Soc., 2022, 518 (2), pp.2794-2809. 10.1093/mnras/stac3228. hal-03815199

HAL Id: hal-03815199

https://hal.science/hal-03815199

Submitted on 18 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Lessons learned from the two largest Galaxy morphological classification catalogues built by convolutional neural networks

T.-Y. Cheng<sup>1,★</sup> H. Domínguez Sánchez,<sup>2,3</sup> J. Vega-Ferrero<sup>1</sup>,<sup>4</sup> C. J. Conselice,<sup>5</sup> M. Siudek<sup>1</sup>,<sup>3,6</sup>

A. Aragón-Salamanca<sup>1</sup>,<sup>7</sup> M. Bernardi,<sup>8</sup> R. Cooke<sup>1</sup>,<sup>1</sup> L. Ferreira,<sup>7</sup> M. Huertas-Company<sup>1</sup>,<sup>4,9</sup>

J. Krywult,<sup>10</sup> A. Palmese<sup>1</sup>,<sup>11†</sup> A. Pieres<sup>1</sup>,<sup>12,13</sup> A. A. Plazas Malagón<sup>1</sup>,<sup>14</sup> A. Carnero Rosell<sup>1</sup>,<sup>3,12,15</sup>

D. Gruen,<sup>16</sup> D. Thomas<sup>1</sup>,<sup>17</sup> D. Bacon,<sup>17</sup> D. Brooks,<sup>18</sup> D. J. James,<sup>19</sup> D. L. Hollowood<sup>1</sup>,<sup>20</sup> D. Friedel,<sup>21</sup>

E. Suchyta<sup>1</sup>,<sup>22</sup> E. Sanchez,<sup>23</sup> F. Menanteau,<sup>21,24</sup> F. Paz-Chinchón,<sup>21,25</sup> G. Gutierrez,<sup>26</sup> G. Tarle,<sup>27</sup>

I. Sevilla-Noarbe,<sup>23</sup> I. Ferrero<sup>1</sup>,<sup>28</sup> J. Annis,<sup>26</sup> J. Frieman,<sup>26,29</sup> J. García-Bellido<sup>1</sup>,<sup>30</sup> J. Mena-Fernández,<sup>23</sup>

K. Honscheid,<sup>31,32</sup> K. Kuehn,<sup>33,34</sup> L. N. da Costa,<sup>12</sup> M. Gatti,<sup>8</sup> M. Raveri,<sup>8</sup> M. E. S. Pereira,<sup>35</sup>

M. Rodriguez-Monroy,<sup>23</sup> M. Smith<sup>1</sup>,<sup>36</sup> M. Carrasco Kind<sup>1</sup>,<sup>21,24</sup> M. Aguena<sup>1</sup>,<sup>12</sup> M. E. C. Swanson,

N. Weaverdyk<sup>1</sup>,<sup>27,37</sup> P. Doel,<sup>18</sup> R. Miquel,<sup>6,38</sup> R. L. C. Ogando<sup>1</sup>,<sup>13</sup> R. A. Gruendl,<sup>21,24</sup> S. Allam,<sup>26</sup>

S. R. Hinton<sup>1</sup>,<sup>39</sup> S. Dodelson,<sup>40,41</sup> S. Bocquet,<sup>16</sup> S. Desai,<sup>42</sup> S. Everett<sup>43</sup> and V. Scarpine<sup>26</sup>

*Affiliations are listed at the end of the paper*

Accepted 2022 November 2. Received 2022 October 26; in original form 2022 August 19

## ABSTRACT

We compare the two largest galaxy morphology catalogues, which separate early- and late-type galaxies at intermediate redshift. The two catalogues were built by applying supervised deep learning (convolutional neural networks, CNNs) to the Dark Energy Survey data down to a magnitude limit of ~21 mag. The methodologies used for the construction of the catalogues include differences such as the cutout sizes, the labels used for training, and the input to the CNN – monochromatic images versus *gri*-band normalized images. In addition, one catalogue is trained using bright galaxies observed with DES (i < 18), while the other is trained with bright galaxies (r < 17.5) and ‘emulated’ galaxies up to r-band magnitude 22.5. Despite the different approaches, the agreement between the two catalogues is excellent up to i < 19, demonstrating that CNN predictions are reliable for samples at least one magnitude fainter than the training sample limit. It also shows that morphological classifications based on monochromatic images are comparable to those based on *gri*-band images, at least in the bright regime. At fainter magnitudes, i > 19, the overall agreement is good (~95 per cent), but is mostly driven by the large spiral fraction in the two catalogues. In contrast, the agreement within the elliptical population is not as good, especially at faint magnitudes. By studying the mismatched cases, we are able to identify lenticular galaxies (at least up to i < 19), which are difficult to distinguish using standard classification approaches. The synergy of both catalogues provides an unique opportunity to select a population of unusual galaxies.

**Key words:** methods: data analysis – methods: statistical – galaxies: structure.

## 1 INTRODUCTION

Galaxy morphology describes the visual features of a galaxy and the structure of its light distribution. Both of these properties are strongly connected with its formation history (e.g. Holmberg 1958; Dressler 1980). In addition, galaxy morphologies are intimately related to their stellar populations, likewise for galaxy stellar masses, star formation rates, ages, and metallicities (e.g. Conselice 2006; Pozzetti et al. 2010; Wuyts et al. 2011; Huertas-Company et al. 2016). Therefore,

obtaining a large number of galaxies with robust morphological classifications is of great importance for understanding galaxy evolutionary history and stages.

Traditionally, the morphological classification of galaxies has been based on visual inspection (de Vaucouleurs 1959; Sandage 1961; de Vaucouleurs 1964; Fukugita et al. 2007; Nair & Abraham 2010; Baillard et al. 2011). However, during the past decades, there was a significant increase in the sizes of galaxy data sets, from the *Hubble Space Telescope*, or the Sloan Digital Sky Survey (SDSS), to current surveys such as Dark Energy Survey, which includes hundreds of millions of galaxies (DES Collaboration 2005; DES Collaboration 2016, hereafter, DES). The number of observed galaxies will increase even more with the advent of future surveys such as the *Euclid Space*

<footnote>* E-mail: ting-yun.cheng@durham.ac.uk</footnote>

<footnote>† NASA Einstein fellow.</footnote>
Telescope (Laureijs et al. 2011) and the Vera Rubin Observatory Legacy Survey of Space and Time (Ivezić et al. 2019), making it impossible to visually classify such extremely large data sets, even with Citizen Science tools such as Galaxy Zoo (Lintott et al. 2008, 2011; Willett et al. 2013). Thankfully, we can now utilize machine learning techniques, which have been applied to a variety of astronomical studies with great success since the 1990s: e.g. star-galaxy separation (Odewahn et al. 1992; Weir, Fayyad & Djorgovski 1995; Soumagnac et al. 2015), photo-z estimation (Collister & Lahav 2004; Alarcon et al. 2021; Schuldt et al. 2021; Soo et al. 2021), galaxy structural measurements (Tohill et al. 2021), strong lensing identification (Jacobs et al. 2017; Petrillo et al. 2017; Lanusse et al. 2018; Cheng et al. 2020b), finding galaxy mergers (Bottrell et al. 2019; Ferreira et al. 2020), and morphological classification of galaxies (Lahav et al. 1995; Banerji et al. 2010; Domínguez Sánchez et al. 2018; Huertas-Company et al. 2018; Siudek et al. 2018a, b; Ghosh et al. 2020; Hausen & Robertson 2020; Walmsley et al. 2020; Cheng et al. 2020a; Turner et al. 2021; Cheng et al. 2021a; Gupta, Srijith & Desai 2022).

The main advantages of machine learning techniques, and in particular, of convolutional neural network (CNN) algorithms, compared to light-profile parametric fitting (e.g. Tarsitano et al. 2018; Everett et al. 2022) are the computation time, orders of magnitude faster once the modes have been trained, and the lack of a model assumption to describe the light profile (of particular importance for asymmetric galaxies, which are more frequent at higher redshift).

Machine learning techniques are applied to astronomical studies using a diversity of approaches, but there are not many comparisons of these different techniques to assess their successes and failures. In this work, we compare the results of the two morphological catalogues presented in Vega-Ferrero et al. (2021, hereafter V21) and Cheng et al. (2021b, hereafter C21), constructed by applying CNNs to the DES data. The differences between the two approaches provide an excellent opportunity to assess, with great statistics, the impact of different deep learning methodologies (e.g. training labels, depth of the training sample, the inclusion of ‘emulated’ images in the training sample, etc). The catalogues are the two largest morphological classification catalogues to date including ~27 million (V21) and ~21 (C21) million galaxies, respectively. The overlapping sample includes over 17 million galaxies. This comparison not only potentially further validates the classification of the two catalogues, but also provides a detailed analysis of different approaches within CNN galaxy morphology studies. Additionally, by combining the two catalogues one can get more than 30 million galaxies with morphological classification.

Since both catalogues have their own related papers, in this work, we focus on their comparison. The arrangement of this paper is as follows. In Section 2, we discuss the differences between the two catalogues in sample selection and methodology. We present a statistical comparison of the two catalogues in Section 3. We discuss the agreement and disagreement between the two catalogues in Section 4. This part of the discussion is divided into bright and faint galaxies and a detailed analysis of the mismatched cases. Finally, a comparison of the physical properties and structural measurements of the two classes reported by each catalogue is discussed in Section 5. We summarize our results in Section 6.

## 2 DATA AND CATALOGUES

In this work, we compare the two largest galaxy morphological classification catalogues to date: V21 which contains ~27 million galaxies with *r*-band magnitude brighter than 21.5, and C21 which

includes ~21 million galaxies with an *i*-band magnitude range 16 ≤ *i* < 21 and at redshift *z* < 1. Both catalogues are built by analysing DES data, which is a wide-field optical imaging survey covering 5000 square degrees (~1/8 sky; Neilsen et al. 2019). The co-add images have a spatial resolution of 0.263 arcsec per pixel and were taken by the Dark Energy Camera (Flaugher et al. 2015), which has a high quantum efficiency in the red wavebands (>90 per cent from ~650 to ~900 nm), and includes ~300 million galaxies with a mean median depth of *g* = 24.33, *r* = 24.08, and *i* = 23.44 at a single-to-noise ratio S/N = 10 (Abbott et al. 2018). The imaging data used in both works is DES Year 3 (Y3) data, but with different selection criteria for the initial samples (see Section 2.1) from the DES Y3 GOLD catalogue (Sevilla-Noarbe et al. 2021).

If not specified, physical properties of galaxies, such as apparent magnitude, redshift, colour, etc., shown in this work are from the DES Y3 GOLD catalogue. Additional stellar mass information is obtained by running the LePhare code (Arnouts & Ilbert 2011) using Bruzual & Charlot (2003) templates, three different metallicities (including solar), Chabrier Initial Mass Function, and exponentially declining star formation histories (similar to Palmese et al. 2020).<fnref n="1" />

Another catalogue built by Tarsitano et al. (2018, hereafter, T18) provides structural measurements, such as Sérsic index, ellipticity, etc. While writing this manuscript, a new parametric light-profile fitting model for DES Y3 GOLD was released (Everett et al. 2022), which could serve as an additional comparison to the predictions of the CNN. However, there are some assumptions in the derivation of those parameters<fnref n="2" /> that may need further examination. Therefore, in this work, we limit the comparison of structural parameters with those from T18. In Section 4.2, we also use the measurements from the VIMOS Public Extragalactic Redshift Survey (VIPERS; Moutard et al. 2016a, b; Krywult et al. 2017; Scodeggio et al. 2018; Siudek et al. 2018b) to check the robustness of the morphological classifications.

### 2.1 Data selection

Different selection criteria for constructing the morphologically classified samples were applied in V21 and C21. The first significant difference are the filters used to define the upper limit of the samples' brightness: V21 imposed *r* < 21.5, while in C21 the selection was 16 ≤ *i* < 21. Fig. 1 shows the magnitude and redshift distribution of the two catalogues. The observed magnitudes are from the DES Y3 GOLD catalogue, measured in an elliptical aperture shaped by the Kron radius in the *i*-band and *r*-band. The photometric redshifts are obtained by the Directional Neighbourhood Fitting algorithm (De Vicente, Sánchez & Sevilla-Noarbe 2016). Galaxies in C21 with the *i*-band magnitude brighter than 18 (the limit for their training sample, see section 2.2) contribute to 3.5 per cent of the total sample. In V21, by including 'emulated' faint galaxies [see (3) of Section 2.2], the magnitude limit of their training samples is *r* < 22.5, covering the full magnitude range of the two catalogues. Fig. 1 shows that the use of different filters for selecting the initial samples results in ~3 and ~2 million unique galaxies in V21 and C21, respectively (i.e. included in V21 and not in C21, and vice versa). Additionally, C21 applied a redshift cut at *z* < 1, because their classifier, which is trained with bright galaxies at low redshift, has difficulty discriminating two

<footnote n="1">¹This unpublished catalogue is built by A. Palmese.</footnote>

<footnote n="2">²For example, the assumption of a DeVaucuers profile or a prior of 0 for the FRACDEV parameter at the faint end.</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Magnitude and redshift distributions of C21 sample (blue-filled histogram), V21 sample (light-yellow-filled histogram), and different subsets of them. The overlapped area is represented as the mix of blue and yellow, i.e. slightly darker blue/green. The solid black lines show the intersection of the two catalogues. The blue-dashed lines and the yellow-dashed lines are for samples only in C21 (C\V) and only in V21 (V\C), respectively. The vertical lines indicate the upper magnitude limit of the training samples such that the blue solid line is for the one of C21 (*i* < 18), the yellow-dotted line is the magnitude limit of the bright training sample in V21 (*r* < 17.7), and the yellow solid line represents the limit of the final training samples in V21, including emulated faint galaxies (*r* < 22.5).</figcaption></figure>

morphology types at *z* ≥ 1 (see C21, section 5.3). This difference results in ~43 000 galaxies included in V21 but not in C21.

Secondly, the two works applied several flags from the DES Y3 GOLD catalogues, which were not identical. For example, EXTENDED_CLASS_COADD distinguishes point-like objects and extended objects. V21 selected a sample with EXTENDED_CLASS_COADD >1 including medium and high-confidence galaxies, while in C21, only high-confidence galaxies were chosen (EXTENDED_CLASS_COADD =3). The different criteria in several flags result in significantly more initial samples in V21 than C21.

Finally, a cut in half-light radius (>2.8 pixels) was applied in V21 to avoid giving a morphological classification to galaxies with not enough spatial resolution. This removes ~1.36 million galaxies from V21, which are included in C21 (corresponding to ~6 per cent of the total samples from C21).

Overall, the different selection criteria applied results in an overlap of ~17 million galaxies, and ~9 and ~3 million unique galaxies in V21 and C21, respectively. The union of the two catalogues increases the total number of morphological classifications to ~30 million galaxies.

## 2.2 Methodology

The CNN architectures used in the two works are different, such that V21 uses four convolutional layers and one dense layer while C21 uses three convolutional layers and two dense layers. The hyperparameters used in the architectures are also different. This could impact the classifications, but this work cannot provide a fair discussion due to many differences between the training data sets as discussed later. This investigation would be better carried out using the same data sets, and with architectures optimized for the same task, to separate the effect from others (see a paper of computer science on this topic, Alzubaidi et al. 2021). Since each work reaches a high accuracy on its own classification task, we simply discuss the impact of other factors on the classifications. Apart from the sample selection

and the CNN architectures, there are also significant differences in the methodology, including: (1) cutout sizes; (2) training labels; (3) initial number of the training sample; (4) brightness of the training sample; and (5) the input to the CNN.

(1) **Cutout sizes:** V21 applied variable cutout sizes that are ~11.4 times the half-light radius centred on the target galaxy. Then, the images are resampled (down or upsampled, depending on the size of the resulting cutout) into a size of 64 × 64 × 3 (where the last three channels correspond to the *g*, *r*, and *i* bands). On the other hand, C21 estimates the size of a galaxy using their own algorithm.<fnref n="3" /> Galaxies with a size smaller than or equal to 30 × 30 pixels, a direct cutout with a size of 50 × 50 pixels is made. For larger galaxies, a cutout of size 200 × 200 pixels is made and then resampled into a size of 50 × 50 pixels.

(2) **Training labels:** Since there were no morphological classifications for DES galaxies with which to train the deep learning models, both studies use the morphological classifications from SDSS imaging. In particular, V21 used the T-Type<fnref n="4" /> presented in Domínguez Sánchez et al. (2018, hereafter, DS18), which are based on a deep learning model trained on the T-Type provided by the visual classification of Nair & Abraham (2010). The V21 training sample was labelled as early-type galaxies (ETGs; for galaxies with T-Type<−0.5) or late-type galaxies (LTGs; for galaxies with T-Type>0.5). Galaxies with intermediate T-Types (−0.5 < T-Type<0.5), where the scatter in the T-Type is large, were excluded from the training sample. This methodology helped the models to converge by removing galaxies with uncertain classifications. Note that by doing so, potential lenticular galaxies are excluded in the training samples. On the other hand, C21 used the classifications of spiral and elliptical galaxies from Galaxy Zoo 1 catalogue (GZ1;

<footnote n="3">³The size was estimated by counting the pixels that have signal to noise >1σ.</footnote>

<footnote n="4">⁴T-Type is a classification scheme that uses continuous values (from −6 to ~10) to categorize galaxy morphology. Elliptical galaxies have negative T-Types and spiral galaxies positive values.</footnote>
Lintott et al. 2008, 2011), and therefore C21 separates elliptical (Es) from spiral galaxies (Sp). A correction in the GZ1 visual classifications was applied due to the better resolution and deeper images of the DES data compared to SDSS (Cheng et al. 2020a, also see Fischer, Domínguez Sánchez & Bernardi 2019; Domínguez Sánchez et al. 2022). This correction provides more accurate morphological labels for ~2.5 per cent of the training set used in C21, and excludes ~0.56 per cent of galaxies that are ambiguous for binary classifications. After this, the machine classifier in C21 is more sensitive to disc structures and proves to correctly identify discy galaxies with rounder and blurred features at faint magnitudes, which humans often incorrectly classify as elliptical galaxies (see the comparison with visual classification in Section 5.2 of C21). Note that, after this labelling correction, the Sp class in C21 includes galaxies with disc structures such as lenticular galaxies.

**(3) Initial number of the training sample:** Due to different available sources with labels for training, both works start with different number of the initial training samples. By matching the GZ1 classifications with the DES observation of SDSS stripe 82, the number of C21's initial training samples is ~2800. It is then upscaled to ~53000 with data augmentation. On the other hand, benefited by using the labels provided from CNN models, V21 has a larger number of initial labelled samples (~17000) and ends up with ~48000 galaxies after augmentation, namely, 'emulated' galaxies at higher redshifts.

**(4) Brightness of the training sample:** Both catalogues use as the basis of their training sample bright SDSS galaxies ($r < 17.7$ in V21 and $i < 18$ in C21) with previous morphological classifications. While C21 only used the DES $i$-band images of bright galaxies as training sample, V21 'emulated' images at higher redshift. They included these faint galaxies in their training sample keeping their original morphological labels, i.e. without changing their 'ground truth' despite their final appearance. Details on the emulation procedure are described in section 2.3 of V21, but in short, the original DES galaxy images were deconvolved by the Point Spread Function (PSF), flux and size corrections due to cosmological dimming were applied, as well as adding k-correction and evolutionary effects. Finally, noise was added and the images were re-convolved with their PSF. In V21, the models were tested on the 'emulated' images (for which true labels were available) with an excellent performance (accuracy ~97 per cent up to $r < 21.5$ mag), demonstrating that their CNN was able to predict correct morphological labels and detect features hidden to the human eye. Additional checks on the morphological classification on real DES faint galaxies were done using available data that correlates, to some degree, with morphology (see Section 5 in V21 and Section 4.2 of this work.)

**(5) Input to the CNN:** V21 used $g$, $r$, and $i$ band images (after normalizing each band individually for each galaxy, to prevent the leak of colour information) while C21 used only $i$-band images, but combined linear, logarithmic, and gradient images. This means that the V21 machine focuses on different structures that are shown in different wavelengths, while the C21 machine considers different structures emphasized in different scales, but uses a single-band image.

## 2.3 Classification definition

Binary classification models return a probability value of belonging to a particular class. In order to assign a class, a probability threshold must be defined. Each catalogue has its own recommended thresholds to determine morphology classes, which is referred to as 'certain' classification throughout the paper. C21 provides the mean value

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Distributions of mean/median probabilities of being Sp [C21; $p(\text{Sp})$] or LTG [V21; $\text{MP\_LTG} = \text{median}(P_i)$, where $P_i$ is defined in Section 2.3]. The light yellow and blue histograms represent all the galaxies in V21 and C21, respectively, while the dashed lines show the galaxies included in only one of the two catalogues (light yellow for V21 and blue for C21). The y-axis is truncated at different scales for optimal visualization.</figcaption></figure>

of the predicted probabilities from five individual models. For each galaxy, an output probability of being an elliptical, $p(\text{Es})$, and a spiral, $p(\text{Sp})$, is assigned by the C21 machine (with $p(\text{Es}) + p(\text{Sp}) = 1$). C21 uses a threshold of 0.8 ($\bar{P} \ge 0.8$) to determine if a galaxy is classified as Es or Sp.<fnref n="5" /> In Fig. 2, we show the mean probability distributions of being Sp, $p(\text{Sp})$, for the C21 sample. Note that the galaxies in C21 that are not in V21 (labelled as $C\setminus V$) have an asymmetrical probability distribution with a preference for spiral classification. This subsample includes a significant fraction of galaxies with an $r$-band radius $\le 2.8$ pixels (~40 per cent of $C\setminus V$ sample).

On the other hand, V21 defined 'robust' classifications for ETG and LTG when $\max(P_i) < 0.3$ and $\min(P_i) > 0.7$, respectively, where $P_i$ represents the median probability obtained from 5 k-folded models. In addition, galaxies are classified as 'secure' when the difference between the maximum and minimum values of the predicted probabilities from the five individual models is smaller than 0.3, i.e. $\Delta P < 0.3$. By definition, a 'robust' classification is a 'secure' classification; galaxies without a robust classification but which satisfies the secure classification criterion are defined as a 'secure intermediate' in V21.

For simplicity, throughout the paper, we focus only on the 'robust' classifications for V21. For C21, we use classifications based on the probability thresholds mentioned above, i.e. $\bar{P} \ge 0.8$. These classifications are referred to as 'certain' type in the discussion. The rest of the galaxies not included in any of these selections are categorized as 'uncertain' type in this paper.<fnref n="6" /> Hereafter, to distinguish the results from the two catalogues, we will refer to Sp/Es for C21 classifications and to ETG/LTG for V21 ones.

## 3 STATISTICAL COMPARISON

Table 1 reports the number of galaxies of each morphological type in the two catalogues and their different combinations. Due to the different selection criteria used to construct the catalogue samples (Section 2.1), C21 and V21 only partially overlap ($C \cap V \cup C \cup V = \sim 0.59$).

<footnote n="5">⁵C21 also defined a confidence level based on the Sérsic index and $g-i$ colour distribution for users who need a further refinement to the classifications that have similar properties in Sérsic index and colour to the one of the training samples.</footnote>

<footnote n="6">⁶Note that, by doing so, the 'secure intermediate' galaxies from V21 are not discussed in this paper.</footnote>
<table><caption>**Table 1.** Number of total galaxies and certain morphological classes in each catalogue (as defined in Section 2.3). Fractions are given in brackets. The ‘C∩V’ reports the number of galaxies in the intersection of the two catalogues; hence, the row of ‘Sp/LTG’ shows the matched case of Sp & LTG while the row of ‘Es/ETG’ is Es & ETG. The ‘C\V’ reports the galaxies included in C21 only, while ‘V\C’ shows the opposite case. Finally, ‘C∪V’ represents the union of the two catalogues. In this case, the row of ‘Sp/LTG’ shows the sum of Sp, LTG, and Sp & LTG while the row of ‘Es/ETG’ is the sum of Es, ETG, and Es & ETG.</caption><thead><tr><th>Number</th><th>V21</th><th>C21<sup>b</sup></th><th>C∩V</th><th>C\V</th><th>V\C</th><th>C∪V</th></tr></thead><tbody><tr><td>Total</td><td>2697 194 5</td><td>2111 910 7</td><td>1782 125 0</td><td>3297 857</td><td>9150 695</td><td>3026 980 2</td></tr><tr><td>Sp/LTG</td><td>1978 980 9 (0.73)</td><td>1753 256 4 (0.83)</td><td>1183 069 3 (0.66)</td><td>2938 910 (0.89)</td><td>7100 049 (0.78)</td><td>2186 965 2 (0.72)</td></tr><tr><td>Es/ETG</td><td>2332 097 (0.086)</td><td>1309 229 (0.062)</td><td>689 332 (0.039)</td><td>35 999 (0.011)</td><td>649 727 (0.071)</td><td>1375 058 (0.045)</td></tr><tr><td>Uncertain</td><td>4850 039 (0.18)<sup>a</sup></td><td>2277 314 (0.11)</td><td>5301 225 (0.30)<sup>c</sup></td><td>322 948 (0.098)</td><td>1400 919 (0.15)</td><td>7025 092 (0.23)<sup>c</sup></td></tr></tbody></table>

<sup>a</sup>This number includes secure intermediate classification from V21. By excluding them, the number of galaxies with uncertain classifications is 3637 301 (0.13).

<sup>b</sup>When selecting samples with a confidence level > 2 in C21 (see section 5.3 of C21), there are 1331 256 8 galaxies (0.63 of the total number of galaxies). Within these selected samples, 1025 951 3 (0.77 of the total selected samples) galaxies are Sp and 1197 604 (0.090) galaxies are Es.

<sup>c</sup>Galaxies in this set are not assigned a certain classification in one of the two catalogues, or the classifications from the two catalogues disagree with each other.

Note that the fraction of Sp/LTG is much larger than the Es/ETG in both catalogues (73 per cent and 83 per cent for V21 and C21, respectively).

Fig. 3 shows the fraction of each morphological class in magnitude bins (both versus *r*-band and *i*-band magnitudes due to the different sample selection). The two catalogues are dominated by disc-like galaxies (i.e. Sp/LTG) at fainter magnitudes. Faint observed galaxies in our sample can be either low-mass systems or at high redshift, more likely consistent with the Sp/LTG or peculiar galaxies, which are likely to be classified as Sp/LTG in binary classifications, based on past studies (e.g. Butcher & Oemler 1984; Dressler et al. 1994; Barnes & Hernquist 1996; Martel, Premadi & Matzner 1998; Conselice, Blackburne & Papovich 2005; Conselice 2014, etc).

However, there is a clear difference between the two catalogues at bright magnitudes: in V21, the sample is dominated by ETG up to *r* ~ 17, where the trend reverses at fainter magnitudes. On the other hand, the C21 sample is dominated by Sp at all magnitude bins, and this fraction keeps increasing with magnitude, reaching 94 per cent at *i* ~ 20.5.

To shed more light on this difference, we compare the results with previous work. In particular, we use the morphological classifications of the COSMOS sample based on support vector machine (SVM) derived by Huertas-Company et al. (2008) and presented in Tasca et al. (2009). This SVM method classifies the COSMOS galaxies into three classes: Early-Type, Spirals, and Irregulars. For comparison, we group the galaxies classified as Spirals and Irregulars as LTG. In order to limit the selection effects in the comparison, we report the fractions for COSMOS galaxies with a DES counterpart and use the magnitude and radius measurements from DES. Fig. 4 shows the comparison of the fraction of ETG/LTG according to V21 (left) and Es/Sp according to C21 (right), as a function of redshift, with the ones from the COSMOS sample. The number of COSMOS galaxies that satisfy the V21 criteria (e.g. magnitude and radius limits) is 6909. When only the magnitude limit is imposed as in C21, the number of COSMOS galaxies increases to 10 350. The fraction of LTG for V21 and COSMOS are very consistent in the full redshift range, while the fraction of ETG is somewhat smaller for V21, especially at intermediate redshifts (*z* ~ 0.5). On the other hand, the fraction of Sp from C21 is above that from COSMOS (and the opposite for the Es), especially at higher redshift (*z* > 0.3). We note, however, that the area covered by COSMOS is very small (2 deg²) compared to the DES one (5000 deg²) and thus the results could be severely affected by cosmic variance. In addition, the morphological classification from COSMOS may suffer from their own uncertainties.

In any case, it is evident that there is an overabundance of Sp (reported by C21) compared to LTG (reported by V21). One possibility is that the C21 model, which was trained with bright galaxies only, is confusing noise with features in the faint and blurred images and therefore is overpredicting the number of Sp. More investigation is carried out in Sections 4.1.2, 4.1.3, and 4.2.

## 4 MORPHOLOGIES AGREEMENT AND DISAGREEMENT

In this section, we study the agreement between the morphological classifications presented in the two catalogues. To be able to compare them one to one, we restrict the analysis to the intersection of the two catalogues (1782 125 0 galaxies; C∩V in Table 1). We define the agreement as the fraction of matched classifications (i.e. Es & ETG or Sp & LTG) from the total number of galaxies with a certain classification (Section 2.3). The result is shown in Fig. 5 as a function of *i* magnitude (used here for convenience since C21 has both lower and upper limits in *i*-band magnitude). The overall agreement is very good, larger than 92 per cent in all magnitude ranges. We also check the agreement between small galaxies (*r*-band radius < 64 pixels) to examine the impact of resampling the cutouts in the pre-processing. A ~95 per cent of agreement for the small galaxy sample indicates no significant impact from this factor.

However, while the agreement at bright magnitudes is due both to Es/ETG and Sp/LTG (contributing roughly equally to the overall agreement at 16 *<i*i**<i>i*<i>*<i>17*), at fainter magnitudes, the agreement is largely driven by the high number of Sp/LTG: in the last magnitude bin, *i* = [20,21], only ~ 0.1 per cent of the galaxies with consistent morphological type between the two catalogues are classified as Es/ETG.

To further investigate the agreement of each morphology class, we show in Fig. 6 the confusion matrices in 4 mag bins. There is an excellent agreement between both morphology classes up to *i* ≤ 19. Assuming that the V21 classification is correct,<fnref n="7" /> this indicates that the CNN predictions of C21 are reliable for samples at least 1 mag fainter than the training sample.

However, at *i* > 19, the agreement vanishes, mostly due to an increasing number of Sp galaxies identified by the C21 catalogue (also see Figs 3 and 4). Fig. 5 shows that at *i* = [16,17], about

<footnote n="7">⁷The V21 catalogue performance is discussed in detail when applied to the emulated galaxies, but also with real faint DES galaxies in section 5 of V21.</footnote>
<figure><img src="image_5.png" /><figcaption>**Figure 3.** Fraction of Es/ETG (red), Sp/LTG (blue), and galaxies with uncertain classifications (grey) with respect to the total in *r* and *i*-band magnitude bins for the V21 (left) and C21 (right) catalogues. The grey dotted line in the left-hand panel (V21) are uncertain galaxies excluding ‘secure intermediate’ classification (see Section 2.3 for the definition of each class).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Figure 4.** Fraction of morphological classes as a function of redshift for the V21 (left) and the C21 (right) catalogues. Es/ETG (red) are compared with the galaxies classified as ETG in COSMOS (orange) and Sp/LTG (blue) are compared to galaxies classified as Spirals and Irregulars in COSMOS (cyan). For the V21 sample the results are shown for the full (solid lines) and the secure (dashed lines) samples (defined in Section 2.3). COSMOS galaxies are selected in a similar way as the corresponding morphological catalogue: 6909 galaxies have *r* < 21.5 mag and $R_e > 2.8$ pixels when compared to V21, and 10 350 galaxies have *i* < 21.5 mag and no cut in radius when compared to C21. Vertical error bars are computed assuming Poisson error (i.e. considering that the error in the number of galaxies is $\sqrt{N}$).</figcaption></figure>

46 per cent of the galaxies are classified as ETG by V21 and Es by C21; the percentage significantly drops, to ~ 25 per cent, at *i* = [18,19], and the fraction is almost negligible (0.1 per cent) at *i* = [20,21]. On the other hand, most of the galaxies classified as LTG in V21 are also classified as Sp by C21.

The differences in the classification of the faint-end galaxies are probably due to the different training strategies: recall that the C21 machine is trained with the images of bright galaxies ($16 \le i < 18$) while V21 included emulated faint DES galaxies up to 22.5 mag in the training sample. In addition, it could also be impacted by the different morphological labels used for training in the two works. Unfortunately, the absence of ‘ground truth’ for the faint DES galaxies prevents us from claiming which are the ‘right’ or ‘wrong’ classifications. In the next Sections 4.1.3, 4.2.1, and 5, we investigate in detail the properties of the galaxies with mismatched classifications to shed some light on their nature and their true morphological class.

## 4.1 Bright galaxies

The previous section suggests that the comparison of the morphological classifications for the bright and faint galaxies is very different,

and so we divide the discussion in the two regimes. In this section, we validate the classifications of bright galaxies ($16 \le i < 18$) taking advantage of the available labels. We compare the classifications with the labels used for training each of the data sets (i.e. GZ1, Section 4.1.1; and DS18, Section 4.1.2) and we discuss the nature of the mismatched cases in Section 4.1.3.

### 4.1.1 Compared with GZ1 labels

First, we compare the classifications of the bright galaxies ($16 \le i < 18$) with the labels from Galaxy Zoo 1, used for training C21 models. Both catalogues show an excellent agreement with the GZ1 labels, with less than 3 per cent of misclassifications. Interestingly, the dominant mismatches of each catalogue are different. In C21, the main mismatch occurs for galaxies classified as Sp in C21 but labelled as Es in GZ1. C21 and Cheng et al. (2020a) discussed that DES imaging data has better resolution and imaging depth than SDSS revealing structures such as spiral arms, which are not clearly visible in SDSS imaging data and which could explain the disagreement in the classifications. Randomly selected examples of this case are shown in Fig. A1. These galaxies are visually discy and/or spiral galaxies, indicating that C21 classifies these galaxies correctly.
<figure><img src="image_7.png" /><figcaption>**Figure 5.** Agreement of certain types (Section 2.3) within different magnitude bins in *i*-band. Grey bars show the percentage agreement of all galaxies with certain classifications. Solid lines represent the fraction of galaxies with matched classifications between the two catalogues. Blue colour shows the match of Sp & LTG while red colour is for Es & ETG. On the contrary, dashed lines and cross points are the fraction of galaxies with mismatched classifications. Blue colour shows the fraction of galaxies that are Sp in C21 but ETG in V21, while red colour presents the fraction of ones with a class of Es in C21 and LTG in V21.</figcaption></figure>

On the other hand, the ~2 per cent of the misclassifications in V21 correspond to galaxies classified as ETG in V21 but labelled as Sp by GZ1. This mismatch is mostly due to lenticular or edge-on galaxies.<fnref n="8" /> Randomly selected examples of this case are shown in Fig. A2

### 4.1.2 Compared with DS18

Next, we compare both catalogues with the labels from DS18 in Fig. 7. V21 reaches an accuracy of 0.95 while C21 has an accuracy of 0.85, with the main source of mismatches being galaxies classified as Sp in C21 but labelled as ETG (T-Type≤0) in DS18. We check whether this mismatch is dominated by lenticular galaxies using the probability of being lenticular (*p*(S0)) reported in DS18. We confirm that most (0.95) of the galaxies classified as Sp by C21 and labelled as ETG by DS18 are, in fact, lenticular galaxies, consistent with the previous section. Randomly selected examples of this kind of mismatch are shown in Fig. A3. In addition, we notice that a few of these galaxies show spiral arms. This is the case where the better quality of the DES images reveals structures that are not visible in SDSS, and the corrected classifier in C21 is able to classify them correctly. In the right-hand panel of Fig. 7, we check the role played by lenticular galaxies [classified as such in DS18 according to *p*(S0)] by excluding them from the computation of the confusion matrix. The agreement with C21 classification is significantly improved for the Sp populations, reaching 99 per cent agreement for this class.

The excellent agreement between the two catalogues and their respective training labels suggests that the inclusion of the three band

images does not improve the classification significantly, at least in the bright regime. This can also be important for the construction of future morphological catalogues where single-band images can be used.

### 4.1.3 Mismatched classifications of bright galaxies

We now study in more detail the misclassifications between the two catalogues at $16 \le i < 18$. There are 17 617 Sp & ETG and 1784 Es & LTG, corresponding to ~3.44 per cent and ~0.348 per cent of the certain types within this magnitude range, respectively.

For the case of Sp & ETG, 0.94 of them are labelled as lenticular galaxies according to DS18. This suggests that bright galaxies with a classification of Sp & ETG are likely lenticular galaxies, in agreement with the finding in Section 4.1.2. Fig. 8 shows structural and physical properties of this mismatch. The distributions indicate that Sp & ETG galaxies have intermediate-to-large Sersic index, high-stellar mass, and similar colour distributions to the Es & ETG. On the contrary, they have a very different ellipticity distribution than Es & ETG, peaking at elongated values. Randomly selected example images are shown below.

By combining the classifications of the two catalogues and looking for Sp & ETG, we are able to 're-discover' lenticular galaxies, a class which was not included in any of the training labels used in the construction of the two catalogues. Note, however, that completeness is not ensured when selecting a sample of lenticular galaxies with this approach.

For the Es & LTG, we show the Sérsic index, ellipticity, apparent colour (g - r), and stellar mass distribution in Fig. 9. Galaxies with a classification of Es & LTG follow the same colour and mass distribution of Sp & LTG and tend to have intermediate Sérsic index, $1 < n < 4$, peaking at $n \sim 1.5$. In addition, they have low ellipticity, i.e. are round objects, suggesting that these galaxies could be disc galaxies seen face-on. Randomly selected examples are shown in Fig. 9. Note that these galaxies have a large bulge component, and the spiral structure, if present, is very faint. Since spiral structures commonly dominate in different wavelengths than the bulge component, the use of multiband images in V21 might be more effective than the use of monochromatic images with different scales when classifying these galaxies. Nevertheless, these galaxies probably correspond to intermediate types – some of them could be S0 or even Es; these galaxy types are difficult to categorize in any of the two classes.

## 4.2 Faint galaxies

Despite the good agreement between the two catalogues at bright magnitudes ($i < 19$ in Fig. 6), this only represents 14 per cent (2507 490 galaxies) of the intersection of the two catalogues. The lack of a labelled sample at $i > 19$ complicates the comparison. Therefore, in this section, we rely on other quantities that correlate with morphology but not uniquely to validate our results. In particular, we use the structural parameters from T18, stellar masses¹ using similar methods to Palmese et al. (2020), photometric measurements (colour) from the DES Y3 GOLD catalogue, and absolute colour, stellar mass (Moutard et al. 2016a, b), and spectral classifications (Siudek et al. 2018b) from VIPERS. Despite a small statistics, we use VIPERS measurements for the absolute colour at fainter magnitudes, because the measurement of absolute magnitudes in DES is less accurate due to the larger uncertainty of the DES photometric redshifts.

<footnote n="8">⁸V21 also provides a classification of edge-on galaxies in their catalogue and warns the users not to trust galaxies classified both as ETG and edge-on. Visual inspection confirmed that many of these could be lenticular galaxies seen edge-on.</footnote>
<figure><img src="image_8.png" /><figcaption>**Figure 6.** Confusion matrices of C∩V samples in different magnitude bins. The red or green text in each quadrant represents the number of galaxies with classification in agreement between C21 and V21. The number above is the fraction of these galaxies with respect to the V21 classification.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>**Figure 7.** Confusion matrices and ROC curves of bright galaxies ($16 \le i < 18$) in C21 (top panel) and V21 (bottom panel) compared with DS18. 'TT' represents 'T-Type'. $TT \le 0$ and $TT > 0$ represents ETG and LTG in DS18, respectively. The right-hand panel shows the confusion matrices that exclude lenticular galaxies (S0), i.e. galaxies with $T$-Type $\le 0$ and the probability of being S0, $p(S0) \ge 0.5$ from DS18.</figcaption></figure>

### 4.2.1 Sp & ETG mismatch

The number of Sp & ETG at $i \ge 18$ is 580 044, corresponding only to ~4.6 per cent of the certain types at $i \ge 18$ (as defined in Section 2.3), but as much as ~41 per cent of the ETG classified as such by V21. Fig. 6 shows that the agreement between the galaxies classified as ETG drops significantly after $i \ge 19$. Hereafter, we separate the discussion into three different magnitude bins: $18 \le i < 19$, $19 \le i < 20$, and $20 \le i < 21$. In Fig. 10, we show the Sérsic index, ellipticity, stellar mass, and apparent colour ($g-r$) distributions in magnitude bins for the Sp & ETG. Note that the colour is based on observed magnitudes, not absolute or k-corrected due to the uncertainty in DES photometric redshift, which complicates the comparison at different redshifts. We use apparent colour distributions of the matched classifications in each magnitude bin as a comparison.

Galaxies classified as Sp & ETG follow a similar Sérsic index distribution of bright Sp & ETG in all magnitude bins, peaking at intermediate range ($n \sim 3$). They are red and massive objects, with very similar colour and mass distributions to the Es & ETG. Interestingly, the ellipticity ($\epsilon$) distributions are very different at different magnitude ranges: the Sp & ETG $18 \le i < 19$ have large ellipticities (i.e. they are elongated galaxies), similar to the bright Sp & ETG, supporting the assumption that they are potential lenticular galaxies. On the other hand, at $i > 19$ they span a much wider range peaking at $\epsilon \sim 0.2$ (i.e. rounder objects).

The final panel shows randomly selected examples of Sp & ETG. While the ones at $18 \le i < 19$ look like lenticulars seen edge-on, the images at fainter magnitudes are very noisy and it is difficult to confirm their morphology by eye. Because of the significant change
<figure><img src="image_10.png" /><figcaption>**Figure 8.** Top: the normalized distribution of Sérsic index, ellipticity, stellar mass, and apparent colour (g − r) from left to right. The light green histograms represent Sp & ETG, while the red and blue lines show the case of Es & ETG and Sp & LTG, respectively, for comparison. The grey-dashed vertical lines note Sérsic index = 2.0 and 4.0 and ellipticity = 0.5. The text within each area separated by the grey-dashed lines shows the fraction of Sp & ETG in each range. Bottom: randomly selected example images of the mismatched case, Sp & ETG.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>**Figure 9.** Same as Fig. 8 but for the Es & LTG, shown as light orange histogram.</figcaption></figure>

in properties, we cannot argue that the Sp & ETG at fainter magnitude
are dominated by lenticular galaxies.

In order to further investigate the nature of these Sp & ETG
galaxies, we use measurements from VIPERS, which provide abso-
lute colours and a classification based on spectroscopic information.
Unfortunately, the number of galaxies in the two catalogues with a
VIPERS counterpart is very small (~4800 in total, ~600–700 Sp &
ETG, 99 per cent of them with *i* ≥ 19), and thus we cannot separate
the sample into magnitude bins. Fig. 11 shows similar trends to

Fig. 10. The distribution of Sérsic index and absolute colour ($M_g - M_r$) of Sp & ETG is even more similar to the one of Es & ETG. In addition, 0.99 of the Sp & ETG have spectral classifications from Siudek et al. (2018b),<fnref n="9" /> consistent with passive or intermediate populations (class = [0,7]). These results suggest that this population

<footnote n="9">⁹Siudek et al. (2018b) used a Fisher Expectation-Maximization unsupervised algorithm to categorize galaxies in 12 spectral classes.</footnote>
<figure><img src="image_12.png" /><figcaption>**Figure 10.** Normalized distributions of Sérsic index, ellipticity, stellar mass, and apparent colour, *g* − *r*, are presented from left to right, and the last panel shows randomly selected examples of Sp & ETG within each magnitude range. The green shaded histograms show the distribution of Sp & ETG within each magnitude range. The number of Sp & ETG galaxies that are used to construct the green-shaded histograms is shown above each graph. Solid red and blue lines are the distribution of Es & ETG and Sp & LTG, respectively, for comparison. The solid dark green lines show the distributions of the Sp & ETG case with 16 ≤ *i* < 18 for comparison. The grey vertical dashed lines note Sérsic index = 2.0 and 4.0 and ellipticity = 0.5. The text within each area separated by the grey lines shows the fraction of Sp & ETG in each range.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Figure 11.** Normalized distributions of Sérsic index, ellipticity, stellar mass, absolute colour $M_g - M_r$ (from VIPERS), and unsupervised spectral classification from Siudek et al. (2018b) are presented. The green shaded histograms show the distributions of Sp & ETG. The solid red and blue lines are the distributions of Es & ETG and Sp & LTG, respectively, for comparison.</figcaption></figure>

comprises distant passive/intermediate massive galaxies with round
shapes, more consistent with ETGs, including potential lenticular
galaxies (although this is very difficult to say from visual inspection
alone due to the noisy images). Note that there are only four galaxies

classified as Sp & ETG at 18 ≤ *i* < 19 with VIPERS’s measurements.
For *i* ≥ 19, we also have small statistics – 605 galaxies; thus, for
this specific magnitude range more precise measurements to make a
robust comparison are needed.
<figure><img src="image_14.png" /><figcaption>Figure 12. Same as Fig. 10 but for the Es & LTG, shown as orange-shaded histograms.</figcaption></figure>

As a side note, the Sp & LTG population at $19 \le i < 20$ contains more massive galaxies (with a peak at $log_{10} M^*/M_{\odot} \sim 11$) than any other magnitude bin. Since there is not an apparent change in Sérsic index, ellipticity, and colour, these massive galaxies may be interesting targets to followup.

### 4.2.2 Es & LTG mismatch

Now we discuss the Es & LTG case, which corresponds to 123 305 galaxies (~1 per cent of the certain types and ~1 per cent of the LTG according to V21), i.e. a much smaller fraction than the mismatch Sp & ETG.

Fig. 12 shows the Sérsic index, ellipticity, stellar mass, and apparent colour distributions for Es & LTG. In general, these galaxies have similar distributions in mass and colour as the Sp & LTG population. However, they are round objects with intermediate values of Sérsic indices. The cutout examples have no signs of spiral or asymmetric structure, suggesting that these could be face-on spiral galaxies with low T-Types (0 < T-Type < 2), or even low-mass elliptical galaxies. Again, these are intermediate galaxies that are difficult to classify, similarly to the brighter population discussed in Section 4.1.3. Note that the fraction of Es & LTG with high Sérsic index (i.e. Sérsic index > 2) increases after $i \ge 19$. In particular, a bimodal distribution appears in the fainter bin ($20 \le i < 21$). These galaxies with bulge structure (Sérsic index >4) are blue galaxies with relatively lower mass (peaking at $log_{10} M^*/M_{\odot} \sim 10.5$) compared to the ones of

Es & ETG. Although the Sérsic index measurements are more uncertain for this kind of galaxies, which complicates the analysis, it is reasonable that these galaxies have uncertain classification since they are uncommon in the initial training samples (i.e. bright galaxies) but also due to their intrinsic faintness, which complicates the morphological classification. This shows that some populations of galaxies could be missing in the training sets. As an alternative, using hydrodynamic simulations to build complete populations of galaxies throughout the magnitude and redshift ranges of the targets in the training process might help solving this bias. By adopting the certain classifications provided in the two catalogues, one could alternatively train a series of machines that contains real faint galaxies.

## 5 COMPARISON OF GALAXIES PROPERTIES

Finally, without limiting to the intersection samples, we study several physical and structural properties in each catalogue for the samples with a certain classification, i.e. either Es or Sp in C21 and either ETG or LTG in V21, in order to further discuss what might impact each machine's decision. Note that the classification criteria used for V21 classifications is the more restrictive version, i.e. robust ETGs/LTGs (Section 2.3).<fnref n="10" />

<footnote n="10">¹⁰One can choose to use a less restrictive criteria to increase the statistics, depending on the scientific goals.</footnote>
<figure><img src="image_15.png" /><figcaption>**Figure 13.** Normalized distributions of *r*-band magnitude, redshift, apparent colour (*g* − *r*), half-light radius, Sérsic index, and ellipticity. The shaded histograms show galaxies with robust classification from V21 while solid lines are for C21. Red and blue colour indicates Es/ETG and Sp/LTG, respectively.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>**Figure 14.** The shaded histograms show the normalized Sérsic index distributions of red Sp with $g - r > 1.5$ (left) and blue Es with $g - r < 1.5$ (right) from C21. The blue and red solid lines show the Sérsic index distributions of LTG and ETG from V21, respectively, for comparison. The number of data in each class that have Sérsic index measurements is shown within brackets in the legend.</figcaption></figure>

Fig. 13 shows the distributions of magnitude, redshift, observed colour, and structural parameters (half-light radius, Sérsic index, and ellipticity) for the two classes (ETG/LTG or Es/Sp) in the two catalogues. While the distributions in size and ellipticity look quite similar for the two catalogues, there are significant differences in the other properties.

In particular, ETG and LTG, classified according to V21, have similar redshift distributions, but show a clear bimodality in colour. The ETG have brighter observed magnitudes compared to LTG but span through the full magnitude range, at least up to 21 mag in the *r*-band. On the other hand, the colours of Es and Sp galaxies, classified as such according to C21, are more overlapped, while the Es population clearly dominates at bright magnitudes (peaking at ~18.5 mag in the *r*-band) and lower redshifts compared to the Sp population.

We further investigate the discrepancy in colour distribution by studying the Sérsic index distribution of red Sp ($g - r > 1.5$) and blue Es ($g - r < 1.5$) in Fig. 14. Note that we use apparent colour in comparison without addressing K-correction; hence, we apply a colour cut at $g - r = 1.5$, at the intersection point of the colour distributions of ETG and LTG (Fig. 13). Blue galaxies ($g - r < 1.5$) classified as Es in C21 do closely follow the Sérsic index distribution of ETG from V21. On the other hand, the red ($g - r > 1.5$) galaxies classified as Sp have an intermediate Sérsic index distribution in between ETG and LTG. This indicates a significant structural difference between red Sp and ETG/LTG, which may indicate that they are potential lenticular galaxies.

Red Sp are ~21 per cent of galaxies at $z < 1$ with a certain classification in C21 (~18 per cent from the total sample). In

Section 4.1.3, we discussed that the Sp & ETG can be potential lenticular galaxies. Over 90 per cent of the Sp & ETG are red Sp (but only 22.5 per cent red Sp are Sp & ETG). However, we note again that there are a variety of uncertainties associated with identifying lenticular galaxies, since they cover a wide range of physical properties (Deeley et al. 2020, 2021). We remark that the pursuit in this paper is not to identify lenticular galaxies, but to assess the impact of different CNN approaches to this undefined class.

# 6 SUMMARY AND CONCLUSIONS

In this work, we compare the two largest galaxy morphological classification catalogues to date: V21, which includes ~27 million galaxies at $r < 21.5$, and C21, which includes ~21 million galaxies with $16 \le i < 21$ and $z < 1$. Due to different initial sample selection, the two catalogues have an overlap of ~60 per cent. Besides differences in the CNN architectures, the two studies have fundamental differences in their approaches such as (1) cutout sizes; (2) training labels; (3) initial number of the training sample; (4) brightness of the training sample; and (5) input to the CNN: multiband versus monochromatic images (Section 2.2). The fact that such different methodologies are applied to the same large data sets allows us to compare the results in a statistically significant way and to assess different machine learning approaches.

We examine the agreement between the two catalogues using the intersection sample ($C \cap V$). The agreement is as high as ~95 per cent (see Fig. 5) of the galaxies with a certain classification, i.e. Es & ETG or Sp & LTG (as defined in Section 2.3). However, the large agreement is mostly driven by the Sp & LTG population, which corresponds to 66 per cent of the intersection of the two catalogues (89 per cent of the samples with a certain classification from one of the two catalogues). On the other hand, the agreement of the Es/ETG population is lower (3.9 per cent), especially at $i > 19$ mag (~1.2 per cent of the total intersection sample), where the number of galaxies classified as Es by C21 decreases (see Fig. 3).

The classifier of C21 was trained with bright low-redshift galaxies only, where reliable visual labels were available, while V21 added faint galaxies to the training sample by emulating bright galaxies to fainter magnitudes. Additionally, the initial number of the training samples in C21 is smaller than the one in V21, which indicates that C21’s sample might have lower diversity than the one of V21. One of the main results from this comparison is the fact that the C21 machine can push its predictions accurately 1 mag fainter than its training samples. The excellent agreement between the two catalogues up to $i < 19$ also indicates that (1) there is no significant impact from the number of the initial training samples between C21 and V21,
and (2) the use of multiband images does not provide a significant improvement or it posts similar effect in the morphological classifications of galaxies when compared to the use of monochromatic images with different scales (i.e. linear, logarithmic, and gradient). The latter can have a significant impact on the machine learning methodologies used for morphological classifications in future Big Data surveys such as Euclid or Rubin/LSST, as it provides a way to help improving classifications when using single-band images.

We have studied in detail the photometric, structural, and spectroscopic properties of the mismatched classifications of the two catalogues divided in the bright ($i < 18$) and faint ($i \ge 18$) regimes. Our main findings are:

(i) Bright Sp & ETG: These are ~3.4 per cent of the galaxies with certain classification in this magnitude range. Their properties and appearance are consistent with possibly being lenticular galaxies (Fig. 8), a class that was not included in any of the training samples. This mismatch could be a potential way to identify lenticular galaxies, but we note that this approach might result in a small completeness.

(ii) Bright Es & LTG: These contribute less than 0.4 per cent to the galaxies with certain classification in this magnitude range. They are generally blue, round galaxies (95 per cent have ellipticity smaller than 0.5, see Fig. 9) with intermediate Sérsic indices. These galaxies are difficult to classify and could be a mixture of face-on disc galaxies with no signs of spiral structure, lenticular, or elliptical galaxies.

(iii) Faint Sp & ETG: This population contributes to ~5 per cent of the galaxies with certain classification in this magnitude range but as much as 41 per cent of the ETG classified as such by V21. At $18 \le i < 19$, their structural measurements and physical properties follow similar trends as the ones at $16 \le i < 18$, supporting the hypothesis that they could be lenticular galaxies as well. In the fainter magnitude bins ($i > 19$), their properties are more consistent with a red, massive, and passive population, although it is almost impossible to confirm via visual inspection due to their noisy images (see Fig. 10).

(iv) Faint Es & LTG: These correspond to ~1 per cent of the galaxies with a certain classification for both catalogues in this magnitude range. These systems are blue galaxies with relatively low masses compared to the Es & ETG, with round shapes and intermediate Sérsic indices. These galaxies are difficult to classify, as in the bright regime (Fig. 12). At $i \ge 19$, there is a bimodal distribution of the Sérsic index, i.e. they could be a population of blue low-mass galaxies with significant bulge component, which may not be well-represented in the training sample.

Finally, we compare the physical properties of the galaxies classified as Es/ETG and Sp/LTG from each catalogue in Fig. 13. The galaxies classified as ETG by V21 have similar structural measurements such as half-light radius, Sérsic index, and ellipticity to the galaxies classified as Es by C21; the same is true for Sp/LTG. However, the distribution in observed magnitude, redshift and colour are very different for the C21 compared to V21. ETG and LTG galaxies span more or less over the full magnitude and redshift range but are clearly separated in colour, with ETGs being redder. On the other hand, the colours of Es/Sp are more overlapped and instead there is a clear difference in the magnitude and redshift of Es and Sp, with Es being more abundant in the bright low-redshift regime. This suggests that the C21 classifier underpredicts Es at fainter magnitudes and high redshifts (see also Fig. 4) probably because it confuses noise with structure. However, interestingly, when examining blue Es ($g - i < 1.5$) and red Sp ($g - i > 1.5$) in Fig. 14, blue Es have similar structure to ETG and red Sp show intermediate structure between ETG and LTG.

This is the first comparison of two largest morphological catalogues up to date, including over 20 million galaxies each, generated by a CNN. This allows us to assess different CNN approaches and to further validate the classifications provided by each catalogue. These classifications could indeed serve as a training sample to classify future data sets at faint magnitude limits. In the case of disagreement between the classifications (~5 per cent of intersection of the two samples), we suggest to complement the classification with additional parameters, such as structural measurements. Furthermore, by combining the two catalogues, the number of total classifications is increased up to ~30 million galaxies for future studies of galaxy evolution. We warn the reader to be aware of the biases and disagreement detailed in this paper when making use of the catalogues for scientific research. In particular, we caution on the robustness of the selection of the ETG population at faint magnitudes due to the large discrepancy of the classifications presented in the two catalogues.

## ACKNOWLEDGEMENTS

We thank Dr Thibaud Moutard for providing the measurements of galaxy stellar mass in VIPERS to this work. TYC acknowledges the support of STFC grant ST/T000244/1 and Royal Society grant RF/ERE/210326, hosted at Durham University. HDS acknowledges support by the PID2020-115098RJ-100 grant from MCIN/AEI/10.13039/501100011033. MS has been supported by the European Union's Horizon 2020 Research and Innovation programme under the Maria Sklodowska-Curie grant agreement (no. 754510) and the Spanish Ministry of Science and Innovation through the Juan de la Cierva-formación programme (FJC2018-038792-I). MS has been supported by the Polish National Agency for Academic Exchange (Bekker grant BPN/BEK/2021/1/00298/DEC/1).

Funding for the DES Projects has been provided by the U.S. Department of Energy, the U.S. National Science Foundation, the Ministry of Science and Education of Spain, the Science and Technology Facilities Council of the United Kingdom, the Higher Education Funding Council for England, the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign, the Kavli Institute of Cosmological Physics at the University of Chicago, the Center for Cosmology and Astro-Particle Physics at the Ohio State University, the Mitchell Institute for Fundamental Physics and Astronomy at Texas A&M University, Financiadora de Estudos e Projetos, Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro, Conselho Nacional de Desenvolvimento Científico e Tecnológico and the Ministério da Ciência, Tecnologia e Inovação, the Deutsche Forschungsgemeinschaft, and the Collaborating Institutions in the Dark Energy Survey.

The Collaborating Institutions are Argonne National Laboratory, the University of California at Santa Cruz, the University of Cambridge, Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas-Madrid, the University of Chicago, University College London, the DES-Brazil Consortium, the University of Edinburgh, the Eidgenössische Technische Hochschule (ETH) Zürich, Fermi National Accelerator Laboratory, the University of Illinois at Urbana-Champaign, the Institut de Ciències de l'Espai (IEEC/CSIC), the Institut de Física d'Altes Energies, Lawrence Berkeley National Laboratory, the Ludwig-Maximilians Universität München and the associated Excellence Cluster Universe, the University of Michigan, the National Optical Astronomy Observatory, the University of Nottingham, The Ohio State University, the University of Pennsylvania, the University of Portsmouth, SLAC National Accelerator
Laboratory, Stanford University, the University of Sussex, Texas A&M University, and the OzDES Membership Consortium.

Based in part on observations at Cerro Tololo Inter-American Observatory, National Optical Astronomy Observatory, which is operated by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with the National Science Foundation.

The DES data management system is supported by the National Science Foundation under grant numbers AST-1138766 and AST-1536171. The DES participants from Spanish institutions are partially supported by MINECO under grants AYA2015-71825, ESP2015-66861, FPA2015-68048, SEV-2016-0588, SEV-2016-0597, and MDM-2015-0509, some of which include ERDF funds from the European Union. IFAE is partially funded by the CERCA programme of the Generalitat de Catalunya. Research leading to these results has received funding from the European Research Council under the European Union's Seventh Framework Program (FP7/2007-2013) including ERC grant agreements 240672, 291329, and 306478. We acknowledge support from the Australian Research Council Centre of Excellence for All-sky Astrophysics (CAASTRO), through project number CE110001020, and the Brazilian Instituto Nacional de Ciencia e Tecnologia (INCT) e-Universe (CNPq grant 465376/2014-2).

## DATA AVAILABILITY

The morphological catalogue of V21 and C21 are available in the Dark Energy Survey Data Management system at the National Center for Supercomputing Applications at the University of Illinois and can be accessed at https://des.ncsa.illinois.edu/releases/other/morphCN.

## REFERENCES

* Abbott T. M. C. et al., 2018, ApJS, 239, 18

* Alarcon A. et al., 2021, MNRAS, 501, 6103

* Alzubaidi L. et al., 2021, J. Big Data, 8, 53

* Arnouts S., Ilbert O., 2011, Astrophysics Source Code Library, record ascl:1108.009

* Baillard A. et al., 2011, A&A, 532, A74

* Banerji M. et al., 2010, MNRAS, 406, 342

* Barnes J. E., Hernquist L., 1996, ApJ, 471, 115

* Bottrell C. et al., 2019, MNRAS, 490, 5390

* Bruzual G., Charlot S., 2003, MNRAS, 344, 1000

* Butcher H., Oemler A. J., 1984, ApJ, 285, 426

* Cheng T.-Y. et al., 2020a, MNRAS, 493, 4209

* Cheng T.-Y. Li N., Conselice C. J., Aragón-Salamanca A., Dye S., Metcalf R. B., 2020b, MNRAS, 494, 3750

* Cheng T.-Y., Huertas-Company M., Conselice C. J., Aragón-Salamanca A., Robertson B. E., Ramachandra N., 2021a, MNRAS, 503, 4446

* Cheng T.-Y. et al., 2021b, MNRAS, 507, 4425(C21)

* Collister A. A., Lahav O., 2004, PASP, 116, 345

* Conselice C. J., 2006, MNRAS, 373, 1389

* Conselice C. J., 2014, ARA&A, 52, 291

* Conselice C. J., Blackburne J. A., Papovich C., 2005, ApJ, 620, 564

* de Vaucouleurs G., 1959, Handbuch Phys., 53, 275

* de Vaucouleurs G., 1964, AJ, 69, 561

* De Vicente J., Sánchez E., Sevilla-Noarbe I., 2016, MNRAS, 459, 3078

* Deeley S. et al., 2020, MNRAS, 498, 2372

* Deeley S., Drinkwater M. J., Sweet S. M., Bekki K., Couch W. J., Forbes D. A., Dolfi A., 2021, MNRAS, 508, 895

* DES Collaboration, 2005, preprint (astro-ph/0510346)

* DES Collaboration, 2016, MNRAS, 460, 1270

* Domínguez Sánchez H., Huertas-Company M., Bernardi M., Tuccillo D., Fischer J. L., 2018, MNRAS, 476, 3661

* Domínguez Sánchez H., Margalef B., Bernardi M., Huertas-Company M., 2022, MNRAS, 509, 4024

* Dressler A., 1980, ApJ, 236, 351

* Dressler A., Oemler, Augustus J., Butcher H. R., Gunn J. E., 1994, ApJ, 430, 107

* Everett S. et al., 2022, ApJS, 258, 15

* Ferreira L., Conselice C. J., Duncan K., Cheng T.-Y., Griffiths A., Whitney A., 2020, ApJ, 895, 115

* Fischer J. L., Domínguez Sánchez H., Bernardi M., 2019, MNRAS, 483, 2057

* Flaugher B. et al., 2015, AJ, 150, 150

* Fukugita M. et al., 2007, AJ, 134, 579

* Ghosh A., Urry C. M., Wang Z., Schawinski K., Turp D., Powell M. C., 2020, ApJ, 895, 112

* Gupta R., Srijith P. K., Desai S., 2022, Astron. Comput., 38, 100543

* Hausen R., Robertson B. E., 2020, ApJS, 248, 20

* Holmberg E., 1958, Meddelanden fran Lunds Astronomiska Observatorium Serie II, 136, 1

* Huertas-Company M., Rouan D., Tasca L., Soucail G., Le Fèvre O., 2008, A&A, 478, 971

* Huertas-Company M. et al., 2016, MNRAS, 462, 4495

* Huertas-Company M. et al., 2018, ApJ, 858, 114

* Ivezić Ž. et al., 2019, ApJ, 873, 111

* Jacobs C., Glazebrook K., Collett T., More A., McCarthy C., 2017, MNRAS, 471, 167

* Krywult J. et al., 2017, A&A, 598, A120

* Lahav O. et al., 1995, Science, 267, 859

* Lanusse F., Ma Q., Li N., Collett T. E., Li C.-L., Ravanbakhsh S., Mandelbaum R., Póczos B., 2018, MNRAS, 473, 3895

* Laureijs R. et al., 2011, preprint (arXiv:1110.3193)

* Lintott C. J. et al., 2008, MNRAS, 389, 1179

* Lintott C. et al., 2011, MNRAS, 410, 166

* Martel H., Premadi P., Matzner R., 1998, ApJ, 497, 512

* Moutard T. et al., 2016a, A&A, 590, A102

* Moutard T. et al., 2016b, A&A, 590, A103

* Nair P. B., Abraham R. G., 2010, ApJS, 186, 427

* Neilsen E. H., Eric H. J. Annis J. T., Diehl H. T., Swanson M. E. C., D'Andrea C., Kent S., Drlica-Wagner A., 2019, preprint (arXiv:1912.06254)

* Odewahn S. C., Stockwell E. B., Pennington R. L., Humphreys R. M., Zumach W. A., 1992, AJ, 103, 318

* Palmese A. et al., 2020, MNRAS, 493, 4591

* Petrillo C. E. et al., 2017, MNRAS, 472, 1129

* Pozzetti L. et al., 2010, A&A, 523, A13

* Sandage A., 1961, The Hubble Atlas of Galaxies. Carnegie Institution of Washington

* Schuldt S., Suyu S. H., Cañameras R., Taubenberger S., Meinhardt T., Leal-Taixé L., Hsieh B. C., 2021, A&A, 651, A55

* Scodeggio M. et al., 2018, A&A, 609, A84

* Sevilla-Noarbe I. et al., 2021, ApJS, 254, 24

* Siudek M. et al., 2018a, preprint (arXiv:1805.09905)

* Siudek M. et al., 2018b, A&A, 617, A70

* Soo J. Y. H. et al., 2021, MNRAS, 503, 4118

* Soumagnac M. T. et al., 2015, MNRAS, 450, 666

* Tarsitano F. et al., 2018, MNRAS, 481, 2018

* Tasca L. A. M. et al., 2009, A&A, 503, 379

* Tohill C., Ferreira L., Conselice C. J., Bamford S. P., Ferrari F., 2021, ApJ, 916, 4

* Turner S. et al., 2021, MNRAS, 503, 3010

* Vega-Ferrero J. et al., 2021, MNRAS, 506, 1927 (V21)

* Walmsley M. et al., 2020, MNRAS, 491, 1554

* Weir N., Fayyad U. M., Djorgovski S., 1995, AJ, 109, 2401

* Willett K. W. et al., 2013, MNRAS, 435, 2835

* Wuyts S. et al., 2011, ApJ, 742, 96

## APPENDIX A: FIGURES OF SECTION 4
## Sp[C21]+Es[GZ1] ($16 \le i < 18$)

<figure><img src="image_17.png" /><figcaption>**Figure A1.** Randomly selected examples of the mismatches that are classified as Sp by C21 and labelled as Es in GZ1. The *p*(Sp) values quoted under each panel is the probability of being Sp provided by C21.</figcaption></figure>

## ETG[V21]+Sp[GZ1] ($16 \le i < 18$)

<figure><img src="image_18.png" /><figcaption>**Figure A2.** Randomly selected examples of the mismatches that are classified as ETG by V21 and labelled as spiral galaxies in GZ1. The MP_LTG, *p*(edge), and *p*(S0) values quoted under each panel represent the median probability of being LTG, edge-on galaxies (provided by V21), and lenticular galaxies (S0; provided by DS18), respectively. Note that most of these are actually edge-on and/or lenticular galaxies.</figcaption></figure>

## ETG[DS18]+Sp[C21] ($16 \le i < 18$)

<figure><img src="image_19.png" /><figcaption>**Figure A3.** Randomly selected examples of the mismatches that are classified as spiral galaxies (Sp) by C21 and labelled as ETG in DS18. The *p*(Sp) is the probability of being a spiral galaxy according to C21 while the *p*(S0) represents the probability of being a lenticular galaxy, according to DS18.</figcaption></figure>

<footnote n="1">¹Centre for Extragalactic Astronomy, Durham University, South Road, Durham DH1 3LE, UK</footnote>

<footnote n="2">²Centro de Estudios de Física del Cosmos de Aragón (CEFCA), àPlaza de San Juan, 1, E-44001 Teruel, Spain</footnote>

<footnote n="3">³Institute of Space Sciences (ICE, CSIC), Campus UAB, Carrer de Can Magrans, s/n, E-08193 Barcelona, Spain</footnote>

<footnote n="4">⁴Instituto de Astrofísica de Canarias (IAC) La Laguna, E-38205, Spain</footnote>

<footnote n="5">⁵Jodrell Bank Centre for Astrophysics, University of Manchester, Oxford Road, Manchester M13 9PL, UK</footnote>

<footnote n="6">⁶Institut de Física d'Altes Energies (IFAE), The Barcelona Institute of Science and Technology, E-08193 Bellaterra (Barcelona), Spain</footnote>

<footnote n="7">⁷School of Physics and Astronomy, University of Nottingham, University Park, Nottingham NG7 2RD, UK</footnote>

<footnote n="8">⁸Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA 19104, USA</footnote>

<footnote n="9">⁹LERMA, Observatoire de Paris, CNRS, PSL, Université Paris Diderot, France</footnote>

<footnote n="10">¹⁰Institute of Physics, Jan Kochanowski University, ul. Uniwersytecka 7, PL-25-406 Kielce, Poland</footnote>

<footnote n="11">¹¹Department of Physics, University of California Berkeley, 366 LeConte Hall MC 7300, Berkeley, CA, 94720, USA</footnote>

<footnote n="12">¹²Laboratório Interinstitucional de e-Astronomia - LIneA, Rua Gal. José Cristino 77, Rio de Janeiro, RJ - 20921-400, Brazil</footnote>

<footnote n="13">¹³Observatório Nacional, Rua Gal. José Cristino 77, Rio de Janeiro, RJ - 20921-400, Brazil</footnote>

<footnote n="14">¹⁴Department of Astrophysical Sciences, Princeton University, Peyton Hall, Princeton, NJ 08544, USA</footnote>

<footnote n="15">¹⁵Universidad de La Laguna, Dpto. Astrofísica, E-38206 La Laguna, Tenerife, Spain</footnote>

<footnote n="16">¹⁶University Observatory, Faculty of Physics, Ludwig-Maximilians-Universität, Scheinerstr. 1, D-81679 Munich, Germany</footnote>

<footnote n="17">¹⁷Institute of Cosmology and Gravitation, University of Portsmouth, Portsmouth PO1 3FX, UK</footnote>

<footnote n="18">¹⁸Department of Physics & Astronomy, University College London, Gower Street, London WC1E 6BT, UK</footnote>

<footnote n="19">¹⁹Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA</footnote>

<footnote n="20">²⁰Santa Cruz Institute for Particle Physics, Santa Cruz, CA 95064, USA</footnote>

<footnote n="21">²¹Center for Astrophysical Surveys, National Center for Supercomputing Applications, 1205 West Clark St., Urbana, IL 61801, USA</footnote>

<footnote n="22">²²Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA</footnote>

<footnote n="23">²³Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas (CIEMAT), Madrid, 28040, Spain</footnote>

<footnote n="24">²⁴Department of Astronomy, University of Illinois at Urbana-Champaign, 1002 W. Green Street, Urbana, IL 61801, USA</footnote>

<footnote n="25">²⁵Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK</footnote>

<footnote n="26">²⁶Fermi National Accelerator Laboratory, PO Box 500, Batavia, IL 60510, USA</footnote>
<footnote n="27">²⁷Department of Physics, University of Michigan, Ann Arbor, MI 48109, USA</footnote>

<footnote n="28">²⁸Institute of Theoretical Astrophysics, University of Oslo, PO Box 1029
Blindern, NO-0315 Oslo, Norway</footnote>

<footnote n="29">²⁹Kavli Institute for Cosmological Physics, University of Chicago, Chicago,
IL 60637, USA</footnote>

<footnote n="30">³⁰Instituto de Fisica Teorica UAM/CSIC, Universidad Autonoma de Madrid,
E-28049 Madrid, Spain</footnote>

<footnote n="31">³¹Center for Cosmology and Astro-Particle Physics, The Ohio State University, Columbus, OH 43210, USA</footnote>

<footnote n="32">³²Department of Physics, The Ohio State University, Columbus, OH 43210,
USA</footnote>

<footnote n="33">³³Australian Astronomical Optics, Macquarie University, North Ryde, NSW
2113, Australia</footnote>

<footnote n="34">³⁴Lowell Observatory, 1400 Mars Hill Rd, Flagstaff, AZ 86001, USA</footnote>

<footnote n="35">³⁵Hamburger Sternwarte, Universität Hamburg, Gojenbergsweg 112, D-21029 Hamburg, Germany</footnote>

<footnote n="36">³⁶School of Physics and Astronomy, University of Southampton, Southampton
SO17 1BJ, UK</footnote>

<footnote n="37">³⁷Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA
94720, USA</footnote>

<footnote n="38">³⁸Institució Catalana de Recerca i Estudis Avançats, E-08010 Barcelona,
Spain</footnote>

<footnote n="39">³⁹School of Mathematics and Physics, University of Queensland, Brisbane,
QLD 4072, Australia</footnote>

<footnote n="40">⁴⁰Department of Physics, Carnegie Mellon University, Pittsburgh, PA 15312,
USA</footnote>

<footnote n="41">⁴¹NSF AI Planning Institute for Physics of the Future, Carnegie Mellon
University, Pittsburgh, PA 15213, USA</footnote>

<footnote n="42">⁴²Department of Physics, IIT Hyderabad, Kandi, Telangana 502285, India</footnote>

<footnote n="43">⁴³Jet Propulsion Laboratory, California Institute of Technology, 4800 Oak
Grove Dr., Pasadena, CA 91109, USA</footnote>

<footnote>This paper has been typeset from a TEX/LATEX file prepared by the author.</footnote>