# Constraints on the in situ and ex situ stellar masses in nearby galaxies obtained with artificial intelligence

Eirini Angeloudi, Jesús Falcón-Barroso, Marc Huertas-Company, Alina

Boecker, Regina Sarmiento, Lukas Eisert, Annalisa Pillepich

► To cite this version:

Eirini Angeloudi, Jesús Falcón-Barroso, Marc Huertas-Company, Alina Boecker, Regina Sarmiento, et al.. Constraints on the in situ and ex situ stellar masses in nearby galaxies obtained with artificial intelligence. Nature Astronomy, 2024, 8, pp.1310-1320. 10.1038/s41550-024-02327-3. insu-04822427

HAL Id: insu-04822427

https://insu.hal.science/insu-04822427v1

Submitted on 3 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Constraints on the in situ and ex situ stellar masses in nearby galaxies obtained with artificial intelligence

Received: 26 February 2024

Accepted: 28 June 2024

Published online: 22 July 2024

<figure><img src="image_2.png" /></figure>

Check for updates

Eirini Angeloudi Ⓓ1,2✉, Jesús Falcón-Barroso Ⓓ1,2, Marc Huertas-Company<fnref n="1" />,2,3,4, Alina Boecker Ⓓ1,2,5, Regina Sarmiento Ⓓ1,2, Lukas Eisert⁶ & Annalisa Pillepich⁶

The hierarchical model of galaxy evolution suggests that mergers have a substantial impact on the intricate processes that drive stellar assembly within a galaxy. However, accurately measuring the contribution of accretion to a galaxy's total stellar mass and its balance with in situ star formation poses a persistent challenge, as it is neither directly observable nor easily inferred from observational properties. Using data from MaNGA, we present theory-motivated predictions for the fraction of stellar mass originating from mergers in a statistically significant sample of nearby galaxies. Employing a robust machine learning model trained on mock MaNGA analogues (MaNGIA) obtained from a cosmological simulation (TNG50), we unveil that in situ stellar mass dominates almost across the entire stellar mass spectrum ($10^9 M_\odot < M_* < 10^{12} M_\odot$). Only in more massive galaxies ($M_* > 10^{11} M_\odot$) does accreted mass become a substantial contributor, reaching up to 35–40% of the total stellar mass. Notably, the ex situ stellar mass in the nearby Universe exhibits notable dependence on galaxy characteristics, with higher accreted fractions favoured being by elliptical, quenched galaxies and slow rotators, as well as galaxies at the centre of more massive dark matter haloes.

In the cosmological model favoured at present, one of the main mechanisms driving the growth of a galaxy's stellar mass and, primarily, its size is through interactions and mergers with its satellite counterparts<fnref n="1" />. This hierarchical process of evolution not only orchestrates the creation of larger structures but also carries crucial consequences in shaping the physical characteristics of galaxies. Mergers are known to trigger structural transformations³, rejuvenate star formation⁴ and fuel the activity of active galactic nuclei⁴, as well as contribute to the building up of massive early-type galaxies (ETGs)⁵. Their multivariate effects imply that discerning the role of accretion in the assembly of a galaxy's stellar mass—particularly in relation to the mass originating from in situ star formation—is key to understanding the complex mechanisms that govern galaxy evolution.

However, our understanding of the balance between in situ mass and ex situ mass in a galaxy's total stellar mass remains incomplete. Although the role of internal star formation in the mass assembly of galaxies, which peaks around redshift $z \approx 2$, is fairly well established⁶, constraining the rate at which galaxy mergers occur across cosmic time and their integrated impact on galaxy evolution is still uncertain. A primary challenge lies in detecting merger events in observational data. Traditionally, this has been accomplished by identifying pairs of galaxies that are sufficiently close both spatially and dynamically, under the assumption that they will merge over a relatively short timescale⁷,⁸. Alternatively, direct evidence of galaxy interactions, such as tidal tails and structural asymmetries, has been utilized to develop non-parametric methods for calculating merger

<footnote n="1">¹Instituto de Astrofísica de Canarias, La Laguna, Spain. ²Dept. Astrofísica, Universidad de la Laguna, La Laguna, Spain. ³Université Paris-Cité, LERMA - Observatoire de Paris, PSL, Paris, France. ⁴SCIPP, University of California, Santa Cruz, CA, USA. ⁵Department of Astrophysics, University of Vienna, Vienna, Austria. ⁶Max Planck Institute for Astronomy, Heidelberg, Germany. ✉e-mail: eirini@iac.es</footnote>
rates across various redshifts⁹,¹⁰. However, both approaches require high-quality observational data and carry distinct biases, such as resolution-sensitive thresholds or misidentification of galaxy fly-bys as merging systems, which may not capture the entire spectrum of merger appearances.

An alternative to direct merger detection is to quantify the integrated effect of mergers in a galaxy, following a more challenging archaeological approach. As a galaxy can be observed only at a single point in cosmic time, tracing its merger history and the impact of that on its evolution is heavily dependent on models or cosmological simulations. Focusing on our cosmic neighbourhood, a number of works have derived the ex situ stellar mass for a small sample of nearby galaxies through N-body simulations that account for stellar halo build-up only through accretion of dwarf satellite galaxies¹¹,¹². Moving further away is an observational challenge, as sufficiently high-quality data are required for unveiling distinctions between the in situ and the ex situ stellar populations. So far, parametric surface brightness models have been used in combination with a direct comparison with cosmological simulations¹³. These have revealed that ex situ stellar mass mainly dominates in the outskirts of galaxies¹⁴⁻¹⁷. In particular for ETGs, several works have attributed the building up of their outer envelopes to merger events at late times¹⁸⁻²⁰. However, such results rely on toy models that might not fully capture the more complex patterns tracing the merger history of a galaxy. Machine learning has emerged as a promising tool in this regime, having already contributed to the detection of various merger aspects such as tidal features²¹, merging phases²² and mergers at higher redshift²³. Deep learning, in particular, has the potential to revolutionize our understanding of the cumulative impact of mergers on galactic evolution due to its capacity to uncover intricate patterns in complex data. This potential has been demonstrated in our recent works²⁴,²⁵, which used neural networks to infer key parameters related to the merger history of galaxies in a cosmological simulation.

Here, we present theory-motivated predictions for the integrated impact of merger events in a statistically significant observational sample of nearby galaxies. Building upon our previous work²⁵, in which we showed that spatially resolved maps of stellar mass and kinematics can serve as robust predictors of the ex situ stellar mass fraction of galaxies across two cosmological simulations, we now obtain estimates for the amount of stellar mass that has been accreted for the ~10,000 galaxies in the Mapping Nearby Galaxies at Apache Point Observatory survey (MaNGA). We investigate whether the complex merger histories of galaxies can be traced by secondary dependencies at a fixed stellar mass and attempt to find correlations with the cosmic environment. The current work is built upon a robust methodology for deriving the ex situ stellar mass fractions, which leads to informed insights about the intricate impacts that mergers can have on galaxy evolution.

## Constraining the in situ and ex situ stellar masses

To investigate the impact of mergers in the nearby Universe across a statistically significant sample, we used MaNGA Data Release 17 (DR17)²⁶, which is the integral-field unit (IFU) of the Sloan Digital Sky Survey phase IV (SDSS-IV) (details in Methods). We trained a self-supervised machine learning model (Methods) on the Mapping Nearby Galaxies with Illustris TNG Astrophysics database (MaNGIA)²⁷, a sample of mock MaNGA analogues originating from the cosmological simulation TNG50 (details in Methods). MaNGIA not only provides the ground truth that we aimed to predict but also includes a desired observational realism specifically matched to MaNGA, rendering it an ideal training sample. Our goal is to train a neural network that can infer the global fraction of accreted stars in galaxies from observable IFU maps of stellar mass surface density, stellar velocity and velocity dispersion, as the radial gradients of these properties have already been proven to be robust predictors of these quantities in a multi-domain cosmological simulation²⁵. We trained a machine learning model on MaNGIA mock galaxies and assessed its predictive accuracy on a separate test set

<figure><img src="image_3.png" /></figure>

**Fig. 1 |** The contributions of the in situ stellar mass and the ex situ stellar mass to the total stellar mass for ~10,000 MaNGA galaxies predicted by the neural network model trained on the MaNGIA mock dataset. The mean total stellar mass in a given stellar mass bin is drawn with a black solid line and should roughly be a straight line. The average contribution of the in situ stellar mass created through internal star formation in each stellar mass bin is drawn with a solid blue line. The red solid line shows the average contribution originating from accretion and mergers (ex situ). The error bars contain 68% of all data. For comparison, we plot the mass of the stellar halo (to distinguish it from dark matter halo mass) for the Milky Way (MW)³⁸ and M31 (ref. 11) with orange stars (a proxy for accreted mass) and a rough projection of the ex situ stellar mass of the Milky Way after it merges with the Large Magellanic Cloud with a green star (details in Discussion). The open circles correspond to independent ex situ stellar mass measurements for two ETGs, NGC 1380 and NGC 1427, which are members of the Fornax galaxy cluster, using population-orbital decomposition methods in combination with the cosmological simulation TNG50 from ref. 43. The red shaded region corresponds to an independent measurement of mass from major mergers for massive galaxies from ref. 41. It is apparent that the in situ stellar mass dominates across the stellar mass range. The ex situ stellar mass becomes substantial only for massive galaxies, which reveals the strong dependency of the level of accretion on the stellar mass. Our measurements are roughly compatible with completely independent measurements in the literature.

reserved from the MaNGIA sample during training. By applying the verified model to the MaNGA galaxies, we were able to acquire predictions for the amount of accreted versus in situ stellar mass across the stellar mass range in the nearby Universe.

In Fig. 1, we plot the mean in situ stellar mass (solid blue line) and ex situ stellar mass (solid red line) for ~10,000 MaNGA galaxies. The in situ stellar mass dominates across the lower and intermediate stellar mass range, with the ex situ stellar mass becoming substantial only at the most massive end. More specifically, the contribution of mergers to the stellar mass assembly of low-stellar-mass galaxies ($M_* < 10^{10.5} M_\odot$) is nearly negligible, and the entire stellar mass is attributed to in situ star formation. At higher stellar masses ($M_* > 10^{11} M_\odot$), the impact of mergers becomes more relevant and the mean value of the ex situ stellar mass exceeds $10^{11} M_\odot$ for the most massive galaxies ($M_* > 10^{11.5} M_\odot$). This indicates that, for such galaxies, the stellar mass originating from mergers constitutes up to 35–40% of their total stellar mass. Our results are in relatively good agreement with previous works for our own Galaxy, the Local Group and galaxies at the massive end of the stellar mass range, as we will discuss later.

Overall, we found a pronounced dependency of the amount of ex situ stellar mass on the stellar mass of a galaxy. Consequently, the more massive a galaxy is, the stronger the effect of mergers is on its
<figure><img src="image_4.png" /><figcaption>**Fig. 2 | Secondary dependencies of the ex situ stellar mass on galaxy characteristics.** a,b, For comparison with the global relation, the mean stellar mass originating from accretion and mergers across the stellar mass range is drawn with a solid black line in both panels. a, The dependency of the average ex situ stellar mass on galaxy morphology. At a fixed stellar mass, elliptical galaxies</figcaption></figure>

(red dashed line) have more accreted stars than SO galaxies (yellow dashed line) and substantially more than spiral galaxies (blue dashed line). **b**, Dependency of ex situ stellar mass on star formation. At a fixed stellar mass, quenched galaxies (red dashed line) have accreted a higher fraction of their stellar mass than star-forming galaxies (blue dashed line). The error bars contain 68% of the data.

stellar mass assembly. This conforms with the hierarchical build-up scenario in which mergers play an important role in the mass assembly of the most massive galaxies¹⁴,²⁸. We emphasize that this dependency was revealed even though no information about the total stellar mass was provided to the model and any global information was removed by normalizing all input images individually during preprocessing, so that the model focused on the radial gradients²⁵. The reported correlation additionally exhibits a considerable scatter, suggesting that secondary properties apart from the stellar mass also correlate with the integrated merger history of a galaxy, as we investigate in the next section.

## Ex situ stellar mass dependencies

We aimed to explain the observed scatter in the relation between accreted stellar mass and total stellar mass by investigating secondary dependencies of the ex situ stellar mass on galaxy characteristics. Specifically, we examine trends for morphology, star-formation activity, rotation and environment. These properties have already been measured for MaNGA galaxies (Methods), allowing us to analyse the ex situ stellar mass dependency within specific subgroups across the stellar mass range. We emphasize here that no prior knowledge of these characteristics was provided as an input to the neural network during training.

In Fig. 2, the mean ex situ stellar mass is plotted for different morphological types and star-formation states. The black solid line represents the mean ex situ stellar mass fraction of the entire MaNGA sample, whereas the coloured dashed lines correspond to the different subgroups. Our findings reveal that secondary dependencies of the ex situ stellar mass indeed exist with morphology and star formation. We found that, on average, elliptical galaxies have accreted more stars than their spiral counterparts at a fixed stellar mass. Interestingly, SO-type galaxies lie between the two other morphological groups, aligning with their expected evolutionary stage and morphological type. A similar pattern emerged for star-formation activity, with quenched galaxies exhibiting higher ex situ stellar mass than star-forming galaxies at a fixed stellar mass. These secondary dependencies were revealed by comparing galaxies at the same stellar mass bins, thus effectively removing the predominant stellar mass dependency. Also note that, although there is considerable overlap in the observed scatter between the different subgroups, the mean values of ex situ stellar mass consistently separate the distinct classes across almost the entire stellar mass range.

We also found that slow rotators²⁹ have accreted a higher fraction of their stellar mass, on average, than fast rotators (Fig. 3). Although our statistics for slow rotators are not extensive, especially at lower masses, their mean ex situ stellar mass consistently surpasses the global relation across all stellar mass bins. Given the limited size of our sample of slow rotators, we also opted to investigate peculiar objects, namely galaxies classified as fast rotators but exhibiting an unexpectedly high ex situ stellar mass fraction (>0.5). Such objects could be anomalies due to erroneous processing or genuinely intriguing objects that have managed to maintain their fast rotation even after substantial merger events. We identified ~40 galaxies with these characteristics (highlighted as orange and black stars in Fig. 3). Upon closer examination, we found that a notable proportion of these indeed appear to be fast rotators. They are compact galaxies with a substantial bulge (left-hand SDSS images in Fig. 3). To further examine these interesting objects, deeper observations might be required. Additionally, we also identified continuing merger events in the outlier data (right-hand SDSS images in Fig. 3), pointing to an erroneous calculation of their rotational properties. Notably, the specific objects have also been flagged as unclean in their fast-rotator classification, indicating an extra visual categorization as merger events.

Finally, we aimed to investigate the impact of the environment on the merger history of galaxies (Fig. 4). Previous studies have reported correlations between the stellar population properties and the mass of the host dark matter halo that a galaxy resides in³⁰⁻³². This prompted us to explore whether such correlations extend to the merger history of galaxies. Using the self-calibrating halo-based galaxy group finder on SDSS from ref. 33, we found that galaxies in the centre of more massive dark matter haloes have accreted more stellar mass than their counterparts with a similar total stellar mass but in the centre of less massive dark matter haloes (isolated galaxies were removed from this analysis to make the halo mass estimates more reliable). This result signifies that the integrated merger history of a galaxy is correlated not only with its stellar mass but also with the properties of its host dark matter halo. Interestingly, we found that this dependency was not present for satellite galaxies (Fig. 4b).

## Discussion

We calculated physics-based measurements of the integrated impact of mergers in the nearby Universe for a statistically significant sample of galaxies. We did this by taking advantage of the power of machine learning to determine the relative contributions of the ex situ and in situ
<figure><img src="image_5.png" /></figure>

**Fig. 3 | Dependencies of the ex situ stellar mass on rotation.** For comparison with the global relation, the mean stellar mass originating from accretion and mergers across the stellar mass range is drawn with a solid black line. At a fixed stellar mass, slow rotators (red dashed line) exhibit a higher ex situ stellar mass than their fast-rotator counterparts (blue dashed line). The error bars contain

68% of the data. The star points correspond to outliers, galaxies classified as fast rotators with a predicted fraction of ex situ stellar mass >0.5. We plot six SDSS images of the outlier sample and identify several intriguing actual fast rotators (left-hand images) as well as galaxies undergoing a merger event (right-hand images). The image labels correspond to the MaNGA ID of each galaxy.

<figure><img src="image_6.png" /></figure>

**Fig. 4 | Dependencies of the ex situ stellar mass on environment.** a, **b,** For comparison with the global relation, the mean stellar mass originating from accretion and mergers across the stellar mass range is drawn with a solid black line in both panels for central and satellite galaxies respectively. **a,** Dependency of the ex situ stellar mass on the dark matter halo mass for central galaxies. At a

<figure><img src="image_7.png" /></figure>

fixed stellar mass, galaxies at the centre of more massive haloes (red dashed line) have accreted a higher fraction of their stellar mass. **b,** Dependency of the ex situ stellar mass on the dark matter halo mass for satellite galaxies. The fraction of ex situ stellar mass in the satellite galaxies is not impacted by the dark matter halo mass at a fixed stellar mass. The error bars contain 68% of the data.

stellar mass components to the total stellar mass of galaxies using IFU MaNGA data. Our findings reveal that galaxies with a low stellar mass ($M_{*} < 10^{10.5} M_{\odot}$) mainly comprise in situ stars with an almost negligible ex situ component. On the other hand, we found that the impact of mergers is more prominent for galaxies at higher stellar masses ($M_{*} > 10^{11} M_{\odot}$), for which the ex situ stellar mass component can reach up to 35–40% of the total stellar mass.

Our findings allow us to put constraints on the integrated amount of stellar mass originating from in situ star formation in comparison to the component accreted from mergers and interactions in the local Universe. By integrating the contributions of the in situ and ex situ stellar masses across our MaNGA sample, we found that ~70% of the total stellar mass in the nearby Universe was attributed by our model to in situ star formation and only ~30% was attributed to accretion through mergers and interactions. In a volume-limited sample, attained

with volume corrections as reported in ref. 34, the fraction of ex situ mass drops even further to ~19%. This result greatly underlines that, although the contribution of mergers to the mass assembly of galaxies is important, it remains a secondary factor in stellar mass growth in comparison to in situ star formation. Notably, our findings fall partially short of predictions from simulations. For example, the Illustris cosmological simulation estimated a global ex situ stellar mass fraction of 30% at redshift $z=0$ (ref. 35) and the semi-analytical model GALFORM predicted a corresponding value of 40% (ref. 36).

The constraints set in the current work are in relatively good agreement with estimates for our own Galaxy and the local neighbourhood. In particular for the Milky Way, the component primarily attributed to the accretion of stars from satellite galaxies is identified as the Milky Way halo<sup>37,38</sup>, which is estimated to have a stellar mass of $-1.3 \times 10^9 M_{\odot}$ (ref. 38). This serves as a lower limit for the amount of accreted mass
in the Milky Way and is below the mean relation of ex situ stellar mass recovered for MaNGA galaxies (orange star in Fig. 1). As a speculative future scenario, we included another point (green star) to represent the rough projected ex situ stellar mass estimate when the Milky Way eventually merges with its most massive satellite, the Large Magellanic Cloud, with a calculated stellar mass of $-1.5 \times 10^9 M_\odot$ (ref. 39). Interestingly, this projection aligns more closely with our mean ex situ stellar mass relation. However, it is important to note that the Milky Way is believed to have had a rather atypical assembly history given its mass$^{40}$. We also plotted the estimated halo mass of M31 (ref. 11). It lies on our mean ex situ stellar mass relation, so that it is consistent with the constraints found in the current work.

Moreover, further validation of our results at the high end of the stellar mass range was obtained by a comparison with findings from studies measuring the rates at which galaxies merge at various redshifts$^{41,42}$. Such works offer a completely independent observational measurement of the ex situ stellar mass that is based on calculating the fraction of galaxies found in close pairs in extended survey samples. By tracking the number of merger events per unit co-moving volume for redshift $z < 3$, one can integrate the total accreted stellar mass in this redshift range when the average stellar mass of the infalling galaxy is known. Specifically in ref. 41, the authors found that for massive galaxies ($M_* > 10^{11} M_\odot$), the amount of ex situ stellar mass attributed to major merger events for redshift $z < 3$ could be approximated as $1.48^{+0.78}_{-0.49} \times 10^{11} M_\odot$ (plotted in Fig. 1 as a shaded red region). This is in relatively good agreement with our results in the high end of the stellar mass range, taking all the uncertainties involved in the calculations under consideration. Additionally, an independent ex situ stellar mass measurement for two external ETGs, NGC 1380 and NGC 1427, which are members of the Fornax galaxy cluster, has been reported using population-orbital decomposition methods in combination with the TNG50 cosmological simulation$^{43}$. These measurements also align closely with our relation (plotted in Fig. 1 as open circles).

Diving further, we found a very prominent dependency of the amount of accreted stellar mass on the total stellar mass of the galaxy. This generally agrees with results from cosmological simulations$^{35,44–46}$, zoom-in simulations$^{28}$ and semi-analytical models$^{47}$. Across these studies, there is a consensus that the more massive a galaxy is, the higher the fraction of its stellar mass originated from accretion and mergers. As in this work, simulations have predicted almost minimal ex situ stellar mass for galaxies with stellar mass $M_* < 10^{10.5} M_\odot$ and a steep increase of the accreted stellar mass fraction for stellar masses above $10^{10.5} M_\odot$. However, apart from ref. 44, all these models predicted very high ex situ stellar mass fractions of the order of -80% for the most massive galaxies ($M_* > 10^{11.5} M_\odot$), levels that were not reached in our results. This discrepancy may be because our MaNGIA training sample originated from the limited-volume TNG50 cosmological simulation. Although TNG50 provides finer resolution, it is confined to a smaller simulation box, resulting in massive galaxies lying, on average, in lower-mass haloes compared to simulations of a larger volume. However, as our MaNGIA sample is matched to MaNGA, with a selection criterion that also does not favour central galaxies in very massive haloes$^{34}$, we consider this limitation acceptable, and so we focus primarily on the statistical interpretation of the results.

Apart from the strong dependency of ex situ stellar mass on the total stellar mass of a galaxy, we additionally observed that the said relation exhibits a notable scatter. In attempting to explain it, we found that the amount of stellar mass originating from mergers in a galaxy has secondary dependencies on morphology, star-formation, rotation and environment. More specifically, we found that elliptical galaxies have accreted a higher fraction of their stellar mass than spiral galaxies at a fixed stellar mass. This was not a surprising result, as merging is known to be a possible mechanism for creating elliptical galaxies$^{2,14,48}$ and similar trends have also been produced in cosmological simulations$^{46,49}$. Additionally, we found that galaxies with quenched star formation have

accreted a greater fraction of their stellar mass, on average, than their star-forming counterparts. This result aligns with previous studies that attribute fast quenching to gas-rich mergers, which can induce intense bursts of star formation and trigger activity by active galactic nuclei, thus depleting galaxies of their gas content$^{50,51}$.

We identified another secondary dependency of the ex situ stellar mass on galaxy rotation. Despite the limited sample of slow rotators in this study, our analysis revealed that slow rotators exhibit a higher ex situ stellar mass component than fast rotators at a fixed stellar mass. This result agrees with previous studies that attribute disturbed rotational properties to merger events$^{52}$. Imprints of mergers and interactions, such as tidal features, have been shown to prevail in slow rotators in comparison to fast rotators, particularly ETGs$^{53}$. Moreover, our approach may serve as a valuable tool for detecting anomalies in data, thus enabling the identification of rare objects, such as fast-rotating galaxies with unusually high ex situ stellar mass components.

Finally, we attempted to assess the impact of the environment and, in particular, the host dark matter properties on the merger history of a galaxy to gain further insights into the interplay between the two. Interestingly, we found that the contribution of ex situ stellar mass to the total mass is higher for central galaxies at the centre of more massive dark matter haloes than their counterparts in less massive haloes. Intriguingly, this trend is not mirrored for satellite galaxies, suggesting that their merger history was not equally influenced by their host halo properties. Previous works have found relations between environment and merger rates, especially dry mergers in over-density regions$^{54}$, but the dependence of merger rates on the halo mass is not conclusive$^{55}$. Notably, a recent study$^{12}$ identified a tight relation between the number of satellites and the accreted mass for several galaxies in our local neighbourhood, further supporting our results.

Overall, this work provides theory-motivated observational constraints on the balance of in situ stellar mass versus ex situ stellar mass in the mass assembly of galaxies in the nearby Universe for a statistically significant sample. We found that the impact of mergers on a galaxy's stellar assembly, although secondary overall in comparison to in situ star formation, is more prominent at the high mass end of galaxies and less important for those with low and intermediate stellar masses. Our findings additionally reveal secondary dependencies of the fraction of accreted stellar mass on galaxy morphology, star-forming activity, rotational properties and the galaxy's environment throughout the entire stellar mass range. Looking ahead, the upcoming high-redshift surveys, combined with the computational capabilities of machine learning, hold the potential to unravel the balance between in situ and ex situ stellar mass evolution across cosmic time.

## Methods

### Overall approach

The goal of the present work was to acquire estimates for the contribution of in situ stellar mass versus ex situ stellar mass for an observational sample of the local Universe that is statistically meaningful. As the effects of mergers (ex situ stellar mass) are neither directly observable nor straightforward to obtain from a galaxy's observational properties, we resorted to cosmological simulations to gain a view of the merger histories of nearby galaxies. We trained a deep learning model to predict the fraction of ex situ stellar mass from observable spatially resolved maps of simulated galaxies. The first challenge that arises in this approach is that training on a particular cosmological simulation can induce bias in the predictions of the model, which can be carried over to the application of the model to real galaxies. Moreover, observational data contain inherent observational effects and noise that are absent in idealized simulated data, which complicates the direct application of a model trained on raw simulation data to observational images.

To address these issues, our overall approach was as follows. We used two state-of-the-art cosmological simulations instead of one to cross-train our neural network model, so that we could identify a set
of inputs that was a robust predictor of the ex situ stellar mass fraction across the distinct subgrid physics that govern each simulation. In our previous work²⁵, we showed that the gradients of the spatially resolved maps of stellar mass, stellar velocity and stellar velocity dispersion are accurate and robust predictors of the global ex situ stellar mass fraction of nearby galaxies across the TNG100 and the EAGLE cosmological simulations. However, the trained models were still missing the realism required for application to real galaxies. As our goal was to investigate statistically the impact of mergers on the mass assembly of galaxies in the local Universe, we employed the machine learning model on MaNGA²⁶, which is at present the largest integral-field spectroscopic survey with data for ~10,000 nearby galaxies. To train the aforementioned model, we required a mock dataset derived from simulations with a realism matched to MaNGA and which was also statistically significant. Thus, we turned to MaNGIA, the largest mock dataset to date. MaNGIA was designed to mimic the MaNGA survey. By training and validating the predictive capabilities of the machine learning model on MaNGIA, we were confidently able to apply it to MaNGA galaxies and obtain a first prediction of the ex situ versus in situ stellar mass contribution to the total stellar mass.

The inputs to the machine learning model were the robust IFU maps of stellar mass density and kinematics. The output was a prediction of the ex situ stellar mass fraction for each galaxy. This selection of inputs was derived from the cross-training across TNG100 and EAGLE in the initial step of our methodology. As MaNGIA originated from the TNG50 cosmological simulation, we expected these inputs to generalize well when applied to both mock and real data. The ground truth of the ex situ stellar mass was available for MaNGIA, as the stellar assembly catalogues have been resolved for the TNG50 simulation⁴⁹. These catalogues provide, among other things, a measurement of the ex situ stellar mass for each galaxy. This measurement was realized by traversing the merger tree of a galaxy and classifying all stars formed in other galaxies other than the main progenitor branch (which either merged or interacted with the main galaxy) as accreted. Note that during preprocessing, we normalized all input IFU maps individually, essentially removing any global information present. Thus, the input images contained only the gradients of the maps and were stripped of all mass dependencies. The model was then trained to learn relations between the gradients and the global ex situ stellar mass fractions.

## MaNGA

MaNGA²⁶ is the integral-field spectroscopic survey of SDSS-IV. It was designed to systematically map the kinematics and spatially resolved spectroscopic properties of nearby galaxies. MaNGA provides spatially resolved spectra for over 10,000 galaxies with stellar masses in the range $10^9 M_{\odot} < M_{*} < 10^{12} M_{\odot}$. The survey covers redshifts in the range 0.01 < z < 0.15 (with a median $z \approx 0.03$) for a wavelength range 3,600–10,300 Å at a resolution of $R \approx 2,000$. Each galaxy was observed using fibre bundles with diameters that varied from 12 inches (19 fibres) to 32 inches (127 fibres). The complete MaNGA sample consists of three subsamples: the primary sample, which has optimized spatial coverage out to $1.5 R_{e}$ (-50%), where $R_{e}$ is the effective radius of the galaxy; the secondary sample, which extends the radial coverage to $2.5 R_{e}$ (-33%); and the colour-enhanced subsample, which supplements the primary sample in poorly sampled regions of the near ultraviolet–/ versus $M_{i}$ colour–magnitude plane (-17%)³⁴. MaNGA provides a statistically significant sample of galaxies across diverse stellar masses and environments. To move to a volume-complete sample, we used the volume-corrected weights provided by ref. 34. In the present work, the stellar mass measurements assumed were obtained from the NASA–Sloan Atlas, which was derived through K-correction fitting for elliptical Petrosian fluxes. We additionally applied a correction to the stellar masses in the NASA–Sloan Atlas using the cosmological parameter $H_{0} = 100 h \text{ km s}^{-1} \text{ Mpc}^{-1}$ to match the simulations. The h constant

assumed in the NASA–Sloan Atlas $h=1$, whereas the value used in the simulations corresponds to that used by the Planck Collaboration 2015⁶, $h = 0.6774$. The correction applied (in logscale units of solar masses) was the following: $M_{\text{corr}}^{*} = M^{*} - 5 \log_{10}(h)/2.5$.

## MaNGIA

MaNGIA²⁷ is a forward-modelled sample of galaxies from the TNG50 simulation⁵⁷,⁵⁸, the highest-resolution simulation of the IllustrisTNG suite. The simulated galaxies from TNG50 were chosen to match MaNGA galaxies in stellar mass, size and redshift and were forward-modelled so that a mock MaNGA-like data cube was constructed for each simulated galaxy. This was done with an extended processing pipeline, which included generating mock spectra from the simulation particles, replicating the fibre bundle configuration of MaNGA and adding observational effects, such as atmospheric seeing and noise. Afterwards, the resolved mock data cubes were analysed with the PIPE3D pipeline⁵⁹,⁶⁰ to produce mock stellar population and kinematic maps. The PIPE3D analysis code was used for the official MaNGA release, thus allowing a direct comparison of the mock data sample with observations. MaNGIA successfully reproduced the overall trends for metallicity, age and kinematics observed by MaNGA, rendering it an ideal sample for machine learning applications, such as the one presented in the current work. Note that, in addition to the publicly available MaNGIA galaxy sample, we augmented our training set with another sample of ~1,600 MaNGIA galaxies, which are not yet included in the public dataset.

## Neural network model architecture

Our aim was to infer the ex situ stellar mass fraction of galaxies from two-dimensional spatially resolved maps of stellar mass density, stellar velocity and velocity dispersion. As our training dataset (MaNGIA) was derived from a simulation for which the ground truth was available, a supervised machine learning approach was considered to be suitable for this task. However, MaNGIA was not created with the current application in mind. As a result, the original dataset is very unbalanced in terms of the distribution of the ex situ stellar mass fractions. More specifically, the vast majority of the mock galaxies in MaNGIA have a very low ex situ stellar mass fraction and only a handful of galaxies have an accreted mass fraction higher than 40%. This may cause a traditional supervised neural network to predict only biased low values. To circumvent this issue, we adopted a self-supervised architecture.

Self-supervised learning is a machine learning approach that is becoming increasingly popular in astrophysical applications. It can extract meaningful representations from unlabelled data by solving pretext tasks. These representations can then be used for a variety of downstream tasks. We selected a self-supervised learning approach because such approaches have been shown to help with unbalanced data by providing representations that are more robust to dataset imbalance than supervised learning⁶¹. We decided to use the bootstrap-your-own-latent (BYOL) self-supervised learning approach⁶², which has been shown to extract robust representations even from limited samples. The BYOL model uses two neural networks with the same architecture that receive different augmented views of the same input image and output a representation in the embedding latent space. The network is trained to minimize the distance between the two embeddings ($z_i$ and $z_i^+$), effectively learning robust representations of the images that are invariant to the augmentations applied. To assist our model in capturing relevant features in the data that correlate with the ex situ stellar mass fraction, we additionally added a regressive head ($q_i$) to the self-supervised network during training. This was designed to learn the accreted stellar mass fractions from the representations. As a result, the loss function of the model is a combination of the BYOL loss term and the regression loss (mean-squared error), weighted with a value of $\lambda$ (in our case $\lambda = 1$). The parameters $Y_i$ and $n$
correspond to the true values of the ex situ fractions for each object in the sample and the size of the sample, respectively:

$$ \mathcal{L} = \mathcal{L}_{\text{BYOL}} + \lambda \mathcal{L}_{\text{regr}} = 2 - 2 \frac{\langle z_i, z_i^+ \rangle}{\| z_i \|_2 \times \| z_i^+ \|_2} + \lambda \frac{1}{n} \sum (Y_i - q_r(z_i))^2. \quad (1) $$

To enhance the performance of the model, we artificially balanced our training dataset before training by including simple augmentations in the data that were specifically targeted for galaxies with high ex situ stellar mass fractions. The said augmentations include random cropping, resizing, and horizontal and vertical flipping. The model architecture was implemented in Keras with a TensorFlow back end. We trained our models on an NVIDIA Tesla P100 GPU to optimize the execution time.

## Uncertainties of the model

Before applying the trained model to the MaNGA dataset, we executed a set of validation steps to ensure the network could recover the MaNGIA mock dataset and to calibrate the model's uncertainties. First, we adopted an ensemble approach in an attempt to reduce epistemic uncertainties and enhance the reliability of the predictions of the model on MaNGA. More specifically, instead of depending on a single trained model, we trained five neural networks, each with the architecture detailed in the previous section but using distinct weight initializations. Employing this ensemble strategy enabled us to combine the predictions from the five models for each galaxy, thus effectively reducing the uncertainty and yielding more robust predictions. We found that this approach indeed increased the accuracy on the unseen test dataset from MaNGIA, for which the ground truth was available so that the accuracy was measurable. The left-hand panel of Supplementary Fig. 1 shows the recovery of the ex situ stellar mass fractions for the MaNGIA test set, along with the assembled predictions of the models for the whole MaNGA sample. The predictions for MaNGIA closely trace the ground truth, thus enhancing our confidence in the results recovered for MaNGA.

Moreover, an essential consideration for evaluating the robustness of a machine learning model when applied to real galaxies involves providing a measure of uncertainty for the model predictions. We addressed this crucial aspect through a second step of training. In this step, we fine-tuned the neural network to predict a probability distribution instead of a single value for the ex situ stellar mass fraction of each galaxy. This involved freezing the representations learned from the self-supervised training and exclusively training the model to effectively map these representations to a posterior distribution of the ex situ stellar mass fraction. By sampling from the posteriors of the five fine-tuned models for each galaxy, we could ensemble the predicted distributions from all samples and utilize the produced standard deviation as a confidence measurement of the models for each prediction.

To establish the reliability of the standard deviation as a measure of uncertainty, we conducted a statistical coverage test on the models' predictions for the MaNGIA test set. This test assessed the accuracy of the predicted posteriors by determining the percentage of times the error fell within a certain percentage of the posterior width. Ideally, this percentage should align with the specified width (for example, 68% of galaxies should have a prediction error $\le 1\sigma$ for a 68% probability volume). Our findings indicated that our models exhibited close-to-perfect calibration while being slightly conservative, as can be seen in the right-hand panel of Supplementary Fig. 1, which underlines that they indeed provided meaningful measures of uncertainty. As the ground truth was not available for MaNGA galaxies, the same coverage test could not be applied to them. However, we found that the uncertainty measurements that the models provided for MaNGIA and MaNGA (top of the right-hand panel of Supplementary Fig. 1) are comparable, with MaNGA exhibiting slightly higher uncertainties, as was expected. Note that after fine-tuning the models, a handful of

galaxies in MaNGA had predicted distributions that lie outside the prior space for ex situ stellar mass fractions (negative mean values). However, by removing these galaxies and reapplying the statistical analysis of the current work, we found that the same trends arose for galaxy characteristics and environment. Thus, these discrepancies did not highly affect the produced results and might even indicate possible outliers in the observations that are not sufficiently covered in the simulated space.

## Implications from the ex situ definition

An important consideration that needs to be addressed in our work is the definition of in situ stellar mass versus ex situ stellar mass in the training set. As MaNGIA originates from the IllustrisTNG (TNG50) cosmological simulation, the ground truth provided during training for the ex situ stellar mass was carried over from that simulation. More specifically, the definition of the ex situ stellar mass fraction was derived from ref. 35, in which the in situ stars are considered to be all particles formed in galaxies that lie along the main progenitor branch of the galaxy in which they presently reside. All stellar particles formed outside the main progenitor branch are considered to be ex situ stars. The said definition results in the primary growth of ex situ stellar mass predominantly occurring at $z=2$ and beyond in the Illustris cosmological model$^{35}$. However, this may be at odds with observations suggesting that massive ETGs experience most of their stellar mass and size growth at lower redshifts ($z<2$)$^{63-65}$. Nevertheless, in the IllustrisTNG simulations, specifically for massive galaxies, the relative contribution of the ex situ fraction to the total stellar mass also exhibits an increasing trend at $1<z<2$ (ref. 35). This trend could increase the ex situ stellar mass fraction from 20% to 50% for galaxies with a stellar mass of $M_* \approx 10^{11} M_\odot$ (ref. 46), thereby it is more consistent with observational results within this stellar mass regime.

## Robustness of the predictions

Although simulation-based inference is key for predicting unobservable properties of galaxies, it can be affected by systematic uncertainties from using a specific simulated dataset for training, which raises concerns about the model's generalizability. Assessing our model's robustness required us to examine it at several levels, each with potential sources of bias: the raw simulation space, the mock space and the observational space.

Focusing initially on the raw simulation space, the primary source of bias originated from the particular subgrid physics of the training simulation, as the model may have learned patterns tied to the underlying galaxy-formation model that do not generalize to different subgrid physics. To address this issue, we built upon our recent work$^{25}$, which identified a set of inputs robustly predicting the ex situ stellar mass fraction across two cosmological simulations. Specifically, we found that IFU-like maps of stellar mass, stellar velocity and velocity dispersion can accurately predict the ex situ stellar mass fraction across the TNG100 and the EAGLE cosmological simulations, whereas other stellar population properties, like age and metallicity, induce bias. Demonstrating this robustness, the left-hand panel of Supplementary Fig. 2 illustrates the recovery of the ex situ versus stellar mass relation for a test set from the EAGLE cosmological simulation at $z=0$, as predicted by a neural network trained on TNG100. This shows that the neural network model generalizes well to different subgrid physics and, therefore, is not anchored to a specific model. Thus, as an initial step towards generalization, we strategically limited the inputs supplied to the neural network in the current work to this robust subset of IFU maps.

The process of mocking, that is, translating simulation outputs to the observational space, also requires assumptions that can introduce systematic uncertainties. To properly quantify this effect, we should ideally perform similar cross-training of the neural network model with different mock datasets and validate the inferred results
on the MaNGA sample. However, this requires at least two different mock samples, each with realism matched to MaNGA and originating from distinct cosmological simulations. Unfortunately, apart from MaNGIA (which originated from TNG50), there was no comparable mock dataset from a different cosmological simulation, and creating one from scratch is a very computationally expensive task and beyond the scope of the current work. Nonetheless, as the MaNGIA mocking pipeline is publicly available, we decided to create a limited mock dataset from the EAGLE cosmological simulation solely for testing. We produced a limited sample of ~150 mock galaxies at $z=0.1$, spanning a stellar mass range of $10^{10} M_{\odot} < M_{*} < 10^{11.5} M_{\odot}$. We then applied the MaNGIA-trained model on the EAGLE mock sample. The results are plotted in the middle panel of Supplementary Fig. 2. We can see that the model satisfactorily reconstructed the stellar mass versus ex situ fraction relation, despite being trained on a higher-resolution cosmological box and different subgrid physics. Although there is an offset of about 5% between the true and predicted relations, the model is capable of closely recovering the slope of the true relation, even though it is distinctly different from the one that it was trained on. This is a reassuring result, as it indicates that the model does not merely project trends from the TNG50 simulation but has indeed learned generalized mappings between the radial gradients and the ex situ stellar mass fractions.

Note that the offset between the true and the predicted relations for the EAGLE sample was observed only in the mock space and not the raw simulation space. This can partially be attributed to the different resolutions of MaNGIA (TNG50) and EAGLE/TNG100 and to the mocking procedure itself. Specifically, calculating a projected measurement of galaxy size in the three-dimensional simulation space equivalent to the half-light radius used for the fibre recombination in MaNGA is challenging and can affect the noise in the resulting mock IFU maps. For the MaNGIA mock dataset, the authors of ref. 27 used the radius of a sphere containing half of its stellar mass ($R^{3D*}$) and fitted it to the observed radius from synthetic images of TNG50 galaxies$^{66}$, using the semimajor axis of the ellipse containing half of the stellar light ($R^{Petro}_{0.5}$). We used the same fit for the EAGLE mock sample, but this conversion may not be optimal for EAGLE galaxies, as it affects the level of noise in the produced maps and, thus, contributes to the offset observed in the mock space.

Finally, reaching the observational space, we examine the impact of the processing pipeline used to recover the MaNGA IFU maps on the predictions of the model. For that, we compare the inferred ex situ stellar mass fractions recovered from the PIPE3D maps with those recovered using IFU maps from a different pipeline. Specifically, we used the products from the Firefly analysis code$^{67}$ to acquire stellar mass density maps and the MaNGA data-analysis pipeline$^{68}$ for the stellar kinematic maps, as an alternative to the PIPE3D maps. In the right-hand panel of Supplementary Fig. 2, we plot with a blue solid line the median of the difference between the predictions acquired for each galaxy in MaNGA when the distinct pipelines were used to resolve the IFU input maps across the stellar mass range. We found that the models can robustly predict the ex situ stellar mass fraction of a galaxy independently of the processing pipeline used to derive the stellar maps. There is an increase in the difference between the predictions for the most massive galaxies, but that is accompanied with the higher uncertainty produced by the models (green error bars). This strengthened our confidence in the predicted results. As a final validation test, we also attempted to train the neural network on a different mock MaNGA dataset (iMaNGA$^{69}$) so that we could make a direct comparison of the inferred results, but the sample's limited size (~1,000 galaxies) did not allow the model to be trained effectively.

## MaNGA resolved properties and environment

To unveil the secondary dependencies of the ex situ stellar mass, we used properties that have already been measured for the MaNGA survey

in previous works. For the morphological type of each galaxy, we used the MaNGA PyMorph morphological catalogue$^{70}$, which provides deep-learning-based morphological classifications for SDSS DR17. We separated the MaNGA galaxies into 2,550 ellipticals, 966 S0-type galaxies and 5,428 spiral galaxies using the following recommended restrictive selection:

* Spiral galaxies: T-type > 0 and $P_{LTG} \ge 0.5$ and VC = 3

* S0 galaxies: T-type $\le 0$ and $P_{S0} > 0.5$ and $P_{LTG} < 0.5$ and VC = 2

* Elliptical galaxies: T-type $\le 0$ and $P_{S0} \le 0.5$ and $P_{LTG} < 0.5$ and VC = 1

where VC corresponds to the visual classification (VC = 1 for ellipticals, VC = 2 for S0-type galaxies, VC = 3 for late-type galaxies (LTGs) and irregulars, and VC = 0 for unclassifiable galaxies), T-type is a number assigned to each classification and $P_{\text{class}}$ corresponds to the probability that a galaxy belongs to that class as output from the machine learning model.

To classify the MaNGA galaxies by their star-formation activity as quenched or star-forming, we used the PIPE3D analysis$^{71}$ applied to MaNGA DR17. We obtained the integrated star-formation rate derived from the Hα emission line from the said analysis, and we separated our galaxies into star-forming and quiescent on the star-formation rate versus stellar mass plane. In Supplementary Fig. 3a, the classification cut used in the current work is shown with a black solid line. The points are additionally colour-coded with the resolved ex situ stellar mass fraction, which reveals the secondary dependency of the accreted stellar mass to the star-formation activity of the galaxy. This separation criterion produced a sample of 5,696 star-forming and 4,519 quenched galaxies.

In terms of rotation, we used the angular momentum proxy $\lambda_{\text{Re}}$ and the ellipticity $\epsilon$ measurements for about 2,300 MaNGA galaxies provided by ref. 72. Plotting these galaxies in the angular momentum versus ellipticity plane (Supplementary Fig. 3b), we employed the classification criterion proposed by ref. 29 to distinguish slow and fast rotators. Following this criterion—labelling galaxies with $\lambda_{\text{Re}} < 0.08 + \epsilon$ and $\epsilon < 0.4$ as slow rotators and the remainder as fast rotators—we identified a subset of 268 slow rotators and 2,520 fast rotators. Notably, this plane also exhibits a discernible trend of ex situ stellar mass with angular momentum, as indicated by the colour-coded representation in the figure.

Finally, for a measurement of the host halo environment properties, we turned to the self-calibrating halo-based group finder applied on SDSS by ref. 33. This catalogue provides, among other things, the likelihood ($P_{\text{sat}}$) of each galaxy being a satellite, the dark matter halo mass ($M_{\text{halo}}$) and the number of satellites within the host halo ($N_{\text{sat}}$) for central galaxies. We used a relatively strict criterion to separate our MaNGA galaxies into centrals ($P_{\text{sat}} < 0.1$) and satellites ($P_{\text{sat}} \ge 0.9$). Additionally, we removed isolated galaxies from our sample (central galaxies that have no satellites) to ensure that the halo mass estimates were more reliable. We ended up with 2,234 central galaxies and 1,508 satellite galaxies. The range of the halo masses provided by the group catalogue for the sample of central galaxies is $10^{11} M_{\odot} < M_{\text{halo}} < 10^{15} M_{\odot}$, peaking at $M_{\text{halo}} \approx 10^{13} M_{\odot}$. The majority of central galaxies in this analysis reside in Milky Way-sized haloes and groups, whereas a small subsample live in clusters (~150 galaxies with $M_{\text{halo}} > 10^{14} M_{\odot}$).

## Data availability

Data directly related to this publication and its figures are available from the Strasbourg Astronomical Data Centre through anonymous ftp to cdsarc.u-strasbg.fr (130.79.128.5) or through the link https://cdsarc.cds.unistra.fr/viz-bin/cat/J/other/NatAs. The MaNGA products (https://www.sdss4.org/dr17/manga/) and the MaNGIA dataset (https://www.tng-project.org/data/docs/specifications) are all publicly available. The associated datasets are available via GitHub at https://github.com/eagel27/galaxy-exsitu.
## Code availability

The associated code and machine learning models are available via GitHub at https://github.com/eagel27/galaxy-exsitu.

## References

1. Somerville, R. S. & Davé, R. Physical models of galaxy formation in a cosmological framework. Annu. Rev. Astron. Astrophys. 53, 51-113 (2015).

2. Kormendy, J., Fisher, D. B., Cornell, M. E. & Bender, R. Structure and formation of elliptical and spheroidal galaxies. Astrophys. J. Suppl. Ser. 182, 216-309 (2009).

3. Patton, D. R., Ellison, S. L., Simard, L., McConnachie, A. W. & Mendel, J. T. Galaxy pairs in the Sloan Digital Sky Survey. III. Evidence of induced star formation from optical colours. Mon. Not. R. Astron. Soc. 412, 591-606 (2011).

4. Ellison, S. L., Patton, D. R., Simard, L. & McConnachie, A. W. Galaxy pairs in the Sloan Digital Sky Survey. I. Star formation, active galactic nucleus fraction, and the mass-metallicity relation. Astron. J. 135, 1877-1899 (2008).

5. Bell, E. F. et al. Dry mergers in GEMS: the dynamical evolution of massive early-type galaxies. Astrophys. J. 640, 241-251 (2006).

6. Madau, P. & Dickinson, M. Cosmic star-formation history. Annu. Rev. Astron. Astrophys. 52, 415-486 (2014).

7. Barton, E. J., Geller, M. J. & Kenyon, S. J. Tidally triggered star formation in close pairs of galaxies. Astrophys. J. 530, 660-679 (2000).

8. Lin, L. et al. The DEEP2 galaxy redshift survey: evolution of close galaxy pairs and major-merger rates up to z~1.2. Astrophys. J. Lett. 617, L9-L12 (2004).

9. Conselice, C. J., Bershady, M. A., Dickinson, M. & Papovich, C. A direct measurement of major galaxy mergers at z≤3. Astron. J. 126, 1183-1207 (2003).

10. Lotz, J. M., Primack, J. & Madau, P. A new nonparametric approach to galaxy morphological classification. Astron. J. 128, 163-182 (2004).

11. Harmsen, B. et al. Diverse stellar haloes in nearby Milky Way mass disc galaxies. Mon. Not. R. Astron. Soc. 466, 1491-1512 (2017).

12. Smercina, A., Bell, E. F., Samuel, J. & D'Souza, R. Relating the diverse merger histories and satellite populations of nearby galaxies. Astrophys. J. 930, 69 (2022).

13. Boecker, A. et al. A galaxy's accretion history unveiled from its integrated spectrum. Mon. Not. R. Astron. Soc. 491, 823-837 (2020).

14. van Dokkum, P. G. et al. The growth of massive galaxies since z=2. Astrophys. J. 709, 1018-1041 (2010).

15. Oyarzún, G. A. et al. Signatures of stellar accretion in MaNGA early-type galaxies. Astrophys. J. 880, 111 (2019).

16. Davison, T. A. et al. Mapping accreted stars in early-type galaxies across the mass-size plane. Mon. Not. R. Astron. Soc. 507, 3089-3112 (2021).

17. Cannarozzo, C. et al. The contribution of in situ and ex situ star formation in early-type galaxies: MaNGA versus IllustrisTNG. Mon. Not. R. Astron. Soc. 520, 5651-5670 (2023).

18. Huang, S., Ho, L. C., Peng, C. Y., Li, Z.-Y. & Barth, A. J. Fossil evidence for the two-phase formation of elliptical galaxies. Astrophys. J. Lett. 768, L28 (2013).

19. Huang, S., Ho, L. C., Peng, C. Y., Li, Z.-Y. & Barth, A. J. The Carnegie-Irvine galaxy survey. III. The three-component structure of nearby elliptical galaxies. Astrophys. J. 766, 47 (2013).

20. Huang, S. et al. Individual stellar haloes of massive galaxies measured to 100 kpc at 0.3 < z < 0.5 using Hyper Suprime-Cam. Mon. Not. R. Astron. Soc. 475, 3348-3368 (2018).

21. Walmsley, M., Ferguson, A. M. N., Mann, R. G. & Lintott, C. J. Identification of low surface brightness tidal features in galaxies using convolutional neural networks. Mon. Not. R. Astron. Soc. 483, 2968-2982 (2019).

22. Bickley, R. W. et al. Convolutional neural network identification of galaxy post-mergers in UNIONS using IllustrisTNG. Mon. Not. R. Astron. Soc. 504, 372-392 (2021).

23. Ćiprijanović, A., Snyder, G. F., Nord, B. & Peek, J. E. G. DeepMerge: classifying high-redshift merging galaxies with deep neural networks. Astron. Comput. 32, 100390 (2020).

24. Eisert, L. et al. ERGO-ML: comparing IllustrisTNG and HSC galaxy images via contrastive learning. Mon. Not. R. Astron. Soc. 528, 7411-7439 (2024).

25. Angeloudi, E. et al. ERGO-ML: towards a robust machine learning model for inferring the fraction of accreted stars in galaxies from integral-field spectroscopic maps. Mon. Not. R. Astron. Soc. 523, 5408-5429 (2023).

26. Bundy, K. et al. Overview of the SDSS-IV MaNGA survey: mapping nearby galaxies at Apache Point Observatory. Astrophys. J. 798, 7 (2015).

27. Sarmiento, R. et al. MaNGA: 10,000 mock galaxies for stellar population analysis. Astron. Astrophys. 673, A23 (2023).

28. Oser, L., Ostriker, J. P., Naab, T., Johansson, P. H. & Burkert, A. The two phases of galaxy formation. Astrophys. J. 725, 2312-2323 (2010).

29. Cappellari, M. Structure and kinematics of early-type galaxies from integral field spectroscopy. Annu. Rev. Astron. Astrophys. 54, 597-665 (2016).

30. Greene, J. E. et al. The MASSIVE survey. II. Stellar population trends out to large radius in massive early-type galaxies. Astrophys. J. 807, 11 (2015).

31. Oyarzún, G. A. et al. SDSS-IV MaNGA: how the stellar populations of passive central galaxies depend on stellar and halo mass. Astrophys. J. 933, 88 (2022).

32. Scholz-Díaz, L., Martín-Navarro, I., Falcón-Barroso, J., Lyubenova, M. & van de Ven, G. Baryonic properties of nearby galaxies across the stellar-to-total dynamical mass relation. Nat. Astron. 8, 648-656 (2024).

33. Tinker, J. L. A self-calibrating halo-based group finder: application to SDSS. Astrophys. J. 923, 154 (2021).

34. Wake, D. A. et al. The SDSS-IV MaNGA sample: design, optimization, and usage considerations. Astron. J. 154, 86 (2017).

35. Rodriguez-Gomez, V. et al. The stellar mass assembly of galaxies in the Illustris simulation: growth by mergers and the spatial distribution of accreted stars. Mon. Not. R. Astron. Soc. 458, 2371-2390 (2016).

36. Huško, F., Lacey, C. G. & Baugh, C. M. The buildup of galaxies and their spheroids: the contributions of mergers, disc instabilities, and star formation. Mon. Not. R. Astron. Soc. 518, 5323-5339 (2023).

37. Bell, E. F. et al. The accretion origin of the Milky Way's stellar halo. Astrophys. J. 680, 295-311 (2008).

38. Helmi, A. Streams, substructures, and the early history of the Milky Way. Annu. Rev. Astron. Astrophys. 58, 205-256 (2020).

39. McConnachie, A. W. The observed properties of dwarf galaxies in and around the Local Group. Astron. J. 144, 4 (2012).

40. Kruijssen, J. M. D., Pfeffer, J. L., Reina-Campos, M., Crain, R. A. & Bastian, N. The formation and assembly history of the Milky Way revealed by its globular cluster population. Mon. Not. R. Astron. Soc. 486, 3180-3202 (2019).

41. Conselice, C. J., Mundy, C. J., Ferreira, L. & Duncan, K. A direct measurement of galaxy major and minor merger rates and stellar mass accretion histories at z<3 using galaxy pairs in the REFINE survey. Astrophys. J. 940, 168 (2022).

42. Mundy, C. J. et al. A consistent measure of the merger histories of massive galaxies using close-pair statistics. I. Major mergers at z<3.5. Mon. Not. R. Astron. Soc. 470, 3507-3531 (2017).

43. Zhu, L. et al. The Fornax3D project: discovery of ancient massive merger events in the Fornax cluster galaxies NGC 1380 and NGC 1427. Astron. Astrophys. 664, A115 (2022).
44. Lackner, C. N., Cen, R., Ostriker, J. P. & Joung, M. R. Building galaxies by accretion and in situ star formation. *Mon. Not. R. Astron. Soc.* **425**, 641–656 (2012).

45. Davison, T. A., Norris, M. A., Pfeffer, J. L., Davies, J. J. & Crain, R. A. An EAGLE's view of ex situ galaxy growth. *Mon. Not. R. Astron. Soc.* **497**, 81–93 (2020).

46. Tacchella, S. et al. Morphology and star formation in IllustrisTNG: the build-up of spheroids and discs. *Mon. Not. R. Astron. Soc.* **487**, 5416–5440 (2019).

47. Lee, J. & Yi, S. K. On the assembly history of stellar components in massive galaxies. *Astrophys. J.* **766**, 38 (2013).

48. De Lucia, G., Springel, V., White, S. D. M., Croton, D. & Kauffmann, G. The formation history of elliptical galaxies. *Mon. Not. R. Astron. Soc.* **366**, 499–509 (2006).

49. Rodriguez-Gomez, V. et al. The role of mergers and halo spin in shaping galaxy morphology. *Mon. Not. R. Astron. Soc.* **467**, 3083–3098 (2017).

50. Rowlands, K. et al. Galaxy And Mass Assembly (GAMA): the mechanisms for quiescent galaxy formation at $z<1$. *Mon. Not. R. Astron. Soc.* **473**, 1168–1185 (2018).

51. Belli, S., Newman, A. B. & Ellis, R. S. MOSFIRE spectroscopy of quiescent galaxies at 1.5<z<2.5. II. Star formation histories and galaxy quenching. *Astrophys. J.* **874**, 17 (2019).

52. Naab, T. et al. The ATLAS3D project. XXV. Two-dimensional kinematic analysis of simulated galaxies and the cosmological origin of fast and slow rotators. *Mon. Not. R. Astron. Soc.* **444**, 3357–3387 (2014).

53. Yoon, Y., Park, C., Chung, H. & Lane, R. R. Evidence for impact of galaxy mergers on stellar kinematics of early-type galaxies. *Astrophys. J.* **925**, 168 (2022).

54. Lin, L. et al. Where do wet, dry, and mixed galaxy mergers occur? A study of the environments of close galaxy pairs in the DEEP2 galaxy redshift survey. *Astrophys. J.* **718**, 1158–1170 (2010).

55. Jian, H.-Y., Lin, L. & Chiueh, T. Environmental dependence of the galaxy merger rate in a $\Lambda$CDM universe. *Astrophys. J.* **754**, 26 (2012).

56. Planck Collaboration. Planck 2015 results. XIII. Cosmological parameters. *Astron. Astrophys.* **594**, A13 (2016).

57. Pillepich, A. et al. First results from the TNG50 simulation: the evolution of stellar and gaseous discs across cosmic time. *Mon. Not. R. Astron. Soc.* **490**, 3196–3233 (2019).

58. Nelson, D. et al. First results from the TNG50 simulation: galactic outflows driven by supernovae and black hole feedback. *Mon. Not. R. Astron. Soc.* **490**, 3234–3261 (2019).

59. Sánchez, S. F. et al. SDSS-IV MaNGA: pyPipe3D analysis release for 10,000 galaxies. *Astrophys. J. Suppl. Ser.* **262**, 36 (2022).

60. Lacerda, E. A. D. et al. pyFIT3D and pyPipe3D – the new version of the integral field spectroscopy data analysis pipeline. *New Astron.* **97**, 101895 (2022).

61. Liu, H., HaoChen, J. Z., Gaidon, A. & Ma, T. Self-supervised learning is more robust to dataset imbalance. Preprint at arxiv.org/Uabs/2110.05025 (2021).

62. Grill, J.-B. et al. Bootstrap your own latent: a new approach to self-supervised learning. In *Proc. 34th Conference on Neural Information Processing Systems* (eds Larochelle, H. et al.) 21271–21284 (Curran Associates, 2020).

63. Daddi, E. et al. Passively evolving early-type galaxies at $1.4 \le z \le 2.5$ in the Hubble ultra deep field. *Astrophys. J.* **626**, 680–697 (2005).

64. van Dokkum, P. G. et al. Confirmation of the remarkable compactness of massive quiescent galaxies at $z \sim 2.3$: early-type galaxies did not form in a simple monolithic collapse. *Astrophys. J. Lett.* **677**, L5 (2008).

65. van der Wel, A. et al. 3D-HST+CANDELS: the evolution of the galaxy size-mass distribution since $z=3$. *Astrophys. J.* **788**, 28 (2014).

66. Rodriguez-Gomez, V. et al. The optical morphologies of galaxies in the IllustrisTNG simulation: a comparison to Pan-STARRS observations. *Mon. Not. R. Astron. Soc.* **483**, 4140–4159 (2019).

67. Wilkinson, D. M., Maraston, C., Goddard, D., Thomas, D. & Parikh, T. FIREFLY (Fitting Iteratively for Likelihood Analysis): a full spectral fitting code. *Mon. Not. R. Astron. Soc.* **472**, 4297–4326 (2017).

68. Westfall, K. B. et al. The data analysis pipeline for the SDSS-IV MaNGA IFU galaxy survey: overview. *Astron. J.* **158**, 231 (2019).

69. Nanni, L. et al. iMaNGA: mock MaNGA galaxies based on IllustrisTNG and MaStar SSPs. II. The catalogue. *Mon. Not. R. Astron. Soc.* **522**, 5479–5499 (2023).

70. Domínguez Sánchez, H., Margalef, B., Bernardi, M. & Huertas-Company, M. SDSS-IV DR17: final release of MaNGA PyMorph photometric and deep-learning morphological catalogues. *Mon. Not. R. Astron. Soc.* **509**, 4024–4036 (2022).

71. Sánchez, S. F. et al. Pipe3D, a pipeline to analyze integral field spectroscopy data. II. Analysis sequence and CALIFA dataproducts. *Rev. Mexicana Astron. Astrofis.* **52**, 171–220 (2016).

72. Graham, M. T. et al. SDSS-IV MaNGA: stellar angular momentum of about 2300 galaxies: unveiling the bimodality of massive galaxy properties. *Mon. Not. R. Astron. Soc.* **477**, 4711–4737 (2018).

## Acknowledgements

E.A. thanks C. Gallart, S. Faber, K. Bundy and L. Scholz-Díaz for their insightful discussions. J.F.-B. and E.A. acknowledge support from the Spanish Ministry of Science, Innovation and Universities (Grant Nos. PID2019-107427GB-C32 and PID2022-140869NB-I00) and through the project TRACES from the Instituto de Astrofísica de Canarias, which is partially supported through the state budget and the regional budget of the Consejería de Economía, Industria, Comercio y Conocimiento of the Canary Islands Autonomous Community. M.H.-C., R.S. and E.A. acknowledge financial support from the State Research Agency of the Spanish Ministry of Science and Innovation under the grants ‘Galaxy Evolution with Artificial Intelligence’ with reference PGC2018-100852-A-I00 and ‘BASALT’ with reference PID2021-126838NB-I00. This research made use of computing time available on the high-performance computing systems at the Instituto de Astrofísica de Canarias. We thankfully acknowledges the technical expertise and assistance provided by the Spanish Supercomputing Network (Red Española de Supercomputación), as well as the computer resources used: the Deimos-Diva Supercomputer and the LaPalma Supercomputer, at the Instituto de Astrofísica de Canarias.

## Author contributions

E.A. performed the data analysis and wrote the paper. E.A., M.H.-C. and J.F.-B. conceived and designed the project. A.B. provided substantial comments on the overall design of the project and the interpretation of the results. R.S. substantially assisted with the creation of the mock dataset from the EAGLE cosmological simulation. L.E. and A.P. commented on the paper and helped improve its overall focus. All authors interpreted and discussed the results and commented on the paper.

## Competing interests

The authors declare no competing interests.

## Additional information

**Supplementary information** The online version contains supplementary material available at https://doi.org/10.1038/s41550-024-02327-3.

**Correspondence and requests for materials** should be addressed to Eirini Angeloudi.
**Peer review information** *Nature Astronomy* thanks the anonymous reviewers for their contribution to the peer review of this work.

**Reprints and permissions information** is available at
www.nature.com/reprints.

**Publisher's note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

© The Author(s), under exclusive licence to Springer Nature Limited 2024