dataesr/scientific-paragraphs-categorization huggingface dataset:
DeepLeGATo DeepLeGATo  (Tuccillo et al. 2018 ) is a software package for estimating galaxy structure based on a supervised deep-learning approach. The code uses CNNs to perform a simple regression between an image centred on a given galaxy and its structural parameters, providing results in very short times (see Appendix B. 1). In the version used in this work, the training was performed with images of fixed size (128×128 pixels) independently of the galaxy effective radius; this likely caused sub-optimal performance on the largest and brightest galaxies. The images used for training were not the ones provided with the EMC dataset; instead, they were idealised analytic 1-or 2-component Sérsic profiles, convolved with the PSF of the corresponding band, to which noise having similar properties to that in the EMC data were added.
ProFit Profit  (Robotham et al. 2017 ) is a software package designed to perform Bayesian two-dimensional photometric galaxy profile modelling. It consists of a low-level C++ library accessible via a command-line interface and documented API, along with high-level R (v. 3. 6. 1) and Python interfaces. The fitting process for each object starts running the source finder ProFound  (Robotham et al. 2018)  3 on a 500 × 500 pixel cutout centred on each target, to create a segmentation map and find nearby sources requiring simultaneous modelling; the output also provides some reasonable initial guess for the profile solution. The actual fitting is then performed by the Highlander core software 4 , which combines a genetic algorithm step with a CHARM  (Turchin 1971; Smith 1984 ) Markov chain Monte-Carlo (MCMC) process, repeated twice; each one is run for 100 steps (where model realisations are modified by the number of free parameters also).
SourceXtractor++ SourceXtractor++  (Bertin et al. 2020; Kümmel et al. 2020)   5  is a ground-up re-write of the widely used SExtractor2 software  (Bertin & Arnouts 1996) , written in C++ with a strong focus on extensibility and model-fitting photometry; the software is under active development and the results submitted to the challenge represent a snapshot in this process (the version of SourceXtractor++ used in the EMC is 0. 12). Each SourceXtractor++ run includes two stages, detection and measurement. The detection stage follows the same procedure as used in SExtractor2. Detection parameters need to be optimised for a compromise between the completeness of the true object list and the number of spurious objects extracted or deblended; over-extraction of sources impacts the performance of the run-time required, and may also reduce the accuracy of morphological measurements if objects are over-deblended.
The chosen priors for the EMC are described in Appendix B. 5. 2. Noticeably, the SourceXtractor++ pipeline used for the EMC includes a pre-processing of the images, namely the extraction and usage of PSFs from images, performed using the PSFEx software  (Bertin 2011) , and a re-downsampling to the original pixel scales of the NIR and LSST images (0. 3 and 0. 2, respectively). This procedure was allowed by the guidelines of the EMC, given that no additional input data were used. However, given that no other participants did anything similar, we decided to also check the performance of the package on standard, nonpre-processed data, finding overall good agreement with a few differences that should be taken into account when considering the overall processing cost of the pipeline. We discuss this in Appendix B.
3, all participants obliged to this, and some also provided results for lower S/N sources. However, not all the fits were successful (i. e. some were given as NaN or default values in the output catalogues), and some were flagged as "bad" or "unreliable" in one or more codes. In general, these sources were not included in the lists used to evaluate the accuracy of the results, which therefore only included the objects for which all the software packages had provided a reliable fit. An important exception is the case of Galapagos-2, which outputs several quality flags describing which component of a galaxy can be considered as reliably fit. In particular, for the DS runs a total of five flags were provided: USE_FLAG_SS indicates that the single Sérsic fit is usable, as it did not run into fitting constraints; USE_FLAG_BULGE_CONSTR and USE_FLAG_DISK_CONSTR serve the same purpose for the bulge and disc components, respectively; in addition, USE_FLAG_BULGE_BRIGHT and USE_FLAG_DISK_BRIGHT indicate whether the bulge and the disc are relatively bright enough (b/t > 0.
(2022). See the Appendix of Euclid Collaboration (2023) for a comprehensive discussion on this topic. To estimate the impact for each software tool, an additional term evaluating the completeness fraction of the output catalogues with respect to the full 5σ list was included in the global metric, as described below. To build the metric, we started considering the relative flux difference of each object with respect to the input true flux, i. e. δ f = ( f measf true )/ f true. We then used δ f to evaluate three diagnostics: the bias B; the dispersion D; and the outlier fractions O. In summary, the three diagnostics were first averaged over the sources belonging to bins of input magnitude (we used 15 bins to divide the full interval of simulated magnitudes, from 14 to 28), to quantify the impact of S/N; then these averages (normalised with weighting factors) were summed, and further combined with the completeness C diagnostic, finally yielding a global score S for each field and realisation.
Each software package has different ways of identifying unreliable fits, and we refer the reader to the publications describing each code for additional information. Here, we simply trusted the participants' verdict on the reliability of their fits. We point out that the definitions used in Euclid Collaboration (2023) are very similar, but since it is difficult to construct meaningful expectations for ideal perturbed distributions of morphological parameters (corresponding to the σ true we use here for the fluxes), some differences were introduced. The interested reader should therefore pay attention to these details. Finally, the global diagnostic S for each run is defined as S = (1 -C) + k B B + k D (D -1) + k O O (2) where we subtract 1 from D when computing the final global statistics, because when the dispersion is 'ideal' the ratio with σ true is 1, and we want the value of all diagnostics to be close to zero for ideal fits.
Finally, as already mentioned the Galapagos-2 runs on DS F0 were performed in a simultaneous multi-band fit, including I E with the other bands; this caused the results to be significantly different from those in the other four I E fields. To avoid any impact on the evaluation, and considering the many different approaches of the other participants (DeepLeGATo only provided the I E fits, Morfometryka did not provide the fit, ProFit only provided non-simultaneous fits, and SourceXtractor++ provided both a simultaneous and a separated fit), we decided to remove F0 from the analysis of I E DS. In summary, I E SS and RM were analysed by averaging F0, F2, F3, and F4 (resulting in 212 000 objects for SS, and 204 229 for RM, where the bright galaxies were not simulated); I E DS by averaging F2, F3 and F4 (207 064 objects); and the other bands on DS F0 alone (because it was the only field simulated with a multi-band dataset; it contains 70 700 objects).
Results In this section we discuss the results obtained with the different software packages. First of all, it is worth pointing out again that the complexity of the challenge caused a significant scatter in the interpretation of its goals and spirit by the participants. This caused substantial differences in the adopted approaches, level of processing and output formats between them. Together with the high dimensionality of the dataset, this makes a direct and comprehensive comparison of the results very challenging. In other words, the strategies and techniques adopted by the participants influenced the overall accuracy of the provided output, and this must be taken into account in the analysis, to ensure a fair overview of each code's capabilities and limitations. Nevertheless, we believe it is possible to draw some interesting general conclusions from the comparison.
Global outcome In the following, we separately analyse the three realisations SS, DS, and RM. We separate the multi-band dataset from the I E -only DS fits, since the results are significantly different (we identify the multi-band case with the addition of the letter 'M' to the acronym DS whenever necessary   metric S for each realisation are listed in Table  2  (where the values of the completeness factor C are also reported, to allow for a better evaluation of its impact), and shown in Fig. 9. A visual summary of the values of each diagnostic used to compute S (i. e. B, D and O), averaged over the magnitude bins and fields, is given in Fig. 10. For the multi-band case, these values are the average of the ones obtained in each of the nine individual bands. We note that in these plots the values are shown before the weighting by the k factors in Eq.
That said, we can claim that with our metric all software packages provided acceptable to good results in at least some of the realisations. Some differences are present in a few cases (e. g. a particular realisation or band, the faint end of the simulated distribution of galaxies, etc. ), but the outputs provided are typically fairly accurate. Unsurprisingly, in the DS realisation the best results were obtained in the I E runs, for most of the software packages, given its high resolution and depth. The multi-band dataset proved to be more demanding, because of the lower S/N and resolution of the images, and also of the resampling procedure which introduced noise correlations. All the participants reached at least 95% global completeness in the I E DS dataset; in the other realisations, some lower scores were obtained (see the numbers in parenthesis in Table  2 ).
4). Interestingly, the fits with free n bulge were in general slightly better than those with fixed n bulge = 4 (except for Galapagos-2), despite the fact that the simulated galaxies had indeed n bulge = 4. Finally, the results for the RM realisation were less accurate than the ones for the other I E images. This should be expected, given the inherently more difficult task of fitting analytical profiles on complex realistic morphologies. Interestingly, here the three codes that provided results have very similar trends; this seems to imply that when dealing with realistic galaxy A101, page 11 of 28 shapes, the impact of prior calibrations, pre-processing of the images, and robustness of the algorithm become of secondary importance with respect to the inherent difficulty of the task. The results for the multi-band dataset show the evident impact of the different strategies followed by the participants.
Noticeably, ProFit behaves differently, slightly underestimating intermediate magnitude sources before starting to overestimate at the faint end. An even more striking exception is given by Morphometryka (SS was the only provided output dataset), which has a clearly declining monotonic trend, reaching δ f -15% at I E 25. 3. The situation is completely different in the RM realisation, where all three of the codes that provided results have similar declining trends in the bias B, with faint sources typically having fluxes underestimated by about 10-15% at I E = 24-25. This is at odds with the results from the other realisations. We checked that this is not an issue in the simulations: while minor inconsistencies between the input true fluxes in the catalogue and the actual realisations of the sources in the images can be present because of the simulation method (see Sect.
The multi-band dataset In the multi-band dataset (for which only Galapagos-2, ProFit, and SourceXtractor++ provided results; see Figs. 13  and 14 ) the trends are generally similar to those of the I E case, with larger values of bias at the faint end for all codes in the NIR and LSST bands. In particular, Profit reaches B bin 1. 5 in the NIR bands, and approximately 4 in the LSST u and g bands, likely because of the independent fits performed on low S/N sources (we do not show the corresponding points in the plots, for readability); there is also a particular trend in the LSST bands, with sources being underestimated at intermediate magnitudes before turning to a strong overestimation at the faint end (we were not able to find an easy explanation for this). Galapagos-2 and SourceXtractor obtain better (and similar) results in NIR thanks to the simultanous fit, reaching B bin 0.
Colours in the multi-band dataset Thus far, we have focussed on the accuracy of the flux estimates. However, for the multi-band dataset, it is also interesting to examine how accurately the three software packages that provided results were able to recover flux ratios among the bands, i. e. colours. Here too, the multi-dimensionality of the data requires some effort to summarise the results in a few informative plots. In each small sub-panel of Figs. 19 (Galapagos-2), 20 (ProFit), and 21 (SourceXtractor++), we show the comparison between a measured colour and its corresponding true value. For each fitted galaxy (colour-coded by A101, page 15 of 28 A&A 671, A101 (2023) Fig. 19. Colours estimates for Galapagos-2. Each small sub-panel on the left part of the plot shows the measured versus true colour (magnitude difference) for a pair of bands: the colour is always x-axis label -y-axis label.
Conclusions This is the first of two papers presenting and discussing the EMC, a project aimed at comparing the results of model-fitting software packages on a set of simulated Euclid observations, with the aim of providing a first quantitative study to define the best-suited algorithm to be included in the Euclid Science Ground Segment processing pipeline. Five model-fitting tools (DeepLeGATo, Galapagos-2, Morfometryka, ProFit, and SourceXtractor++) were tested in the challenge, providing partial or complete results. It is worth stressing that the results obtained for the EMC and discussed in this paper and in Euclid Collaboration (2023) were used by the developers to improve and, in some cases, significantly upgrade the software packages, which is per se a relevant outcome of this work. While the companion paper (Euclid Collaboration 2023) focusses on results concerning the morphological parameters, in this work we have presented the simulated dataset and focussed on the results concerning photometric estimates only.
We used the code Egg  (Schreiber et al. 2017)  to produce mock galaxy catalogues, which were then exploited to create synthetic images in three different realisations: two using GalSim  (Rowe et al. 2015) , so as to generate SS and DS analytical profiles (the latter with n bulge =4 and n disc =1 and varying bulge-to-total luminosity ratios); and one with the method described in Euclid Collaboration (2022a) and  Lanusse et al. (2021)  to obtain realistic morphologies by means of a CNN technique (RM). We did not simulate irregular galaxies, which constitute a substantial fraction of the high-redshift population, and we did not include AGNs. Furthermore, the distribution of the bulge-tototal ratios of the simulated galaxies is more skewed towards disc-dominated objects than in the real Universe (see Fig.
The requested output consisted of the photometric (and morphological) estimates for all of the objects in these lists. To allow for a quantitative analysis of the results and for a comparison of the accuracy of the codes, we defined a metric as described in Sect. 4, computing three quantities related to the median bias in the flux measurements (B), the mean dispersion of the measured flux distribution (D), and the outlier fraction (O). These quantities were evaluated on a subset of the dataset including only the sources for which all participants provided a fit, and we also computed a further value quantifying the fraction of sources of the input list for which a software package succeeded to provide a successful fit (C). Finally, we combined them in a single figure of merit S; the lower the value of S, the better the global fit was, with S ≤ 1 being considered optimal performance.
The global results are summarised in Table  2  and Figs. 9-14. We also analysed how accurately colours were retrieved, and the reliability of the uncertainty estimates. The computational times and some in-depth analysis for each individual software package are discussed in the appendix. In general, all the participants provided acceptable to good results in at least some of the tests, with a few differences from case to case. A thorough comparison could only be performed with the results of ProFit, Galapagos-2, and SourceXtractor++ (the first two provided all the requested outputs, and the third only lacked one part of a set, namely the free n bulge run in the double Sérsic multi-band realisation). DeepLeGATo only provided fits for the I E analytical realisations, yielding good results with the caveat of a S/N-dependent calibration of the models, while Morfometryka only provided data for the single Sérsic set; therefore, their contribution to the challenge can only be considered as indicative of their potential.
A101, page 19 of 28  A&A 671, A101 (2023)  Considering the S values, among the three codes that provided the complete output, the best performance was obtained by SourceXtractor++ in all realisations. It is worth stressing again that the pipeline it adopted for the EMC was the most tuned on the dataset, with the inclusion of a substantial pre-processing of the data, and the priors being modelled using the samples of true values that were provided along with the simulated images. In the analytical realisations (SS and DS), SourceXtractor++ and Galapagos-2 reached typical values in the bias of the measured fluxes with respect to the true values below 1% at a S/N above 10 in all bands, and above 5 in I E (in at least one of the possible configurations of the codes for which results were provided).
For these two codes, the dispersion is typically slightly larger than, but comparable to, the one expected from a theoretical distribution obtained by perturbing the input true fluxes with simulated observational noise consistent with the nominal depths of the images; a small fraction of outliers (objects with a relative error in the fitted flux larger than 5 times the expected theoretical distribution) is always present. ProFit also yielded good results in SS, but was less accurate on the DS realisation (Galapagos-2 also being sub-optimal at the faint end in I E for this realisation). The three software packages performed similarly on the RM dataset, and in all cases results were poorer with respect to the analytical realisations; this seems to imply that the overall quality of the fits might be sub-optimal when dealing with real data.
However, further investigation is needed to assess to what extent the quality of the simulation might have impacted the results. All codes tend to underestimate the uncertainties at the bright end, but statistically provide a reasonable estimate for intermediate and faint sources. Finally, a relevant outcome is the importance of the simultaneous fitting of the multi-band dataset: using the information provided by the deep and high resolution I E band helps to obtain more accurate measurements on the shallower NIR bands, and noticeably also on the ground-based LSST bands, both in terms of absolute fluxes and colours. Once again this highlights how the synergy between the two surveys will be of paramount important for the success of both. We did not attempt to investigate the performance of the model-fitting algorithms in the context of the Euclid Deep Survey (see  Laureijs et al.
) and the other parameters not included in the metric that should be taken into account (such as the computing time, required resources, compatibility with the current pipeline, accuracy of uncertainty budget estimates, etc. ), it is important to stress that the results presented in this work and in Euclid Collaboration (2023) must be interpreted with caution. We followed one possible approach to analyse, in a compact way, a very complex and multi-layered dataset; and we emphasise that some of the software packages might be better suited than the ones that obtained the best score here for other specific science cases. We invite the interested reader to check the full set of results using the online tool. Future work will include testing at least some of the software packages in more realistic environments using Euclid official simulations and, finally, the implementation in the Euclid pipeline of the chosen algorithms.
B. 2. Galapagos-2 Galapagos-2 employs the down-hill gradient minimisation method of Galfit/GalfitM, so it is typically fast, and can be reasonably parallelised to use 16 cores (due to an IDL limitation; ways around this limit exist, as it can also run several sessions in parallel). For the EMC, the code was run on a machine with 48 cores (2. 3 GHz each) at 96 threads and ample (512 GB) memory, using 12 threads (at 1. 15 GHz) at a time (and 4 sessions in parallel) for multi-band fits. The run times of the Galfit fits themselves were measured and stored, and account for the majority of the time used, with the wrapper not adding significant computational time. The typical GalfitM fitting times were approximately as follows: -DS F1 single-band single Sérsic fit, 160 hours (about 5. 7 seconds per galaxy); -DS F1 single-band double Sérsic fit, 150 hours (about 5.
We point out that since the other teams did not provide single Sérsic flux estimates for DS runs, we could not check whether this particular result is a general one, or only concerns Galapagos-2. Interestingly, the fit of the LSST i band on the DS F0 dataset by Galapagos-2 is substantially worse than the fits provided for the other bands. After an in-depth analysis, we concluded that this is actually due to the polynomial fitting method used in the software. In short, Galapagos-2 works with effective central wavelengths, and when performing a multi-band fit on a dataset including two bands with a very close central wavelength, the shallower of the two bands might get a sub-optimal fit with a systematic offset. This is exactly the case here, with the LSST i band having a central wavelength very close to the I E one (λ I E = 710 nm and λ i = 754 nm), and very different filter widths; see Sect.
2. 1 and Fig. 1. Noticeably, this does not affect the whole catalogue, but only a fraction of the sources. Despite trying to find a correlation with some other input or output parameter of the dataset, we were not able to identify the criteria resulting in a good or a bad fit for a given source. Nevertheless, we verified the reliability of our conclusion by picking one source having a bad fit, and looking at the residual images produced by Galapagos-2 in the full multi-band run, and in a test run in which the I E band information was not used to fit the i band. The result of the test is shown in Fig. B. 5: while the other LSST bands are well-fitted in both cases, when the I E band is used the i band has an evident residual, which flattens out when excluding I E. Of course, this cannot be considered a viable, definitive 'solution' to the issue, since I E is the most important and deepest band, so it cannot be excluded from the fit to improve the fit in another band; on the contrary, one might A101, page 24 of 28   want to exclude the i band from the fit, although this would imply entirely discarding a band and its information.
Therefore, for the sake of comparison with the other software packages we agreed to keep the delivered dataset including this issue, although it non-negligibly worsens the statistics. Finally, we note that the issue is more evident when using the fluxes of the single Sérsic fit, while it is somehow mitigated if the double Sérsic fit is considered (Galapagos-2 provides both for the DS realisation, which is the only one including LSST bands).