dataesr/scientific-paragraphs-categorization huggingface dataset:
The dynamical evolution of the system should be very fast, with the clumps moving inwards in less than 100 Myr and producing a massive spheroid  (Dekel et al. 2009 ). In addition, a central BH could be activated and produce a luminous AGN very similar to most of the quasi-stellar objects at redshifts above 6 where a luminous AGN coexists with a powerful starburst in the nucleus. Thus, SPT0311-58 could be exhibiting the evolutionary phase immediately previous to the infrared phase of a luminous quasistellar object. Alternative scenarios predicting the formation and evolution of massive clumps as a consequence of interaction and merger can also be relevant in this galaxy. The combination of high angular resolution JWST and ALMA data and hydrodynamical cosmological simulations of massive structures in the early Universe are required to further illuminate the formation and subsequent evolution of systems like SPT0311-58 during the first Gyr after the Big Bang.
Acknowledgements. The authors thank to the anonymous referee for useful comments. J. A. -M. , A. C. G. , L. C. , A. L. acknowledge support by grant  PIB2021-127718NB-100 , P. P. -G. by grant  PGC2018-093499-B-I00. A. A-H. by grant  PID2021-124665NB-I00  from the  Spanish Ministry of Science and Innovation/State Agency of Research   MCIN/AEI/10. 13039/501100011033  and by "ERDF A way of making Europe". F. W. and M. N. acknowledge support from the  ERC Advanced  Grant  740246  (Cosmic_Gas), A. B. and G. O. acknowledge support from the  Swedish National Space Administration (SNSA). O. I. acknowledges the funding of the  French Agence Nationale de la Recherche  for the project  iMAGE  (grant  ANR-22-CE31-0007 ), J. H. was supported by a  VILLUM FONDEN Investigator grant  (project number  16599 ).
K. I. C. acknowledges funding from the  Netherlands Research School for Astronomy (NOVA)  and the  Dutch Research Council (NWO)  through the award of the  Vici Grant   VI. C. 212. 036. J. P. P. acknowledges financial support from the  UK Science and Technology Facilities Council , and the  UK Space Agency. T. P. R. would like to acknowledge support from the  ERC  under advanced grant  743029  (EASY). The  Cosmic Dawn Center  (DAWN) is funded by the  Danish National Research Foundation  under grant No. 140  The work presented is the effort of the entire MIRI team and the enthusiasm within the MIRI partnership is a significant factor in its success. MIRI draws on the scientific and technical expertise of the following organisations:  Ames Research Center, USA ;  Airbus Defence and Space, UK ;  CEA-Irfu ,  Saclay, France ;  Centre Spatial de Liège, Belgium ;  Consejo Superior de Investigaciones Científicas, Spain ;  Carl Zeiss Optronics ,  Germany ;  Chalmers University of Technology, Sweden ;  Danish Space Research Institute, Denmark ;  Dublin Institute for Advanced Studies, Ireland ;  European Space Agency, Netherlands ;  ETCA, Belgium ;  ETH Zurich, Switzerland ;  Goddard Space Flight Center, USA ;  Institut d'Astrophysique Spatiale, France ;  Instituto Nacional de Técnica Aeroespacial, Spain ;  Institute for Astronomy, Edinburgh, UK ;  Jet Propulsion Laboratory, USA ;  Laboratoire d'Astrophysique de Marseille (LAM), France ;  Leiden University, Netherlands ;  Lockheed Advanced Technology Center (USA) ;  NOVA Opt-IR group  at  Dwingeloo, Netherlands ;  Northrop Grumman ,  USA ;  Max-Planck Institut für Astronomie (MPIA) ,  Heidelberg, Germany ;  Laboratoire d'Etudes Spatiales et d'Instrumentation en Astrophysique (LESIA), France ;  Paul Scherrer Institut, Switzerland ;  Raytheon Vision Systems, USA ;  RUAG Aerospace, Switzerland ;  Rutherford Appleton Laboratory (RAL Space), UK ;  Space Telescope Science Institute, USA ;  Toegepast-Natuurwetenschappelijk Onderzoek (TNO-TPD) ,  Netherlands ;  UK Astronomy Technology Centre, UK ;  University College London, UK ;  University of Amsterdam, Netherlands ;  University of Arizona, USA ;  University of Cardiff, UK ;  University of Cologne, Germany ;  University of Ghent ;  University of Groningen, Netherlands ;  University of Leicester, UK ;  University of Leuven, Belgium ;  University of Stockholm, Sweden ;  Utah State University, USA.
Protects The Individual Updates The Aggregate Secure Aggregation Central DP Local DP Among many data analysis methods, this paper focuses on singular value decomposition (SVD). Plainly, SVD factorizes a matrix into three new matrices. Originating from linear algebra, SVD has several interesting properties and conveys crucial insights about the underlying matrix. Hence, SVD has essential applications in data science, such as in recommendation systems  [8, 9] , principal component analysis  [10] , latent semantic analysis  [11] , noise filtering  [12, 13] , dimension reduction  [14] , clustering  [15] , matrix completion  [16] , etc. Existing federated SVD solutions fall into two categories: SVD over horizontally and vertically partitioned datasets  [17]. In real-world applications, the former is much more common  [18, 19] ; therefore, in this paper, we choose the horizontal setting and focus on the privacy protection challenges.
Related Work The concept of privacy-preserving federated SVD has been studied in several works, which are briefly summarized below. In the literature, many anonymization techniques have been proposed to enable privacy protection in federated machine learning and other tasks. Ref. [20]  proposed substitute vectors and length-based frequent pattern tree (LFP-tree) to achieve the data anonymization. It focuses on what data can be published and how they can be published without associating subjects or identities. With the concept of data anonymization in mind, Ref. [21]  proposed a strategy by decreasing the correlation between data and the identities. However, the utility of the data will be affected. And, Ref. [22]  focused on high-dimensional dataset, which is divided into different subsets; then, each subset is generalized with a novel heuristic method based on local re-coding.
Definition 1 ((ε, δ)-Differential Privacy). A randomized mechanism M : X → R with domain X and range R satisfies ε-differential privacy if for any two adjacent inputs x, x ∈ X and for any subset of output S ⊆ R it holds that Pr(M(x) ∈ S) ≤ e ε • Pr(M(x ) ∈ S) (2) The variable ε is called the privacy budget, which measures the privacy loss. It captures the trade-off between privacy and utility: the lower its value, the more noise is required to satisfy Equation (2), resulting in higher utility loss. Another widely used DP notion is approximate DP, where a small additive term δ is added to the right side of Equation  (2). Typically, we are interested in values of δ that are smaller than the inverse of the database size. Although DP has been adopted to many domains  [7]  such as recommendation systems  [31] , we are not aware of any work besides  [18]  which adopts DP for SVD computation.
The FedPower Algorithm Following Guo et al. [18] , we assume there are n node devices, and each device i holds an independent dataset, an s i -by-d matrix M i. Each row represents a record item, while the columns of each matrix correspond to the same feature space. Moreover, M denotes the composition of matrices M i such that M T = [M T 1 , M T 2 ,. , M T n ] ∈ R s×d , with s = ∑ n i=1 s i. The solution proposed by Guo et al. [18]  is presented in Algorithm 1 with the following parameters. σ = T/p ε•min i (s i ) 2 log 1. 25 T/p δ σ = T/p max i (p i ) ε•min i (s i ) 2 log 1. 25 T/p δ Algorithm 1 Full participation protocol of FedPower by Guo et al. [18]. Input: Datasets {M i } n i=1 , target rank k, iteration rank r ≥ k, number of iteration T, synchronous set I p T , and the variance of noises (σ, σ ) Output: Approximated eigenspace Z T 1: initialise Z (i) 0 = Z 0 ∈ R d×r ∼ N(0, 1) d×r 2: for t = 1 to T do 3: each node device i computes Y (i) t = M i Z (i) t-1 , where M i = 1 s i M T i M i 4: if t ∈ I p T then 5: each node device i computes Ŷ(i) t = Y (i) t D (i) t (orthogonal transformation) 6: each node device i adds the Gaussian noise: Y (i) t = Ŷ(i) t + N (i) ∼ N(0, ||Z (i) t-1 || 2 max σ 2 ) d×r 7: each node device i sends Y (i) t to the server 8: the server performs perturbed aggregation with an extra Gaussian noise: Y t = ∑ n i=1 s i s Y (i) t + N ∼ N(0, max i ||Z (i) t-1 D (i) t || 2 max σ 2 ) d×r 9: the server broadcasts Y t to all node devices each node device i performs orthgonalization: Z (i) t = orth(Y (i) t ) 13: end for 14: return approximated eigenspace Z T = ∑ n i=1 s i s Z (i) T D (i) T+1 if T / ∈ I p T ∑ n i=1 s i s Z (i) T otherwise.
Enhancing the Utility of FedPower Adversary Model. Throughout this paper, we consider a semi-honest setup, i. e. , where the clients and the server are honest but curious. This means that they follow the protocol truthfully, but in the meantime, they try to learn as much as possible about the dataset of other participants. We also assume that the server and the clients cannot collude, so the server cannot control node devices. Utility Analysis of FedPower. It is not a surprise that adding Gaussian noise twice (i. e. , the local and the central noise in Step 6 and 8 in Algorithm 1) severely affects the accuracy of the final result. A straightforward way to increase the utility is to eliminate some of this noise. As highlighted in Table  1 , the local noise protects the individual clients from the server.
Algorithm 2 Utility-enhanced FedPower. Input: Datasets {M i } n i=1 , target rank k, iteration rank r, number of iteration T, synchronous trigger p, the variance of noise σ, and key pair (sk hm , pk hm ) Output: Approximated eigenspace Z T 1: initialise Z (i) 0 = Z 0 ∈ R d×r ∼ N(0, 1) d×r with orthonormal columns and generate an r × r zero matrix P and another all-ones matrix P of the same size 2: for t = 1 to T do 3: each node device i computes Y (i) t = M i Z (i) t-1 , where M i = 1 s i M T i M i 4: if t ≡ 0 (mod p) then 5: each node device i computes Ŷ(i) t = Y (i) t D (i) t (orthogonal transformation) 6: each node device i adds Gaussian noise: Y (i) t = Ŷ(i) t + N (i) ∼ N(0, σ) d×r 7: SA protocol is executed among the server and all node devices, with inputs Y (i) t and output Y t 8: the server chooses one random index j ∈ [1, n] and encrypts P and P : C (j) = Enc pk h (P) and C (j ) = Enc pk h (P ) for j ∈ [1, n] \ {j} 9: the server sends value C (j) and C (j ) to the appropriate node devices 10: each node device i computes for all i ∈ [1, n] \ {j}, the server decrypts the receiving messages C (i) = N (i) • C (i) which is Enc pk h (N (i) • P ) if i = j C (i) to obtain N (i) ≡ N (i) • P = Dec sk h (C (i) ) 13: the server updates aggregation result as Y t = Y t -∑ i∈[1,n]\{j} N (i) 14: the server performs orthogonalization Z t = orth(Y t ) 15: the server broadcasts Z t to all node devices  Z T = ∑ n i=1 s i s Z (i) T D (i) T+1 if T / ∈ I p T ∑ n i=1 s i s Z (i) T otherwise.
• Step 5: We remove this client-side operation from our privacy-enhanced solution, as it is not essential; only the convergence speed would be affected slightly. The FedPower protocol with enhanced privacy is present in Algorithm 3, where besides the orthogonalization, clipping is also performed with ẑ. The only client operation which must be considered for the sensitivity computation (i. e. , before noise injection) is Step 3. We calculate its sensitivity in Theorem 1. • √ r • m • ẑ. Proof. To make the proof easier to follow, we remove the subscript round counter from the notation. Let us define M and M such that they are equal except at position 1 ≤ i, j ≤ d. Now, multiply these with Z from the left results in Y and Ỹ, respectively, which are the same except in row i: [m i1 • z 11 + • • • abs(•)≤ m• ẑ +m ij • z j1 + • • • + m id • z d1 ,.
The best practice is to set δ as the inverse of the size of the underlying dataset, so there is a direct connection between the variance σ and the privacy parameter ε. Theorem 2. If T = 1, then Algorithm 3 provides (ε, δ)-DP, where = 8 • r • log (1. 25/δ) • m • ẑ σ Proof. Can be verified by combining the provided formula with the appropriate sensitivity. Algorithm 3 Privacy-enhanced FedPower. Input: Datasets {M i } n i=1 , target rank k, iteration rank r, number of iteration T, the clipping bound ẑ, the variance of noise σ Output: Approximated eigenspace Z T 1: initialise Z (i) 0 = Z 0 ∈ R d×r ∼ N(0, 1) d×r with orthonormal columns 2: for t = 1 to T do 3: each node device i computes Y (i) t = M i Z (i) t-1 , where M i = 1 s i M T i M i 4: each node device i adds Gaussian noise: each node device i calculates the latest Z Y (i) t = Y (i) t + N (i) ∼ N(0, (i) t = clip(orth(Y (i) t ), ẑ) 12: end if 13: end for 14: return approximated eigenspace Z T = ∑ n i=1 s i s Z (i) T D (i) T+1 if T / ∈ I p T ∑ n i=1 s i s Z (i) T otherwise.
Empirical Comparison In order to compare our proposed schemes with FedPower, we implement the schemes in Python  [36]. As we only encrypt 0 and 1 in Section 4, we optimize the performance and take advantage of the utilized Paillier cryptosystem. More specifically, we re-randomize the corresponding ciphertexts to obtain new ciphertexts. In addition, we also exploit the homomorphic property, and instead of decrypting each value (d × r × |number o f node devices| times), we first calculate the product of all the ciphertexts (elementary matrix multiplication) and then perform the decryption on a signal matrix. In this way, we obtain the sum of all Gaussian noises more efficiently. The decryption result is the sum of noise which will be canceled in Algorithm 2. Furthermore, we prepare the M i = 1 s i M T i M i , Z (i) 0 and all keys of SA offline for each node device i.
Setup. For our experiments, we used the well-known NETFLIX rating dataset  [37] , and we pre-process it similarly to  [38]  (instead of 10, we removed users and movies with less than 50 ratings). It consists of 96. 310. 835 ratings corresponding to 17. 711 movies from 324. 468 users. We split them horizontally into 100 random blocks to simulate node devices. Moreover, we set the security parameter to 128; thus, we adopt 3072 bits for N in Paillier cryptosystem (this is equivalent to RSA-3072, which provides a 128-bit security level  [39] ). The number of iteration rank and top eigenvectors is set to r = k = 10, and we keep the same synchronous trigger p = 4 as  [18]. To compare FedPower with our enhanced solutions, we set the noise size for these algorithms as σ = σ = 0. 1. Moreover, for Algorithm 3 we bounded M i with 0.
Conclusions Motivated by Guo et al. 's distributed privacy-preserving SVD algorithm based on the federated power method  [18] , we have proposed two enhanced federated SVD schemes, focusing on utility and privacy, respectively. Both use secure aggregation to reduce the added noise, which reverts to the initial design intent and interest. Yet, the added cryptographic operations trade efficiency for superior performance (×10 better results) while providing either similar or superior privacy guarantee. Our work leaves several future research topics. One is to further investigate the computational complexity, particularly the secure aggregation, to achieve more efficient solutions. Another is to investigate the scalability of the proposed solutions, regarding larger datasets and different datasets in applications other than recommendation systems.
Reference DTMs The baseline referencing data of this work is the 59 m/pixel SLDEM that is available from the USGS (the United States Geological Survey) site: https://astrogeology. usgs. gov/ search/map/Moon/LRO/LOLA/Lunar_LRO_LOLAKaguya_DEMmerge_60N60S_512ppd (accessed on 17 December 2022). The SLDEM is based on the GRAIL (Gravity Recovery and Interior Laboratory mission) controlled LOLA DTM  [35] , which has a horizontal accuracy of ~10 m and vertical accuracy of ~0. 5 m  [36] , but with improved spatial coverage using photogrammetric DTMs that are independently derived from the SELENE TC stereo images  [37]. The SLDEM has a slightly lower vertical accuracy of about 3-4 m but an improved spatial resolution of 59 m/pixel compared to the LOLA 118 m/pixel DTM  [12]. SLDEM provides the most accurate geodetic framework of the Moon to date.
However, the resolution gap between the SLDEM and the input LROC NAC images and DTMs are too large to achieve sensible co-registration and co-alignment directly. In this work, the CE-2 global photogrammetric DTM, i. e. , CE-2 DTM, produced by the National Astronomical Observatories, Chinese Academy of Sciences (NAOC), is used as an intermediate referencing dataset to bridge the resolution gap between SLDEM and LROC-NAC. The 20 m/pixel CE-2 DTM is available publicly from the NAOC site (https://moon. bao. ac. cn/ce5web/moonGisMap. search (accessed on 17 December 2022)). The CE-2 DTM was produced using 384 selected single-strip 7 m/pixel CE-2 stereo images (with forward and backward viewing angles of 7. 98 • and -17. 2 • , respectively)  [14, 15]. The CE-2 DTM is currently the highest-resolution global DTM of the lunar surface.
based on the GRAIL (Gravity Recovery and Interior Laboratory mission) controlled LOLA DTM  [35] , which has a horizontal accuracy of ~10 m and vertical accuracy of ~0. 5 m  [36] , but with improved spatial coverage using photogrammetric DTMs that are independently derived from the SELENE TC stereo images  [37]. The SLDEM has a slightly lower vertical accuracy of about 3-4 m but an improved spatial resolution of 59 m/pixel compared to the LOLA 118 m/pixel DTM  [12]. SLDEM provides the most accurate geodetic framework of the Moon to date. However, the resolution gap between the SLDEM and the input LROC NAC images and DTMs are too large to achieve sensible co-registration and co-alignment directly. In this work, the CE-2 global photogrammetric DTM, i. e. , CE-2 DTM, produced by the National Astronomical Observatories, Chinese Academy of Sciences (NAOC), is used as an intermediate referencing dataset to bridge the resolution gap between SLDEM and LROC-NAC.
Input LROC NAC Images The input data of this work are the LROC NAC single-strip images. The LROC NAC instrument  [18]  captures repeat-pass line-scanning panchromatic images at 0. 5-2 m/pixel resolution over a swath width of 2. 5-10 km (~10,000 pixels wide) and a swath length of ~25 km (~52,000 pixels long). Each of the two LROC NAC cameras has a 700 mm focal length telescope and a 5064 pixels CCD (charge-coupled device) line array providing a cross-track field of view of ~2. 85 NAC CCD arrays provides doubled observation swath width. The raw LROC NAC records have a bit depth of 12-bit and are compressed to 8-bit images for a better signal-to-noise ratio  [18]. The LROC NAC EDR (Engineering Data Record) images are publicly available from the LROC PDS archive (https://pds. lroc. asu. edu/data/LRO-L-LROC-2-EDR-V1.
Input LROC NAC Images The input data of this work are the LROC NAC single-strip images. The LROC NAC instrument  [18]  captures repeat-pass line-scanning panchromatic images at 0. 5-2 m/pixel resolution over a swath width of 2. 5-10 km (~10,000 pixels wide) and a swath length of ~25 km (~52,000 pixels long). Each of the two LROC NAC cameras has a 700 mm focal length telescope and a 5064 pixels CCD (charge-coupled device) line array providing a cross-track field of view of ~2. 85°. A cross-track overlap of ~135 pixels for the two LROC NAC CCD arrays provides doubled observation swath width. The raw LROC NAC records have a bit depth of 12-bit and are compressed to 8-bit images for a be er signal-tonoise ratio  [18]. The LROC NAC EDR (Engineering Data Record) images are publicly available from the LROC PDS archive (h ps://pds.
Network Training and Testing The training dataset for the MADNet Moon model consists of 22,084 ORI and DTM pairs (512 × 512 pixels subsets at 5 m/pixel) which were formed from 392 pairs of downsampled LROC NAC PDS ORIs and DTMs. We follow the same methods that are described in  [25]  to form the training dataset. These include a downsampling process (from 2-3 m/pixel to 5 m/pixel for DTMs and from 0. 5-1 m/pixel to 5 m/pixel for ORIs) in order to average out several high-frequency photogrammetric artefacts, a structural similarity index measurement (SSIM)  [48]  assisted manual screening process to remove the low-quality ORI and DTM pairs, and data augmentation that uses horizontal and vertical flipping. The.
Network Training and Testing The training dataset for the MADNet Moon model consists of 22,084 ORI and DTM pairs (512 × 512 pixels subsets at 5 m/pixel) which were formed from 392 pairs of downsampled LROC NAC PDS ORIs and DTMs. We follow the same methods that are described in  [25]  to form the training dataset. These include a downsampling process (from 2-3 m/pixel to 5 m/pixel for DTMs and from 0. 5-1 m/pixel to 5 m/pixel for ORIs) in order to average out several high-frequency photogrammetric artefacts, a structural similarity index measurement (SSIM)  [48]    At the initial training and tuning stage, we left out 1000 training pairs to form the test dataset. For testing purposes, root mean squared errors (RMSEs) and mean SSIMs  [48]  are used as evaluation metrics and are periodically monitored throughout the initial training process.
SSIM is the locally computed structural similarity index metric derived using patterns of pixel intensities among neighbouring pixels with normalised brightness and contrast  [48]. RMSE measures the pixel-wise differences between the inference results and the ground-truth DTMs, while SSIM complementarily measures the differences in structural features between the inference results and the ground-truth DTMs. Both RMSE and mean SSIMs are widely used as loss functions and evaluation metrics in monocular depth/height estimation studies (e. g. ,  [49] [50] [51] ). depth/height estimation studies (e. g. ,  [49] [50] [51] ). Figure  5  shows four randomly selected examples from the test results in comparison to the input images and ground-truth height maps from the test dataset. The corresponding RMSE and mean SSIM measurements of the exemplars are shown in Figure  5.
11 m, standard deviation 104. 42 m) using our in-house 3D co-alignment pipeline that is described in  [38, 39]. The co-aligned 20 m/pixel CE-2 DTM is then refined into 14 m/pixel using the same MADNet Moon model with the 7 m CE-2 ORI (refer to Section 2. 4). In step (2), a series of USGS-ISIS (Integrated Software for Imagers and Spectrometers) based pre-processing functions are applied to the raw input LROC NAC PDS images. These include the data format conversion (lronac2isis), radiometric calibration (lronacccal), echo effects removal (lronacecho), ancillary information initialisation (spiceinit), and map projection and orthorectification with respect to the SLDEM (cam2map). After pre-processing, the data cubes (in USGS-ISIS format) are converted to GeoTiff images using GDAL (Geospatial Data Abstraction Library; refer to https://github.
4). In step (2), a series of USGS-ISIS (Integrated Software for Imagers and Spectrometers) based pre-processing functions are applied to the raw input LROC NAC PDS images. These include the data format conversion (lronac2isis), radiometric calibration (lronacccal), echo effects removal (lronacecho), ancillary information initialisation (spiceinit), and map projection and orthorectification with respect to the SLDEM (cam2map). After pre-processing, the data cubes (in USGS-ISIS format) are converted to GeoTiff images using GDAL (Geospatial Data Abstraction Library; refer to h ps://github. com/OSGeo/gdal/releases/tag/v3. 6. 2 (accessed on 17 December 2022)). In step (3), the pre-processed LROC NAC images are automatically co-registered with the CE-2 image using the ENVI ® Modeler software (h ps://www.