Interpretable Encoding of Galaxy Spectra

Yan Liang
Department of Astrophysical Sciences, Princeton University
Princeton NJ 08544, USA
yanliang@princeton.edu
Peter MelchiorDepartment of Astrophysical Sciences, Princeton University,
Center for Statistics & Machine Learning, Princeton University
Princeton NJ 08544, USA
peter.melchior@princeton.edu
Sicong Lu
Department of Physics and Astronomy, University of Pennsylvania
Philadelphia PA 19104, USA
siconglu@upenn.edu

Abstract
We present a novel loss function to train autoencoder models for galaxy spectra.
Our architecture reliably captures intrinsic spectral features regardless of redshift,providing highly realistic reconstructions for SDSS galaxy spectra using as little
as two latent parameters. But the interpretation of encoded parameters remains
difficult because the decoding process is non-linear and the latent space can behighly degenerate: different latent positions can map to virtually indistinguishable
spectra. To resolve this encoding ambiguity, we introduce a new similarity loss,
which explicitly links latent-space distances to data-space distances. Minimizing
the similarity loss together with the common fidelity loss leads to non-degenerate,
highly accurate spectrum models that generalize over variations in noise, masking,and redshift, while providing a latent space distribution with clear separations
between common and anomalous data.

1Introduction

Spectroscopy is a critical tool to probe the physical mechanisms that drive the formation andevolution of present-day galaxies. Despite the vast amount of available galaxy spectra provided bylarge spectroscopic surveys, extracting physical knowledge from them is still a difficult task. Ideally,
one would infer galaxy properties by directly fitting the observed spectrum to a theoretical model,
but analytical models are not yet sophisticated enough to reproduce typical individual high S/Ngalaxy spectra, especially the strong emission lines [1, 2]. The physical processes contributing to theobserved spectral features may be still poorly understood, thus using oversimplified models could
lead to biased interpretation of the data.
Alternatively, one may construct a fully data-driven model via unsupervised learning. The main
challenge in this approach is to properly disentangle the intrinsic physical spectra from redshift(causing a stretching of the spectra), noise level, and artifacts such as telluric contamination. Linearmodels, i.e. a combination of empirical or theoretical templates [3, 4], are commonly used for redshift
Machine Learning and the Physical Sciences workshop, NeurIPS 2022.

RedshiftzSpectrum

Encoderxx1

Latent Variablefθ(x, z)

x2

Restframe Model

Decodersgϕ(s)

x′x′1 x′2

s1 s2

Reconstructionx′′h(x′, z)Redshift
Resamplex′1′x′2′￼￼￼￼￼￼￼￼￼￼

￼

￼￼￼

Similarity loss:If x′1 ≈ x′2, then s1 ≈ s2
Fidelity loss: x ≈ x′′

Figure 1: Our autoencoder loss function combines fidelity loss, which compares separately input
spectra (x1 , x2 ) to their redshifted and resampled reconstructions (x′′1 , x′′2 ), with a novel similarity
loss, which links the distance in latent space |s1 − s2 | between two spectra to their distance inrestframe |x′1 − x′2 |. Spectra of two physically similar galaxies observed at different redshifts (z1 , z2 ),for which the underlying restframe models are very similar, will thus yield similar latent vectors.

estimation and spectral classification [5, 6]. The reconstruction power of linear models is limited bytemplate quality, and often requires many components to achieve a good fit.
Autoencoders (AE) can yield models with good fidelity and small latent dimensionality [7]. But
for conventional AEs, all galaxy spectra needed to be de-redshifted and resampled to a common
restframe, restricting either redshift or wavelength ranges that can be probed. [8] showed that by
explicitly adding a redshift transformation to the decoder path, one can utilize the entire spectrum forgalaxies at all redshifts. However, the authors noted that redshift-invariant encoding was achieved
only over a limited redshift range, where a set of important “consensus features” was observablein the each spectrum [9]. We now introduce a new loss term that solves this problem by relating
latent space distances to distances between reconstructed restframe models. Our approach resolves
degeneracies in the decoding process and establishes an encoding that is robustly invariant to changes
in redshift. At no reduction in fidelity, the latent space also becomes directly interpretable—thespectra of physically similar galaxies cluster, and latents in the same neighborhood reconstructsimilar-looking spectra.

2Data

We obtain 500,000 galaxy spectra spanning redshift z ∈ [0, 0.5] from Sloan Digital Sky Surveys
Data Release 16 [10]. Our sample includes all optical spectra that are classified as galaxies andhas redshift error zerr < 10−4 . Approximately 70% of the samples are used for training, 15% forvalidation, and 15% is held for test. All spectra are normalized by the median flux and zero-padded toa homogeneous wavelength λobs = 3784...9332Å. We mask out telluric contamination by assigningzero weights to within 5Å of the top ∼100 telluric lines, amounting to 12% of the data vectors.

3Method

Our model architecture is taken from [8] (see Figure 1 for an overview and the aforementioned paperfor details). Let x ∈ RM denote an input spectrum with M = 3921 elements. It gets encoded, bya modified version of the CNN encoder from [11], into a low-dimensional latent respresentation,s ∈ R2 . A standard MLP with (256, 512, 1024) nodes and a leaky ReLU activation generates a
restframe spectrum x′ , whose 7000 spectral elements are chosen to create a mildly super-resolved
representation with an extended wavelength coverage (λrest = 2359 ... 9332 Å). The reconstructed
spectrum x′′ is then redshifted and linearly interpolated from x′ to the same wavelengths as x.
2

This architecture allows for a redshift-invariant encoding as the reshift transformation is explicitly
performed in the generator.
Our extended loss function
Ltotal = Lfid + Lsim + Lc
(1)
operates on all four of the stages shown in Figure 1. The fidelity loss term Lfid quantifies thereconstruction quality, assuming normally distributed noise. It measures the mean log-likelihood of
the reconstruction of spectral elements averaged over batches of N spectra with spectral size M :N

Lfid =

1 X
wi ⊙ (xi − x′′i )2 ,2N M i(2)where xi is the i-th input spectrum, wi its inverse variance, and ⊙ the element-wise multiplication.
We define a similarity loss term that, unlike the fidelity loss, operates on the two intermediate stages.
Let s = fθ (x, z) be the encoded latent vector and x′ = gϕ (s) be the restframe model, where fθ , gϕ
are parameterized encoder and decoder functions. Ideally, if two restframe models, x′i and x′j aresimilar, their latent positions si and sj should be similar as well; otherwise, the mapping from latent
space to spectrum space becomes difficult to interpret. On the other hand, distinctively different
models should have well-separated latent positions. While one could intuitively expect an autoencoder
to establish such a relation in its latent space, we find empirically that this is often not the case.
However, the desired relation can be satisfied naturally if the latent distance is proportional to aspectral distance: |s1 − s2 |2 ∝ |x′1 − x′2 |2 , which motivates us to set up a loss term as follows:1
1Sij = |si − sj |2 −|w′ ⊙ (x′i − x′j )|2(3)2MN N
1 XXsigmoid(k1 Sij − k0 ) + sigmoid(−k1 Sij − k0 )(4)
Lsim (k0 , k1 ) = 2N i jwhere w′ is the inverse-variance weight defined at restframe wavelengths, and k0 , k1 are adjustablehyper-parameters that control the steepness of the slope. Lsim encourages pairwise latent space
distances proportional to the spectra (dis)similarity defined in Equation 3. The double sigmoid
function in Equation 4 serves two purposes. It limits the effect of the similarity loss by setting
Lsim ≤ 1, such that it is only important when the fidelity loss is low Lfid ≲ 1. And it provides
relatively smooth gradients (compared to e.g. Sij2 ), improving the trainability of the model. It iscrucial to measure the similarity between restframe pairs rather than input pairs, because the former
provides a stable measure independent of redshift and observational window. In addition, restframe
models usually contain less noise than the raw data (see Figure 2, bottom left panel).
Inspired by [12], we add a the consistency loss as a more guided form of the similarity loss:

N
1
1 X
2
Lc =sigmoid|s−s|− 0.5, σs = 0.1,iaug,iN i2σs2(5)where si and saug,i are the latent positions of the original and its corresponding augment spectrum,that has been redshifted by a random znew from a uniform distribution between [0, 0.5]. Minimizing
Lc reduces the latent distances between galaxy spectra and their augments, improving the encoding
stability against redshift. The consistency loss reinforces that physically similar spectra pairs arepulled together in latent space, while dis-similar pairs are moved apart by the (dis)-similarity loss
term Equation 3.

4Results

We implement the autoencoder architecture using pytorch. To evaluate the impact of different loss
terms, we trained three models using different strategies: In the first training, we optimize the modelwith Lfid alone to serve as the baseline. In the second model, we optimize Lfid + Lsim simultaneously,with an “inverse-annealing” cycle to gradually increase the slope k1 , while k0 is held constant. For
the third model, we add the consistency regularization and optimize the full loss from Equation 1.
We train each model on a NVIDIA A100 GPU using the same training and validation data set for
700 epoch and observe convergence. The results are summarized in Table 1. The best performance3

Training objective

Lfid

Lsim

Lc

|si − saug,i |fidelity
fidelity + similarity
fidelity + similarity + consistency

0.476
0.473
0.470

0.158
0.155

0.001

2.496
0.450
0.007

Table 1: Training performance using different strategies, evaluated on the test set. Lsim is evaluatedassuming k0 = 2.5, k1 = 1.0. Note that when the latent distance and spectral distance is perfectlyaligned, the minimum similarity loss Lsim is 0.151.

Figure 2: Left: Observed spectrum (at z = 0.04, black) from the test data, its reconstruction (red),and reconstructions of augmented spectra with artificially altered redshifts (color-coded). Zoomed-in
versions are shown on the top. The colored-bar in the bottom-left shows the observed wave-length
range of the augments. Right: 2D latent space distribution of 10,000 SDSS spectra randomly selectedfrom the test set. The red circle marks the example spectrum, and the triangles mark its augments(essentially indistinguishable in the plot).

is achieved with all three losses combined. In particular, our best model reduces the average latentdistance of augmented redshifted spectra, as a measure of redshift invariance in latent space, by afactor of 350 comparing to the fidelity-only model. This improvement has been achieved without any
decrease in fidelity. On the contrary, minor gains in fidelity could indicate that the model benefits
from a non-degenerate latent space, where each latent position maps to more spectra from a wider
redshift range. We suggest that the consistency loss should not be used without the similarity loss
because it seeks to collapse the latent distribution at the origin of the latent space. Instead, it is best
added once a model trained with fidelty and similarity losses is available.
With the best model, we achieve a desirable redshift-invariant behavior. Figure 2 shows the reconstructed restframe models and encoded latent positions of an example galaxy. The latent positions of
the original and redshift-augmented spectra are stably grouped together (colored markers in rightpanel), even for the z = 0.45 augment (colored in orange) where the dominant Hα emission line
is outside of the observable wavelength range, i.e. unavailable to the encoder. The reconstruction
quality is evidently robust to such missing features.
45Conclusion

We introduced a novel method to establish an interpretable, redshift-invariant encoding of galaxy
spectra. Our model generates highly realistic spectra for the entirety of the SDSS spectroscopic galaxy
sample with only two latent parameters, making it a powerful dimensionality reduction method. We
define a new loss term to relate distances in latent space to distances in restframe space. It discourages
latent-space degeneracies and pulls intrinsically similar galaxies together, regardless of redshift. The
decoder can thus learn a more complete underlying restframe model over the entire redshift range.
In addition, the new loss term encourages an encoding that is well-suited for anomaly detection:
Outliers will be pushed away from more common samples, who themselves will tend to cluster.

6Limitations

For simplicity, we have fixed the latent space dimension to 2. This choice allows for direct visualization and does not noticeably affect the quality of the reconstructed spectra. Even though the detailed
performance may differ, the usefulness of our proposed similarity loss will transparently translate tohigher-dimensional latent spaces.
Because of the magnitude limit of SDSS, galaxies with z > 0.3 galaxies are underrepresented in ourdataset. This issue can be addressed by training the autoencoder jointly with additional datasets suchas the Baryon Oscillation Spectroscopic Survey (BOSS)[13].

7Broader Impact

The similarity loss term is agnostic to the specific design of the autoencoder. We expect it to establish
interpretable, non-degenerate, outlier-sensitive latent space distributions for other types of data.

References[1] Rita Tojeiro, Alan F Heavens, Raul Jimenez, and Ben Panter. Recovering galaxy star formationand metallicity histories from spectra using vespa. Monthly Notices of the Royal AstronomicalSociety, 381(3):1252–1266, 2007.
[2] Rita Tojeiro, Will J Percival, Alan F Heavens, and Raul Jimenez. The stellar evolution ofluminous red galaxies, and its dependence on colour, redshift, luminosity and modelling.
Monthly Notices of the Royal Astronomical Society, 413(1):434–460, 2011.
[3] Michael JI Brown, John Moustakas, J-DT Smith, Elisabete Da Cunha, TH Jarrett, Masatoshi
Imanishi, Lee Armus, Bernhard R Brandl, and Josh EG Peek. An atlas of galaxy spectral energy
distributions from the ultraviolet to the mid-infrared. The Astrophysical Journal Supplement
Series, 212(2):18, 2014.
[4] M Polletta, M Tajer, L Maraschi, G Trinchieri, CJ Lonsdale, L Chiappetti, S Andreon, M Pierre,O Le Fevre, G Zamorani, et al. Spectral energy distributions of hard x-ray selected active
galactic nuclei in the xmm-newton medium deep survey. The Astrophysical Journal, 663(1):81,2007.
[5] Ashley J Ross, Julian Bautista, Rita Tojeiro, Shadab Alam, Stephen Bailey, Etienne Burtin,
Johan Comparat, Kyle S Dawson, Arnaud De Mattia, Hélion du Mas des Bourboux, et al.
The completed sdss-iv extended baryon oscillation spectroscopic survey: Large-scale structure
catalogues for cosmological analysis. Monthly Notices of the Royal Astronomical Society,498(2):2354–2371, 2020.
[6] Zachary J Pace, Christy Tremonti, Yanmei Chen, Adam L Schaefer, Matthew A Bershady,Kyle B Westfall, Médéric Boquien, Kate Rowlands, Brett Andrews, Joel R Brownstein, et al.
Resolved and integrated stellar masses in the sdss-iv/manga survey. ii. applications of pca-basedstellar mass estimates. The Astrophysical Journal, 883(1):83, 2019.
[7] Stephen KN Portillo, John K Parejko, Jorge R Vergara, and Andrew J Connolly. Dimensionality
reduction of sdss spectra with variational autoencoders. The Astronomical Journal, 160(1):45,2020.
5

[8] Peter Melchior, Yan Liang, ChangHoon Hahn, and Andy Goulding. Autoencoding Galaxy
Spectra I: Architecture. arXiv e-prints, page arXiv:2211.07890, November 2022.
[9] Peter Melchior, ChangHoon Hahn, and Yan Liang. Autoencoding galaxy spectra. In ICML
Workshop Machine Learning for Astrophysics, 2022.
[10] Romina Ahumada, Carlos Allende Prieto, Andrés Almeida, Friedrich Anders, Scott F Anderson,
Brett H Andrews, Borja Anguiano, Riccardo Arcodia, Eric Armengaud, Marie Aubert, et al.
The 16th data release of the sloan digital sky surveys: first release from the apogee-2 southern
survey and full release of eboss spectra. The Astrophysical Journal Supplement Series, 249(1):3,2020.
[11] Joan Serrà, Santiago Pascual, and Alexandros Karatzoglou. Towards a universal neural network
encoder for time series. In CCIA, pages 120–129, 2018.
[12] Samarth Sinha and Adji B. Dieng. Consistency regularization for variational auto-encoders.
CoRR, abs/2105.14859, 2021.
[13] Kyle S Dawson, David J Schlegel, Christopher P Ahn, Scott F Anderson, Éric Aubourg, StephenBailey, Robert H Barkhouser, Julian E Bautista, Alessandra Beifiori, Andreas A Berlind, et al.
The baryon oscillation spectroscopic survey of sdss-iii. The Astronomical Journal, 145(1):10,
2012.

6