Searching for the Weirdest Stars: A Convolutional Autoencoder-Based Pipeline for Detecting Anomalous Periodic Variable Stars
Abstract
The physical processes of stars are encoded in their periodic pulsations.
Millions of variable stars will be observed by the upcoming Vera Rubin Observatory’s Legacy
Survey of Space and Time. Here, we present a convolutional autoencoder-based pipeline as an automatic approach to search for anomalous periodic variables within The Zwicky Transient Facility Catalog of Periodic Variable Stars (ZTF CPVS).
We encode their light curves using a convolutional autoencoder, and we use an isolation forest to sort each periodic variable star by an anomaly score with the latent space.
Our overall most anomalous events share some similarities: they are mostly highly variable and irregular evolved stars.
An exploration of multiwavelength data suggests that they are most likely Red Giant or Asymptotic Giant Branch stars concentrated in the disk of the Milky Way.
Furthermore, we use the learned latent feature for the classiﬁcation of periodic variables through a hierarchical random forest.
This novel semi-supervised approach allows astronomers to identify the most anomalous events within a given physical class, accelerating the potential for scientiﬁc discovery.
Introduction
Anomaly detection is a vital aspect of making discoveries in astronomy.
Examples include the anomalies in the CMB temperature anisotropies [1], quasi-stellar objects [2] and dark energy [3].
As deep-sky surveys via modern telescopes continue to exponentially increase our discovery rates of galactic and extra-galactic transients, researchers are turning towards automated methods of anomaly detection [4–6].
Advanced techniques to search for anomalous astrophysical events are essential in the era of upcoming observatories.
In particular, the Legacy Survey of Space and Time (LSST) conducted by the Vera
Rubin Observatory is expected to commence operations in 2024 [7] and is anticipated to observebillion objects within its 10 years of operation [8].
It is reasonable to expect anomalous periodic variables stars (PVSs) which defy expectations.
Indeed, their discoveries have already been challenging our understanding of the Galactic metallicity [9], the physics of accretion and mass transfer [10, 11], etc.
Despite the impact brought by these discoveries, studies on PVSs that utilize machine learning have only been made on classiﬁcations [12–14], and deep generative modeling for parameter estimation [15].
While Malanchev et al.
[16] searched for anomalous transient detected with The
Zwicky Transient Facility, they are not speciﬁcally aiming for PVSs.
Here, we provide an anomaly detection pipeline to effectively search for peculiar PVSs detected with The Zwicky Transient Facility.
The Zwicky Transient Facility contains numerous publicly available data which serve as a good training set for big data problems in astronomy which will arise with LSST.
We anticipate that our pipeline can be applied once the observational data from the Vera Rubin Observatory (or other, new facilities) are made available.
Method
Here we describe our training set and methodology.
Our training set consists of the ZTF CPVS presented in Chen et al.
The ZTF CPVS utilizes the Data Release 2 archive of The Zwicky Transient Facility [18] to search for and classify new PVSs down to a r − band magnitude, a measurement of brightness, of ∼ 20 . 6 .
They ﬁnd a total of 781 , 602 PVSs, of which 621 , 702 are newly discovered.
The data are given in two ﬁlters ( g − band and r − band) which are observed asynchronously.
The ZTF
CPVS provides periods, which we use to phase-fold the light curves.
To phase-fold a light curve, we cut the time series into multiple sub-series with a time duration equal to the period and stack these sub-series on top of each other.
Because the data are irregularly sampled by the telescope and taken in both bands, we choose to interpolate the phase-folded light curves using the multivariate Gaussian process [MGPR, 19, 5, 20]) with periodic boundary conditions.
The MGPR has a mean function η ( φ, λ ) and a covariance function K .
We set η ( φ, λ ) = 0 , where φ is the temporal phase and λ is a wavelength in scaled units.
We choose the following covariance function:
[Equation]
Here, (cid:126)r = ( (cid:126)φ,(cid:126)λ ) is a high dimensional position vector, while l φ and l λ measure the correlation along the phase and wavelength direction respectively, C is a constant, and δ measures the white noise level of the raw data.
After ﬁtting the kernel function, we generate 160 evenly spaced data points along with the phase direction for both g − and r − bands.
Following the approach of Villar et al.
[21] and Villar et al.
[6], we stack both of them horizontally to form an ‘image’ of size 2 × 160 .
The ‘images’ will be encoded through the convolutional variational autoencoder [C-VAE, 22] with a
LeNet structure[23].
We select 730 , 184 of data and split them into a train to validation to test ratio of 7 : 2 : 1 .
The architecture of the encoder is described as follows:
Input layer of size 2 ×2.
3 Convolutional layers with the ReLu activation with Dropout 3.
Dense layer with 256 neurons, Linear Activation 4.
Latent space of size 2 ×[-
-]
The ﬁlter size of the convolutional layer increases from 32 → 64 → 128 , and the dropout fraction is set to 0 .
Given the variational nature of the autoencoder, the bottleneck latent space is described by a mean vector (cid:126)µ and the diagonal covariance matrix σ ii .
Inspired by the work of Zhang and Bloom [13], we apply periodic padding to the ‘images’ during the convolution to enforce periodic boundary conditions.
Our decoder is the symmetric counterpart of the encoder; however, no periodic padding and no dropout are applied.
We train with early stopping with an epoch of 895 , which takes roughly 6 hours running on the NVIDIA RTX2080-Ti GPU.
Once trained, we next use an isolation forest to rank each periodic variable star by its anomaly score.
The isolation forest works by building an ensemble of binary trees, which work to isolate samples of the population.
The anomalies are identiﬁed as requiring few trees to isolate the event [24]. In this work, we focus on events with the top 100 most anomalous scores.
We use the latent vectors (cid:126)µ , the log of period log 10 τ , and the difference between the average g − and r − band magnitude ( m gr := (cid:104) m g (cid:105) − (cid:104) m r (cid:105) ) as input features of the isolation forest [25].
We note that the period and magnitudes are explicitly included because this information is lost in the pre-processing used to train our autoencoder; however, we believe that they will be valuable in ﬁltering out anomalies.
We use the isolation forest implemented in the scikit-learn package with based estimators of 100 , 000 1 .
The use of the isolation forest is inspired by its simplicity and robustness. We noticed that there are detection scores specialized for deep-generative networks, such as the Mahalanobis conﬁdence score [26–29], which computes distances between data points to a distribution (assumed Gaussian). It is not applicable in our case, because we have appended extra variables into our feature space in which they are not necessary Gaussian. Furthermore, we are ultimately interested in the relative rankings but not the absolute scores of the anomalies.
A simple yet robust isolation forest should meet our expectations.
We remark that reconstruction scores are also widely used as indicators for anomalies [30, 31], but our empirical ﬁndings suggest that they perform roughly similar to the isolation forest.
Finally, we encourage readers to refer to Figure 1 as a review of our pipeline.
We additionally use our learned latent space to classify 2 the ZTF CPVS by their physical origin.
We note that the ZTF CPVS provides class labels for PVSs, but the classiﬁcations are based on hand-engineered features only.
Here we provide an alternative classiﬁcation method that makes use of a hierarchical random forest classiﬁer.
We extracted events labels from the SIMBAD catalog [32] by cross-matching (using the python package Astroquery [33]) their sky-coordinates with those listed in the ZTF CPVS.
The SIMBAD catalog contains class labels obtained, typically, through spectroscopic analysis, which is more reliable but often expensive.
We found 31 , 541 of successfully cross-matched objects.
We then construct 13 classes in 2 levels.
The ﬁrst level includes Active Galactic Nuclei-like (AGNL), Cepheid (CEP), Eclipsing Binaries (EB), Long-Period Variables (LPV),
Mira variables (Mira), other Pulsating Variables (Pul oth ), RR Lyrae (RR), and Peculiar Variables (Pec).
The second level is further classiﬁcations of the Pec type, and it includes Carbon stars (CType), Horizontal Branch stars (HB), Red Giant Branch stars (RGB), S-Type stars (S-Type), Young
Stellar Object-like (YSOL), and Other Variables (V oth ).
They will be serving as the data set of our classiﬁcation model.
We split the data set into a training-to-test set ratio of 7 : 3 by using the python package scikit-learn [34].
We note that our training set is highly imbalanced, with the largest set containing 10 , 745 events and the smallest containing just 41 events.
We balanced the training set using the python package imbalanced-learn [35] with default learning parameters, which performs synthetic minority resampling [36, 35]. Finally, we train the hierarchical random forest classiﬁer provided by imbalanced-learn , with no hyperparameter tuning performed.
[Figure]
[Figure]
We ﬁnd that our latent space exhibit an annular structure, which inspires us to transform the latent vector into an N-dimensional spherical coordinate.
We run our isolation forest on both sets of coordinates and compare results, selecting the top 100 most anomalous events in both cases.
Our anomaly detection algorithm is seemingly sensitive to the irregular oscillating which consists of several dominant Fourier modes.
In general, these anomalous events are both multi-modal and highly irregular, with some examples exhibiting larger ﬂuctuations that span over several magnitudes.
Some of the anomalies also show weak or even anti-correlation between the g − and r − band light curves, suggesting signiﬁcant temperature variations within the pulsations.
To better understand the nature of the selected anomalies, we extracted Gaia [39] G-band absolute magnitudes M G , and the difference between the Gaia B-band and R-band absolute magnitudes
G BP − G RP for the top 100 anomalies in both latent spaces.
We plot their distribution in Figure 3 (a).
We note that this is a common diagnostic phase space, which roughly correlates with the temperature and luminosity of the stars.
The majority of the anomalies are cool , with G BP − G RP >and luminous with 0 < M G < 2 .
These properties correspond to evolved, luminous, and cold stars. Furthermore, we show the distribution of the top anomalies in the Milky Way galactic coordinates in A G N L C E P E B
L [Table] V
M
[Table]
r a
[Table]
RR
Predicted label
AGNL
CEP
EB
LPV
Mira
Pec
Puloth
RR
T r u e l a b e l
[Table]
Predicted label
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table]
[Table] [Table] [Figure] Figure 3 (b).
The majority of the anomalies concentrate in the Milky Way galactic disk, implying that (1) there is signiﬁcant interstellar reddening due to the dust for these events and (2) the progenitor systems of these events are consistent with young and massive stars.
Taken together, the observational evidence points to highly anomalous, young, cool, and massive Red Giant or Asymptotic Giant Branch stars.
Spectroscopic follow-up observations and detailed light curve modeling is essential in fully understanding the anomalies detected in our data-driven pipeline; however, we note that these anomalies were discovered with limited survey observations.
Similar techniques will be invaluable for future missions.
Finally, we highlight the classiﬁcation results from our hierarchical classiﬁer in Figure 4 (a) and (b).
We classify for substantially more classes with our newly developed method.
For example, we ﬁnd that 97 .
9% of the AGNL objects (objects associated with supermassive black holes in other galaxies) are labeled as semi-regular galactic variable stars in previous studies.
We hope that a closer investigation of these labels can lead to improved purity in the ZTF CPVS. We note that we do not use the same class labels as the original ZTF CPVS, making it difﬁcult to directly use the ZTF CPVS as a baseline model.
A detailed comparison between classiﬁcation methods will be left to the future. Nonetheless, we ﬁnd that the new classiﬁcation model for the ZTF CPVS that uses our learned latent space is reasonably accurate.
Last but not least, we plot the latent distribution using our new labels in Figure 2, to show the robustness of our autoencoder in differentiating objects that belongs to different categories.
Conclusion
We present a convolutional autoencoder-based pipeline as an automatic yet robust approach to (1) search for anomalous PVSs within a sea of data.
(2) create a new classiﬁcation model for PVS.
Our pipeline can be applied to PVS data obtained from deep-sky surveys in the future.
Last but not least, our pipeline generates a list of anomalous PVSs.
Detailed spectroscopic follow-up is essential to reveal their true identity and how they ﬁt into our current understanding of late-stage stellar processes.
Astrophysical big data has become a rapidly growing ﬁeld in recent years.
We anticipate our method can contribute to the community, such as detecting other kinds of anomalous periodic/nonperiodic transients or better classifying different categories of astronomical objects.
Furthermore, our unsupervised learning pipeline can make use of supervised object labels to look for categorical-wise anomalies, which could potentially help better understand the diversities within classes and improve the labeling and classiﬁcations of stellar objects.
Techniques similar to those presented here can be used in broader applications to identify anomalies in periodic/non-periodic, multi-variate time series.  Acknowledgement
We thank the Flatiron institute for providing computer cluster access.
We thank Prof. Ming-Chung,
Chu in the Chinese University of Hong Kong for providing the NVIDIA RTX2080-Ti GPU as the computational resources for the neural network training.
We also thank Prof. Maria Drout and
Anna O’Grady at the University of Toronto; Matteo Cantiello, Mathieu Renzo, and Adam Jermyn at the Center of Computational Astrophysics, Flatiron Institute for their valuable discussion on the anomalous periodic variables stars.
VAV acknowledges support by the Simons Foundation through a
Simons Junior Fellowship (#718240) during the early phases of this project, as well as support in part by the NSF through grant AST-