AmortizedPeriod: Attention-based Amortized Inference for Periodicity Identification
Authors: Hang Yu
Cong Liao
Ruolan Liu
Jianguo Li
Hu Yun
Xinzhe Wang
Venue: ICLR 2024 poster (2024)
Keywords: Periodicity Identification
Probabilistic Models
Amortized Inference
Attention
Time Series Analysis
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: A novel amortized inference framework is proposed for accurate and fast periodicity identification

Abstract: Periodic patterns are a fundamental characteristic of time series in natural world, with significant implications for a range of disciplines, from economics to cloud systems. However, the current literature on periodicity detection faces two key challenges: limited robustness in real-world scenarios and a lack of memory to leverage previously observed time series to accelerate and improve inference on new data. To overcome these obstacles, this paper presents AmortizedPeriod, an innovative approach to periodicity identification based on amortized variational inference that integrates Bayesian statistics and deep learning. Through the Bayesian generative process, our method flexibly captures the dependencies of the periods, trends, noise, and outliers in time series, while also considering missing data and irregular periods in a robust manner. In addition, it utilizes the evidence lower bound of the log-likelihood of the observed time series as the loss function to train a deep attention inference network, facilitating knowledge transfer from the seen time series (and their labels) to unseen ones. Experimental results show that AmortizedPeriod surpasses the state-of-the-art methods by a large margin of 28.5% on average in terms of micro $F_1$-score, with at least 55% less inference time.

AMORTIZEDPERIOD ATTENTION-BASED AMORTIZED INFERENCE FOR PERIODICITY IDENTIFICATION
Hang Yu, Cong Liao, Ruolan Liu, Jianguo Li; Yun Hu, Xinzhe Wang
Ant Group, Hangzhou, 310013, China
ABSTRACT
Periodic patterns are fundamental characteristic of time series in natural world, with significant implications for range of disciplines, from economics to cloud systems. However, the current literature on periodicity detection faces two key challenges: limited robustness in real-world scenarios and lack of memory to leverage previously observed time series to accelerate and improve inference on new data. To overcome these obstacles, this paper presents AmortizedPeriod, an innovative approach to periodicity identification based on amortized variational in- ference that integrates Bayesian statistics and deep learning. Through the Bayesian generative process, our method flexibly captures the dependencies of the periods, trends, noise, and outliers in time series, while also considering missing data and irregular periods in robust manner. In addition, it utilizes the evidence lower bound of the log- likelihhodd of the observed time series as the loss function train deep attention inference network, facilitating knowledge transfer from the seen time series (and their labels) to unseen ones. Experimental results show that AmortizedPeriod surpasses the state-of-the-art methods by a large margin of 28.5% on average in terms of micro Pi-score, with least 55% less inference time
INTRODUCTION
Periodic patterns are omnipresent in this physical world, since the cyclic movements of the earth and the moon give rise recurring events, and further influence the behavior of life on this planet. Identi- fying periods in real-world time series and investigating their causes shed light upon understanding how the world works, and further aid in more informed and accurate modeling and decision- making. Indeed, periodicity detection plays a pivotal role in time series-related tasks, such as classifica- tion (Orsenigo & Vercellis) (010)), clustering (Aghabozorgi al., 2015), decomposition (Wen et al.)
2019, anomaly detection (Zhao et al. 2019). and forecasting (Oreshkin et al. [2019] Liu et al. 2022
Ni et al. 2023). It thereby finds applications and permeates the literature in various domains, such economy and finance (Nerlove et al. 2014), physics and earth science (Kong al. (020). biology and neuroscience (Jing et al. 2022), database and cloud operations (Rzadca et al. 2020), etc.
The large body of literature on periodicity detection can be broadly divided into four groups. The first group identifies periods in the time domain via autocorrelation or partial autocorrelation (Elfeky et al.) 2004 |Breitenbach et al., 2023), whereas the second exploits discrete Fourier transform (DFT) and finds periods in the frequency domain (Vlachos al., 2004 Bauer al. 2020). The third one further combines the time and frequency methods and allows them to complement each other (Vlachos et al. 2005 Wen et al. 202T). However, both autocorrelation and DFT are originally intended for periodicity detection, and so their use requires arduous preprocessing and postprocessing steps to alleviate their deficiencies. Instead, the fourth group (Tenneti & Vaidyanathan 2015) 2016) projects the time series onto the orthogonal Ramanujan subspaces (Vaidyanathan) 2014a b), each representing an exact period, and detects periods by identifying projections with significant energy. Despite the progress, the practical utility of the above approaches is still hindered by two limitations:
Limited robustness: In real time series, multiple periodic components often interlaced with each other, and are further contaminated with trend, noise, outliers, and incomplete observations. Moreover, there may exist irregular periods, such the monthly pattern. Note that the number of
"Corresponding author.
Coode is available http://jiihaa.comaaapi////mmaart

---

days often differs in consecutive months and monthly periods are crucial for real applications such as resource scheduling. is thus imperative to design a periodicity detection approach that robust to 1) multiple and 2) possibly irregular periods well as 3) trend, 4) noise, 5) outliers, and 6) missing data. Unfortunately, the existing approaches only address some of these requirements (cf. Appendix [B). In particular, none of them explicitly consider irregular periods. Additionally, they typically tackle the desiderata sequentially in a pipeline, and may suffer from the issue of error accumulation.
In the aforementioned approaches, each time series is processed in isolation and the inferences are never reused. More specifically, even if the same time series has been analyzed before, analyzing it in the future still requires the same amount of computational cost. Consequently, these methods may incur a daunting computational burden when tackling the huge number of time series that can be found in real applications nowadays (Rzadca et al.) 2020; Wu et al. [023). This problem exacerbated for some methods in the third and the fourth group, as they are more computationally intensive than those in the first two groups. Furthermore, memorylessness indicates that labeling
Memorylessness:
information from the past cannot help to improve the performance of these algorithms in the future, even though human feedback can be available every now and then.
To lift these limitations, we propose AmortizedPeriod, an amortized variational inference based method for periodicity detection. Amortized inference can be viewed as a marriage of Bayesian statisties and deep learning (Zhang et al. 2018, Ganguly et al.) (022). The Bayesian paradigm offers a flexible way to jointly model the period, trend, noise, and outlier components and accounts for the uncertainties resulting from these components as well as the missing data. The deep learning- based inference network further maps the observed time series to the parameters of the variational periodicity distributions. By maximizing the evidence lower bound (ELBO) of the log-likelihood of the observed time series in a self-supervised manner, the resulting inference model learns to mimic the effect of probabilistic inference, such that on testing data, we can enjoy the benefits of probabilistic modeling without paying further cost for inference from scratch. Moreover, the labeling information can be incorporated to train the inference model a semi-supervised fashion. Viewed another way, the periodicity detection problem can be recast as multi-class classification problem. The inference model is the classifier and is trained using the proposed sellf/semi- -ttp///siiiseeeeiieeeedee framework. In a nutshell, the Bayesian aspect of AmortizedPeriol addresses the challenge of limited robustness, while the deep learning part tackles the issue of memorylessnes, Our contribution is threefold:
We propose novel method for periodicity detection named AmortizedPeriod. To the best of our knowledge, AmortizedPeriod is the first model that can perform period learning in the presence of multiple and irregular periods as well as trends, noise, outliers, and missing data, whereas existing methods only address some of these problems (see Appendix B
We propose an inference network based on cross-channel attention and derive the ELBO as the loss to train this network in self/semi-suppervisee manner. As result, AmortizedPeriod is also fresh attempt to leverage the knowledge from the previously observed time series (and their labels) accelerate and improve the inference on new data.
We conduct extensive experiments on four datasets, and find that AmortizedPeriod outperforms the state-offin satie--ff-heeaaae art methods by 28.5 385 on average with above 55% less amount of inference time. Moreover, a relatively small fraction of labels (e.g., 15%) can significantly boost its performance.
2 RELATED WORKS
As mentioned in the introduction, existing methods for periodicity detection can be categorized into four groups The first group resorts to the (partial) autocorrelation function (ACF) in the time domain that finds the local maxima of the similarity between the shifted and unshifted version of a time
series (Elfeky al. 2004; Wang et al. 22055] 2006 Toller & Kern 2017 BBriieenachh et al. 2023). However, this group tends to overlook smaller periods while emphasizing larger ones (Vlachos et al. 2005). They also struggle with multiple periods. In this case, the least common multiple (LCM) of the hidden periods represents the actual period, which can be significantly greater than the data length and thus may not appear in the local maxima of the ACF all. Additionally, the effectiveness of these methods can be hampered by noise and outliers in the time series.
The second group of methods instead utilizes the DFT and detects periods based on the periodogram in the frequency domain (Vlachos et al. 2004. Tominaga 2010 et al., 2010 Drutsa et al. 2017] Bauer et al.) 2020). Nevertheless, the accuracy of these methods diminishes for larger periods, in contrast to the ACF (Vlachos et al. 2005). Furthermore, the periodogram aims to provide the Fourier terms that best reconstruct a time series when added together, and therefore may produce numerous
2

---

false periods when trying approximate the shape of non-sinusoidal time series. Finally, similar to the first group, these methods are sensitive to trend changes and outliers.
The third group of methods intends to borrow the strength of both time and frequency methods (la
chos al. 2005 Toller et al., 2019 Puech et al., 2020 Wen et al., 2021 2023). They typically first select a list of candidates in the frequency domain using the DFT, and then identify the exact period in the time domain using the ACF. The fundamental idea is that a valid period revealed by
the periodogram should correspond to a peak in the ACF. RobustPeriod (Wen et al. (021) further enhances the robustness by first taking preprocessing step that removes the trend, noise, and outliers using the wavelet transform to balance the time and frequency resolution, and finally detecting and validating the period in each scale respectively based on Huber-perriooogram and Huber-ACF where the Huber loss promotes robustness. Recent advancements
filter, then decomposing the time series into multiple scales via discrete
this field also address the presence of (block) missing data (Wen et al.) 2023]. While the third group outperforms the previous ones, each stage (e.g.. denoising, decomposition, detection, and validation) is processed sequentially in a pipeline, implying that biases in one stage can negatively impact the subsequent stages. Moreover, the performance of each stage relies on manually chosen hyperparameters, posing a practical challenge in their selection.
The aforementioned difficulties largely stem from the inadequacy of both the ACF and the DFT for detecting periods. As a remedy, the fourth category of approaches (Tenneti & Vaidyanathan 2015: 2016: Deng & Han 2017 Zhang et al. 2020) utilizes the Ramanujan transform, which is specifically tailored for periodicity detection. This technique involves projecting the time series onto the orthogonal Ramanujan subspaces, each representing a distinct period. Identifying periods is then posed as a linear regression problem, whose objective is to select the Ramanujan subspaces that can best explain the time series. Despite these advancements, these techniques still exhibit limited robustness and are not able to transfer knowledge from previously observed time series to new ones.
3 AMORTIZEDPERIOD
The limited robustness observed in previous studies can be attributed to the neglect of non-periodie components, such as trends, noise, and outliers. While certain methods, like RobustPeriod, ac- knowledge these components, they eliminate them individually in deterministic and sequential way, leading to error accumulation. Moreover, the performance of each stage is sensitive to the choice of the corresponding hyperparameters. To overcome these difficulties, we propose a compact Bayesian generative model that offers flexibility in modeling the interaction between all components as well as their corresponding hyperparameters in joint and probabilistic manner. On the other hand, existing approaches derive period estimates for each time series from scratch by solving optimization problems iteratively. Moreover, even if we correct the estimated periods for a time series, applying existing methods would still yield the same erroneous results. To provide a remedy, we develop an attention-based inference network that directly maps observed time series to the posterior distribution of different components and the hyperparameters, including the distribution of the periods. As result, the inference model can provide period distributions for new time series without iterative optimization. Moreover, the inference model can be trained using labeling information, as it can memorize the labels. In the sequel, before delving into the generative and inference model, we first introduce the Ramanujan subspaces, which form the foundation of AmortizedPeriod.
3.1 RAMANUJAN SUBSPACES AND PERIODICITY IDENTIFICATION
The definition of Ramanujan subspaces is cumbersome to describe and not illuminating, so we present instead the definition of y-periodic time series as a starting point. From there, we proceed to discuss the properties of Ramanujan subspaces relevant to periodicity detection and define exactly q-periodic time series. We refer the readers to Appendix for the formal definition of Ramanujan subspaces.
Definition 1. (q-periodic) A time series s() with time stamp of length L. (1 S SLS S T.) is q-periodic if q is the smallest integer such that s(t + q) s(t) for all t, and q is called the period of st)).
This definition eliminates the ambiguity that a time series of period is also of a period that is a multiple of g, which plagues the time-domain methods for periodicity detection. However, another problem comes along with this definition, that is, a time series with a period of 6, for instance, may be further decomposed into two periodic components with periods 2 and 3 respectively, making it necessary to detect these fundamental exact periods in practical scenarios. To attack this problem. Vaidyanathan (2014a] b) proposed the Ramanujan subspaces, which construct a distinet linear subspace
3

---

Sg for each period q. The time series spanned by subspace S, can only possess period of q and cannot be decomposed into smaller periods. Moreover, two arbitrary subspaces Sq. and Sy are orthogonal to each other if qı # 92. As result, can define the exact period as:
Definition 2. (exactly q-periodic (Muresan & Parks 2003)) A time series s()) is of exact period q if s(t) is in the Ramanujan subspace St and the projection of s(t) onto Sq is zero for all q < q.
Let R, be the linear basis for Sg and D [R].... R.... the resulting Ramanujan dictionary with a maximum period of Pmax. We can ascertain the periods of a time series s(t) by decomposing it as
Da, where is the coefficient vector. Ideally, a should have non-zero entries only at locations where the periodic components of s reside. Estimating a given s and D can be formulated as a sparse vector recovery problem, such basis pursuit or lasso (Tenneti & Vaidyanathan (016).
3.2 GENERATIVE MODEL
Unfortunately, the above formulation susceptible the presence of trends, noise, outliers, and incomplete observations that often coexist with periodic components in time series, as well as the hyperparameter promoting sparsity in basis pursuit or lasso. Conversely, Bayesian models or networks offer a compact, flexible, and interpretable integration of various components and their associated hyperparameters in time series, while also accounting for the accompanying uncertainty. However, their application in the context of periodicity identification remains unexplored. To bridge the gap, we propose a reformulation of the periodicity detection problem from the Bayesian perspective.
Suppose that a time series x = x(1 L) after normalization can be decomposed into periods s, trend т, noise e, and outliers 8, that is, æ + T + E + 8. Next, we introduce the prior distribution for each component individually. More details on the priors can be found in Appendix D
The periodic component s: follows from Section 3.1 that = Do, where D is the Ramanujan dictionary and a represents the coefficient vector. 0g 7 O indicates that the period characterized by Dj exits in the time series x. To distinguish between the zero and non-zero entries in a. we resort to the horse-shoe prior (Carvalho et al. 2009), which can be interpreted as scale mixture of Gaussians:
p(ayloj, v) = W(az;
p(a,) = (0,
(1)
where C+ (0, is standard half- Calf-Cauchy distribution the positive reals Rt, and v and dj respectively denote the global and local shrinkage parameters. The global shrinkage parameter v shrinks all ay to zero, while the heavy-tailed half-Cauchy priors for the local shrinkage parameters o allow some 0g to escape from the shrinkage. The resulting a = [a1,' ... .4.] would therefore be sparse. One attractive property of the horse-shoe distribution is that the shrinkage weight, 1/(1 + oz), has a U-shaped density (i.e.. horse-shoe shape) Beta (0.5, 0.5), which is a Beta distribution with shape parameters a = b- 0.5, as shown in Appendix D.2 It reaches the lowest value at 0.5 but is unbounded at 0 and 1, indicating that this prior prefers of to be either very small or very large and can well separate the zero and nonzero values in a. To facilitate the amortized inference, we innovatively reparameterize the horse-shoe prior as:
=N(ayj: where A = 1/02 and Wj = =o= (1 + of) As a result, the expectation of wj provides an indicator of whether a; = 0. We then specify the hyperprior on to be a Gamma distribution Gamma (ax, bx).
p(aylwz, A) =N ay; (1 (x--w)
p(w,) Beta(wy; 0.5, 0.5)
(2)
Wj
where the shape ax and rate 6y are set small (c.g., 10- +) such that the hyperprior is non non-informative.
The trend component T: The trend is assumed to vary smoothly across time. Therefore, we employ the thin-plate model (a.k.a the second-order intrinsic Gauss-Marker random field) (Rue & Held 2005 Yu & Dauwels, 2016) to describe this behavior, since it penalizes the second-order difference (i.e.,
curvature). Concretely, model the second-order differences as Gaussian distribution: 2 (1)
(( In other words, the density function of thin-plate model is (cf. Appendix D.3):
p(T) x exp (- 3 (T(t-1) - (t) + (t (())))))) exp (-
L-1
B1 (-11-TRR
Σ
- -TTKK KITT).
(3)
2
2
where the smoothness parameter B1 controls the curvature of the trend over time, and Kip is the weighted Laplacian matrix. It can be observed from (3) that the thin-plate model is invariant to the addition of constant, and more importantly, linear function of time t. As such, this prior can accommodate the linear trends without penalty. Moreover, as the Ramanujan dictionary D has a constant subspace So that can describe the overall mean level of the time series, we specify the mean
4

---

of the trend component T to be O. The resulting prior on can be expressed as:
1 - ( I + $1 Ktp)))
p B1) x exp
2
where Bg measures how close to zero (t) is for all t. We again impose con-informative Gamma hyperprior Gamma(10-{,100 1) on Bo and 3,.
The outliers ô and the noise E: Typically, most of the elements in the outliers ò are O, suggesting that 6 is sparse. To this end, we impose the Student's 's f-distribution on (t). which is commonly used shrinkage prior in the literature of sparse Bayesian learning (Tipping 2001 Yu et al., 2020a). Equivalently, it can be written in a hierarchical manner as (cf. Appendix
p()(t) (s())) = ((((t); O, vs(e)),
p(vace) = Inv- Luv-Garmaa( (USCe); (a(E), bs(e)), (5)
where Inv- Inv-Gamma (15()); (5((): bs(o)) denotes the inverse Gamma distribution with shape parameter aō(1) and scale parameter ba(e). Again, we set ( = by(l) 10-4 to make the inverse gamma hyperprior non-informative. On the other hand, we assume that (0, ve. is the Gaussian white noise and further posit the non-informative conjugate inverse Gamma prior on ve.- Note that we use the t-prior instead of the horse-shoe prior for the period coefficient vector here. The latter is more complex but comes along with a Beta distributed variable W, that indicates whether the corresponding period exists, which is required for the sake of periodicity detection. However, since our main focus is not on whether the outlier exists (i.e., d(t) 0 or not), the simpler t-prior is more suitable choice.
Since the outliers and the noise are independent, the distribution of their sum <(t) + 5t) can be simplified as (0, & + (.())). As a consequence, the distribution the observed time series a given the latent components can be expressed as:
p(z]a,, T, vz) N Wax Da u. diag(us)),
where I denotes the identity matrix, and diag (vs) is a diagonal matrix with ta(1) on the diagonal.
The overall generative model: Taken together, the overall Bayesian network w.rt. (with regard to) all random variables can be factorized as:
p(æ, a, w, X, T, Bo, B1, Ue: v8) = p(æla,, T, ue (s)p( A)p())(())))))
p((| 31 (1)p(()))))))))))))))))) ((1))))(()))))p((p))))))))
(7)
We further consider a more general scenario where multiple time series z( for i = 1,..., .N share the same periods (as indicated by w) but possibly with different values of the coefficients (9) In other words, the zero pattern of a (+) is the same for all i, but the non-zer values in (+) can differ. The remaining three components 7 700, e(t), and g10) can also be different. The resulting Bayesian network can be factorized as:
N
(1x(1 (1:N), w, X, T (1:N), Bo, B1, vf (1.N) u
. (L:))) = II p( latt) -(0) -(0,,(t)
:(aa w, A)p(T(00 3, )p( pp( p(W)((A(())))31)
(8)
where the priors on [i} is the same for all i and guarantees that the estimated periods are the same for (I:N]. The corresponding factor graph representation (Kschischang al. 2001 of ( depicted
in Fig. [(a] Note that the above equation amounts to when N = 1. This formulation is widely applicable to periodicity detection of time series in various fields in practice (Zhang et al., 2020). such the electricity usage and the workloads of data centers, to name few. Moreover, real- wol-world time series are often associated with a given fundamental period (e.g., daily period), and other periods (e.g., weekly and monthly periods) are multiple of this fundamental period. Under this scenario, the i-th time point in consecutive days can form new time series (+), Suppose that there are N time points in a day, and we can obtain N time series x (I:N) with common periods.
3.3 INFERENCE MODEL
The ELBO: Owing to the U-shaped prior Wj. the posterior distribution of wj also follows a U-shape, as observed in YYu et al.] 2019) Chen et al. 2023). As a result, our ultimate goal is to detect the periodicity by checking how close to w, is given the observed time series x (I:N), namely, p(u; = 1|2(1=N3). To this end, we need to compute the exact posterior p( (I:N,, w, X, A,:N(, ,,, 31, of (1:N) Uu (1:N) (oc Since this posterior is intractable, we in- stead approximate it with inference model q( ,,, A, (1:N), Bo, 91, ve
(1:N} ug (2:N)). which
5

---

Par
P/o
Pp.
Pany
P.
Pd.
M
P
Pr
Rj
Pv
Praso
Pote
(a) Generative model
(b) Inference model
Figure I: Factor graph representation (Kschischang al. 2001 of AmortizedPeriod. The circle and square nodes respectively denote variables and factors the generative (8) and inference (10) model. The rectangular plate labeled 4 or L means that there are or subgraphs of this kind. We abuse the edge direction to denote the sampling process in the two models. Note that the inference model maintains the dependence structure of the generative model to better approximate the true posterior. It further includes the addition (blue dashed lines) and alteration (red solid lines) of several edges ensure that all latent variables are dependent the observation æ.
a probabilistic encoder outputting the latent variables given the observations, using the framework of amortized variational inference (Zhang et al.) 2018 Ganguly et al. (022). in analogy to the VAE (Kingma Welling 2014). The parameters of both the generative and the inference model can be jointly estimated by maximizing the evidence lower bound (ELBO) L of log ((:(1:N)):
L = Fyy log ((x: (1:N), w, X, (11N) N0. 31, of (1::N) .0! (1:N)) + Hlg,
(9)
where Fy denotes expectation over the distribution, and HH, denotes the entropy of the g distribution.
The q distributions: However, to guarantee that the inference process is tractable, the VAE (Kingma & Welling 2014) and the commonly-used mean-field amortized inference method (Zang et al. 2018) Ganguly al. 2022) typically assumes that all latent variables are independent. Consequently. the approximation accuracy is limited, especially when the latent variables in the original generative model are highly dependent such as in our model (8) with hierarchical structure, as pointed out in (Turner& & Sahani] 2011 Ranganath et al., 2016). As a remedy, we design a structured inference network (Lin et al. 2018,] Ambrogioni et al. [202]] Agrawal & Domke 2021;; [Ruillard & Wassermann 2022) that can incorporate the dependence structure of the generative model into the inference model in straightforward manner. As shown in the factor graph representation in Fig. [(b] we consider an inference model that factorizes in similar fashion to the generative model, but is conditioned on the observation x! (I:N);
N
q(a (1:N) w. X, (1::N), Bo, S1, vt ((L:N) 2g (LNN) = II ( (1) -(()) (1))) w. A, Bo. BI, ".0 (0)
+
q( (2t0)
x (1)) II g(w, 1x (2:N) (g(A|z) (2:N)
(10)
Next, we elaborate on the specification of the factors in the inference model (10) One crite- rion for choosing the distributions is to facilitate the computation of the expectation w.r. g in (9). The expectation is often computed via a Monte Carlo estimator based on the "reparam- eterization trick" (Kingma & Welling, 2014): variate U with simple distribution that is inde- pendent of all parameters 0 of g is first defined, and reparameterization function F' is derived such that F(U,0) has distribution q. On the other hand, the q distributions should be similar to (if not the same as) the corresponding p distributions for the sake of better approximation. To this end, we specify q( (()))) ((oo)x(1:N)) ((31 | (I:N3)) q(v= (z(()), and g( (1) (+))) to be log-normal distributions in order to approximate the Gamma and Inverse Gamma distri- butions, and q(w, (x (I:V) ) to be the Kumaraswamy distribution (Nalisnick & Smyth 2017) to approximate the Beta distribution. In addition, consider the dependence structure in the genera- tive model, we assume that g(ar11), 7(0 (x(t)) w. A, 3o, B1, (.()) u (1h) takes the functional form of p(a((), - [0)]z (x0 w, A, fo, Br, of (. which is Gaussian distribution as proven below.
Proposition 1. Given p(ar (0) (0) - (.) .ug (,))), p(a!") in 8) p(a - 700 [0,(1) w X, Bo, B1 ! .ug ofit) = -N-KK1
(2) |wا A), and p(7( (8,, B1) as defined 11, K-1) with precision matrix (inverse
(8

---

Tenser
Uneer
Activation Furction
Muttihead Attinttion Block
Muthead Attention
Belbt
Betlec
Bafle/2(+6)
Betbef
Red
CAB
SAS
ReLu
af g(F.)
Avdut
BxANC
BvdaC
dvdix2
SVS
mea
(aa)
The inference network of AmortizedPeriod whose outputs the parameters of the q distributions. Here the time series can exchange information via the SAB as they share common periods, but the basis cannot as they encode distinct periods. To detect periods, only operations inside the red dashed box are required.
Figure 2:
covariance) K and potential vector h given by:
Adaag
(1-w))
DJ diag 56)
(1))
D diag (v!
+ 0g (1))
K (1)
w/
diag (vf ( + u38 )
Bot - K diag (vf0) -
D
h
diag (o (1 + - 20)
Proof. See Appendix E
The inference network:
The parameters distributions w.r.t. w, A, Ue, and vs are expressed as functions of the observations x (1:N) through a deep neural network, which is based on the multihead cross attention block (CAB) /Vaswani et al. 2017). To begin with, let us define the CAB:
(11) where LN denotes layer normalization, FFN denotes the row-wise feedforward network, and MHA Q, K, V) is the multihead attention layer as defined (Vaswani et al.) 2017) where Q K, and V serve queries, keys, and values. CAB can be interpreted selecting keys K that similar the queries Q and aggregating the corresponding values V to update the queries. Note that CAB(A, A) amounts performing self-attention on A, denoted as, SAB(A).
CAB(A, B) LN(H FFN H)),
H L-(MHHAAAA (A B B) A),
In the proposed model, the latent variables can be partitioned into two groups: those responsible for determining which bases to be used (i.e., w, and X) and those associated with other components in the time series (i.c.. So. 31. Ue, and Uz). For the former group, our objective is to update the parameters for basis selection by checking the seasonality behavior of the N time series. Hence, after passing the N time series x (I:N) that share common periods through two layers of the celf-attention block SAB(w (I:N),, we apply two layers of CAB(D, D, x (I:N)) in which the queries Q are the Ramanujan basis D and the keys K and values V are the representations of the N time series x (1:N) given by the last SAB. For the latter group, we aim to remove the periodic components from the time series and estimate the latent variables related to trend, noise, and outliers. To this end, we employ two Ca D) layers with the N time series as the queries and the basis D as the keys and values. These are each followed by self-attention block SAB((x (1:N)).
We remark that the attention operation is computed over different time series and bases, unlike the con- ventional approach of operating over the time dimension when acquiring time series representations through Transformers (Zhang & Yan 2023). As result, such an operation permutation equivariant to the order of the times series and the Ramanujan bases, which is desirable characteristie. The overall architecture of the inference network sketched in Figure. [ Note that during testing, we only need to compute q(w, xe (I:N)) Hence, only operations within the red dashed box are required.
3.4 SEMI-SUPERVISED LEARNING
One advantage AmortizedPeriod that the labels of the periods can be leveraged to train the inference network semi-supervisedly. For a time series with period of q, it is likely that all bases within the Ramanujan subspaces associated with and its divisors dy will be selected, as the period series equals the LCM of the exact periods of all its periodic components (cf. Proposition (2. However, the specific bases chosen are uncertain and depend on the characteristies of the time series. By contrast, given the exact period and its divisors, the bases corresponding other periods can be excluded.
time
7

---

Table 1: The miero Fi score and run time (seconds) averaged over all time series resulting from all benchmark methods for four datasets. The first two datasets consist of synthetic data with different noise variance v. and proportion of outliers 8%. The standard deviation resulting from 5 trials is shown in Table in the appendix.
Methods
0.1, 8%
0.05
F/
1, 8%
0.1
Yahoo
App Flow
Train
Test
Time
Train
Test
Time
Train
Test
Time
Truin
Test
Time
AutoPeriod
0.29
0.30
1.76e-1
0.29
0.28
9,88c-2
0.75
0.73
1.50e-1
0.40
0.45
7. 7.68e-2
RobustPeriod
0.32
0.30
5.91c2
0.31
0.29
4.41c2
0.84
0.82
1.24c2
0.57
0.56
3.72c2
RPD
0.24
0.23
5.39
0.18
0.17
5.06
0.83
0.84
1.17
0.59
0.60
8.47
AmortizedPeriod
0.80
0.76
3.23e-2
0.70
0.57
3.32o-2
0.86
0.87
3,48c-2
0.89
0.87
3.43o-2
Accordingly, let Sy and Sd, denote the Ramanujan subspace with exact period q and its divisors respectively. For each exact period q. we first compute the log-likelihood log g(w; = | (1:N) for all j E Sy U Sd,,- We then maximize the maximum of the above set of the likelihoods, since least one basis D, from S. USa will be chosen. On the other hand, for j E S, USA,, we maximize the minimum of the log-likelihoode log ((w) = 0) ():3() for all j. in order to learn the inference network such that the likelihood of wy = 0 is maximized for those non-existing periods. In summary, to conduct supervisd learning, we add the following two terms to the ELBO L ( min J4S,USa
max (log g(ws = 1/z(1::0)) +
(log ((og(wo = 0/ (1:N))
(12)
JES,
3.5 IRREGULAR PERIODS AND MISSING DATA
For irregular periods (c.g., monthly period), we assume that the true period is still regular (i.e., 31 days for the monthly period) but the observations x are irregularly sampled. We then remove those rows in the Ramanujan basis corresponding to the days that do not exist in reality, and use the remaining rows in our model. On the other hand, missing data can be handled in natural way by ignoring the corresponding factors in the generative model. In the inference model, we also ignore the missing data
when computing ((x+(+) - 710 |==]) w, X, Bo. B1, (:) D. (1)), where xO denotes the observed data. This conditional distribution has closed-form expression explained in Appendix [E] Moreover, when training the inference network in a self or semi-supervised manner, we replace missing values with zero. Consequently, the inference network can regard zero as missing values.
EXPERIMENTS
We demonstrate the usefulness of AmortizedPeriod on four datasets, including two synthetic datasets (with different noise variance v. and proportions of outliers 8%) and two real datasets (Yahoo data and App Flow data). As discussed in detail in Appendix [ the identification of periods becomes progressively less challenging for synthetic data, App Flow data, and Yahoo data. The experiment configuration can be found in Appendix H and the complete results are presented in Appendix [
We first benchmark the proposed AmortizedPeriod with three state-of-the-art (SOTA) methods, including AutoPeriod (Vlachos et al. 2005) and RobustPeriod (Wen et al. [202]] in the third group, as well as the Ramanujan Periodic Dictionary based method (Tenneti & Vaidyanathan 2015) (referred to as RPD) in the fourth group. As mentioned in Section 2 AutoPeriod and RobustPeriod are time- frequency methods that borrow the strength of both time and frequency methods for periodicity identification. For RPD, we use the Lasso formulation in (Tenneti & Vaidyanathan) (015) which can automatically determine whether the coefficient associated with a key in the Ramanujan dictionary is zero with soft-thresholding. We assess all methods in terms of micro F -scoee and the average run time. The former is commonly used to evaluate the performance of multilabel classification methods, and periodicity detection can be regarded as a multilabel classification problem. The latter measures the efficiency of the methods. In practice, the number of time series can be huge, and therefore, a fast approach is desired. The results for all four methods are listed in Table 1
AmortizedPeriod demonstrates the highest miero Fj Fi-score while maintaining the shortest run time, especially for the challenging synthetic dataset. It is worth noting that both AmortizedPeriod and RobustPeriod consider the non-periodic components in the time series (e.g.. trend changes, noise, and outliers). However, AmortizedPeriod is even better than RobustPeriod, since it models all components and their associated hyperparameters jointly in a flexible probabilistic manner, whereas RobustPeriod models each component sequentially in pipeline without fully considering their relations and requires manual determination of hyperparameters. Furthermore, the lengthy sequential pipeline in RobustPeriod significantly increases run time compared to other methods. In contrast, the inference
8

---

train
0.85
train
0.6
test
test
Micro Fi-score
0.80
0.70
0.2
0.60
0.2
0.2
0.4
1.0
Proportion of missing data
Proportion of labeled data
Figure 3: The micro score resulting from AmortizedPeriod as a function of the proportion of (a) missing data and (b) labeled data for the synthetic data with v. and 8% 0.1.
Table 2: The micro Fi-score for the ablation study. The standard deviation is shown in Table 7 in the appendix.
0.1, 8% 0.05
8%
0.1
Yahoo
App Flow
Methods
Train
Test
Train
Test
Train
Test
Train
Test
AmortizedPeriod
0.80
0.76
0.70
0.57
0.86
0.87
0.89
0.87
Trend
0.58
0.55
0.58
0.48
0.85
0.86
0.88
0.86
Outliers
0.51
0.48
0.28
0.29
0.86
0.86
0.55
0.50
Month
0.71
0.70
0.53
0.49
0.85
0.84
amortization in AmortizedPeriod leads to the shortest run time, highlighting its superiority over the remaining methods that perform inference from scratch for each time series. Recall that the inference model of AmortizedPeriod is a neural network. Thus, the run time can be further shortened when the model is deployed on GPUs On the other hand, AutoPeriod and RPD are more susceptible to the non-periodic components in the time series. Although RPD performs relatively well for Yahoo and App Flow data, where the periodic components dominate the time series (see Figure 7 in Appendix G), its performance declines when applied to synthetic data. By integrating the Ramanujan subspaces into the proposed framework, we significantly boost the performance of RPD.
We further investigate the impact of missing data and labeled data on AmortizedPeriod, The results are presented in Fig. 3 As demonstrated in Fig. (3a] missing data does not significantly affect the performance of AmortizePPerodd when its proportion is less than or equal to 30%. However, when the proportion exceeds 30%, the performance deteriorates rapidly. On the other hand, leveraging labeled data to train the inference network can significantly improve the performance of AmortizedPeriod. enhancing the miero Fiore by above 15% for both training and testing data. Notably, for testing data, the accuracy improves faster when the first 15% of labels are introduced, and stabilizes thereafter, indicating that a small proportion of labeled data can significantly enhance performance. This is especially relevant in practice, where human experts can only label a small fraction of time series data, yet such data can benefit AmortizedPeriod considerably. Other methods, however, cannot take advantage of the labeling information.
Finally, we conduct an ablation study to verify the effectiveness of different modules in AmortizedPe- riod. Specifically, we consider removing the trend, outliers, and the monthly basis in AmortizedPeriod.
The results are presented in Table 2 Our findings indicate that it is crucial to consider the trend, outliers, and irregular periods during periodicity detection when such components exist in the data. As illustrated in Appendix G Yahoo data are almost free of trends, outliers, and monthly periods, App Flow data is mainly corrupted by outliers, while the synthetic data consists of all these components. The results in Table 2 are consistent with these observations.
5 CONCLUSION
In this work, we have introduced AmortizedPeriod, a new model for periodicity detection that borrows the strength of both Bayesian statistics and deep learning. We have shown that AmortizedPeriod is more robust to trend changes, noise, outliers, missing data, and irregular periods than existing SOTA methods. The micro Fi-score resulting from AmortziedPeriod can surpass those of SOTA methods by 28% on average. Moreover, the inference amortization technique excludes the need to conduct inference from scratch for every time series. Thus, the resulting run time is at least 55% shorter than other methods. In addition, a small proportion of labeled time series can significantly boost the performance of AmortizedPeriod.. We believe that this is the first self suupervised/semi-uppeeeee model for periodicity identification that successfully solves the burning issues of limited robustness and memorylessness to the SOTA methods.
9

---

REFERENCES
Saced Aghabozorgi. Ali Seyed Shirkhorshidi, and Teh Ying Wah. Time-series clustering-a decade review. Information systems, 53:16-38, 2015.
Abhinav Agrawal and Justin Domke. Amortized variational inference for simple hierarchical models. Advances in Neural Information Processing Systems, 34:21388-21399 2021.
Kevser Ovaz Akpinar and Ibrahim Ozcelik. Methodology to determine the device-level periodicity for anomaly detection in ethercat-basel industrial control network. IEEE Transactions on Network and Service Management, 18(2):2308222 -319,, 2020.
Luca Ambrogioni, Gianluigi Silvestri, and Marcel van Gerven. Automatic variational inference with cascading flows. In International Conference on Machine Learning, pp. 254-263. PMLR, 2021.
Kasun Bandara, Rob J Hyndman, and Christoph Bergmeir. Mstl: A seasonal trend decomposition algorithm for time series with multiple seasonal patterns, arXiv preprint arXiv:2107. 1....13.4 13462, 2021.
André Bauer, Marwin Züffe, Nikolas Herbst, Samuel Kounev, and Valentin Curtef. Telescope: An automatic feature extraction and transformation approach for time series forecasting a level- playing field. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp. 1902-1905. IEEE, 2020.
George EP Box and Gwilym M Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Society. Series C (Applied Statistics), 17(2):11-109, 1968.
Tim Breitenbach, Bartosz Wilkusz, Lauritz Rasbach, and Patrick Jahnke. On a method for detecting periods and repeating patterns in time series data with autocorrelation and function approximation. Pattern Recognition, pp. 109355, 2023.
Carlos M Carvalho, Nicholas G Polson, and James G Scott. Handling sparsity via the horseshoe. In Artificial intelligence and statistics, pp. 73-80. PMLR, 2009.
Chaoyu Chen, Hang Yu, Zhichao Lei, Jianguo Li, Shaokang Ren, Tingkai Zhang, Silin Hu, Jianchao Wang, and Wenhui Shi. Balance: Bayesian linear attribution for root cause localization. In Proceedings of the 2023 ACM SIGMOD International Conference on Management of Data, 2023.
Shi-Wen Deng and Ji-Qing Han. Ramanujan subspace pursuit for signal periodic decomposition. Mechanical Systems and Signal Processing, 90:79-96, 2017.
Alexey Drutsa, Gleb Gusev, and Pavel Serdyukov. Periodicity in user engagement with search engine and its application to online controlled experiments. ACM Transactions on the Web (TWEB), 11(2):1-35, 2017.
Mohamed Elfeky, Walid G Aref, and Ahmed Elmagarmid. Using convolution to mine obscure periodic patterns in one pass. In Advances in Database Technology-EDBT 2004: 9th International Conference on Extending Database Technology, Heraklion, Crete, Greece, March 14-18, 2004 9, pp. 605-620 Springer, 2004.
Ankush Ganguly, Sanjana Jain, and Ukrit Watchareeruetai. Amortized variational inference: Towards the mathematical foundation and review. arXiv preprint arXiv:2209. 10888, 2022.
Riccardo Guidotti, Lorenzo Gabrielli, Anna Monreale, Dino Pedreschi, and Fosca Giannotti. Discov- cring temporal regularities in retail customers' shopping behavior. EPJ Data Science, 7(1):1-26. 2018.
Jin Jing, Wendong Ge, Aaron F Struck, Marta Bento Fernandes, Shenda Hong, Sungtae An, Safoora Fatima, Aline Herlopian, Ioannis Karakis, Jonathan J Halford, et al. Interrater reliability of expert elettroenccphalographess identifying seizures and rhythmic and periodic patterns in electroen- cephalograms. Neurology, 2022.
Diederik Kingma and Max Welling. Stochastic gradient vb and the variational auto-encoder. In International Conference Learning Representations, volume 19, pp. 121, 2014.
10

---

Linghe Kong, Tian Huang, Yongxin Zhu, and Shenghua Yu. Big Data in Astronomy: Scientific Data Processing for Advanced Radio Telescopes. Elsevier, 2020.
Frank R Kschischang, Brendan Frey, and H-A Loeliger. Factor graphs and the sum-product algorithm. IEEE Transactions on information theory, 47(2):498 -198885511222000 2001.
Zhenhui Li, Bolin Ding, Jiawei Han, Roland Kays, and Peter Nye. Mining periodic behaviors for moving objects. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1099-1108, 2010.
Wu Lin, Nicolas Hubacher, and Mohammad Emtiyaz Khan. Variational message passing with structured inference networks. In International Conference on Learning Representations, 2018.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dust- dar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference learning representations, 2022.
D Darian Muresan and Thomas W Parks. Orthogonal, exactly periodic subspace decomposition. IEEE Transactions on Signal Processing, (0l0))2227-222999 2003.
Eric Nalisnick and Padhraic Smyth. Stick-breaking variational autoencoders. In International Conference on Learning Representations, 2017.
Marc Nerlove, David M Grether, and Jose I. Carvalho. Analysis of economic time series: a synthesis. Academic Press, 2014.
Zelin Ni, Hang Yu, Shizhan Liu, Jianguo Li, and Weiyao Lin. Basisformer: Attention-basee time series forecasting with learnable and interpretable basis. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In International Conference on Learning Representations, 2019.
Carlotta Orsenigo and Carlo Vercellis. Combining discrete svm and fixed cardinality warping distances for multivariate time series classification. Pattern Recognition, 43 43(11):3787-3794 2010.
Tom Puech, Matthieu Boussard, Anthony D' Amato, and Gaötan Millerand. A fully automated periodicity detection in time series. In Advanced Analytics and Learning on Temporal Data: 4th ECMI, PKDD Workshop, AALTD 2019, Würzzurg, Germany, September 20, 2019, Revised Selected Papers 4, pp. 43-54. Springer, 2020.
Srinivasa Ramanujan. On certain trigonometrical sums and their applications in the theory of numbers. Trans. Cambridge Philos. Soc, 22(13):259-276, 1918.
Rajesh Ranganath, Dustin Tran, and David Blei. Hierarchical variational models. In International conference on machine learning, pp. 324-333. PMLR, 2016.
Louis Rouillard and Demian Wassermann. Adavi: Automatic dual amortized variational inference applied to pyramidal bayesian models. International Conference on Learning Representations, 2022.
Havard Rue and Leonhard Held. Gaussian Markov random fields: theory and applications. Chapman and Hall/CRC, 2005.
Krzysztof Rzadca, Pawel Findeisen, Jacek Swiderski, Przemyslaw Zych, Przemyslaw Broniek, Jarek Kusmierek, Pawel Nowak, Beata Strack, Piotr Witusowski, Steven Hand, et al. Autopilot: workload autoscaling at google. In Proceedings of the Fifteenth European Conference on Computer Systems, pp. 1-16, 2020.
Yusufu Shehu and Robert Harper. Efficient periodicity analysis for real-time anomaly detection. In NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium, pp. 1-6. IEEE, 2023.
11

---

Sean J Taylor and Benjamin Letham. Forecasting scale. The American Statistician, 72(1):37-45. 2018.
Srikanth Tenneti and PP Vaidyanathan. Nested periodic matrices and dictionaries: New signal
representations for period estimation. IEEE Transactions Signal Processing, 63(14):3736-3750 2015.
Srikanth V Tenneti and PP Vaidyanathan. unified theory union of subspaces representations for period estimation. IEEE Transactions Signal Processing. 64(20):5217-523) 2016.
Anurak Thungtong, Matthew F Knoch, Frank Jacono, Thomas E Dick, and Kenneth A Loparo. Periodicity: characteristic of heart rate variability modified by the type of mechanical ventilation after acute lung injury. Frontiers in Physiology, 9:772, 2018.
Michael E Tipping. Sparse bayesian learning and the relevance vector machine. Journal of machine learning research, (Jun):211-2444) 2001.
Maximilian Toller and Roman Kern. Robust parameter- free season length detection in time series. In MILETS 2017@ International Conference Knowledge Discovery and Data Mining, 2017.
Maximilian Toller, Tiago Santos, and Roman Kern. Sazed: parameter-free domain season length estimation in time series data. Data Mining and Knowledge Discovery, 3375)11755-1798, 2019.
Daisuke Tominaga. Periodicity detection method for small-sample time series datasets, Bioinformatics and biology insights, -.BBI, 2010.
RE Turner and M Sahani. Two problems with variational expectation maximisation for time-series models. Bayesian Time series, 3615:109- 1300 2011.
PP Vaidyanathan. Ramanujan sums in the context of signal processing part i: Fundamentals. IEEE transactions on signal processing, 62(16)::145-4455 2014a.
PP Vaidyanathan. Ramanujan sums the context of signal processing part ii: Fir representations and applications. IEEE Transactions Signal Processing, (2(16)4455-41727 2014b.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Michail Vlachos, Christopher Meek, Zografoula Vagena, and Dimitrios Gunopulos. Identifying similarities, periodicities and bursts for online search queries. In Proceedings of the 2004 ACM SIGMOD international conference Management of data, pp- 131-142, 2004.
Michail Vlachos, Philip Yu, and Vittorio Castelli. On periodicity detection and structural periodic similarity. In Proceedings of the 2005 SIAM international conference on data mining, pp. 449-460. SIAM, 2005.
Jiandong Wang. Tongwen Chen, and Biao Huang. Cyclo estimation for setcrete-timme cyclo- stationary signals. IEEE transactions on signal processing,
2005.
Xiaozhe Wang, Kate Smith, and Rob Hyndman. Characteristic-basse clustering for time series data. Data mining and knowledge Discovery, 13:335-364 2006.
Qingsong Wen, Jingkun Gao, Xiaomin Song, Liang Sun, Huan Xu, and Shenghuo Zhu. Robuststl: A robust seasonal-tres trend decomposition algorithm for long time series. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pp- 5409-5416, 2019.
Qingsong Wen, Zhe Zhang, Yan Li, and Liang Sun. Fast robuststl: Efficient and robust ceasonal-trend decomposition for time series with complex patterns. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 2203-22133 2020.
Qingsong Wen, Kai He, Liang Sun, Yingying Zhang, Min Ke, and Huan Xu. Robustperiod: Robust line-frequency mining for multiple periodicity detection. In Proceedings of the 2021 ACM SIGMOD International Conference Management of Data, pp.
2021.
12

---

Qingsong Wen, Linxiao Yang. and Liang Sun. Robust dominant periodicity detection for time series with missing data. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023.
Hang Yu and Justin Dauwels. Modeling spatio-temporal extreme events using graphical models.
IEEE Transactions Signal Processing, 64(5):1101-11) 2016.
Hang Yu, Luyin Xin, and Justin Dauwels. Variational wishart approximation for graphical model selection: Monoscale and multiscale models. IEEE Transactions on Signal Processing, 67(24): 6468 6482, 2019.
Hang Yu. Sylvain Barbot, Justin Dauwels, Teng Wang, Priyamvada Nanjundiah, and Qiang Qiu. Outlier-insensitie bayesian inference for linear inverse problems (outibi) with applications to space geodetic data. Geophysical Journal International, (1133)333 350, 2020a.
Hang Yu, Songwei Wu, Luyin Xin, and Justin Dauwels. Fast bayesian inference of sparse networks with automatic sparsity determination. The Journal Machine Learning Research, 21(1):4883- 4936, 2020b.
Cheng Zhang, Judith Bütepage, Hedvig Kjellström, and Stephan Mandt. Advances in variational inference. IEFE transactions on pattern analysis and machine intelligence, 41(8):2008-2026 2018.
Lin Zhang, Alexander Gorovits, Wenyu Zhang, and Petko Bogdanov. Learning periods from incomplete multivariate time series. 2020 IEFE International Conference on Data Mining (ICIM), pp. 1394-1399. IEEE, 2020.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representations, 2023.
Nengwen Zhao, Jing Zhu, Yao Wang, Minghua Ma, Wenchi Zhang, Dapeng Liu, Ming Zhang, and Dan Pei. Automatic and generic periodicity adaptation for kpi anomaly detection. IEEE Transactions on Network and Service Management, 16(3):1170-11 188,, 2019.
Juntang Zhuang, Tommy Tang, Yifan Ding, Sekhar C Tatikonda, Nicha Dvornek, Xenophon Pa- pademetris, and James Duncan. Adabelief optimizer: Adapting stepsizes by the belief in observed gradients. Advances in neural information processing systems, 05518795-188000 2020.
13

---

A SYMBOLS AND NOTATIONS
Table 3: Notations and their meanings.
Notation
Size Cunstan
Meaningg
posct porne for sinessaicc seriee variational q-distibutions time series with time viamp! periediceersional compenenes with stamp Ramampan sabigace with perind g Linear baik coreepoouingg the subspacs Sq Rumanjaa distionary with musimum period Paus ieefficients wirl each hosis the Remamjion disiooary trend componene.
se a0
S
& ot0)
Lx Yourr olo)
rorr())
coro
noise compooent
outleer compasest
global shrinkage paramaterr of the honasshoee price
shrinlge parmeter
-hoe prior shape parameners of the beta pñor for 4y
periodic confficient oy
global shrinkage perameter
shoe pricr
181
inverse local shrakage panameter in the hosse shoe prior for each periodic coetficiens
stupe parantter the Gamma prioe A
panumcter of the Gamuaa prier for
second ceder differences, that AFr(() = (r(t + 1) 33--7t -1f1) - (r(1) rit 1)) that controls the curvature
smoothaess porameter the thin parameser of the this pluc prior that contol how close to zcro the trend is weightal Laplasiun matris the thin-phie priut ideurity motris
trend over time
I.X.
shape porameter Gamna proor for rate perameter of the Comma price for Ao shape porameter
TXT
Gamna proor for
bo.
181
rate puzameter the Gamma prir for 3;
varisnee of scm uezn Cuussian pri for the outice comperactt d(2) timeslanp
shap gorameter the imenz Gamma prior for fest scale perameter the imenee Gamma proor for toun
vuriance the meun Gaussan prior the noise componeus the 2-th time scrics, whane the superscrips denotes the indes
all cimestamps
2x11
(o olg) (200fen sig) (: + x
prscision the Cauveian voriatioaa distiiutionn glreli), =11 (rt0)) A, R. (
the poteniial vector the Guussian vaiiaional disriutioo (f yd+) g/ w. A. (e.e))
the mean of the log normal variational detribution for 120 als
1x1
which is finction of ri-) determined by the infermnce netwoek varimee logaoomal variational distitttion for (1 sinho lnf0))
which is fenction of riill determined by the infermnce netwook the mema the legonemall varistinall disritation o! ofv! (el)))
which is function of determined by the inferrnce nemwork the veinnce the lngenemal varosionaa distrbution for n2 gir. (111)) which is fanation of gii dats mincd by Kumarawany distibation uy afwe N)). which funcrion gfll detemmined by the inforenoe network the mean the lepacemal varistional dicribation for of Ale LLN)) which is function of a( (10) detemmined by the infevence artwork the veriance the log-nomaal vanatioaal distnbation for of for IoN)
1x1
the shape par mmmiicess
1x1
1x1
which function alllon decemiina
1x1
ils of lee" which funeton of glll detemmined by the inference network the veriane of the lng-neral varintionall distihation for is yl ar (illy
mean of the loguncuall vaiitiiual
1x1
which is function of 2!1-) detenminal by the inferense work the of the log normal vatiatioaal diaribation for fi, 9l am (.)) inforenc
1x1
g!ll) dosmmind the varimee the logemenal variniioed distibation for & vl., Wi))
which
1x1
which function of g!t-M) determined by the inference network
FIS Daclolo)
expectation of function over distriuatoon KL. divergence berween dieriiotions o and y the FLRO
II,
the entropy of distibution
A.
the unentio block (CAB)
Q.K. V
query, key, und value tetsues in the ispat of the mnbthead amtcstion (MHA)
multiccdd stumiio
FFN
feedforward netwoek
layer nomm
CAB
cross uneetion block welf stteation beak
B MORE DETAILS ON RELATED WORKS
Table presents comprehensive comparison of the proposed AmortizedPeriod with existing methods discussed in Section 2 based on the six essential criteria: 1) capability to handle multiple periods, 2)
ability to accommodate irregular periods, 3) consideration of trend, 4) robustness to noise, 5) handling
14

---

of outliers, and 6) handling of missing data. Analysis of the table reveals that the existing methods address only a subset of these requirements, whereas AmortizedPeriod demonstrates robustness by effectively addressing all of them.
Table 4: Comparision between AmortizedPeriod and the existing works the four groups for periodicity identification in terms of robustness.
Single Period
Multiple Periods
Irregular Periods
Missing
Group
Method
Trends
Noise
Outliers
Data
Elfcky et al
2004
Yes
Yes
No
No
Yes
No
No
1. ACF
Wang
2006
Yes
Yes
Yes
No
Toller & Kern
2017
Yes
No
No
Yes
Yes
No
Yes
Breitenbach el al.
2023
Yes
No
No
Yes
Yes
No
No
Tominaga
2010
Yes
Yes
No
2. DFT
Drutsa
2017
Yes
Yes
No
No
Yes
No
No
Bauer ot
2020
Yes
Yes
Yes
Yes
No
No
AutoPeriod
(Vlachos
Yes
3. DFT& ACF
Sazed (Toller et al.
2019
Yes
No
No
Yes
Yes
No
No
Puech et al.
2020
Yes
Yes
No
Yes
Yes
No
No
RobustPeriod Wen
2021
Yes
Yes
No
Wen et al. 2023
Yes
Yes
No
Yes
Yes
Yes
Yes
4. RPD
Tenneti & Vaidyanathun 2015
2016
Yes
Yes
No
No
Yes
No
No
Zhang ct 2020
Yos
Yes
Yes
Yes
AmortizedPeriod
Yes
Yes
Yes
Yes
Yes
Yes
Yes
C FORMAL DEFINITION OF THE RAMANUJAN SUBSPACES
The Ramanujan Subspaces can be defined from two distinct perspectives, namely the Fourier basis and the Ramanujan sum.
From the Fourier viewpoint, the q-th Ramanujan subspace S, encompasses all exactly q-periodic time series (t) that cannot be further decomposed into smaller periods. It can be represented as:
sy(t)
ag Wkt.
(13)
(k.2)=1
where W, = exp(j2n//g is the Fourier basis, Oλ are the coefficient for the frequency 2rrk/g and (k, 2) means that the greatest common divisor (GCD) of k and g is 1. Note that S. only contains non-zero coefficients at the "coprime frequencies" 2Tk/ where 1 < k S q and &: is coprime to and therefore, the exact period of st) q and smaller. follows that the rank of this subspace Sy is Φ(g), which is the Euler totient function signifying the number of integers l in < k S satisfying (k, 4) (Vaidyanathan 2a)
On the other hand, the renowned Indian mathematician Srinivasa Ramanujan introduced the concept of the Ramanujan sum 1918 (Ramanujan 1918). which is given by
(t)
Wkt
(14)
9
(k.g)=1
Ramanujan employed this sum to demonstrate that various standard arithmetic functions number theory can be expressed as linear combinations of (t). For any fixed integer q, (t)() is a sequence with periodicity q that cannot be further decomposed. Interestingly, different from the complex-vaie valuee Fourier basis, the Ramanujan sum is always integer-valued, which is appealing characteristic. Moreover, it can be easily obtained via efficient recursive computations, as demonstrated previous works (Vaidyanathan 2014a).
15

---

By leveraging the definition of the Ramanujan sum, the g-th Ramanujan subspace can also be spanned
by all the ((q) circularly shifted versions of rt(() (Vidyanathan 2014b):
((1)
st(() Σ k=0
be - k).
(15)
where by denotes the real number coefficients. Since the rank of Sg is ((g) as previously mentioned, any set of k(g) linearly independent vectors from S, can span the entire subspace. The Φ09) circularly shifted versions of ri (t) serve as a set of such linearly independent vectors.
Consider the sum x=()= = E. Fg (1), where Tyy (1) is an arbitrary a d-period time series (cf. Def- inition D the period of xxt) is equal to the least common multiple (LCM) of qt, i.c. LCM(qq). or a divisor of LCM())). as proven in VVaidyanathan 2014a; Tenneti & Vaidyanathan 2015). For instance, given two 2-periodic time series (1, 1,
). the period of their sum is 1, which is a divisor of [2 2) = 2. However, if the exact period of Feg, (t) is qe, that is, Fz.
and (-1,1,-,,
S,, the period of (tt) can only be LCM(q:) and not smaller. This is another attractive property of the Ramanujan subspace. Conversely, any arbitrary q-periodic time series, as defined in Definition T can be spanned by the set of Ramanujan subspaces Sg., where q represents all divisors of g, including both 1 and 9. For example, a 6- 6-periodic time series can be obtained by adding periodic components from S2 and Sz. S₁ and Sg, or S₁, S2, S3, and Sg. as long as the LCM of the exact periods corresponding to these subspaces equal 6. In other words, we can summarize this concept with the following proposition:
(t) (t)
Proposition 2. According to the definition of periods in Definition and exact periods in Definition 2 the period of time series is equal to the least common multiple (LCM) of its exact periods.
In order to determine the fundamental exact periods of a time series æ+)). we can decompose at1 as
x
Dox
(16)
where = RR R2. P.... denotes the Ramanujan dictionary, and R. are the matrices of size L x σ(q) whose columns are the (04) shifted versions of (1) (14). Ideally, the coefficient vector a has non-zero zero entries only at locations where the periodic components of æ reside. The number of columns in D is ((P = m'max p(q). that is, O(P Hence, D can be fat matrix with more columns than rows. The resulting problem of estimating a given x and D can be formulated sparse vector recovery problem such basis pursuit or lasso (Tenneti & Vaidyanathan 2016
D A BRIEF INTRODUCTION TO THE PRIORS FOR AMORTIZEDPERIOD
D.1 STUDENT'S S I-PRIORS
Recall that we use student's t-priors to describe the behavior of outliers. To remain meaningful, the outliers must be relatively rare. This means that the vector should be sparse, with most of its entries being zero. Assuming that the components of 5 are independent and identically distributed random variables, each entry (t) of the outlier vector may follow a normal distribution with zero mean and variance V5()), as represented by the equation:
p()(t) (a(e)) (o(t)): e3())),
(17)
The outlier precision Ux(e) is also random variable, and since we have no prior knowledge of its value, we adopt a non- non-informative conjugate prior:
p(es())) = Inv- aav-Gamma(Pstt)) Qs(t). bx(e))
(18)
Here, Inv-Gamma(Psro) (s())) b5())) refers to the inverse Gamma distribution with shape parameter
as(1) and scale parameter ba(1). By setting both as(e) and bō(e) small values, such 10-4 the prior becomes non-informative.
As a result, the marginal distribution for each outlier can be obtained by integrating out va())- yielding (Yu et al. (020a)::
p(5(t)) = / p(5(t) vsto) ((((()) dusce)
(19)
16

---

student-t
Laplace
horse-shoe
0.7
4.0
3.5
0.6
3.0
0.5
densiyy
denelyy
2.5
0.4
2.0
1.5
0.2
1.0
0.1
0.5
0.0
0.0
0.6 (b) Shrinkage weights density
1.0
(a) Sparse promoting priors
Figure 4: The density of the commonly used sparse promoting priors, including the Student's T, Laplace, and horse-shoe prior (a), and the density of the corresponding shrinkage weights 1/(1 + σ²) (b).
aa(1)
T(ag()) + 2 r(as())) (
(20)
o,,²
20a(e)
This equation represents Student's t-distribution. On hand, the Student's 's t-distribution tends to shrink most entries towards zero due to its sharp peak at zero. On the other hand, the long tail of the distribution allows for some elements 5(t) to be far from zero. Consequently, the resulting vector will be sparse.
D.2 HORSE- SHOE PRIORS
The horse-shoe prior also a Bayesian sparsity-prane promoting prior that assumes that each coefficient a; be expressed scale mixture of Gaussians (Carvalho et al. 2009 Yu al. 2020b):
playlo,, =N(a;;0,
(21)
p(a,) = C0,)),
(22)
where the standard deviation oj follows half-Cauchy distribution C+ (0, 1). Here, v is referred to as the global shrinkage parameter, and o, as the local shrinkage parameter.
Figure ((a) displays the densities for commonly-used sparsity-promotig priors, including the horse- shoe, Laplacian, and Student's priors. The horse-sher -hose prior is particularly useful as a shrinkage prior for sparse Bayesian learning due to its unique characteristics. Its flat, Cauchy- like tails allow the coefficients the strong periods remain large, while its infinitely tall spike at the origin provides severe shrinkage for the zero elements in a. This property is not present in the Laplacian and / prior.
Viewed another way, we can represent the shrinkage weight W as Wj 1/(1 + +o)), and depict its density for the horse-shoe, Laplacian, and prior Figure 4(b) Choosing of ~ C+ (0, 1) implies wj Beta(0.5 0.5), which is a symmetric density unbounded at both 0 and 1. In other words, the horse-shoe prior yields a shrinkage profile that exhibits two expected outcomes: large nonzero values (wy z 0, implying little shrinkage), and zeros (wy indicating total shrinkage). In contrast, the Laplacian prior leads to shrinkage profile that approaches a fixed constant as wy approaches 1, while vanishing entirely as wi approaches O. The prior exhibits an unbounded density near wi = O, while being bounded near wi = 1.
In Section 3.2
we further reparameterize the horse-shoe prior using Wy and À = 1/v² to utilize Wy as an indicator of which periods exist.
17

---

D.3 THIN-PLATE PRIORS
The thin-plate model (Rue & Held 2005 Yu & Dauwels 2016), commonly used as smoothness prior, a Gaussian Markov random field (GMRH) that penalizes second-order differences. Specifically, the second-order differences are modeled as a Gaussian distribution:
(1) ~ (0, (T
(23)
For one-dimensional problem with variables (t) evenly distributed along chain, the second-order difference at (t) can be defined as (t) = T(t (f---) 2T(t) + 7( + 1). Consequently, the density function of thin-plate model with a chain structure can be expressed as:
81
(T( 2T + 1))²) x exp 1app--5 (- (-
p(T) x exp
(24)
where the smoothness parameter 31 controls the curvature, and Kip takes the form:
K tp A A,
(25)
-2
A =
(26)
-2
Note that is a (L 2) x L matrix. Apparently, the precision matrix (i.e., inverse covariance matrix) of a thin-plate model is given by ag Km. From equation (24), it is evident that the thin-plate model remains unchanged by the addition of constant a linear function along the Markov chain. In other words, ktp1
and &1 = O, where is a column vector ones, and 81 2 Pl". As result, this prior accommodate linear trends without any penalty. Additionally, it can be inferred that Kip has two zero eigenvalues, making it rank deficient. Consequently, an improper density is often employed in practice (Rue & Held 2005). given by:
1 -
p(T) |K exp
(27)
where [K]+ denotes the product of the positive eigenvalues of the precision matrix K. Eq. (25)
eveals that the conditional mean of one variable T(t), conditioned on the remaining variables 1, ... L.), can written as (Rue Held 2005):
T(1, .t 1 1,
(LT()))(L,,)-- ,t 1, L)) (T + 1) + + 1)) 1 (T(++ + + T(t -tt-22) 2))..
(28)
This can be seen as second-order polynomial interpolation based on four nearby variables +T(t 2). T(t - 1), 7(t + 1), and 7( + 2) without an overall level. Therefore, the thin-plate model allows for deviations from an overall mean level without requiring the specification of the overall mean level itself. This property is often desirable in practice.
E PROOF OF PROPOSITION
As mentioned in Section 3.3 we set ((a60,,(1)) (00)))00)) 80. p(a(1, (1) w, X, 3o 31, ws
31, v1 (.
o ()) in order to capture the dependencies between fi and fil given the remaining latent variables. Recall that
(0) (1)
(0)
(i) (f (1) x exp
x - Da 7) (v. diag(vs)-11)) Da Da-r))) -7)).
1 Wj 0 (0)))
p(a." (wij, A) x exp
Wj
p(T(1)]88) B1) exp
- T T(30I + 11(K)
18

---

All these distributions are Gaussian. According to the properties of Gaussian distributions (Lin et al. 2019 8
we have
p(a11, w. Bo, 31, ol)) u )
a p( 00 (0 ,(0 v
10 31) p(a ( -وية 2) p(T 1o)
j=1
aa
(0)
-(1) 7
K(0
a00
-(177 1
exp
2
-(1
[a
=N
7(0 7(0)
K [i)
h0 K(0-1
(29)
where
A Aiiag
1 1-ws) + DJ diag (v!") + + (000))110 D
DJT diag (ve") + +08 b
K(O1 (1)
Wj diag (v! + vz8" )
-1 D
BoT + B1 Kip + diag (v) (5)
(1)
8
(30)
DT
h (4)
diag (o )
-1 10)
(31)
In addition, when there exist missing data, the time points / in the time series x: (3) (1 L) can be partitioned into posterior distribution of (1) and 7(+ can be computed in natural way
M and O, representing the missing and observed set respectively. The
p(a T (x X, 3o Bi, of x
x p( (
(1)la 10 (1), 1 (1), 68
( 80))
II
play (0) Jaoj, A, 11. Bi).
(32)
IED
The above distribution still Gaussian, and the corresponding K (i) and h (() can be computed by replacing (+ (t) for E MI with in (30) and (3I.
F DERIVATION OF THE ELBO
Recall that the generative model (i.e., the p distribution) can be factorized as:
p(a (1:N) a (1:N) w. X, 7 (1:N), Bo. $1. u! (1:N) oo ( (1:0))
N
(1) II p( (0 (0) (0 ug p(aa 11 31 (e(!! ((ug))
+ II (l.s))()))(3)))((111 1=1
(33)
where
p(x111 [1)
(i) - (1) of rg) (2 Da ( + +61, ofth
I +
diag(u!"))
(34)
p(a. (1) wejs, A) N N(a5;0, 0,, (a 1- ws) 7
(35)
Wj
p(7(10 (1) 18,, 31) = N (7(() 0. (Bod + 3(Kpp) )
(36)
p(o(()) = Inv-Gamma :a ((09)
(37)
p(v)))
(1)
Inv-Ganmn Gamma
800) (())
(38)
19

---

p(w,) = Beta (uy:awsi bus):
(39)
p(A) =
Gamma
(x:x;bx):
(40)
p(5)) =
Gamma (0: )180):
(41)
p(31) =
Gamma
[31 68g
(42)
(43)
and the inference model (i.e., the distribution) can be factorized as:
q(a (1:N), w. w,,,, (1:N1, Po, $1, of2:,) vg (1:N)
/ (1:N)
II
g ((a (r) - 1111)11111 w. x, Su, Br, wft 2g 10) (a(1! |210 2 1111)))))))))
2=1
+
II ((wy)a:(1100 ==1
[1:N] (006
(1:N) (9 (1(111111100)
(44)
where
(((()) -10 10) w, A, So, 31, / =N ((
[a(3)
:K 1100) : (x()-)
(45)
7(1
Lognormal (v/0) 100 vt (z)
(46)
((1560 (1)))
(1)) = Lognormal (1 (000)) (z: (1)) 1 510 18) (ze (1))
(1)
(47)
aww, (o0))
Kumaraswamy (u (1 (1:N3) d dww
(48)
q( (A/2011:01)
Lognormal (= 2, (L)
(49)
((3 ((3o]ax(1.N))
Lognormal mo (ax (LNN). uso (z0 (.(1))))))
(50)
q(3 (((1)))() Lognormal (B (ar (LNN)) ug (=(li))))
(51)
f Note that the parameters of q(vv [z(()))
(( (i))))), g(wو (AAlxx (( (e
and ((11 (1x))) are explicit functions of x ( (I:N), which is parameterized by the inference network.
By substituting the
33 and distributions (44) into the ELBO we can obtain:
N
L Egg
log (( 10 110) wf() ug (1) + logr(a (0) |z, No, A1) + log log (6) 180, |10,0 31)
+ logg(v!!t) + logr(o!")) + Σ
logg(ws;) + logg(A)) + + 1ogr((3)) + 1(g)(33)
j=1
log (1) 111//0011 (1000((1,, ,2 Ao, >1, 31, of (ft + log (1
+ log (0 (1) (101)
log ((w x (1:N)) - log (A12211101) - log ((6llzz xx (1:N))
log ((213311e))) 12: (1:N1)
Rgg log (+) (00 wfft ()) log (((() A) + 10g)(=(0))p B1)
20

---

- log logg((a!)) 100 w, 1, 31, ol) ,08
DEL
- DKL (1106"/=(")>)000005
DKL ((ls)laltii)))1lllc DEL (1(A)=/1)))))101AA)
-KL ( (0(oo)ool1))))1ll DKL (1(3))(()))))((())
(52)
where PKL denotes the KI. (Kullback-Leibler divergence between two distributions. We now delve into the expectations in the above expression.
follows from Appendix that the first term amounts to
Fg
logp(x
(1) (1) (0)
(tt) + log + logp(r(())))) (1)
1og (1)
(i)
A. So B1, u!! U (0))
(log { 1 (x-=))
Σ (log (v! + (
Wi
1=1
(log det (301 + k1
1 (log det K)
xl1)
(diag (v + + -1 1)x0
2
DTT diag (v! + v/) - 2o (53)
1 x ( (diag (vf0) +
D K
(i)
where K() is defined in 29 and (-) denotes the expectation over the distributions. Note that the term (e + B1 )p))
can be further simplified as:
Σ (log(Bo + Brrr)),
(log del (B. + B3 K (p)))
(54)
where ra denotes the t-th eigenvalue of Kip- We therefore use this simplification when computing the above expectation. In addition, the expectations in (53) do not have close-form expressions, and only be computed via Monte-Carlo approximations. More specifically for those variables whose g distributions are log-normal, we can use the reparameterization trick introduced in (Kingma & example,
Welling 2014 Take vf [f] Lognormal(m!." (1) (zz(1) v. (f))
o V vee (ali))e m!! (f) (1),
(55)
where e (O0 1). On the other hand, for wy Kumaraswamy(wJj (wy; ews (2o(1oN)), ) duy (x((1.))) samples of wj can be reparameterized as:
W = (1 u Sayg (x(liN))) (w)(actlNN3)
(56)
where u ~ U(0, 1).
The second and third terms DKL (q(v." (+) (z (llp(( and ( (1lp(( (1) are the KL divergence between log-normal distribution and an inverse Gamma distribution. Suppose that the mean and variance of the log-normal distribution are m and and the shape and scale of the inverse gamma distribution are and b. The resulting KL divergence (after removing the constant terms) can be expressed as:
am + bexp (-m+
1 こ²
1 log
(57)
The fourth term DKL (k(((((()))) [xN is the KL divergence between a Kumaraswamy (Nalis nick & Smyth 2017) with shape parameters and d and Beta distribution with shape parameters
21

---

Algorithm I AmortizedPeriod
Require: N time series (1:: that share common periods, each with length L, and the Ramanujan dictionary D with size 4;;
Ensure: g(w, (z) for = =1, 1,
1: repeat
Pass (1:N)) and D through the inference network to get the parameters for g(w, |ax (1:N))), g (A)(x(1))))
((B6)z=(1))) (([3]/z((:))) o(vE") [x( and q( ( (=(()));
Draw samples from the Kumaraswamy distribution g(w, lar (2.N) using the reparameterization trick
in (56):
4:
Draw samples from the log-normal distributions Q(A|z((1-N)) (( (1:))) ((51
q(v(") (()), and [(=55"1<<f (zx**) using the reparamcterizztioon trick in 55
5:
Draw i' samples from q( (1) ((4) [< w. So. 31, vf v/ given the samples in the above two
steps using the reparameterization trick for Gaussian distributions (Kingma & Welling 2014)
Compute the negative ELBO via Monte Carlo approximations (52):
7:
Update the parameters of the inference network via gradient descent;
8: until convergence;
9:
Perform inference by computing g(wy (x) via the inference network for new time series;
and b, that is, E.g log (wor (2:N)) - logg(w;)
log e log d + (d (logg wwg) - (a c)og w) (b (log(11) w,)),
(58)
c + log d + d (1 (1-
1
(v(()) (v(t)) + (b (log(1 wwy))
+>)
(59)
where y denotes the digamma function, y denotes the Euler-Mascheroni constant, and (5) holds since
(logw) = - ((()) (
(60)
(log(1 - w(]) = d'
(61)
Note that the expectation (log(I-w;)) cannot be computed analytically, and so Monte Carlo estimates are required by sampling from the Kumaraswamy distribution as in (56).
The last three terms DKL
DKL (IN)))))) and DKL (g (1:0)) [p(::) are the KL divergence between the log-normal distribution with mean m and variance v and the Gamma distribution with shape and rate b:
am + bexp (m " ")² log
(62)
2
The training and inference algorithm for AmortizedPeriod is summarized in Algorithm
G DATASET DESCRIPTION
Here, provide the details regarding the datasets used in experiments.
Synthetic Data:
We generate periodic synthetic data with noise, outliers, and changing trends. As first step, we determine the number of periodic components in a time series by randomly picking an integer from (O......) .3). where 0 means there are no periodic components and so Wj should be () for all j. For each periodic components, we further choose the shape of the base periodic signal from sinusoidal, square, and triangle waves at random, all of which are frequently seen in the real world. The corresponding periods are chosen from {1, 283. We further multiply the chosen periods with 24, corresponding to 24 hours in a day. The amplitude of the base signal is sampled uniformly from [1, 1.5). We also add an irregular monthly period to 25% of the time series. On the other hand, the trend component is also randomly selected as sinusoidal, square, triangle wave, with period as long the length of the time series L. We specify
22

---

L = 120 days x 24 hours 2880 such that the monthly period component can repeat for about four times. The corresponding amplitude is sampled uniformly from [3, 10]. Finally, we set the variance of the Gaussian white noise v. and the proportion of outliers 8% to be (0.1, 0.05) and (1) respectively to generate two datasets, in order to evaluate the periodicity detection approaches under mild and severe conditions. For each dataset, we simulate 5000 samples and split them into training and testing sets with ratio of 4: 1. In particular for AmortizedPeriod, each sample is further partitioned into N 24 time series with the same periods, as mentioned in Section 3.2 We depict a random selection of four time series for both datasets and their corresponding decomposition in Figure [n [ We observe that the periodic components are heavily corrupted by trends, noise, and outliers, especially for the dataset with v, = and 8% 0.1.
Yahoo Data We utilize the publicty available multiple-period data from Yahoo's webscope S5 datasets, specifically the Yahoo-A3 and Yahoo-A4 datasets. These datasets comprise 200 time series in total, each of which contains 1680 data points across 3 period lengths: 12, 24, and 168. The ratio between the training and testing set is 4 1. Similar to the synthetic data, we divide each sample into N = 12 time series with the same periods for AmortizedPeriod. We plot out in Figure 7 four randomly selected from this dataset. It can seen that the noise level associated with time series in this dataset the lowest comparison with the other three datasets.
App Flow Da We further collect the real traffic flow data for 311 on 18 logic data centers in the cloud system of Ant Group. These data are used for workload forecasting, anomaly detection, and auto-scaling of cloud databases and computing in practice. This dataset consists of 1000 time series, each of length 2880 (120 days x 24 hours). The candidate periods include 0 (i.e., non-periodie), 1, 7, and 15 days as well as the irregular monthly periods. It can be observed that the periods are corrupted by changes in the periodic and trend patterns, and the presence of noise, outliers, and even blocks of missing data. All time series are labeled manually by experts. The ratio between the training and testing set is 4 1, and we again split each sample into N 24 time series with the same periods for AmortizedPeriod. We further present four time series from this dataset selected at random in Figure & As demonstrated in this figure, the trend of the time series is relatively stable. However, there exist outliers every now and then. Moreover,
deployed
observe weekly (the second figure) and monthly periods (the third figure) in this dataset. is important to note that the monthly pattern observed in the traffic flow of Alipay users can be attributed to activities such as pre-authorized debit, auto-deduction of utility bills (e.g., electricity, water), and auto-pas payment of credit card bills. Although these activities occur sporadically, they may affect all Alipay users within brief time frame. As result, it is essential to consider monthly patterns during resource scheduling, since they may significantly increase the traffic flow.
H EXPERIMENT SETUP
Unless otherwise specified, in all of our experiments. we set the hidden dimension in the CAB and SAB to 192. For optimization, we use Adabelief (Zhuang et al.) 2020) with 81 0.5, Bg 0.999, and learning rate 1 x 10-4, since it offers more stable results than Adam. We conduct training for 5000 epochs and select the checkpoints with the lowest training loss as the final model. All simulations of training AmortizedPeriod are run using 8 NVIDIA TESLA A100 GPUs with 80 GB of VRAM. All inferences are run on MacBook Pro (16-inch, 2019) with 6-core Intel i7 CPU and 16 GB of RAM. All experiments are averaged over 5 trials.
2 htttss//wwbsccope..e wwtp:e/bbsoppeeddbb..eeeee wwwpoyyahhhoooooccc.oeeoee www.cataloccccccccooooooo wttp///ctallggggpplaaaaatttoeeee puayy/aaattyyp-ssuuccuueeerr
data.zip.
*a. The data set does not contain any Personal Identifiable Information (PII): b. The data set is desensitized and encrypted; c. Adequate data protection was carried out during the experiment to prevent the risk of data copy leakage, and the data set was destroyed after the experiment; d. The data set is only used for academic research, it does not represent any real business situation.
23

---

trend
monthly
periods
periods
noise and outliers
final time series
Amplitude
Amplitude
10
500
1000
1500
2000
2500
500
1000
1500
2000
2500
A0
Amplitude
Amplitude
500
1000
1500
2000
2500
500
1000
1500
2000
25500
Ampiltude
Ampltuuee
1000
2000
2500
500
1000
15500
2000
2500
10
Amplitude
Amplitude
A
1000
2000
500
1500
2500
Figure 5: Time Series from the synthetic dataset with v, 0.1 and 8% = 0.05.
trend
monthly
period;
periodz
noise and outllers
final time series
Ampitude
Amplltude
20
20
500
1000
1500
2000
2500
500
1000
1500
2000
2500
Amplitude
Amplitude
20
20
500
1000
1500
2000
2500
500
1000
1500
2000
2500
20
Amplitude
Amplitude
0
-20
20
500
1000
1500
2000
2500
500
1000
1500
2000
2500
20
Ampll Iude
20
-20
20
0
500
1000
1500
2000
2500
500
1000
1500
2000
2500
Figure 6: Time Series from the synthetic dataset with v. and 8%
0.1.
EXPERIMENTAL RESULTS
In this section, we present a more comprehensive comparison between AutoPeriod, RobustPeriod, RPT, and the proposed AmortizedPeriod over the four datasets, supplement to the results presented in Table Coneretely, apart from the micro F, -core reported in Table we also employ other performance metrics to evaluate the benchmark methods, namely the macro Fs-score, weighted
24

---

Ampituce
5000
2500-
200
400
660
800
1000
1200
1400
1800
Time
Amplkude
200
400
600
6D0
1000
1200
1400
1600
Time
Ampitude
2000
-4000
200
400
10000
1200
1400
1000
Time
4000-
Ampltuude
2000
200
800
1000
1400
Figure 7: Time Series from the Yahoo dataset.
Ampikude
2000
2500
Ampittaa
50000
1mm M MMM Vin
Ampiltude
500
1000
1500
2500
Anpitude
500
sáo
1000
1500
2000
2500
Time
Figure 8: Time Series with span of months from the App Flow dataset.
Fi-score and sampled Fi- Fcose... Prior to presenting our findings, we provide an explanation of these scores. The Fi-score is widely used metrie for assessing classification model performance, combining precision and recall into single value that represents their balance. The different variations of the Fy-score are defined follows:
25

---

Table 5: Full comparison between the benchmark methods the four datasets.
Metrics
AutoPeriod
RobustPeriod
RPD
AmortizedPeriod
train
0.29
0.32
0.24
0.80
Micro score
test
0.30
0.30
0.23
0.76
& = 0.1, 6% 0.05
train
0.18
0.19
0.18
0.65
Macro Fi-scoee
test
0.17
0.19
0.18
0.57
train
0.30
0.31
0.39
0.87
Weighted Fi-score
test
0.29
0.29
0.39
0.84
train
0.23
0.29
0.23
0.80
Sampled Fi-score
test
0.22
0.27
0.22
0.75
Time
1.76e-1
5,91e2
5.39
3.23e-2
Mcmory
2.57MB
609.48MB
144.60MB
2.79KB
train
0.29
0.31
0.18
0.79
Micro Pi-score
test
0.30
0.29
0.17
0.57
train
0.18
0.18
0.10
0.58
Macro score
test
0.18
0.17
0.10
0.45
Ui 8%
train
0.27
0.28
0.22
0.86
Weighted Fi-score
test
0.27
0.26
0.22
0.72
train
0.22
0.28
0.12
0.73
Sampled F -core
test
0,22
0.26
0.12
0.54
Time
9.88e-2
4.41e2
1.24e2
3.72e2
Memory
2.56MB
93.98MB
144.60MB
2.90KB
train
0.75
0.84
0.83
0.86
Miero -core
test
0.73
0.82
0.84
0.87
train
0.29
0.09
0.59
0.81
Macro Fi-score
test
0.23
0.12
0.38
0.82
Yahoo
train
0.87
0.98
0.81
0.81
Weighted Ficsc
test
0.92
0.98
0.84
0.82
train
0.69
0.84
0.86
Sampled Fissore
0.71
0.83
0.84
0.86
Time
1.50e-1
1.24e2
1.17
3.48e-2
Memory
1.33MB
50.20MB
35.87MB
2.31KB
train
0.40
0.57
0.59
0.88
Micro -score
test
0.45
0.56
0.60
0.85
train
0.03
0.07
0.04
0.28
Macro F Score
test
0.04
0.08
0.26
App Flow
train
0.39
0.78
0.62
0.90
Weighted Fi-ceree
test
0.46
0.82
0.65
0.88
train
0.30
0.55
0.62
0.82
Sampled Fisscoe
test
0,36
0.55
0.63
0.79
Time
7.68c-2
3.72c2
8.47
3.43e-2
Memory
2.57MB
94.17MB
144.6IMB
2.66KB
Micro Fi-score: This metric considers the overall count of true positives, false positives, and false negatives across all classes. It computes the precision and recall for the entire dataset and then derives the Fi-score based on these aggregated values. treats all classes equally and suitable for imbalanced datasets.
Macro Fi-score: This metric calculates the Fi-score for each class individually and then takes the average across all classes. Each class is given equal importance, irrespective of its size or distribution in the dataset. It is appropriate when all classes are considered equally significant.
Weighted F score: Similar to the macro Fi-sc,re, this metric accounts for the class distribution within the dataset. It calculates the Fi-score for each class, weighting them by the number of
26

---

Table 6: The miero score and run time (seconds) averaged over all time series resulting from all benchmark methods for four datasets. The first two datasets consist of synthetic data with different noise variance v. and proportion of outliers 0%. The results are averaged over 5 trials and the corresponding standard deviation is provided in the brackets below.
0.1.
8%
0.0G
1.0%
0.1
Yalooo
App Flow
Methods
Tzain
Test
Time
Tain
Test
Time
Train
Test
Time
Train
Test
Time
AmaPPrind
0.29
0.30
1.76e-1
0.29
0.28
9.88e-2
0.75
0.75
1.50e-1
0.40
0:45
7.68022
(0)
(0)
(1.676-2)
(0)
(0)
(2.11e-2)
(0)
(0)
(7.17e-3)
(U)
(4.98e-3
RohoePeriod
0.32
501c2
0.31
4.41c2
0.84
1.3462
0.57
0.56
3.73x2
(1.S4e-3)
(7.75e44)
(3.76el)
(3.88e-3)
(2.31e3)
(4.41e1)
(0)
(0)
(7.33)
(8)
(5.06e1)
RPD
0.24
0.23
5.39
0.18
0.17
5.06
0.83
0.84
1.17
0.59
0.60
8.47
(5.08e-5)
(7.99-55)
(4.59--1)
(1.80e55)
(7.42e-5)
(4:1o-1)
(0)
(0)
(9.52-7)
(4:46e-5)
(0)
(2.68o-1)
AmurtzeePPrrid
0.80
3.36-2
0.70
3.32e22
3:43-22
(1.62e-2)
(2.02e-2)
(2.10e-3)
(1.58e-2)
(9.37e:3)
(4.69e-3 3)
(2.36e
(2.54e-2)
(2.92e 3)
2.00-2)
(2.57e-2)
(4.1le-3)
instances in each class, and then takes the average. This evaluation provides more balanced assessment when the classes have varying sizes.
Comparable to the macro Fi-score, this metric calculates the Fi score for each individual sample and subsequently takes the average across all samples. It assigns equal importance to each sample, irrespective of its class or distribution. This metric is useful for evaluating classifier performance on per-sample basis, without considering class imbalances or class-specific performance.
Sampled Fi-score:
Moreover, in addition to the running time shown in Table [1 we also show the peak memory consumption during inference. All of these results are summarized in Table
In terms of estimation accuracy, AmortizedPeriod demonstrates superior performance compared to other methods for most datasets and variants of the F1-score, establishing its dominance. However, for the Yahoo dataset, RobustPeriod surpasses AmortizedPeriod in terms of the weighted Fi-score while falling short in other Fi score variants. This discrepancy can be attributed RobustPeriod generating more false positives for nonexistent periods, yet yielding fewer false negatives for existing periods. Notably, when computing the weighted Fi-score, the weights assigned non-existing periods are set to zero, thereby disregarding the false positives produced by RobustPeriod and inflating the resulting weighted Fi -iccce..
In terms of efficiency, AmortizedPeriod exhibits remarkable superiority over other methods, as it requires the least running time and memory resources, Unlike RobustPeriod and RPT, which involve iterative optimization algorithms with computationally intensive operations such matrix inverses for accurate period estimation, AmortizedPeriod solely relies on the inference network that takes the original time series as input and outputs the estimated periods. Specifically, we focus exclusively on inferring the existence of period (i.c., ((w)) during inference, making the distribution of other components irrelevant. Thus, we employ only the inference network within the red dashed box depicted in Figure resulting in a lightweight network architecture. Additionally, when applying AmortizedPeriod, we leverage strategy where the original time series divided into N subseries with identical periods, based on the base periods of the time series. This approach is elaborated upon the end of Section 3.2 and in Appendix G when introducing the datasets. For instance, consider a time series spanning 120 days, with each day comprising N = 24 time points. By aggregating the i-th time point across consecutive days, we create N 24 time series, each with a length of 120. Consequently, we can collectively detect common periods (e.g., daily, weekly, and monthly) for these 24 time series. This practice effectively reduces the number of potential bases in the Ramanujan dictionary D. Recall that the attention is computed between the set of time series and the basis vectors, and so shrinking D can enhance efficiency. Unfortunately, this strategy is not applicable to other methods, they can only detect periods for a single time series instead of identifying common periods among a set of time series.
Furthermore, report the standard deviation computed over five trials in Table Our analysis reveals that the standard deviation for all methods is relatively small in comparison to the mean, especially for the SOTA methods.
27

---

Table The micro Fi -core for the ablation study. The results are averaged over 5 trials and the corresponding standard deviation provided in the brackets below.
Methods
V, 0.1. 8%
0.05
%
1.
Yahoo
App Flow
Train
Test
Train
Test
Train
Test
Train
Test
0.80
0.76
0.70
0.57
0.86
0.87
0.89
0.87
AmortizePPeriod
(1.62e-2)
(2.02e-2)
(1.58c-2)
(9.37c-3)
(2.36c-2)
(2.54c-2)
(2.00e-2)
(2.57e-2)
0.58
0.55
0.58
0.48
0.85
0.86
0.88
0.86
Trend
(7,52e-3)
(7.13e-3)
(4,33e-3)
(3. (6-3)))3
(2.87e-2)
(6.29e-3)
(5.61e-3)
(1.11e-2)
0.51
0.48
0.28
0.29
0.86
0.86
0.55
0.50
Outliers
(2.75e-2)
(4.83e-2)
(2.91e-2)
(2.85e-2)
(1.1le-3)
(3. (3.12e-3)
(4.88e-2)
(5. (2.e2)
0.71
0.70
0.53
0.49
0.85
0.84
Month
(1 (8)8))
(2.62e-2)
(1.62e-2)
(1.21e-2)
(3.84e-2)
(5.51e-2)
Table Full comparison between the benchmark methods on the CRAN dataset.
Metrics
AutoPeriod
RobustPeriod
RPD
AmortizedPeriod
train
0.05
0.23
0.26
0.50
Micro Ficrr
test
0.12
0.18
0.20
0.25
train
0.02
0.13
0.12
0.25
Macro Fi-score
test
0.08
0.12
0.10
0.12
train
0.05
0.42
0.49
0.51
Weighted Fi -coor
test
0.06
0.31
0.46
0.18
train
0.02
0.25
0.34
0.50
Sampled Fi -score
test
0.06
0.17
0.26
0.25
Time
1.76e-1
4.96e2
1.14c1
1.47
Mcmory
2.57MB
29.84MB
103.36MB
2.40KB
J ADDITIONAL EXPERIMENTS ON THE CRAN DATASET WITH SINGLE PERIODS
In this section, we evaluate the four methods for period detection using the CRAN data. The dataset consists of univariate time series extracted from open-source packages listed in the "Time Series Data" section of the CRAN Task View on Time Series Analysis Toller et al. 2019 5 It includes 82 time series in diverse range of application domains, including economic indicators like employment rates and retail sales, as well as environmental measurements such as pollution levels and sunspot counts.
It should be noted that the time series within this dataset have varying lengths. To enable the application of AmortizedPeriod to this dataset, a preprocessing step undertaken involving linear interpolation and truncation. This ensures that all resulting time series have the same length of 1040. Specifically, if a time series (4) has a length I,(+) less than 1040, we perform linear interpolation by introducing [1040/L(0) + new points between each pair of consecutive time points in the original series, where |.] represents the floor function. When L [+) is greater than or equal to 1040, truncate the original series, retaining the first 1040 time points. Following this preprocessing step, is observed that the true periods for all time series are multiples of 8, with the exception of two series() Consequently, these two series are excluded from the analysis. The resulting periods for all remaining time series range from 8 to 528. Similar to the aforementioned datasets, each sample partitioned into N 8 time series, each sharing the same period. The training and testing sets maintain a ratio of 4:1, respectively. For AmortizedPeriod, we train the inference network with 5e4 epochs and choose the checkpoints with the lowest training loss. The remaining settings are the same as before.
https://caan.-prjjeec. wwtp://wwwwwwwwwwww//eeeeeiee
"These series initially have periods of 7 and 52, respectively.
28

---

The results of our experiments presented in Table where it can be observed that the proposed AmortizedPeriod consistently outperforms other methods in terms of accuracy and efficiency across most evaluation criteria. It is worth noting that the performance of AmortizedPeriod data is comparatively lower than that on the training data. This discrepancy arises due to the limited number of time series in the training set (i.e., 64), which restricts the generalization capability of the learned inference network to unseen testing data. Nevertheless, we would like to emphasize that the periods for the training data are estimated in a self-supervised manner. Consequently, the superior Fi-score achieved by AmortizedPeriod indicates the versatillity of the proposed generative model in describing the training data, while the inference model effectively maps the observed time series to the distribution of the periods.
the testing
On the other hand, the hyperparameters for AutoPeriod and RobustPeriod were set to be the same as those used in the four datasets discussed in Section These parameters were manually tuned to yield good performance on those datasets. Although tuning these parameters specifically for the CRAN data might potentially enhance their performance, it would require additional effort. This underscores the importance of adaptive hyperparameter learning from the data, a feature effectively addressed by AmortizedPeriod.
Finally, for RPD, we employed cross- validaaii to select the single hyperparameter in all our experiments, which contributes to its relatively robust performance. However, since RPD does not account for the non-periodic components the data, its performance remains inferior to that of AmortizedPerridd particularly for the training data.
K LIMITATIONS, DISCUSSIONS, AND FUTURE WORK
K.1 LIMITATIONS OF AMORTIZEDPERIOD
First, AmortizedPeriod is more suitable for scenarios where there are hundreds or thousands of time series, as some training data is required to train the inference network. When there are only a few time series, we recommend utilizing the existing SOTA methods. In addition, before using it for inference, AmortizedPeriod needs to be trained for relatively long time. When the training dataset is larger, the training process can be time-consuming. However, after AmortizedPeriod is well-trained, the inference process relatively fast, shown in experiments.
K.2 DISCUSSIONS THE USAGE OF PERIODICITY DETECTION
As mentioned in Section periodicity detection offers benefits for other time series-related tasks, such as classification, clustering, decomposition, anomaly detection, and forecasting. In this regard, we will now delve deeper into how accurately estimated periods can boost the performance of these tasks.
Time series classification and clustering: Periodic patterns in time series data often contain infor- mative and discriminative features that can distinguish different classes clusters with the noise and irrelevant variations in time series filter out. For example, in heart rate monitoring, periodicity detection can identify recurring patterns, such as beat-to-beat intervals, enabling the differenti- ation between normal heart rhythms and irregularities related to cardiac conditions (Thungtong ct al. 2018). Similarly, in customer behavior analysis, periodicity detection helps identify regular patterns in purchasing behavior, facilitating the clustering of customers based on their shopping habits (Guidotti al. 2018).
Time series decomposition: Many typical time series decomposition methods, such MSTL (Ban dara et al.) 2021) and robustSTL. (Wen et al.) 2019). rely on predefined periods for extracting seasonal components. By accurately detecting periodic patterns,
extract underlying seasonal variations more effectively, leading to a cleaner and more accurate decomposition.
Periodicity detection aids in accurately identifying and characterizing normal patterns or behaviors in time series data. Establishing baseline for normal behavior based on the underlying periodic patterns enables the detection of anomalies or deviations from expected periodic behavior (Shehu & Harper 2023). For instance, in network traffic analysis. periodicity de- tection helps identify regular communication and data transfer patterns, enabling the identification of anomalies that deviate from the expected periodic behavior (Akpinar & Ozcelik) 2020
Anomaly detection:
29

---

Time series forecasting: Periodicity detection allows for the identification and modeling of seasonal, trend, and noise patterns in time series data, as mentioned above. By accurately capturing the underlying periodic components, variations more accurately, while reducing the influence of unpredictable noise. Periodicity patterns have been exploited explicitly implicitly in various forecasting algorithms, including ARIMA (Box & Jenkins 1968), prophet (Taylor & Letham 20188). N-BEATS (Oreshkin et al. 2019). Pyraformer (Liu al. 2022). TimesNet (Wu et 2023). etc.
extract and model seasonal and trend
K.3 FUTURE WORK
In future research, an intriguing avenue to explore is the concept of pre-training in the context of time series analysis. Specifically, we can investigate the feasibility of pre-training inference network using a large and diverse set of time series data. This pre-trained network can then be directly applied to perform periodicity detection without requiring additional training. Additionally, it would be valuable to assess whether the features extracted by the pre-trained inference network are also effective for other time series-related tasks, such as classification and forecasting. To this end, the pre-trained inference network could be utilized for downstream tasks with some fine-tuning, further investigating its potential for enhancing performance across various time series analysis applications.
30


Reviews:
[{"review_id": "B7FM1yOywd", "reviewer": "Reviewer_Qy29", "summary": "The paper studies the periodicity detection problem with the proposed AmortizedPeriod method by considering Bayesian modeling and deep learning. The motivation is good, the literature review is comprehensive, and the proposed method has competitive experimental performance. On the other side, some concerns are raised about the paper writing and the technical design and illustration. For more details, please refer to the following sections.", "questions": "Please refer to the first and the third points in the above section.", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The literature review is quite comprehensive, including the periodic time series, Ramanujan subspaces, and different kinds of priors.\n\n- The Bayesian modeling designed for periodicity identification seems novel and comprehensive.\n\n- The experiments are extensive, and the provided results are very competitive.", "weaknesses": "- The first and biggest concern raised by the reviewer is how the proposed two challenges get addressed by the proposed model. For the first challenge \"robustness\", how do the designed Bayesian modeling and encoder-decoder model address it? More importantly, how does the proposed AmortizedPeriod solve the \"memorylessness\" challenge, i.e., the second challenge, especially when the experimental running time of the proposed AmortizedPeriod is superior?\n\n\n- Section 3.1 is somehow deep for some audiences and does not motivate Section 3.2 very well, the plain definition occupies much space but neither motivates the formulation proposal in Section 3.2 nor adds values for the statement of why it is sensitive to trends, noises, and outliers.\n\n\n- The setting of semi-supervised learning in Section 3.5 is not very clear, e.g., what is the role of sub-period and how it works in the learning and classification.\n\n\n- The symbols, notions, and indexing are somehow messy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JE1UcqAQqr", "reviewer": "Reviewer_M5wN", "summary": "The authors propose a new approach for identifying periodic patterns in time-series data. They introduce AmortizedPeriod, which integrates Bayesian statistics and deep learning using an attention-based amortized variational inference model. AmortizedPeriod is capable of capturing various components of time series data, including multiple and irregular periods, trends, noise, outliers, and missing data. They also present a self/semi-supervised learning framework for the inference model to leverage knowledge from previously observed time series. Extensive experiments on four datasets demonstrate that AmortizedPeriod outperforms existing state-of-the-art methods with significant computational efficiency gains.", "questions": "(1) If CRAN is included, can it be described in two parts (single-periodicity detection and multi-periodicity detection) in the experimental analysis? I think that providing such an experimental analysis helps to further demonstrate the generalization of AmortizedPeriod.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "(1) The proposed model has good robustness in dealing with a variety of data anomaly scenarios, and addresses the limitations of current methods in terms of memorylessness. The authors present sufficient mathematical proofs in the method and appendix. \n(2) The attention-based amortized variational inference model is innovative. The self/semi-supervised learning framework allows the inference model to leverage past knowledge, improving the inference efficiency on new data. \n(3) The experiments implemented are relatively complete. From the experimental results, AmortizedPeriod has achieved a very significant advantage compared with state-of-the-art methods. The experimental section demonstrates the effectiveness and computational efficiency of AmortizedPeriod.", "weaknesses": "(1) The structure of thesis, however, is a little bit unreasonable. The description of the method takes up a lot of space, resulting in a somewhat inadequate analysis of the experiments that follow (although some are also presented in the appendix).\n(2) The baselines in this work may not be enough. It is better to supplement some methods proposed in recent years to baselines. (e.g., Robust Dominant Periodicity Detection for Time Series with Missing Data\", published at ICASSP 2023). \n(3) To enhance the solidity of this work, it is suggested to add some public data sets (e.g., CRAN) to evaluate the performance on AmortizedPeriod and the baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qXRItFF174", "reviewer": "Reviewer_ue83", "summary": "To tackle the problems of limited robustness and lacking memory, the authors proposed AmortizedPeriod for Periodic patterns combining Bayesian statistics and deep learning. The proposed method can model the correlation of the periods, trends, noise, and outliers in time series, and take use of missing data and irregular periods at the same time. The authors also conduct several experiments to prove the effectiveness of the proposed framework.", "questions": "1.\tCan the author include more preliminaries?\n2.\tCan the author include more error metrics?\n3.\tCan the author add more downstream tasks?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. The periodicity detection task is essential and interesting.\nS2. The authors provide rigorous theoretical analysis for their proposed methods.\nS3. The authors have conducted several experiments to indicate the effectiveness of their proposed methods.", "weaknesses": "W1. The authors may include more preliminaries to explain the tasks, such as problem definition. This can be beneficial for readers without sufficient background of periodicity detection tasks to understand the paper.\nW2. The authors may include more error metrics to measure the performance of periodicity detection tasks like other classification tasks.\nW3. The authors may add more downstream tasks to further evaluate the effectiveness of periodicity detection tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CYCmm4E2kV", "reviewer": "Reviewer_KTuk", "summary": "This paper focuses on the periodicity detection problem, and proposed a model AmortizedPeriod which can address the limited robustness and memorylessness problems in SOTA by applying Bayesian paradigm and variational inference. Extensive experiments are conducted.", "questions": "Please address the questions above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well organized and well presented.\n2. A novel model AmortizedPeriod is proposed which can address the limited robustness and memorylessness problems in the existing works.\n3. Extensive experiments are conducted to validate the model effectiveness, the results look promising.", "weaknesses": "1. Since the proposed model is trying to deal with time series data with self-attention, which also requires a lot of memory, there should be some experiments to compare the memories used by the proposed method and other methods.\n2. In the experiment, the results of multiple periodicity scenarios should be provided, for example, when the data has multiple periodicity, irregular periods, noise, outliers, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)