Topological Attention for Time Series Forecasting
Authors: Sebastian Zeng
Florian Graf
Christoph Hofer
Roland Kwitt
Venue: NeurIPS 2021 Poster (2021)
Keywords: Time series forecasting
Persistent homology
Attention
Topological Data Analysis

Abstract: The problem of (point) forecasting univariate time series is considered. Most approaches, ranging from traditional statistical methods to recent learning-based techniques with neural networks, directly operate on raw time series observations. As an extension, we study whether local topological properties, as captured via persistent homology, can serve as a reliable signal that provides complementary information for learning to forecast. To this end, we propose topological attention, which allows attending to local topological features within a time horizon of historical data. Our approach easily integrates into existing end-to-end trainable forecasting models, such as N-BEATS, and, in combination with the latter exhibits state-of-the-art performance on the large-scale M4 benchmark dataset of 100,000 diverse time series from different domains. Ablation experiments, as well as a comparison to recent techniques in a setting where only a single time series is available for training, corroborate the beneficial nature of including local topological information through an attention mechanism.

Topological Attention for Time Series Forecasting
Sebastian Zeng
Florian Graf
Department of Computer Science University of Salzburg sebastian settaattaannznn@ppaaeee www.@plusglp....ee.. ac. at
Department Computer Science
University of Salzburg ceriiinngra@@lllee florian. wrw.graffffe graff@lus.... ac.at at
Christoph Hofer
Roland Kwitt
Department of Computer Science University of Salzburg chofer&cosy. sbg. ac
Department of Computer Science
University of Salzburg
roland, roland.kwitttpluue www.ttplus..u...mmooooouk ac
Abstract
The problem of (point) forecasting univariate time series is considered. Most approaches, ranging from traditional statistical methods to recent learning-based techniques with neural networks, directly operate As an extension, we study whether local topological properties, as captured via persistent homology, information for learning to forecast. To this end, we propose topological attention, which allows attending local topological features within time horizon of historical data. Our approach easily integrates into existing end-to-end trainable forecasting models, such N-BEATS, and, in combination with the latter, exhibits state-of-the-arr performance on the large-scale M4 benchmark dataset of 100,000 diverse time series from different domains. Ablation experiments, comparison to broad range of forecasting methods in setting where only single time series is available for training, corroborate the beneficial nature of including local topological information through an attention mechanism.
time series observations.
reliable signal that provides complementary
well as a
1 Introduction
Time series are ubiquitous in science and industry, from medical signals (e-g., FEG), motion data (e.g., speed, steps, etc.) or economic operating figures to ride/demand volumes of transportation network companies (e.g.. Uber, Lyft, etc.). Despite many advances in predicting future observations from historical data via traditional statistical [5], or recent machine learning approaches [29, 42, 34, 32, 24, 44], reliable and accurate forecasting remains challenging. This not least due to widely different and often heavily domain dependent structural properties of time related sequential data.
In this work, we focus on the problem of (point) forecasting univariate time series, i.e., given length-T vector of historical data, the task is to predict future observations for given time horizon H. While neural network models excel in situations where large corpus of time series is available for training, the case of only single (possibly long) time series is equally important. The arguably most prominent benchmarks for the former type of forecasting problem are the "(M)akridakis" competitions, such as M3 [26] or M4 [28]. While combinations (and hybrids) of statistical and machine learning approaches have largely dominated these competitions [28, see Table 4], Oreshkin et al. [29] have recently demonstrated that pure coaming-based model (N-BEATS) attains state-of- the-art performance. Interestingly, the latter approach is simply built from a collection of common neural network primitives which are not specific to sequential data.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Kindowy
(Length
Windowy
(longth
Time (1)
Persi homology
Barcode
Pexational encading
Ve
2
R
R
(a Ow +
Tr Transformer Encodor
Figure 1: Illustration of topological attention, computed on time series ohservations I1,- TT. The signal is decomposed into collection of W overlapping windows of size For each window, a topological summary. ie.. persistence barcode B, is computed. These local topological summaries then vectorized (in R") via differentiable map Ve. fed through several transformer encoder layers [41] (implementing multi-head self-attention mechanism) with positional encoding at the input, and finally mapped to v E R by an MLP (best-viewed in color).
However, the majority of comming-basede approaches directly operate on the raw input signal, implicitly assuming that viable representations for forecasting can be learned via common neural network primitives, composed either a feedforward or recurrent manner. This raises the question of whether there exist structural properties of the signal, which are not easily extractable via neural network components, but offer complementary information. One prime example' are topological features, typically obtained via persistent homology [7, 13]. In fact, various approaches [31, 16, 11, 21, 15] have successfully used topological features for time series analysis, however, mostly in classification settings, for the identification of certain phenomena in dynamical systems, or for purely exploratory analysis (see Section 2).
Contribution. We propose approach to incorporate local topological information into neural forecasting models. Contrary to previous works, we do not compute global topological summary of historical observations, but features of short, overlapping time windows which the forecasting model can attend to. The latter is achieved via self-attention and thereby integrates well into recent techniques, such as N-BEATS [29]. Notably, in our setting, computation of topological features (via persistent homology) comes with moderate computational cost, which allows application in large-scale forecasting problems.
Problem statement. In practice, neural forecasting models typically utilize the last T' observations of time series in order to yield (point) forecasts for given time horizon H. Under this perspective, the problem boils down to learning a function (parametrized as a neural network)
b: RRT
+ R
x d(x)
from given collection of inputs (i.e., length-T vectors) and targets (i.e., length- H vectors). Specifi- cally, we consider two settings, where either (1) a large collection of time scries is available, as in the M4 competition, or (2) only have access to single time series. In the latter setting, model has to learn from patterns within a time series, while the former setting allows to exploit common patterns across multiple time series.
Related work
Persistent homology and time series. Most approaches to topological time series analysis are conceptually similar, building on top of work by de Silva et al. [10] and Perea & Harer [31, 30]. Herein, time series observations embedding [38] from which Vietoris-Rie Aitoriss-iisss (VR) persistent homology is computed. The resulting topological summaries, i.e., persistence barcodes. are then used for downstream processing. Within this regime, Gidea et al. [15] analyze the dynamics of cryptocurrencies using persistence landscapes [6]. Khasawneh al. [21] study chatter classification synthetic time series from turning processes
transformed into point cloud via time-delay coordinate
Although leaseing-based based approaches [36] exist approximate topological summaries (without guarantees).

---

and Dlotko et al. [11] identify periodicity patterns in time series. In [22], Kim et al. actually compute one-step forecasts for Bitcoin prices and classify price patterns, essentially feeding barcode statistics as supplementary features to a IP/CN--baeed regression model. Surprisingly, very few works deviate from this pipeline, with the notable exception of [16], where VR persistent homology not computed from time-delay coordinate embedding, but rather from assembling observations (within sliding windows of size n) from a d-variate time series into a d-dimensional point cloud, followed by VR persistent homology computation.
Although these works clearly demonstrate that capturing the "shape" of data via persistent homology provides valuable information for time series related problems, they (1) rely on handcrafted features (i.e., predefined barcode summary statisties, or fixed barcode- vectoortccttiiniaiion strategy), (2) consider topological summaries as the single source of information and (3) only partially touch upon forecast- ing problems (with the exception of [22]). Furthermore, in this existing line of work, sweeping a sliding window over the time series is. first and foremost, a way to construct a point cloud which represents the entire time series. Instead, in our approach, each window yields its own topological summary in the form of persistence barcode, reminiscent to representing sentence as sequence of word embeddings in NLP tasks. When combined with learnable representations of persistence barcodes [19, 8). this perspective paves the way for leveraging recent techniques for handling learning problems with sequential data, such as attention [41], and allows to seamlessly integrate topological features into existing neural forecasting techniques.
Neural network approaches to time series forecasting, Recently, various successful neural net- work approaches to (mostly probabilistic) time series forecasting have emerged, ranging from auto-regressive neural networks as in DeepAR [34], to (deep) extensions of traditional state space models, such as DeepFactors [42] or DeepState [32]. While these models are inherently tailored to the sequential nature of the forecasting problem, Li et al. [24] instead rely on the concept of (log-sparse) self-attention [41]. fed by the outputs of causal convolutions, and Oreshkin et al. [29] even abandon sequential neural network primitives altogether. The latter approach, solely based on operations predominantly found in feed-forward architectures, achieves state-of-the-arr performance for (point) forecasts across several benchmarks, including the aarge-scale M4 competition.
Yet, a common factor in all aforementioned works is that raw time series observations are directly input to the model, assuming that relevant structural characteristies of the signal can be learned. While we choose an approach similar to [24], in the sense that we rely on self-attention, our work differs in that representations fed to the attention mechanism are not obtained through convolutions, but rather through topological analysis step which, by its construction, captures the "shape" of local time series segments.
3 Topological attention
The key idea of topological attention to analyze local segments within an input time series, æ, through the lens of persistent homology. As mentioned in Section 2, the prevalent strategy in prior work is to first construct a point cloud from æ via a time-delay coordinate embedding and to subsequently compute VR persistent homology. Historically, this is motivated by studying structural properties of an underlying dynamical system, with a solid theoretical foundation, e.g., in the context of identifying periodicity patterns [31, 30]. In this regime, is encoded as point cloud in R²" by considering observations within a sliding window of size n as a point in R".
While the time-delay embedding strategy is adequate in settings where one aims to obtain one global topological summary, it inadequate when local structural properties of time series segments are of interest. Further, unless large (computationally impractical) historical time horizons are considered, one would obtain relatively sparse point clouds that, most likely, carry little information.
3.1 Time series local topological summaries
Instead of time-delay coordinate embedding, we follow an alternative strategy: time series signal, a, is still decomposed into a sequence of (overlapping) windows, but not to yield a point cloud element, but rather to be analyzed in isolation. In the following, we only discuss the necessities specific to our approach, and refer the reader to [14, 7, 4] for a thorough treatment of persistent homology.
3

---

tracking topaloggical changes while sarpping
(00)
(2-8)) 22-22/19.00 (19.0.9).0.0)
(19)
12- (() ( (.9.))
Tire sur 25
23
birth event
Barcodo: B2 ((rs, oc), (sta, (ra.
Figure Illustration of O-dimensional persistent homology computation for a time series æ length The barcode Ba encodes topological changes, in the form (birth, death) tuples, as we sweep through the growing sequence X₂ C .... C K of subsets of K. For example, the connected component born at 7a, dies at 22, caused by the merge with the connected component born (best-viewed in color).
To topologically analyze length-T time series, over the time steps {I...., [T]. computa- tionally tractable manner, lets consider a 1-dimensional simplicial complex of the form
K ((1).... (T); (1.2)...... (T- -,,1,
where {i] denote O-simplices (i.e., vertices) and I-simplices { (i.e., edges) are in K. if and only if i and are two consecutive time indices. Topologically. K. carries the connectivity properties of time series of length T. which equivalent to straight line. This is the same for all time series of length I' and thus offers no discriminative information.
Persistent homology, however, lets combine the purely topological representation of the time series with its actual values. For specific æ, let a1 I .... I or denote its increasingly sorted values and consider
c20 = O,
=o EK: 5 for E [T)
In fact, -= x2 C C S .... S K forms an increasing sequence of subsets of IC, i.e., a filtration. Importantly, while K. is the same for all time series of length T. the filtration, (K)) == is determined by the values of æ. Persistent homology then tracks the evolution of topological features throughout this sequence and summarizes this information in the form of persistence barcodes.
In our specific case, as topologically equivalent straight line, we only get O-dimensional features, i.e., connected components. Hence, we obtain one (0 degree) barcode B2. This barcode a multiset (birth, death) tuples, representing the birth (b) and death (d) of topological features. Informally, we may think of building K. piece-by-piece according to the sorting of the z4, starting with the lowest value, and tracking how connected components appear / merge, illustrated Fig.
Remark 3.1. The information captured throughout this process has noteworthy properties. First, it is stable in the sense that small changes in the observation values may not cause arbitrary changes in the respective barcodes, see [9]. Second, one may equally order the negative observations, i.e.. -x, and thus obtain B_ In that manner, the signal is analyzed from below and above.
Finally, extract local topological information, we do not compute one single barcode for æ, but one for each sliding window of size n, see Fig. 1. Given decomposition of x into W subsequent windows, we obtain W barcodes. ....... Bw. which constitute the entry point for any downstream operation. Informally, those barcodes encode the evolution of local topological features over time.
3.2 Barcode vectorization
Although persistence barcodes concisely encode topological features, the space of persistence bar- codes, denoted as BB, carries no linear structure [40] and the nature of barcodes as multisets renders them difficult ranging from fixed mappings into a vector space (e.g., [6, IJ) to kernel techniques (e.g., [33, 23)) and, more recently, learnable vectorization schemes (e.g.. [18, 81). Here, we follow the latter approach, as it integrates well into the regime of neural networks. In particular, the core element in learnable vectorization schemes differentiable map of the form
in learning settings. Myriad approaches have been proposed to alleviate this issue,
Vg B R,
B A
Σ (b,d)E
(e d)
(2)

---

where IR² - IR denotes a so called barcode coordinate function [18], designed to preserve the
stability property Remark 3.1. Upon assembling a collection E N such coordinate functions and subsuming parameters into O one obtains e-dimensional vectorization of B E B via
Ve B R
................................ B - (Ve, (B),..., Ve. (B))
Taking into account the representation of W persistence barcodes, we summarize the vectoriza- tion step as
TopVec: BW + Rxe, (BI,...,Bw'
aw) (Ve(B.),.... Ve(Bw)) (4)
This is distinctly different to [31, 30, 21, 22] (see Section 2), where one barcode is obtained and this barcode represented in a fixed manner, e.g., via persistence landscapes [6] via barcode statistics.
3.3 Attention mechanism
In order to allow forecasting model to attend to local topological patterns, as encoded via the aj. we propose to use the encoder part of Vaswani et al.'s [41] transformer architecture, implementing repeated application of (multi-head) self-attention mechanism. Allowing
attend local time series segments is conceptually similar to Li et al. [24], but differs in the way local structural properties are captured: not via causal convolutions, but rather through the lens of persistent homology, In this setting, the scaled dot-product attention, at the heart of a transformer encoder layer, computes
(AW*))
with A Eq.(4) (a1. .........w aw)
O = softmma
AW
(5)
Vah
and W Rexd, W E Rexd, W E Rexd, denoting learnable (key, value, query) projection
matrices. Recall that A w holds all e-dimensional vectorizations of the W persistence barcodes. In its actual incarnation, one transformer encoder layer', denoted as AttnEnc IR W'x + RWxe computes and concatenates M parallel instances of Eq. (5), (i.e., the attention heads), and internally adjusts du such that Md.. Composing E such AttnEnc maps, one obtains
TransforrmrFnnooerr nxxx xe W'ee
(6)
AttnEnci AttnEncH(A))
Finally, we use a two-layer MLP R We + RT (with ReLU activations) to map the vectorized output of the transformer encoder to T-dimensional representation. Topological atlention thus implements
TopAttn: BW - IRT
(B (B....., Bw) MLEFvoo(Transformeereccoaa TopVec(B,..... Bw))
where vec(.) denotes row-major vectorization operation.
Remark 3.2. Notably, as the domain of TopAttn is W error backpropagation stops at the persistent homology computation. However, remark that, given recent works differentiating through the persistent homology computation [17, 25], one could even combine topological attention with, e.g., [24], in the sense that the outputs of causal convolutions could serve the filter function for persistent homology. Error backpropagation would then consequently allow to learn this filter function.
3.4 Forecasting model
While the representation yielded by topological attention (see Eq. (7)) can be integrated into different neural forecasting approaches, it specifically integrates well into the N- N-BEATS model of Oreshkin et al. [29]. We briefly describe the generic incarnation of N- BEATS next, but remark that topological attention be similarly integrated into the basis-expansion variant without modifications,
omitttin the additional normalization and projection layers for brevity
5

---

Essentially, the generic N-BEATS model is assembled from stack of L double-residual blocks. For 1 < L, each block consists non-linear map (implemented as a MI.P)
N-BEATS
N-BEATS
S R R
with denoting the internal dimensionality, and two subsequent maps that yield the two-fold output of the 1-th block as
g
x' = (S' (a (c),))), and y' (ss((a))))
with U' E RT V E _ and æ" = While the æ! yield the connection to the following computation block, the y' are used to compute the final model prediction y E R via (component-wiss -wise) summation, i.e., y = yy' +... +..+2 y
(x'
St
Importantly, in this computational chain, the æ' can be leveraged an interface integrate additional information. our case, we enrich the input signal each block by the output of the topological attention mechanism through concatenation (hence, S² g2 + IRa, see figure to the top-right), i.e.,
Topatta = (x', v) where e = TopAttn(BB,......r BB))
This means that the time series signal is (I) imput (in its raw form) N-BEATS and (2) its topological attention representation, v. is supplied to each block as a complementary signal. In a similar manner (i.e., through concatenation), e can be included in much simpler models as well (see Section 4.2).
Computational complexity. Aside from the computational overhead incurred by the (multi-head) attention module, need to compute O-dimensional persistent homology for each sliding window. This can be done efficiently, using union find data structures, with complexity O (maa-
where m [K]] = 2n- 1 with the sliding window size and a (-) denoting the inverse of the Ackermann function. As the latter grows very slowly, computational complexity roughly linear for this part; see suppl. material for a detailed runtime study.
4 Experiments
We assess the quality of point forecasts in two different settings and perform ablation studies to isolate the impact of topological attention in the proposed regime.
Throughout all experiments. we compute persistent homology from x and - (see Remark 3. using Ripser [3]. Barcode vectorization, see Eq. (3), is based on rational hat coordinate functions [18] with the position parameters (i.e.. the locations of each coordinate function in R²) initialized by k-means++ clustering over all barcodes in the training data (with & set the number of coordinate functions). This yields representation E 2e per sliding window. Full architecture details can be found in the suppl. material.
Ablation setup. When assessing each component of topological attention in isolation, we refer to +Top as omitting the TransformerEncee time series observations to the transformer encoder, i.e., omitting TopVec in Eq. (7).
part in Eq. (7), and +Attn as directly feeding the
4.1 Evaluation metrics
To evaluate the quality point forecasts, two commonly used metrics are the symmetric mean absolute percentage error (sMAPE) and the mean absolute scaled error (MASE). Letting ú (#T+1...... ,.....,,.HHH.
2T+HH ) denote the length-At H forecast, y = (IT+I...,, +T+)) the true observations
and x (#1. .......,33 IT) the length-T history of input observations, both scores are defined as [28]
200
IT+ TT+:l
i= ITT+ ST+el
sMAPE( 9)
Σ
MASE(x: y. 9) Lizm+1 |
(10)
H
pr++l + [
Ii-m!

---

with m depending on the observation frequency. For results on the M4 benchmark (see Section 4.3). we adhere the competition guidelines and additionally report the overall weighted average (DWA) which denotes the arithmetic mean of sMAPE and MASE (with m pre-specified), both measured relative to naïve (seasonally adjusted) forecast (also provided by the M4 competition as Naive2).
4.2 Single time series experiments
We first consider the simple, yet frequently occurring, practical setting of one-step forecasts with historical observations available for only single length-N time series. Upon receiving time series R (with T < N), a model should yield forecast for the time point T + (i.c., H = 1).
4.2.1 Dataset
To experiment with several single (but long) time series of different characteristics, we 10 time series from the publicly available electricity [12] demand dataset and four (third-party) time series of car part demands, denoted as car-parts. Based on the categorization scheme of [37], the time series are chosen such that not only smooth time series (regular demand occurrence and low demand quantity variation) are represented, but also lumpy ones (irregular demand occurrence and high demand quantity variation). For electricity. the respective percentages are 70% vs. 30%, and, for car-parts, 75% vs. 25%. All observations are non-negenn non-negative. In case of electricity, which contain measurements in I 15min intervals, we aggregate (by summation) within 7h windows, yielding total of 3,762 observations. For car car-pare car-parts, demand is measured on daily basis across a time span of 7-8 years (weekends and holidays excluded), yielding 4,644 observations For each time series, 20% of held-out consecutive observations are used for testing, 5% for validation.
average.
4.2.2 Forecasting model
We employ simple incarnation of the forecasting model from Section 3.4. In particular, we replace N-BEATS by a single linear map (with bias), implementing
((B,... Bw), + w ToT(a (æx, TopAttn(B,,.... Bw:)) +
(11)
with (z, TopAttn(B)..... ..... Bw) )) denoting the concatenation of the topological attention signal and the input time series æ, as in Eq. (9). During training. we randomly extract T + 1 consecutive observations from the training portion of the time series. The first 7 observations are used input æ, the observation at T + 1 is used as target. Forecasts for all testing observations are obtained via a length-T rolling window, moved forward one step at time.
In terms of hyperparameters for topological attention, we use single transformer encoder layer with four attention heads and 32 barcode coordinate functions. We minimize the mean-squared-error via ADAM over 1.5k (electricityy and 2k (car-parts) iterations, respectively, with batch size of 30. The initial learning rate of the linear map in Eq. (11) is set to 9e-2, the initial learning rates for the components of Eq. (7) are listed in Scction 4.3.2, scaled up by a factor of 10. Regarding the latter, empirically found that models trained single time series typically require larger learning rates, most likely due to the reduced variation in the batches throughout training (as we sample from a single time series). All learning rates are annealed following cosine learning rate schedule.
4.2.3 Results& & Ablation study
We compare against several techniques from the literature that are readily available to a practitioner. This includes autoARIMA [20], Prophet [39], a vanilla LSTM model, as well as several approaches implemented within the GluonTs [2] library. With respect to the latter, we list results for single- hidden-layer MLP, DeepAR [34] and MQ- BIQ-CNN/NG RNN [43]. By Naive, we denote baseline, yielding IT as forecast for ZT+1- Importantly, each model is fit separately to each time series in the dataset.
For fair comparison, we further account for the fact that forecasting models typically differ in their sensitivity to the length of the input observations, æ. To this end, we cross-validate T (for all methods) using the SMAPE on the validation set. Cross-validation points are determined by the topological attention parameters W and n, i.c., the number and lengths of the sliding windows. For n ranging from 10 to 200 and W ranging from 5 to 45, we obtain wide range of input lengths, from
hnteps://archive.it uci http:///www../ae/tce/ccccaaaaaaa
7

---

Table 1: Single time series experiments car-parta and electricity, using the aMAPE performance criterion. Listed are (1) the average rank (Ø Rank) each method, as well as (2) the average percentual difference (% Dift.) to the Rank-1 approach per time series. | denotes GluonTS [2] implementations.
(a) car-parta (#time series:
(b) electricity (#time series: 10)
Method
Rank
% Diff.
Method
Rank
Diff.
Lin +TopAttn
1.50
1.73
Lin. +TopAttn
1.60
5.43
Prophet
2.75
4.05
"DeepAR
1.80
7.71
MML
3.00
5.99
'MLP
2.90
14.54
'DeepAR
3.50
8.24
'MQ-CNN
4.70
45.07
autoARIMA
5.25
13.82
autoARIMA
5.10
45.31
LSTH
6.00
16.54
Prophet
6.10
61.28
MMQ-RNN
7.50
34.65
LSTM
7.40
69.70
Naive
7.75
30.39
"MQ-RNN
7.50
68.53
MMQ-CNN
7.75
29.49
Naive
7.90
69.82
14 to 244. Instead listing absolute performance figures, we focus on the average rank within the cohort of methods, well as the average percentual difference the best-ranking approach per time series.
Table lists the overall statistics for electricity and car-parts. We observe that, while the overall ranking per dataset differs quite significantly, Lin. San.+TopAtten consistently ranks well. Second, the average percentual difference to the best-ranking approach per time series is low, meaning that while Lin.+Tonp +TopAttn might not yield the most accurate (wrt. the sMAPE) forecasts on a specific time series, it still produces forecasts of comparable quality.
Table provides the same performance statistics for an ablation study of the topological attention com- ponents. Specifically, we combine the linear model of Eq. (11) with each component of topological attention in isolation.
Some observations are worth pointing out: First, the linear model (Lin.) alone already performs surprisingly well.
Table 2: Ablation study
This can possibly be explained by the fact that the task only requires one-step forecasts, for which the historical length-2' observations (directly preceding the forecast point) already quite informative. Second, directly including topological features (i.e., +Top) has a confounding effect, We hypothesize that simply vectorizing local topological information from all sliding windows, without any focus, obfuscates relevant information, rather than providing reli- able learning signal. This also highlights the importance of attention in this context, which, even when directly fed with observations from each sliding window (i.e., +Attn), ex- hibits favorable performance (particularly on car-parts). However, in the latter strategy, the input dimensionality for transformer encoder scales with the sliding window size
X Rank
% Diff.
car car-parts
(#time series: 4)
Lin.
2.75
0.15
+Top +Attn
3.50
1.45
1.25
0.15
+TopAttn
2.50
0.64
electricity (#time series: 10)
Lin.
2.40
5.95
+Top
3.40
12.20
+Attn
2.30
2.41
+TopAttn
1.90
2.10
Contrary to that, in case of topological attention. the input dimensionality is always fixed to the of number coordinate functions, irrespective of the
sliding window size
4.3 Large-scale experiments on the M4 benchmark
Different to Section 4.2, we now consider having nultiple time series of different lengths and characteristics available for training. Further, instead of one-step forecasts, the sought-for model needs to output (multi-step) point forecasts for time horizons H 1.
'he sMAPE determines the rank of method per time series; these ranks then averaged all time series

---

Table 3: Performance comparison the M4 benchmark in of sMAPE/ / DWA, listed by subgroup. N-BEATS and T-BEATS+TopAttn denote an ensemble formed by training multiple models, varying from 2// to 5H and randomly initializing each model ten times (i.e., a total of 40 models per subgroup). Forecasts are obtained by taking the median over the point forecasts of all models. f denotes results from [27, 28).
Yearly (23k)
Quarterly (24k)
Monthly (48k)
Others (5k)
Average (100k)
Method
WWinner M4 [35]
13.176/ 0.778
9.679 0.847
12.126 0.836
4.014 /0.920
11.374 / 0.821
"Benchmark
14.848 / 0.867
10.175 / 0.890
13.434 / 0.920
4.987
1.039
12.555 / 0.898
NNaive2
16.342 / 1.000
11.011 1.000
14.427 1.000
4.754 1.000
13.564 / 1.000
N- BEATE [29]
13.149 0.776
9.684 0.845
12.054 0.829
3.789 0.857
11.324 0.814
N- BEATSSTTppttn
13.063 0.771
9.687 0.845
12.025 0.828
3.803 / 0.860
11.291 0.811
4.3.1 Dataset
Experiments are based on the publicly available M4 dataset', consisting of 100,000 time series from six domains, aggregated into six subgroups that are defined by the frequency of observations (i.c., yearly, quarterly, monthly, etc.). Forecasting horizons range from 6 (yearly) H 48 (hourly). The test set is fixed and contains, for all time series per subgroup, exactly H observations to be predicted (starting at the last available training time point); see suppl. material for dataset statistics.
4.3.2 Forecasting model
We employ the forecasting model of Section 3.4 and closely follow the architecture and training configuration of [29, Table 18). Our implementation only differs in the hidden dimensionality of N-BEATS blocks (128 instead of 512) and in the ensembling step. In particular, for each forecast horizon (i.e., for each subgroup), [29] train multiple models, varying T from = 2H to T = 7H, using ten random initializations and three separate loss functions (sMAPE, MASE, MAPE). One final forecast per time series is obtained by median- aggregatinn of each model's predictions. In our setup, we solely rely on the sMAPE loss function, vary T only from T 2H to T 5/1, but still use ten random initializations. Even with this (smaller) ensemble size (40 models per subgroup, instead of 180), N- BEAATS alone already outperforms the winner of M4 (see Table 3). As we are primarily interested in the effect of integrating topological attention, sacrificing absolute performance for smaller ensemble size is incidental.
In terms of hyperparameters for topological attention, the length, n, of sliding windows is set to 007 . Tj. where 7 varies per subgroup as specified above. The model uses 20 transformer encoder layers with four attention heads and 64 structure elements for barcode vectorization. For optimization, we use ADAM with initial learning rates of le-3 (for N-BEATS and the MLP part of www.orroernoedeeeeeeeeee All learning rates are annealed according cosine learning rate schedule over 5,000 iterations with batch size of 1,024.
Eq. (7)), 8e-3 (TopVec) and 5e-3
4.3.3 Results & Ablation study
Table 3 lists the sMAPE and OWA for the winner of the M4 competition [35], as well as the Naive2 baseline (with respect to which the OWA is computed) and the M4 benchmark approach, obtained as the arithmetic mean over simple, Holt, and damped exponential smoothing.
In terms of the OWA, we see an overall 0.4% improvement over N-BEATS and 1.2% improvement over the M4 winner [35]. In particular, topological attention performs well on the large yearly monthly subgroups of 23k and 48k time series, respectively. While OWA scores are admittedly quite close, the differences are nog-negligible, considering the large corpus of 100k time series. In fact, several methods in the official M4 ranking differ by an even smaller amount with respect to the OWA measure.
Table Ablation study
Method
sMAPE
OWA
N-BEATS
11.488
0.827
+Top +Attn
11,505
0,920
11.492
0.826
+Toptttn
11.466
0.824
Savailable at https: httppp//ggisa......mmmptttttooot wwwp///coopeeiiiiioo//eeeeeeeeee
based the N-BEATS reference implementation https://gittuu/ wwwp://ttww..ccmeeeeeeeeeeeeeeee wwt.//////eeee///////eeme AI/N-BEATS

---

Similar to the ablation results of Section 4.2, the ablation study in Table 4 (conducted for T = 2H only) reveals the beneficial effect of topologcal attention, in particular, the beneficial nature of allowing to attend to local topological features. Contrary to the ablation in Table 2, we see that in this large-scale setting, neither topological features (+Top), nor attention (+Attn) alone yield any improvements over the already strong N- BEBEATS model. In fact, when integrated separately into N N-BEATS, both components even deteriorate performance in terms of the sMAPE.
Number of parameters. Regarding a comparison of N-BEATS and N-BEATS-TopAtttn in terms of the number of parameters, we note that the particular N-BEATS incarnation in our experiments has a total of 11.7M parameters; our approach adds about 700k additional parameters. To assess whether the reported improvements are simply due to a larger overall model, we ran additional experiments increasing the size of N -BEATS (by doubling the hidden dimensionality) to =6.4M parameters, As in Table 4, this study was performed using a historical time horizon of = 2H and all results are averaged over 10 random initializations of each model. In terms of the OWA, the larger N-BEATS model achieves an average score of 0.826 ys. 0.827 for N-BEATS and 0.824 for N-BEATS+ToPAttta (the latter two numbers correspond to the results in Table 4). Hence, substantially increasing the number of parameters for N- -EAAT indeed improves the overall score (as expected), but does not reach the OWA result of our model.
5 Conclusion
While several prior forecasting works have pointed out the relevance of local structural information within historical observations (e.g.. [24]), it is typically left to the model to learn such features from data. Instead, we present direct approach for capturing the "shape" of local time series segments via persistent homology. Different to the typical application of the latter in signal analysis, we capture the evolution of topological features over time, rather than a global summary, and allow forecasting model to attend to these local features. The so obtained topological attention mechanism yields a complementary representation that easily integrates into neural forecasting approaches. In combination with N-BEATS [29], for instance, large-scale experiments on the M4 benchmark provide evidence that including topological attention indeed allows to obtain more accurate point forecasts.
Societal impact. Due to the ubiquity of time series data, forecasting in general, certainly touches upon variety of societally relevant and presumably sensible areas. As our work has potential impact in that sense, we perform large-scale experiments over a wide variety of time series from different domains, thereby obtaining a broad picture of the overall forecasting performance.
Source code is publicly available at https: //t///ggttt/////////// com hhap:plusrkwwitt//ee
Acknowledgments & Disclosure of funding
This research was supported in part by the Austrian Science Fund (FWF): project FWF P31799-N38 and the Land Salzburg (WISS 2025) under project numbers 20102- F1901 166-KZP and 20204- W1SS/225/197-2019, The first author also gratefully acknowledges financial support from Porsche Holding Austria and Land Salzburg within the WISS 2025 project "KFZ" (P1900123). Finally, like to thank all anonymous reviewers for their constructive feedback during the review process.
References
[1] H. Adams, T. Emerson, M. Kirby, R. Neville, C. Peterson, P. Shipman, S. Chepushtanova, E. Hanson, F. Motta, and L. Ziegelmeier. Persistence images: A stable vector representation of persistent homology. JMLR, 18(8):1-35, 2017.
[2] A. Alexandrov, K. Benidis, M. Sablke-Schneider V. Flunkert, J. Gasthaus, T. Januschowski, D.C. Maddix, S. Rangapuram, D. Salinas, J. Schulz, L. Stella, A.C. Türkmen, and Y. Wang. GluonTS: Probabilistic and neural time series modeling in Python. JMLR, 21(116): 2020.
[3] U. Bauer. Ripser: efficient computation of vietoris-rips persistence barcodes. arXiv, 2021. https://arxiv.. org/ at//ab///199 /1908 02518.
[4] J.-D. Boissonnat, F. Chazal, and M. Yvinec. Geometric and Topological Inference. Cambridge
Texts in Applied Mathematics. Cambridge University Press, 2018.
10

---

[5] P.J. Brockwell and R. Davis. Time Series: Theory and Methods. Springer Series in Statistics. Springer, New York, 1991.
[6] Bubenik. Statistical topological data analysis using persistence landscapes. JMLR, 16(1):77-
102, 2015.
[7] G. Carlsson. Topology and data. Bull. Amer. Math. Soc., 46:255-308, 2009.
[8] M. Carriere, F. Chazal, Y. Ike, Lacombe, M. Royer, and Y. Umeda. PersLay: A neural network layer for persistence diagrams and new graph topological signatures. In AISTATS, 2020.
[9] D. Cohen-Steiner, Edelsbrunner, and Harer. Stability of persistence diagrams. Discrete Compul. Geom., 3731):103-120, 2007.
[10] V. de Silva, P. Skraba, and M. Sijdemo-Johanseee Topological analysis of recurrent systems. In NIPS Workshop on Algebraic Topology and Machine Learning, 2012.
[11] P. Diotko, W. Qiu, and S. Rudkin. Cyclicality, periodicity and the topology of time series, arXiv, 1900 12118.
2019, https://araa
[12] D. Dua and C. Graff. UCI machine learning repository. http://archive. ics. uci uci.edu/ml, 2017.
[13] H. Edelsbrunner and J. L. Harer. Computational Topology An Introduction. American Mathematical Society, 2010.
[14] H. Edelsbrunner, D. Letcher, and A. Zomorodian. Topological persistence and simplification. Discrete Comput. Geom., 28(4):511-53: 2002.
[15] M. Gidea, D. Goldsmith, Y. Katz, P. Roldan, and Y. Shmalo. Topological recognition of critical transitions in time series of cryptocurrencies. Physica A: Statistical mechanics and its applications, 548: 123843, 2020.
[16] M. Gidea and Y. Katz. Topological data analysis financial time series: landscapes of crashes. Physica A: Statistical mechanics and its applications, 191:820-834, 2018.
[17] C. Hofer, F. Graf, M. Niethammer, and R. Kwitt. Graph filtration leaming. In ICML., 2020.
[18] C. Hofer, R. Kwitt, M. Dixit, and M. Niethammer. Comeetivity-oppimized representation learning via persistent homology. In ICML, 2019.
[19] C. Hofer, R. Kwitt, and M. Niethammer. Learning representations of persistence barcodes. JMLR, (0(126):1-45, 2019.
[20] R.J. Hyndman and Khandakar. Automatic time series forecasting: The forecast package for R. Journal of Statistical Software, 27(2). 2008.
[21] F.A. Khasawneh, F. Munch, and J.A. Perea. Chatter classification in turning using machine learning and topological data analysis. IFAC, 51(14):195-200, 2018.
[22] K. Kim, J. Kim, and A. Rinaldo. Time series featurization via topological data analysis. arXiv, 2018. https: httppp://aaxivvcoom www.xivv..ogree.o hatg//a///88 httbs/////..... 01812.0098777
[23] G. Kusano, K. Fukumizu, and Y. Hiraoka. Persistence weighted Gaussian kernel for topological data analysis. In ICML, 2016.
[24] S. Li, X. Jin, Y. Xuan, X. Zhou, W. Chen, Y.-X. Wang, and X. Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In NewrIPS, 2019.
[25] F. Chazal M. Carriere, M. Glisse, Y. Ike, and H. Kannan. Optimizing persistent homology based functions. In ICML, 2021.
[26] S. Makridakis and M. Hibon. The M3-competition: results, conclusions and implications. International Journal of Forecasting, 16(4):451-44 2000.

---

[27] S. Makridakis, E. Spillooii,, and V. Assimakopoulos. The M4-competition: Results, findings, conclusion and way forward. International Journal of Forecasting, 34(4):802 2018.
[28] Makridakis, E. Spiliotis, and Assimakopoulos. The M4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1):54-74, 2020.
[29] B.N. Oreshkin, D. Carpov, N. Chapados, and Y. Bengio. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. ICLR, 2020.
[30] J.A. Perea. Persistent homology of toroidal sliding window embeddings. ICASSP, 2016.
[31] J.A. Perea and J. Harer. Sliding windows and persistence: application of topologiaa methods signal analysis. Foundations of computational mathematics, 15:799-883 077999888, 2015.
[32] S.S. Rangapuram, M. Seeger, Gasthaus, L. Stella, Wang, and T. Januschowski. Deep state space models for time series forecasting. In NeurlPS, 2018.
[33] R. Reininghaus, U. Bauer, S. Huber, and R. Kwitt. A stable multi-scale kernel for topological machine learning. In CVPR, 2015.
[34] D. Salinas, V. Flunkert, J. Gasthaus, and T. Janschowski. Deepar: Probabilistic forecasting with
autoregressive recurrent networks. International Journal of Forecasting, 36(3), 2020.
[35] Smyl. A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting. International Journal of Forecasting. 36(1):75-88 85, 2020.
[36] A. Som, H. Choi, K.N. Ramamurthy, M.P. Buman, and P. Turuga. PI-Net: A deep learning approach to extract topological persistence images. CVPR, 2020.
[37] A.A. Syntetos, J.E. Boylan, and J.D. Croston. On the categorization of demand patterns. Journal of the Operational Research Sociery, 56(3):495-55) 2005.
(38] F. Takens. Detecting strange attractors in turbulence. In Dynamical Systems and Turbulence. Springer, 1981.
[39] S.J. Taylor and B. Letham. Forecasting at scale. Peerl Preprints 5:e3190v2, 2017. https: //doi.org// 7287 /128//peerj..... prrpprnts.. 3190v2.
[40] K. Turner, Y. Mileyko, Mukherjee, and J. Harer. Fréchet means for distributions persistence diagrams. Discrete Comput. Geom, 52(1):44-70 2014.
[41] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, K. Jones, A.N. Gomez, L. Kaiser, and
I. Polosukhin. Attention all you need, 2017.
[42] Y. Wang, A. Smola, D.C. Maddix, J. Gasthaus, D. Foster, and T. Januschowski. Deep factors for forecasting. In ICMI., 2019.
[43] R. Wen, K. Torkkola, B. Narayanaswamy, and D. Madeka. A multi-horizon quantile recurrent forecaster. arXiv, 2017. https: /ttp///axvvvv..////v///////aaa wttp:///////.........
[44] H.-F Yu, N. Rao, and I.S. Dhillon. Temporal regularized matrix factorization for high- dimensional time series prediction. In NIPS, 2016.
12


Reviews:
[{"review_id": "vwRaLLeyb7f", "reviewer": "Reviewer_Erx9", "summary": "The paper presents an approach to combine topological features to time-series forecasting.\nEssentially, the method computes 1D topological barcodes at each time-steps, pass them to a transformer and then provides those features as additional input to NBEATS. Experiments are conducted on two datasest when performing local forecast (e.g. forecasting the time-series independently) and on M4 when using global forecast (learning the model jointly across time-series). An improvement of 0.4%/1.2% is reported on M4 for OWA/sMAPE.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is clearly written and well positioned compared to existing work both in forecasting and application of persistent homology for ML. The method is original and while reusing existing techniques from persistent homology, the contribution on time-series is novel and interesting on its own.\n\nMy main concern for this paper lies on the experiments. Tiny improvements are reported compared to existing approach and it was not clear how much improvement can be attributed to the technique proposed compared to other factors (noise, HPO).\n\nFirst, the paper reports a specific setup for their method where learning rates are adjusted per components: for instance for the univariate setting, the learning rates are 9e-2 (for topological feature mapping), 5e-2 for the transformer taking topological features and 9e-2 for the output linear projection. For the M4 setting, different learning rates values are used. Introducing different learning rates is problematic, it would require an additional HPO budget (especially given that the two settings have different values) and it is not clear if the improvement reported by the authors would hold if a similar setup would have been used for NBEATS. I believe, a single learning rate should be used in all models to have a more apple-to-apple comparison.\n\nSecond, the results reported by the authors are very small (0.4%/1.2% improvement on M4 for OWA/sMAPE). This also raises a few questions. The noise on M4 is very large: different network initializations may give results differences that are close to the range reported by the author. As such, multiple seeds should be run to clearly show that the improvement reported is not simply due to noise. Also, while a useful ablation study has been carried out, it is a bit insufficient in my view. Part of the improvement could be done solely to the increase of parameters. As such, an ablation should also consider a model trained with roughly the same number of parameters to validate that the improvement comes from the topological features and could not have been achieved by \"just\" increase the hidden size of NBEATS. \n\nI hope those points can be improved by the author during the rebuttal. One thing I was hopping when reading the paper was also to provide more insights or qualitative visualizations on the topological features as they are quite new for time-series/ML readers (why do they work, how do they help to detect complex seasonality pattern).\n\nAdditional remarks:\n\n- l148 the description barcode coordinate function should be included in order to make the paper self-contained, some space can be gain in this section by just saying in words that you concatenate the e topological features at each time-steps\n- l196: I agree that the computational complexity would not suffered with the computation of barcodes. However, a more important discussion should be on the number of additional parameters to the initial method.\n- l249: how did you find the learning-rate per component? It would be challenging for the method to tune 3 learning-rate on each dataset and this diminish the value of the contribution.\n- Table 2 what are (2) and (10)?\n- l288: \"However, in the latter strategy, the input dimensionality for the transformer\n encoder scales with the sliding window size n.\" I do not find this argument convincing, one could slice the number of observations to the same dimensionality as your topological representation.\n- l324: \"1.2% improvement over the M4 winner\" this number has to be compared with the noise obtained with multiple seeds.\n- l69: *Single* time series experiments\n\n\n-- edit after rebuttal\n\nI would like to thank the authors for the clear answer to the points raised in my reviews regarding hybrid learning-rates, difference in number of parameters and improvement reported compared to noise sensitivity (the latter point that seems has been noted by the other reviewers as well).\n\nThe answers and detailed analysis clarified my doubts, there is an improvement that can be clearly attributed to the method proposed (but require careful examination, I hope the analysis can be included in the manuscript). While the improvement is small, the benchmark considered (M4) is also highly competitive and the method is original and promising, as noted by other reviewers, as such I will raise my score.\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3.5"}, "extra_text": {}}, {"review_id": "ZZUg7Hfx-J", "reviewer": "Reviewer_DXFs", "summary": "The paper proposes topological attention, a type of self-attention mechanism that captures the topological information of the signals. In contrast to earlier works, this work focuses on local instead of global topological summaries. The main difference is attributed to the extraction of such summaries for each sliding window and not over the global historic data. The paper discusses how such sets of local summaries can be vectorized and incorporated into neural network architectures. The experimental results on an existing forecasting approach show the benefit of topological attention vs. the regular (global) attention mechanism.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-written and the introductory figures help in the understanding of the proposed approach. \n\nThe novelty of the paper is somewhat limited, considering that, conceptually, the main difference is the use of an existing approach over sliding windows. However, the vectorization and integration of such extracted topological summaries are not trivial and, therefore, there is merit in having such an approach documented.\n\nThe results are not impressive. For the single time series experiments the method used to incorporate the topological attention seems to already perform surprisingly well due to the simplicity of the task. However, it is interesting to see that standalone topological features without attention mechanism reduce performance and, therefore, the benefit of using both approaches combined becomes evident.\n\nThe results on the M4 forecasting competition show similarly not impressive trends: the performance depends on the horizon, some times a simple attention mechanism performs better. However, even a 1% improvement might be statistically significant considering the size of the corpus here. The ablation study creates a much clearer picture of the benefits.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "63RZ1F3TW96", "reviewer": "Reviewer_RPLH", "summary": "This paper proposes a new approach for one-dimensional time-series forecasting that is based on augmenting the data with topological information. Specifically, the authors design an attention module that extract the local connections in their windowed persistent homology barcodes, and concatenates this information to the original time-series. The authors evaluate their method on two scenarios: a single series and multiple time-series elements. Their techniques shows modest improvement over existing approaches.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This submission proposes an interesting idea: in addition to operating on the raw data, it suggests to augment the data with local topological information that is drawn from persistent homology computations of the data. In practice, the proposed method includes a module that generates the persistent barcodes, and an attention module which extract local connections in the windowed barcodes. To the best of my knowledge, this idea is novel and was not suggested before. In general, I like this approach, as it allows to connect a fundamental problem (time-series forecasting) with an established theory and practice (persistent homology). As such, this work has the potential to inspire further research that combines persistent homology in the context of time-series processing or other machine-learning problems. Overall, the exposition is clear, and the authors made an effort to include many details about persistent barcodes and their computation. I believe their description in 3.1 & 3.2 could be further, however, I do not have a concrete suggestion on how to do it---perhaps use less general descriptions. For instance, you can directly write what is the particular $mathcal{V}_\\theta$ you used right below eq. 2.\n\nThe main issue I found with this submission is the somewhat on-par results in the multi-series setting. In particular, the improvement over N-BEATS in Table 3 seems marginal. Moreover, the comment at the end of pg. 8 makes me think that the problem is more fundamental. Namely, since the authors did not use the ``best'' version of N-BEATS, and improve on a subpar version of it, it might be that the proposed mechanism yields no improvement when added to the full N-BEATS. At this point---we have no way of knowing. With that being said, I am still on the positive side, given the potential of this work.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See above.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a form of local topological attention to improve the performance of a time series forecasting model. The reviewers agree that the proposed method is interesting and relevant, and the writing is clear. 
On the other hand, all reviewers caution that the performance improvements demonstrated empirically are small (~ 1% on M4), and two reviewers see the novelty as somewhat limited. However, the authors were able to alleviate further concerns regarding the empirical evaluation during the discussion period, leading all reviewers to vote for (weak) acceptance. The reviewers belief that the merits outweigh the flaws of the paper, and I concur with their assessment, recommending acceptance.
Decision: Accept (Poster)