Targeted Attacks on Timeseries Forecasting
Authors: Yuvaraj Govindarajulu
Avinash Amballa
Pavan Kulkarni
Manojkumar Parmar
Venue: Submitted to ICLR 2023 (2023)
Keywords: timeseries
forecasting
targeted attacks
adversarial ml
ai security
apgd

Abstract: Real-world deep learning models built for Time Series Forecasting are used in several critical applications from medical devices to the security domain. Many previous works have shown how deep learning models are prone to adversarial attacks and studied their vulnerabilities. However, the vulnerabilities of time series models for forecasting due to adversarial inputs are not extensively studied. While attack on a forecasting model might be intended to deteriorate the performance of the model, it is more effective, if the attack is focused on a specific impact on the model's output. In this paper, we propose a novel formulation of Directional, Amplitudinal, and Temporal targeted adversarial attacks on time series forecasting models. These targeted attacks create a specific impact or hide a potential high-impact area on the forecasting output. We use the existing adversarial attack techniques from the computer vision domain and adapt them for time series. Additionally, we propose a modified version of the Auto Projected Gradient Descent attack for targeted attacks. We explore the impact of the proposed targeted attacks against untargeted attacks. We use KS-Tests to statistically prove the impact of the attack. Our experimental results demonstrate how targeted attacks on time series models are practical and are more powerful in terms of statistical similarity. It is, hence difficult to detect through statistical methods. We believe that this work opens a new paradigm in the time series forecasting domain and is an important consideration for developing better defenses.

Under review as a conference paper at ICLR 2023
TARGETED ATTACKS ON TIMESERIES FORECASTING
Anonymous authors
Paper under double-blind review
ABSTRACT
Real-world deep learning models developed for Time Series Forecasting are used in several critical applications ranging from medical devices to the security do- main. Many previous works have shown how deep learning models are prone adversarial attacks and studied their vulnerabilities. However, the vulnerabilities of time series models for forecasting due to adversarial inputs are not extensively explored. While attack on forecasting model might aim to to deteriorate the performance of the model, cific impact on the model's output. In this paper, propose novel formulation Directional, Amplitudinal, and Temporal targeted adversarial attacks on time series forecasting models. These targeted attacks create specific impact the amplitude and direction of thre output prediction. We use the existing adversar- ial attack techniques from the computer vision domain and adapt them for time series. Additionally, we propose modified version of the Auto Projected Gradi- ent Descent attack for targeted attacks. We examine the impact of the proposed targeted attacks verses untargeted attacks. We use KS-Tests to statistically demon- strate the impact of the attack. Our experimental results demonstrate how targeted attacks on time series models are viable and are more powerful in terms of sta- tistical similarity. It is, hence difficult to detect through statistical methods. We believe that this work opens new paradigm in the time series forecasting domain and represents an important consideration for developing better defenses.
more effective, if the attack is focused on a spe-
INTRODUCTION
Time Series Forecasting (TSF) tasks are seen in many real-world problems across several domains. The wide range of domains of applications include demand forecasting Carbonneau et al. (2008), anomaly detection Laptev et al. (217)), stock price prediction jaee Kim (2003), electrical pricing Crespo Cuaresma et al. (004) and weather forecasting Grover et al. (2015). Improved availability of data and computation resources has reflected in the recent efforts (Rasul et al. (2020), Wen et al.
(217)), Oreshkin et al. (2019) of applying deep learning techniques for forecasting tasks. The wide applications of such deep learning models have led to threat due to adversaries and hence also the work towards exploration and prevention (Rathore et al. (2021), Cao & Gong (2017). Li & Li (020 of such adversarial attacks.
For a given classification model, the goal of the adversary could be either targeted or untargeted. In targeted attacks, the adversary tries misguide the model to particular class other than the true class, In an untargeted attack, the adversary tries to misguide the model to predict any of the incorrect classes. The definition of targeted are well-defined for classification tasks and have been used in several previous works (Goodfellow et al. (2014)). Kurakin et al. (2016), Croce & Hein (2020). These definitions are not applicable for regression tasks such as TSF In the adversarial machine learning domain, time series tasks have received significantly less attention as compared to those of computer vision. Also, the adversarial attacks and defenses studied the computer vision domain are not always useful for time series, requiring specific adaptations and re-definitioss.
In this paper, we address the above-mentionel shortcomings by providing a formulation for targeted attacks on TSF. To do this, extend the popularly known adversarial attacks from the computer vision domain to time series forecasting. Together with popular attacks such as Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD), also propose modified variant of Auto PGD attacks. We perform KS-tests on the loss attributes of the output forecasts (predictions) study the statistical properties of the proposed targeted attacks.

---

Under review as a conference paper at ICLR 2023
The contributions of our work are as follows:
1. We define and formalize targeted attacks on deep learning time series forecasting.
2. We propose modified Auto PGD attack for Time Series Forecasting (MAPGD-TSS)) an extension of the AutoPGD algorithm for targeted time series attacks, which can be extended to any regression task.
3. Through statistical tests, we show that inputs with targeted perturbations much more indistinguishable than untargeted attacks through empirical studies on the Google Stock and Household electric power consumption datasets.
2 RELATED WORK
Most of the approaches on adversarial attacks were first started on image classification in the deep learning domain. [Szegedy et al. (2013) proposed adversarial examples for image recognition. which initiated direction to investigate adversarial attacks in various domains. [Gooffello et al. (2014b) proposed the Fast Gradient Sign Method (FGSM) which is a single-step attack. In a similar line, Kurakin et al. (2018) presented an iterative version of FGSM called the Basic Iterative Method (BIM). These attacks pave the way for the current offte-of-art adversarial attacks in computer vision. Khamaiseh et al. (2022) summarizes the existing white-bos and black-box adversarial attacks and compares them on the basis of time taken, strength, and transferability of the attacks. Additionally,
by focusing on how they are employed in real-world applications, they have offered thorough description of the most popular defense mechanisms against adversarial attacks.
Oregi et al. (2018) introduced the first adversarial attacks Time Series Classification (TSC). Fawaz et al. (2019) used existing adversarial attack mechanisms such as the FGSM, BIM to de- crease the accuracy of residual networks for Time Series Classification. |Rathore et al. (021 brings the first notion of targeted attack on time series data. The targeted attacks are however focused on TSC tasks. Yang et al. (2022) brings the notion of a black-box method called TSady on the TSC task. This work suggests a gradient-free black-box strategy to attack DNNs for TSC with local perturbations based on time series shapelets and differential evolution, which contrast to prior work that required gradient information and global perturbations.
Pialla et al. (2022) introduced Smooth Gradient Method (SGM) attack based on gradient method and shows how adversarial training is a good way to improve a time series classifier's (TSC) robust- ness against smoothed perturbations by enforcing a smoothness condition on generated perturbations that contains spike and sawtooth patterns. The work Karim et al. (2019), takes into consideration that the time series models are sensitive to abnormal perturbations in the input and stringent re- quirements perturbations. To address this, the work crafts time series adversarial based the importance of measurement. The adversarial inputs are subjected to models performing time series prediction tasks such LSTNet, CNN-, RNN- and MHANET-based models. An importance-bast adversarial attack needs much smaller perturbations compared to other existing adversarial attacks. The work, however, does not formulate or address the targeted attacks in time series forecasting.
Another work Mode & Hoque (2020) on time series forecasting, explores the vulnerabilities of deep learning mule series regression models adversarial samples. The work also focuses on gradient-based white box attacks on deep learning models such as CNNs,
Units (GRU), and Long-Short Term Memory (LSTM) models. The vulnerabilities are shown to be trans- ferable and have the ultimate consequences. This work also focuses on untargeted adversarial at- tacks with an aim to increase the error of the deep learning model's output. Dang-Nhu et al. (2020) considers an adversarial setting in probabilistic framework auto-regressive forecasting models. This work uses Monte-Carle estimation in approximating the gradient of expectation and addresses the challenge of effectively differentiating through Monte-Carlo estimation using reparametrization and ecore-function estimators. This also proposes an under-estimation attack and an over-estimation attack electricity consumption prediction for reparametrization and ccore-function estimators.
2

---

Under review as a conference paper at ICLR 2023
3 SETTINGS AND FORMULATIONS
3.1 THREAT MODEL
Time Series Forecasting. Given a time series X [7T time series forecasting task predicts the value of 2T+1 based on the previous samples [ -1-w+1, ...rr.. where to is the window size under consideration. The sample IT41 corresponds the forecasted value and is often represented by Y.
Adversarial Time Series. An adversarial perturbation typically superposed on a given input time ...,....
series X, to construct X given by [411. ..,, ..r.. The adversarial time series X (Xade) is intended to significantly worsen the output prediction Y of TSF model.
Goal of Adversary The goal of the attacker is to create a targeted output impact on the time series. We consider Lo -bounded perturbation that causes targeted attack. We consider the definition of white-box access, where the gradients of the model are available for loss calculation. The Threat model can also be extended in transfer attack scenario, where Oracle model created through extraction in a black-box setting can be used as white-box. Further, the range of the input after the perturbation is assumed be accepted by the model. The inputs otherwise, limited to the acceptable input range of the model. We denote the regressor f R(M) R(N) with parameters 8, the predicted output for input E (M() is represented as y y fxx).
Properties of the perturbation. In general, adding a perturbation to the input should detoriate the performance of the model prediction. Additionally, It is also important for the perturbation to satisfy additional requirements including: 1. Small changes to the input to create bigger perfor- mance degradation on the output. Larger perturbations to the inputs are also easily detectable by the input plausibility check modules. It is notable that it more expensive to achieve higher input perturbation. 2. Imperceptible perturbations, attributing to reduced risk due to detection of the input perturbation. The perturbation is hence formulated as a constrained optimization problem, where f the Deep TSF model under consideration and indicates the strength of the attack.
= macimizzll Ý11
(1)
s.t ||X - 1 e, where Y = f(X) and Ý = f(x)
3.2 KS TESTS IN TIMESERIES
popular non-parametric test method in statistics, to test whether sample follows a reference probability distribution (one sample KS-Test) or where two samples follow a given probability distribution. Given a single sample, the distance between the reference probability distribution and the empirical distribution of the given sample is measured. Corresponding to the above distance, a p-value is calculated. Given a significance value a. the null hypothesis that the sample follows the reference distribution is acceptable, if the p-value is acceptable if is greater than the significance value a.
Kolmogorov-Sminn Sminnov Test (KS Test) Massey (1951)
In order to compare the attributes of actual prediction and outputs due to adversarial inputs, we sider the error of a TSF model output within given window. We consider the Root Mcan Squared Error (RMSE) between the original prediction with the outputs due to targeted and untargeted at- tacks. The distribution of the MSE across all the windows in the given dataset roughly follows a normal distribution, making KS-test usable for statistical analysis.
4 TARGETED TIME SERIES ATTACKS
Considering the limited number of work in the time series adversarial area and the need for the definition of targeted attacks the time series domain, we go about the formulation. We take into account the practical aspects that create maximum impact on the output. Depending on the type of impact targeted on the forecasting output, the adversarial attacks are classified as follows:
DIRECTIONAL (DTA). In a Directional Targeted Attack (DTA), the attacker crafts the attack in a way, a minor perturbation on the input causes a shift in the direction of the output. The forecasted output could be shifted upwards downwards.
3

---

Under review as a conference paper at ICLR 2023
AMPLITUDINAL (ATA), In Amplitudinal Targeted Attack (ATA), the attacker crafts the attack such that a minor perturbation the input causes the output amplitude to be limited within prescribed threshold. In such given scenario, the attacker would want to hide any high-impact areas on the output for his benefit.
TEMPORAL (TTA). In a Temporal Targeted Attack (TTA), the attacker crafts the attack such that a minor perturbation in the input causes a specific time region in the output to be manipulated. In the given region, the attacker would want to change the direction of the output (DTA) or the amplitude of the output (ATA) in specific time window (f1, t2). The attacker performs DTA or ATA specifically in the target time window to realise TTA. TTA can hence be considered sub-category of DTA or ATA.
The attacks are formalized in the context of the respective adversarial attacks (FGSM, PGD, and APGD) the further sections. In the further equations, the input time series without perturbation is denoted as zo and the adversarial time series Tade- For amplitudinal attacks (ATA), denotes the threshold to limit the output amplitude. In case of directional attacks (DTA), the factor a accordingly. For temporal attack (TTA),
attributes to the direction of the attack and is either +1 the type of impact to be created within a given time window (t1, t2) could be either DTA or ATA and is referred as (.) in the below equations. All the below attacks are defined for L. norm.
4.1 TARGETED FGSM
Fast Gradient Sign Method (FGSM), The FGSM attack Goodfellow et al. (2014a) introduces as notion of gradient-based adversarial attack. The adversarial input is crafted by adding small amount of perturbation that increases the loss of the true class making the model misclassify the input Todv- For classification example, the perturbation noise is calculated as the gradient of the loss function L with respect to the input image for the given true output class. On the other hand, targeted attacks decrease the loss with respect the target.
Todu = x + 6 sign( C(,( x. Mtrue)
Targeted FGSM. Targeted Attacks for TSF using FGSM are defined as in equation where L represents the loss or cost function that was used as an optimization function during model training.
dta = To sign( (V, (C(0, x. + a (yl))
Lodu,ata To sign V (C(0, (im(T,
4.2 TARGETED PGD
Projected Gradient Descent (PGD). The PGD attack is an extension of Iterative-FGSM (popularly called Basic Iterative Method (BIM) Kurakin et al. (2016). In this iterative method, after each step of perturbation, the adversarial example is projected back into the ball of x using the projection function II. The perturbed image after iterations is denoted by xade
[1 +e- V2(C(0, (0-00==1111 (trrae))
x adv
Targeted PGD. The targeted PGD attacks for TSF extend the targeted attack techniques of FGSM in the PGD context.
Fddv [1 - (2(( x-1 yy)) where, y/ E Ey + a lvl, lim v), at)()
(5)
4.3 TARGETED APGD
Auto Projected Gradient Descent (APGD). Auto PGD (or simply APGD) an extension of the
PGD attack addressing the sub-optimal step size and objective function issues. APGD Croce & Hein 2020) proposes 1. adding momentum to the gradient step while progressively reducing step size, 2. two conditions to update the step size when there is successful increase in the objective function and to carly-stop when there is no improvement towards the objective function. In algorithm [ we propose MAPGD-TSE, an extension of the APGD algorithm and is discussed in detail in section
4

---

Under review as a conference paper at ICLR 2023
Algorithm modified (Targeted) APGD for Time Series Forecasting
Require: Regression Model f. Space S, input 2!",, step size momentum parameter a, iterations Niterr checkpoints W, target yr, Loss L
x(1) Ps(x(0) (L(((x(") v()
if ((((x()) ),3)) ((((x()) ) y() then
Smin t -(x=()) and min f- x: (0)
else:
fmin f(x(1)) and Umin f- x1)
end
for n 2(++1 t- (s(x(1) - -n- (L(f(x(()), )(1)
1, Nuter - do
x(++1) t- ps(x(()) + (2(n+1) x( + (1 (xx(t) xl(-11))
((f(x((+1)), 3()) < [(fmin yr) then
fmin + f(x(1+1)) and Emin f- (a+1) end if
n E W then if Conditionl or Condition2 then
17 + 2 and g(n+1) + Imin
end if
end if end for
In the targeted APGD attacks, we first go about presenting modified Auto PGD attacks for the Time Series forecasting (mAPGD-TSF) a The modified variant applies to any regression model general. The following modifications are made the algorithm in comparison to AutoPGD to account for targeted time series applications. The changes are namely:
Loss function L. The confidence or the output accuracy is replaced with loss function L. The goal to increase the confidence function is replaced with goal to minimize the loss with respect to the target.
Output Target. The output target is adapted as per the targeted attacks DTA, ATA and TTA respectively: y/ {y + ly), y), alt(y(()))
Initialization. One-step PGD is performed before the start of an iterative reduction of loss. The one-step PGD is considered as an initialization, as the step initializes the minimal loss fmin-
The conditions related to step size selection are retained from the original AutoPGI Work Croce & Hein (2020). The step size is halved if one of the conditions is true. With p(n) is the lowest objective value with the least loss in the first n-iterations, below are the two conditions:
1. -1
< p. (w, wz-1),
-
2. 17 (1 -1) (w5)) and ((w5-1) ((w,)
EXPERIMENTS
5.1 SETUP: DATA AND MODEL
Energy prediction is a classical problem in time series forecasting. These prediction results need to be accurate however, an attacker could potentially generate adversarial samples in order to alter the predictions which might lead to significant losses for customers and the energy markets. Here thus study the impact of targeted adversarial attacks on the household electric power consumption data Repossioryy (. This dataset comprises measurements of electric power in household from 2006 to 2010 with 1-minute sampling rate. In addition to date and time, the dataset contains seven other variables including global active power, global reactive power, voltage, global intensity,
5

---

Under review as a conference paper at ICLR 2023
capha-)))
3.-
1205 00:3
W
sigral
preditted FCOM Lergeeed
FSSM Lapsied
0.6-
86
(a) Targeted DTA-Up attack 0.1)
Targeted ATA-Amp attack 0.1)
cos Down (aisha
Untargeted artauks
1.0-
predices
GG2 La gutud
ME FOOM unttepeted P-D paredd
8.
APCD MM
0.3
(c) Targeted DTA-Down attack (c
0.5)
(d) Untargeted attack (c 0.1)
Figure 1: Google stock data prediction on Targeted attacks (a), (b), (c) and untargeted attack (d)
and sub-metering to 3). We use seven variables or input features (global active power, global reactive power, voltage, global intensity, and sub-metering (1 to 3)) to forecast global active power after re-sampling the dataset from minutes to hours on the normalized data.
Stock prediction is high-demand, high-impact and high-risk area in the financial markets. Stock market prediction can have direct impact on individuals, organizations, and nation as a whole, thus inaccurate predictions are seen as high-impactful and high-risky. However, these models are also vulnerable to attackers, where one can launch an adversarial attack to make the necessary predic- tions. Here study the impact of targeted attacks on the google stock dataset (Dataset (220 This data set comprises google stock prices for the past 5 years i... 2017 to 2022 with I I-day sam- pling rate (except holidays). In addition to the date, the dataset consists of open, high, low, close, and volume. We use five variables or input features (open, high, low, close, and volume) to forecast the stock open prices on the normalized data.
For both, Google stock and Household electric power consumption datasets, we divide the overall time series into several time windows of size 5. The split for the Google Stock data into train and test datasets with a 4:1 ratio (train data over 4 years: test data over year). For the electrical power consumption data, we split them in 2:3 ratio (train data over 2 years: test data over years). Different model architectures were experimented for both the datasets including LSTM Hochreiter & Schmidhuber (997). Vanilla-RNN (Rumelhart & McClelland (9877), Rumelhart & McClelland (1987a)), GRUCho et al. (2014)annd variants of CNN (Hoseinzade & Haratizadeh 2018), Borovykh ct al. (2017). Out of which, LSTM best fits both datasets with minimum RMSE on the test data. Trained the LSTM models using MSE loss with an ADAM optimizer for 50 epochs with early stopping. RMSE on 60% household electric power consumption test data is 0.085, RMSE on 20% google stock test data is 0.040.
5.2 TARGETED ATTACKS
We run the proposed targeted attacks i.e. DTA, ATA and TTA using FGSM, PGD, mAPGD for e = [0.01, 0.1, 0.5, 1.0, 1.5]. For DTA, we denote a = for direction up. a = -1 for direction down for both the datasets. Threshold in ATA, denoted by is taken be 0.9 for google stock
data, т = 0.2 for houschold electric power consumption. For TTA, take (2) = (100, 150) and
(1,, (2) (50, 100) for both the datasets. We use the MSE loss function in calculating the gradient

---

Under review as a conference paper at ICLR 2023
Temporal--emp
0.7
colona prodirte
oiggis proaicte
PGD targsted
PCD APGD digie
APGD largited
Move
0.
53
300
1.00
200
256
100
1.9
300
250
bours
(a) Targeted TTA Up attack (r- 0.1)
(b) Targeted TTA-Amp attack 0.1)
Anoiiudinal MTrccholSS-
0.1, Uncorgesed
origina
ofigiial
0.4-
PRI targettd
untorocted
targited
unta
24
02-
62
-22-
200
1.50
200
250
1000
150
200
220
(c) Targeted ATA attack (E- 0.1)
(d) Untargeted attack (E attack
0.1)
Figure 2: Household electric power consumption data prediction on Targeted attacks (a), (b), (c) and untargeted attack (d)
with respect to inputs. Additionally, to compare the efficacy of the targeted attacks proposed, we run the untargeted attacks using FGSM, PGD, and APGD for different e values. Untargeted attacks for FGSM, PGD, and APGD follow a notation where the attack results in an increased output loss with respect to the original output. MSE is used as loss function for gradient calculations as previously done for targeted attacks.
RESULTS AND DISCUSSION
6.1 COMPARISON OF RESULTS
Figure I shows the stock open price (original, predicted, FGSM predicted, PGD predicted, APGD predicted) vs across the number of days for targeted attacks with different values. We see that the prediction output for FGSM, PDG, and APGD overlap with each other for smaller e-values on DTA and TTA-Up, TTA-Down attacks. For the DTA-Up attack Figure [). the entire predicted output has shifted up compared to the original prediction. Similarly, for the D'TA-Down in Figure (c)) the prediction has moved down. Figure
[c)) also shows the impact of higher epsilon on the output prediction. For TTA-Amp in Figure [[). only the prediction between time steps (t1, 12) has clipped to the threshold for PGD and APGD. However, here FGSM is not performing as expected. FGSM, instead of clipping the output prediction to the threshold, bringing the prediction down.
Figure Saaure[showa shows the global active power(original, predicted, FGSM predicted, PGD predicted, APGD predicted) vs days for different sorts of targeted attacks. We see that the prediction output for PGD, and APGD overlap with each other, whereas FGSM slightly deviates a bit for DTA and TTA-Up, TTA-Down attacks for smaller e-values. For TTA-Up Flg, the predicted output between (21, has shifted up compared to the original prediction. Similar case with TTA-Amp in Fig- 2re Figuree((c) shows the ATA attack resulting in clipping the output prediction to the thresh- old. Similar to google stock data, FGSM deviates from PGD, and APGD in ATA attack here as well. On the other hand, untargeted attacks on google stock data Figure d) and household electric power consumption Figure 2 d) results show that there is control over the prediction output. All three attacks (FGSM, PGD, APGD) increase the loss with respect to true output, but we can't 't really infer meaningful results from the output prediction since it doesn't 't have any pattern specified. Hence in
7

---

Under review as a conference paper at ICLR 2023
Derodtional Up (aiph=1))
Dlroct ona Down (aloha-11)
FCSP lareted
FG24
IGSM
PCD lagieed
10-
FGD saguted
POR targolid
APCE largtied
AFCD largeaa
AFTOD uargesed
2.4
0.2
PRD Largoed
(orgtee)
23-
20-
A400 Depited
033
0.0
404tin
epeion
opoion
output RMSE loss vs e, input norm vs DTA- Up attack on google data
output RMSE loss vs input L2 norm vs for DTA-
Down attack google data
Tampord al (t 12-150)
Tampora Anpiitadial 10.500 12-100)
3.4 1G24
43
FCSM
FGSM targeted
PGD cargited
PCD targined
PcB targeied
3.2
APGD sargesed
APGB largeted
25
AP-2 targsted
FG2 targened
E
0-2
23
WPCD targesed
3.4
<poion
epd km
output RMSE loss c, input vs for TTA- Amp allack on google data
(d) output RMSE vs input norm for TTA-
Amp atlack household electric power consumption duta
Figure 3: Loss (RMSE, L2 norm) vs epsilon plots for targeted FGSM, PGD, APGD on google stock data (a), (b), (c) and household electric power consumption data (d)
all practical scenarios, targeted attacks are preferred over the untargeted for their properties, and use cases and are also widely self-explainable compared to untargeted attacks.
6.2 Loss VS. AMOUNT OF PERTURBATION
Figure shows the RMSE loss between adversarial output and true output vs epsilon, Lz norm of the adversarial input and true input vs epsilon. We consider La norm as the distance metric for analysis since Loo can reveal only the maximum perturbation. Results in figure3]c.d) demonstrate that in ATA attack, and TTA-Amp attack, FGSM has a higher loss input space as well as in output space compared to PGD and APGD. This justifies the results in figure bb), figure for FGSM not achieving the desired target in the ATA, and TTA-Amp attacks due to the fact that FGSM is single-step attack and performs poorly for targets near the original input (in cases like ATA). On the other hand, PGD and APGD perform well for ATA, and TTA-Amp attacks. PGD and APGD) are iterative and successful in reducing the input distance achieve the attacks. For DTA, TTA-Up, and TTA-Down attacks Figure a.),) FGSM, PGD, and APGD perform similarly for lower values of epsilon. But for higher values of epsilon, FGSM has higher loss in input space compared to PGD and APGD. Thus, we recommend using PGD and APGD techniques for targeted attacks.
also studied the effect of the perturbation the input based several attributes in the time domain such as the L2 norm. For smaller time windows, the perturbations do not statistically affect the input attributes and hence For larger window sizes, the impact becomes significant but however not observable in the time domain. It is observed that the perturbations tend to differ from the inputs because of their higher- frequency components. These high-frequency components can then be detected through simple input frequency plausibility tests.
In experiments,
immune any input statistical tests.
6.3 KS TEST RESULTS
We perform statistical analysis on the targeted attacks using the KS-test and analyze the output statisties using histograms. For output statisties, plot the RMSE loss between the adversarial prediction and the true output for all the targeted attacks in addition to the original RMSE loss, and untargeted RMSE loss in histogram with window size of 5 for both the datasets considered.
8

---

Under review as a conference paper at ICLR 2023
(a) PGSM (TTX-Amp, 0.1) on google
(b) PGD (TTA-Amp,
0.1) on google
(c) APGD (TTA-Amp,
0.13 google
(d) FGSM (DTA-Up.e 0.1) on houschold electric power data
(c) PGD (DTA-Up. 0.1) on bousehold electric power data
(1) APGD (DTA-Up.e = 0.13 on bousehold electric power data
Figure 4: Histograms (output statistics) representing the original loss, targeted loss ,and untargeted loss distributions for FGSM, PGD, APGD attacks on the google stock data (a), (b), (c) and household electric power consumption data (d), (c), (f)
Google stuck
Honsehold
Attack
FGSM
PGD
APGD
FGSM
PGD
APGD
T-U
TU
DTA-up
0.876
0.676
0.582
1.99
DTA-doow
0.708
0.514
0.788
0.959
0.514
0.708
0.959
0.514
0.992
1.0
0.788
0.572
1.0
0.916
0.916
AIA
0.271
0.929
0.769
0.113
0.999
0.959
0.113
0.99
0.859
0.436
1.0
0.936
0.12
1.0
0.984
0.116
1.0
0.984
TTA-up
0.137
0.959
0.931
0.337
0.999
0.931
0.137
8999
0.931
0.208
1.0
0.848
0.152
1.0
0.968
0.152
1.0
0.968
TTA-down
0.157
0.959
0.858
0.357
0.999
0.858
0.157
8.99
0.858
0.196
1.0
0.94
0.34
1.0
0.998
0.14
1.0
0.988
-TA-amp
0.117
0.959
0.882
0.044
0.959
0.989
0.044
0.959
0.959
0.084
1.0
1.0
0.006
1.0
1.0
0.036
1.0
1,0
Table 1: KS test (Output statistics) for all the proposed targeted attacks based on comparing the orig- inal loss, targeted loss, untargeted loss distributions on google, household electric power consump- tion dataset for = 0.1. Here O-T, O-U, T-U represent the KS statistics between Original-TTareet Original- Untargeted, and two distributions.
attacks. The lesser the values imply the closer the
Figure 4 demonstrates that PGD and APGD targeted are statistically similar to the original compared to FGSM targeted.
Table shows the statistical difference between original, targeted, and untargeted attacks for O.I. Table[[jindicate that in google stock data, FGSM is statistically far from original compared to PDG, and APGD in ATA attack. Also for household electric power consumption data, FGSM is far from the original compared to PGD and APGD in almost all the attacks. It observable from the table and figure that that i) targeted attacks are more statistically similar to the original predic- tion compared to untargeted attacks, this infers that targeted attacks are less likely to be detectable compared to untargeted attacks. ii) targeted PGD and APGD perform better compared to FGSM.
7 CONCLUSION
In this paper, we introduced the first notion of targeted attacks on Time Series Forecasting. We define and formalize three types of targeted attacks using existing white box attack techniques. Together with FGSM and PGD. proposed modified version of AutoPGD attacks for regression tasks to apply the formulation. Our experimental results prove successful targeted attacks on the forecasting outputs. Through histograms and KS-test, we statistically show that the targeted attacks are closer and hence undetectable compared to untargeted attacks. Further work in the directions includes blx extensions and analysis of input statistics in the transformed frequency domain. This work opens up a new direction toward the exploration of defenses for TSF.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Anastasia Borovykh, Sander Bohte, and Cornelis W. Oosterlee. Conditional time series forecasting with convolutional neural networks, 2017.
Xiaoyu Cao and Neil Zhenqiang Gong. Mitigating evasion attacks to deep neural networks via region-basee classification. CoRR, abs/1709.05583, 2017. URL BtLptt::://erxvvorgg/
Real Carbonneau, Kevin Laframboise, and Rustam Vahidov. Application of machine learn- ing techniques for supply chain demand forecasting. European Journal of Operational doi: hutps//dbb.org//110006....o. 2006.12.004 URL https:/ http::/wwwwwwwwweeeeeeeeeeees httt:s///ww..eeeeeeeeee.oc.cc/// http:///ww......cceeeeeeeeaaaaa. 50377221706012000
Research, 184(3): 140-)114401154, 2008. ISSN 0377-2217.
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülgehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical ma- chine translation. CoRR, abs/1 406.1078, 2014. URL http:///axxiv..or/bbbssss/
1078
Jesús Crespo Cuaresma, Jaroslava Hlouskova, Stephan Kossmeier, and Michael Obersteiner. Forecasting electricity spot-prices using linear univariate time-series models. Applied En- ergy, 77(1)87-106,, January 2004. Utt.httss::///itt http://daaarrp...oooo.oo http:///eeeeeeeeeeeeeeeeeeee aptp://////eeapppeee//// appene/
v v77y200411p87-1066 .tm.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with ensemble of diverse parameter-free free attacks, 2020. URL htttp:://axxv..ot wttp://w...gg/a////03..... 01 01690
Raphaël Dang-Nhu, Gagandeep Singh, Pavol Bielik, and Martin Vechev. Adversarial attacks on probabilistic autoregressive forecasting models. In International Conference on Machine Learn- ing. pp. 2356-2365, PMLR, 2020.
Google Stock Dataset. Google stock dataset (www.nasdaq.com) https:: www.wnasda..com com/
syybol//goo//historricoo 2020.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alait Muller. Adversarial attacks on deep neural networks for time series classification. 20/9 Inter national Joint Conference on Neural Networks (IJCNN) pp. 1-8. IEEE, 2019.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, 2014a.
6572.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 14 2014b.
Aditya Grover, Ashish Kapoor, and Eric Horvitz. deep hybrid model for weather forecasting. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '15, pp. 379-386, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781 4781450336
org/100.114577782358.. 1
doi: 10.1145/783258.7887777 URL https://doi.
Sepp Hochreiter and Jürgen Schmidhuber: Long Short-Term Memory. Neural Computation, (8):1735-1780, 1997. ISSN 0899-7667. doi: (..1162/neco.19911 1997.9.8. 1735. URL https:
//doi.org/10. 1et.:1112/nnneee
1997.9...... 9977..8..773
Ehsan Hoseinzade and Saman Haratizadeh. Cnnpred: Cnn-based stock market prediction using several data sources, 2018.
wttp://gg/......ooooo/o 08923
Kyoung jae Kim. Financial time series forecasting using support vector machines. Neurocom- puting, 55(1):307-319, 2003. ISSN 0925-2312. doi: http:///hiinng/0000000//9992/222
00372-2. URL https: /tt:://www... www.e//eieeetttco//eeeeeeeeeeeee
50925231203003772 Support Vector Machines.
10

---

Under review as a conference paper at ICLR 2023
Fazle Karim, Somshubra Majumdar, and Houshang Darabi. Adversarial attacks time series, 2019.
URL httpss:/aaxiii....//pppp abs/ 190..1007770.
Samer Y. Khamaiseh, Derek Bagagem, Abdullah Al-Alaj, Mathew Mancino, and Hakam W. Alo- mari. Adversarial deep learning: A survey on adversarial attacks and defense mechanisms on image classification. JEEE Access, pp. 1-1, 2022. doi: 10. 10.1109/ 20009/AACCESSS222222811113333
Alexey Kurakin, lan J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. CoRR, abs/1607.0225 02233,,, 2016. UtL.http://arxivv http:///aawiii..oo/o////b/////// 02533
Alexey Kurakin, lan J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pp. 99-112. Chapman and Hall/CRC, 2018.
Nikolay Pavlovich Laptev, Jason Yosinski, Li Erran Li, and Slawek Smyl. Time-series extreme event forecasting with neural networks at uber. 2017.
Deqiang Li and Qianmu Li. Adversarial deep ensemble: Evasion attacks and defenses for malware detection. CoRR, abs/2006. 16545, 2020. URLhttps:: /ttp//aaxvvv/..////v//e////aalvl http//axwvvooo./a/a/// abgg//////// 2006. 6545
Frank J. Massey. The kolmogoroy-smminov test for goodness of fit. Journal of the American Statis-
fical Association, 46(253):68-78, 1951. ISSN 01621459. URL http: www. jstor org/
stabl e 2280095
Gautam Raj Mode and Khaza Anuarul Hoque. Adversarial examples in deep learning for multivari- ate time series regression. CoRR, abs/2009. 11911, 2020. URL www.hhtss///axxii.crrrett
2009. 11911
Izaskun Oregi. Javier Del Ser, Aritz Perez, and Jose Lozano. Adversarial sample crafting for time series classification with elastic similarity measures. In International Symposium on Intelligent and Distributed Computing, pp. 26-39. Springer, 2018.
Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-BEATS: neural basis expansion analysis for interpretable time series forecasting. CoRR, abs/1905.10437, 2019. URL
htttp://axiv..oom ort. abs 9000 10437
Gautier Pialla, Hassan Ismail Fawaz, Maxime Devanne, Jonathan Weber, Lhassane Idoumghar, Pierre-Alain Muller, Christoph Bergmeir, Daniel Schmidt, Geoffrey Webb, and Germain Forestier. Smooth perturbations for time series adversarial attacks. In João Gama, Tianrui Li, Yang Yu, Enhong Chen, Yu Zheng, and Fei Teng (eds.), Advances in Knowledge Discovery and Data Mining, pp. 485-496, Cham, 2022. Springer International Publishing. ISBN 978-3-031- 05933-9.
Kashif Rasul, Abdul-Saboor Sheikh, Ingmar Schuster, Urs Bergmann, and Roland Vollgraf. Multi-variate probabilistic time series forecasting via conditioned normalizing flows. CoRR, abs/2002.06103, 2020. URL UR..tpps::/aaxi..ott///a/b// 20100
Pradeep Rathore, Arghya Basak, Sri Harsha Nistala, and Venkataramana Runkana. Untargeted, targeted and universal adversarial attacks and defenses on time series. CoRR, abs/2101.05639, 2021.
org http://wwwwr.aaa/////////// 05639911..66699
UCI Machine Learning Repository. Individual household electric power consumption data
set. bttts::/arccive..ie uci http://t.a...///aaaaaaa///aaa/aa
2lectrrictpowert-coosuupp on| 2012.
David Rumelhart and James McClelland. Learning Internal Representations by Error Propa-
gation, pp. 318-362. 1987a.
David Rumelhart and James L. McClelland. Learning Internal Representations by Error Propa- gation, pp. 318-362. 1987b.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1312,6/99 6999 2013.

---

Under review as a conference paper at ICLR 2023
Ruofeng Wen, Kari Torkkola, Balakrishnan Narayanaswamy, and Dhruv Madeka. A multi-horizon 0bs/1711.11053
quantile recurrent forecaster, 2017. URL httttttss//xxi..ott
Wenbo Yang, Jidong Yuan, Xiaokang Wang, and Peixiang Zhao. Tsadv: Black-box adver-
sarial attack on time series with local perturbations. Engineering Applications of Artificial Intelligence, 114:105218, 2022. ISSN 0952-1976.
doi:
2022. 105218. URL http://www...eeaee.cccccc/////aa $095219762003025
12


Reviews:
[{"review_id": "nelIJpjnlHb", "reviewer": "Reviewer_oRvB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose to study targeted attacks on deep learning methods for time series forecasting. They adapt existing approaches to attack trained neural networks for image classification and run a number of experiments.", "summary_of_the_review": "The paper appears to apply established techniques to a related problem. This contribution appears to be marginal in terms of novelty.", "strength_and_weaknesses": "The paper nicely summarizes various popular attack patterns for deep learning methods and uses them in a time series forecasting setup. Notably absent from the paper is a meaningful defense mechanism to a targeted attack.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "j0D2VWO82J", "reviewer": "Reviewer_6F4s", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper tackles the problem of adversarial attacks against Machine Learning (ML) methods for Time Series Forecasting (TSF). The paper rightly points out that there are few works in this domain: hence, the paper proposes an original formalization of “targeted attacks” against such methods, which also works for \"regression\" ML problems (typical in TSF, but rare in the 'traditional' computer vision setting envisioned in adversarial ML research). The proposed attacks are then empirically validated on two datasets (representing two diverse real-world settings) and the effectiveness of the attacks is validated via statistical tests.", "summary_of_the_review": "First, I appreciate the line of research tackled by this paper. It is true that there have been few papers that investigated “adversarial attacks against ML methods for time series forecasting”. Furthermore, it is also true that these methods are becoming increasingly popular in the real world, as they are being integrated into production systems. Therefore, it is also crucial to study this subject from a security viewpoint.\n\nUnfortunately I cannot recommend acceptance of this paper – at least not in its state, and definitely not for ICLR. Indeed, the paper is affected by several and critical weaknesses, which affect both its “security” focus and its technical rigour (required for ICLR). I will elaborate below on the many problems that affect the paper, and provide suggestions for improvement.\n\nBefore I do this, however, I must warn the authors that I will gladly engage in a discussion with them, but that I am also very unlikely to change my mind on the submission, as I believe the amount of effort required to put it into an “acceptable” state for ICLR is simply too much to address during the rebuttal phase.\n\n**Limited scope.** \nAccording to Section 3.1, the considered setting assumes the usage of Time Series Forecasting only for predicting the “next” value of a given time-series. Such a setting can be a bit simplistic as there are several cases in which the forecasting may entail the prediction of “multiple” future values. Albeit one can argue that predicting multiple values can be done by iteratively predicting a single one, it is unclear whether this scenario also falls within the threat model of the paper.\n\n**Vague definitions.** \nThe following “definition” of adversarial time series is not rigorous: “The adversarial time series $\\hat{X}$ ($X_{adv}$) is intended to significantly worsen the output prediction $\\hat{Y}$ of a TSF model.” Given the scope of the paper (let me quote from the introduction: “We define and formalize targeted attacks on deep learning time series forecasting.”), such “vagueness” is unacceptable. The same issue affects the definition of the attacker’s “goal”, which I quote: “The goal of the attacker is to create a targeted output impact on the time series.” What is a “targeted output impact”? Furthermore, I do not agree with this statement: “Properties of the perturbation. The performance of the model prediction would reduce as the strength of the perturbation increases.” It may be intuitive to assume that “larger” perturbations have a greater impact, but such a statement is not universal. Do the authors *require* their perturbations to have a stronger impact as their magnitude is increased? Finally, the following sentence is problematic: “This attributes to the cost of data manipulation. It is notable that it is more expensive to achieve higher input perturbation”: first, there is no mentioning of this “cost”, thereby increasing the vagueness; second, because “greater perturbation = greater cost” is also not universally true (see [A]). Put simply, the authors attempt to formalize adversarial attacks on TSF, but the provided definitions are vague and the formalization – which appears to be universal – is limited to very few specific cases (which are not even properly defined). \n\n**Extremely Powerful Attacker.** \nThe proposed threat model envisions an extremely powerful attacker, who has white-box knowledge of the targeted ML model. This is highly unrealistic in the real world; furthermore, although the authors claim that “the attacks work even in black-box settings”, the way in which such attacks work is by iteratively querying the targeted ML model---thereby assuming an attacker who has “oracle access” to the ML model. Both scenarios are highly unlikely in real world settings (see, e.g., [A]), especially because the attacker – on top of knowing/being able to query the targeted ML model – also needs to introduce the perturbations (i.e., they must be able to manipulate the input data stream analyzed by the ML model). Simply put, from a security standpoint, the considered “threat model” is unrealistic and I am not surprised in the slightest that the attacks “work”. I invite the authors to provide some specific use cases of the proposed threat model, which should be provided with concrete justifications as to why an attacker would invest so much effort in setting-up, and then launching, a similar offense. Otherwise the authors should remove these considerations and simply state that the proposed “contribution” is devoted to assessing the robustness of ML models for TSF to adversarial examples (which are not necessarily malicious). I point the authors to a recent work that summarizes all these issues, [A]\n\n\n**The definition of “temporal targeted attacks” (TTA) is flawed.**\nAccording to Section 4, these attacks are defined as follows: “In a Temporal Targeted Attack (TTA), the attacker crafts the attack such that a minor perturbation in the input causes a specific time region in the output to be manipulated. In the given region, the attacker would want to change the direction of the output (DTA) or the amplitude of the output (ATA) in a specific time window $(t_1, t_2)$.”. This definition is in stark contrast with the definition of “time series forecasting” provided in Section 3, which I quote: “Given a time series X = [$x_1$, $x_2$, … $x_T$], a time series forecasting task predicts the value of $x_{T+1}$ based on the previous samples [$x_{T−w}, x_{T−w+1}, ..., x_{T}$], where $w$ is the window size under consideration.” Indeed, if the goal of time series forecasting is predicting $x_{T+1}$, then how can the attacker influence the output in a specific time window $(t_1, t_2)$? This is only possible if $t_1=t_2$, which means that the window is not a window in the first place. Therefore, either one of the following is true: the definition of time series forecasting is flawed, or TTA are redundant.\n\n\n**No (realistic) justification for the Experimental Evaluation.**\nThis is a follow-up issue to the “extremely powerful attacker” described above. Let me quote from Section 5: “Stock prediction is a high-demand, high-impact and high-risk area in the financial markets. Stock market prediction can have a direct impact on individuals, organizations, and nation as a whole, thus inaccurate predictions are seen as high-impactful and high-risky. However, these models are also vulnerable to attackers, where one can launch an adversarial attack to make the necessary predictions.” Specifically, “how can an attacker launch an adversarial attack against ML models tasked to predict stock market values?” According to the above, the attacker must be able to manipulate the data, and either know the targeted ML model (which is an IP of a company), or be able to query such ML model. How is this realistically feasible? Reality is far more complex and my impression is that the paper is being oversold (from a security perspective), thereby increasing the confusion (endemic among practitioners) about the real threat of adversarial attacks against ML models [A]. (Note that the same comment applies also for the “energy prediction” scenario: why, and how, would an attacker attempt to do so?)\n\n**Biased experiments.**\nFor the evaluation, the authors adopt some parameter settings without properly motivating them. Why was the Google Stock Prices split into train:test with a 4:1 split? Why were the time windows set to 5? Maybe there is a logic, but as it is not described in the paper, I am inclined to believe that the authors arbitrarily “cherry picked” the values that confirmed their attacks to be successful.\n\n**Unclear statistical validation.**\nIt is unclear how the KS test was carried out. Table 1 shows the resulting p-value of the many comparisons performed in the paper. However, it is not clear what was actually being compared: did the authors simply aggregate the overall results obtained by a (single) ML model on the the (single) train:test split with a (single) parameter configuration? In other words, how large is the size of the “populations” that are being compared? My guess is that the authors performed such tests by considering a single “configuration”, which is not fair and definitely not enough to support any claim about the effectiveness of the attacks. I invite the authors to consider diverse configurations (e.g., different splits, different window sizes), and then use all such diverse configurations to perform a (more transparent) statistical test by also adopting appropriate “correction” techniques (e.g., Bonferroni Correction). Doing so would tremendously help supporting the effectiveness claims made in the paper!\n\nSome additional issues:\n\n•\tThere is no buildup for Section 3.2. \n\n•\tThis sentence makes no sense (Section 3.2): “Corresponding to the above distance, a p-value is calculated. Given a significance value α. the null hypothesis that the sample follows the reference distribution is acceptable, if the p-value is acceptable if it is greater than the significance value α.”\n\n•\tThe quality of writing in Section 4 is very low\n\n•\tI am inclined to believe that the proposed “ATA” are just a subset of “DTA”. A more rigorous definition is necessary\n\n•\tThe following statement is wrong: “The FGSM attack Goodfellow et al. (2014a) forms the basis of any known adversarial attack techniques.” There are a plethora of “adversarial attacks” wherein the adversarial examples are created by means of techniques that have nothing to do with FGSM. I invite the authors to avoid making such statements.\n\n•\tThe paper should make an effort to cite some existing works that proposed adversarial attacks against TSF for the two considered use cases (a quick search on Google Scholar would reveal several papers that evaluated their attacks on such data).\n\n•\tThe proposed attack is evaluated only on \"univariate\" time series. Will it work also on multivariate time series? Why has this setting not been considered in the paper?\n\n\nEXTERNAL REFERENCES\n\n[A]: Apruzzese, G., Laskov, P., de Oca, E. M., Mallouli, W., Rapa, L. B., Grammatopoulos, A. V., & Franco, F. D. (2022). The Role of Machine Learning in Cybersecurity. Digital Threats: Research and Practice.", "strength_and_weaknesses": "STRENGTHS:\n+ Trendy topic that has been not very explored\n+ The statistical validation is interesting\n\n\nWEAKNESSES\n- Very poor presentation\n- Lack of scientific rigour (in clarity and quality)\n- Unrealistic assumptions\n- Biased experiments\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "JMz94CbTYvR", "reviewer": "Reviewer_X1J3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the targeted attack on time-series forecasting tasks. Specifically, this paper proposes three different targeted attack goals: directional attack, amplitudinal attack, and temporal attack. This paper then designs the corresponding FGSM, PGD, and APGD attacks for these goals. All attacks are evaluated on the google stock dataset and household electric power dataset, and the evaluation results show that the proposed attacks achieve adversarial attack goals.", "summary_of_the_review": "Thanks for the paper, I thoroughly enjoyed reading the paper. My concern about the paper is the effectiveness of the targeted attacks.", "strength_and_weaknesses": "Strength: \n\n+ Clean writing: The paper is well-written and easy to read.\n\n+ Novel method: The paper proposes targeted adversarial attacks against time-series forecasting tasks based on three different attack goals: directional attack, amplitudinal attack, and temporal attack. \n\n+ Empirical experiment support: The paper evaluates the proposed methods on two time-series datasets and shows that the proposed attack method achieves the attack goals.\n\nWeakness:\n\n+ Lack of methodology details: I think the most interesting part of the paper is the different targeted attack goals and their corresponding strategies to achieve these goals. In equation 3, amplitudinal and temporal attack needs $lim$ and $att$. Although there is a discussion before Section 4.1 on these two functions, I am still not quite clear about these functions are defined. It will be great to describe these two functions more formally to avoid unnecessary confusion.\n\n+ The effectiveness of the attack: In Figure1, if we compare the delta between the original, predicted, and attacked predicted values, we can find that the delta is always less than $0.1$. Given the $\\epsilon=0.1$, a $0.1$ change in predicted values can be considered expected. For example, for a DTA-up attack, if we simply increase the input value by $0.1$, we may also be able to increase the predicted value. This change makes me feel the adversarial attack is not effective enough. It will be good to visualize the perturbations and the adversarial examples so that readers can better understand how adversarial perturbations cause unexpected changes in the forecasting results.\n\n+ Also, where are the red lines (PGD) and green lines (FGSM) in Figure 1? And why is only the DTA-down attack with $\\epsilon = 0.5$? What’s the performance of the DTA-down attack with $\\epsilon =0.1$? Based on the value on the y-axis, $0.5$ is a very large perturbation.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "GxA_4Z25sA", "reviewer": "Reviewer_zM1v", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper extends targeted adversarial attacks to time series forecasting, and performs a statistical evaluation to demonstrate the effectiveness of the methods.", "summary_of_the_review": "Because of serious concerns about novelty and originality, I can not recommend acceptance.", "strength_and_weaknesses": "The paper is clear and the experiments appear sound. The use of proper statistical tests is appreciated. However, I have serious concerns about the originality and novelty of the contribution, because the paper overlooks directly related work on targeted attacks for time series forecasting. Besides, Section 4 (4 -Targeted Time Series Attacks) looks more like a review of existing attacks rather than a proper contribution, though I acknowledge that the APGD algorithm has been adapted.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject