MIA: A Framework for Certified Robustness of Time-Series Classification and Forecasting Against Temporally-Localized Perturbations
Authors: Ruoxin Chen
Ruizhe Zhong
Jiawei Sun
Jie LI
Chentao Wu
Junchi Yan
Venue: Submitted to ICLR 2023 (2023)
Keywords: Certified robustness
time series forecasting
time series classification

Abstract: Recent literature demonstrates that times-series forecasting/classification are sensitive to input perturbations. However, the defenses for time-series models are relatively under-explored. In this paper, we propose \textbf{M}asking \textbf{I}mputing \textbf{A}ggregation (MIA), a plug-and-play framework to provide an arbitrary deterministic time-series model with certified robustness against temporally-localized perturbations (also known as $\ell_0$-norm localized perturbations), which is to our knowledge the first $\ell_0$-norm defense for time-series models. Our main insight is to let an occluding mask move across the input series, guaranteeing that, for an arbitrary localized perturbation there must exist at least a mask that completely remove out the perturbation, so that our prediction on this masked series is uninfluenced. Remarkably, MIA is high-availability as it still works even if we only have query access to the pretrained model. Furthermore, as there is no dedicated defense against $\ell_0$-norm perturbations for time-series models, we specifically adapt two matrix-based defenses to time-series models for comparison. Extensive experiments show that MIA yields stronger robustness as well as practicality.

Under review as a conference paper at ICLR 2023
MIA: A FRAMEWORK FOR CERTIFIABLY ROBUST TIME- SERIES CLASSIFICATION AND FORECASTING AGAINST TEMPORALLY-LOCALIZD PERTURBATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Recent literature demonstrates that times-series forccasing/classifcattinn are sen- sitive to input perturbations. However, the defenses for time-series models are rel- atively under-explored. In this paper, we propose Masking Imputing Aggregation (MIA), plug-and-pp -lay framework to provide arbitrary deterministic time- series model with certified robustness against temporally-localzee (also known fo-norm localized perturbations), which is our knowledge the first no-momm defense for time-series models. Our main insight is to let an occluding mask move across the input series, guarantecing that, for an arbitrary localized per- turbation there must exist at least one mask that completely occlude the perturbed area, so that the prediction on this masked series is certifiably unaffected. MIA is flexible as it still works even if we only have the query access to the pretrained model. To further validate the superior effectiveness of MIA, we specifically com- pare MIA to two baselines extended from prior randomized smoothing approaches. Extensive experiments show that MIA yields stronger robustness.
perturbations
INTRODUCTION
Time series forecasting/Clasiffcttinn (TSF/TSC) have been widely applied help businesses make informed decisions and plans (Miyato et al. 2017 2018). However, wide range of literature demonstrate that time-series models are vulnerable to adversarial input perturbations (Connor et al.
Zhou et al., 2019 SChleeg et al. 2019 Park et al.)
1994 Gelper et al. 2010 Ding et al.
2022 Yang
cct al. 2020 Dang-Nhu et al. 2020 Oregi et al. 2018 Han et al. 2020). c.g.. an elaborately designed
imperceptible perturbation could control the prediction (Karim al. 2020 Fawaz et al. 2019). So far related literature mainly focusing on detecting the outliers (Ruf et al. 2018 Yairi et al. 2017).
the adversarial robustness of time-series models is relatively under-explored, especially to -orm robustness, e.g., (Yoon et al. 2022) only explore the f2-norm adversarial robustness for probabilistic forecasting models. In the present work, focus the robustness against remporally-localized perturbations, as we notice there already exists corresponding powerful attacks (Yang et al.) 2022).
Generally, defenses can be divided into two types, heuristic defenses and certified defenses. Heuristic defense can yield better empirical robustness but lack robustness guarantees. From the experience on image classification (Athalye al., 2018 Carlini & Wagner 2017 Athalye & Carlini 2018), the heuristic defenses would be useless when confronted with the newly designed adaptive attacks, e.g., Athalye et (2018) leverage Backward Pass Differentiable Approximation technique to successfully circumvent almost all the heuristic defenses at that time. To end such a "cat and mouse" game between the adaptive attacks and the heuristic defenses, the concept of certified defense proposed, with unbreakable robustness certificates.
Current certified defenses can produce robustness certificates but often require the user to retrain the base model from scratch, e.g., Yoon al. (2022); Li et al. (2020); Cohen et al. (019) retrain the base model as these defenses do perform poorly on naturally-traine
models. The requirement for retraining could bring additional challenges when it comes to the real-world deployments. In addition, the certified defenses on sequence- based data are quite under-explored, since almost all the certified defenses focusing matrix-based data (e.g. image).

---

Under review as a conference paper at ICLR 2023
Classify
Class
Class
Rewet an Agreement
Mas, M!
Mask
Impute
Classify
Agpregate
Aiste
Fast 2 MUU
M!!))
Exist dissgreer
Abstain
Overview of MIA pipeline. Inputted a series N1:to MIA first masks different periods of Xi:ta to construct the masked series Xi:ta @ (()), k = O....,M. M. Then MIA imputes the masked series with the imputation model G(.). We classify the imputed series with the pretrained model. If the predictions of all the imputed series are Class 0, MIA will return Class 0 with the robustness guarantee that the output is clean, otherwise MIA will return Abstain.
Figure| 1:
To address these issues, in this paper, we propose Masking Imputing Aggregation (MIA), flexible framework to arm an arbitrary TSFTSSC deterministic model with robustness certificates against temporally-localize perturbations. Different from the requirement for retraining in prior defenses, MIA only an imputation model for recovering the masked areas, which can be easily learned in an unsupervised setting. Specifically, MIA works follows: 1) masking: MIA first masked series via sliding a mask through the input series; 2) imputing: MIA imputes the masked series with the impu- tation model; 3) aggregation (checking agreement): MIA only returns the the class if the pretrained model outputs the same for all the imputed series, otherwise returns Abstain. With the above three steps, can guarantee that all the predictions from MIA is clean. Furthermore, to two baselines extended from randomized smoothing, as we notice that randomized smoothing has achieved a widespread success in defending different adversarial attacks. The contributions are:
compare MIA
1) We propose MIA, a plug-and-play framework to arm an arbitrary TSFTSS model with certified robustness against comporally-localizer perturbations, which is our knowledge the first norm certified defense in time series domain.
2) We propose randomized masked training, specialized training algorithm for training the impu- tation model of MIA, to further boost the performance of MIA.
3) We compare MIA to two baseline methods comprehensively on three aspects. 1) robustness: extensive experiments on different datasets validate that superior robustness of MIA. 2) Practicality: MIA stronger as it is plug-and-play and not require retraining. 3) Inference cost: the inference time of MIA is comparable to the time cost of two baselines.
2 RELATED WORK
Heuristic defenses for time-series models. Prior works on robust TSF/TSC can be divided into two general categories: outlier detection and deep learning. The former is to filter the outliers in statistical way, including M-Means clustering (Yang al. (017). one-class SVM clustering (Schölkopf
al. 2001). Kalman filters (de Bézenac et al. 2020) and support vector data description (Tax &
Duin 2004). The latter leverages the strong representation ability of neural networks to recover the perturbed series, including robust feature-based approaches. (Guo et al. 2016 Yang Fan Xu et al. 2018 Schlegl et |l.]| 2019), GNN based methods (Zhao al.) 2020 Deng Hooi (021). }), association discrepancy (Xu et al.) LSTM-based methods (Hundman et al. 2018 Tariq et al.) 2019). However, these empirical methods lack robustness guarantees, hinting that they would be meaningless once new adaptive attack is found. For that reason, certified defenses are crucial because their mathematical robustness certificates are permanently unbreakable.
2022
reconstruction-basse methods (Li et al. 2021 2019
2022
Certified adversarial defenses. In the field of image classification, there has been much work on the certified defenses, including randomized smoothing (Cohen et al. 22019] Salman et al.) 2020). convex polytope (Wong& & Kolter 2018), CROWN-IBE et al., 20. Among them, the to norm defenses include derandomized smoothing (Levine & Feizi 2020a), randomized ablation (Levine & Feizi 2020b Zhang al. 2020) and series of bask-based
(Zhang et al. 2019) and Lipschitz bounding (Cisse
2

---

Under review as a conference paper at ICLR 2023
defenses (Xiang & Mittal 2021 McCoyd et al., 2020 Han al. 2021 Xiang et al. 2021| 2022 In stark contrast, the certified defenses for time-series data are rarely explored. To our knowledge, Yoon et al. 2022) and (Li et al., 2020) are the only two defenses that produce nn-m robustness certificates, but a common downside is that they both additionally require retraining the base model over Gaussian augmented samples, which imposes a large amount of additional training costs.
3 PRELIMINARIES
Time series classification (TSC), The time series classification modeled as: inputted a to ty-length series (denoted by X1:fo [x1, .2....,, Tt,l), TSC model returns a class f(-) X1:to - y/-
Time series forecasting (TSF). Given the "past observations" Xi:to the forecasting model returns the "future values" f(-) Xi:to + too In this paper we mainly focus on the classic and commonly studied short-term forecasting setting (Ke et al.) 2017). which is to forecast a single time point f(-) Xi:to + I (not necessarily the next point Fto+1). The short-term forecasting problem is sufficiently representative as the problem of long-term forecasting f((1:to) + Xto +ofor can be decomposed into short-term forecasting subproblems, in which the i-th (i = -1.....?) forecaster predicts the (tu + i)-th time point. We discuss the multivariate tasks later in this paper.
perturbation 6 I:taa Laavl) In lemporally-localized per- turbation attack, the adversary is allowed to perturb arbitrary subseries w.r.f. the given ly-Rorm constraint. Let Ladv be the Ey- norm constraint on the localized perturbation. We can formulate all the perturbed series w.r.t. the Eg fo-morm constraint as follows:
Definition (Temporally-localiz)
X1:fo + 8 (add +
X1:to + [0,..., O, dt Stadv + Luai: O, .,....O
_Ixi,..., Itad--1 + 8taevt1, + Statu+hane .... Dtol
Perturbed subseries
where unbold &, refers to the single perturbation value added to the 1-th time point. ladv + 1 and tadv + Lady refer the starting point and the ending point of the perturbation respectively, which explicitly restricts its to norm as tIw Il., = (fadv + Ladv) (tadv + +1)+1= Ladv- Significance of cemporally-locaizee perturbation. Temporally-localzzr perturbation is espe- cially representative in real-world scenarios. Temporally-Iocalized perturbation can represent short- term volatility and local anomaly, both of which can be regarded the normal data added with semporally-localize perturbation. The resistance to short-term volatility is important in long-term foreccstting/predittoo,, in which the long-term value is considered unaffected by the short-term volatility. A typical example is the well-known investment philosophy, "Value Investing" (Piotroski) 2000), where the "intrinsic value" of business is considered to be robust against short-term volatil- ity. Moreover, the detection of local anomaly is practically useful in real-world scenarios. For instance, detecting a subsequent time interval of abnormal heart rate in electronic health records is a problem of local anomaly detection. We can also adopt the method of detecting cemporally-localzze perturbation for detecting the abnormal network traffic for loT Time-Series Data. Furthermore, to highlight the risk of cemporally-oocalizer perturbations, we empirically show how much a fo-norm perturbation can change the output of an undefended forecaster in Appendix. We also compare the attacking performance of fo-norm perturbation to lo-norm perturbation, and the empirical results suggest that forecasting models might be more sensitive to fy-nrrm perturbations.
PROPOSED FRAMEWORK: MASKING IMPUTING AGGREGATION
4.1 PIPELINE OVERVIEW
MIA includes three steps: 1) masking; 2) imputing; 3) aggregation (checking agreement).
1. Masking, We denote a mask by M Meeee. where X1:to @ M[ is replacing the values of X among X1:to with zeros. Let Temast be the size of the mask. Inputted series X1:t, and the to norm of the temporally-tocalized perturbation Ladv. we slide the mask thorough the input series with the step size a Lmaas Ladv + 1, and then obtain the following masked series
X1:ta ② MiI+ka Lmmar + to))) k 0, ..., [(to ILmast)
(2)
where a Linas Ladv +
[[c] returns the smallest integer larger than equal
3

---

Under review as a conference paper at ICLR 2023
Algorithm 1: Algorithm of Masking Imputing Aggregation.
Input:
The pretrained TSF/TSC model f()), the imputation model G(-), the input series X1:to> the mask size Temaske the length of lemporally-tocalizee perturbation Ludv, the discretization parameter for TSF task.
Compute the step size of masking
maskk Ladv
Generate the masked series via sliding the Lmask-Size mask
X1:to 3 M +ka)) k = 0,... ....: (4o Lmaak) )/
Utilize the imputation model impute the masked series
Xido (X @ Motkkommiatto Lmmss
(k)
Compute the output (denoted by (8)) for each imputed series, as follows:
for TSC task
y (k)
fais (x 1:10
f-ir())orr TSF task
# fais (k) if (0) ())
(x1:to] is computed as Eq.
5p:
... (M) then
Output: (
else
Output: Abstain.
We set the step size Temask Tradv for guaranteeing all the temporally-localized perturbations of Ladv can be covered. min(Tmmane + ko, (n) is to prevent the mask from exceeding 40- 2. Imputing. Our second step is to recover the masked values with the imputation model G(-):
(k)
lite G(x1:to @ Mm +ka)) k = 1,..., [(to /mmas)/aa]
(3)
This step is to make Xitto approximate the normal time series, so that the pretrained model could perform similarly on these imputed series. We discuss G(-) later this section.
Aggregation (Checking Agreement). We input the imputed series 11to into the pretrained model
f()). If the pretrained model's ouputs on all Xi:fo reach agreement unanimously, MIA classifier SMIA (x1=)) will output this unanimously approved label/prediction, otherwise output Abstain to alert that the input series might have been attacked by the temporally-localizes perturration...
[ 1 f(x = =... = =f (x ([((oo-mmmkk/aaa))
fMIA (X1:0)
Abstain
Otherwise
Discretization technique for MIA on TSF. We note that TSF models are impossible to forecast exactly the identical value on different series, so that MIA would output Abstain all the time on TSF. To address this, we substitute the original pretrained forecaster f(-) with its discretized version fais()) in Eq. (4). where fais (A) ), ... [(to Lmmas)/aa compute as follow:
(5) where A is discretization parameter that controls the trade-off between the discretization error and the success rate achieving agreement. As decreases, the discretized forecasts retain more information from the original forecasts while the agreement rate decreases. If take A 0.5,
fais (x (K) (x
= A [+x(x())) //A
(8) faie (x) is to round up the value of fais (x to the nearest integer.
4.2 DISCUSSION ON THE MASK SIZE Lmask-
The only requirement of Masking (Step 1) is to ensure for arbitrary temporally-localized perturbation of Leady, there always exists a mask to occlude that perturbation. Thus prerequisite is Truack 2 Ludv. We can control the trade-off between the the imputation quality and the inference cost with Lmask. As we increase Limask the imputation quality will decrease since the number of missing values increases. Meanwhile, the number of masked series decreases subsequently, so the inference cost reduced. In the extreme case where Lmack = to where the masked series are all equal 0 O2:t, MIA always outputs (G((1.))) )) regardless of the input series. The imputation quality extremely poor and the inference cost is the smallest. The practical implementation of MIA is showed in A Algorithm]]
4

---

Under review as a conference paper at ICLR 2023
Remark (MIA on Probabilistic Models). We notice line of time-series forecasting models probabilistic (e.g., DeepAR (Salinas et al. 2020),, which models the forecasted value f(11:t)) as a
random distribution g(y/
X1:to] rather than single value, follows:
(6) The exact forecasting value of probabilistie models is inaccessible (prior works perform Monte-Carlo inference for approximation). which makes applying MIA to probabilistic models challenging. Al- though we can utilize Clopper-Pearee method (Clopper & Pearson 1934) to estimate the discretized
f(x = : Ni.aol [
forecasts fais (X1:to) with confidence level, the inference cost would be expensive for confidence interval estimation.
4.3 ROBUSTNESS CERTIFICATE OF MIA
Proposition 1 (Robustness Certificate of MIA). The forecast/label (not Abstain) returned by Algo- rithm cannot be changed by any temporally-localize perturbation whose Ey norm is no larger than Ladv (see proof in Appendix).
Remark 2 (Robustness Certificate). The robustness certificate is for ( rather than (x1:t)
because is almost impossible to derive the certificate for pretrained model without any require- ment. Our aggregation does not allow any tolerance because the certificate would not hold once a disagreement allowed. Note that, with Masking (Step 1), we can guarantee there exists masked series that is unaffected, and all other masked series retain the perturbed area. If we allow a dis- agreer, the ensemble prediction would be totally under the adversary's 's control, because all except one masked series are perturbed (the only one not affected would become the disagreer). We point out that the certificate also holds for multivariable TSC/TSF. We can easily apply MIA to multivariable tasks through repeating Masking (Step 1) and Imputing (Step 2) on each variable.
4.4 TRAINING IMPUTATION MODEL G(-)
The performance MIA highly depends on the imputation model G(.). We notice that there already exists much work on time series imputation (Cao et al. 2018 Du et al. 2022, Moritz & Bartz- Beiclstein 2017 Fortuin et al., 2020; Cao et al. 2018 Luo et al. 2019; Yozgatigil et al. 2013). However, all these imputation models aim to recover the discrete missing values, which not want. To train an imputation model to recover consecutive missing values, we propose randomized masked training algorithm,, which minimizes the MSE loss the masked noisy series, as follows:
| (2)
C+1
(G + 0(2:ta)) @ Mp Bolnnkkkaa/ammmmaaaaaaaa kha X1:to (7)
where C = [(to- Lmmas))/a and 01::0)) -N o?) is a Gaussian noise series, of which each entry is i..dd sampled from Gaussian distribution. We specifically add Gaussian noise is to make the imputation model robust to the random noise and avoid overfitting, since prior works (Foster et al., 1992 Passalis et al., [222 Hwang et al. 1998) show the time series data is generally noisy. We emphasize that we do not add any noise in inference stage.
4.5 COMPARISON TO RANDOMIZED SMOOTHING DEFENSES
Randomized smoothing (Cohen et al., 2019) is a well-know model-agnostic method in the field of certified defenses, which has been applied defend various types attacks and achieves superior certified robustness in their respective fields. Comparing MIA to randomized smoothing can better demonstrate the advance of our method. We extend two image-specific randomized smoothing defenses, Derandomized Smoothing (Levine & Feizi 2020a) and Randomized Ablation (Levine & Feizi 2020b> to the time series domain, as the basclines.
Derandomized smoothing for time-series models. In the time-series version of DS, given a time series X1: and the base classifier f(-), DS (denoted by fus) classifies as follow
(xs((1:a) = arg max yE>
Σ
[x = y}
Xeub ESub(x1..))))
2W notice that a recent work (Yoon et al. but our definitions of robustness are different. Yoon et al. (2022) bounds the local Lipschitz constant, while objective is much stricter, aiming guarantee the forecast is invariant under the perturbation. (T{) } is the indicator function.
2022 derives robustness certificate for probabilistic forecasters,
5

---

Under review as a conference paper at ICLR 2023
Table 1: (DistalPhalanxTW) Comparison among three defenses on a TSC dataset.
Defense
%
2 %
3 %
%
6 %
7
%
%
10 %
MIA (Amaok = 10%)
67.3%
66.3%
67.3%
67.3%
66.3%
64.4%
66.3%
64.4%
64.4%
63.4%
DS (n 10%)
28.1%
28.1%
28.1%
28.1%
28.1%
28.1%
28.1%
28.1%
28.1%
28.1%
RA (17 10%)
5.8%
5.8%
5.8%
5.8%
5.8%
5.8%
5.8%
5.8%
5.8%
5.8%
MIA (Lmook = 15%)
62.4%
65.3%
65.3%
64.4%
64.4%
62.4%
63.4%
65.3%
63.4%
64.4%
DS (1- 15%)
30.2%
30.2%
30.2%
30.2%
30.2%
30.2%
30.2%
30.2%
30.2%
30.2%
RA 15%)
9,4%
9,4%
9.4%
9.4%
9.4%
9,4%
9.4%
9.4%
9.4%
9.4%
We first let the base classifier make predictions on these subsequences, and then SDS (Xi:to) outputs the majority label. The prediction is robust if
where Sub(X1:t, 7) consists of the subsequences X1:7, x,+1:22 X23+1:33
Xto Noo-
(x = 5)
max y79
Σ
[Xf(b))b) = y) > 2(1 + Ladv 1)
(9)
Xeub (1:to)))))))))
Neut (Su
Randomized ablation for time-series models. RA (denoted by SnA classifies as follows:
(x1.))) arg max yEy
Pr Xeub X Samplee(Nito
[f vl
(10)
where Naub ~ RA 17) is to randomly sample 17 time points without replacement to construct the subseries Xaub. and ablate all other points. fRA (x1:0) returns the label that f(-) is most likely to classify Xsub as. ÿ = fRA((N:s (Xisto) is robust if
(1D Lod-add))
Pr Xealy Sample-a (1)))))))
(Xsub) = 0 > 2
(11)
80
Comparison to DS and RA. We note that the pretrained models of DS and RA make predictions on subseries (Xxub) instead of normal series. Since the data distribution of the subseries are fundamentally different from the normal data, we can expect that these two defenses would perform poorly on the naturally-trained models. Therefore, we need train the base classifiers from scratch on the subseries. In stark contrast, MIA is aodg-and-ply framework that can be directly applied to TSF/TSC pretrained models. In MIA, the main cost of training stage preparing the imputation model. We point out that the imputation model of MIA can be trained in an unsupervised manner, saving us from labeling the data. Furthermore, we empirically show that MIA attains a significantly better robustness than DS and RA in Section
5 EXPERIMENTS
Experimental setup. We We MIA on both TSC and TSF datasets. TSFincludes Exchange Rate, Traffic and UCI Electricity (Alexandrov et al. 2019)) and TSC datasets include PistalPhalanxTW. MiddlePhalanxTT and ProximaIPhalanx (smail Fawaz et al. 2019a). We use MLP-Mixer (Tol- stikhin et al.) 2021), MLP and LSTM (Hochreiter & Schmidhuber 1997) as the pretrained model. Our experiments are conducted on the clean trainsets, following the common setting of certified adversarial defenses (Yoon al.] 2022 Li et., 2020 Cohen al. 2019 Chiang et al. 2020 Zhhng let al., (019)). Unless otherwise specified, We use MLP-Mixer as the base model for MIA, DS and RA, and A- 1.5. The experiments are conducted on CPU (16 Intel(R) Xeon(R) Gold 5222 CPU @ 3.80GHz) and GPU (one NVIDIA RTX 2080 Ti). More details are omitted to Appendix.
Evaluation metries. For TSC, we evaluate the defense by certified accuracy (CA) under the temporally-localized perturbation, which is defined by the fraction of the test samples that are correctly classified and certifiably robust to the perturbation. For TSC, we evaluate the defense by: forecasting rate (FR), mean square error (MSE) and mean absolute error (MAE FR is the fraction of the test samples on which MIA outputs the forecast instead of Abstain. MSE/MAE measures the mean square error/mean absolute error between MIA forecasts (Abstain are excluded) and groundtruth. We omit the evaluation on multivariate tasks to Appendix due to space limitations
5.1 COMPARISON TO PEER METHODS
Comparison on TSC. Table [ reports the certified accuracy of three methods in defending semporally-locaiizer perturbations. The pretrained/base model architectures of three defenses are all
We omit the evaluation of MAE Appendix.
'In our experiments, T 2% Teady = 2% or 17 = 2% refer to 2% to-

---

Under review as a conference paper at ICLR 2023
Table 2: (Exchange) Comparison among three certified defenses on TSF dataset.
Metric
Defense
2%
3%
4 %
5
7%
8%
9 %
10 %
MIA (Lmark 10%)
82.2
82.2
83.2
82.2
82.2
81.2
81.2
81.2
80.2
79.2
DS (1 = 10%)
24,8
24.8
24.8
24.8
24.8
24.8
24.8
24.8
24.8
24.8
FR (%)
(1) 10%)
16.8
16.8
16.8
16.8
16.8
16.8
16.8
16.8
16.8
MIA (fraast 15%)
64.4
71.3
69.3
69.3
71.3
68.3
69.3
71.3
62.4
65.3
DS (1- - 15%)
20.8
20.8
20.8
14.9
14.9
11.9
11.9
11.9
11.9
10.9
(1) 15%)
23.8
23.8
23.8
23.8
23.8
23.8
23.8
23.8
23.8
23.8
MIA (Lmast 10%)
0.143
0.141
0.145
0.141
0.141
0.139
0.137
0.137
0.135
0.134
DS (1) = 10%)
0,192
0.192
0.192
0.192
0.192
0.192
0.192
0.192
0.192
0.192
MSE
(1 10%)
0.202
0.202
0.202
0.202
0.202
0.202
0.202
0.202
0.202
0.202
MIA mest 15%)
0.126
0.134
0.130
0.132
0.137
0.132
0.129
0.130
0.123
0.125
DS (1) 15%)
0.144
0.144
0.144
0.065
0.065
0.069
0.069
0.069
0.069
0.060
(17 15%)
0.248
0.248
0.248
0.248
0.248
0.248
0.248
Table 3: Comparison of the inference time (millisecond) of three defenses on TSC datasets.
Model
FCN
MLP-Mixer
MLP
ResNet-18
Defense
2%
5%
10%
25h
5%
10%
2%
5%
10%
2%
5%
10%
MIA (mask 10%)
2.0
2.0
3.0
2.7
2.7
4.6
1.7
1.7
2.7
3.8
3.8
5.3
DS (1) = 10%)
0.6
0.6
0.6
2.0
2.0
2.0
0.3
0.3
0.3
2.6
2.6
2.6
RA (1) 10%)
25.2
25.2
25.2
259.7
259.7
259.7
0.8
0.8
0.8
130.3
130.3
130.3
MIA (Imask = 15%)
2.0
2.0
2.0
2.7
2.7
2.7
1.7
1.7
1.7
3.8
3.8
3.8
DS (1) = 15%)
0.6
0.6
0.6
1.9
1.9
1.9
0.3
0.3
0.3
2.6
2.6
2.6
RA (1 15%)
25.3
25.3
25.3
260.6
260.6
260.6
0.8
0.8
0.8
130.4
130.4
130.4
Table 4: (Traffic) The performance of MIA different pretrained models. (e) c2%) reports (MSE, FR%) of MIA. Baseline is MSE of the pretrained model without MIA. The lowest MSE and the highest FR for each pretrained model is shown in bold-face.
Model
Buseline
L.ooh
3%
5%
1.2
2%
10%
0.065
0U072 77.25
MLP Mixer
0.224
0,067 75.15
OUN8 7O.
0.019 80.2%
0 78
90.155
0,143 19.13
0.058 77.2%
0,009 76.2%
0,066 69.3%
0.079 80.2%
90,2%
0.075 76.21
0.343 91.1%
0,141 90.1%
6,139 85.1%
2%
0.007 68.77%
0,070 71.355
0L143 S0.156
GRU
0.243
5%
0,072 7.2.7%
0,070 04.355
0.075 76.
0 74:
0,145 91.1%
0,141 8S 136
10%
0.070) 74.3%
0,071 73.3%
0J069 63.4%
0.004 77.2%
0.074
77.2%
0J066 70L3%
0,145 91.1%
0.143 90.1%
0.137 86.1%
0J868 66.3%
0,071 76.
0,1SE 91.1%
LSTM
0.229
0,059 66.3%
65.55%
0
77.25%
SOL15%
0.147
0.070 65.855
0.871
66.3%
0.066 61.4%
0 77.25
77.2%
0J064 72.31%
0.150 SO.15E
90.1%%
0.149
0,064
72.35%
90.155
MLP
0.222
0.057 71.3%
550
72.3%
0.145
0.148 13.
0,066 70007%
0,00Y 70.3%
0,063 66.3%%
0.064 73.3%
0.064 72.5%
0.063 7003%
0.146 91.256
0.146 91.1%%
0,146 89.1%
2%
0,874 64.47%
0.087 75,
0.149 09. 156
ResNer18
0.248
5%
0.077
68.3%
0.077
05..37
0.088
79.2%
01089
76.2%
0,150 89, 1%1
0.846 87.1%
10%
0,077
07:35%
0,0788 67.5%
0J076 60.4%
0,086 70.27%
0.086 76.2%
0J083 73.3%
0.150 58.15
0,149 85.139 25
0,145 23.2%
MLP-Mixer. An interesting observation is that the certified accuracy of DS and RA keeps constant to different Teddy- The reason is that, the probability score of DS/RA models often concentrates on a single class, causing most classifications (including both (correct and wrong classifications) of DS and RA are of high robustness. The results show that the certified accuracy of MIA is more than twice of DR and RA across different Ladv. The reason is that the pretrained model of MIA classifies the masked series, while the base model in RS/DS classifies the subseries. MIA can attain higher certified accuracy because the masked series contains much more information (to- Lmask unmasked time points) than the subseries (17 sampled time points).
Comparison on TSF. Table repports FR and MSE of three defenses on Exchange. where the model predicts the next 30 values. Here we utilize discretization technique to make the TSF task feasible to DS and RA. The table shows that MIA offers a significantly higher FR than DS and RA, implying that MIA return the forecasting results much more frequently than other two defenses. The reason for the superior FR is same as TSC. We also observe that DS (17 15%) achieves a lower MSE than MIA at Lady 2 4%, which partially owing to its low FR. Since a low FR implies that the aggregation step of MIA filters a large portion of distrustful forecasts, reducing the difficulty of achieving lower MSE for the remained forecasts. Based on the results, MIA is better than other two defenses when we jointly consider FR and MSE
7

---

Under review as a conference paper at ICLR 2023
Table 5: (Electricity) (01 02% report (MSE FR%) of MIA on different pretrained models.
Boseline
9%
26
0,893 05
0994 63.4%
0.136 77.25
MI M-
0
55
0.005 49.3%
0,093
68.3%%
0,086 67.3%
01094 66.3%
0.134 78.2%
01154-8..5
01134
10%
0,055 07.075
0.095 07.5
0,089 55.4%
0.091 64,455
0,091 64,
0JASM 59.45%
0.134 77,215
0.134
77:23
66.3%%
216
0.009
02-17
0.08 101 175
0.138 68-35%
GRU
0.420
5%
0,300
64-455
0.101
63.4%
OLDE7 62.4%
0.087
62.4%
0.129 69.3%
0.139
69.3%
10%
0.102 65.3%
0,999 02
0.103 49 50%
0L08S 55L,455
OLUS7 01 4.%
0J086 53.5%
0.139 69.5%
0.139 69.3%
64.20
0:100 61:475
0,0R2 12.855
LSTM
0.438
0.100 66.3%
65.3%
55,4%
71.3%
0.342
OLORT
OLIP4
6.119.5.45%
216
0:100 75-3%
(J882 61.45
0.136 70.35%
MLP
0.402
5%G
0.108 97.2%
0.105 15.2%
OLDN 68.3%)
0J0KS 63.4%
0.131 97.2%
01133
0.130.76,25
10%
0,108
75.235
0,108 75.2%
ILO0K 60.4%%
067 01.458
0.087 61.455
0LI84 55.4%
0.331 77.2%
0.136
76.2%
0.108 66.3%
216
D.06/ 51.55%
0000 50, E76
0.136 06.35%
ResNe118
0.554
0.089 51 11.55
BJIRI $0.59%
CLORD 68L4%%
0.0A8 69.4%
03
68.3%
140
68.3%
10%
0.094 52.5%
000933 51.5%
0.892 43.6%%
0,079 56,455
GLU81 58:4%
0.076 54.5%
0.141 07.35%
0.136 6OL
61.4%
Table 6: Comparison of different training algorithms on TSC datasets.
Model
Training
5%
DistalPhalanss TW
10%
15%
5%
MiddlePttllinTTW TW
10%
15%
5%
ProsinsiPtalamsTT
10%
15%
5%
62.4%
52.5%
65.3%
Random
10%
4%
58.4%
53.5%
50.5%
73.3%
71.3%
MLP-Miner
15%
59.4%
59.4%
58.4%
54.5%
68.3%
67.3%
61.4%
5%
65.3%%
64.4%
72.3%
Masked
10%
66.3%
63.4%
67.3%%
60.4%
76.2%
74.3%
15%
64.4%
64.4%
60.4%
66.3%
62.4%
59.4%
76.2%
75.2%
74.3%
5%
63.4%
53,5%
68.3%
Random
10%
66.21
63.4%
52.5%
70.3%
67.3%%
PCN
15%
66.3%
65.3%
65.3%
57.4%
57.4%
56.4%
66.3%)
66.3%
65.3%
5%
70.3%
64.4%
75.2%
Masked
10%
70.3%
69.3%
66.3%
68.3%
73.3%
71.3%
15%
70.3%
67.3%
66.3%
68.3%
63.4%
63.4%
75.2%%
73.3%
73.3%
65.3%
10%
4%
60.44%
76.2%
73.3%
MLP
15%
63.4%
61.4%
61.4%
61.4%
55.4%
55.4%
71.3%
71.3%
68.3%
5%
64.4%
69.3%%
79.2%)
Masked
10%
65.3%
64.4%
70.3%
67.3%
79.2%)
78.2%
66.3%
63.4%
69.3%
66.3%
66.3%
78.2%
78.2%
77.2%
Random
10%
59.4%%
57.4%
57.4%
56, 36.4%
74.3%)
33%
ResNet-18
15%
4%
56.4%
55.4%
58.4%
58 4%%
58.4%
74.3%
73.5 33
66.3%
5%
65.3%
67.3%
78.2%
Masked
10%
66.3%
64.4%
63.4%
63.4%
78.2%
76.2%
64.4%
61.4%
59.4%
65.3%
62.4%
62.4%
78.25)
78.2%
78.2%
Comparison on inference time. Table compares the inference time of three defenses, which is averaged among three TSC datasets. We observe that the inference time of MIA is larger than DS, but significantly smaller than RA. Specifically, MIA's larger inference time than DS is owing to the cost in running the imputation model. RA's large inference time is for the confidence interval estimatior We also observe that the inference time of MIA increases with Ladve and decreases with Lasks because the number of masked series is [(to /Lmass)((Lmack. Ladv + 1)7 + 1. We omit the inference time analysis on TSF datasets to Appendix.
5.2 ANALYSIS OF MIA ON DIFFERENT PRETRAINED MODELS
Table and Table 5 report the performance of MIA different pretrained models, where the model forecasts the next 24 points. We observe that MIA (A = 1.0, 1.5) consistently lowers MSE as compared to that of the original pretrained models, suggesting MIA could also be effective plugin for performance improvement. Specifically, MIA improves the forecasting performance in the way of filtering these distrustful forecasts, sacrificing the availability (the decrease of FR) for lower MSE as well as certified robustness, which is common trade-off in the field of certified defenses (Cohen) al. 2019: Levine& & Feizi 2020a Liu et al.) [202T] Han et al. (022I). We can control the trade-off between MSE and FR by A, as the decrease of A can reduce MSE and FR.
5.3 ANALYSIS ON IMPUTATION MODEL OF MIA AND ABLATION STUDY
Impact of training algorithm. Table 6 compares our masked training random training, which trains the imputation model on the randomly masked series. Through extensive comparisons on different imputation model architectures and datasets, we convince that masked training consistently
o Following the official implementation of RA [eevine & Feizi 2020b), we take 100, 000 samples for the confidence interval estimation.
8

---

Under review as a conference paper at ICLR 2023
2::0
Aell
2-14
2.02
Forcartting farn (911
Curt Accuracy
0.3
0.00
0.00
1%1
Leam 196
Lran (85)
MSE Imask
(b) FR v.8 Lynack
CA Lynack
54
leak
10%
155
105
15%
6.5
00
23
05
1.0
3.5
2.0
25
35
6.5
1.0
1.5
2.5
25
30
(d) MSE v.S A
(c) FR V.S
A
Figure 2: Top: impact of Lmask on TSC dataset ProximalPhalanxTW Bottom: impact of A on TSF
dataset Traffic (Ladv 3%).
Table 7: Comparison of four imputation models. The best results are shown in bold-face.
Model
Lunk
Electricity 5%
10%
Exchange 5%
10%
2%
Traffic 5%
10%
87.1%
MLP Miser
5%
0.134 78.2%
0.134 78.2%
0,144 83.2%
0.138 81.2%
0.141 90.1%
0.143 89.1%
10%
0.134 77.2%
0.134 77.2%
0.116 66.3%
0.341 82.2%
0.141 52.25
0.134 79.2%
O. 91.1%
0.141 90.1%
0.139 88.1%
0.118 65.355
0.118 40.69%
0,142 87.195
BRITS
512
0.110
71.3%
0.899
60.4%
0.118 49.5%
0.00% 15
0.136 84./295
0.130 75.2%
10%
0.099
61.45%
0,100
60.4%
0.004 45.555
0.207 10.0%
0.277 1.005
0.000 0.053
0,128 70.20%
0.129 77.2%
0.123 73.3%
2%
0.101
58, 45%
0.095 25, 7%
0,120 67.5%
SAITS
5%
0J991
58.0%
0,090
62.7%
0.137 48.00%
0,301 47. 756
0,112 714.96
0.121 68
10%
0.104
56,456
0,103 64.5%
0.094 44.6%
0J07O 7,956
0.048 8.90%
0J001 4.0%%
0.125 07.39%
0.332 00.356
0.124 090.99
2%
OJIMS 52.5%
0.145 23.9%
0.130
68.395
Transfemner
5%
0.110 52.6%
0.113 42.6%
0,072 6,03%
0 2.095
0.127
74:595
0.123
0.096 60.455 45%
0,093
0,093
0.029 8.0
7:995
0.001 5.0%
0,124
75.2%
0.127 74.3%
0,123 61.475
outperforms random training for MIA by non-trivial gap. The gap becomes even larger at larger Ladv- The results suggest that masked training is suitable for MIA imputation model.
Impact of architecture of imputation models. Table 7 reports the performances of different imputation model architectures on MIA. Our results show that MLP-Mixer can offer higher FR and lower MSE simultaneously, suggesting MIP--Mxer is intrinsically more robust than other models.
Impact of Lmask. Fig. 2a 2b show: 1) MSE roughly keeps constant w.r.t. Lunakk because the discretization technique can diminish the difference between the forecasts that are close to each other. 2) FR decreases with the increase of Lmaak. because our imputation quality decreases with Lmaase making harder reach agreement unanimously. Fig. 2c show that the impact of Lmast on CA is not significant. Although the increase of Lyask reduces our imputation quality, it reduces the number masked series simultancously.
Impact of A. Fig. 2d 2c report the impact of A. As A increases, we observe that MSE and FR increase, validating our statement about A in Section 4.1
CONCLUSION
In this paper, we propose the first framework for time-series models to certifiably defend against fo-norm perturbations. Notably, MIA is plug-and-play defense, which can be easily applied to any TSF/TSC pretrained model. The only requirement of deploying MIA is to train an imputation model, which has been extensively explored in this work. Moreover, extensive experiments validate the effectiveness of MIA. We expect our work can inspire more studies on the 4n fn-norm robustness for time-series models. Interesting future works include applying MIA to probabilistic models.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Alexander Alexandrov, Konstantinos Benidis, Michael Bohlke-Schneider, Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Danielle Maddix, Syama Rangapuram, David Salinas, Jasper Schulz, et al. Gluonts: Probabilistic time series models in python. arXiv preprint arXiv: 05264, 2019.
Anish Athalye and Nicholas Carlini. On the robustness of the cvpr 2018 white-box adversarial example defenses. arXiv preprint arXiv:/80.003286, 2018.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent imputation for time series. In NeurIPS, 2018.
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pp. 3-14, 2017.
Ping-yeh Chiang, Michael Curry, Ahmed Abdelkader, Aounon Kumar, John Dickerson, and Tom Goldstein. Detection as regression: Certified object detection with median smoothing. NeurIPS, 2020.
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülççerre Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, 2014.
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In /CML, 2017.
Charles Clopper and Egon S Pearson. The use of confidence fiducial limits illustrated in the case of the binomial. Biometrika, 1934.
Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing. ICMI., 2019.
Jerome Connor, R Douglas Martin, and Les E Atlas. Recurrent neural networks and robust time series prediction. IEEE transactions neural networks, 1994.
Raphaël Dang-Nhu, Gagandeep Singh, Pavol Bielik, and Martin Vechev. Adversarial attacks on probabilistic autoregressive forecasting models. In ICML, 2020.
Emmanuel de Bézenac, Syama Sundar Rangapuram, Konstantinos Benidis, Michael Bohlke- Schneider, Richard Kurle, Lorenzo Stella, Hilaf Hasson, Patrick Gallinari, and Tim Januschowski. Normalizing kalman filters for multivariate time series analysis. In Neural Information Processing Systems, 2020.
Ailin Deng and Bryan Hooi. Graph neural network-based anomaly detection in multivariate time series. AAAI, 2021.
Daizong Ding, Mi Zhang, Yuanmin Huang, Xudong Pan, Fuli Feng, Erling Jiang, and Min Yang- Towards backdoor attack on deep learning based time series classification. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pp. 1274-1287. IEEE, 2022.
Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-basedd imputation for time series. arXiv preprint N2X::2202.0051666 2022.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. Adversarial attacks on deep neural networks for time series classification. In IJCNN, 2019.
Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. Gp- Deep probabilistic time series imputation. In International conference on artificial intelligence and statistics, 2020.
10

---

Under review as a conference paper at ICLR 2023
WR Foster, F Collopy, and LH Ungar. Neural network forecasting of short, noisy time series. Computers & chemical engineering, 1992.
Sarah Gelper, Roland Fried, and Christophe Croux. Robust forecasting with exponential and holt-
winters smoothing. Journal of forecasting, 2010.
Tian Guo, Zhao Xu, Xin Yao, Haifeng Chen, Karl Aberer, and Koichi Funaya. Robust online time series prediction with recurrent neural networks. In DSAA. leee, 2016.
Husheng Han, Kaidi Xu, Xing Hu, Xiaobing Chen, Ling Liang, Zidong Du, Qi Guo, Yanzhi Wang, and Yunji Chen. Scalecert: Scalable certified defense against adversarial patches with sparse superficial layers. In NeurlPS, 2021.
Xintian Han, Yuxuan Hu, Luca Foschini, Larry Chinitz, Lior Jankelson, and Rajesh Ranganath, Deep learning models for electrocardiograms are susceptible to adversarial attack. Nature medicine, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv: arXiv 2016.
arXiv preprint
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural compulation, 9(8):
1735-1780, 1997.
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Söderström. Detecting spacecraft anomalies using istms and nonparametric dynamic thresholding. KDD, 2018.
Jeng-Ren Hwang, Shyi-Ming Chen, and Chia-Hoang Lee. Handling forecasting problems using fuzzy time series. Fuzzy sets and systems, 1998.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. Deep learning for time series classification: review. Data Mining and Knowledge Discovery, 3374)99 2019a.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. Deep learning for time series classification: review. Data mining and knowledge discovery, 2019b.
Fazle Karim, Somshubra Majumdar, and Houshang Darabi. Adversarial attacks on time series. PAMI, 2020.
J. Ke, H. Zheng, Yang, and X. M. Chen. Short-term forecasting of passenger demand under on-demand ride services: A spatio-temporal deep learning approach. Transportation research part Emerging technologies, 2017.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization, 2014.
Alexander Levine and Soheil Feizi. (de) randomized smoothing for certifiable defense against patch attacks. In NewrlPS, 2020a.
Alexander Levine and Soheil Feizi. Robustness certificates for sparse adversarial attacks by random- ized ablation. AAAI, 2020b.
Dan Li, Dacheng Chen, Lei Shi, Baihong Jin, Jonathan Goh, and See-Kiong Ng. Mad-gan: Mul- tivariate anomaly detection for time series data with generative adversarial networks. In ICANN, 2019.
Hui Li, Yumpeng Cui, Shuo Wang, Juan Liu, Jinyuan Qin, and Yilin Yang. Multivariate financial time-series prediction with certified robustness. IEEE Access. 2020.
Zhihan Li. Youjian Zhao, Jiaqi Han, Ya Su, Rui Jiao, Xidao Wen, and Dan Pei. Multivariate time series anomaly detection and interpretation using hierarchical inter-metric and temporal embedding. KDD, 2021.
11

---

Under review as a conference paper at ICLR 2023
Hongbin Liu, Jinyuan Jia, and Neil Zhenqiang Gong. classification. In CVPR, 2021.
Pointguard: Provably robust 3d point cloud
Yonghong Luo, Ying Zhang, Xiangrui Cai, and Xiaojie Yuan. E2gan: End-to-end generative adver- sarial network for multivariate time series imputation. In Proceedings of the 28th international joint conference on artificial intelligence, 2019.
Michael McCoyd, Won Park, Steven Chen, Neil Shah, Ryan Roggenkemper, Minjune Hwang, Jason Xinyu Liu, and David Wagner. Minority reports defense: Defending against adversarial patches. In International Conference on Applied Cryptography and Network Security, 2020.
Takeru Miyato, Andrew M Dai, and Ian Goodfellow. Adversarial training methods for semi- supervised text classification. In ICLR, 2017.
Steffen Moritz and Thomas Bartz- Bertz-Beielsteim. imputets: time series missing value imputation J., pp. 2017
2017.
Izaskun Oregi, Javier Del Ser, Aritz Perez, and Jose A Lozano. Adversarial sample crafting for time series classification with elastic similarity measures. In International Symposium on Intelligent and Distributed Computing, pp. 26-39. Springer, 2018.
Daehyung Park, Yuuna Hoshi, and Charles C. Kemp. A multimodal anomaly detector for robot- assisted feeding using an Istm-based variational autoencoder. RA-L, 2018.
Nikolaos Passalis, Manos Kirtas, George Mourgias-Alexandris. George Dabos, Nikos Pleros, and Anastasios Tefas. Training noise-resilient recurrent photonic networks for financial time series analysis. In 2020 28th European Signal Processing Conference (EUSIPCO), 2021.
Joseph D Piotroski. Value investing: The use of historical financial statement information to separate winners from losers. Journal of Accounting Research, pp. 1-41, 2000.
Lukas Ruff, Nico Görnitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Robert A. Vandermeulen, Alexan- der Binder, Emmanuel Müller, and M. Kloft. Deep one-class classification. In ICML, 2018.
David Salinas, Michael dimensional multivariate forecasting with low-rank gaussian copula processes. NeurlPS, 2019.
Laurent Callot, Roberto Medico, and Jan Gasthaus. High-
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, pp- 1181-1191, 2020.
Hadi Salman. Mingjie Sun, Greg Yang. Ashish Kapoor, and Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. In NeurIPS, 2020.
T. Schlegl, Philipp Seeböck, S. Waldstein, G. Langs, and U. Shhmidt-Erfurth f-anogan: Fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal., 2019.
B. Schölkopf, John C. Platt, Shawe-Taylor, Alex Smola, and R. C. Williamson. Estimating the support of high-dimensional distribution. Neural Comput., 2001.
Shahroz Tariq, Sangyup Lee, Youjin Shin, Myeong Shin Lee, Okchul Jung, Daewon Chung, and Simon S. Woo. Detecting anomalies in space using multivariate convolutional 1stm with mixtures of probabilistic pca. KDD, 2019.
D. Tax and R. Duin. Support vector data description. Mach. Learn., 2004.
Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. In NeurIPS, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NewrIPS, 2017.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In ICMI, 2018.
12

---

Under review as a conference paper at ICLR 2023
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transform-
ers with Coto-Correlatin for long-term series forecasting. In NeurlPS, 2021.
Chong Xiang and Prateek Mittal. Patchguard++: Efficient provable attack detection against adver- sarial patches. arXiv, 2021.
Chong Xiang, Arjun Nitin Bhagoji, Vikash Sehwag, and Prateek Mittal. {PatchGuard]; A provably robust defense against adversarial patches via small receptive fields and masking. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2237-2254, 2021.
Chong Xiang, Saeed Mahloujifar, and Prateek Mittal. {PathCleanser)): Certifiably robust defense against adversarial patches for any image classifier. In s/ USENIX Security Symposium (USENIX Security 22), pp.
2022.
Haowen Xu, Wenxiao Chen, N. Zhao, Zeyan Li, Jiahao Bu, Zhihan Li, Y. Liu, Y. Zhao, Dan Pei, Yang Feng, Jian Jhen Chen, Zhaogang Wang, and Honglin Qiao. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. WWW, 2018.
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. In ICLR, 2022.
Takehisa Yairi, Naoya Takeishi, Tetsuo Oda, Yuta Nakajima, Naoki Nishimura, and Noboru Takata. A data-driven health monitoring method for satellite housekeeping data based on probabilistic clustering and dimensionality reduction. IEEE Trans. Aerosp. Electron. Syst. 2017.
Bo Yang. Xiao Fu. Nicholas D Sidiropoulos, and Mingyi Hong. Towards k-means- friunnd spaces: Simultaneous deep learning and clustering. In ICMI., 2017.
Wenbo Yang, Jidong Yuan, Xiaokang Wang, and Peixiang Zhao. Tsadv: Black-box adversarial attack on time series with local perturbations. Engineering Applications of Artificial Intelligence, 114: 105218, 2022.
Yun Yang and ChongJun Fan. Efficient and robust time series prediction model based on remd- remd-mmip with temporal-window. Expert Systems with Applications, 2022.
Zhongguo Yang, Han Li, Mingzhu Zhang, Jingbin Wang, and Chen Liu. A method for resisting adversarial attack on time series classification model iot system. In International Conference on Web Information Systems and Applications, pp. 559-566. Springer, 2020.
TaeHo Yoon, Youngsuk Park, Ernest K Ryu, and Yuyang Wang. Robust probabilistic time series forecasting. In AISTATS, 2022.
Ceylan Yozgatligil, Sipan Aslan, Cem lyigun, and Inci Batmaz. Comparison of missing value imputation methods in time series: the case of turkish meteorological data. Theoretical and applied climatology, 2013.
Huan Zhang, Hongge Chen, Chaowei Xiao, Bo Li, Duane Boning, and Cho-Jui Hsieh. Towards stable and efficient training of verifiably robust neural networks. arXiv preprint arXiv: arXiv:1906. 2019.
Zhanyuan Zhang, Benson Yuan, Michael McCoyd, and David Wagner. Clipped bagnet: Defending against sticker attacks with clipped bag-of-features. In 2020 JEEE Security and Privacy Workshops (SPW), pp. 55-61. IEEE, 2020.
Hang Zhao, Yujing Wang, Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. Multivariate time-series anomaly detection via graph attention network. ICDM, 2020.
Bin Zhou, Shenghua Liu, Bryan Hooi, Xueqi Cheng, and Jing Ye. Beatgan: Anomalous rhythm detection using ad versarially generated time series. In IJCAI, 2019.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, 2021.
13

---

Under review as conference paper at ICLR 2023
A PROOF FOR PROPOSITION
Proposition 2 (Robustness Certificate MIA). The forecast/label (not Abstain) returned by Algo- rithm I cannot be changed by any temporally-localizzd perturbation whose &n norm is no larger than Lady (see proof in Appendix).
Proof. Here prove the robustness certificate for MIA (TSC). The proof for MIA (TSF) is analogous to this proof. Assume that the adversary has changed the classification result of MIA from y1 to y2 via the cimporrallylocalizee perturbation ó (fg norm is L. For notational simplicity we denote
M M[+kkrmmin (aaaniinmmmaattaaa tka, by (()) l: O,--.,
.. [(lo [ and denote Ifadv t by
(lad)) badv = 0, .,,,..............
....,,gg to Ladv. Then, we have:
f( 3 ) = f(x C M(a) -...=
(12)
f((x1:10 8(tuar)) @ (() ) = f((x1:t 8((ta))) 2 M(
32
(13)
Then our step is prove that there exists mask (())) m {0,.... ... [to / /a]} that can
occlude the perturbation. Specifically, we show that the mask M Lلdt can cover the perturbation.
First, we show the presence of the mask M fadv by proving fuil
[(1o [mmask) //x] as follow:
tadv to Liask
to - Ladv - to + Lyaak
Lmaak LLdv
< 1
(14)
Lmaas
Ladv +1
Lmaas
Ladi +
Second, we show that (( covers the perturbation by comparing the starting/end point of the
mask M (adx)) and the perturbation ö. For the starting point, we have:
fadv
(a]
+ 1)
(tady + +1) S
(15)
Perturbation
Musk
In terms of the end points, we have:
(al tady J mmakk) (badv + Lady)
(16)
a
Perturhation
Mask
=ac tadv + (Irnack Loav + 1) fadv a
(17)
=a(1 tady J + tadv 20 0
(18)
As (()) occludes the perturbation, thus (X1st)) @ (() (X1:1 + )llaar)) @ (m)) 11 1/2
Our proof is completed.
B EMPIRICAL EVALUATION ON RISK OF TEMPORALIY-LOCALIZEE PERTURBATIONS
To support our statement about the risk temporally- perturbations, we specifically propose an algorithm for generating the lemporally-localized perturbations. We then evaluate the attack performance visually.
B.I RESTATE DEFINITION OF TEMPORALLY-LOCALIIEE PERTURBATION
Definition 2 (Temporally-localize perturbation). Temporally-locale perturbation is to perturb consecutive time points of Xista W.r.f. fn-norm constraint. The perturbed series is:
X1sto + 8 1and + 1:l
subject to |18 LLdvl S
Ladv
X1:to + [0, .0...... 8e .l... Stad Laav
O,....0)
(19)
=(x1, Staas +1 + 8 +1,
Dtadv + Loav
Stway + Loddy
Fttl
Perturbed subsequence
where tadv +1 and Laav are the starting point and the & norm of the cemporally-Lcaaliin portttl-tocliiiieeettttttttoooo perturbation tway + lit +nav)-
14

---

Under review as a conference paper at ICLR 2023
Table 8: (Traffic) Evaluate MSE between the clean forecasts and the perturbed forecasts. The temporally-localin perturbations is generated subject to different fn-norm constraints (Ladv 2%, 5%, 10%) and fn-norm constraints (1.0, 1.5, 2.0, 2.5, 3.5).
Model
Lady
1.0
1.5
2.0
2.5
3.0
3.5
2%
1.614
1.675
1.731
1.547
1.482
1.469
MLP
5%
1.437
1.375
1.295
1.288
1.472
1.590
10%
1.318
1.204
1.137
1.418
1.686
1.865
2%
0.156
0.156
0.188
0.283
0.400
0.425
MLP-Mixer
5%
0.154
0.502
0.322
0.148
0.148
0.148
10%
0.071
0.321
0.282
0.201
0.220
0.150
2%
0.059
0.132
0.152
0,422
0.212
0.176
LSTM
5%
0.323
0.503
0.646
0.630
0.852
0.929
10%
0.166
0.290
0.652
0.803
1.301
1.753
B.2 ALGORITHM OF GENERATING TEMPORALLY-LOCALLZRR PERTURBATIONS.
The objective of our algorithm is to maximize MSE between the original forecasts and the perturbed forecasts, with respect to the tu norm constraint. Specifically, given the forecasting model fx11to) > too our objective can be formulated as follows:
arg max [x + 5) f
(20)
where 8 corresponds to the perturbation defined in Eq. 19 Actually, the problem of computing the camporally-localizn perturbation can be decomposed into sub-problems: P1 Search for the period Vadv+1, +
+ +Lav]] to perturb. P2 Fix the period Vadv+1. 1, + L.dv]. compute the value
of the perturbation t +tady Here solving P2 is hard. If we have determined +taay+1..... + we can maximize the following loss to compute the perturbation values via projected gradient descent (PGD).
max Bt ady Lady
[X + - fx
(21)
Then the main challenge is to determined which period to perturb. Here we solve P1 by enumerating
all the possible perturbing positions [tady + 1 fadv + Ladv], fadv = O, .,....to - Lady and compute the corresponding attacks. Finally, return the one with the largest MSE loss among to Jeady + perturbations. However, in practice we found that computing the values of perturbation (P2 ) w.r.t the fixed period is hard to converge, the l norm of the temporally-localicn approach co. We believe that perturbation attack with co &2 norm is meaningless in practice. In the sake of practicality, we additionally consider &g norm for the camporally-localizee perturbations besides Ey norm constraint for the sub uub-problem P2 as follows:
perturbation will
[x 8) [X1.t, susbbectt to to||0||3 <
(22)
8 & 1i ndv Ladv
where c is the preset upper bound of the perturbation lg norm.
B.3
EMPIRICAL EVALUATION OF TEMPORALLX-LOCALIZED PERTURBATIONS.
B.4
QUANTIFY
OF TEMMORAILX+..CAIID2R PERTURBATIONS.
Table 8 quantifies the risk of lemporally-oocaizze perturbations via computing MSE between the clean torecasts and the perturbed forecasts w.r.t. the normorm (Taavv) and the f--m (c) constraints. We observe that MLP-Mixer model provides the highest empirical robustness among three models, which partially explains why MLP-Mixer outperforms other models on MIA. In particular, we further compare MLP to MLP+MIA (6 1.5) in Table 4 of the main paper. Specifically, MSE of MLP under temporally-localizee perturbations (e = 3.0) is 5% 1.472, 10% 0.1.686 while MLP+MIA is 5%: 0.146, 10% 0.144. MIA reduces the MSE to roughly one tenth of the original, which indicates that MIA can effectively prevent our forecasting results from being influenced by the remporally-oocaizee perturbations.
15

---

Under review as conference paper at ICLR 2023
Table 9: Dataset information for TSF and TSC.
Dataset
Context length
Forccusting length
Number of classes
Electricity
96
24
N/A
Exchange
120
30
N/A
Traffic
96
24
N/A
DistalPhalanxTW
80
N/A
6
MiddIePhalanxTTW
80
N/A
6
Proxima[PhalanXTW
80
N/A
6
B.5 COMPARE ATTACKING PERFORMANCE OF &o ATTACK TO E2 ATTACK
We compare the ty attack and &g attack under under norm constraints 3 (attack rate) time series forecasting task. Results are shown in Table 30 31 and 32 MSFe, MSE,, X 100%. MSE22
The values in tables are calculated as
B.6 VISUALIZE THE RISK OF TEMPORALLY- PERTURBATIONS.
Fig. jillustratvelly shows the effect of lemporally-localize perturbations on our forecasting results. We observe that temporally-localized perturbations of Latk 10% can significantly change our forecasts.
C EXPERIMENTAL SETUPS
C.1 DATASET INFORMATION
Table shows the details of each dataset, including context length, forecasting length (for TSF datasets) and number of classes.
Traffic Hourly occupancy rate, between 0 and 1, of 963 San Francisco car lanes Salinas et al., 2019
Electricity Hourly time series of the electricity consumption of 370 customers (Salinas et al. 2019).
Exchange
Daily exchange rate between 8 currencies Salinas et al. 2019
DistalPhalanxTW, MiddlePhalanxTTW ProxinaIPhalanxIIv This series of 11 classification problems were created as part of Luke Davis's PhD titled "Predictive Modelling of Bone Ageing". They designed to test the efficacy of hand and bone outline detection and whether these outlines could be helpful in bone age prediction. Note that these problems are aligned by subject, and hence be treated multi-dimennsioal TSC problem. The final three bone classification problems, DistalPhalanxTW. MiddlePhalanxTTV and ProximalPhalanx TW, involve predicting the Tanner-Whitchouse score (as labelled by a human expert) from the outline.
Data Pre- PreeProcessing We pre-process the input series with savgol with window length 15 and polyorder 5 on both training and testing datasets. Besides, we normalize each input series with its mean value and standard deviation. Mean value and standard deviation will be 0 and I respectively for each normalized input series. We use the instance normalization method on both trainsets and testsets.
16

---

Under review as conference paper at ICLR 2023
Perturbed
Perturbed
Perturbed
1.0
Perturbed
M
0.0
-1.0
-2
-2.0
20
40
60
100
120
40
60
100
120
40
60
100
120
40
00
100
120
Perturbed
Perturoed
Perturbed
Perturbed
20
40
00
100
120
40
60
80
100
120
40
60
80
100
120
40
00
100
120
Perturbed
Perturoed
Perturbed
Perturbed
-1
20
40
60
130
120
40
6U
80 100
120
40
60
80 100
120
40
80
100
120
Porturbed
Porturoed
Perturbed
Perturbed
Clear
-2
20
40
60
130
120
40
60
80
100
120
40
60
100
120
40
60
8C
100
120
Perturbed
Perturoed
Perturbed
Perturbed
-2-
2
20
40
60
100
120
40
60
80
100
120
40
60
80
100
120
40
60
BC
100
120
Perturbed
Perturoed
Perturbed
Perturbed
-2
25
50
75 100
125
150
50
75
100
125
150
50
75
100
125
150
50
75
100
125
150
Perturbed
Porturoed
Perturbod
Perturbed
1.0
M
-0.5
-1
-1.5
w
25
50
75
100
125
150
50
75 100
125
150
50 75 100
125
150
50
75 100 125 150
Figure 3: The effect of temporally-locan localiize perturbations (Latk 10% and e 3.0) on different datasets. Clean and Perturbed refer to the normal input series and the temporally-localized pertur- bations respectively. Row 1, 2, 3: Traffic. Row 4, Electricity. Row 7: Exchange rate. The red background denotes the position of the location of the perturbation. The blue background denotes the output series.
17

---

Under review as conference paper at ICLR 2023
structure
Table 10: MLP structure
Input dim
Output dim
Activation
LeakyRel.L 0.2)
Length of sequence
96
96
96
LeakyReL - 0.2 =0.2
96
LeakyReLUU =0.2)
96
96
LeakyReLU(e =0.2)
96
Length of sequence
LeakyReLU(a -0.2)
Table 11: LSTM/GRU structure
Number of layers
Hidden dim
32
Table 12: The structure of MLP-Miss Block, and we stack 4 MLP-Mixer blocks to construct MLP- Mixer for forecasting and imputation.
Type
Input dim
Output dim
Activation
LayerNorm
96
96
Linear
128
128
GELU Hendrycks & Gimpel 2016
Linear
128
GELU
LayerNorm
Linear
Length of sequence Length of sequence
Length of sequence Length of sequence
GELU GELU
Linear
Table 13: The structure of Fully Convolutional Network (FCN) for TSC.
Layer
Input channel
Output channel
Kernel
Conv Id
96
3
BatchNormid
N/A
ReLU
96
96
N/A
Conv ld
96
96
3
BatchNorrid
96
N/A
ReLU
96
96
N/A
Conv
BatchNormid
96
96
N/A
ReLU
96
96
N/A
Convid ld
96
3
BatchNormid
96
96
N/A
ReLU
96
N/A
GlobalPooling
96
96
N/A
Linear
N/A
IRSOODCTTION TO PRETRAINED MODELS
D INTRODUCTION IXTRODUCTION
Model architecture. We use the classical forecasting and classification models, MLP, MLP- Mixer (Tolstikhin et al. (021). GRU (Cho et al., 2014), LSTM (Hochreiter & Schmidhuber 1997).
FCN (Ismail Fawaz et al. 20195) and ResNet- 18 (He et al. 2016) as the pretrained models. We
thehitecctur show the architecture of these models in Table T0
13
Training. we uniformly adapt Adam optimizer Kinggma & Ba 2014) with 0.0001, (0.9, 0.999), e = 10 8, weight decay = O, epochs = 20 for all the pretrained models.
MORE EXPLRIMENTAL RESULTS
Comparison to peer methods on Mean Absolute Error (MAE). Table 15 compares MAE and FR of three defenses on Traffic. Analogous to the comparison (Table in main paper) MSE, MIA achieves both the lowest MAE and highest FR on Lady = 1%, 2%, 3%. DS outperforms MIA at
Trady 4%,..., .., 10% for the great sacrifice on its FR.
18

---

Under review as a conference paper at ICLR 2023
Table 14: Comparison of imputation quality (MSE between the imputed series and the original
series) of different imputation models on imputing the masked series of different mask length Lmaak = 5%, 10,, 15%, 20%. Fixing makk. we first construct to -maak + masked series of
Lmaak and compute the average imputation MSE over imputing these to - Lmaak 1 masked series.
Bold indicates the best among four generators.
Traffic
Electricity
Exchange
Generator
5%
10%
15%
20%
10%
15%
20%
5%
10%
15%
20%
MLP-Mixer
0.0007
0,0082
0.0170
0.0240
0.0033
0.0320
0,0458
0.0631
0.0002
0,00288
0.0104
0.0212
SAITS
0,1652
0.1110
0.1786
0.2476
0.0878
0.1654
0.2687
0,3586
0.0191
0.0410
0.0631
0.0824
Transformer
0,0642
0.1183
0.1854
0.2629
0.0821
0.1836
0.2404
0.3271
0.0210
0.0389
0.0595
0.0787
BRITS
0.0159
0.0419
0.0656
0.0927
0.0410
0.1192
0.1613
0.2109
0.0099
0.0252
0.0516
0.0705
Table 15: (Exchange) Comparison among three certified defenses on TSF dataset.
Metric
Defense
%
%
%
4 %
5 %
6 %
7 %%
8 %%
9 %
10 %
MIA (L 10%)
82.2
82.2
83.2
82.2
82.2
81.2
81.2
81.2
80.2
79.2
DS (1) 10%)
24.8
24.8
24.8
24.8
24.8
24.8
24.8
24.8
24.8
24.8
FR (%)
RA (1) 10%)
16.8
16.8
16.8
16.8
16.8
16.8
16.8
16.8
16.8
16.8
MIA (Lumm 15%)
64.4
71.3
69.3
69.3
71.3
68.3
69.3
71.3
62.4
65.3
DS (1) = 15%)
20.8
20.8
20.8
14.9
14.9
11.9
11.9
11.9
11.9
10.9
RA (1) 15%)
23.8
23.8
23.8
23.8
23.8
23.8
23.8
23.8
23.8
23.8
MIA (Luma 10%)
0,332
0,330
0.334
0.330
0.330
0.327
0.326
0.326
0,323
0,320
DS (1) 10%)
0.408
0.408
0.408
0.408
0.408
0.408
0.408
0.408
0.408
0.408
RA (1) 10%)
0.413
0,413
0.413
0.413
0.413
0.413
0.413
0.413
0.413
0.413
MAE
MIA (Limm 15%)
0.307
0,320
0.313
0.316
0.322
0.317
0.311
0.314
0.301
0.306
DS (1) 15%)
0.320
0.320
0.320
0.215
0.215
0.222
0.222
0.222
0.222
0.204
RA (1) 15%)
0.446
0,446
0.446
0.446
0.446
0.446
0.446
0.446
0.446
0.446
Table 16: (Traffic) The performance of MIA on different pretrained models. (c1 %2% reports (MAE, FR%) of MIA. Baseline is MAE of the pretrained model without MIA. The lowest MAE and the highest FR for each pretrained model is shown in bold-face.
Model
Boseline
1.0
5%
2%
10%
0.227
0,213 17.25
0.265
0:281
0.233 75.55
0,225 80.2%
90.15
(1233 77.2%
76.2%
0.227
0.225 80.2%
0.216 76.2%
0,336
0,333
25%
0.234 05.3%5
0,210 73.555
(L339 50.
CRUU
0.283
5%
0:342 72.55
0.239 68 555
0.219 76.255
0.214
24.35
0.340 91.1%
0,337 85, 135
10%
0.237 74.3%
0,239 71.3%
0.236 63.4%
77.2%
0,217
77.2%
0.283 70.3%
0,340 91.1%
0.338 90.1%
0,329
(L238 66.3%
0.212 N0-255
0.380
LSTM
0.244
5%
0.257 66.3%
0.237 65.35%
0.215
77.2%
0.213
26.2%
0.345 80L176
0,345 8X. 136
10%
0.237 60.075
0,239 66.3%
0.251 61.4%
0,215 77.2%
0.215
77.2%
0.201 72.3%
0.347 S0.15E
0.349 90.1%
87.14
0.326
0.201 73.3%
900.1
0.288
0.251
0.301 73.3%
72.3%
0,343 92.1%
0.344 DO.
0:330 70.275
0,230 70.3%
66.3%
0,201 73.3%
72.3%
0.199
0.341 91.115
0,308
0.344 64.475
0.258 76,255
88.155
ResNet18
0.290
0.249 68.3%
0.249 65.35%
0.239 79.2%
0.24] 76.2%
0,345 89.1%
0.340
57.1%%
0.249 67.775
0,251 67.3%
0.246 60.4%
0.235 76.27%
0,335 76.2%
0.231 73.3%
0.345 88.156
0,343 96.13%
0,537 83.%
Table 17: (Electricity) (c) 22%) report (MAE FR%) of MIA on different pretrained models.
Model
Buseline
2%
5%
1.0
10%
2%
12 5%
10%
2%
5%
1.5
10%
0.300 05.11%
0.202
77.2%
Mixer
0.430
0.254 69.3%
68.3%
0.266
66.3%
0,308
75.2%
0.308
0,364 07.27%
0,259 151.4%
0.255 59.4%
0.368
77.2%
0,308
0.287 06.3%
0.369
0.347 60,455
08.35%
GRU
01441
0.273 61.45
0.274 63.4%
0,250
62.4%
0,317 69.3%
0.317 69.3%
0.275 45.3%
0.270) 02, 455
0.282 69.
0.269 61.4%
0,250 53.5%
0,31Y 09.3%
0.317 69.3%
6.295
2%
0.373 64.47%
0.243 50.57%
0.319 711.3%
LSTM
0.447
5%
(1.773
66.3%
0,225
68.3%
0.257 30
0.257 55.4%
0,316 71.5%
0.319 70.3%
10%
0.273
60.3%
0.272 61.4%
0.208 51.5%
0.257 56.4%%
0.257
55.4%%
0.262 45.55%
00.116 71.3%
0.320 711.3%
0.294599.45
0.283 73.17%
0.243 51.4%
0.313 74.35%
0.421
01387 77.2%
75.2%
0.249 65.3%)
63,4%
0.304 77.2%
0.306
0.283 75.2%
75.2%
60.4%%
0.250 61.455
0.250 61 11%
0.346 55.4%
0.304 77.2%
0.299
0.280 66.3%
0:256
50.155
60.35%
ResNa188
0.569
0,348
50.35%
0.242 60.4%)
0.324
0.257 52.5%
0.256 51.5%
0.255
0.249
0.244 08
0.237 54.5%%
0.323
0.318
0.302 61.4%
Evaluation of MAE on Traffic and Electricity. Table 16 Table 17 report MAE and FR of MIA on Traffic, Electricity and Exchange respectively, as a supplement to Table 4 and Table 5 in the main paper. We observe that MIA consistently reduces MAE of the pretrained models, similar to
19

---

Under review as a conference paper at ICLR 2023
Table 18: (Exchange) 0i 02% report MSE FRY of MIA at different Latk and Ldef-
Model
Buaciin:
2%
4 1,0
5%
10%
2%
3 122
5%
10%
2%
A 1.5
5%
10%
2%
0.054 5%.1%
0,100 57.1%
0.158 87.1%
MLP Mixer
0.030
5%c
0.855 83.2%
0 50.25
DUNN 54.25
DONN 84.2%
0.164 S0.256
0,138
83.235
0/066 63.2%
0,053 81.2%
IJISI 7802%%
ILM9S 51.255
54.2%
OJIS8 S0L21)
(L141
82.999
0.134 79.25)
0,040
0,000
80.2%
(UISK
54.2%
84.2%
0.153 54.2%
0,142
216
(J861 53.2%
6,098 87.1%
0. 85.1%
LSTM
0.042
55G
0.062 52.95%
0J858 78
0.99 50.155
0.09N 65 255
(L159 55-955
0,182
19-25
10%
0,003 92.27
0.062 90.2%
0.055 75.2%%
0.099 $0,155
0:099 85.1%
0J098 82.2%
0.150 $4,216
0.150
$4.2%
N2.2%
2%
0.065 27.7%
0.110 41,655
0.210 56-4%
MLP
0.658
5%C
0.857
25.77
0J072 10.55
0.105 44.6%
0.094 27.7%
0.209 50.51%
0,231 60.635
10%
0.057 29.7%5
0,057 26.7%
0.069 18,8%%
0.107 44.6%
13.00-43..6
0.105 35.6%
0.213 51.03%
0.216.52.5%
0,240 39.6%
0.060
OLORG
ResNe118
0.033
0,068
78..2%
0J190
0.159
Table 19: (Exchange) ej 02% report MAE FR% of MIA at different Lady and Lmask-
Medel
Boaelin
5%
10%
5%
10%
(1209 58,
0.210-87.1% 1%%
0,349 87.1%
MLP-Misrr
0.133
5%
0.199 83.255
0.194
50.25%
0.264 $4.255
0.364
94.2%
0.353 68.2%
0,326 81 81.235
10%
0.200 83.2%
0.195
81.2%
0,190 76.2%
0.263 $4,255
0.263
84.2%
0.252 80L2%
0.330 52.2%
0.330 82.2%
0,320 79.2%
25
0.211 93.2%
0.274 $3.255
0.342 84.2%
GRU
0.152
0,206
79.215
0.203
76.25%
0.276 85.155
0.267 80.2%
0.339 68.2%%
0,325 27:235
10%
0.210 62.255
0,209 90.2%
0.203 68.3%%
0.274 56.1%
0.270 84.2%
0.263 75.2%
0.342 84.2%
0.342 54.2%
78.2%
0.309 83.2%
0.266
15.1
LSTM
0.159
0:310
78.25
96.155
0.364
88.25)
0.340 70:235
0:315 82.25
90.156
82.2%
50.256
0.213
0.264 41.655
56.4%
MLP
0.639
0,196
229
0,274 44.6%
0.260
50.55%
0,196 77%
0.196 26.7%
0.227 18,8%
0.277 46.0%
0,214 43,6%
0.275 35.0%
51.055
0.416 52.5%
0.658 39.0%
25%
0.205 78.25
0.263 83.25%
0.355 85.1%
ResNer18
0.181
5%
0.201 79.2%
0.201 77.296
0.257 51.25
0:257 90,2%
0.254
S7.15
0,M7 NS.
10%
0,302 75.2%
0,205 78.2%
0.201 72.3%
0.254 75522%
0.256 78 25
0,253 76621
0.354 87.15
0..51
0,349 85.1%
the results on MSE. Our comprehensive experiments suggest that MIA can effectively improve our forecasting quality in the way of filtering the unconfident forecasts.
Evaluation of MSE and MAE for MIA Exchange. Table and Table 19 evaluate MSE/MAE of MIA on exchange. We observe that, MIA moderately increase MSE/MAE of the pretrained models because of the information loss for the discretization technique. Specifically, we can see that MSE/MAE of the pretrained models are commonly much smaller than that of Traffic and Electricity because Exchange dataset is much simpler. The better forecasting performance implies that we might lose more information for the discretization technique. On Exchange, information loss plays more conspicuous role than the filtering function of MIA, causing the increase of MSE/MAE. This suggests that discretization technique might lower the forecasting performance when the pretrained models are precise enough.
Analysis of training algorithm of imputation models. Table 20 reportS MSE and FR of masked training and random training the TSF dataset Traffic. Similar to the comparison on TSC datasets (Table 6 in the main paper), our masked training consistently achieves lower MSE than random training on different imputation models.
Analysis of Different Pretrained Models on MIA. Table 21 reports the MAE of MIA with different pretrained models, as a supplement to Table 7 in the main paper. The results demonstrate that MLP-Mixer outperforms all the other defenses.
F MIA oN MULTIVARIATE TIME SERIES FORECASTING (MTSF)
Apply MIA to Multivariate. Suppose the input series is a do-dimension to-length matrix X E Rox We can represent the multivariate series uniquely by do univariate series l:t: d = 112,...,.e. do-
X
(1)
2
(2)
(do) into
(23)
20

---

Under review as a conference paper at ICLR 2023
Table 20: (Traffic) Comparison between mask methods. ei %%% report MSE TPR% of MIA across
Laei 2%, 5%, 10%, Latk- = 2%, 5%, 10%, A 1.0, 1.2, 1.5. c1 MSE(c)) reports the forecasting performance.
Medel
Misk Methed
2%
A - 1.0
5%
10%
2%
5%
10%
24
Amm 1.5 5%
10PR
IJI8N
BJPI BSL
0.344 TO.05E
mandum
0U171
0.055
BJKA 50.55E
IUIE (8.05
T5-35L
0.073
0.000
0,075 20L,55
49,25
0L048
65.000
0,85 40L03S
64.458
08
0.073
Nock
0.073
B,75
0,033
OLOT3 65.3%
0.148
0,145
S1.2%
0/973
0.174 63.4%
MAPTO 57.40%
01074
0L094 64.4%
01055 SHUAY
0.148 84.2%
0.148
84.2%
0,145 79.2%)
250
0J064 53: 65C
0 02-155
0,144 51-255
randon
5%
0.663 377
0,800
25
0,051 20.556
ILOSS DSARS
0.129 07.35%
0.130
C5.456
10%
0J865
37
0 27.775
0:049 34.955
0J060 04.55
ILOSS 05.505
0/846 30.
0.334 18LU55
1.128
61.
0.220 53.
0.071
0.144 85.1%
0.067
0.067 56.4%
0.009
0.069
0.067 65.35%
0.144
0.142
25
0.073 64.55
0.085 06.550
0,147 T0.25
rindom
5%G
BU77 51.075
0.867
77
B 50L455
IUI61 37.556
9.337 TEL154
B.142
MLB MiR-M
10%
0 67.3%
0J072 42 655
0.073 $0.055
0:869 20.775
0J0RS 74.3%%
0.065 07.455
(L067 63.05
0.061 45.0%%
0.151 86.1%
0.134 T4LSS
0.130
70.55
0.130 01
0.006
0.078 61.4%
DURR
OL00N
0.151 86.1%,
0.1.4 83.24
0.079 63.4%
0
0J076 55.4%%
0,051
OL077
0,078 69.3%%
0.151 96.1%
0.150
82.25%
0.069
0.074 G1.45
0.145 76-2%%
mandom
S/G
(1012 01 61.655
0.011 07.695
0.089 58. 15.
IL144 07.09G
0.158 TS.
IL138
07:
MLP
10%
0.072
0
60.4%
42 073
0,073 40.075
0.077 34.7%
BJANN 68.3%
0 53,555
OJ062 02.05
0,055 48.035
D. 146 84.2%
0.149 75.25
0.140
75.25
0.833 67.835
Mack
5/G
0,871
61.45
8,071 58.4%
0.070 66.3%,
0.000 66.3%
0.145 86.1%
0.147
$5.1%%
10%
0,073 58.4%
0J072 59.45%
BUNGM 57.5%%
DURS 66.3%,
0,005 66.3%
0.005 62.4%
0.146 55.2%
0,147
85.1%
0,144 82.2%
0.071 07
0
55-455
Rod
252
0,073 63.4%
0.967 12,
0.070
0,006 37.55
0.089 71.3%
0.059 20.574
(UI61 49,57%
0,847 37.63
0.146886115 16.1%
61.4%%
0L130
0,137 40,626
Nock
5/6
0,074 63.4%
0,074 62.4%
0.085 66.3%
0,004 67.3%
87.1%
02044 86.1%
10%
0.075
60.4%
0,072 99.4%
00770 57.4%
0.087
68.3%
0.004 64.3%
62.4%62..4%
01143 84.2%
0,143 84.2%
O, 140 K2.2%
Table 21: Comparison of four imputation models. The best results are shown in bold-face.
Ganenior
Mesic
2%
5%
2%
Etchunge
10%
5%
10%
0.312:5 97.2%
0,349 87.1%%
0,340 1%
MI P-Miurr
MAR
308 78.2%
0. 308 78.2%
0,585 83.2%
81.2%
0,336 1%
0,340 89.1%
10%
300 77.2%
0,308 77.2%
0.287 66.3%)
0,390 82.2%)
0,390
82.24
0,300 79.2%%
0,338 98.1%
0,396 90.1%,
0:333 85,
2%
0.290 13.33%
0.299 40 875
0.338 57 116
BRITS
MAR
5%
01261
71.3%
0.264 50, 6%
0.288
49) 595
0.246
15.656
0 (.922 S1.255
0 0.930-75 75:
10%
0.271
61.445
0:272 00. 096
0.265 595
D. 18981 10 10.995
0.526 1.055
0.815 79.255
0.319 77.255
0,8 Q.812733355
0.269 4%
0.258 195
0.MP -56
SAFTS
MAR
0.261 53.5%
0.256
0.312 05.655
0.264 756
0 08
10%
0.274 50.4%
0.278 54.595
0L263 61.695
0.179 2.995
0.123
0,038 1.005
0.316 07.666.555 .555
(0.355
0,315 59.155
Transfoence
MAR
(1768
0.812:65355
0.274 $1.795
0.037 5.0%
6.31261-155
1. We generate the masks in the same way as Masking (Step 1). The main difference is the way we mask the multivariate series with the univariate mask. Masking multivariate series X' with the mask M[cc]] is computed as follow:
X' 4 M[xco]
x
lsfo
(2) 22ta ④ M [uso]
(do) Lito c M
Muso]
T ]
(24)
2. With the imputation model G(-), Imputing (Step 2) for multivariate series X' is computed as follow:
G(X Munee) =
[
x (1) l:to c
Munw)
G
((x(2)
X2:10
(2)
@
G (xf (do) @ M ppcv)]
T
(25)
3. We aggregate all the predictions of the imputed multivariate series in the same way as Aggregation (Step 3) for univariate series.
Evaluation of MIA on multivariate tasks. In terms of the multivariate time series forecasting (MTSF) task, we follow the work (Wu et al. 2022] and evaluate our MIA framework on four datasets
(ETTh2 Zhou et al. 2021). ETTm² (Zhou al. 2021), weather 3 and illness 3 Corresponding results are presented in Table 22 to 29 Extensive experiments demonstrate that MIA behaves
similarly to that of univariate forecasting tasks.
htttps:/www.bgc--r httus://ww..kgr-jjeaaapp..eeeeet
21

---

Under review as a conference paper at ICLR 2023
Table 22: (ETTh2) Evaluate MAE of MIA with pretrained models on multi-variate time series forecasting (MTSF).
2%
0.363 74.555
0.434 52.475
0.590 10.35
MI M-
0.112
5%
11 963 755.8%
73 MSC
Du435 83.6%
0.430
81
0.552
MBU8%
U1547 40.45
10%
0,360 73 09%
0.356 74.2%
0,346 DOUSSS
ILA31 55.055
0.427
ILAIS 77.515
0,548 575
0,542 90.1%
0.43
MLP
66.0%
(6.71%
0 76.0%%
9,544
70.054
M6.6%
11541
85.15
10.
0,368 07.058
0.365 66.0X
0.356 64.4%
0:438 TOLL55
0.435
75.9%)
0.424 73.3%
0.545 56.
0.541
85.8%
0.527 63.5%
5%
01574
69.11%
IL444 TOU655
0.556 SG 735
LSTM
0:375
69.
Q.440 97.1%
0.443
85.9K)
64
0,173
65. 476
68.0%
0,370
0,445 7O.75
74.0%
0.550
85.3%
2.%
0,377 91.45%
0:441 T5
0.563 57.1%)
GRU
0.782
5%
01.744 72.4%
0.372 TILISS
SU442
75.5%
0,440
77.556
0.565 97.65%
11550
55.115
10%%
0,372 72.05%
0.372 76.6%
0,369 65.1%%
IL440 75.05%
0L439
77.6%%
0.435 75.44%
0,562 57.555
0.556 86.9%
IL545 54.%%
0.264
0.454
87.6%%
10%55
0,361 09
0,199 68 05C
0.37% 67..1%
(L482 78.75
0,451 355
0L44K 76.9%
0,589 ST.53)
0,596 87.0P%
[L5S1 SK.I%
II 805 67.0%
QUARE 75.3%%
0.595 83.8%%
TaaeformenNmmmal
0357
0-803 66.1%
01-00 15
74.8%
(1903 476
0.191 83.39
10%
0,400 65.355
0,393 63.455
0.478 74.255
0,477
7.44%
$2,415
1%
(1546 57.9%
U--27 05.355
0,555 76.7%)
Transfrrnnraaaling
0:981
0.346 07.658
0.346
57.8%
0:426 65.355
0,426
05.555
0.552 76 835
0.552 76.5%
10%
0,344 57. 656
0,348 57.8%
0,364 57.8%
BJ425 65.47%
0.425 65.0%%
11424 65.3%%
0,588 76 07.55
0,150 76.5%
IL550 7A.5%
61.1%
0.427 10:215
79.45L
Transtonnstoone
0:352
(8.6%
0.426
60.255
0,348
0,433
68.7%
0,531
Table 23: (ETTh2) Evaluate MSE of MIA with different pretrained models on multi-variate time series forecasting (MTSF).
Midel
Buscline
5.55
10% 55
2%%
5%
10%
2.%
4
5%%
10%
0.1MM
11:10 47.4%
MLP-Mieer
0.930
0.331
0.228
74.2%
0.347
2%
0,862 65,
0.230 76,355
0,367 55 55.85
MIP
1.143
5%
(1 163 M0.0..
0:160666710
66.716
0.331 76.6%
6.23N 71 555
0. 3.362 96.8%
1.366
65 1%
10.
0.062 07.058
0.159 60.63
0,153 64.4%
0.230 70.255
0.225
75.9%)
0.219 73.3%
0,360 56
0.257
55.8%
0,342 53.5%
LSTM
0.250
0.234 76.0%
0.164 65.45%
AUMMS
0,862 66,71
0.236
0.233 76.1%
11.226 74.0%
0,10 85.9%%
0,16 85.3%%
IL357 K3.A%
2%%
0.866 21.655
0.236 (8.
0.37% 62 135
GRU
3.042
167 72.4%
B.168 T0LUE
OLES7 78.3%%
0.255
72
0.3MI 87.6%%
10%
0.866 72.095
71.6%
0.861 68.1%%
0.235 78.07%
0.233
75.4%%
54.2%%
0.171 09.14
0.244 751075
0,375 87,6%
RNN
3,074
[112] 70.2%
0,1700
0800%
SU343 75.2%%
0.243
78.4%
0,375 47.65%
0.773
10%
0.170 65.55%
8.169 65 USE
0.166 67.0%
0.242 78.75
0.341
55.355
0.238 76.9%
0 S7 1955
0,370 87.0%%
0,366 56.1%
67.0%
0.364 75.2%
83.85%
TransfemenNommll
74.8%
63.316
GLL77
0,377
0.261
0,259
0.256 73.4%
125
51594
5%
II 149 97.9%
11212 05 15.335
0. 3636 76.7%
1.576
%
0,849 07.
0.149 57.8%
0.222 65.355
0.222
05.
0.361 76.025
0.361 76.5%
10%
57. 855 0.848
0.148 57.3%%
0.148 07.87%
0.321 65.4%
0.221 60.4%%
01221 68.31%
76. 5/%
6,369 76.5%
0.899766
61.1%%
70.215
79.45)
Condstemettoon
1.444
0.154 01 (16
0.15%
0.224 DELNT
01227 69.7%
79.4%
0.351 7964%
10%
0.153 00,258
0.152 GOLISE
0.151 59.7%
0.232 (0.275
0.231 09.655
0,219 68.7%
0 70
0.349 20.115
0.347 7A.1%
G IMPUTATION MODELS.
For the imputation model architectures, we take SAITS, Transformer and BRITS and MLP-Mixer. Specifically, for SAITS, we use the code from 10 For SAITS (Du et al. 2022), we set dmodel 32, Mlaayrrs 2, diuner 16, Nhead = 4, dg 8, d 8. For Transformer Vaswani et al.) 2017 we set dmadel = 32, aayyr = 2, d'nner = 16, Rhead = 4, d2 = 8, d, = 8. For BRITS (Cao et al. 2018), we set haidden 32. For MI.P-Mixer (Tolstikhin et al. 2021). we use the same structure as Table 12 In terms of training, we use optimizer Adam with Ir = 0.0001, 3 = (0.9, 0.999). = 10 weight_decay = 0 and train the model for 30 epochs.
G.1 CHOICE or IMPUTATION MODEL ARCHITECTRR.
Table 14 compares the imputation quality of different imputation models on Traffic. The imputation quality quantified by the mean square error (MSE) between the imputed series and the original series. We observe that MLP-Mixer consistently outperforms other three models across different imputaly datasets and T.adv, indicating the superior imputation ability of Mi.e-Mixer architecture
" hattpss//gituub..cc////eeeee///p
22

---

Under review as a conference paper at ICLR 2023
Table 24: (ETTm2) Evaluate MAE of MIA with different pretrained models on multi- time series forecasting (MTSF).
A-12
2%
0.329 07, .55
0.404 76.255
0.536 50 735
MI P-M
0,858
5%
I1 540
68.
8.512
59-55C
04421 76.1%
0,385
70-755
0.548 84.3%
IU994 SU
10%
0:337
07 156
0:316
66.7%
0.29 A6,47S
0.444 75-395
0.413
01361 GULISS
0.533 995
14.25%
0.427
MLP
11306
76.3%
0
9,488
87.4%
U.537 N5
10.15
0.357 04.658
0,351
72,0%
0.324 61.9%
0.445 63,65%
0..4%%
0.640
52.4%
0,411 72.5%
0.569 56 835
0.567
87.0%
0.529 S0.8%
5%
01364 68.6%
OL462 72.9%
0.554 82.8%
LSTM
0:363
61.256
K3.0%)
0.460 72.-15
0.454
71.3%
0,789 63.556
0,347 67,E%S
0,456 71.355
0,552
51.77
87.877
2.%
558 62.4%
0UE2N 69.9%
0,619 26,9%
GRU
3,005
5%
0,358 61.
0,371 69.2%
0L479 D0.675
0.4636746%
0.463
67.67%
0.020 7%
1L694 70.255
10%%
0.382 65.055
0.573 SALORE
0,353 64,79 54.7%)
IL476 67.35%
0.488 06, 06.555
0.445 62.31%
R616 71.25
77.4%
0,589 74.2%
0.363
10%
[ 908 64.6%
0.209 A0.5%
0,260 49.71%
0U323 71.6%
0.558 68.0%
0.310.90.5%
0.42%
0.465 75.2%
EL410 60.5%
%
II 233 97.7%6
OUTRT 67.57%
0.008 77.8%)
TaaesfemenNommal
0.98
6.318:50
DUMR 31
0.369 61
(1473 07-
53.455
51.4%
0.283 40.0%5
0,377 64.275
0.369
54.6%
76.725
0.457 74.1%
1
11577
B:445 60.7%%
0,559 73,3%
Transfrrnntaaaliing
1,096
%
0,392 45. 558
0,393 44,0%
0.459 20.35%
0.459 54.0%
0 71.45)
0.551
68,3%
10%
0,378 Ue. 056
0,577 31,95C
0.373 27,0%
0:947 68-37%
0.445 05,755
H.434 57.3%%
0.595 03 075
0.35%
01
IUS43 54.6%
87.5%
0.401 12.355
Transtonnesoone
79.3%
0.312
0,208
0.384
0.375
58.7%
0.667
Table 25: (ETTm2) Evaluate MSE of MIA with different pretrained models on multi-variate time series forecasting (MTSF).
Medkl
Braciin:
10.5
10%
0,3.9 67 15%
112066 75-35%
775
MLPMMier
1.750
5%
0.145
68 2%
0.128 62.85E
0.219 76.1%
0.192
20.718
0,362 94,3%)
0.319
SOL115
10.%
0.845 07 256
0.142 66.7%
0,121 A6.4%
11266 75-3%
0.213 PK.0%
VL1TH 65.0%
0.563 52 9P5
0.148 82.8%
IL287 75.6%
0.222
01343
0.155 74.05
72/0%
0.356
03233
72.5%
0.370
5%
11155 64.6%
SU542 72.9%
0 82.4%%
LSIM
2,858
0.157
0.152
0.241 772.55
0126671.14
0U301
0.386 51.655
65.255
0,149 62.6%%
0.238 11.315
0.235
51.75
0.396 81.8%
0,381
OL260
78,9%)
GRU
2,685
0,114 01. 855
(9.2%)
0.261 (0.45%
0.247
67.6%
0.436
16-5%)
1.410 10.355
10%
0.171 09.558
0.168 S0L0CS
0,154 54.7%
0.240 07.275
0,254 06,
0.236 62.7%
77.22 0,422
0.41777.19
0,315 74.2%
0.023
ILEFT TOUT55
64.258
0.104 GELESS
OLISS
0.159
78.25%
2%
0.131 57.7%
CL186 67.5%
0.286
77.6%%
TransbemenNomal
2.100
1%
0.128 56 556
0.121 $1.999
0.178 ASC
0L708 TO 475
11269
10%
0J124
63.455
0.120 51.4%
0,108 40.0%
0L1S1 64.25%
0.173 62.4%%
0.163 54.6%
0.279 76.755
0.208 74.1%
0.249 68.7%
Transermaaaaliingg
555
0.245
0.243
0.367
10%
0,066 05,
0.168 34,55
0,164 27.00%
0.236 48.29%
0.236 15.
0.226 37.3%%
0.367
68 (5.055
0.365 61.87%
0.353 54.6%
01139 67.5
11203 72.455
0,302
TransformeCuuvs
0.195
73.8%%
0.175 AE
99.25%
6,291
0.13X 67.756
60.0%
50.4%
0.193
01174 (5.2%%
77.075
70.155
G.2 IMPACT OF GAUSSIAN AUGMENTATION
Table 33
Table 34 Table 35 Table 36 and Table 37 report the impact of Gaussian augmentation 0.01, 0.02, 0.03, 0.04, 0.05 time series classification dataset DistalPhalanxTWW (Lmaskk 15%) respectively. We compare the MIA with Gaussian augmentation to MIA without Gaussian augmentation (baseline) on # #*Certified Accuracy* at Latk = 5%, 10%, 15%. The table reports the relative improvement (abs %) on certified accuracy.
G.3 VISUALIZE THE IMPUTATION QUALITY.
Fig. iittuitively shows the imputation performance of different imputation models on dataset Traffic.
23

---

Under review as a conference paper at ICLR 2023
Table 26: (llness) Evaluate MAE of MIA with different pretrained models on multi-variate time series forecasting (MTSF).
Model
Biselise
10%
5%
10%
10%
2%
182232 $1
0,351 80.5%
0.314 98.87S
Mixer
0.712
5%
0:217
27:
0,234 T6.6%
1.236 S1.15
0.227
H6.7%
0,309 125
0.263 57,755
0.216
77.2%
0.230 73.0%
0.254 54.555
0.252
0.247 82.5%%
995
0.313 88.9%
0.286
5%%
54,4%
0.353
80.5%)
MIP
0.090
0,258 51.555
0.258
50.9%
0.306 62.65
0,301
0 50
0.376 55.
10.15
0,268 50.558
0.363
50.85%
0.256 50.3%
0,328 00,275
0.325 65.5%%
0,311 63.7%
0,388 53
0.789
0.373 7M.5%
(1,29 68.6%
8.291 RRUATS
3.35 77,85%
LSTM
01244-40
45.6%
0.297
0.298
0.385
0.239
0.292 99.6%
70.075
0.351
2%%
0.300 54,4%
0.356 67.3%
0 90.8%%
GRU
0.987
5%
0.296 54.4%
0.299
52.052
0L354 67.3%%
0.343-66775
0,368 78
0.503
78.055
10.%
I1294 54.4%%
0.294 54.4%
0.2W6 54.4%
BL382 67.8%%
0,331 67.37%
67.35
0.374 99.15
0.373 80.1%
38.6%
OL765
63.7%)
KNN
0.300 166
SUUS
03361 500.55
0.353
1.964 05 15-125
0.308
10%
0,297 55.058
0.297 38.0%
0,253 35,0%
0.360 52:85%
0,357 02,055
0.354 52.0%
0.394 65
0.390
62.875
0.387 62.6%
01244 58.5%
OLEUI 70L2%%
0.377 80.5%%
TaanformatNmmaal
0.877
0:253
65.05Z
0.280 57.555
0.265 06.755
0.242 7%.7%
55.6%
0,233 54.4%5
0.2N0 07.575
0.277
0,272 66.1%)
79.5%%
7.336-78.5%
2%%
IL299 54,4%
BL326 65L0%
0. 0.162 $4.3%%
TrassiennePoolling
0:945
5%
0 12 678
0,305
90.3%;
0.325
(5.L55
0.336
64.356
0,353 51 $1.935
0,358 51.155
10%
0,200
8,298 (0)
0.296 48.0%%
0.341 67 67.35%
0.546 66.7%)
0.351 65.5%%
(L922 65 695
62.55%
0,381 51.0%
0,316
10%
0,258 63,
6.26%
0,263 66.7%
(L388
0,302 2O.95E
FL389 67.8%
(1 0.993 76 9/%
0.169 78.9%
EL:00 77.8%
Table 27: (llness) Evaluate MSE of MIA with different pretrained models on multi-variate time series forecasting (MTSF).
Model
Beselite
Leswt:
2.gg
A 1.0
5.5
10.9.
2.%
A - 1.2
$15
10%
2.%
A L5
5.%
10%
0L095
90.6%)
MLP-Miser
0.970
0,878
0.000
77,
77.2%
01033 64.2%%
IUIRO K2.5%
M95
0.140 8K.9%
0.130 AX.%%
2%
II 104 54.4%
OUTRY 66.1%
0.208
82.5%%
MLP
1,265
5%
0J885 01. 216
OJI84 50.9%
OLBR (2..55
0.812
B.186 50.175
0.186
79.35%
00.050
50.55
0.896 50.3%%
0.132 65.355
0.131
0.121 63.7%%
0.195
0.18I 79.5%
2%%
45.6%
0.120
77,8%)
LSTM
1.596
IM027 45.6%
01076 45.6%
0.122 59.6%
0.128 50.6%
0.126 77.2%)
0,172
77.2%
10%
15.6645.6%%
BJI76 45.6%
1887445.69
0.123 59.6%%
0.123 SM66F)
0.121 SH.6%
0.179 77.22
0.176
70.075
0.143 67.0%
0.110
0.112
OL341 67.3%
0.545
79.0%
54.4%
0.198
2%
0.101 38.6%
OLI6S 55.0%
0.20L 63.7%
RNN
1.329
1%
0.814335.00 (96
6.166 SUUYS
11161 52.05%
(1203 IS 195
6.194
0.819338.00 096
38,005
IL160 50.655
0.158 01 07.056
82.0%
01203 15
0. 071.55
6.194.26%
ThandhenenNomaal
0,087 55.00
0.122
0,118
00
T6-075
U.103 4%
10%
0.064 66.758
0.078 55.6%
0,878 54.4%
0 67.
0.114 67.8%%
0,130 66.1%
0.165 50. 19.155
0.159
79.5%
7..57 99.5%
0.115 54.4%%
0L344 69U0%%
0.157 84.2%
Tranformaahaaliing
0:122
50.3%)
0.143 (9L.15
0.842
61.555
55)
51.3%
50.05%
0.155 07.4%%
0.157
53
8..9%
105 66.1%
0.146 73.75
0.319 90.8%
TrastornerCorv
3.17
0,059 63.256
0.303 60U806
3.146 74.9%
0.143 72.5%%
0 73 79-035
0,242 78.9%
10%
0.806 63.25%
0.097 61.4%
0,805 56.7%
1L137 10.5%%
0,139 70.255
0L137 67.8%
0.215 76
0.210 78.9%
0.206 77.8%
Table 28: (Weather) Evaluate MAE of MIA with different pretrained models on multi variate time series forecasting (MTSF).
Medel
Braclin:
10%
10.%
10%
0:360 71
01640 7800%
0.506 81.8%
MLPMMier
0.793
5%
0.367 71.5%
0,389 68
DU445 75.0%
0,436
76.555
0.571 54 54.5
0.560 53.1%
10%
0.162 700
0.961 7RUPS
0,347 66.1%
1L,39 T7-3%
01436
77.3%%
W.455 74.5%%
0.563 55 0%%
0.561 8399
IUS45 K2.0%
0.465 70.055
0.47%
10%
0.401 77.356
70.4%
0,30M 63.0%
O1478 7AU8T
0.475
79.2%
1LA61 70.3%
11506
0.564 26,17
2.%
0.432 72
8.516 77.2%
0,645
53.6%
LSIM
0.818
0/432
72
B-431 71.496
0L512 77.2%%
0.536 76,
0L645
0L644 50.0%
72.6%
0.43272.3%
0.428 76.4%
0,515 77.0%%
0.51677.11%
0.511 76..%%
83.6% 0.546
0.646 83.6%
53.2%%
1%
51.776
0,514
73.655
GRU
1.385
0,419 52.555
8.414 10.65
0L516 63.8%
0.512
0,65S
24.3%)
IU652 12.85
10%
11419 52.4%
0,477 52.1%
0,410 -89.0%
01515 13..7%
0.513 63.3%)
0.586 59.6%
0.654 74
06652 73.7%
BLGA6- 70L0%
0,188
ILANK 72.6%
87%
RNN
0.777
74.8%
71.2%
0 75.375
0 75.356
83.7%%
0.363
04475
72.0%
0.567 57.9%
2%%
0.412 68.4%
DU49L 73.1%
0,510 90,0%%
TrandeeenNomall
0.591
0:412 05 256
0-009
66.9%
OU691 73..%
689
72.6%
(1616 TH 195
0.614
10%
0.40 05.055
0.407 65.45
0,399 65.2%
0.485 72.65
0.486
72.01%
0..997 71.5%%
0,609 7%
0.609 79.07%
0L601 77.7%
TransfemnsPalling
0,644
0,392 70:055
0.479
0.474
10%
0.367 05.055
0.385 60LIFF
0,382 68.2%
0.474 750,1%5
0,433 P6.01S
0.469 75.5%%
0.604 52 835
0U683 52.775
0.600 $2.6%
012944 66,
OU475 74.5%
0.558 83.3%
Tranffrmanruus
0:393
66.
74.31% 0.475
0.596 8..6%%
0,10 (16
0,384
0.868 71.4%
0.466
7005%
51.155
24

---

Under review as conference paper at ICLR 2023
Table 29: (Weather) Evaluate MSE of MIA with different pretrained models on multi-variate time series forecasting (MTSF).
Midiel
Baaciin
2.%
3
10.%
2%
À m 1.2
5%
101
2.%
10%
0.188
10.1%
20:216
POUPS
0.34M
00.34
0.233
74.5%
0.375
0,359 $2.0%%
0.887 20-005
IL263 T8
8.013
MLP
1.455
01187 71.9%G
0.180 61.0%
11265 70,655
0.255
23-116
0.405 96.6%)
0.395 54.155
10.50
71.056
70.4%
0.363
0,204
F2.35%
0,203 $7.20%
83.6%
LSTM
3,500
5%
0,204 72.858
71.851
0L283 27.2%
0.292
06.715
0.456 50
0.455
S0.075
10.%
61.75 16.005 72.6%
0.304 72.3%
0,312 70.4%
1120% 77.055
0.281
77.1%
01.209 76.51%
0.456 83.8%
0.456 0445688449
IL611 KL2%
0.290
52.1%%
0.189
0.290
70457
2%
0.371 73.356
0.249 T7.675
0,302 52 975
RNN
1.591
(1123 74.8%
0.168 71.25
0.252 70.355
0.234
15.555
0.392 83.7%)
0.371 70:455
10%
0.876 74.7%
0.176 73.4%
0.115 65.4%
0.256 75667%
01234 72.07%
0.410 N3.7%
0.399 85.E%
I.373 77.7%%
TrancfenneNommal
0,150 256
0U272
TO
0442
10%
0.387 60,076
B.387 G6.076
0,382 65.2%%
0.246 72,475
0.268
72,014
01243 71.9%
0.423 875
0.423
78
77.74%
D.178 70.1%
OL263 7669%%
0-418 83.5%
5
0.129 66.9%
0.261 74.5%
0.410 82.3%)
ConddeemenCore 1.999
%
0.177 pe 055
8.178 66-11%
112611 71.015
2:9
71.7%
01410 52-995
0-407 81.6%
10%
0.176 65.055
B.375 60LERE
0.171 63.9%
0.256 72.475
0.254 72.076
0,248 70.5%
0,415 51
0.404 51.155
0,385 79.9%
Table 30: Compare Lo-no fo-norm localized perturbation to ε2-norm perturbation (computed by the algorithm [2]) on the MSE between the original forecast and the perturbed forecast. The table reports the relative improvement of the fo-norm perturbation over the f2-norm perturbation (averaging among 128 randomly selected samples). The positive value indicates that our fo-norm perturbation outperforms fo-norm perturbation. For fairness, the tn or &g norm of the perturbation is restricted to be no larger than Bx the average value among the to &2 norm of all the testing samples. Values in tables are calculated as (MSE,, M //SE
Attack Rate (3)
Model
10
20 %
30
40 %
50 %
MLP-Mixer
+769.9
+89.5 %
+73.3 %
+12.3 %
+53.1 %
GRU
+2.5 %
-1.6 %
-8.3 %
-3.3 %
-6.5 %
LSTM
+23.1 %
+15.1 %
-33.5 %
-14.8 %
+2.0 %
MLP
+265.0 %
+376.3 %
+211.6
+114.1 %
+58.3 %
Table 31: (Exchange) Time series attack. Difference of MSE between and lz attack.
Attack Rate (3) %
Model
10 %
20 %
40
50 %
MI-P-Mixer
+ +1000.1 %
+332,2 %
+114.7 %
+131.7 %
+84.6 %
GRU
-48.6
-45.1
-39.7 %
-31.7 %
-20.9 %
LSTM
-8.8 %
-28.3 %
-24.9 %
-16,2 %
-6.6 %
MLP
+528.9
+101.8
+36.9 %
+8.8 %
+6.2 %
Table 32: (Traffic) Time series attack. Difference of MSE between to and attack.
Attack Rate (3)
Model
10
%
50
MLP-Mixer
+3310.1 %
+841.8 %
+328.5 %
+317.7 %
+205.5 %
GRU
+147.0 %
+28.2 %
+7.3 %
-4.7 %
+21.6 %
LSTM
+239.0 %
+66.8 %
+14.9
+2.5
-2.8 %
MLP
+1760.4
+145.7
+38.4 %
+7.5 %
-7.6 %
25

---

Under review as conference paper at ICLR 2023
Table 33: (DistalPhalanxTW Lmask 15%) 0.01
DistalPhalanxTW
MiddllePhalanXTW
ProximalPhulaxTTK
Merric,
Trnace
5 %
10 %
5%
10 %
15
5%
10%
15
5 %
1.0
-3.0
0.0 %
ACC
10
%
0.0%%
%
-1.0 %
-1.0%
-3.0%
15 %
1.0%
3.0% %
0.0%%
-4.0
-3.0 %
-40 %
-1.0%
1.0%
3.0 %
%
MLP.
ACC
10%
-2.0 S
-2.0 %
0.0 %
-1.0% %
-1.0%
-1.0% %
1.0
5%
-1.0 %
2.0
0.0 %
MLP
10 10%
%
0.0
%
0.0
15%
1.0%
1.0 %
3.0
2.0 %
4.0 %
2.0
1.0 %
0.0
0.0 %
5%
-2.0% %
-1.0%
0.0 %.
ResNet-18
ACC
10
3.0
0.0 %
%
0.0 %
0.0 %
1.0
15%
-1.0% %
-3.0%
-1.0% %6
0.0
1.0%
1.0%
-1.0%
-2.0 %
-1.0%
Table 34: (DistalPhalanx TW Lmask = 15%) = 0.02
Dista/PhallnXTW
MiddiePhalanxIIN
ProcimulPhulanx 10%
Model
Metric, 0,020
Linaak
10%
15%
5%
10
15
5%
5%
-1.0
0.0 %.
0.0 %.
ACC
0.0%
13 %
5.0 %
6.9 %
3.0 %
0.0 %
-1.0 %
-2.0 %
0.0 %
1.0 %
2.0 %
MLP-Mixer
ACC
10%
1.0 %
0,0 %
0.0 %
2.0 %
1.0 %
-1.0 %
15%
5%
0.0 %
2.0 %
0.0 %
ACC
2.0
15 %
0.0 %
2,0 %
3.0 %
2.0 %
3.0 %
3.0 %
0.0 %
0.0 %
0.0 %
ResNet-18
ACC
10 %
5.0 %
1.0%
-1.0
1.0 %
0.0 %.
0.0 %
Table 35: (DistalPhalanxTW Lmask 15%) = 0.03
DistalPhalanxTW
MiddlePPhulanxTV TW
Prosimu/PhalaxTTV
Metric,
Lmunk
10%
5%
10% %
15
5%
10% %
15 %
5%
-2.0 S.
0.0 %
1.0%
FCN
ACC
-2.0 %
%
-1.0
-3.0 %
-3.0 %
-2.0%
15 %
2.0 %
3.0
2.0%
-2.0 %
-3.0 %
-2.0%
-1.0% %
2.0
2.0 %
0.0
1.0 %
2.0
MI.--Mixer
ACC
10 %
0.0 %
1.0
-1.0 %
1.0%
1.0 %
0.0 %
0.0%
5%
0.0 %
2,0 %
0.0 %
15 %
1.0 %
3.0 %
4.0 %
0.0 %
L0%
1.0 %
-1.0 %
0.0
-1.0%
%
0.0
R8sNet-18
ACC
10 %
0.0 %
-1.0%
0.0 %
10%
0.0 %
10%
15.%
-1.0% S
-1.0%
0.0
0.0 %
1.%%
1.0%
0.0
0.0%
0.0
Table 36: (DistalPhalanxTW Lmask 15%) 0.04
DistalPhalanxTW
MiddlsPhalanxTTK TW
ProximalPhalanxTWV
Model
Metric,
Lanek
15%
5%
2.0 %
-1.0 %
0.0 %
ACC
%
2.0 %
-2.0 %
-2.0 %
-2.0
15 %
2,0 %
1.0%
0.0 %
-1.0 %.
-2.0 %
.2.0%
-3.0 %
1.0
1.0%
MLP.Mixeer Mixer
ACC
10 %
0.0 %.
1.0 %
0.0 %
-1.0 %
0L0 %
0.0 %
5%
1.0 %.
1.0
1.0% %
ACC
15 %
1.0 %
3.0 %
4.0 %
1.0
3.0 %
3.0 %
-1.0% %
0.0 %
0.0 %
1.0%
-1.0%
ResNet-18
ACC
10 %
0.0 %.
0.0 %
0.0%
1.0 %
1.0 %
1.0
26

---

Under review as conference paper at ICLR 2023
Table 37: (DistalPhalanxTTW TW Limask = 15%) = 0.05
DisralPholanxTW
MiddlePhallaxxTV TW
ProximalPhalanx TW
Model
Metric, 0.050
Imnoc
10
15%
5%
10%
15
5%
10%
15
5 %
1.0 %
0.0%
1.0%
FCN
ACC
10 %
0,0
1.0
-1.0 %
-4.0 %
-1.0 %
0.0
5.0
4.0%
2.0
0.0
-1.0
-2.0%
0.0
2.0%
1.0%
5%
-1.0 %
-2.0 %
1.0
MLP-Mixer
ACC
1.0
%
-2.0
-1.0 %
-1.0
-2.0
15 %
2.0 %
%
0.0
3,0 %
1.0%
0.0 %
0.0 %
-1.0 %
2.0 %
1.0%
2,0
0.0
MI.P
ACC
10 %
4,0 %
1.0 %
-1.0 %
1.0%
0.0 %
0.0 %
5%
-2.0 %
0.0 %
-1.0 %
ResNet-18
0.0
15 %
1.0 %
1.0 %
0.0
1.0 %
1.0
1.0 %
-1.0 %
0.0
-1.0%
27

---

Under review as conference paper at ICLR 2023
Modell tyge ML Mixiee
Model type
ML P-Mixer
Model type
All P-Mixer
1.5
imputed original.
2.0
imputed ginal
imputed
1.0
original
1.0
0.0
0.5
0.5
0.0
-155
-1.5
20
40
60
A0
20
40
60
aC
20
40
60
80
Model tyge MLP-Mixer
Model type MLP-Mixer
Model type MLP-Mixer
4
imputed original
imputed ginal
1.5
imputed
3
2
1.0
original
0.5
0.0
1-
0
0
-1.0
-1
-1.5
80
Model type SAISS
Model tyge SAITS
Model type SAITS
1.5
Imputed original
2.0
Imputed ginal
imputed
1.0
1.5
original
0.5
1.0
0.0
-0.5
0.0
-0.5
-1.5
-1.0
-2.0
-1.5
40
60
80
40
60
80
60
80
Model type SAITS
Model Lype SAITS
Model lype SAITS
Imputed original
3
Imputed ginal
1.5
imputed
3
2
ariginal
0.5
1
0.0
0
0.5
0-
-1.0
-1
1.5
80
80
Model BRITS
Model tyoe BRITS
Model type BRITS
1.5
Imputed original.
2.0
Imputed ginal
imputed original
1.0
1.5
1.0
0.0
-0.5
0.0
-0.5
-1.5
-2.0
-1.5
40
60
80
40
60
80
40
60
80
type
type
type
4
imputed original
imputed
1.5
imputeo or originel
3
2
ginal
0.5
0.0
0-
-1.0
-1
-1.5
10
60
80
10
80
80
Model type Transoomerr
Model type Transfomeer
Model type Transtormer
1.5
Imputed original
2.0
1.5
Imputed ginal
imputed originel
0.5
1.0
1.0
0.0
-1.0
-1.5
-1.0
1.5
40
60
80
20
40
60
80
40
60
80
Model type Tansformer
Model type Transformer
Model type Transforme
imputed original
3
Imputed ginal
1.5
Imputed
3
2
1.0
iigiell
0.5
1
0.0
0
0.5
-1.0
-1.5
80
Figure 4: (Traffic) Imputation quality with different imputation models. We set Lmask = 10%. Original and imputed refer to the original time series and the imputed time series respectively.
28


Reviews:
[{"review_id": "qui2m7AdqB", "reviewer": "Reviewer_SiPr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes Masking Imputing Aggregation (MIA), a framework based on masking and aggregation, to achieve certified robustness for time series forecasting models against temporally localized $\\ell_0$-norm-bounded perturbations. For the masked time steps, an imputation model is trained to fill in and then new time series is fed into the old model for prediction. When consensus is achieved, certified robustness of the pipeline is achieved. Experiments on Exchange Rate, Traffic, UCI Electricity, DistalPhalanxTW, MiddlePhalanxTW, ProximalPhalanxTW against two baselines extended from previous work (Derandomized Smoothing and Randomized Ablation) demonstrate the effectiveness.", "summary_of_the_review": "Due to the limited technical novelty and rough writing, the submission may be not ready for publication at the current stage. However, I appreciate the novel application of masking and imputation framework in the certified robustness of the time series domain.", "strength_and_weaknesses": "Strengths:\n- A practical and rigorous certification framework with significantly better performance than baselines.\n- The framework is novel since no prior work achieves certified time series prediction against $\\ell_0$ predictions and no prior work can protect base models without retraining in this domain.\n\nWeaknesses:\n- The idea is relatively straightforward: it is like an application of [1,2] in the time series domain. The unique point of imputation model is spiritually similar to [3].\n\n- The writing quality needs further improvements. For example, many typos exist even on the first page:\n  1. First para: $ell_0$-robustness -> $\\ell_0$-robustness\n  2. Second para: \"Athalye et al. (Athalye et al., 2018)\" repeated - you can use `\\citet{}`\n  3. Second para: \"the concept of certified defense is proposed. guaranteeing the robustness for each prediction mathematically.\" -> additional \".\"\n\nIn later sections:\n\n4. Eqn. (1): the first line $x_{1:t_0} + \\delta, \\text{subject to} \\|\\delta\\|_0 \\le L\\_{adv}$ does not contain the temporal localization constraint, so it is not equal to the second line.\n\n5. Eqn. (3): $G(\\mathbf{x}^{(m)}\\_{1:t_0})$ -> $G(\\mathbf{\\hat x}^{(m)}\\_{1:t_0})$\n\n6. Remark 1, last sentence: the inference cost could not be 0. It should at least be 1.\n\n7. What is the $\\sigma$ value used in practice in Eqn. (7)? It is not specified in later sections. Also, why Gaussian augmentation is needed here? Unlike other randomized smoothing work, here no Gaussian augmentation is used in the inference time, so the use of Gaussian augmentation in the training time may induce domain mismatch and needs further justifications.\n\n8. Eqn. (9): I think the guarantee of RHS should be $2 * (k+L_{adv}+1)$ instead of $k + L_{adv}+1$, since both terms of LHS can change by 1 for each step perturbation leading to gap reduced by 2.\n\n9. Section 5, first paragraph: MiddlePhalanxTW ProximalPhalanx -> MiddlePhalanxTW, ProximalPhalanxTW\n\n[1] PatchGuard: A Provably Robust Defense against Adversarial Patches via Small Receptive Fields and Masking. USENIX Security 21\n\n[2] PatchCleanser: Certifiably robust defense against adversarial patches for any image classifier. In 31st USENIX Security Symposium (USENIX Security 22)\n\n[3] Denoised Smoothing: A Provable Defense for Pretrained Classifiers. NeurIPS 2020", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "TE5-Yn5B_j6", "reviewer": "Reviewer_k4J1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a method for detecting $l_{0}$ norm perturbation in time series. The key idea is to do multiple masking over the time series by sliding windows, impute the masked areas individually, and measure the disagreement of the predictions using the imputed time series. A disagreement is an indicator of perturbation. The method is a flexible framework and can be applied to different base predictive models.", "summary_of_the_review": "The paper presents a method for dealing with a new problem on detecting $l_{0}$ norm perturbation in time series. The method is a flexible framework to different forecasting/classification methods. The practical significance of the studied problem requires some justification. The presentation of the method design should be improved for clarity. The current paper is not easy to evaluate the validity of the proposed method. See details in strength and weakness.", "strength_and_weaknesses": "Strength\n1. The paper studies a new problem and introduces a three-step approach for detecting $l_{0}$ norm perturbation in time series, which is easy to apply to different time series predictive models.\n2. The paper provides some theoretical analysis about the robustness of the proposed method, and comparison with randomized smoothing defenses.\n3. The experimental results demonstrate the effectiveness of the proposed method by comparing with a couple of baseline methods.\n\nWeakness\n1. The practical significance to detect $l_{0}$ norm perturbation in time series is obscure. The paper could provide some examples to illustrate why detect $l_{0}$ norm perturbation in time series is practically useful, compared to $l_{2}$ norm perturbation.\n2. The presentation of the technical development is hard to follow. Many equations are given without sufficient description on their intuition. For example, Eq. 1, it is unclear why the norm $L_{adv}$ is used as the time point index and what is the relationship between $\\delta$ and $\\delta_{t}$. Eq. 4, it is unclear on why should all M predictions be consistent instead of allowing some tolerance. Some theoretical analysis is desired. Similarly, Eq 5, 7, etc., are presented without introducing intuition.\n3. Below Eq 1, the authors claim in most cases short-term events should not have large impacts on the long-term outcomes. Then the question is why resistance to the short-term perturbation is essential.\n4. The proposed method requires the masks to cover arbitrary temporally-localized perturbation of $L_{adv}$. It is better to provide some guidelines on how to ensure this requirement is satisfied.\n5. In proposition 1, to conclude the forecast/label cannot be changed, it is better to quantify the change and define what is considered unchanged quantitatively.\n6. In Remark 3, it is better to also describe the impact of certificate only for $f_{MIA}$ instead of $f$.\n7. It is good to see the comparison to randomized smoothing defenses. It is better to further justify on why this comparison is important.\n8. Is there any discussion on whether the proposed method useful for both univariate and multivariate time series?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8VKHlLxRyV", "reviewer": "Reviewer_F33Z", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a plug-and-play method for robust estimation of perturbed data in time series identification and prediction. The proposed method assumes perturbations in a part of time series data and performs estimation by aggregating multiple masked and imputed time series data outputs. Experimental results with multiple data sets and multiple machine learning models are reported.", "summary_of_the_review": "Overall, the reviewer leans toward rejecting this paper. A response from the authors regarding the above weaknesses could improve the score.", "strength_and_weaknesses": "Strengths\n1. Extensive experimental results are reported using multiple methods such as MLP, RNN, CNN, Transformer, and MLP-Mixer on multiple datasets of time series data.\n\nWeaknesses\n1. The authors should proofread the manuscript repeatedly. There are examples found in Section 1.\n  - ell_0-robustness -> \\ell_0-robustness\n  - guaranteeing the robustness for each prediction mathematically. -> The first letter of the word at the beginning of a sentence should be capitalized.\n  - BDPA should be first introduced by its correct name, not the abbreviation.\n  - The authors should break paragraphs before describing the first contribution point.\n1. Mathematical explanations are also hard to follow.\n  - In equation (1), what does it mean that the delta index decreases as the x index increases?\n  - Most critical point is that the definition of the mask in equation (2) is unclear; the reader is ultimately forced to imagine it. Surprisingly, there are three different capital M's in equation (2) and Algorithm 1: Roman, Italic, and Calligraphic Ms. As for Roman's M and Calligraphy's M, their definitions cannot be understood. Since the purpose is to mask the time-series data, the reviewer can only imagine that Ms are series data, with some being 1 and others being 0; the indexes for the value 0 are not stated clearly. Some figures showing mask patterns would be helpful.\n1. (Yoon et al., 2022) is cited but not compared to the proposed method in the experimental results.\n1. The second paragraph of the introduction makes the point about heuristic defense methods, arguing that a certified defense is needed to end the cat-and-mouse game. However, the algorithm in this paper also assumes that only a local part of the time series data is perturbated. Would the cat-and-mouse game continue even with the proposed method when other types of attacks are considered?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject