Original: MIA: A Framework for Certifiably Robust TimeSeries Classification and Forecasting Against Temporally-Localized Perturbations
Revised: MIA: A Framework for Certifiably Robust TimeSeries Classification and Forecasting Against Temporally-Localized Perturbations

Original: Anonymous authors Paper under double-blind review
Revised: Anonymous authors Paper under double-blind review

Original: Abstract
Revised: Abstract

Original: Recent literature demonstrates that times-series forecasting/classification are sensitive to input perturbations.
Revised: Recent literature demonstrates that times-series forecasting/classification are sensitive to input perturbations.

Original: However, the defenses for time-series models are relatively under-explored.
Revised: However, the defenses for time-series models are relatively under-explored.

Original: In this paper, we propose M asking I mputing A ggregation (MIA), a plug-and-play framework to provide an arbitrary deterministic timeseries model with certified robustness against temporally-localized perturbations (also known as ℓ 0 -norm localized perturbations), which is to our knowledge the first ℓ 0 -norm defense for time-series models.
Revised: In this paper, we propose M asking I mputing A ggregation (MIA), a plug-and-play framework to provide an arbitrary deterministic timeseries model with certified robustness against temporally-localized perturbations (also known as ℓ 0 -norm localized perturbations), which is to our knowledge the first ℓ 0 -norm defense for time-series models.

Original: Our main insight is to let an occluding mask move across the input series, guaranteeing that, for an arbitrary localized perturbation there must exist at least a mask that completely remove out the perturbation, so that our prediction on this masked series is uninfluenced. Remarkably,
Revised: Our main insight is to let an occluding mask move across the input series, guaranteeing that, for an arbitrary localized perturbation there must exist at least one mask that completely occlude the perturbed area, so that the prediction on this masked series is certifiably unaffected.

Original: MIA is high-availability as it still works even if we only have  query access to the pretrained model.
Revised: MIA is flexible as it still works even if we only have the query access to the pretrained model.

Original: Moreover, as there is no dedicated defense against ℓ 0 -norm perturbations for time-series models, we specifically adapt two matrix-based defenses to time-series models for comparison.
Revised: To further validate the superior effectiveness of MIA, we specifically compare MIA to two baselines extended from prior randomized smoothing approaches.

Original: Extensive experiments show that MIA yields stronger robustness as well as practicality.
Revised: Extensive experiments show that MIA yields stronger robustness.

Original:  Introduction
Revised:  Introduction

Original: Time series forecasting/classification (TSF/TSC) have been widely applied to help businesses make informed decisions and plans (Miyato et al., 2017;
Revised: Time series forecasting/classification (TSF/TSC) have been widely applied to help businesses make informed decisions and plans (Miyato et al., 2017;

Original: Zhou et al., 2019; Schlegl et al., 2019; Park et al., 2018).
Revised: Zhou et al., 2019; Schlegl et al., 2019; Park et al., 2018).

Original: However, a wide range of literature demonstrate that time-series models are vulnerable to adversarial input perturbations (Connor et al., 1994;
Revised: However, a wide range of literature demonstrate that time-series models are vulnerable to adversarial input perturbations (Connor et al., 1994;

Original: Gelper et al., 2010; Ding et al., 2022; Yang et al., 2020; Dang-Nhu et al., 2020; Oregi et al., 2018; Han et al., 2020), e.g., an elaborately designed imperceptible perturbation could control the prediction.
Revised: Gelper et al., 2010; Ding et al., 2022; Yang et al., 2020; Dang-Nhu et al., 2020; Oregi et al., 2018; Han et al., 2020), e.g., an elaborately designed imperceptible perturbation could control the prediction (Karim et al., 2020;

Original:  So far related literature is mainly focusing on detecting the outliers (Ruff et al., 2018; Yairi et al., 2017), the adversarial robustness of time-series models is relatively under-explored, especially ℓ 0 -norm robustness, e.g., (Yoon et al., 2022) only explore the ℓ 2 -norm adversarial robustness for probabilistic forecasting models.
Revised: Fawaz et al., 2019). So far related literature is mainly focusing on detecting the outliers (Ruff et al., 2018; Yairi et al., 2017), the adversarial robustness of time-series models is relatively under-explored, especially ℓ 0 -norm robustness, e.g., (Yoon et al., 2022) only explore the ℓ 2 -norm adversarial robustness for probabilistic forecasting models.

Original: In the present work, we investigate the ell 0 -robustness against localized perturbations, as we notice there already exists corresponding powerful attacks (Yang et al., 2022).
Revised: In the present work, we focus on the robustness against temporally-localized perturbations, as we notice there already exists corresponding powerful attacks (Yang et al., 2022).

Original: In terms of heuristic  defenseapproaches, they can yield better empirical robustness but lack robustness guarantees.
Revised: Generally, defenses can be divided into two types, heuristic defenses and certified defenses. Heuristic defense can yield better empirical robustness but lack robustness guarantees.

Original: According to the experience from the heuristic defenses for the computer vision tasks (Athalye et al., 2018; Carlini & Wagner, 2017; Athalye & Carlini, 2018) (e.g., Athalye et al.(Athalye et al., 2018)
Revised: From the heuristic defenses on image classification (Athalye et al., 2018; Carlini & Wagner, 2017; Athalye & Carlini, 2018) (e.g., Athalye et al.

Original:  leverage BPDA technique to  circumvent most prior heuristic defenses successfully), we have reason to worry that these heuristic defenses would be invalidated when confronted with the newly-designed adaptive attacks.
Revised: (2018) leverage Backward Pass Differentiable Approximation technique to successfully circumvent almost all the heuristic defenses at that time), it is concerned that these heuristic defenses would be useless when confronted with the newly designed adaptive attacks.

Original: To end such a ”cat and mouse” game between the adaptive attacks and the heuristic defenses, the concept of certified defense is proposed. guaranteeing the robustness for each prediction mathematically.
Revised: To end such a ”cat and mouse” game between the adaptive attacks and the heuristic defenses, the concept of certified defense is proposed, with unbreakable robustness certificates.

Original: In terms of certified defenses, they can produce robustness certificates but often require the user to train the base model from scratch, e.g., (Yoon et al., 2022; Li et al., 2020; Cohen et al., 2019) need to train the  model on Gaussian augmented data for an acceptable performance, since these defensesare often ineffective on the natural models.
Revised: In terms of certified defenses, they can produce robustness certificates but often require the user to retrain the base model from scratch ( Yoon et al. ; Li et al. ; Cohen et al. (2019) retrain the base model on Gaussian augmented data), because these defenses often perform poorly on the naturally-trained models.

Original: Such  requirement may bring additionaluncertainties and challenges in real-world deployments.
Revised: Such retraining requirement could bring additional challenges in the realworld deployments.

Original: Moreover, so far almost all the certified defenses are focusing on matrix-based data (e.g., image), the defenses for sequence-based data are quite terra-incognita.
Revised: Moreover, so far almost all the certified defenses are focusing on matrix-based data (e.g., image), the defenses on sequence-based data are quite under-explored.

Original: G ( · ) .
Revised: G ( · ) .

Original: We classify the imputed series with the pretrained model.
Revised: We classify the imputed series with the pretrained model.

Original: If the predictions of all the imputed series are Class 0 , MIA will return Class 0 with the robustness guarantee that the output is clean, otherwise MIA will return Abstain .
Revised: If the predictions of all the imputed series are Class 0 , MIA will return Class 0 with the robustness guarantee that the output is clean, otherwise MIA will return Abstain .

Original: To address these issues, in this paper, we propose M asking I mputing A ggregation (MIA), a plugand-play framework for all the TSF/TSC deterministic models to provide robustness guarantees against temporally-localized perturbations.
Revised: To address these issues, in this paper, we propose M asking I mputing A ggregation (MIA), a plug-andplay framework to arm TSF/TSC deterministic models with robustness guarantees against temporallylocalized perturbations.

Original:  MIA  onlyrequires an imputation modeladditionally for imputing the masked series, which  can be trained in an unsupervised manner.
Revised: All MIA needs is only an imputation model for recovering the masked areas, which we can easily train in an unsupervised setting.

Original: Specifically, MIA works as follows: 1) masking: MIA first generate a list of masked series via moving a mask across the input series, with the guarantee that there is no temporally-localized perturbation (w.r.t. ℓ 0 -norm constraint) can influence all the masked series simultaneously; 2) imputing: MIAthen imputes the masked series with an imputation model; 3) aggregation: MIA will return the result only if the pretrained model outputs the same prediction on all the imputed series.
Revised: MIA works as follows: 1) masking: MIA first generate a list of masked series via sliding a mask through the input series; 2) imputing: MIA imputes the masked series with our imputation model; 3) aggregation (checking agreement): if the pretrained model returns the same class for all the imputed series, MIA will return the this class. Otherwise MIA will alter the possible temporally-localized perturbation.

Original: Finally, we can guarantee that  the final output must equal the prediction on the uninfluenced masked series.
Revised: With the above three steps, we can guarantee that all the predictions from MIA is clean.

Original: Furthermore, we compare MIA to  derandomized smoothing (Levine & Feizi, 2020a) and randomized ablation (Levine & Feizi, 2020b), two randomized smoothing based defenses that are firstly adapted to time-series models by us.
Revised: Furthermore, we compare MIA to two ℓ 0 -norm certified defense baselines, derandomized smoothing (Levine & Feizi, 2020a) and randomized ablation (Levine & Feizi, 2020b), highlighting the superior effectiveness and practicality of MIA over other certified defenses.

Original: The contributions are: 1) We propose MIA, a plug-and-play framework to produce robustnesscertificates against temporally-localized perturbations for both TSF and TSC deterministic models, which is to our knowledge the first ℓ 0 -norm certified defense for time-series models.
Revised: The contributions are: 1) We propose MIA, a plug-and-play framework to arm an arbitrary TSF/TSC model with certified robustness against temporally-localized perturbations, which is to our knowledge the first ℓ 0 -norm certified defense in time series domain.

Original: We propose  masked training, a  training algorithmspecialized for  the imputation models of MIA, which exploits
Revised: We propose randomized masked training, a specialized training algorithm for training the imputation model of MIA, to further boost the MIA performance. We compare MIA to two baseline methods on

Original: the nature of consecutive masked series.
Revised: practicality and robustness.

Original: Experiments validate that the imputation models trained by masked training can yield higher robustness. We firstly adapt two matrix-based certified defenses (used in vision) to the time-series defenses.
Revised: In terms of practicality,

Original: In comparison, MIA is more practical because MIA can be directly applied to the pretrained models, while the other two defenses need to train the base model over noisy samples in a supervised setting. Extensive experiments  validate that MIA can offer stronger robustness.
Revised: MIA is stronger as MIA can be directly applied to the pretrained models. In terms of effectiveness, extensive experiments on different datasets and models validate that superior robustness of MIA.

Original:  Related Work
Revised:  Related Work

Original: Heuristic defenses for time-series models.
Revised: Heuristic defenses for time-series models.

Original: Prior works on robust TSF/TSC can be divided into two general categories: traditional outlier-based approaches and deep learning approaches.
Revised: Prior works on robust TSF/TSC can be divided into two general categories: traditional outlier detection based approaches and deep learning approaches.

Original: The former is to filter the outliers in a statistical way, including k-Means clustering (Yang et al., 2017), one-class SVM clustering (Sch¨olkopf et al., 2001), Kalman filters (de B´ezenac et al., 2020) and support vector data description (Tax & Duin, 2004).
Revised: The former is to filter the outliers in a statistical way, including k-Means clustering (Yang et al., 2017), one-class SVM clustering (Sch¨olkopf et al., 2001), Kalman filters (de B´ezenac et al., 2020) and support vector data description (Tax & Duin, 2004).

Original: The latter leverages the strong representation ability of neural networks, including robust feature-based approaches (Guo et al., 2016; Yang &
Revised: The latter leverages the strong representation ability of neural networks, including robust feature-based approaches (Guo et al., 2016; Yang &

Original: Fan, 2022), reconstruction-based methods (Li et al., 2021; 2019; Xu et al., 2018; Schlegl et al., 2019), GNN-based methods (Zhao et al., 2020; Deng & Hooi, 2021), association discrepancy (Xu et al., 2022), LSTM-based methods (Hundman et al., 2018; Tariq et al., 2019).
Revised: Fan, 2022), reconstruction-based methods (Li et al., 2021; 2019; Xu et al., 2018; Schlegl et al., 2019), GNN-based methods (Zhao et al., 2020; Deng & Hooi, 2021), association discrepancy (Xu et al., 2022), LSTM-based methods (Hundman et al., 2018; Tariq et al., 2019).

Original: However, all of these  methods lack robustness guarantees, hinting that they could become invalid once the adversary designs a new adaptive attack to bypass them.
Revised: However, all of these empirical methods lack robustness guarantees, hinting that they will be meaningless once the adversary designs a new adaptive attack that breaks them successfully.

Original: For that reason, certified defenses are crucial since their mathematical robustness certificates are permanently valid.
Revised: For that reason, certified defenses are crucial since their mathematical robustness certificates are permanently valid.

Original: Certified adversarial defenses.
Revised: Certified adversarial defenses.

Original: In the field of image classification, there has been much work on the certified adversarial defenses, including randomized smoothing (Cohen et al., 2019; Salman et al., 2020), convex polytope (Wong & Kolter, 2018), CROWN-IBP (Zhang et al., 2019) and Lipschitz bounding (Cisse et al., 2017).
Revised: There has been much work on the certified defenses for robust image classification, including randomized smoothing (Cohen et al., 2019; Salman et al., 2020), convex polytope (Wong & Kolter, 2018), CROWN-IBP (Zhang et al., 2019) and Lipschitz bounding (Cisse et al., 2017).

Original: In particular, the certified defenses of the similar objective to our work ( ℓ 0 -norm robustness) include derandomized smoothing (Levine & Feizi, 2020a), randomized ablation (Levine & Feizi, 2020b; Zhang et al., 2020) and a series of mask-based defenses (Xiang & Mittal, 2021; McCoyd et al., 2020; Han et al., 2021; Xiang et al., 2021; 2022).
Revised: In particular, the certified defenses on ℓ 0 -norm robustness include derandomized smoothing (Levine & Feizi, 2020a), randomized ablation (Levine & Feizi, 2020b; Zhang et al., 2020) and a series of mask-based defenses (Xiang & Mittal, 2021; McCoyd et al., 2020; Han et al., 2021; Xiang et al., 2021; 2022).

Original: In stark contrast, the certified defenses for time-series models are rarely explored.
Revised: In stark contrast, the certified defenses for time-series data are rarely explored.

Original: To our knowledge, (Yoon et al., 2022) and (Li et al., 2020) are the only two defenses that produce ℓ 2 -norm robustness certificates, but a common downside is that they  require the base model to perform well over Gaussian augmented samples, as most natural models do not meet this requirement.
Revised: To our knowledge, (Yoon et al., 2022) and (Li et al., 2020) are the only two defenses that produce ℓ 2 -norm robustness certificates, but a common downside is that they additionally require the base model to perform well over Gaussian augmented samples.

Original: Therefore, we need to train thebase model  on noisy samples from scratch, imposing a large amount of additional training costs.
Revised: Since most natural models do not meet this requirement, the user is asked to retrain the model from scratch on noisy samples, which imposes a large amount of additional training costs.

Original:  Preliminaries and Problem Setup
Revised:  Preliminaries and Problem Setup

Original: Time series classification (TSC).
Revised: Time series classification (TSC).

Original: First we model the task of time series classification. Specifically, inputted atime series of length t 0  (denoted by x 1: t 0 =
Revised: We model the task of time series classification as: inputted a t 0 -length time series (denoted by x 1: t 0 =

Original: [ x 1 , x 2 , . . .
Revised: [ x 1 , x 2 , . . .

Original: , x t 0 ] ), TSC model returns the  classof the inputted series (denoted by f ( · ) : x 1: t 0 → y ).
Revised: , x t 0 ] ), TSC model returns the predicted class (denoted by f ( · ) : x 1: t 0 → y ).

Original: Time series forecasting (TSF).
Revised: Time series forecasting (TSF).

Original: Given “past” (denoted by x 1: t 0 ), the forecasting model returns the “future” (denoted by x t 

Revised: Given the “past observations” (denoted by x 1: t 0 ), the forecasting model returns the “future values” (denoted by x t

Original: +1: t 0 + τ ).
Revised: +1: t 0 + τ ).

Original: For ease of understanding,  we consider the classic and commonly studied short-term forecasting setting (Ke et al., 2017): forecasting a single  time point (not necessarily the next time point x t 0 +1 )
Revised: For ease of understanding, in this paper we focus on the classic and commonly studied short-term forecasting setting (Ke et al., 2017), which is to forecast a single future time point (not necessarily the next time point x t 0 +1 )

Original: f ( · ) : x 1: t 0 → R .
Revised: f ( · ) : x 1: t 0 → R .

Original: Since the problem of long-term forecasting f ( x 1: t 0 ) → x t 0 +1: t 0 + τ can be decomposed into τsub-problems of short-term forecasting, of which the i -th ( i = 1 , . . .
Revised: The short-term forecasting problem is sufficiently representative because the problem of long-term forecasting f ( x 1: t 0 ) → x t 0 +1: t 0 + τ can be decomposed into τ short-term forecasting subproblems, of which the i -th ( i = 1 , . . .

Original: , τ ) forecaster predicts thetime point x t 

Revised: , τ ) forecaster predicts the t

Original: + i .
Revised: + i -th time point.

Original: Definition 1 (Temporally-localized perturbation) .
Revised: Definition 1 (Temporally-localized perturbation δ

Original: Temporally-localized perturbation is  to perturb consecutive time pointsamong the input series w.r.t.
Revised: [ t adv +1: t adv + L adv ] ) . In a temporally-localized per- turbation attack, the adversary is allowed to perturb an arbitrary subsequence of time points w.r.t.

Original:  ℓ 0 -norm constraint .
Revised: the given ℓ 0 -norm constraint .

Revised: Let L adv be the ℓ 0 -norm constraint on the localized perturbation.

Original: The perturbed series is:
Revised: We can formulate all the perturbed series of L adv as follows:

Original: [Equation]
Revised: [Equation]

Original: subject
Revised: where δ t denotes the perturbation value added to the t -th time point. t adv + 1 and t adv + L adv refer to the starting point and the ending point of the perturbation respectively. The ℓ 0 norm of the perturbation is

Original: [Equation]
Revised: [Equation]

Original: where t adv + 1 and L adv denote the starting point and the ℓ 0 norm of the perturbation. Temporally-localized perturbationcorresponds to the attack scenario where the time-series data is influenced by a short-term event, which is  representative in  the real world, e.g., the stock price drops abnormally for a short period due to the CEO scandal.
Revised: Significance of temporally-localized perturbation. Temporally-localized perturbation is espe- cially representative in real-world scenarios. Temporally-localized perturbation can represent shortterm volatility and local anomaly , both of which can be regarded as the normal data added with temporally-localized perturbation.

Original: We believe that, in most cases, a short-term event should not have a large impact on the long-term outcomes, thus enhancing the resistance of time-series models to temporally-localized perturbations is essential.
Revised: The resistance to short-term volatility is important in long-term forecasting/prediction, in which the long-term value is considered unaffected by the short-term volatility. A typical example is the well-known investment philosophy, ”Value Investing” (Piotroski, 2000), where the ”intrinsic value” of a business is considered to be robust against short-term volatility. Moreover, the detection of local anomaly is practically useful in real-world scenarios. For instance, detecting a subsequent time interval of abnormal heart rate in electronic health records is a problem of local anomaly detection.

Original: To highlight the risk of temporally-localized perturbations, we propose an efficient generation algorithm and show how much a  perturbation can change the forecasting results of  undefended forecasters in Appendix. 
Revised: We can also adopt the method of detecting temporally-localized perturbation for detecting the abnormal network traffic for IoT Time-Series Data. Furthermore, to highlight the risk of temporally-localized perturbations, we empirically show how much a ℓ 0 -norm perturbation can change the output of an undefended forecaster in Appendix. We also compare the attacking performance of ℓ 0 -norm perturbation to ℓ 0 -norm perturbation, and the empirical results suggest that forecasting models might be more sensitive to ℓ 0 -norm perturbations.

Original:  Proposed Framework: Masking Imputing Aggregation 4.1 Pipeline Overview
Revised:  Proposed Framework: Masking Imputing Aggregation 4.1 Pipeline Overview

Original: MIA includes three steps: 1) masking; 2) imputing; 3) aggregation.
Revised: MIA includes three steps: 1) masking; 2) imputing; 3) aggregation (checking agreement).

Original: Masking.
Revised: Masking.

Original: Inputted a testing time series x 1: t 0 , we generate the masked series ˆ x 1: t 0 = x 1: t 0 ⊙ M ( t mask , L mask ) , where ⊙ M ( t mask , L mask ) is to remove out the values of the time period [ t mask +1 : min( t mask + L mask , t 0 )] .
Revised: We denote a mask by M [ u : v ] . x 1: t 0 ⊙ M [ u : v ] is removing out the values of x u : v (replacing the original values with zeros). Let L mask be the size of the mask. Given a series x 1: t 0 and the

Original: Specifically, fixing L adv and L mask , the masked series (denoted by ˆ x ( m ) 1: t 0 , m = 0 , . . , M ) are constructed as follows:
Revised: [Equation] [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: where α = L mask − L adv

Original: [Equation]
Revised: [Equation]

Original: 1 , M =

Original: [Equation]
Revised: [Equation]

Original: where α is the step size of our mask moving.
Revised: ℓ 0 norm of the temporally-localized perturbation L adv , we slide the L mask -size mask thorough the input series x 1: t 0 with the step size α = L mask − L adv + 1 . Then we obtain the masked series:x 1: t 0 ⊙ M [1+ kα :min( t 0 ,L mask + kα )] , k = 0 , . . . , ⌈ ( t 0 − L mask ) /α

Original: The number of the masked series is M + 1 .
Revised: ⌉ where α = L mask − L adv +(2) 2.

Original: Imputing.
Revised: Imputing.

Original: Denote G ( · )  a time-series imputation model (we will discuss G ( · ) later this section).
Revised: Let G ( · ) be a time-series imputation model (we will discuss the imputation model later this section).

Original: Our second step is to recover the masked values of ˆ x ( m ) 1: t 0 with G ( · ) , as follows:[Equation]
Revised: Our second step is to recover the masked area with G ( · ) , as follows:

Original: [Equation]
Revised: [Equation]

Original: This step is to make x ( m ) 1: t 0 approximate the normal time series, so that the pretrained model  perform similarly on these imputed series.
Revised: This step is to make x ( k ) 1: t 0 approximate the normal time series, so that the pretrained model could perform similarly on these imputed series.

Original: Aggregation.
Revised: Aggregation (Checking Agreement).

Original: We input x ( m ) 1: t 0 into the pretrained model f ( · ) .
Revised: We input x ( k ) 1: t 0 into the pretrained model f ( · ) .

Original: If the pretrained model’s ouputs on all x ( m ) 1: t 0 reach agreement unanimously, MIA (denoted by f MIA ( x 1: t 0 ) ) will output this unanimously approved label/prediction, otherwisewill output Abstain to alert that the input  might have been attacked by the temporally-localized perturbations.
Revised: If the pretrained model’s ouputs on all x ( k ) 1: t 0 reach agreement unanimously, MIA (denoted by f MIA ( x 1: t 0 ) ) will output this unanimously approved label/prediction, otherwise output Abstain to alert that the input series might have been attacked by the temporally-localized perturbations.

Original: [Equation]
Revised: [Equation]

Original: Abstain
Revised: Abstain

Original: Otherwise
Revised: Otherwise

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Discretization technique.
Revised: Discretization technique.

Original: We notice that, it is almost impossible for TSF models to forecast exactly the same value onM + 1 different imputed series, causing MIA to output Abstain all the time.
Revised: We notice that, it is almost impossible for TSF models to forecast exactly the same value on different imputed series, so that MIA would always output Abstain .

Original: To address this problem, we add a layer after the forecaster to discretize its outputs.
Revised: To address this problem, we add a layer after the forecaster for discretization.

Original: Given the pretrained model f ( c ˙) , the discretized forecaster (denoted by f dis ( · ) ) works as follows:
Revised: Given the pretrained model f ( · ) , the discretized forecaster (denoted by f dis ( · ) ) works as follows:

Original: [Equation]
Revised: [Equation]

Original: where ∆ is a discretization parameter to control the trade-off between the retained information after discretization and the success rate of achieving the agreement.
Revised: where ∆ is a discretization parameter that controls the trade-off between the retained information after discretization and the success rate of achieving the agreement.

Original: As ∆ increases, the discretized forecasts retain more information from the original forecasts while the agreement rate decreases.
Revised: As ∆ decreases, the discretized forecasts retain more information from the original forecasts while the agreement rate decreases.

Original: Remark 1 ( Mask Size L mask) .
Revised: Discussion on the mask size L mask .

Original: The onlynecessary requirement of masking (step 1) is to ensurethat for an arbitrary temporally-localized perturbation of L adv , there  exists at least one mask  thatoccludes the perturbation completely , implying that L mask must be no smaller than L adv .
Revised: The only requirement on Masking (Step 1) is to ensure for an arbitrary temporally-localized perturbation of L adv , there always exists at least one mask to occlude that perturbation . Thus a prerequisite is that L mask ≥ L adv .

Original: As L mask increases, the imputation quality decreases as we need to impute more missing values.
Revised: If we increase L mask , the imputation quality will decrease because we need to impute more missing values.

Original: Meanwhile,  the number of masked series linearly decreases with themask size, suggesting that large L mask helps save inference cost.
Revised: Meanwhile, if we decrease L mask , the number of masked series will decrease subsequently, then the inference cost is reduced.

Original: In the extreme case where L mask =
Revised: In the extreme case where L mask =

Original: t 0 , we have ˆ x (0) = ˆ x (1) = . . .
Revised: t 0 , all the masked series are equal to 0 1: t 0 .

Original: = ˆ x ( M ) = 0 1: t 0 . MIA always outputs f (  0 1: t 0 ) regardless of the input series, thus the inference cost is 0 .
Revised: MIA always outputs f ( G ( 0 1: t 0 )) regardless of the input series, then the inference cost is the smallest.

Revised: The practical implementation of MIA is showed in Algorithm 1.

Original: Remark 2 (MIA on Probabilistic Models) .
Revised: Remark 1 (MIA on Probabilistic Models) .

Original: A line of time-series forecasting models are the probabilistic models (e.g., DeepAR (Salinas et al., 2020)), which forecast over a random distribution (denoted by q [ y | x 1: t 0 ] ), as follows:
Revised: A line of time-series forecasting models are the probabilistic models (e.g., DeepAR (Salinas et al., 2020)), which forecast over a random distribution (denoted by q [ y | x 1: t 0 ] ), as follows:

Original: [Equation]
Revised: [Equation]

Original: Applying MIA to probabilistic models is challenging because the exactvalue of forecast is inaccessible, hindering our aggregation (Step 3).
Revised: Applying MIA to probabilistic models is challenging because the exact forecast values are inaccessible.

Original: Although we can utilize Clopper-Pearsonestimation method (Clopper & Pearson, 1934) to estimate  f dis ( x 1: t 0 ) with a confidence level, the inference cost on probabilistic models is much higher than deterministic models forthe confidence interval estimation, reducing the practicality of MIA. 4.2 Implementation of MIA The practical implementation of MIA is showed in Algorithm 1.
Revised: Although we can utilize Clopper-Pearson method (Clopper & Pearson, 1934) to estimate the discretized forecasts f dis ( x 1: t 0 ) with a confidence level, the inference cost would be expensive for confidence interval estimation.

Original: 4.3 Robustness Certificate of MIA
Revised: 4.2 Robustness Certificate of MIA

Original: Proposition 1 (Robustness Certificate of MIA) .
Revised: Proposition 1 (Robustness Certificate of MIA) .

Original: The forecast/label (not Abstain ) returned by Algorithm 1 cannot be changed by any temporally-localized perturbations whose ℓ 0 norm is no larger than L adv (see proof in Appendix).
Revised: The forecast/label (not Abstain ) returned by Algorithm 1 cannot be changed by any temporally-localized perturbations whose ℓ 0 norm is no larger than L adv (see proof in Appendix).

Original: Remark 3 ( Robustness Certificate ) .
Revised: Remark 2 (Robustness Certificate) .

Original: We emphasize that, our certificate is only for f MIA ( x 1: t 0 ) , instead of f ( x 1: t 0 ) , as f MIA ( x 1: t 0 ) and f ( x 1: t 0 ) are not necessarily equal.
Revised: We emphasize the certificate is only for f MIA ( x 1: t 0 ) , instead of f ( x 1: t 0 ) . Our aggregation does not allow any tolerance because the robustness of MIA would not hold if a disagreement is allowed.

Original: To certify the robustness for a sequence of time series x t 0 +1: t 0 + τ , we just need to implement the algorithm 1 on x t 0 +1 , x t 0 +2 , . . .
Revised: Note that, under our masking scheme, we can only guarantee there is a masked series that is unaffected by the perturbation.

Original: , x t 0 + τ one by one, to certify the robustness for each time point.
Revised: Meanwhile, the other masked series all retain the perturbed area. If we allow a disagreer, the ensemble prediction would be under the adversary’s control, because all except one masked series are under the adversary’s control (the unaffected masked series would become that disagreer). We also point out that MIA can be applied to multivariable tasks through repeating the process of univariate version of Masking (Step 1) and Imputing (Step 2) on each variable.

Original: 4.4 Training Imputation Model G ( · )
Revised: 4.3 Training Imputation Model G ( · )

Original: The performance of MIA frameworks highly depends on the imputation model G ( · ) that we use.
Revised: The performance of MIA frameworks highly depends on the imputation model G ( · ) .

Original: We notice that there exists much work on time series imputation (Cao et al., 2018; Du et al., 2022; 
Revised: We notice that there exists much work on time series imputation (Cao et al., 2018; Du et al., 2022; Moritz &

Original: Moritz & Bartz-Beielstein, 2017; Fortuin et al., 2020; Cao et al., 2018;
Revised: Bartz-Beielstein, 2017; Fortuin et al., 2020; Cao et al., 2018;

Original: Luo et al., 2019; Yozgatligil et al., 2013).
Revised: Luo et al., 2019; Yozgatligil et al., 2013).

Original: However, all these imputation models aim to recover the discrete missing values, thus perform poorly on imputing a consecutive sequence of missing values.
Revised: However, all these imputation models aim to recover the discrete missing values, which is not we want.

Original: To address this problem, we propose randomized masked training to train the imputation model G ( · ) , which is to minimize the MSE loss over M + 1 masked series with Gaussian augmentation, as follows:
Revised: Therefore, we propose randomized masked training to train the imputation model G ( · ) , minimizing the MSE loss over M + 1 masked series with Gaussian augmentation, as follows:

Original: [Equation]
Revised: [Equation]

Original: where  δ 1: t 0  ∼ N (0 , σ 2 ) is a length- t 0 vector, of which each entry is i.i.d.
Revised: where C = ⌈ ( t 0 − L mask ) /α ⌉ and δ [1: t 0 ] ∼ N (0 , σ 2 ) is a noise series, of which each entry is i.i.d.

Original:  from Gaussian  N (0 , σ 2 ) .
Revised: sampled from Gaussian distribution N (0 , σ 2 ) .

Original: We adopt MLP-Mixer (Tolstikhin et al., 2021) as the architecture of our imputation model.
Revised: Adding Gaussian noise is to make the imputation model robust to the random noise and avoid overfitting, as prior works (Foster et al., 1992; Passalis et al., 2021; Hwang et al., 1998) show the presence of random noise in time series data.

Original: 4.5 Comparison to Randomized Smoothing Defenses
Revised: 4.4 Comparison to Randomized Smoothing Defenses

Revised: Randomized smoothing (Cohen et al., 2019) is a well-know model-agnostic method in the field of certified defenses, which has been applied to defend various types of attack and achieves superior certified robustness in their respective fields.

Original: Weshow how to extend derandomized smoothing (DS) (Levine & Feizi, 2020a) and randomized ablation (RA) (Levine & Feizi, 2020b), to the certified defenses against temporally-localized perturbations.
Revised: We extend two randomized smoothing ℓ 0 -norm defenses, Derandomized Smoothing (Levine & Feizi, 2020a) and Randomized Ablation (Levine & Feizi, 2020b) as the baselines.

Original: We compare MIA to these two defenses.
Revised: [Table]

Original: Derandomized smoothing for time-series models.
Revised: Derandomized smoothing for time-series models.

Original: In the time-series version of DS, given a time series x 1: t 0 and the base classifier f ( · ) , DS (denoted by f DS ) classifies as follows 3 :
Revised: In the time-series version of DS, given a time series x 1: t 0 and the base classifier f ( · ) , DS (denoted by f DS ) classifies as follows 3 :

Original: [Equation]
Revised: [Equation]

Original: where Sub ( x 1: t 0 , k ) consists of the subsequences x 1: k , x k +1:2 k , x 2 k +1:3 k , . .
Revised: where Sub ( x 1: t 0 , k ) consists of the subsequences x 1: k , x k +1:2 k , x 2 k +1:3 k , . .

Original: , x t 0 − k +1 ,t 0 .
Revised: , x t 0 − k +1 ,t 0 .

Original: We first let the base classifier make predictions on these subsequences, and then f
Revised: We first let the base classifier make predictions on these subsequences, and then f

Original: DS ( x 1: t 0 ) outputs the majority label.
Revised: DS [Equation] x 1: t 0 ) outputs the majority label.

Original: The predictionˆ y = f DS ( x 1: t 0 ) isprovably robustto temporally-localized perturbations ( ℓnorm is L adv ) if
Revised: The prediction is robust if

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: Randomized ablation for time-series models.
Revised: Randomized ablation for time-series models.

Original: RA (denoted by f RA ( · ) ) classifies as follows:
Revised: RA (denoted by f RA ( · ) ) classifies as follows:

Original: [Equation]
Revised: [Equation]

Original: where x sub ∼ RA ( x 1: t 0 , k ) is to randomly sample k time points without replacement to construct the subseries x sub , and ablate all other points.
Revised: where x sub ∼ RA ( x 1: t 0 , k ) is to randomly sample k time points without replacement to construct the subseries x sub , and ablate all other points.

Original: f RA ( x 1: t 0 ) returns the label that f ( · ) is most likely to classify x sub as.
Revised: f RA ( x 1: t 0 ) returns the label that f ( · ) is most likely to classify x sub as.

Original: ˆ y =
Revised: ˆ y =

Original: f RA ( x 1: t 0 ) isprovably robust ifthe top-1 probability score satisfies
Revised: f RA ( x 1: t 0 ) is robust if

Original: [Equation]
Revised: [Equation]

Original: Compare MIA to DS and RA.
Revised: Comparison to DS and RA.

Original: The base classifier of DS and RA are making predictions on subseries, instead of normal series, implying that the pretrained model would perform poorly on these two defenses.
Revised: The base classifier of DS and RA are making predictions on subseries, instead of normal series, implying that the pretrained model would perform poorly on these two defenses.

Original: Therefore, to apply these two defenses, we need to train the base classifiers from scratch on the subseries.
Revised: Therefore, to apply these two defenses, we need to train the base classifiers from scratch on the subseries.

Original: In stark contrast, MIA is a plug-and-play framework that can be applied to TSF/TSC pretrained models.
Revised: In stark contrast, MIA is a plug-and-play framework that can be applied to TSF/TSC pretrained models.

Original: The major cost of MIA is to train the imputation model.
Revised: The major cost of MIA is to train the imputation model.

Original: Remarkably, MIA imputation model can be trained in an unsupervised manner, saving us from labeling the data.
Revised: Remarkably, MIA imputation model can be trained in an unsupervised manner, saving us from labeling the data.

Original: Furthermore, our experimental results consistently demonstrate that MIA outperforms DS and RA on the certified accuracy, as further discussed in Section 5.
Revised: Furthermore, our experimental results consistently demonstrate that MIA outperforms DS and RA on the certified accuracy, as further discussed in Section 5.

Original:  Experiments
Revised:  Experiments

Original: Experimental setup.
Revised: Experimental setup.

Original: We evaluate MIA on both TSC and TSF datasets.
Revised: We evaluate MIA on both TSC and TSF datasets.

Original: TSF includes Exchange Rate,
Revised: TSF includes Exchange Rate,

Original: Traffic and UCI Electricity (Alexandrov et al., 2019), and TSC datasets include DistalPhalanxTW,
Revised: Traffic and UCI Electricity (Alexandrov et al., 2019), and TSC datasets include DistalPhalanxTW,

Original: MiddlePhalanxTW ProximalPhalanx (Ismail Fawaz et al., 2019a).
Revised: MiddlePhalanxTW ProximalPhalanx (Ismail Fawaz et al., 2019a).

Original: We use MLP-Mixer (Tolstikhin et al., 2021), MLP and LSTM (Hochreiter & Schmidhuber, 1997) as the pretrained model.
Revised: We use MLP-Mixer (Tolstikhin et al., 2021), MLP and LSTM (Hochreiter & Schmidhuber, 1997) as the pretrained model.

Original: We emphasize that, our experiments are conducted on the clean trainsets, following the common setting in the field of certified adversarial defenses (Yoon et al., 2022;
Revised: We emphasize that, our experiments are conducted on the clean trainsets, following the common setting in the field of certified adversarial defenses (Yoon et al., 2022;

Original: Li et al., 2020; Cohen et al., 2019;
Revised: Li et al., 2020; Cohen et al., 2019;

Original: Chiang et al., 2020; Zhang et al., 2019).
Revised: Chiang et al., 2020; Zhang et al., 2019).

Original: MLP-Mixer is set as the default pretrained model/base model for MIA, DS and RA.
Revised: MLP-Mixer is set as the default pretrained model/base model for MIA, DS and RA.

Original: We use ∆ = 1 . 5 unless otherwise specified.
Revised: We use ∆ = 1 . 5 unless otherwise specified.

Original: All our experiments are conducted on CPU (16 Intel(R)
Revised: All our experiments are conducted on CPU (16 Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Gold 5222 CPU @ 3.80GHz) and GPU (one NVIDIA RTX 2080 Ti).
Revised: Gold 5222 CPU @ 3.80GHz) and GPU (one NVIDIA RTX 2080 Ti).

Original: More details about our experimental setups are omitted to Appendix.
Revised: More details about our experimental setups are omitted to Appendix.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: For TSC, we evaluate the defense by the certified accuracy (CA) to the temporally-localized perturbation of L adv , defined by the fraction of the test samples that are correctly classified and certifiably robust to the perturbation.
Revised: For TSC, we evaluate the defense by the certified accuracy (CA) to the temporally-localized perturbation of L adv , defined by the fraction of the test samples that are correctly classified and certifiably robust to the perturbation.

Original: For TSC, we evaluate the defense by three metrics: forecasting rate (FR), mean square error (MSE) and mean absolute error (MAE) 4 .
Revised: For TSC, we evaluate the defense by three metrics: 3 I {} is the indicator function. [Table] [Table]

Original: FR is the fraction of the time points on which MIA outputs the forecast instead of Abstain .
Revised: FR is the fraction of the time points on which MIA outputs the forecast instead of Abstain .

Original: MSE/MAE measures the mean square error/mean absolute error between forecasts ( Abstain are excluded) and groundtruth.
Revised: MSE/MAE measures the mean square error/mean absolute error between forecasts ( Abstain are excluded) and groundtruth.

Original: 5.1 Comparison to Peer Methods
Revised: 5.1 Comparison to Peer Methods

Original: Comparison on TSC.
Revised: Comparison on TSC.

Original: Table 1 reports the certified accuracy of three methods in defending temporally-localized perturbations.
Revised: Table 1 reports the certified accuracy of three methods in defending temporally-localized perturbations.

Original: The pretrained/base model architectures of three defenses are all
Revised: The pretrained/base model architectures of three defenses are all

Original: MLP-Mixer.
Revised: MLP-Mixer.

Original: An interesting observation is that the certified accuracy of DS and RA keeps constant to different L adv .
Revised: An interesting observation is that the certified accuracy of DS and RA keeps constant to different L adv .

Original: The reason is that, the probability score of DS/RA models often concentrates on a single class, causing most classifications (including both (correct and wrong classifications) of DS [Table] [Table] [Table]
Revised: The reason is that, the probability score of DS/RA models often concentrates on a single class, causing most classifications (including both (correct and wrong classifications) of DS and RA are of high robustness.

Original: and RA are of high robustness.

Original: The results show that the certified accuracy of MIA is more than twice of DR and RA across different L adv .
Revised: The results show that the certified accuracy of MIA is more than twice of DR and RA across different L adv .

Original: The fundamental reason is that the pretrained model of
Revised: The fundamental reason is that the pretrained model of

Original: MIA classifies the masked series, while the base model in RS/DS classifies the subseries.
Revised: MIA classifies the masked series, while the base model in RS/DS classifies the subseries.

Original: MIA can offer a higher certified accuracy with more information, as the masked series contains much more information ( t 0 − L mask unmasked time points) than the subseries ( k sampled time points).
Revised: MIA can offer a higher certified accuracy with more information, as the masked series contains much more information ( t 0 − L mask unmasked time points) than the subseries ( k sampled time points).

Original: Comparison on TSF.
Revised: Comparison on TSF.

Original: Table 2 reports FR and MSE of three defenses on Exchange, where the model predicts the next 30 values.
Revised: Table 2 reports FR and MSE of three defenses on Exchange, where the model predicts the next 30 values.

Original: Here we utilize discretization technique to make the TSF task feasible to
Revised: Here we utilize discretization technique to make the TSF task feasible to

Original: DS and RA.
Revised: DS and RA.

Original: The table shows that MIA offers a significantly higher FR than DS and RA, implying that MIA return the forecasting results much more frequently than other two defenses.
Revised: The table shows that MIA offers a significantly higher FR than DS and RA, implying that MIA return the forecasting results much more frequently than other two defenses.

Original: The reason for the superior FR is same as TSC.
Revised: The reason for the superior FR is same as TSC.

Original: We also observe that DS ( k = 15% ) achieves a lower MSE than
Revised: We also observe that DS ( k = 15% ) achieves a lower MSE than

Original: MIA at L adv ≥ 4% , which partially owing to its low FR.
Revised: MIA at L adv ≥ 4% , which partially owing to its low FR.

Original: Since a low FR implies that the aggregation step of MIA filters a large portion of distrustful forecasts, reducing the difficulty of achieving lower
Revised: Since a low FR implies that the aggregation step of MIA filters a large portion of distrustful forecasts, reducing the difficulty of achieving lower

Original: MSE for the remained forecasts.
Revised: MSE for the remained forecasts.

Original: Based on the results, MIA is better than other two defenses when we jointly consider FR and MSE.
Revised: Based on the results, MIA is better than other two defenses when we jointly consider FR and MSE.

Original: Comparison on inference time.
Revised: Comparison on inference time.

Original: Table 3 compares the inference time of three defenses, which is averaged among three TSC datasets.
Revised: Table 3 compares the inference time of three defenses, which is averaged among three TSC datasets.

Original: We observe that the inference time of MIA is larger than DS, but significantly smaller than RA.
Revised: We observe that the inference time of MIA is larger than DS, but significantly smaller than RA.

Original: Specifically, the larger inference time than DS is owing to theadditional cost in running the imputation model of MIA.
Revised: Specifically, MIA’s larger inference time than DS is owing to the cost in running the imputation model.

Original: The seriously large inference timeof RA is because of [Table] [Table] its confidence interval estimationfor the probability score, which requires a large number of samples of noisy classifications 6 .
Revised: RA’s large inference time is for the confidence interval estimation 6 .

Original: We also observe that the inference time of MIA increases with L adv 
Revised: We also observe that the inference time of MIA increases with L adv , and decreases

Original: [Table]
Revised: [Table]

Original: and decreases with L mask , since the number of masked series equals ⌊ (  t 0 − 1) /
Revised: [Table] [Table] with L mask , because the number of masked series is ⌈ ( ⌉ t 0 − L mask ) /

Original: ( L mask − L adv +1) ⌋ +1 .
Revised: ( L mask − L adv + 1) ⌉ + 1 .

Original: We omit the inference time analysis on TSF datasets to Appendix.
Revised: We omit the inference time analysis on TSF datasets to Appendix.

Original: 5.2 Analysis of MIA on Different Pretrained Models
Revised: 5.2 Analysis of MIA on Different Pretrained Models

Original: Table 4 and Table 5 report the performance of MIA on different pretrained models, where the model forecasts the next 24 points 7 .
Revised: Table 4 and Table 5 report the performance of MIA on different pretrained models, where the model forecasts the next 24 points.

Original: We observe that MIA ( ∆ = 1 . 0 , 1 . 5 ) consistently lowers MSE as compared to that of the original pretrained models, suggesting MIA could also be an effective plugin for performance improvement.
Revised: We observe that MIA ( ∆ = 1 . 0 , 1 . 5 ) consistently lowers MSE as compared to that of the original pretrained models, suggesting MIA could also be an effective plugin for performance improvement.

Original: Specifically, MIA improves the forecasting performance in the way of filtering these distrustful forecasts, sacrificing the availability (the decrease of FR) for lower MSE as well as certified robustness, which is a common trade-off in the field of certified defenses (Cohen et al., 2019; Levine & Feizi, 2020a;b; Liu et al., 2021; Han et al., 2021).
Revised: Specifically, MIA improves the forecasting performance in the way of filtering these distrustful forecasts, sacrificing the availability (the decrease of FR) for lower MSE as well as certified robustness, which is a common trade-off in the field of certified defenses (Cohen et al., 2019; Levine & Feizi, 2020a;b; Liu et al., 2021; Han et al., 2021).

Original: We can control the trade-off between the forecasting performance and FR via modifying ∆ , as the decrease of ∆ canfurther reduce MSE at the cost of FR. 
Revised: We can control the trade-off between MSE and FR by ∆ , as the decrease of ∆ can reduce MSE at the cost of FR. [Figure] [Figure] [Table]

Original: 5.3 Analysis on Imputation Model of MIA and Ablation Study
Revised: 5.3 Analysis on Imputation Model of MIA and Ablation Study

Original: Impact of training algorithm.
Revised: Impact of training algorithm.

Original: Table 6 compares our masked training to random training, which trains the imputation model on the randomly masked series.
Revised: Table 6 compares our masked training to random training, which trains the imputation model on the randomly masked series.

Original: Through extensive comparisons on different imputation model architectures and datasets, we convince that masked training consistently [Figure] [Figure]
Revised: Through extensive comparisons on different imputation model architectures and datasets, we convince that masked training consistently outperforms random training for MIA by a non-trivial gap.

Original: [Table]

Original: outperforms random training for MIA by a non-trivial gap.

Original: The gap becomes even larger at larger
Revised: The gap becomes even larger at larger

Original: L adv .
Revised: L adv .

Original: The results suggest that masked training is suitable for MIA imputation model.
Revised: The results suggest that masked training is suitable for MIA imputation model.

Original: Impact of architecture of imputation models.
Revised: Impact of architecture of imputation models.

Original: Table 7 reports the performances of different imputation model architectures on MIA.
Revised: Table 7 reports the performances of different imputation model architectures on MIA.

Original: Our results show that MLP-Mixer can offer higher FR and lower MSE simultaneously, suggesting MLP-Mixer is intrinsically more robust than other models.
Revised: Our results show that MLP-Mixer can offer higher FR and lower MSE simultaneously, suggesting MLP-Mixer is intrinsically more robust than other models.

Original: Impact of L mask .
Revised: Impact of L mask .

Original: Fig.
Revised: Fig.

Original: 2a, 2b show: 1) MSE roughly keeps constant w.r.t.
Revised: 2a, 2b show: 1) MSE roughly keeps constant w.r.t.

Original: L mask , because the discretization technique can diminish the difference between the forecasts that are close to each other.
Revised: L mask , because the discretization technique can diminish the difference between the forecasts that are close to each other.

Original: FR decreases with the increase of L mask , because our imputation quality decreases with
Revised: FR decreases with the increase of L mask , because our imputation quality decreases with

Original: L mask , making it harder to reach agreement unanimously.
Revised: L mask , making it harder to reach agreement unanimously.

Original: Fig.
Revised: Fig.

Original: 2c show that the impact of L mask on
Revised: 2c show that the impact of L mask on

Original: CA is not significant.
Revised: CA is not significant.

Original: Although the increase of L mask reduces our imputation quality to decrease FR, it also reduce the number of masked series, benefiting the agreement.
Revised: Although the increase of L mask reduces our imputation quality, it reduces the number of masked series simultaneously.

Original: Impact of ∆ .
Revised: Impact of ∆ .

Original: Fig.
Revised: Fig.

Original: 2d, 2e report the impact of ∆ .
Revised: 2d, 2e report the impact of ∆ .

Original: As ∆ increases, we observe that MSE and FR increase, validating our statement about ∆ in Section 4.1.
Revised: As ∆ increases, we observe that MSE and FR increase, validating our statement about ∆ in Section 4.1.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we propose the first framework for time-series models to certifiably defend against ℓ 0 -norm perturbations.
Revised: In this paper, we propose the first framework for time-series models to certifiably defend against ℓ 0 -norm perturbations.

Original: Notably, MIA is a plug-and-play defense, which can be easily applied to any
Revised: Notably, MIA is a plug-and-play defense, which can be easily applied to any

Original: TSF/TSC pretrained model.
Revised: TSF/TSC pretrained model.

Original: The only requirement of deploying MIA is to train an imputation model, which has been extensively explored in this work.
Revised: The only requirement of deploying MIA is to train an imputation model, which has been extensively explored in this work.

Original: Moreover, our extensive experiments validate the effectiveness of MIA.
Revised: Moreover, our extensive experiments validate the effectiveness of MIA.

Original: We expect our work can inspire more studies on the ℓ 0 -norm robustness for time-series models.
Revised: We expect our work can inspire more studies on the ℓ 0 -norm robustness for time-series models.

Original: Interesting future works include applying MIA to probabilistic models.
Revised: Interesting future works include applying MIA to probabilistic models.

Original: A Proof for Proposition
Revised: A Proof for Proposition

Original: Proposition 2 (Robustness Certificate of MIA) .
Revised: Proposition 2 (Robustness Certificate of MIA) .

Original: The forecast/label (not Abstain ) returned by Algorithm 1 cannot be changed by any temporally-localized perturbation whose ℓ 0 norm is no larger than
Revised: The forecast/label (not Abstain ) returned by Algorithm 1 cannot be changed by any temporally-localized perturbation whose ℓ 0 norm is no larger than

Original: L adv (see proof in Appendix).
Revised: L adv (see proof in Appendix).

Original: Proof.
Revised: Proof.

Original: Here we prove the robustness certificate for MIA (TSC).
Revised: Here we prove the robustness certificate for MIA (TSC).

Original: The proof for MIA (TSF) is analogous to this proof.
Revised: The proof for MIA (TSF) is analogous to this proof.

Original: Assume that the adversary has changed the classification result of MIA from y 1 to y 2 via the temporally-localized perturbation δ ( ℓ 0 norm is L adv ).
Revised: Assume that the adversary has changed the classification result of MIA from y 1 to y 2 via the temporally-localized perturbation δ ( ℓ 0 norm is L adv ).

Revised: For notational simplicity we denote M [1+ kα :min( L mask + kα,t 0 )] by M ( k ) .

Original: Then, we have:
Revised: Then, we have:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Without loss of generality, suppose the temporally-localized perturbation is located at [ t adv + 1 : min( t adv + L adv , t 0 )]
Revised: Without loss of generality, suppose the temporally-localized perturbation is located at [ t adv + 1 : min( t adv + L adv , t 0 )]

Original: ( t adv = 0 , . . .
Revised: ( t adv = 0 , . . .

Original: , t 0 − 1 ), we prove that there exists a mask M ( ˆ m ) , ˆ m =
Revised: , t 0 − 1 ), we prove that there exists a mask M ( ˆ m ) , ˆ m =

Original: ⌊ t
Revised: ⌊ t

Original: adv α ⌋ that can completely occlude it .
Revised: adv α ⌋ that can completely occlude it .

Original: Specifically, we compare the starting/end point of the mask M ( ˆ m ) and the perturbation δ .
Revised: Specifically, we compare the starting/end point of the mask M ( ˆ m ) and the perturbation δ .

Original: For the starting point, we have:
Revised: For the starting point, we have:

Original: [Equation]

Original: Mask

Original: [Equation]

Original: Perturbation

Original: [Equation]
Revised: [Equation]

Original: In terms of the end points, we have:
Revised: In terms of the end points, we have:

Original: [Equation]
Revised: [Equation]

Original: As M ( ˆ m ) occludes δ , thus ( x 1: t 0 ) ⊙ M ( ˆ m )
Revised: As M ( ˆ m ) occludes δ , thus ( x 1: t 0 ) ⊙ M ( ˆ m )

Original: = ( x 1: t 0 + δ ) ⊙ M ( ˆ m ) ⇒ y 

Revised: = ( x 1: t 0 + δ ) ⊙ M ( ˆ m ) ⇒ y

Original: = y 2 .
Revised: = y 2 .

Original: Our proof is completed.
Revised: Our proof is completed.

Original: B Empirical Evaluation on Risk of Temporally-Localized Perturbations
Revised: B Empirical Evaluation on Risk of Temporally-Localized Perturbations

Original: To support our statement about the risk of temporally-localized perturbations, we specifically propose an algorithm for generating the temporally-localized perturbations.
Revised: To support our statement about the risk of temporally-localized perturbations, we specifically propose an algorithm for generating the temporally-localized perturbations.

Original: We then evaluate the attack performance visually.
Revised: We then evaluate the attack performance visually.

Original: B.1 Restate Definition of temporally-localized Perturbation
Revised: B.1 Restate Definition of temporally-localized Perturbation

Original: Definition 2 (Temporally-localized perturbation) .
Revised: Definition 2 (Temporally-localized perturbation) .

Original: Temporally-localized perturbation is to perturb consecutive time points of x 1: t 0 w.r.t.
Revised: Temporally-localized perturbation is to perturb consecutive time points of x 1: t 0 w.r.t.

Original: ℓ 0 -norm constraint .
Revised: ℓ 0 -norm constraint .

Original: The perturbed series is:
Revised: The perturbed series is:

Original: [Equation]
Revised: [Equation]

Original: , subject to 
Revised: subject to ∥

Revised: δ [ t adv +

Original: [Equation]
Revised: [Equation]

Original: δ
Revised: adv + L adv

Original: [Equation]
Revised: [Equation]

Original: adv

Original: [Equation]
Revised: [Equation]

Original: where t adv + 1 and L adv are the starting point and the ℓ 0 -norm of the temporally-localized perturbation δ  .
Revised: where t adv +1 and L adv are the starting point and the ℓ 0 -norm of the temporally-localized perturbation δ [ t adv +1: t adv + L adv ] .

Original: B.2 Algorithm of Generating Temporally-Localized Perturbations.
Revised: B.2 Algorithm of Generating Temporally-Localized Perturbations.

Original: The objective of our algorithm is to maximize MSE between the original forecasts and the perturbed forecasts, with respect to the ℓ 0 -norm constraint.
Revised: The objective of our algorithm is to maximize MSE between the original forecasts and the perturbed forecasts, with respect to the ℓ 0 -norm constraint.

Original: Specifically, given the forecasting model f ( x 1: t 0 ) → x t 0 +1 ,t 

Revised: Specifically, given the forecasting model f ( x 1: t 0 ) → x t 0 +1 ,t

Original: + τ , our objective can be formulated as follows:
Revised: + τ , our objective can be formulated as follows:

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Original: where δ corresponds to the perturbation defined in Eq.(18).
Revised: where δ corresponds to the perturbation defined in Eq.(18).

Original: Actually, the problem of computing the temporally-localized perturbation can be decomposed into two sub-problems: P1 ) Search for the period [ t adv +1 , t adv + L adv ] to perturb.
Revised: Actually [Table] the problem of computing the temporally-localized perturbation can be decomposed into two sub-problems: P1 ) Search for the period [ t adv +1 , t adv + L adv ] to perturb.

Original: P2 )
Revised: P2 )

Original: Fix the period [ t adv +1 , t adv + L adv ] , compute the value of the perturbation δ t adv +1 , . . .
Revised: Fix the period [ t adv +1 , t adv + L adv ] , compute the value of the perturbation δ t adv +1 , . . .

Original: , δ t adv + L adv .
Revised: , δ t adv + L adv .

Original: Here solving P2 is not hard.
Revised: Here solving P2 is not hard.

Original: If we have determined δ t adv +1 , . .
Revised: If we have determined δ t adv +1 , . .

Original: , δ t adv + L adv , we can maximize the following loss to compute the perturbation values via projected gradient descent (PGD).
Revised: , δ t adv + L adv , we can maximize the following loss to compute the perturbation values via projected gradient descent (PGD).

Original: [Equation]
Revised: [Equation]

Original: Then the main challenge is to determined which period to perturb.
Revised: Then the main challenge is to determined which period to perturb.

Original: Here we solve P1 by enumerating all the possible perturbing positions [ t adv + 1 : t adv + L adv ] , t adv = 0 , . . .
Revised: Here we solve P1 by enumerating all the possible perturbing positions [ t adv + 1 : t adv + L adv ] , t adv = 0 , . . .

Original: , t 0 − L adv and compute the corresponding attacks.
Revised: , t 0 − L adv and compute the corresponding attacks.

Original: Finally, we return the one with the largest MSE loss among t
Revised: Finally, we return the one with the largest MSE loss among t

Original: [Equation]
Revised: [Equation]

Original: L adv +1 perturbations.
Revised: perturbations.

Original: However, in practice we found that computing the values of perturbation ( P2 ) w.r.t.
Revised: However, in practice we found that computing the values of perturbation ( P2 ) w.r.t.

Original: to the fixed period is hard to converge, as the ℓ 2 norm of the temporally-localized perturbation will approach ∞ .
Revised: to the fixed period is hard to converge, as the ℓ 2 norm of the temporally-localized perturbation will approach ∞ .

Original: We believe that a perturbation attack with ∞ ℓ 2 norm is meaningless in practice.
Revised: We believe that a perturbation attack with ∞ ℓ 2 norm is meaningless in practice.

Original: In the sake of practicality, we additionally consider ℓ 2 norm for the temporally-localized perturbations besides ℓ 0 -norm constraint for the sub-problem P2 , as follows:
Revised: In the sake of practicality, we additionally consider ℓ 2 norm for the temporally-localized perturbations besides ℓ 0 -norm constraint for the sub-problem P2 , as follows:

Original: [Equation]
Revised: [Equation]

Original:  (21) where ϵ is the preset upper bound of the perturbation ℓ 2 norm.[Table]
Revised: subject [Equation] (21) where ϵ is the preset upper bound of the perturbation ℓ 2 norm.

Original: B.3 Empirical Evaluation of Temporally-localized Perturbations. 
Revised: B.3 Empirical Evaluation of Temporally-localized Perturbations. B.4 Quantify the risk of temporally-localized perturbations.

Original: Quantify the risk of temporally-localized perturbations.

Original: Table 8 quantifies the risk of temporally-localized perturbations via computing MSE between the clean forecasts and the perturbed forecasts w.r.t.
Revised: Table 8 quantifies the risk of temporally-localized perturbations via computing MSE between the clean forecasts and the perturbed forecasts w.r.t.

Original: the ℓ 0 -norm ( L adv ) and the ℓ 2 -norm ( ϵ ) constraints.
Revised: the ℓ 0 -norm ( L adv ) and the ℓ 2 -norm ( ϵ ) constraints.

Original: We observe that MLP-Mixer model provides the highest empirical robustness among three models, which partially explains why MLP-Mixer outperforms other models on MIA.
Revised: We observe that MLP-Mixer model provides the highest empirical robustness among three models, which partially explains why MLP-Mixer outperforms other models on MIA.

Original: In particular, we further compare MLP to MLP+MIA ( δ = 1 . 5 ) in Table 4 of the main paper.
Revised: In particular, we further compare MLP to MLP+MIA ( δ = 1 . 5 ) in Table 4 of the main paper.

Original: Specifically, MSE of MLP under temporally-localized perturbations ( ϵ = 3 . 0 ) is 5% : 1 .
Revised: Specifically, MSE of MLP under temporally-localized perturbations ( ϵ = 3 . 0 ) is 5% : 1 .

Original: 1 . 686 while MLP+MIA is 5% : 0 .
Revised: 1 . 686 while MLP+MIA is 5% : 0 .

Original: MIA reduces the MSE to roughly one tenth of the original, which indicates that MIA can effectively prevent our forecasting results from being influenced by the temporally-localized perturbations.
Revised: MIA reduces the MSE to roughly one tenth of the original, which indicates that MIA can effectively prevent our forecasting results from being influenced by the temporally-localized perturbations.

Revised: B.5 Compare Attacking Performance of ℓ 0 Attack to ℓ 2 Attack

Revised: We compare the ℓ 0 attack and ℓ 2 attack under under norm constraints β (attack rate) on time series forecasting task.

Revised: Results are shown in Table 30, 31 and 32.

Revised: The values in tables are calculated as

Revised: MSE ℓ 0 − MSE ℓ 2 MSE ℓ

Original:  Visualize the risk of temporally-localized perturbations.
Revised: × 100% . [Table] B.6 Visualize the risk of temporally-localized perturbations.

Original: Fig.
Revised: Fig.

Original: 3 illustratively shows the effect of temporally-localized perturbations on our forecasting results.
Revised: 3 illustratively shows the effect of temporally-localized perturbations on our forecasting results.

Original: We observe that temporally-localized perturbations of L atk = 10% can significantly change our forecasts.[Table]
Revised: We observe that temporally-localized perturbations of L atk = 10% can significantly change our forecasts.

Original: C Experimental Setups C.1 Dataset Information
Revised: C Experimental Setups C.1 Dataset Information

Original: Table 9 shows the details of each dataset, including context length, forecasting length (for TSF datasets) and number of classes.
Revised: Table 9 shows the details of each dataset, including context length, forecasting length (for TSF datasets) and number of classes.

Original: Traffic Hourly occupancy rate, between 0 and 1, of 963 San Francisco car lanes Salinas et al.
Revised: Traffic Hourly occupancy rate, between 0 and 1, of 963 San Francisco car lanes (Salinas et al., 2019).

Original: Electricity Hourly time series of the electricity consumption of 370 customers Salinas et al.
Revised: Electricity Hourly time series of the electricity consumption of 370 customers (Salinas et al., 2019).

Original: Exchange Daily exchange rate between 8 currencies Salinas et al.
Revised: Exchange Daily exchange rate between 8 currencies (Salinas et al., 2019).

Original: DistalPhalanxTW, MiddlePhalanxTW, ProximalPhalanxTw 

Revised: DistalPhalanxTW, MiddlePhalanxTW, ProximalPhalanxTw

Original: This series of 11 classification problems were created as part of Luke Davis’s PhD titled ”Predictive Modelling of Bone Ageing”.
Revised: This series of 11 classification problems were created as part of Luke Davis’s PhD titled ”Predictive Modelling of Bone Ageing”.

Original: They are designed to test the efficacy of hand and bone outline detection and whether these outlines could be helpful in bone age prediction.
Revised: They are designed to test the efficacy of hand and bone outline detection and whether these outlines could be helpful in bone age prediction.

Original: Note that these problems are aligned by subject, and hence can be treated as a multi-dimensional TSC problem.
Revised: Note that these problems are aligned by subject, and hence can be treated as a multi-dimensional TSC problem.

Original: The final three bone classification problems, DistalPhalanxTW, MiddlePhalanxTW and ProximalPhalanxTW, involve predicting the
Revised: The final three bone classification problems, DistalPhalanxTW, MiddlePhalanxTW and ProximalPhalanxTW, involve predicting the

Original: Tanner-Whitehouse score (as labelled by a human expert) from the outline.
Revised: Tanner-Whitehouse score (as labelled by a human expert) from the outline.

Original: Data Pre-Processing We pre-process the input series with scipy.signal.savgol filter with window length 15 and polyorder 5 on both training and testing datasets.
Revised: Data Pre-Processing We pre-process the input series with scipy.signal.savgol filter with window length 15 and polyorder 5 on both training and testing datasets.

Original: Besides, we normalize each input series with its mean value and standard deviation.
Revised: Besides, we normalize each input series with its mean value and standard deviation.

Original: Mean value and standard deviation will be 0 and 1 respectively for each normalized input series.
Revised: Mean value and standard deviation will be 0 and 1 respectively for each normalized input series.

Original: We use the instance normalization method on both trainsets and testsets.
Revised: We use the instance normalization method on both trainsets and testsets.

Original: D Introduction to Pretrained Models
Revised: D Introduction to Pretrained Models

Original: Model architecture.
Revised: Model architecture.

Original: We use the classical forecasting and classification models, MLP, MLPMixer Tolstikhin et al. (2021), GRU Cho et al. (2014), LSTM Hochreiter & Schmidhuber (1997),
Revised: We use the classical forecasting and classification models, MLP, MLPMixer (Tolstikhin et al., 2021), GRU (Cho et al., 2014), LSTM (Hochreiter & Schmidhuber, 1997),

Original: FCN Ismail Fawaz et al. (2019b) and ResNet-18 He et al.
Revised: FCN (Ismail Fawaz et al., 2019b) and ResNet-18 (He et al., 2016) as the pretrained models.

Original: as the pretrained models.

Original: We show the architecture of these models in Table 10, 11, 12, 13.
Revised: We show the architecture of these models in Table 10, 11, 12, 13.

Original: Training.
Revised: Training.

Original: we uniformly adapt Adam optimizer Kingma & Ba (2014) with lr = 0 . 0001 , β = (0 . 9 , 0 . 999) , ϵ = 10 − 8 , weight decay = 0 , epochs = 20 for all the pretrained models.
Revised: we uniformly adapt Adam optimizer (Kingma & Ba, 2014) with lr = 0 . 0001 , β = (0 . 9 , 0 . 999) , ϵ = 10 − 8 , weight decay = 0 , epochs = 20 for all the pretrained models.

Original: E More Experimental Results

Original: Comparison to peer methods on Mean Absolute Error (MAE).

Original: Table 15 compares MAE and FR of three defenses on Traffic.

Original: Analogous to the comparison (Table 2 in main paper) on MSE, MIA achieves both the lowest MAE and highest FR on L adv = 1% , 2% , 3% .

Original: DS outperforms MIA at

Original: L adv = 4% , . . .

Original: , 10% for the great sacrifice on its FR.

Original: [Table]
Revised: [Table]

Original: Length 
Revised: Length of

Original: [Table]
Revised: [Table]

Original: LeakyReLU
Revised: LeakyReLU

Original: ( α = 0 . 2)
Revised: ( α = 0 . 2)

Revised: [Table]

Original: LeakyReLU
Revised: LeakyReLU

Original: ( α = 0 . 2)
Revised: ( α = 0 . 2)

Original: [Table]
Revised: [Table]

Original: LeakyReLU
Revised: LeakyReLU

Original: ( α = 0 . 2)
Revised: ( α = 0 . 2)

Original: [Table]
Revised: [Table]

Original: LeakyReLU
Revised: LeakyReLU

Original: ( α = 0 . 2)
Revised: ( α = 0 . 2)

Revised: [Table]

Original: Length of sequence
Revised: Length of sequence

Original: LeakyReLU
Revised: LeakyReLU

Original: ( α = 0 . 2)
Revised: ( α = 0 . 2)

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Original: [Table]
Revised: Comparison to peer methods on Mean Absolute Error (MAE). Table 15 compares MAE and FR of three defenses on Traffic. Analogous to the comparison (Table 2 in main paper) on MSE, MIA

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Revised: achieves both the lowest MAE and highest FR on L adv = 1% , 2% , 3% .

Revised: DS outperforms MIA at

Revised: L adv = 4% , . . .

Revised: , 10% for the great sacrifice on its FR.

Original: Evaluation of MAE on Traffic and Electricity.
Revised: Evaluation of MAE on Traffic and Electricity.

Original: Table 16, Table 17 report MAE and FR of MIA on Traffic, Electricity and Exchange respectively [Table] as a supplement to Table 4 and Table 5 in the main paper.
Revised: Table 16, Table 17 report MAE and FR of MIA on Traffic, Electricity and Exchange respectively, as a supplement to Table 4 and Table 5 in the main paper.

Original: We observe that MIA consistently reduces MAE of the pretrained models, similar to the results on MSE.
Revised: We observe that MIA consistently reduces MAE of the pretrained models, similar to the results on MSE.

Original: Our comprehensive experiments suggest that MIA can effectively improve our forecasting quality in the way of filtering the unconfident forecasts [Table] [Table] [Table]
Revised: Our comprehensive experiments suggest that MIA can effectively improve our forecasting quality in the way of filtering the unconfident forecasts.

Original: [Table]

Original: Evaluation of MSE and MAE for MIA on Exchange.
Revised: Evaluation of MSE and MAE for MIA on Exchange.

Original: Table 18 and Table 19 evaluate MSE/MAE of MIA on exchange.
Revised: Table 18 and Table 19 evaluate MSE/MAE of MIA on exchange.

Original: We observe that, MIA moderately increase MSE/MAE of the pretrained models because of the information loss for the discretization technique.
Revised: We observe that, MIA moderately increase MSE/MAE of the pretrained models because of the information loss for the discretization technique.

Original: Specifically, we can see that
Revised: Specifically, we can see that

Original: MSE/MAE of the pretrained models are commonly much smaller than that of Traffic and Electricity because Exchange dataset is much simpler.
Revised: MSE/MAE of the pretrained models are commonly much smaller than that of Traffic and Electricity because Exchange dataset is much simpler.

Original: The better forecasting performance implies that we might lose more information for the discretization technique.
Revised: The better forecasting performance implies that we might lose more information for the discretization technique.

Original: On Exchange, information loss plays a more conspicuous role than the filtering function of MIA, causing the increase of MSE/MAE. 
Revised: On Exchange, information loss plays a more conspicuous role than the filtering function of MIA, causing the increase of MSE/MAE. This [Table] [Table]

Original: This suggests that discretization technique might lower the forecasting performance when the pretrained models are precise enough.
Revised: [Table] suggests that discretization technique might lower the forecasting performance when the pretrained models are precise enough [Table]

Original: Analysis of training algorithm of imputation models.
Revised: Analysis of training algorithm of imputation models.

Original: Table 20 reportS MSE and FR of masked training and random training on the TSF dataset Traffic.
Revised: Table 20 reportS MSE and FR of masked training and random training on the TSF dataset Traffic.

Original: Similar to the comparison on TSC datasets (Table 6 in the main paper), our masked training consistently achieves lower MSE than random training on different imputation models.
Revised: Similar to the comparison on TSC datasets (Table 6 in the main paper), our masked training consistently achieves lower MSE than random training on different imputation models

Original: [Table]
Revised: [Table]

Original: [Table] 
Revised: [Table] [Table] [Figure] [Table]

Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: Analysis of Different Pretrained Models on MIA.
Revised: Analysis of Different Pretrained Models on MIA.

Original: Table 21 reports the MAE of MIA with different pretrained models [Table] as a supplement to Table 7 in the main paper.
Revised: Table 21 reports the MAE of MIA with different pretrained models, as a supplement to Table 7 in the main paper.

Original: The results demonstrate that MLP-Mixer outperforms all the other defenses
Revised: The results demonstrate that MLP-Mixer outperforms all the other defenses.

Original: [Table]
Revised: MIA framework can be easily implemented into multi-variate time series forecasting and classification.

Original: [Table]{+For the multi-variate time series forecasting (MTSF) task, we follow the work (Wu et al., 2021) and evaluate our MIA framework on four datasets (ETTh2 (Zhou et al., 2021), ETTm
 (Zhou et al., 2021), weather 8 and illness 9 . Corresponding results are presented in Table 22 to 29. Extensive experiments demonstrate that MIA behaves similarly to that of univariate forecasting tasks.+}
Revised: {+For the multi-variate time series forecasting (MTSF) task, we follow the work (Wu et al., 2021) and evaluate our MIA framework on four datasets (ETTh2 (Zhou et al., 2021), ETTm
 (Zhou et al., 2021), weather 8 and illness 9 . Corresponding results are presented in Table 22 to 29. Extensive experiments demonstrate that MIA behaves similarly to that of univariate forecasting tasks.+}

Original: F Imputation Models.
Revised: G Imputation Models.

Original: For the imputation model architectures, we take SAITS, Transformer and BRITS and MLP-Mixer.
Revised: For the imputation model architectures, we take SAITS, Transformer and BRITS and MLP-Mixer.

Original: Specifically, for SAITS, we use the code from 9 .
Revised: Specifically, for SAITS, we use the code from 10 .

Original: For SAITS Du et al. , we set d model = 32 , n layers = 2 , d inner = 16 , n head = 4 , d k = 8 , d v = 8 .
Revised: For SAITS (Du et al., 2022), we set d model = 32 , n layers = 2 , d inner = 16 , n head = 4 , d k = 8 , d v = 8 .

Original: For Transformer Vaswani et al. , we set d model = 32 , n layers = 2 , d inner = 16 , n head = 4 , d k = 8 , d v = 8 .
Revised: For Transformer (Vaswani et al., 2017), we set d model = 32 , n layers = 2 , d inner = 16 , n head = 4 , d k = 8 , d v = 8 .

Original: For BRITS Cao et al. (2018), we set h hidden = 32 .
Revised: For BRITS (Cao et al., 2018), we set h hidden = 32 .

Original: For MLP-Mixer Tolstikhin et al. (2021), we use the same structure as Table 12.
Revised: For MLP-Mixer (Tolstikhin et al., 2021), we use the same structure as Table 12.

Original: In terms of training, we use optimizer Adam with lr = 0 . 0001 , β = (0 . 9 , 0 . 999) , ϵ = 10 − 8 , weight decay = 0 and train the model for 30 epochs. 
Revised: In terms of training, we use optimizer Adam with lr = 0 . 0001 , β = (0 . 9 , 0 . 999) , ϵ = 10 − 8 , weight decay = 0 and train the model for 30 epochs. [Table]

Original: Comparison of the imputation quality on imputing masked series.
Revised: [Table] G.1 Choice of Imputation Model Architecture.

Original: Table 14 compares the imputation quality of different imputation models on Traffic.
Revised: Table 14 compares the imputation quality of different imputation models on Traffic.

Original: The imputation quality is quantified by the mean square error (MSE) between the imputed series and the original series.
Revised: The imputation quality is quantified by the mean square error (MSE) between the imputed series and the original series.

Original: We observe that MLP-Mixer consistently outperforms other three models across different datasets and L adv , indicating the superior imputation ability of MLP-Mixer architecture.
Revised: We observe that MLP-Mixer consistently outperforms other three models across different datasets and L adv , indicating the superior imputation ability of MLP-Mixer architecture.

Revised: G.2 Impact of Gaussian Augmentation

Revised: Table 33, Table 34, Table 35, Table 36 and Table 37 report the impact of Gaussian augmentation at σ = 0 .

Revised: on time series classification dataset DistalPhalanxTW ( L mask = 15% ) respectively.

Revised: We compare the MIA with Gaussian augmentation to MIA without Gaussian augmentation (baseline) on **Certified Accuracy** at L atk =

Revised: The table reports the relative improvement (abs % ) on certified accuracy.

Original:  Visualize the imputation quality.
Revised: G.3 Visualize the imputation quality.

Original: Fig.
Revised: Fig.

Original: 4 intuitively shows the imputation performance of different imputation models on dataset Traffic. 
Revised: 4 intuitively shows the imputation performance of different imputation models on dataset Traffic. [Table] [Table] [Figure]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]