Towards Editing Time Series

Authors: baoyu  jing
shuqi  gu
tianyu  chen
zhiyu  yang
dongsheng  li
jingrui  he
kan  ren
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 17:08:00

Abstract: Synthesizing time series data is pivotal in modern society, aiding effective decision making and ensuring privacy preservation in various scenarios. Time series are associated with various attributes, including trends, seasonality, and external information such as location. Recent research has predominantly focused on random unconditional synthesis or conditional synthesis. Nonetheless, these paradigms generate time series from scratch and are incapable of manipulating existing time series samples. This paper introduces a novel task, called Time Series Editing (TSE), to synthesize time series by manipulating existing time series. The objective is to modify the given time series according to the specified attributes while preserving other properties unchanged. This task is not trivial due to the inadequacy of data coverage and the intricate relationships between time series and their attributes. To address these issues, we introduce a novel diffusion model, called TEdit. The proposed TEdit is trained using a novel bootstrap learning algorithm that effectively enhances the coverage of the original data. It is also equipped with an innovative multi-resolution modeling and generation paradigm to capture the complex relationships between time series and their attributes. Experimental results demonstrate the efficacy of TEdit for editing specified attributes upon the existing time series data. The project page is at https://seqml.github.io/tse.

Main paper: 
 introduction: 
 Time series data analysis plays a crucial role in various modern business sectors, including climate monitoring [22, 13, 14] , healthcare [12, 3, 46] , urban management [43, 16, 15] , and online service [35] . Time series are derived from diverse sources with inherent characteristics, such as system configurations, alongside external influential factors like environmental status. These factors are considered attributes of the time series. In many real-world scenarios, such as healthcare and urban monitoring, time series data tend to be sparse and privacy-sensitive, particularly concerning specific attributes that are rarely observed in real-world scenarios.Synthesizing time series data has emerged as a prominent research area aimed at addressing these challenges. Existing methods for time series synthesis range from unconditional generation [47, 31] to conditional generation [39, 27, 4] . Unconditional generation produces outputs based solely on the underlying distribution of the data. Without relying on any input conditions, the generated results are highly uncontrollable. Conditional generation controls the outcomes of the generated data based on the input conditions. However, conditional generation tends to produce samples around the data mean [40] , as demonstrated in our experiments, which could easily overlook detailed characteristics of the data, such as high-frequency patterns and local structures. Both unconditional and conditional generation paradigms synthesize time series from scratch, and they are incapable of manipulating existing samples. As a result, these paradigms are unable to answer "what if" questions in time series synthesis: given a time series, what would it become if some of its attributes are modified?Figure 1 : Illustration of the time series generation paradigms. Conditional generation generates time series from scratch, which usually generates samples around the dataset mean. Time series editing allows for the manipulation of the attributes of an input time series sample, which aligns with the desired target attribute values while preserving other properties.In this paper, we introduce a novel task, called Time Series Editing (TSE), for sample-level time series manipulation. More specifically, our objective is to directly manipulate specific attributes of an input time series to target values while maintaining consistency in other characteristics. For instance, consider observing air quality under season conditions in different cities as shown in Fig. 1 . On the left side, two example air quality time series for the spring and summer of London are illustrated. The spring one exhibits a lower amplitude with more noise, whereas the summer one features a higher amplitude with less noise. What would the resulting time series look like if we modify the season attribute of the spring one to summer while maintaining all other properties? As illustrated in Fig. 1 , it could have a larger amplitude with much noise. The task of TSE is complex due to several challenges. Firstly, the time series data distribution over the full composited attribute space is biased and may not be adequately covered, leaving gaps in our understanding, especially concerning unobservable or poorly defined attributes. For example, in climate data analysis, attributes like temperature and humidity are observable and well-defined. However, attributes like atmospheric pressure variations or localized microclimates may be challenging to observe or define accurately. Secondly, different attributes influence time series at varying resolutions. For example, trends have a global impact, while seasonality exerts a more localized influence. Modeling these multi-scale attributes and time series associations, while effectively controlling them, presents significant difficulties.To address these challenges, we introduce a novel method called Time Series Editor (TEdit), which is based on predominant generative models, specifically diffusion models [10, 38, 39, 27] . To address the data coverage issue, we propose a novel bootstrap learning algorithm, which leverages the generated data as pseudo-supervision for subsequent model learning. This algorithm helps improve the coverage of the whole attribute space and enhance the generation performance. To capture the intricate multi-scale associations between time series and attributes, we introduce a multi-resolution modeling and generation paradigm. The proposed multi-resolution paradigm can manipulate the given time series and attributes both effectively and efficiently. Our experiments, conducted on both synthetic and real-world datasets, demonstrate the effectiveness of our proposed solution. Specifically, our approach excels in generating precise time series under specified attributes, while keeping consistency in other attributes, showcasing the practical utility of our method.In summary, our main contributions are threefold. (1) We introduce and formulate the novel task of Time Series Editing (TSE). To the best of our knowledge, this is the first work exploring editing time series according to the given attribute configuration upon the input data. (2) We propose a novel multi-resolution diffusion model, called TEdit, with a bootstrap learning algorithm, which can flexibly capture the patterns at diverse granularity and gradually improve the generation performance.(3) We conduct and release a benchmark for TSE, including synthetic data and several carefully curated real-world datasets with a comprehensive evaluation protocol, aiming to facilitate further research in the community. 
background 2.1 time series generation: 
 A time series sample x ∈ R L is a series of L chronologically ordered sample points. Each time series sample adheres to some attributes a ∈ N K + , such as trend and the cycle number, and the k-th item a k corresponds to N k categorical options where k ∈ {1, ..., K}. In real-world scenarios, each time series may have several attributes, e.g., the air quality time series is associated with K = 2 attributes: location and season. Note that, not all the attributes can be observed due to practical limitations.Before describing the main task of the paper, we first briefly review the background of Time Series Generation. Unconditional Time Series Generation (UTSG) samples the time series data x upon the modeled data distribution p(x) such that x ∼ p(x), of which the generation process is uncontrollable. Conditional Time Series Generation (CTSG) generates the time series sample x based on the conditional distribution such that x ∼ p(x|a), where a is the input condition. CTSG tends to produce data samples near the dataset mean [40] . Both UTSG and CTSG paradigms generate time series from scratch and lack the ability to directly manipulate the existing samples.Diffusion models [10, 37, 38] learn to estimate and remove the random noise, which was added in the forward process onto the real-world data, through a sequence of sampling processes. Specifically, during training, random noise is gradually added to the original data sample x 0 = x1 via a Gaussian Markov transition q(x t |x t-1 ) := N ( √ 1 -β t x t-1 , β t I), where t ∈ [1, T ] indicates the diffusion step, and {β t } T t=0 are the predetermined variance schedule. The expression of latent variable x t can be simplified asx t = √ α t x 0 + √ 1 -α t ϵ, ϵ ∼ N (0, I), α t := Π t s=1 (1 -β s ).The learnable component of the diffusion models is a noise estimation network ϵ θ (x t , t, a), where a denotes the additional condition such as attributes. ϵ θ (x t , t, a) is trained by estimating the noise added to x t :min θ L(x 0 ) := min θ E ϵ∼N (0,I),t∼U (1,T ) ∥ϵ -ϵ θ (x t , t, a)∥ 2 2(1) where x 0 ∼ q(x 0 ) is sampled from the real data distribution, x t is a noisy version of x 0 .After training, given an attribute a, we can generate a sample x0 from random noise xT ∼ N (0, I) using ϵ θ (x t , t, a) with a sampler, e.g., deterministic Denoising Diffusion Implicit Model (DDIM) [38] :EQUATIONEQUATIONwhere f θ (x t , t, a) can be regarded as a prediction of x 0 at the diffusion step t.Adapting diffusion methods from the computer vision domain to the time series domain is not trivial. Time series is different from images, and it poses unique challenges for modeling and generating time series data, such as the complex multi-scale entanglement between time series and attributes. 
time series editing: 
 Given a time series and its associated attributes (x, a), a natural question arises: "what would it become if some of its attributes are modified?" In our work, we take a novel perspective of directly editing the given time series with the specified attribute modifications.In this section, we first formally formulate the problem of Time Series Editing (TSE) in Sec. Recall that, each time series sample x ∈ R L is associated with a set of attributes a ∈ N K + . Each attribute a k has N k options, k ∈ {1, ..., K}. For example, the trend type may contain N k = 4 values like linear, quadratic, exponential, and logistic. Let (x src , a src ) be a pair of source time series and source attributes to be edited, and a tgt be the desired target attributes. Comparing a tgt and a src , there are K edit edited attributes, e.g., trend types, and K prsv preserved attributes, e.g., cycle numbers, where K edit + K prsv = K. Now, we can formally define the task as below.The time series editing task is to build a function Φ θ to generate a target time series xtgt = Φ θ (x src , a src , a tgt ) by modifying the set of K edit edited attributes A edit and maintaining the set of K prsv preserved attributes A prsv as well as other information of x src .Given a pair of source time series and attributes (x src , a src ), generating the target time series xtgt corresponding to the target attributes a tgt entails two requirements. On one hand, the edited attributes A edit should be satisfied after the generation. On the other hand, the preserved attributes A prsv as well as other characteristics, e.g., noise, need to be maintained. Conditional generation [27] , such as directly adopting the conditional diffusion model described in Sec. 2.2, would fail the editing task because it does not take into consideration the detail characteristics of the source time series.In this paper, we propose a diffusion based two-stage procedure for TSE: Φ θ (x src , a src , a tgt ), which is illustrated in the upper part of Fig. 2 . In the first stage, we encode both the attribute semantics [6] and the detail characteristics [10] of the source time series x src 0 into the latent variable x src T via the deterministic forward DDIM process [18] :EQUATIONIn the second stage, originating from the latent variable xtgt T = x src T , we gradually generate the final target time series xtgt 0 with the consideration of the target attribute a tgt via the deterministic reverse DDIM process [38] :EQUATIONwhere f θ is given in Eq. (3) . Till now, we have detailed the procedure of the proposed TEdit. In the following subsections, we introduce the model architecture and the training algorithm.The key component of our TEdit is the noise estimator ϵ θ (x, t, a) in Eqs. (4) (5) . Though many diffusion model realizations have been proposed in other fields [18, 10, 37, 38, 24] and the time series domain [39, 27] , we found them inefficacious in modeling and generating time series data, especially when considering attributes. Few of them consider that different attributes can influence the time series on varying scales. We argue that it is important to take into account these differences. For instance, the trend types have a global impact on time series, while the seasonality affects the time series in local areas.In this paper, we propose a multi-resolution noise estimator. It first slices the input time series x t into several patch sequences of different resolutions r ∈ {1, ..., R}, where R is the total number of resolutions. Then it processes the patch sequences along with other information, e.g., attributes a and diffusion step t, to estimate the noise εr t for resolution r. Finally, the estimated noise for different resolutions are mixed together to obtain the final estimated noise εt . An illustration is presented in Fig. 3 . We elaborate on the details in the following content 2 .Patchifying. Following [29] , we patchify the input time series x ∈ R L , into a sequence of patch tokens P = {p 1 , ..., p Np }, where p i ∈ R Lp , L p is the window size and N p = ⌊Lp ⌋ is the patch number. After that, we encode them into embeddings P = {p 1 , ..., pNp }, where pi ∈ R D .Multi-Resolution Modeling and Generation. To model multi-resolution patterns and better control the conditional generation at multiple scales, we propose a multi-patch design with various patch lengths. Specifically, as shown in the left part of Fig. 3 , following the above patchifying operation, we encode the input time series x into patch embedding sequences { Pr } R r=1 of R resolutions, where Pr = {p r 1 , ..., prN r p }, pr i ∈ R D , N r p = ⌊ L-L r p L r p⌋ is the patch number, L r p is the window size. We set the patch length as L r p = b r-1 to produce exponential receptions, where b ∈ N + is the base. Thereafter, the processing module operates self-attention as the Transformer [42] to capture the input patterns for each resolution r, and it incorporates the attribute a and diffusion step t information to produce the embeddings { Pr } R r=1 . The details of the processing module are presented in Appendix D. The processed output { Pr } R r=1 is still in the form of multi-resolution sequences. The model subsequently decodes the diffusion noise at different resolutions back into the original space and integrates the outcomes at different resolutions to produce the final noise estimation, as shown on the right side of Fig. 3 . Specifically, for each Pr , the decoder transforms it to εr ∈ R L in the original time series space. The final estimated noise is obtained via a mixing upon the concatenation of estimated noise of different resolutions as ε = MLP([ϵ 1 , ..., ϵ R ]), where MLP(•) denotes Multi-Layer Perceptron.Parallel Processing. Though more effective, the proposed multi-resolution modeling and generation brings efficiency issues since it derives multiple token sequences with various sequence lengths. Iteratively processing each sequence is of low efficiency. Herein we design a novel parallelization with an attention masking mechanism in the processing module in Fig. 3 to harness the parallelism of the Graphics Processing Unit (GPU). Specifically, we first concatenate the input patch embedding sequences of different resolutions into a single vector P = [ P1 , ..., PR ] ∈ R D×(N 1 p +...+N R p ) . When calculating pair-wise self-attention scores as in the Transformer [42] , we use a mask matrix to mask out the inter-sequence attention operations across different sequences while preserving intra-sequence attention. Please refer to Appendix D for more details.Although any pretrained conditional diffusion model ϵ θ can be directly used to perform editing as shown in Sec. 3.2, the pretrained model is an essentially conditional generator, which has a great ability in generating time series directly from attributes but it might be less effective in modifying the attributes of an existing time series. The simplest way to improve the model's ability is to finetune ϵ θ via the ground-truth source (x src 0 , a src ) and target (x tgt 0 , a tgt ) pairs. However, the imaginary target x tgt 0 , which satisfies both a tgt and the details of x src 0 might be very rare or even does not exist in the real world. Thus, a key challenge for finetuning is how to effectively learn the information of x tgt 0 . Fortunately, the pretrained model is capable of generating some edited samples of a certain quality via xtgt 0 = Φ θ (x src 0 , a src , a tgt ). An example is shown in Fig. 2 . In this paper, we propose a bootstrap learning algorithm, which first pretrains ϵ θ based on the noise estimation loss L in Eq. ( 1) and then finetunes ϵ θ based on xtgt 0 with top confidence scores. Here, we briefly present the key steps of the bootstrap learning, and the full algorithm is given in Appendix E. Given a batch (X src 0 , A src ), whereX src 0 ∈ R B×L , A src ∈ N B×K +, B is the batch size, the finetuning works as follows:• Compose target attributes A tgt . For each (x src 0 , a src ) ∈ (X src 0 , A src ), we compose the imaginary target attributes a tgt based on a src by randomly sampling values for edited attributes A edit and keeping the values of the preserved attributes A prsv . Then we have the tuple (X src 0 , A src , A tgt ). dataset contains the ElectroEncephaloGram (EEG) data of subjects, who were required to imagine the movements of either the tongue or the left small finger. Each sample takes a length of 150 with 2 attributes: 2 movements and 64 channel ids.Each of the three datasets is associated with a pertaining dataset and several finetuning datasets. The pretraining datasets only contain the source time series x src and its attributes a src . During pertaining, the noise estimator ϵ θ is trained to denoise x src t with a src via Eq. ( 1). Each finetuning dataset corresponds to a specific split of A edit and A prsv . For example, the Synthetic dataset has 3 attributes, and thus it has 6 = 2 i=1 3 i different splits, namely, finetuning datasets. Similarly, both Air Quality and Motor Imagery datasets have 2 finetuning datasets. Finetuning datasets contain the source time series x src , source attributes a src , and target attributes a tgt . The process of generating the target attributes mirrors the target attribute composition step in bootstrap learning. Note that for Synthetic, the target time series x tgt is also available. During finetuning, we use the proposed bootstrap learning algorithm to further improve the model. More details can be found in Appendix F. Evaluation Metrics. A good editor should be able to generate xtgt that aligns with each a tgt k ∈ a tgt . The cornerstone of our evaluation is the Contrastive Time series Attribute Pretraing (CTAP) model, which extracts time series and attribute embeddings by learning their alignment. Our CTAP is similar to CLIP [33] . For details of CTAP, please refer to Appendix G. Based on the CTAP model, we introduce two metrics: CTAP score and Log Ratio of Target-to-Source (RaTS). (1) The CTAP score is similar to the CLIP-I score [23, 33] , which measures the alignment between the generated time series and the real-world time series which are associated with the given attribute value a tgt k . Specifically, we first use the CTAP model to extract embeddings ĥx , which is the embedding of xtgt , and hx |a tgt k , which is the average embedding of the time series associated with a tgt k in the training data. Then we calculate the cosine similarity of ĥx and hx |a tgt k , where the higher similarity indicates better alignment. ( 2) The RaTS score measures whether xtgt is closer to a tgt k than x src . Formally, the RaTS score for a tuple (x tgt , x src , a tgt k ) is defined as:EQUATIONwhere p(a k |x) is calculated by applying a softmax over the similarity scores of all (h x , h a k =i ) pairs, where h x is the CTAP embedding of x, and h a k =i , i ∈ {1, ..., N k }, is the CTAP embedding of the i-th possible value for the k-th attribute a k .We mainly evaluate xtgt from two perspectives: editability and preservability for edited attributes A edit and preserved attributes A prsv , respectively. For the edited attributes A edit , the higher RaTS and CTAP the better. For the preserved attributes A prsv , the higher CTAP the better. Since RaTS can take negative values, we use |RaTS| for A prsv , which measures the semantic divergence of xtgt from x src w.r.t a k . The lower |RaTS| means the better preservability. For more details, please refer to Appendix H. In addition to the semantic level evaluations, for the synthetic data, since we have ground truth target time series, we also use Mean Square Error (MSE), Mean Absolute Error (MAE) to perform the point level evaluation.Compared Methods. Since there is no existing method for TSE, we modify the popular time series diffusion model CSDI [39] and the recent conditional diffusion model Time Weaver [27] for TSE.As CSDI is unable to process attributes, we add an extra attribute encoder to incorporate attribute information. For Time Weaver, we slightly modify its attribute encoder for our settings. After pretraining the modified CSDI and Time Weaver, we use them to edit time series via the editing procedure described in Sec. 3.2. Our proposed TEdit-CSDI and TEdit-TW are implemented by incorporating core processing modules of CSDI and Time Weaver into our proposed multi-resolution noise estimator Sec. 3.3, and trained via Sec. 3.4. Architecture details are presented in Appendix D.Implementation Details. For all experiments, we set the number of diffusion steps as T = 50, embedding size for attributes and time series as 64, and use Adam optimizer [20] to train the model. For pretraining, we set (batch size, learning rate) as (256, 1e-3); for finetuning, we set them as (64,1e-7) for Synethtic and (32,1e-7) for Air Quality and Motor Imagery. We conduct a grid search for the hyperparameters of the multi-resolution. Considering the balance between performance and efficiency, we chose a compromise ratio for bootstrap. Specifically, (R, L p , ψ) = (3, 2, 0.5) for Synthetic, (3, 2, 0.5) for Air Quality, (3, 3, 0.5) for Motor Imagery. All our experiments were conducted on a single Nvidia-A100 GPU.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
main results: 
 In this section, we quantitatively evaluate the performance of comparison methods for editing and preserving attributes (RQ1) for all the datasets. For the edited attributes A edit , we report the RaTS and CTAP scores to depict the models' ability to edit attributes. For the preserved attributes A prsv , we report the |RaTS| and CTAP scores to depict the models' ability to preserve attributes. For the Synthetic dataset, since we have the ground truth, we also include MSE and MAE to evaluate the overall disparity between the ground truth x tgt and the generated xtgt .In Tab. 1, we present the averaged results over all the finetuning sets for each dataset. The detailed results for each finetuning sets are presented in Appendix I.1. Firstly, for the overall performance (MSE and MAE), the proposed TEdit could significantly outperform baselines. Secondly, for the edited attributes A edit , TEdit-CSDI and TEdit-TW could respectively outperform CSDI and Time Weaver on RaTS and TAP, showing that xtgt generated by TEdit could better fulfill the desired target A edit . Thirdly, for the preserved attributes A prsv , TEdit could basically maintain |RaTS| and TAP scores, which indicates TEdit is capable of preserving A prsv . In summary, these observations show that TEdit, including the multi-resolution and bootstrap learning, could improve the ability of conditional diffusion models to edit A edit and preserve A prsv .In this subsection, we conduct ablation studies on the Synthetic and Motor datasets based on the TEdit-TW to investigate the impact of the proposed multi-resolution modeling (RQ2) and the bootstrap training algorithm (RQ3). The averaged results over all the finetuning sets are presented in Tab. 2.Firstly, we compare TEdit-TW trained with BS (BootStrap) with TEdit-TW trained with GT (Ground Truth) on the Synthetic dataset 3 . For GT, we finetune TEdit-TW by minimizing MSE between the generated xtgt and the ground truth x tgt . It can be observed that GT performs much better than BS on the overall metrics (MSE and MAE), and it is only slightly better than BS on the attribute level metrics for both edited and preserved attributes, demonstrating that BS has a strong ability to capture the attribute semantic information. Secondly, we remove BS from TEdit-TW and compare its performance with the full TEdit-TW model. For the Synthetic dataset, BS could improve the overall scores, and maintain the performance on other attribute level metrics. For the Air dataset, BS primarily improves the scores of the edited attributes, with a trade-off in the scores of the preserved attributes. We hypothesize that there might exist some intrinsic connections between the two attributes: city and season. For the Motor dataset, BS can enhance the performance on the edited attribute while maintaining the performance on the preserved attributes. Thirdly, we investigate the contribution of MR (Multi-Resolution). For the Synthetic and Air datasets, MR could improve the scores on all metrics. For the Motor dataset, MR helps improve the scores for the edited attributes and mostly sustains the performance on the preserved attributes. These improvements highlight the effectiveness of the proposed MR.We further conduct an in-depth analysis of the effectiveness and efficiency of our TEdit, aiming to reveal how it works through quantitative comparison and visualization.Editing vs. Conditional Generation. Although conditional generation xtgt = Φ θ (∅, ∅, a tgt ) can produce time series that satisfy the specified target attribute values, it often struggles to retain sufficient details, particularly for difficult or unobservable attributes, since it generates samples from scratch. In contrast, editing xtgt = Φ θ (x src , a src , a tgt ) is designed to modify the attributes of existing time series, and thus is able to preserve much detail information. We first quantitatively compare the two modes on the Synthetic data. The averaged MSE and MAE over all the finetuning datasets are presented in Tab. 3, which shows that editing could significantly outperform conditional generation with much better (lower) MSE and MAE scores. We further showcase how editing and conditional generation work at the sample level. Fig. 4 , plots the generated results of these two modes under. We can observe that the edited time series could better align with the target attributes, Fig. 4 (a)(b), and preserve the characteristics of the source time series, e.g., bias in Fig. 4 (a). In comparison, conditional generation tends to produce smooth samples around the dataset mean. Additionally, we can also see in Fig. 4 Considering that multi-resolution modeling and generation brings additional efforts for processing different resolutions, as discussed in Sec. 3.3. We compare the running time of serial processing and parallel processing implementations of the multi-resolution paradigm with total resolution number R = 3 and patch length L r p = 3 r , r ∈ {0, ..., R -1}. The experiment is conducted on the Synthetic dataset with a batch size of 32. The serial processing iterative processes a single resolution at a time, whereas the parallel processing concatenates multiple resolution sequences along the length dimension and processes them simultaneously. We record the time it takes for the full noise estimator and the processing module to complete the forward diffusion process. As shown in Tab. 4, our designed parallel mechanism can significantly improve the inference efficiency.Bootstrap Improves the distributional coverage of the attribution space. Fig. 5 visualizes the dimension reduced distribution of (a) the raw data, (b) the generated data and (c) the mixed data, on the Synthetic dataset using t-SNE [41] . We can observe that the newly generated data fulfill the uncovered region of the original raw data therefore enhance the data coverage over the whole space, which explains how the bootstrap learning helps generative model training. More visualization analysis can be found in Appendix I.5.In this subsection, we conduct sensitivity analysis to study the impacts of hyper-parameters of the multi-resolution mechanism and bootstrap learning. Multi-resolution hyper-parameters. We perform a sensitivity study for the hyper-parameters of the proposed multi-resolution, i.e., the patch length L p and the number of resolutions R, on one Synthetic finetuning set, which aims to edit trend directions while preserving other attributes. The results are presented in Fig. 6 . More results on other Synthetic finetuning sets can be found in Appendix I.2.To analyze the impact of R, we fix L p = 2 and vary R. As shown in Fig. 6 (a), trend types prefer R = 3, whereas seasonal cycles perform better with R = 4. To examine the impact of L p , we fix R = 3 and vary L p . According to Fig. 6 (b), trend types show a preference for L p = 2 while seasonal cycles prefer L p = 3. Besides, we find that our proposed multi-resolution paradigm always performs better than the vanilla single resolution time-series modeling paradigms, i.e., L p = 1 or R = 1, which corroborates the discussion in Sec. 3.3 that attributes may influence the time-series generation at different scales and granularities.Bootstrap learning hyper-parameters. We conduct a sensitivity study on the hyper-parameters of bootstrapping, specifically the bootstrap ratio ψ within the range {0.1, 0.3, 0.5, 0.7, 0.9} on a Synthetic finetuning set, which aims to edit the trend direction while preserving others. As shown in Fig. 7 , the model exhibits a poor performance then the bootstrap ratio is low, e.g., 0.1. The model has a good performance with medium to high bootstrap ratios. We hypothesize that this is because tuning with only a few top samples might lead to certain mode collapse. In this paper, we introduced the task of Time Series Editing (TSE), which enables controllable time series synthesis by manipulating specific attributes of the input time series while maintaining consistency in other properties. There are two major challenges of TSE. Firstly, in the real world, the distribution of time series and attributes is usually biased, and the full space might not be adequately covered. Secondly, time series and attributes exhibit complex multi-scale entanglement.To address these two challenges, we propose a novel diffusion based approach, called TEdit, which incorporates a bootstrap learning algorithm and a multi-resolution modeling and generation paradigm.Comprehensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of our proposed TEdit in generating precise time series with specified attributes. In practice, our method still has certain limitations. For example, different attributes have varying difficulty degrees to edit; attributes may have complex inter-dependencies, and some attributes may be difficult to edit without affecting others. Despite this limitation, we believe our work lays the groundwork for controllable time series synthesis, potentially benefiting applications in fields such as climate monitoring, healthcare, and urban management.Time Series Generation. Time series generation has become a research hotspot in recent years [47] .From the view of modeling, early attempts for time series generation utilize Generative Adversarial Networks (GANs) [47, 31] , which trains the generator adversarial to the discriminator of correct classification on real or synthetic data. The latest research moves the focus to Diffusion models (DMs) [27, 36, 39] which have illustrated much higher fidelity and generation performance than GANs [45] . From the perspective of the generation paradigm, there are two branches. The first one is unconditional generation [47, 31, 34, 48] , which employs a generative model to generate data samples without relying on any conditions for regulation. The second paradigm is conditional generation [39, 27, 36, 44] making use of additional conditional information like category labels, textual descriptions, features, and other kinds of metadata to guide the generation process and enabling high-quality generation that is tightly correlated with the conditions. Among them, CSDI [39] enhances time-series imputation by treating observed time series as conditions in the diffusion model. TIME WEAVER [27] directly encode the metadata into the diffusion model to achieve controlled generation. Though they achieved better quantitative results, these methods suffer from unexpected outcomes that inadvertently emphasize trivial attributes while overlooking critical ones, especially with a low availability of time-series data over various attributes.Diffusion Model. Diffusion model [10, 37] has become increasingly prominent for data generation across various fields [26, 28, 2, 21] , which learns to gradually regress and remove the noise that was added to the clean data in the forward process, allowing data generation aligned with real distribution. Our work is highly related to conditional generation, which can be divided into two main streams of techniques. Classifier-guided generation [6, 19, 24] adds the control signal from the gradient of the classifier to bias the generation towards the specific class in the denoising process of the diffusion model, which requires training an additional classification model. Another stream of works [11, 30] stem from classifier-free guidance in data generation, without training any additional classifier.Data Editing. Data editing aims to manipulate the content of the given input to meet the requirements of the target data attributes, such as image style [17] , semantics [25] , and structure [7] , with the utilization of multimodal methods such as text-to-image synthesis [9] , [28, 49] and audio based image editing [50, 51] . In text-based scenarios, some works also generate texts under the instruction of specific attributes like sentiment [5] and structural information [32] . However, directly adopting the existing solution from other fields to time series editing is not feasible because of the significantly different data formats and poor data coverage of real-world time-series data w.r.t. different attributes.Our method offers several potential positive societal impacts in many real-world scenarios. For example, in climate monitoring, it enhances climate models and forecasts, aiding in climate change mitigation. For urban management, it supports better decision-making in areas such as traffic management, pollution control, and resource allocation. Additionally, in research, it facilitates robust experiments and innovative solutions by providing high-quality, customizable datasets.However, there are potential negative societal impacts to consider. There are data privacy risks associated with the manipulation of sensitive information, necessitating strong data protection measures. In financial markets, our method could be exploited for fraudulent activities, undermining market integrity. An overreliance on synthetic data may lead to biased or inaccurate models, resulting in flawed decision-making. Furthermore, there are ethical concerns regarding the potential misuse of our method in creating deceptive artificial intelligence models, which raises issues of transparency and accountability. While our method offers significant benefits, it is crucial to carefully manage these risks to maximize positive impacts and minimize negative consequences.In this section, we give a review of the diffusion process. Diffusion models are a class of two-stage generation models, consisting of a forward process and a reverse(denoising) process. The forward process is a Markov chain which gradually adds random noise on the origin data sample x 0 and intermediate latent variables x t through a Gaussian transition q(x t |x t-1 ) := N ( √ 1 -β t x t-1 , β t I), where t = 1, • • • , T indicates the diffusion step, and {β t } T t=0 are predetermined variance schedule. The latent variable x t can be described as:EQUATIONIn the reverse process, a denoiser ϵ θ (x t , t, a), where a ∈ N K is the given attributes, is needed to approximate the noise in each diffusion step to help to sample backward from x T through another Gaussian transition p θ (x t-1 |x t ) := N (x t-1 ; µ θ (x t , t), σ θ (x t , t)I). To obtain µ θ (x t , t), the denoiser should be trained by optimizing the loss:EQUATIONHere q(x 0 ) denotes the data distribution of x 0 (or x), t ∼ U(1, T ) indicates that the diffusion step t follows the uniform distribution between 1 and T .After training, given a condition a, we can generate the sample x0 from a random noise xT ∼ N (0, I) using the trained ϵ θ (x t , t, a) with a diffusion sampler, such as the deterministic denoising diffusion implicit model (DDIM) [38] :EQUATIONEQUATION)where f θ (x t , t, a) can be regarded as a prediction of x 0 at the diffusion step t.In this section, we will show the architecture utilized in our modified backbone models CSDI 8 and TIME-WEAVER 9, especially how the attributes are incorporated into the models. It should be mentioned that all the attributes in our experiment setting are categorical.Consider a batch of time series samples of size (N batch , F, L), where N batch represents the number of samples per batch, F represents the number of channels in the time series, and L represents the horizon. The corresponding attributes a is of shape (N batch , L, K), where K represents the number of the attributes. While performing the time-series inputs, each attribute option will be encoded to a corresponding embedding vector.As for CSDI, our modified part is highlighted in red. First, the attribute embedding will be linearly projected to shape (N batch , d, F, K), then concatenated with the time series embedding to form the prompt with shape (N batch , d, F, K + L). Next, the prompt is added with the expanded diffusion step embedding and the sum will be inputted into the transformer layer. After that, the attribute embedding will be discarded, and the shape gets back to (N batch , d, F, L).For TIME-WEAVER, We only make some slight changes to its original structure: 1. Our experiment setting doesn't include continuous type attributes, as mentioned. 2. We replace the self-attention modules with multi-layer perceptrons (MLPs).As mentioned in Sec 3.3, we introduce a novel parallelization with an attention masking mechanism to accelerate the training and inference. The attention mask is of the following form, denoted asM ∈ R (N 0 p +...+N R-1 p )×(N 0 p +...+N R-1 p) where M r ∈ 0 N r p , the residual value is negative infinity. Through adding this attention mask into the original attention calculation [42] and softmax operation softmax(z) i = e z i N j=1 e z j receiving z ∈ R N , such attention mask would mask out inter-sequence attention while preserving intra-sequence attention.M =     M 1 • • • -∞ . . . . . . . . . -∞ • • • M R-1As mentioned in Sec. 3.4, we propose a bootstrap learning algorithm that enables the model ϵ θ to learn the information of the imaginary target. Given the pretained model ϵ θ , the detailed algorithm for a batch (X src 0 ∈ R B×L , A src ∈ N B×K), where B is the batch size, L is the length, and K is the number of attributes, is presented below.Input:(X src 0 ∈ R B×L , A src ∈ N B×K + ), where B, L, K are batch size, length, and attribute number. Attributes to be edited A edit and preserved A prsv . The ratio ψ of the top bootstrapped samples to include for tuning.1: # 1. Compose tuple (X src 0 , A src , A tgt ).2: for k = 1, • • • , K do 3: if k ∈ A edit then 4: A tgt [:, k] ∼ U{1, • • • , N k }▷ Random sample a new value for the k-th attribute. We generate the synthetic time series according to the following formula:EQUATION)Table 5 shows the summary of the attributes for the synthetic dataset.For example, the attribute {Trend Type = linear, Trend Direction = up, Season Cycles = 2} is one of the total 32 (4 Trend Types × 2 Trend Directions × 4 Season Cycles) attribute combinations. We split the 32 combinations into train, validation, and test sets by 24:4:4.In the pretraining dataset, for each attribute combination, we sample 300 time series according to the random variances described in the following subsections. For the finetuning dataset, we treat each attribute combination in the 24:4:4 split as the target attribute. For each target attribute, we compose its source attribute based on the desired attributes to be edited A edit and preserved A prsv . For example, let A edit = {trend type}, A prsv = {trend direction, season cycles}. Suppose the target attribute a tgt = {linear, up, 2}, then its source could be a src 1 = {linear, up, 2}, a src 2 = {quadratic, up, 2}, a src 3 = {exponential, up, 2} and, a src 4 = {logistic, up, 2}. Then for a source and target attribute pair (a src , a tgt ), we generate the source and target time series (x src , x tgt ) based on the process described in the following subsections. Note that for each (a src , a tgt ) attribute pair, we first generate the deterministic part, e.g., trend type, for a src and a tgt , denoted by x src deter and x tgt deter . Then generate 10 randomnesses x rand , including noise, bias etc, and then we add these randomnesses to the deterministic part to ensure the source and target time series have the same randomness x rand :x tgt = x tgt deter + x rand , x src = x src deter + x rand . The finetuning dataset contains 6 subsets for editing different attribute combinations. For modifying trend type, there are 760 train samples, 140 validation samples, and 160 test samples. For modifying trend direction, there are 420 train samples, 70 validation samples, and 80 test samples. For modifying season cycles, there are 760 train samples, 140 validation samples, and 160 test samples. For modifying trend type and trend direction, there are 1440 train samples, 280 validation samples, and 320 test samples. For modifying trend type and season cycles, there are 2880 train samples, 560 validation samples, and 640 test samples. For modifying trend direction and season cycles, there are 1440 train samples, 280 validation samples, and 320 test samples. In addition, the same random variance and bias pair is used both on the source time series and target time series in a source-target pair respectively.In the following formulas, t i indicates the i-th sampling point in the original synthetic time series(t) and x i indicates the i-th sampling point in a contain form of x. For instance, the total length of a time series L = 128, then i = [1, . . . , 128].Trend Types. As described in table 5, there are 4 trend types: linear, quadratic, exponential, and logistic. As described above, t is used to obtain x. For the linear trend:x trend = t, in this case t i ∈ [0, 1], x i ∈ [0, 1]. For the quadratic trend: x trend = t 2 , in this case t i ∈ [0, 1], x i ∈ [0, 1]. For the exponential trend: x trend = 2 t 1024 , in this case t i ∈ [-10, 10], x i ∈ [0, 1].x trend is needed to range from 0 to 1, so t i is in [-10,10] . For logistic trend:x trend = 1 1+exp(-t) , where t i ∈ [-10, 10], x i ∈ [0, 1]. Similar to the exponential trend, we repeat a scaling process in logistic trend. To train the model more easily,x trend = (x -0.5) × 2 is used to normalize x trend to [-1, 1].Trend Directions. There are totally 2 directions: up and down. For instance, in the Cartesian coordinate system, a linear line from coordinates (0, 0) to (1, 1) represents an "up" trend, while another line from coordinates (0, 0) to (1, -1) represents a "down" trend. All 4 trend types introduce above are originally "up" trends. Therefore, for an "up" trend, we set x trend = x trend and for a "down" trend, we set x trend = -x trend .Random Variance. We randomly perturb the scale of the trend by: x trend = s • x trend where s is the random scale obtained by the following processs = m • c + (1 -m) • 1 c , m ∼ Bern(0.5), c ∼ U(0.8, 1.0) If m = 1, then s = c ∈ [0.8, 1.0]; else s = 1 c ∈ [1.0, 1.25].The season attribute is represented by the number of cycles in a time series. [0,1,2,4] sinusoidal wave cycles are randomly added.x season = a sin(2πt + ϕ)wheret ∈ [0, n cycle ], n cycle ∈ [0, 2 0 , 2 1 , 2 2 ].The Random Variances of these two variables follow uniform distributions: a ∼ U(0.4, 0.6), ϕ ∼ U(0, 2π).We simulate the real-world noise by a combination of Gaussian noise and high-frequency noise.x noise = x g + x hf (13) The Gaussian noise is sampled through x g = N (0, σ), σ ∼ U(0.04, 0.06). The high-frequency noise is portrayed by a sinusoidal wave:x hf = a sin(2πt + ϕ), t ∈ [0, n cycle ], n cycle ∈ [2 4 , 2 5 , 2 6], a ∼ U(0.08, 0.10), ϕ ∼ U(0, 2π).The last step is to add bias to the previous synthesized sample, which is randomly sampled from a uniform distribution x bias ∼ U(-0.5, 0.5). We processed the raw data and split it into train, validation, and test sets. We only selected the data of PM2.5, which is common in all stations. Season attributes are generated according to month. We slice the series by week to a length of 168 (24hour × 7day). For pretraining dataset, there are totally 3650 time series samples, in which we randomly pick 2825 time series samples as train, 353 time series samples as validation, and 472 time series samples as the test. For finetuning dataset, it's composed of multi-subsets where each subset corresponds to an editing setting, e.g. modifying the "season".In any subset, a sample is composed of a source time series, source attributes, and target attributes where the source time series are sampled from the corresponding splits of pretraining data. For each source time series, we randomly change the source attributes 2 or 3 times to get the target attributes and pair the source time series and target attributes together. Finally, we get (train: 2000, valid: 600, test: 600) samples for modifying "city" subset and (train: 3000, validation: 900, test: 900) samples for modifying "season" subset.The Motor Imagery dataset [1] contains some Electroencephalogram (EEG) records. During the Brain-Computer Interface experiments, the subjects have to perform imagined movements of either the tongue or the left small finger. All recordings are collected at a sampling rate of 1000 Hz with 64 channels. Every record contains 3000 time stamps (3 seconds measurement). For pertaining data, there are 19353 samples in the train set, 2419 in the validation set, 2419 in the test set. For fine-tuning data, the processing operation is similar to the Air Quality. In the subset of modifying movement, there are (train:2000, valid:1000, test:1000) samples. In the subset of modifying channel id, there are (train:3000, valid:1500, test:1500) samples.Our Contrastive Time-Series Attribute Pretraining (CTAP) model is similar to the popular CLIP model [33] . The purpose of this model is to train the time series encoder and the attribute encoder by learning the alignment between the time series x ∈ R L and its associated attributes a ∈ N K + . Note that we use separate attribute encoders for different attributes a k = a[k], k ∈ {1, ..., K}. An illustration of CTAP for a batch of X ∈ R B×L and their associated k-th attribute A ∈ N B + is presented in Fig. 10 . Note that the attribute index k is dropped for clarity. The time series encoder and attribute encoder extract the embeddingsH x = {h xi } Bi=1 and H a = {h ai } B i=1 . Following [33] , we calculate the pair-wise similarities between the h xi and h aj , and the encoders are trained by distinguishing whether h xi and h aj are from the same data pair. The pseudocode of the CTAP model is shown in Algorithm 2. For the synthetic dataset, MSE and MAE are two of the metrics used in our experiments. For the reason that we have targets in our synthetic datasets, MSE and MAE can be applied to evaluate the generated time series. The two formulas are listed below:EQUATIONEQUATION)Here |D| is the size of the test set, x i is the i-th target time series, and xi is the corresponding generated time series.The CTAP score is obtained through the Contrastive Time series -Attribute Pretraining (CTAP) model, which is similar to the popular CLIP model [33] . The details of the CTAP model are given in Appendix G. Suppose a tgt k is one of the target attributes, the CTAP score is similar to the CLIP-I score [23] , which uses the pretrained CTAP model to measure the alignment, e.g., cosine similarity, between the generated xtgt and the real-world time series that also associated with the attribute a tgt k . Specifically, we first use the CTAP model to extract embeddings ĥx , which is the embedding of xtgt , and hx |a tgt k , which is the average embedding for the time series associated with a tgt k in the training data. Then we calculate the cosine similarity of ĥx and hx |a tgt k , where the higher similarity indicates better alignment between the generated and real-world time series.We introduce Log Ratio of Target-to-Source probability (RaTS) score to measure whether the generated time series xtgt is closer to the target attribute a tgt k than the source time series x src . Formally, the RaTS score for a tuple (x tgt , x src , a tgt k ) is defined as:EQUATIONwhere p(a k |x) is calculated by applying a softmax over the similarity scores of all (h x , h a k =i ) pairs, where h x is the CTAP embedding of x, and h a k =i , i ∈ {1, ..., N k }, is the CTAP embedding of the i-th possible value for the k-th attribute a k .RaTS > 0 represents the model doesn't retain characteristics of the source time series x src to a certain degree. RaTS < 0 means that the attribute a tgt k is reinforced compared to the source x src . For the edited attributes A edit , the higher the RaTS score, the closer the edited xtgt is to the target attributes. For the preserved attributes A prsv , the lower the |RaTS| score, the better the preservation of the attributes. To analyze the influence of R, we fix L p = 2 and vary R. To examine the impact of L p , we fix R = 3 and vary L p . We found that different attributes exhibit varying preferences for multi-resolution parameters across different subsets. This observation supports the motivation for proposing a multi-resolution approach: distinct attributes exert different scales of influence on time series data. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: The information about the computer resources have been claimed in the appendix.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: This paper has discussed broader impacts in appendix.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper does not release models or real-world datasets, all the experiments are based on synthetic data and public datasets.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: All the assets used in this paper are open-source materials and are properly credited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.x0 and x are interchangeable in this paper.We discuss the noise estimator in the diffusion step t here, thus, we omit the notation of the diffusion step t without causing confusion.Ground truth is only available in the Synthetic dataset. 
 
 Appendix:  


Decision: This paper presents a novel approach to time series editing (TSE), a critical advancement in time series data synthesis. Unlike traditional methods that generate time series from scratch, which may lead to unexpected outcomes or trivial attribute prioritization, this work focuses on editing existing time series data to match specified attributes while preserving others. The authors propose a multi-resolution modeling framework, combined with bootstrap tuning on self-generated instances, to achieve this. The experiments demonstrate that this method is effective on both synthetic and real-world datasets, showcasing its potential to directly modify attributes while maintaining data integrity.

The significance of this work lies in its practical applicability and the innovation it brings to time series analysis. By enabling controlled edits on time series data, the proposed method addresses key challenges such as data sparsity and bias in time series generation. This paper not only offers a robust solution for generating high-quality time series data but also lays the groundwork for future research in domains where precise control over data attributes is necessary. The combination of a strong methodological foundation, comprehensive experimentation, and practical relevance makes this paper a valuable contribution to the field.

Reviews: review: paper_summary: paper_summary: This work proposes a new task as time series editing (TSE), that edits existing time series samples based on given sets of attributes. The goal is to perform edits that are only relevant to the changed attributes while preserving other existing attributes. The authors propose a multi-resolution modeling and generation model, and they perform bootstrap tuning on self-generated instances to perform the experiments. The authors show the proposed method can be used for modifying specified time series on synthetic dataset and real-world datasets.
strength_weakness: strength_weakness: While I personally like the idea a lot, there are a few non-neglectable issues (rooms for improvement):

1. The work could use better applications.
There are a couple of tasks/applications that could clearly benefit from TS editing. For example, see the works below:

[1] Cheng, J. Y., Goh, H., Dogrusoz, K., Tuzel, O., & Azemi, E. (2020). Subject-aware contrastive learning for biosignals. arXiv preprint arXiv:2007.04871.

[2] Liu, R., Azabou, M., Dabagia, M., Lin, C. H., Gheshlaghi Azar, M., Hengen, K., ... & Dyer, E. (2021). Drop, swap, and generate: A self-supervised approach for generating neural activity. Advances in neural information processing systems, 34, 10587-10599.

[3] Yi, K., Wang, Y., Ren, K., & Li, D. (2024). Learning topology-agnostic eeg representations with geometry-aware modeling. Advances in Neural Information Processing Systems, 36.

The above works include datasets with very specific attributes, e.g. subject information; task information; geo/channel information. If the proposed method could be applied to the above tasks, and demonstrate robustness, the paper could be much stronger.

2. Experimental results are overall not convincing enough. The datasets are too small, and there is no enough information about the synthetic dataset. There are not a lot of baselines.

3. Some parts of the method section can use better presentation:
- Section 3.1 How does the Time Series Editing task differ from the normal editing task in vision? Why do the authors want to define a new definition for it?
- Section 3.2 The authors should argue why is diffusion necessary for performing the task. What advantages would it bring?
- Section 3.4 The authors should compare the proposed method with traditional SSL methods like BYOL & Simclr and discuss about their differences.
questions: questions: NA
limitations: limitations: The authors discussed limitations.
paper_summary: paper_summary: This paper introduces time series editing (TSE), a novel approach for modifying existing time series to match specified attributes while preserving other attributes. Unlike traditional methods that generate time series from scratch, TSE employs a multi-resolution modeling and bootstrap tuning framework, enhancing data coverage and generation precision. The paper presents a multi-resolution diffusion model and a comprehensive benchmark dataset, demonstrating the method’s effectiveness on both synthetic and real-world data.
strength_weakness: strength_weakness: - The paper focuses on editing specific attributes of time series while preserving others but does not extensively discuss the handling of diverse attribute types and their interactions.
- The reproducibility is uncertain without code and detailed implementation information in Appendix I.
- There are minor writing issues, such as “levela” in line 291, page 8, that need correction.
questions: questions: I have no specific questions.
limitations: limitations: The authors adequately addressed the limitations.
paper_summary: paper_summary: The paper discusses the challenges of synthesizing time series data influenced by intrinsic and extrinsic factors. It highlights the limitations of existing benchmarks and methods and suggests a new task, Time Series Editing (TSE), which modifies given time series based on specified properties while preserving other attributes. In addition, the paper suggests a new method based on generative bootstrapping and the incorporation of multi-resolution modeling to enhance data coverage, demonstrating its efficacy in experiments.
strength_weakness: strength_weakness: - The work "On the Constrained Time-Series Generation Problem" [1] introduces time series generation and editing. Although there are certain differences between the current work and the above work, they can be seen as similar. However, the paper does not reference or discuss the similarities and differences. I think that incorporating such a comparison is crucial. In addition, this somehow negatively affects the first novelty that the offers suggested. Nevertheless, I still think the task is novel enough.

- The main experiment results in Tables 1 and 2 are on specific attributes; to show robustness, it is necessary to show or average all the possible preserved-changed attribute groups. 

Multi-resolution models for time series tasks are already in use [2, 3], which limits the novelty. I want to caveat this weakness by stating that [2] is not in the context of generation, and [3] is almost concurrent with this work.


[1] - On the constrained time-series generation problem. Coletta, Andrea and Gopalakrishnan, Sriram and Borrajo, Daniel and Vyetrenko, Svitlana.
[2] - Multi-resolution Time-Series Transformer for Long-term Forecasting. Yitian Zhang, Liheng Ma, Soumyasundar Pal, Yingxue Zhang, Mark Coates
[3] - Multi-Resolution Diffusion Models for Time Series Forecasting. Lifeng Shen, Weiyu Chen, James Kwok
questions: questions: - Can the author present all the possible preserved-edited groups in the main results? Currently, in my opinion, the main results are not extensive enough. 

- Why did the author not mention "On the Constrained Time-Series Generation Problem"?
limitations: limitations: detailed on the weakness
paper_summary: paper_summary: This paper introduces Time Series Editing (TSE), a method for generating time series data with control over their attributes. TSE generates a sample by modifying an existing time series through manipulation of specific attributes while maintaining consistency in others. This approach leverages the hypothesis that patterns in time series synthesis are transferable between attribute values with consistent shared attributes. The authors propose a novel multi-resolution diffusion-based model for sample generation, that is trained using a bootstrapped learning algorithm. They show that this architecture and training procedure choice overcomes challenges related to biased data distribution and varying degrees of influence different attributes have on time series generation.
strength_weakness: strength_weakness: $\textbf{Edited attribute set}$: In the experiment section, the results are for a specific type of change (a specific set of edited attributes). It is not clear why this specific combination was selected. I think the performance should be reported for all possible combinations, or the average of all possible combinations. Another interesting extension to this would be to look into these averaged performances, as a function of the size of the manipulated set of attributes. This might not be helpful for the selected real-world datasets because they only have 2 attributes, but at least the authors can assess this in the synthetic setting. 

$\textbf{Standard errors}$: Some of the reported performances are very close and it is difficult to assess their significance. The authors need to add confidence intervals to the reported numbers in all tables. 

$\textbf{RaTS}$: It is not clear how $p(a_k^{tgt}|x)$ is estimated in order to calculate the RaTS performance score? 

$\textbf{MSE for real data}$: In general, it is difficult to know the ground-truth for edited time series in real-world dataset. However, I believe in the existing datasets, this is not impossible. For instance, in the Air dataset, if the changed attribute is the city, we have the corresponding time series in the data. This way the authors can also report MSE/MAE for the real-world datasets as well. This is important because even though the other metrics assess important qualities for the model, they cannot tell how well or realistic the generated signal is.  Adding this evaluation can help better assess the performance of TSE on real-data.
questions: questions: $\textbf{Correlated attributes}$: In many real-world applications, the underlying attributes are not independent. As a result, randomly changing attributes while fixing others can result in out of distribution or unrealistic samples. I'm wondering if this becomes a problem especially in the bootstrap self-scoring step where the model will end up overfitting to out of distribution samples. 

$\textbf{Distributional coverage}$: I'm not sure the conclusion drawn from Figure 5 is correct. The figure shows T-SNE projection of the data and the bootstrapped data, and the conclusion is that the generated samples fill in the gaps in the original data space. However, with methods like T-SNE, the projection is a function of the data they are trained on. Which means comparing the three subfigures that are representing different data cohorts is not possible. Can the authors provide more explanation on this?

$\textbf{Unseen attributes}$: Does the approach extend to unseen attribute values? Can you generate a time series with an edited attribute where the value has never been seen in the training data?

Note: I would be willing to increase my score if some of the concerns mentioned in the review are addressed.
limitations: limitations: $\textbf{Evaluating generated time series samples}$: One of the biggest challenges for generative models in time series is evaluating the generated samples. The proposed metrics and evaluations are useful in assessing different aspects of the performance of TSE, but there is no discussion on how we can assess whether a generated sample is realistic or not. Even a low MSE by itself doesn't guarantee high-quality generated samples. I think this is something worth discussing in the paper.

Comments: title: Look forward to hearing from you
Thanks
Response
Response from Reviewer ZaGr
Respose
Rebuttal response
comment: Thank you for your feedback!

We will revise our paper according to your suggestions.
Thank you!

We are happy to hear that all the concerns have been addressed!
We are pretty much encouraged by your constructive and insightful comments and feedback! 
We will further revise our paper according to your suggestions.
Dear Reviewer tfFr,

Thanks for your valuable comments and feedback!
We hope our response could help address your concerns.
Today is the last day of discussion. We look forward to hearing from you and hope to discuss any concerns or suggestions to further improve our work.

Thank you!

Best, 

Authors of the 10940 submission
Overall I like the work, especially the possibilities it brings to the field. I'd suggest the authors include more discussions during revision, especially regarding the applications that I suggested to encourage future follow-ups. The rebuttals also provided some other evidence for me to further advocate for the work and thus I will raise my score.
TAP is similar to the popular CLIP [2], and the purpose of TAP is to align time series $\mathbf{x}$ and attributes $\mathbf{a}$ in the embedding space, which is trained by distinguishing the positive and negative ($\mathbf{x}$,$\mathbf{a}$) pairs.
The details of TAP are presented in Appendix G, including architecture, pseudocode for training, and its performance.
TAP has two encoders: (1) a time series encoder, which is based on PatchTST [3] and (2) an attribute encoder (an MLP). 
For different datasets, parameters are tuned e.g., patch lengths, 8,16,32 .., learning rate 1e-2, 1e-3, 1e-5 ...
On average, top1/top2 accuracy for different attributes of the Synthetic, Air and Motor datasets are 0.9145/0.9797, 0.9397/0.9746, 0.9258/0.9682 respectively.

## 2. Model Architectures
Our implementation of the noise estimator $\bf{\epsilon}_\theta$ is based on the public CSDI [1] code, and most of the architecture configurations follow CSDI such as variance schedules $\beta_t$ and $\alpha_t$ of diffusion. 
For detailed values of the critical hyper-parameters, please see the table below.
Illustrations of model architectures are presented in Figure 3 in our main paper and Figures 9 and 10 in Appendix H. 

Here, we summarize the key components of the model architectures.
Given a time series $\mathbf{x}\_t$, our model will first slices $\mathbf{x}\_t$ into sequences with $R$ different resolutions (with different patch lengths) $\\{\mathbf{P}^r\\}\_{r=0}^{R-1}$.
Each sequence $\mathbf{P}^r$ will be fed into the processing module to predict the noise of $\mathbf{P}^r$.
We implement two versions of processing modules TEdit-CSDI and TEdit-TW based on CSDI [1] and Time Weaver [4], respectively.
Detailed figures, dimension sizes, and operations are presented in Figures 9 and 10 in Appendix H.
Our modified parts are highlighted in red in Figures 9 and 10.
Briefly speaking, CSDI does not take attributes as inputs and TEdit-CSDI incorporates attributes as prompt embeddings used in [5].
For Time Weaver, it is also based on CSDI and uses another strategy to incorporate attribute information.
We only make a slight modification by replacing the self-attention for processing continuous attributes with MLPs, since we do not consider continuous attributes in the current version.


## 3. Hyper-parameters

There are two types of hyper-parameters: model parameters and training parameters, which are listed below.
Many model parameters follow the settings of CSDI [1].
In the revised version, we will add more details.

- **Table 1: model parameters**

|Hyper-parameters|Descriptions|Values|
|:------:|:------:|:------:|
|$d$|Model dimension|128|
|$N$|The number of layers of the noise estimator|3|
|$R$|The number of resolution|2,3,4|
|$L_P$|Patch length|2,3|
|$T$|Diffusion steps|50|
|$S$|The start value for variance schedule|0.0001|
|$E$|The end value for variance schedule|0.5|
|$\beta_t$|Variance schedule for diffusion|$[\frac{S-E}{T-1}+S*(t-1)]^2$|

- **Table 2: training parameters**

|Hyper-parameters|Descriptions|Values|
|:------:|:------:|:------:|
|${E}_{pt}$|The epoch number of pretraining|300~1000|
|${lr}_{pt}$|The learning rate of pretraining|1e-3|
|${E}_{ft}$|The epoch number of finetuning|50|
|${lr}_{ft}$|The learning rate of finetuning|1e-5~1e-7|
|$\psi$|Bootstrap ratio|0.0~1.0|
|$\omega$|Bootstrap weight|0.0~1.0|

[1] Tashiro et al. CSDI: Conditional score-based diffusion models for probabilistic time series imputation. NeurIPS'2021

[2] Radford et al. Learning transferable visual models from natural language supervision. ICML'2021.

[3] Nie et al. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. ICLR'2023

[4] Narasimhan et al. Time weaver: A conditional time series generation model. ICML'2024

[5] Jia et al. Visual prompt tuning. ECCV'2022
Thanks for your positive feedback!

We are happy to hear that most of your concerns have been addressed.
As for the replication, we have sent AC an anonymous version of the code and data for the replication test, which is allowed under the regulation of NeurIPS rebuttal.
Due to the rebuttal policy of NeurIPS, we are not allowed to add external links in the comments.
We promise that we will publish the whole framework including the data preparation, training, and evaluation pipeline, for reproducibility upon the acceptance of our paper.

Here, we provide the details of our implementation as below, including pseudo code for training and evaluation, model architectures, and hyper-parameters, hoping to better help improve the reproducibility of our work.

## 1. Pseudo code 
>**Pre-training**
>
>Inputs: \#epochs $N_{epoch}$, \#batches for each epoch $N_{batch}$, batch size $B$, noise estimator $\epsilon_\theta$, pretraining dataset $\mathcal{D}$, total diffusion step $T$, variance schedule $\\{\beta_t\\}_{t=1}^T$
>
>For epoch $n_{epoch}<N_{epoch}$:
>
>$\qquad$For batch $n_{batch}<N_{batch}$:
>
>$\qquad\qquad$ \# 1. Load data
>
>$\qquad\qquad$ Load a batch of $\mathbf{X}, \mathbf{A}$
>
>$\qquad\qquad\qquad$ \# 2. Train $\Phi$ via Eq.1
>
>$\qquad\qquad\qquad$ For each sample $\mathbf{x}\in\mathbf{X},\mathbf{a}\in\mathbf{A}$:
>
>$\qquad\qquad\qquad$ $t\sim Uniform(1,T)$ \# Sample a diffusion step $t$ 
>
>$\qquad\qquad\qquad$ $\epsilon\sim\mathcal{N}(0,1)$ \# Sample Gaussian noise $\epsilon$
>
>$\qquad\qquad\qquad$ $\mathbf{x}\_t=\sqrt{\alpha\_t}\mathbf{x}\_0+\sqrt{1-\alpha\_t}{\epsilon},(0,\mathbf{I}),~\alpha\_t:=\Pi\_{s=1}^t(1-\beta\_s)$ \# Get noisy time series $\mathbf{x}\_t$
>
>$\qquad\qquad\qquad$ $\hat{\epsilon}=\epsilon_\theta(\mathbf{x\_t},t,\mathbf{a})$ \# Predict the noise
>
>$\qquad\qquad\qquad$ $l=||\hat{\epsilon}- \epsilon||^2$ \# Noise estimation loss
>
>$\qquad\qquad\qquad$ Update the noise estimator $\epsilon_\theta$ by minimizing $l$

>**Finetuning**
>
>Inputs: \#epochs $N_{epoch}$, \#batches for each epoch $N_{batch}$, batch size $B$, diffusion model $\Phi$ (the core component is the noise estimator $\epsilon_\theta$), bootstrap ratio $\psi$, finetuning dataset $\mathcal{D}$
>
>For epoch $n_{epoch}<N_{epoch}$:
>
>$\qquad$For batch $n_{batch}<N_{batch}$:
>
>$\qquad\qquad$ \# 1. Load data
>
>$\qquad\qquad$ Load a batch of $\mathbf{X}^{src}, \mathbf{A}^{src}, \mathbf{A}^{tgt}$
>
>$\qquad\qquad$ \# 2. Bootstrapping
>
>$\qquad\qquad$  $\hat{\mathbf{X}}^{tgt}\leftarrow\Phi(\mathbf{X}^{src},\mathbf{A}^{src}|\mathbf{A}^{tgt})$ \# Generate the target $\mathbf{X}^{tgt}$ by editing $\mathbf{X}^{src}$ as Eqs.4-5
>
>$\qquad\qquad$ $\hat{\mathbf{X}}^{src}\leftarrow\Phi(\hat{\mathbf{X}}^{tgt},\mathbf{A}^{src}|\mathbf{A}^{tgt})$ \# Back-translation Eqs.4-5
>
>$\qquad\qquad$ $\mathbf{s}\leftarrow MSE(\hat{\mathbf{X}}^{src}, \mathbf{X}^{src})$ \# Get self-score via MSE
>
>$\qquad\qquad$ $\hat{\mathbf{X}}^{tgt}\leftarrow Sort(\hat{\mathbf{X}}^{tgt}|\mathbf{s})$ \# Sort $\hat{\mathbf{X}}^{tgt}$ based on $\mathbf{s}$ 
>
>$\qquad\qquad$ $\hat{\mathbf{X}}^{tgt}\leftarrow\hat{\mathbf{X}}^{tgt}[:,B\cdot\psi]$ \# Keep the top $\psi$ samples with lowest MSE scores $\hat{\mathbf{x}}^{tgt}\in\hat{\mathbf{X}}^{tgt}$
>
>$\qquad\qquad$ \# 3. Calculate loss
>
>$\qquad\qquad$ Calculate the noise estimation loss based on the bootstrapped sample $\hat{\mathbf{X}}^{tgt}$. \# Similar to Eq.1 for pertaining 
>
>$\qquad\qquad$ Update the model $\Phi$ via backpropogation.

>**Evaluation**
>
>Inputs: model $\Phi$, dataset $\mathcal{D}$, \#batches $N_{batch}$, Time series Attribute Pretraining model (TAP)
>
>For batch $n_{batch}<N_{batch}$:
>
>$\qquad$ \# 1. Load data
>
>$\qquad$ Load a batch of $\mathbf{X}^{src}, \mathbf{A}^{src}, \mathbf{A}^{tgt}$, $\mathbf{X}^{tgt}$  \# $\mathbf{X}^{tgt}$ is only available for Synthetic data
>
>$\qquad$ \# 2. Generate $\hat{\mathbf{X}}^{tgt}$ by editing $\mathbf{X}^{src}$
>
>$\qquad$ $\hat{\mathbf{X}}^{tgt}\leftarrow\Phi(\mathbf{X}^{src},\mathbf{A}^{src}|\mathbf{A}^{tgt})$
>
>$\qquad$ \# 3. Evaluate $\hat{\mathbf{X}}^{tgt}$
>
>$\qquad$ Calculate RaTS scores (Appendix F.3) and TAP scores (Appendix F.2) based on the TAP model (Appendix G).
>
>$\qquad$ Calculate $MSE(\hat{\mathbf{X}}^{tgt}, \mathbf{X}^{tgt})$ if $\mathbf{X}^{tgt}$ is available
Thank you. You have addressed all of my concerns. After the rebuttal and the reviewer's comments, I think the paper will be more substantial and impactful with the revised version. Therefore, I will raise my score. 

If I may add a comment, I find this task very interesting. I hope that more works that will add real-world datasets or complex synthetic data will come in the future and challenge this task, as today is pretty simple. Thanks.
Thank you so much for your insightful comments!

For the first concern, indeed, leaving $\mathbf{x}^{src}$ and $\mathbf{a}^{src}$ empty, the editing task will degenerate to the conditional generation task, as is mentioned by your comments. 
Therefore, *conditional generation could be regarded as a special case of editing*.
Actually, we can directly evaluate the performance of conditional generation on our editing datasets by masking out the information of the source $\mathbf{x}^{src}$ and $\mathbf{a}^{src}$.
Specifically, for the editing task, each sample has three inputs $\mathbf{x}^{src}$, $\mathbf{a}^{src}$ and $\mathbf{a}^{tgt}$; for the conditional generation task, only $\mathbf{a}^{tgt}$ is available to models.

To further enhance our experiments regarding to your suggestions, we conducted additional experiments on the conditional generation task, which will be put into the revised paper version as a supplementary experimental setting of our work.
In the table below, we report the averaged performance of conditional generation of Time Weaver and our TEdit-TW on different Synthetic datasets: all 6 possible edit-preserve attribute groups.
Please note that the RaTS score is not suitable for the conditional generation and we have not reported it in this new experiment,
since RaTS scores are calculated based on both $\mathbf{a}^{src}$ and $\mathbf{a}^{tgt}$, and $\mathbf{a}^{src}$ is unavailable for the conditional generation task.
It can be seen that our method TEdit-TW can significantly outperform Time Weaver for the conditional generation task.
Following your suggestions, we will add the conditional generation task to our benchmark in the revised version. Thank you for your valuable suggestion.

#### Tabel 1. Averaged performance of conditional generation on 6 different Synthetic datasets: all 6 possible edit-preserve attribute groups.
|Method|MSE$\downarrow$|MAE$\downarrow$|TAP_trend_type$\uparrow$|TAP_trend_direction$\uparrow$|TAP_season_cycle$\uparrow$|
|:--------:|:--------:|:--------:|:--------:|:---------:|:--------:|
|Time Weaver|0.3509|0.4691|0.7685|0.9880|0.2035|
|TEdit-TW|0.2414|0.3958|0.8353|0.9899|0.2568|


Regarding the second concern, we are sorry that the "different attributes" in the title caused the confusion. 
The "different attributes" actually means "all possible combinations of edit-preserve groups". 
For the synthetic dataset, it includes the following (edit | preserve) sets: 

- (trend type | trend direction, season cycle)
- (trend direction | trend type, season cycle)
- (season cycle | trend type, trend direction)
- (trend type, trend direction | season cycle)
- (trend type, season cycle | trend direction)
- (trend direction | trend type, season cycle). 

For the Air dataset, it includes the following (edit | preserve) sets: 
- (city | season) 
- (season | city)

The scores in Tables 1 and 2 in the previous response are averaged over all these combinations, and the scores for each edit-preserve group are presented in the PDF files of the general response. 

As for the third concern, as is mentioned by your comments, our novelty lies in the multiresolution modeling in time series editing and generation task, which is unique and different compared to the concurrent related works mentioned by your previous review comments.
We are encouraged by your positive comments and we think that a more proper description would help better understand these works in the revised paper.

Thank you once again for acknowledging our work. We will carefully revise our paper in line with your valuable suggestions, including:
* Unifying the definition framework of editing with conditional generation and providing a clearer discussion of their differences and distinctions.
* Incorporating additional experiments on our method within the conditional generation setting, to enhance the comprehensiveness of our experiments.
* Refining our claims to more properly describe our contributions, particularly in relation to the multiresolution mechanism in generation and editing tasks.

We are happy to have more discussions if you have any other concerns or suggestions.
Thank you so much for your recognition and we are very much encouraged by your positive feedback and increased scores. 

We will further improve our paper according to your suggestions. If there are any other suggestions or questions you'd like to discuss, please don't hesitate to let us know. We are happy to further discuss the opportunities and challenges of this new and exciting Time Series Editing (TSE) task.
Thank you for your detailed response, which addresses some of my concerns. I appreciate the effort you put into it. I will maintain my current rating unless more direct replication evidence is provided.
Thank you for your response.

Regarding the first concern, I agree that clarifying the differences in task definitions is important. In my opinion, contrasting these definitions clearly is crucial. Additionally, a universal formulation for both tasks should be considered. For instance, if x_source  and a_source are left empty, both tasks could be viewed as the same. Therefore, I believe that these problems should be unified under a single benchmark. I encourage the authors to explore how this framework can be unified in the final revision of the paper.

As for my second concern, I am unclear if the reported results are averaged across all editing scenarios. The title mentions "different attributes," which suggests that it may only cover a subset of the possible editing scenarios. I believe an extensive evaluation across all possible editing scenarios should be conducted on both datasets to thoroughly assess the method's robustness. Could you clarify this?

Lastly, thank you for providing clarification on the third concern. I believe the approach you've provided is solid enough to serve as a reliable baseline for benchmarking, which is significant. As for the claimed novelty, I think it should be specifically attributed to adapting the multiresolution approach for time series editing and generation.
Thank you for the detailed rebuttal. I believe the added evaluations and results will be a good addition to the paper. The rebuttal has covered most my concerns and I'm happy to edit my score.