VerbalTS: Generating Time Series from Texts
Authors: Shuqi Gu
Chuyue Li
Baoyu Jing
Kan Ren
Venue: ICML 2025 poster (2025)
Keywords: time series
conditional generation
diffusion model
multi-modality
Primary area: applications->time_series
TL;DR: VerbalTS is a framework that generates time series from unstructured text. It uses multi-focal alignment for improved quality and semantic match, outperforming existing approaches on various datasets.

Abstract: Time series synthesis has become a foundational task in modern society, underpinning decision-making across various scenes. Recent approaches primarily generate time series from structured conditions, such as attribute-based metadata. However, these methods struggle to capture the full complexity of time series, as the predefined structures often fail to reflect intricate temporal dynamics or other nuanced characteristics. Moreover, constructing structured metadata requires expert knowledge, making large-scale data labeling costly and impractical. In this paper, we introduce VerbalTS, a novel framework for generating time series from unstructured textual descriptions, offering a more expressive and flexible solution to time series synthesis. To bridge the gap between unstructured text and time series data, VerbalTS employs a multi-focal alignment and generation framework, effectively modeling their complex relationships. Experiments on two synthetic and four real-world datasets demonstrate that VerbalTS outperforms existing methods in both generation quality and semantic alignment with textual conditions.

VerbalTS: Generating Time Series from Texts
Shuqi Gu Chuyue Li!
Baoyu Jing?
Kan Ren
Abstract
Sample speoific information
Sequence events
Time series synthesis has become a foundational task in modern society, underpinning decision- making across various scenes. Recent approaches primarily generate time series from structured con- ditions, such as attribute- based metadata. How- ever, these methods struggle to capture the full complexity of time series, as the predefined struc- tures often fail to reflect intricate temporal dy- namics or other nuanced characteristics. More- over, constructing structured metadata requires expert knowledge, making large-scale data label- ing costly and impractical. In this paper, we intro- duce VERBALTS, a novel framework for generat- ing time series from unstructured textual descrip- tions, offering a more expressive and flexible so- lution time series synthesis. To bridge the gap between unstructured text and time series data, VERBALTS employs a multi-focal aligmment and generation framework, effectively modeling their complex relationships. Experiments on two syn- thetic and four real-world datasets demonstrate that VERBALTS outperforms existing methods in both generation quality and semantic alignment with textual conditions. The project page is https://seqml.
Unutractareds
The season cyde 6-4
The time serdes hav. knnor toend
Unstoodarol:
woateer is showing light rain The weother overall B expected to be partly broken dloud wits modorate nan eter on.
the maming
The current
The beginning part hax double paoks
Sirumtueet
Souutured
Trand type linzar
Waather:
Trndd diresion Seoscnoyyee
Light on
Good
Figure 1. Illustration of the different control conditions. Com- pared to structured conditions, unstructured conditions are better at capturing unique, sample-specific information, c.g., special local shapelets that are not present in all samples (left figure). Addition- ally, structured conditions fail to effectively describe the sequence of events, common characteristic of time series (right figure).
scratch (Narasimhan et al., 2024) or modifying existing ones (Jing al., 2024a), enabling richer datasets and ad- vancing time series modeling.
Current time series generation tasks can be broadly classi- fied into unconditional (Desai et al., 2021; Yuan & Qiao, 2024; Pei et al., 2021; Yoon et al., 2019) and conditional generation (Wang et al., 2023; Lee et al., 2023; Coletta et al., 2024). Unconditional generation involves randomly sampling from the learned time series distribution, offering minimal control over the properties of the generated samples. In contrast, conditional time series generation usually lever- ages structured conditions such as metadata (Narasimhan et al., 2024), time series attributes (Jing et al., 2024a) or class labels (Li et al., 2022; Wang et al., 2023).
1. Introduction
Time series modeling plays a crucial role in modern soci- cty, with applications spanning finance (Gao et al., 2024), medicine (He et al., 2023; Chen et al., 2024b; Jarrett al., 2021), climate (Jing et al., 2021: 2022; 2024b;c), and en- ergy (Lai al., 2018). However, challenges like complex data collection and the rarity of extreme conditions limit access to large-scale. high-quality time series data. To ad- dress this, research has explored generating time series from
Generating time series from structured conditions is straight- forward but comes with notable limitations. First, much of the real-world information about time series is unstructured, making the extraction of structured features from large, un- organized datasets both time- cone-consuming and iabor-intenssve. Second, as illustrated in Fig. 1, this process often leads to significant information loss. For instance, sample-specific details that cannot be encapsulated within a uniform struc- ture are frequently omitted. Similarly, sequential conditions that govern the chronological characteristics of time series may be inadequately captured by structured representations.
'School of Information Science and Technology, ShanghaiTech University, Shanghai, China Univeersity of Illinois at Urbana- Champaign, Illinois, United States. Correspondence to: Kan Ren <renkan @ shanghaitech.edu.cn>.
In this paper, we propose to generate time series using text descriptions as control conditions. Text, as prevalent form of unstructured information, inherently conveys rich seman- tic content that allows for more nuanced and detailed ex-
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

VerbalTS: Generating Time Series from Texts
pressions compared to traditional structured conditions, as illustrated in Fig. 1. By leveraging text descriptions for time series generation, our approach enables fine-grained control and communicates richer semantic information, offering users enhanced flexibility and control over the generation.
the complex relationships between text and time series, we incorporate innovative multi-focal alignment and genera- tion framework. (iii) We construct comprehensive bench- mark with multi-facet time series datasets and textual in- formation, which evaluates both generation and semantic editing performance of different methods.
This novel task presents several challenges stemming from the target of mulli-modaliin modeling and the unique char- acteristics of time series, which cannot be effectively ad- dressed using image generation approaches (Song et al., 2021; Peebles& & Xie, 2023). First, unstructured data, such as text, often carries intricate semantic information, mak- ing it difficult to distinguish relevant content from irrele- vant noise (Yin et al., 2019). Additionally, time series data exhibit multivariate characteristics and complex temporal dependencies (Torres al., 2021; Wu et al., 2021), which fundamentally differ from the spatial struetures typically encountered in image generation tasks. Furthermore, the control conditions for time series generation encompass multi-semantic information and exert varying influences. For example, textual conditions like "raining the whole day" and "raining after sunny weather" introduce nuanced distinc- tions that must be accurately reflected in the generated time series. Finally, the scarcity of paired time series and text data in real-world scenarios hinders the learning of robust connections between the two modalities.
2. Related Work
2.1. Conditional Time Series Generation
Conditional time series generation has attracted significant attention in recent years (Ang et al., 2023; Yang et al., 2024; Zhang et al., 2024c). Early approaches primarily relied on generative adversarial networks (GANs) (Esteban et al., 2017; Wang et al., 2023; Li et al., 2022) where a generator and a discriminator were jointly trained to refine genera- tion quality through adversarial learning. Variational au- toencoders (VAEs) (Lee et al., 2023) were also explored, optimizing the evidence lower bound (ELBO) to approxi- mate the underlying data distributions. More recently, diffu- sion models (Coletta et al., 2024; Narasimhan et al., 2024; Tashiro et al., 2021) have emerged as a more stable and reliable alternative, consistently demonstrating superior per- formance in time series synthesis.
From the perspective of control conditions, time series gen- eration can be categorized into structured and unstructured conditions. Structured conditions follow a fixed format, such as discrete categories (Li et al., 2022: Lee et al., 2023), metadata (Narasimhan et al., 2024) or preset constraints (Co- letta et al., 2024). While easy to learn, these approaches are inherently constrained by the predefined condition space, restricting the diversity of generated samples. For instance, DiffTime (Coletta al., 2024) incorporates constraints as conditions, struggling with conditions that cannot be easily formulated as constraints, ultimately limiting its general- izability. In contrast, unstructured conditions offer greater flexibility. Several works (Chen et al.; Chung et al., 2023) attempt to utilize clinical text reports generate electro- cardiograph data. However, these methods rely on domain knowledge, which are largely domain-specifee and lack the generalization capability required for broader time series applications.
To tackle these challenges, we propose VERBALTS, a novel multi-focal alignment and generation framework based on vanilla diffusion models (Ho et al., 2020) for text-to-time- series generation. We first recognize that time series exhibit multi-semantic information, reflected through representa- tions and generations at different resolutions. VERBALTS captures both the hierarchical semantics in text conditions and their multi-resolution influences on the generated time series. Specifically, low-resolution representations capture global trends, while high- high-resolution representations focus on finer details, such as local shapelets. Second, consider- ing that diffusion models process information and generate content at varying granularities (Zhang et al., 2024a; Fan et al., 2024), VERBALTS segments the generation process into multiple stages, applying condition information dynam- ically. Additionally, our framework extends to multivariate time series generation with nuanced semantic information alignment. Beyond traditional diffusion models (Song et al., 2021), our proposed method VERBALTS integrates multi- view time series noise estimator with multi-focal text processor, capturing the nuanced interplay between textual conditions and time series dynamics.
2.2. Time Series with Multi-modal Data
Prior studies (Liu et al.; Wang et al., 2024c) suggest that numerical time series alone often lack sufficient informa- tion, while external multi-modal data can provide valuable complementary insights. This has sparked growing interest in jointly modeling time series alongside other modalities. Some works (Han et al., 2024; Duan et al., 2023; Akbari et al., 2021) focus aligning different modalities such as text, images, and time series to learn unified represen-
Our main contributions are summarized as follows. (i) We introduce the novel task of time series generation from un- structured data, to the best of our knowledge, marking the first work to address this challenge and enabling more fine- grained controlled generation. (ii) To effectively model
2

---

VerbalTS:
Generating Time Series from Texts
tations. Others (Xu al., 2024; Srinivas et al., 2024; Jia et al., 2024; Xue & Salim, 2023; Liu et al., 2024) utilize the coarse-grained textual information to enhance time se- ries forecasting. Additionally. some exploration like (Li et al., 2023) aims to describe time series in natural language, thereby improving their interpretability. Together, these ef- forts collectively highlight the feasibility and necessity of integrating multi-modal data to improve time series model- ing. Building on this trend, explore multi-modal time series generation, aiming to strengthen connections between time series and other modalities.
During training, noise is gradually added to the original data distribution ((x))² via Gaussian Markov transition:
T
(xx1: [x0)
II t=1
((x, (x--1);
((x, = -N(xx-1 -1- ,,, (,1),
and produces the noisy sample x; at each diffusion step E [1,T]. Here { are the predetermined variance schedule. x; can be expressed as X; = va + vI a,E, where ~ ((,, I) and a2 == II -1.)...
2.3. Multi-scale Time Series Modeling
Then, a learnable noise estimation network <gl [x,, 4, (c)c) with the condition processor Je are trained by estimating the noise added to X,, which is used in the reverse process of diffusion model described later. The objective function is to minimize the noise estimation loss as
The complex temporal dependencies inherent in time series present significant modeling challenges, motivating the de- velopment of multi-resolution and multi-scale approaches. These methods are particularly prominent in forecasting tasks. For example, TimeMixer (Wang et al., 2024b) and TimeMixer++ (Wang et al., 2024a) decompose time series into trend and seasonality components, employing top-down and bottom-up mixing strategies at different resolutions, re- spectively. Pathformer (Chen et al., 2024a) introduces multi-scale routing mechanism for dynamically selecting appropriate transformers. MG-TSD (Fan et al., 2024) mod- els the denoising process in diffusion as a progressive re- finement from low to high resolution, using downsampled series as supervision signals at various diffusion steps. In time series editing, TEdit (Jing et al., 2024a) operates variable patch sizes to enable parallel multi-resolution edit- ing. For representation learning, TS2Vec (Yue et al., 2022) performs hierarchical contrastive learning to obtain robust contextual representations. However, these methods primar- ily focus on intra-modal multi-scale structures, overlooking the hierarchical information present in other modalities and the cross-modal interactions that are crucial for effective multi-modal modeling.
min [(xo) min Wio.,,,.,,,. ||c - cs (x L, g. [c) 0,6
0.
(2)
Io (c) denotes the encoder of the condition and will be further discussed in Sec. 4.2, Xu ~ g(x)) is sampled from the real data distribution, X2 is a noisy version of Xo-
During inference, given a condition c processed by go(c), sample Xa can be generated from random noise Xy (0, I) through multiple steps of denoising which uses the trained noise estimator Eg (X,, t, (a(c) with a sampler, such as the deterministic Denoising Diffusion Implicit Model (DDIM) (Song et al., 2021)
vai-1
81-1
(&t - VI ORED (Ri, t, gc()))
voz
(3)
/1- 11-aa-- 011-00-100$t go(c)))
Consequently, it is critical for the noise estimator in time series generation to effectively capture the ppatio-temporal properties while accounting for the nuanced influences of textual information on the generation process. signifi- cant challenge lies in bridging the gap between unstructured textual descriptions and the time series modality. Aligning the generation of time series with the multi-semantic infor- mation embedded in text is non-trivial task, as illustrated in Sec. 1. In the following section, we will introduce our proposed method, VERBALTS.
3. Text To Time Series Generation
3.1. Problem Formulation
Consider a sample pair of time series x and its corre- sponding textual description c. Given a text description E Na with M tokens, we aim learn generative model f, to generate time series that conforms to the text * f(c) E K where K and L are the number of variables and the length, respectively.
4. VERBALTS: Multi-focal Alignment and Generation
3.2. Conditional Diffusion Model Framework
In this section, we introduce our method, VERBALTS, which consists of two key components: a multi-view noise esti- mator eg and multi-focal text processor Jø- The noise estimator eg is designed to model and generate time series
Our model is built upon the conditional diffusion model (Sa- haria et al., 2022), We briefly review the training and infer- ence procedures below.
xx and Xo are interchangeable in this paper.
3

---

VerbalTS: Generating Time Series from Texts
Compound
Reprogram
Mult-resolution Mixer
Estimated noise
Text embedding
Spatial
A (opont
Patch Decoder
Time series
Anchor vectors
Residual Layer
representation
Text representation
Temporal
Z (rrmp)
SSA
Eq. (8)
TSA
Eq. (6)
(ttmpp)
Eq. (5)
A Desup)
MLP
Eq.18)
Eq. (6)
E
P.Y
2
Diffusion
A (dan
Multirrsslluion Encoder
Estimator cg
Multi-focal Text Processor Sp
Diffusion step
Stage
Stage
Stage S
Figure 2. The overall model architecture of VERBALTS, including multi-view noise estimator and multi-focas text processor.
with finer granularity, effectively capturing detailed tem- poral dynamics, incorporating spatial interactions among variables, and ensuring nuanced condition control through- out the diffusion process (Sec. 4.1). Meanwhile, the text processor Ys transforms textual information into multi- semantic structured latent space, enabling more precise representation of the complex relationships between text and time series modalities (Sec. 4.2). Finally, the multi- sealtiicmmnnic information would be aligned to the time series generation process, with an elaborately designed conditional control mechanism (Sec. 4.3). The overall model architecture is presented in Fig. 2.
the following content, we first provide an overview of the proposed multi-view noise estimator and then elaborate on each individual view.
Overview. As demonstrated in Fig. 2, given noisy time se- ries x,, the noise estimator 60 (xxx4x,) 4) first encode it into em- bedding space via the patch encoder, which will be passed through J residual layers and finally decoded by the patch decoder and multi-resolution mixer to obtain the estimated noise Ct, that will be removed from x,- Each residual layer contains blocks: temporal self-attention TSA(-), spatial self-attention SSA(.) (Vaswani, 2017), with ford-forward network FFN Please refer to Appendix B for more de- tails about the model architecture.
4.1. Multi-view Noise Estimator
Temporal View. The input noisy time series x, E R RRKL is modeled with a total R resolutions in the temporal dimension. First, at each resolution r, X, is sliced into N, = L'= patches of size L,, get Xt,r E X Xer is then encoded into embeddings Pt. E R via a linear layer, where D is the size of the embedding. Then we concatenate the = ΣR N patch embeddings from R resolutions into single tensor P. K which will be fed into J residual layers. Within the j-th residual layer, we design a TSA(-) block to capture multi- resolution temporal information of P(i-1) by performing the self-attention (Vaswani, 2017) on the temporal dimen- sion. Note that operations within the same resolution. More details can be referred to Appendix B.
Textual conditions provide the flexible descriptions of the target time series from various perspectives, as mentioned in Sec. 1. Generating time series from these flexible and complex textual conditions requires fine-grained control over the generation process. In this paper, we derive the fine- grained time series generation from the three key views of the generation process. First, from the temporal view, time series data exhibit strong sequential characteristies. Jing al. (2024a) has demonstrated that the influence of control conditions on time series generation varies across different time spans, making finer-grane emer-granularity temporal modeling essential. Second, from the spatial view, real-world time series data usually contains multiple variables, yet most existing studies focus on univariate time series generation (Jing et al., 2024a), overlooking variable-level interactions. Furthermore, from the diffusion view, recent studies (Zhang et al., 2024a; Fan et al., 2024) have discovered that different steps in the denoising process in the diffusion model derive the generation details at varying levels of granularity. In
mask matrix to restrict attention
Spatial View. Within the j-th layer, we design a SSA(:) block, similar to TSA(), to capture the spatial information by applying the self-attention over the K variables in the spatial dimension of 10-1)

---

VerbalTS: Generating Time Series from Texts
The complete process of temporal and spatial modeling in the j-th layer of the noise estimator can be expressed as
To address these challenges, we propose a multi-focal text processor, denoted as (o(c), for better information align- ment. This text processor transforms complete text sen- tence into multi-semantic components from the views of temporal, spatial dimensions and diffusion process, through semantic reprogramming (Chen, 2024), as introduced as below. We term it "multi-focal" because textual semanties are distributed across different tokens, and our approach selectively focuses on the most relevant ones.
p(. - -P((-1) + FFN(SSA(TSA(P! ee)),, (4)
P.. e, E RDD is the embedding of the current where P denoising time step. with spatial and temporal self-attention mechanism mentioned above. More details are presented in Appendix B.1.
Diffusion View. Now we move attention onto the overall dif- fusion process Eg (x, f, -), E [1, TI. We evenly divide the total T denoising steps into S stages. For the denoising step 1, it is assigned to the s-th stage where & = xx At each stage, the diffusion model generates time series at different granularities, guided by corresponding levels of textual in- formation, as detailed in Sec. 4.3. For clarity, we represent the multi-view time series representations across the entire diffusion process as a unified variable P E R''xXXNxx D concatenating the temporal and spatial representation P. for each diffusion step t E T..
Multi-focal Reprogramming. Given text condition с E N' of M tokens, we first encode e into embeddings E E T MRD via a pre-trained text encoder, e.g.. Long-clip (Zhang et al., 2024b). Then, we reprogram E into I seman- tic levels Z E TR based on the learnable anchor vectors A R where the i-th row corresponds the i-th se- mantic level. Specifically, the reprogram (RPG) operation is defined by:
Z = T-RGGQQA, KE, VE) Soofmax ( QiK E
VVE. (5)
Till now, the proposed multi-view time series modeling and generation framework in the noise estimator has been clari- fied. It is worth noticing that, the control of textual informa- tion on the three views, i.e., different temporal resolutions, spatial variables and multiple diffusion stages, would be further described in Sec. 4.3. Unlike existing time series dif- fusion models (Tashiro et al., 2021; Narasimhan et al., 2024; Coletta al., 2024), our approach models the generation of time series from three views: spatiotemporal relationships and the noise reduction process. This design, assembly with the multi-focal alignment module described in Sec. 4.3, en- ables more nuanced generation and leads to higher-quality outputs, as is shown in the Sec. 5.2.
where QA AWA, Q. Kg EW K. VE EW v. are query, key, and value matrices, respectively, and W (.) are learnable weights.
From the three views of multi-focal modeling, we specifi-
cally define: (i) Diffusion semantic anchors A (dit)) E RSSD where S is the stage number of denoising process; (ii) Temporal semantic anchors A (temp) E RR where R is the resolution number of time series modeling: (iii) Spa- tial semantic anchors A (spat) RRKDD where K is the variable number. The corresponding text representations can be calculated by the reprogramming mechanism, re- sulting in (ditt) E RSxIxixb, ((xmp) E R RIXIIRxD ( E RIXKXIXD, respectively. By aligning multi- semantic conditions with the corresponding time series components (Sec. 4.3) and updating them through gradi- ent backpropagation, distinct anchor vectors learn unique representations, as demonstrated in Sec. 5,4.3.
4.2. Multi-focal Text Processor
Textual conditions encode complex multi-semantic informa- tion within an unstructured token sequence. The effective- ness of the noise estimator eg is strongly influenced by how well the textual conditions are modeled. However, modeling this information presents several challenges. First, aligning time series components with relevant textual semantics is non-trivial. From the temporal view, phrases like "at the beginning" and "around the middle" describe local features, while "overall" and "global" refer to broader trends. From the spatial view, terms like "light rain" primarily affect the variable of precipitation while having minimal influence on the variable of wind direction. From the diffusion view, tokens conveying global information are likely play larger role in early diffusion stages, whereas tokens with fine-grained details become more relevant in later stages. Second, textual data includes grammatical fillers (e.g., "the", "is", and "of") that carry little semantic weight and should not be overemphasized during modeling.
Further, we unify the semantic representations of the temporal, spatial and diffusion process views to a compound matrix Z E Sx XxX as below.
Z = Z (dilf)
Z (spa))
@ Z (emp) where
L
(6)
Z
Z (diff)
Z (spat)
+ (temp)
The @ the operation of matrix addition with broadcast- ing. The representation not only encodes various multi- semantic textual information but also enables multi-modal control for time series generation, which will be further discussed in Sec. 4.3.
Unlike the text processor in visual generation (Rombach et al., 2022), which uses a unified text condition throughout, our multi-focal text processor leverages hierarchical textual
5

---

VerbalTS: Generating Time Series from Texts
information. This design facilitates multi-semantic pattern discovery and enables more refined generation, demon- strated in Secs. 5.3 and 5.4. Furthermore, we introduce the learnable anchor vectors to eliminate the need for man- ual extraction of structured information from unstructured text, enhancing both flexibility and efficiency and exhibiting more superior performance than the strong baselines based on structured conditions, as shown in Sec. 5.2.
5. Experiment
In this section, we present the experimental settings and the corresponding results. Our analysis follows the below research questions (RQs). RQ1: Does the unstructured condition conveys more information than the structured one for time series generation? RQ2: How does the multi-focal generation mechanism work? RQ3: How does the proposed method VERBALTS build the alignment between text and time series? We have released all the reproducible code and benchmarking datasets at https//seeqll github. wartiubbboo//eeee /o//erbalTSS/
4.3. Multi-modality Semantic Alignment
Using the multi-view noise estimator and multi-focal text processor described earlier, we obtain multi-view represen- tations for the time series, P RT*XXNxD and for the text, Ze E RX The key challenge now lies in effec- tively aligning and applying the semantic information from the text descriptions into the diffusion denoising process.
5.1. Experiment Setup
Datasets. To solve the problem of scarcity of paired text and time series data, we construct datasets using data from three different sources: (i) Full synthetic datasets with man- ually constructed texts and the corresponding ground-truth time series, including Synth-U with univariate time series and Synth-M with multivariate time series. (ii) Realdworld datasets including Weather (Xu et al., 2024) of climtime in- dicators and Blind Ways (Kim et al., 2024) of blind people's trajectories. both originating from real-world textual annota- tions and time series. (iii) Augmented real-world datasets including ETTml (Zhou et al., 2021) and Traffic (Leo, 2024) datasets with the real-world time series and the corre- sponding textual descriptions annotated by the external tool. For detail information of the dataset construction, please refer to Appendix A.
To address this, we propose an adapter that aligns the multi- semantic information of the text across its three views with the corresponding components of the time series, ensuring seamless integration during the diffusion process. Inspired by the adaptive layer normalization (Xu et al., 2019), the pro- posed adapter rectifies the time series embedding P based on the textual condition Z via a gate a, a scaler "y and a shifter B, which are calculated through a Multi-Layer Per- ceptron (MIP)):
a. 7,, = MLP(Z), where a, 3,7 SXKXRRD
(7)
Evaluation metrics. We evaluate the quality of the gener- ated time series from two perspectives. (i) Fidelity: Frechet Inception Distance (FID) (Heusel al., 2017) and Joint Frechet Time Series Distance (J-FTSD) (Narasimhan et al., 2024) are used to evaluate the fidelity by measuring the discrepancy between the generated data distribution and the real data distribution. (ii) Semantic Alignment: Contrastive Time series Text Pretraing (CTTP) score is used to measure the semantic similarity between the generated time series x and the text condition Similar to CLIP score (Rad- ford et al., 2021), proxy model trained through contrastive learning calculates the similarity between the generated time series and the condition text in the latent space. We further provide the details of the model architecture of CTTP model and prove its reliability in Appendix C.
Each dimension of the controlling parameters manages the influence time series modeling along with R resolutions and K variables within the S stages of the diffusion process correspondingly, as explained in Sec. 4.1. The representa- tions within the diffusion model are controlled via
P = a @ SSS((SS(77 @
(P P++el ())) ())),
@ and @ are custom element-wiss multiplication and ad- dition between matrices. The adapter parameters o., 3,8 y undergo broadcasted expansion through temporal and dif- fusion segment replication, achieving dimensional compat- ibility with tensor P. with detailed calculations provided in Appendix B.3. The conditionally altered representation P E XTX will be passed into FFN in each residual layer for further calculation.
Baselines. Since there are few works studying the general unstructured text to time series generation problem, we compare our proposed method VERBALTS with attribute- based generative models Time Weaver (Narasimhan et al., 2024) and TEdit (Jing et al., 2024a), constrained generation model DiffTime (Coletta et al., 2024) and class- cansscoondiioneed generation model TimeVQVAR (Lee et al., 2023). The conditions for these methods are transformed from textual descriptions. More implementation details are provided in Appendix A.4.
With the proposed adapter, we align multi-semantic informa- tion across temporal, spatial, and diffusion views between the two modalities. Unlike Liu et al. (2022), which adjusts weights for decomposed conditions, our approach considers the correspondence between control conditions and gen- erated data, a crucial factor for fine- -reagga conditional generation. The effectiveness is detailed in Sec. 5.
6

---

VerbalTS: Generating Time Series from Texts
Multivariate setting
Syothetic dataser
Roal-world darasets
Coodition
Method
JFID
Symth-M JIFTSD
1 CTTP
JFID
BlinWWys LIFTSD
1 CTTP
Weather LIPTS
1 CTTP
Class
TimeVQVAE (Lee et. al., 2023)
88.17 10.13
94.1610.10
50.23
29.48 30.00
36.87.g so
2..0
84.28 30
22.19.0.:0
Constrint
DiffTime (Coletta et el., 2024)
48.8774..2
96.86
21.60 40
68.32
06.21
7.1440.75
60.00
68.8
14.48
Time Weaver (Narasimhan el al., 2024)|
43.38
59.59 g..00
52. 25..co
51.31 196.97
89.22
2.50
16.07
19.79
27. 27.27a
Amribute
TEdi (fing et al., 2024g)
41 11.9922222
58.08
52.55+0.20
28. 28.96
36. 36.78
4.06
14.86+0.31
18.33
27 27.507
Text
VERRAITS (ouns)
32.91 29.00 ca
60 64.13go. o6
27.6331 19
24.40
2.18
6.18
2.50
30.6229.05
Univariate setting
Synihetic dataset
Real-world datasets
Coudition
Method
JFLD
Synth-U JIFTSD
CTTP
LFID
LIFTDD
ET1ml
1 CTTP
IPID
Trfttic LFFTD
1 CTTP
Class
TimeVQVAF (1..c 2023)
41 1..2a
54.47.1.01
26.94.0.07
26.8410.91
30.19go #1
10.56go.1
40.477...5
42.68
8.52
Constraint
DidTime (Coletia et al., 2024)
45 43.77
12.94
12.79
59.14
g6.37
2.33
100.600
104.27
1.07
Attribute
TeneWeaver (Narasimhus 2024)
Tidii (Jin et al., 20244)
36 36.0222955
36 36.38 20
49.9220
49
1..06
27.95.20.c
28,09 20 0o
26,2620 s: 32.90g0.a0
35.0410 45
36.35go 46.
10.34
10.34g0 0s
44.34g1
45.82g1
eu
20.06
48. 19
8,2010
Test
VERBAITSS (ouns)
28.29 22.26.0.23
37.56.0.or
24.19
20.10
11.10
37.03-1.03
39.03-1.44 1.54
8.4340.03
Table 1. Averaged performance with standard deviation (tean-tstd) of multivariate and univariate settings on synthetic (left) and real- world (right) datasets with three random runs. The best performance is with bold font, t (L) means the higher (lower), the better.
Ditrest
Matric
VIRBAITS
wio A"
who A'&
1.0
Ground Thuth
1.9
Ground Thuth Timellioneer
VerbwiTS
LFID JFFSD
29. 29.05.0.45
29:95 29.98.0..s
34.17.0.55
24.18:0.30
Synth M
32. 01....2
33.91 03.9120.05
40 40.95.g0.51
41 11.4010.01
CCTPP
84.18
63 63.86.0.04
60.17.0.a9
60.07
0.6
0.5
Weother
LPDD LIFTS
8.50 31
6.13
2.01.
7:3020.25
7.8022.43
10-122n.s
0.4
0.4
1CTTP
30.02:0.03
30.32.0.05
30.231:0.0s
20.90
0.2
Table 2. Ablation studics on Synth-M -M and Weather datascts. A A' and A are the abbreviations of A (sp4), A (temp) and A (dit) which refer to the anchor vectors of spatial, temporal and diffusion in the multi-focal text processor, respectively.
0.0
0.0
0.4
0.6
0.2
0.4
1.0
1.0
Ground Thuth VerbbiTT
1.0
Ground Thuth
Timelleaver
5.2. Quantitative Results
Weatrer
0.6
0.6
0.4
0.4
In this section, we quantitatively evaluate and report the performance of the compared methods for time series gener- ation from given textual conditions (RQ1) across all datasets. The average results for multivariate and univariate time se- ries generation are presented in Tab. 1.
0.0
0.0
0.0
0.2
0.8
1.0
Figure 3. the VERBALTS (left) and TimeWeaver (right) on Synth-M and Weather datasets.
Comparison of generated data distribution between
As shown in Tab. 1, VERBALTS demonstrates superior fi- delity on both synthetic and real-world datasets, achieving over 20% improvements on the Synth-M/U and Weather datasets. Moreover, VERBALTS achieves significantly bet- ter semantic alignment between the generated time series and the descriptive texts, as evidenced by its much higher CTTP performance. These results highlight not only the richer information embedded in unstructured text conditions but also the exceptional ability of our proposed method to generate high-quality time series that are semantically well-aligned with the given conditions. More qualitative evaluation results are provided in Appendix F.2.
tial, temporal, and diffusion views by sequentially removing the anchor vectors A (spat), A (temp), and A (dift), Correspond- ingly, we simplify the calculation of the text representation Ż E IIxIxII by averaging the text sequence embedding E along the text length dimension. As shown in Tab. 2, each ablation operation results in a significant decline in performance, highlighting the positive effects of multi-focal modeling in VERBALTS for generating time series data that are both high-fidelity and semantically well-aligned with the textual conditions.
5.3. Ablation Study
5.4. Extended Analysis
We conduct an ablation study of VERBALTS on the Synth-M and Weather datasets to evaluate the impact of the proposed multi-focal text processor and milti-view noise estimator, as described in Sec. 4.2 and 4.1 (RQ2).
5.4.1. UNSTRUCTURED V.S. STRUCTURED CONDITION
We conduct analytical experiments, trying to uncover the effects of the unstructured text conditions versus the struc- tured condition on time series generation (RQ1). We also analyze the behavior patterns of different compared models on two condition types.
In this study, we successively ablate the multi-focal model- ing and generation capabilities of VERBALTS across the spa-
7

---

VerbalTS: Generating Time Series from Texts
Synth-M
Weather
0.1
Stage
60
30
6.0
Stage
25
8.0
Stage
40
0.0
20
TimeWeaver
TimeWeaver
0.1
Resalition
VerbalTS MFTP
VerbalTSS weo MFTP
0.0
20
VerbalTS
VerbalTS
61
Resoluttonn
61
Resolationn
35
6.0
Epochs
Eporhs
Figure 4. The learning curves of CTTP score on the validation set of Synth-M and Weather datasets. Findings: (D: Structured con- ditions are easier for model to learn. @: Unstructured conditions provide additional information. (:: The multi-focal text processor (MFTP) in VERBALTS makes learning more effective.
Spmni
0.1
Vortable
01
Vortabll1
0.0
Figure 6. Distribution of the averaged attention weight in Eq. (5) of each token within the vocabulary. VERBAI.S exhibits various attention distrihutions on different textual information from differ- ent views, showcasing the effect of multi-focal processing.
Time series generated from unstructured text conditions are more diverse and better aligned with the ground-truth distribution than those from structured condi- tions. As shown in Fig. 3, t-SNE (Van der Maaten & Hinton, 2008) visualization reveals that text-based conditions pro- duce more diverse samples, while structured conditions lead to stronger clustering and less variety. This highlights the limitations of structured conditions and the value of flexible controls like text for generating realistic time series. For more visualization results, please refer to Appendix F.I.
Finding 1:
5.4.2. TEXT EFFECT IN TIME SERIES GENERATION
We further explore how text information impacts time series generation in VERBALTS, addressing RQ3.
Finding 3: VERBALTS focuses on semantically relevant information in raw text, with the multi-focal text processor enhancing this ability. To analyze the impact of relevant and irrelevant information, we manually labeled the token vocabulary into a relevant token set Voeel (e.g.. descriptions of trend, shape, and seasonality) and an irrelevant token set yúir) (e.g., meaningless pronouns and stop words). More de- tails are in Appendix 1.1.5. We then conducted an interven- tion experiment by randomly masking relevant or irrelevant tokens and evaluating the effect on time series generation performance. Fig. 5 shows the quantitative analysis of the CTTP performance ratio CTIP varying ratios. The results demonstrate that masking seman- tically relevant tokens significantly degrades the generation performance (left of Fig. 5), whereas masking irrelevant tokens has minimal impact (right of Fig. 5). This resilience is attributed to the multi-focal text processor and alignment mechanisms in VERBALTS. Moreover, VERBALTS per- forms much better than baseline Time Weaver even masking 90% irrelevant tokens in text, highlighting the robustness of our method. We further show a case study in Appendix F.3, to qualitatively illustrate how our method works.
Finding 2: Unstructured text conditions convey richer in- formation but introduce additional noise, while this can be effectively handled by the multi-focal mechanism of VER- BALTS. We present the learning curves on the Synth-M and Weather datasets in Fig. 4. Initially, Time ex- hibits faster learning (gap 1), as the structured conditions are noise-free, allowing the model more easily extract useful information. However, it is later outperformed by VERBALTS (gap 2), owing to the richer, fine-grained de- tails provided by the unstructured text. Furthermore, the multi-focal mechanism in VERBALTS enables more effec- tive utilization of this additional information, leading to faster convergence and ultimately superior performance in text-to-time series generation (gap 3))
CTTF after masking tokens at
Modking velovon tollen
Masking kvebsove tooee
WilUTS w/l WFTP
Tmilthoe howlee
Teaiiluse basiins
5.4.3. EFFECT OF MULTI-FOCAL TEXT PROCESSING
02-
0.3
055
07
Finding 4: The multi-focal text processor enables VER- BALTS differentiate and utilize diverse textual informa- tion for fine-grained time series generation. To evaluate how VERBALTS processes nuanced textual details (RQ2), we analyze the averaged attention values of each token in the vocabulary from Eq. (5). (See Appendix E.2 for more details.) As shown in Fig. 6, VERBALTS assigns varying
Figure 5. Intervention experiment on Synth-M dataset. We com- pare VERBALTS with and without the multi-focal text processor (MFTP). TimeWeaver baseline I and 2 serve as references without masking, representing the CTTP ratio of TemeWeaver to VER- BALTS and VERBALTS w/o MFTP, respectively.
8

---

VerbalTS: Generating Time Series from Texts
attention to tokens across different focuses, demonstrating the fine-grained modeling capabilities of the multi-focal text processor, outlined in Sec. 4.2.
Sareaz waae god a The of
juat Loud seat oog.
CTT 60.52
Carpea hag Taqurry demporans oo woat oepiniig thoeiics
(rawert
5.4.4. SENSITIVITY STUDY
Macopaapen CTT 56554
Rrver The space spiiic ieserrppo recuens sinal 01 rncurrde The trand
Finding 5: VERBALTS demonstrates robustness to hyper- parameter settings. Beyond a certain threshold, the benefits of multi-resolution and multi-stage modeling become evi- dent. To evaluate the sensitivity of VERBALTS to specific hyperparameters, we conduct sensitivity study on Synth-M and Weather datasets to evaluate the impact of the multi- resolution number R and multi-stage number S. As shown in Fig. 7, VERBALTS maintains stable performance across a wide range of hyperparameter values. Beyond a certain threshold, the benefits of incorporating multiple resolutions and stages become significant.
Figure 8. Illustration of editing task. Column I: the raw time series before editing. Column 2: result edited by TEdit (Jing et al., 2024a); Column 3: result edited by our VERBALTS; Column 4: the condition prompts for editing, with (source -> target) properties.
underlying mechanisms. In this work, addressed the problem of generating time series from text by proposing VERBALTS, which integrates a novel multi-focal text pro- cessor with a multi-view diffusion model to achieve fine- grained semantic alignment between the two modalities. Experiments on two synthetic and four real-world datasets demonstrated the effectiveness of approach through both quantitative and qualitative analyses. VERBALTS also showed strong capabilities in extracting meaningful text information, enabling high-quality generation and flexible applications such as controlled generation and editing.
60
50
50
SynthM
30
20
20
10
However, our method has limitations, such as slower gen- eration efficiency due to the long-range reverse process in diffusion models, which may hinder real-worls applications. Addressing this limitation presents an important direction for future research.
30
25
IFTSD
CTIP
CTTP
Wasthor
20
20
15
10
10
Impact Statement
This work introduces a novel method for generating time series from texts, bridging the gap between natural language understanding and time series synthesis. We foresee several potentially positive societal impacts: (i) providing richer time series data across various fields, (ii) reducing the risk of personal privacy leakage in data, and (iii) advancing the study of refined and controlled time series generation.
Figure 7. Sensitivity study on Synth-M and Weather datasct. is the resolution number and S is the stage number of diffusion.
5.4.5. WIDER APPLICATION: TIME SERIES EDITING
Finding 6: VERBALTS enables flexible and precise time se- ries editing. Adopting the approach from (Jing et al., 2024a), we integrate VERBALTS into the editing process. Specif- ically, noise is added to the source time series x via the forward process (Eq. (1)), and the reverse process (Eq. (3)) generates the edited time series ~ under the target text condition via our proposed VERBALTS. Fig. 8 showcases the ability of VERBALTS to retain original text characteris- ties while aligning with modified descriptions. Compared to TEdit (Jing et al., 2024a), which relies on attributes, VERBALTS achieves finer edits by precisely locating and following the adjusted tokens in the target text description. More case study results are provided in Appendix F.4,
While we do not anticipate immediate or direct negative societal consequences arising from this contribution, we acknowledge that, like other generative technologies, it may be vulnerable to misuse. As such, responsible use, ethical oversight, and continued monitoring are essential to ensure that the technology is applied for beneficial purposes.
A Acknowledgment
The research was supported by National Natural Science Foundation of China (Grant No. 62406193). The authors also gratefully acknowledge further assistance provided by Shanghai Frontiers Science Center of Human-centered Arti- ficial Intelligence, MoE Key Lab of Intelligent Perception and Human-Machine Collaboration, and HPC Platform of ShanghaiTech University.
6. Conclusion
Text offers rich and nuanced information, making it a power- ful modality for describing time series details or uncovering

---

VerbalTS: Generating Time Series from Texts
References
Esteban, C., Hyland, S. L., and Rätsch, G. Real-valued (medical) time series generation with recurrent condi- tional gans. arXiv preprint arXiv: 1000:: 776632663, 2017.
Akbari, H., Yuan, L.,, Qian, R., Chuang, W.-H., Chang, S.-F., Cui, Y., and Gong, B. Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text. Advances in Neural Information Processing Systems, 34: 24206-24221, 2021.
Fan, X., Wu, Y., Xu, C., Huang, Y., Liu, W. and Bian, J. MG-TSD: -uutiigraaullrrtyy time series diffusion models with guided learning process. In The Twelfth International Conference on Learning Representations, 2024. URL /tthhtts:://ppeneeeee /oppnrevieneeeeeeeeeeeeeeee net forum? id=czi
Ang. Y., Huang, Q. Bao, Y., Tung. K., and Huang, Z. Ts- gbench: Time series generation benchmark. Proceedings of the VLDB Endowment, 17(3):305-318 2023.
Gao, Y., Chen, H., Wang. X., Wang, Z. Wang. X., Gao, J., and Ding, B. Diffsformer: A diffusion trans- former on stock factor augmentation. arXiv preprint S0li::20020066555 2024.
Chen, J., Lai, Y., Zhang. D., Wang, Y., Geng, S., Li, H., and Hong, S. Diffusets: 12-lead ecg generation conditioned on clinical text reports and -ptient-specific information. In Artificial Intelligence and Data Science for Healthcare: Bridging care.
Al and People- CentrcC Health
Han, J., Gong, K., Zhang, Y., Wang, J., Zhang, K., Lin, D., Qiao, Y., Gao, P., and Yue, X. Onelim: One framework to align all modalities with language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26584-26595, 2024.
Chen, P.. ZHANG, Y., Cheng, Y., Shu, Y., Wang, Y., Wen, Q. Yang, B., and Guo, C. Pathformer: Multi- scale transformers with adaptive pathways for time se- ries forecasting. In The Twelfth International Confer- ence on Learning Representations, 2024a. URL https: //openrevvew.neeeeeeeeeeeee wwwp://wnwwooooooodoooeoo.aaaaad wat.:/ummmiid1ooooePaaa22
He, H., Zhao, S., Xi, Y., and Ho, J. C. Meddiff: Generat- ing electronic health records using accelerated denoising diffusion model. arXiv preprint 2rXiv22302.03333 2023.
Chen, P.-Y. Model reprogramming: Resource-efficient cross-domain machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 22584-22591, 2024.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6855 2020.
Chen, Y. Ren, K., Wang, Y., Fang, Y., Sun, W., and Li, D. Contiformer: Continuous-ting transformer for irregular time series modeling. Advances in Neural Information Processing Systems, 36, 2024b.
Jarrett, D., Yoon, J., Bica, I., Qian, Z., Ercole, A., and van der Schaar, M. Clairvoyance: A pipeline toolkit for medical time series. In International Conference Learning Representations, 2021. URL https:// openreview net ftr.mrorrmm// iinrrumiid-see wew.ddxxnCyYW.XXE.
Chung, H., Kim, J., Kwon, J.-m., Jeon, K.-H., Lee, M. S., and Choi, E. Text-to-ecg: 1 12-lead electrocardiogram synthesis conditioned on clinical text reports. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1-5. IEEE, 2023.
Jia, F., Wang, K., Zheng, Y., Cao, D., and Liu, Gpt4mts: Prompt-basee based large language model for multimodal time- series forecasting. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 23343-23351, 2024.
Coletta, A., Gopalakrishnan, S., Borrajo. D., and Vyetrenko, S. On the constrained time-series generation problem. Advances in Neural Information Processing Systems, 36, 2024.
Jing, B., Tong, H., and Zhu, Y. Network of tensor time series. In Proceedings of the Web Conference 2021, pp. 2425-2437, 2021.
Desai, A., Freeman, C., Wang, Z., and Beaver, Timevae: A variational auto-encoder for multivariate time series generation. arXiv preprint 21Xiv:211.008 2021.
Jing, B., Zhang, S., Zhu, Y., Peng, B., Guan, K., Margenot, A., and Tong, H. Retrieval based time series forecasting. arXiv preprint arXiv:2209, 13525, 2022.
Duan, Y., Zhou, J., Wang, Z., Wang, Y.-K., and Lin, C.-T. Dewave: discrete eeg waves encoding for brain dynam- ics to text translation. In Proceedings of the 37th Inter- national Conference on Neural Information Processing Systems, pp. 9907-9918, 2023.
Jing, B., Gu, S., Chen, T., Yang, Z., Li, D., He, J., and Ren, K. Towards editing time series. Advances in Neural Information Processing Systems, 2024a.
10

---

VerbalTS: Generating Time Series from Texts
Jing, B., Wang, Y., Sui, G., Hong, J., He, J., Yang, Y., Li, D., and Ren, K. Automated contrastive learning strategy search for time series. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 4612 2-20,,00, 2024b.
Narasimhan, S. S., Agarwal, S., Akcin, O., Sanghavi, S., and Chinchali, S. P. Time weaver: A conditional time series generation model. In ICML, 2024. URL https: / //openrevieww. net formm? d--ppDDe xmFr.
Nie, Y., H. Nguyen, N., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
Jing, B., Zhou, D., Ren, K., and Yang, C. Causality-awand spatiotemporal graph neural networks for spatiotemporal time series imputation. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 1027-1037, 2024c.
Nils Braun, Maximilian Christ, A. K.-L. tsfresh, 2024. URL https://g thub. com/bl hte/bbuuu-yyodrrr tsfresh.
Kim, H. J., Sengupta, K., Kuribayashi, M., Kacorri, H., and Ohn-Bac Bar, E. Text to blind motion. In The eiiht Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https: /oopeneeview net/ forum? mete/eorrmdddiIJDDLLLL di=iiiddiQQGGT
OpenAl. ChatGPT-3.5; Language model for conversational AI, 2023. URL https: //op/naiippppea com/ chat Available at https://opmmii.co//hatttt
Lai, G., Chang, W.-C., Yang, Y., and Liu, H. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR confer- on research & development in information retrieval, pp. 95-104, 2018.
Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 4195-4205, 2023.
Pei, H.,, Ren, K., Yang, Y., Liu, C., Qin, T., and Li. D. Towards generating real-world time series data. In 202/ IEEE International Conference on Data Mining (ICDM), pp. 469-478. IEEE, 2021.
Lee, D., Malacarne, S., and Aune, E. Vector quan- tized time series generation with a bidirectional prior model. In International Conference on Artificial Intelligence and Statistics, 2023. URL https: //api. www.stiisccooooeee....e. org.ooopurrr////.D....... 257405229.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Leo. Istanbul traffic index. Kaggle, 2024. URL https: //www, //www.kaggle...mm con/ /ttam//aaasts//ttaam /attaattttsttttaaoo//// /leoaard000//// i istanbul -taauull-rrrrae traffllttrffffff-- ceweeeiindx//ee data.
Li, X., Ngu, A. H. H., and Metsis, V. Tts-egan: A trans- former series conditional gan for biosignal data aug- mentation. arXiv preprint arXiv:2206. / 13676, 2022.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference computer vision and pattern recognition, pp. 10684-10695, 2022.
Li, Y., Gao, Y., Cai, J., Zheng, G., Shi, H., and Liu, X. Repr2seq: A data-to-text generation model for time se- ries. In 2023 International Joint Conference Neural Networks (IJCNN, pp. 1-8. IEEE, 2023.
Saharia, C., Chan, W., Saxena, S., Li, L.,, Whang. J., Denton, E. L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al. Photorealistic text-to-image dif- fusion models with deep language understanding. Ad- vances in neural information processing systems, 35: 36479-36494, 2022.
Liu, H., Xu, S., 7hao, 7.,, Kong, L.,, Kamarthi, H., Sasanur, A. B., Sharma, M., Cui, J., Wen, Q., Zhang, C., et al. Time-mmd: Multi-doman multimodal dataset for time series analysis. In The Thihty-eight Conference on Neu- ral Information Processing Systems Datasets and Bench- marks Track.
Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. URL https: /wwp//vien....eeeeeeeeeee net /
Liu, N., Li, S., Du, Y., Torralba, A., and Tenenbaum, B. Compositional visual generation with composable dif- fusion models. In European Conference on Computer Vision, pp. 423-439. Springer, 2022.
Srinivas, S. S., Ravuru, C., Sannidhi, G., and Runkana, V. Reprogramming foundational large language mod- els (Ilms) for enterprise adoption for spatio-temporal forecasting applications: Unveiling a new era in copilot- guided cross- cross-modal time series representation learning.
Liu, Y., Qin, G., Huang, X., Wang, J., and Long, M. Auto- times: Autoregressive time series forecasters via large lan- guage models. arXiv preprint arXiv:2402.02377. 2024.

---

VerbalTS: Generating Time Series from Texts
In Proceedings of the AAAI-2024 Workshop on Deploy- able Al (DAI), 2024. URL htttp:://rxiv... abs/ 2408. 14387.
Xue, H. and Salim, F. D. Promptcast: A new prompt- based learning paradigm for time series forecasting. IEFE Transactions Knowledge and Data Engineering, pp. 1-14, 2023. doi: 10. 00.119/KKDEE.023.52222555
Tashiro, Y., Song. J., Song. Y., and Ermon, S. Csdi: Con- ditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804-24816, 2021.
Yang. Y., Jin, M., Wen, H., Zhang, C., Liang, Y., Ma, L., Wang, Y., Liu, C., Yang, B., Xu, Z.,, et al. A survey on diffusion models for time series and spatioltemporal data. arXiv preprint arXiv:2404. 22222444.888888 18886, 2024.
Torres. J. F., Hadjout, D., Sebaa, A., Martínez-Alvare F., and Troncoso, A. Deep learning for time series forecast- ing: a survey. Big Data, 3 2021.
Yin, G., Liu, B., Sheng, L., Yu, N., Wang, X., and Shao, J. Semantics disentangling for text-to-image generation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 2327-2336, 2019.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
Yoon, J., Jarrett, D., and Van der Schaar, M. Time-series generative adversarial networks. Advances in neural in- formation processing systems, 32, 2019.
Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Yuan, X. and Qiao, Y. Diffusion-TS. Interpretable diffu- sion for general time series generation. In The Twelfth International Conference on Learning Representations, 2024. URL https: wwwpp/epeneeeeeeeeeeepeeeeeeeewe net /frrmmrrm i 3=44hapFjo0
Wang. L., Zeng, L., and Li, J. Acc-gan: adversarial error correction gans for auto-regressive long time-series gener- ation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 10140-10148, 2023.
Wang, S., Li, J., Shi, X., Ye, Z., Mo, B., Lin, W., Ju, S., Chu, Z., and Jin, M. Timemixer+:: general time series pattern machine for universal predictive analysis. arXiv preprint ar 2rxiv:2410. 16032, 2024a.
Yue, Z., Wang. Y., Duan, J., Yang. T., Huang, C., Tong, Y., and Xu, B. Ts2vec: Towards universal representation of time series. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 8980-8987, 2022.
Wang, S., Wu, H., Shi, X., Hu, I., Luo, H., Ma, L., Zhang, J. Y., and J. Timemixer: Decomposable multi- scale mixing for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024b. URL https:/t////...... http://pppeeeeeeeeeeeeee forum? 7oshffEff....
Zhang, B., Gao, Z., Qu, Y., and Xie, H. How con- trol information influences multilingual text image gen- eration and editing? In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024a. URL htth:: /ttp:://////e///e/w net forum? GcOwGG0GGG
Wang, X., Feng, M., Qiu, J., Gu, J., and Zhao, J. From news to forecast: Integrating event analysis in LLM-based time series forecasting with reflection. Annual Conference on Neural Information Processing Systems, 2024c. URL https: http://///eeeeeeeeeeeeeem
Zhang, B., Zhang, P., Dong, X., Zang, Y., and Wang, J. Long-CLIP: Unlocking the long-text capability of CLIP. In Leonardis, A., Ricci, E.,, Roth, S., Russakovsky, O., Sattler, T., and Varol, G. (eds.), Computer Vision - ECCV 2024, volume 15109 of Lecture Notes in Com- puter Science pp. 310-325, Cham, 2024b. Springer. doi: URL https: //doi. org/10. 001.11007997----0333 02900301112998221
The Thirty-eighth
Wu, H., J., Wang, J., and Long, M. Autoformer: Decom- position transformers with auto-correlation for long-term series forecasting. Advances in neural information pro- cessing systems, 34:22419-22430, 2021.
Zhang, Q., Wang, H., Long, C., Su, I.., He, X., Chang, J., Wu, T., Yin, H., Yiu, S.-M., Tian, Q. et al. A survey of generative techniques for ppatial-teppore data mining. arXiv preprint 2rXiv:2405.09592 2024c.
Xu, J., Sun, X., Zhang, Z., Zhao, G., and Lin, J. Under- standing and improving layer normalization. Advances in neural information processing systems, 32, 2019.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In The Thirty- Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, volume 35, pp. 11106-11115. AAAI Press, 2021.
Xu, Z., Bian, Y., Zhong, J., Wen, X., and Xu, Q. Be- yond trend and periodicity: Guiding time series forecast- ing with textual cues. arXiv preprint 135522405..3522, 2024.
12

---

VerbalTS: Generating Time Series from Texts
Attribute
Value
Trend Types
[Linear, Quadratic, Exponential, Logistic]
Trend Directions
[Up, Down]
Season Cycles
[0. 2, 4]
Local Shapelets
[None, Single Peak, Sag, Double Peaks]
High Frequency Components
[0. 16, 32, 64]
Multivariable
[X-axis Flip, Y-axis Flip, Shift Forward, Shift Backward]
Table 3. Summary attribute options for synthetic datasets.
A. Datasets
In this section, we provide detailed instructions for constructing the datasets, which include six datasets of three categories, as mentioned Sec. 5.1.
A.1. Full synthetic datasets
The Full synthetic represents the datasets that both time series and text description are manually generated, including the Synth-U and Synth-M datasets.
Specifically, we first manually define 6 types of attributes, as shown in Tab. 3. Based on these attributes, time series data are generated using specific mathematical formulas and multivariate rules; While textual descriptions are generated through predefined text templates. Thus, our processed dataset combines three types of data: generated time series, textual descriptions, and predefined attributes.
We generate Synth-U dataset according to the following formula:
X = Xtrend + Xacason + Xlocal + Xaf + Xpoise
Among them, trend and season are the main components of time series, while noise, high-frequency components, and local shapelet are supplementary details. Therefore, we divide the control attributes into two categories: primary and secondary. The primary attributes are shared by all data in the data set, including Trend Types, Trend Directions, and Season cycles. The secondary attributes are attributes that are sample-specific and are difficult to be uniformly described, including Local Shapelets and High Frequency Components.
Synth-M follows a construction method similar to that of Synth-U but in a multivariate setting with an extra attribute: Multivariable. The second variable of the time series is transferred from the first variable following four kinds of rules: X-axis Flip, Y-axis Flip, Shift Forward, Shift Backward. We present all the attribute information in Tab. 3.
We sample 1000 samples for 32 combinations of primary attributes (4 Trend Types 2 Trend Directions x Season Cycles) and get 32000 samples. We randomly split the samples into training set, validation set, and test set in a ratio of 6: 1: 1. Finally, we get 24000 training samples, 2400 validation samples, and 2400 test samples.
A. PRIMARY ATTRIBUTES
The primary attributes are attributes that all data in a dataset have in common, such as trend and season in all time series. Such attributes are often represented in a structured format, but lack the ability to represent unique characteristies of the data.
Trend Types
There are 4 trend types: linear, quadratic, exponential, and logistic. As described above, t is used to obtain x. For linear
trend: Xuend = t, in this case f/ E [0, 1), a; E [0, 1]. For quadratic trend: Xirend = t?, in this case f; E [0, 1). x; E [0. 1). For exponential trend: Xtead = 1 in this case 4 E [-10, 10), x, E [0, 1). Xuend is needed range from to 1, so 4, is in [-]0,10]. For logistic trend:
Xtrnnd T=exp[-E)> where &; E [-10, 10), x4 [0, 1]. Similar the exponential trend
operation, we repeat a scaling process in logistic trend. To train the model more easily, Xucad = (x 10--0 0.5) x is used to normalize Xiren to 1..
13

---

VerbalTS: Generating Time Series from Texts
Trend Directions
There totally 2 directions: up and down. For instance, in the Cartesian coordinate system, linear line from coordinates (0,0) to (1,1) represents an "up" trend, while another line from coordinates (0,0) to (1,-1) represents "down" trend. To implement this idea, we used simple method to generate these two types of data as the following formulas: up trend is
Xtrend Xtrnn and down trend is Xrend
Xtrend-
Season Cycles
The Season represents the number of cycles in time series. We add [0,1,2,4] sinusoidal waves into the original synthetics time series.
Xgeason a (2nt + +)
where 1 [0, Meyclle, Meycle E 20, The Random Variances of these variables follow uniform distributions: U(0.4, 0.6), @ u(, 2m).
22,
A.1.2. SECONDARY ATTRIBUTES
The secondary attributes are attributes that only some samples have. Such attributes are often hard to be represented in a structured format, containing sample-specific details.
Local Shapelets
We define three local shapelets: [single peak, sag, double peaks], which are used to simulate local details in real-world time series. A single peak consists of a linear rise and linear fall, with length of both ends, the highest midpoint in the range of [1.0, 1.2]. Sag is single peak with downward concave shape that is symmetrical about the x-axis. Double peaks are two single peaks spliced together. We also divide the time series into three segments of equal length: the beginning, the middle, and the end, and add 0 or shapelet to a random position in each segment. The probability of adding nothing is 0.7, and the probability of single peak, sag, and double peaks is 0.1.
High Frequency Components
High-frequency signals are common components in time series. We add high-frequency components simulate real-world time series. The period of high-frequency components ranges from [0, 16, 32, 64] and is constructed in the same way as season where U(0.1,0.3) ø (0, 2m).
Noise
The noises are added to simulate the real-world time series. Since noise is difficult to model, adding noise is more to increase the randomness and diversity samples. Since noise is difficult to model, adding noise is more to increase the randomness and diversity of the sample. The noise will be sampled from a Gaussian distribution Xaoise NNu,, d²), where [0.04, 0.06).
A.1.3. VARTIVARIABLE
The multi variable transfer rules include X-axis Flip, Y-axis Flip, Shift Forward, and Shift Backward. X-axis Flip and Y-axis Flip are flipping the time series of the first variable along the x-axis and y-axis to get the second variable, Shift Forward and Shift Backward are shifting the time series along the time dimension where the shift distance dahin E [20, 40).
1.1.4. ATTRIBUTES TEXT DESCRIPTIONS
The text descriptions are generated from the attributes with prompt templates. Here are examples of text descriptions for each attribute in Synth-M dataset:
Trend Type & Trend Direction: "The trend is up and linear."
Season Cycles: "The season cycle is 4."
High Frequency Component: "There is a high frequency component with a period of 64."
Local Shapelet: "There is sag at the beginning. The end part has double peaks."
Multivariable: "Flip the variable along x-axis can get variable 2."
14

---

VerbalTS: Generating Time Series from Texts
A.5. RELEVANT AND IRRELEVANT TOKENS
As mentioned in Sec. 5.4.2, we manually annotated all tokens within the synthetic datasets, categorizing them as either relevant irrelevant. Specifically, tokens associated with the attribute value (refer to Tab. 3) were labeled relevant, while all other tokens were labeled irrelevant. The results are presented as follows:
Relevant tokens: "linear", "quad", ""exp", "log", "up". "down", "peak", "peaks", "sag", "double", "single", "beginning", "end", "middle", "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "x", "y", "flip", "forward", "backward", "shift".
Irrelevant tokens: "trend", "type", "direction", "cycle", "season", "the", "description", "time", "series", "variable", "area", "part", "has", "a", "and", "equal", "., "is", "high", "frequency", "component", "with", "period", "along". "axis", "signal", "on", "at", "in", "of".
A.2. Augmented real-world datasets
The Augmented real-world represents the datasets that are composed of real-world time series and manually generated text description, including the ETTml and Traffic.
Specifically, the original datasets only contain time series data. We first employ tsfresh (Nils Braun, 2024) library to extract 6 time series features, serving as attributes for baseline input. Then, text annotations are generated from extracted features through prompt templates. Details are given in Sec. 1. Thus, our processed datasets combine three types of data: time series data, text descriptions, and extracted attributes.
A. FEATURE EXTRACTION AND TEXT ANNOTATION
To annotate these real-world datasets with their corresponding text and attributes, we performed series of processing steps, including feature extraction and text annotation. Specifically, used the tsfresh (Nils Braun, 2024) library to extract four global features (Skewness, Kurtosis, Linear Trend Slope, FFT Frequency) and two local features (Local Linear Trend Slope and Number of Peaks) from the time series data. These features were then used to generate attributes and text descriptions. The extraction methods for each feature and the corresponding text generation process are detailed below.
Feature Name
Description
Skewness
Measures the asymmetry of the time series value distribution. Measures the sharpness of the time series value distribution.
Kurtosis
Global Features
Linear Trend
Describes the overall trend direction and rate of change in the time series. Identifies the dominant periodicity in the frequency domain.
FFT Frequency
Local Linear Trend
Captures the trend direction within each segment. Counts the number of local maxima within each segment.
Local Features
Number of Peaks
Table 4. Summary of global and local fcatures extracted using free for augmented real-world datasets.
Global Features Extraction and Text Annotation
The global features describe the overall characteristics of the time series, which include:
Skewness: The skewness of the time series distribution is calculated through ts skewness which measures the asymmetry of the value distribution. Based on the skewness value:
If skewness < -0.5, the text description is: "The distribution of the value in time series shifted to the negative."
If skewness > 0.5, the text description is: "The distribution of the value in time series is shifted to the positive."
Otherwise, the text description is: "The distribution of the value in time series is symmetrical."
2. Kurtosis: The kurtosis of the time series distribution calculated through tsfresh. kurtosis, which measures the sharpness of the distribution. Based on the kurtosis value:
If kurtosis < -0.5, the text description is: "and has low kurtosis."
15

---

VerbalTS: Generating Time Series from Texts
If kurtosis > 0.5, the text description is: "and has high kurtosis."
Otherwise, the text description is: "and has normal kurtosis."
Linear Trend: The overall trend the time series is calculated through tsfresh. hwapeeshh.iieeaaee linear.trend, which describes the trend direction and intensity. For example:
A strongly positive slope generates the description: "The time series going up rapidly."
A weakly positive slope generates the description: "The time series is going up slowly."
A strongly negative slope generates the description: "The time series is going down rapidly."
A weakly negative slope generates the description: "The time series is going down slowly."
A near-7ero slope generates the description: "The time series has no obvious trend direction."
FFT Frequency: The dominant frequency of the time series is identified through fsfresh, fft coefficient The frequency with the highest magnitude is used to generate the description: "The main season eycles around n pi," where is the index of the dominant frequency.
Local Features Extraction and Text Annotation
We divide the time series into three segments: beginning, middle, and end along the time dimension equally. The local feature extraction is performed on each segment independently, which includes:
Local Linear Trend: The local trend of the time series is calculated through tsfresh. Ainear.trente which describes the trend direction and intensity. For example:
"At the beginning, the time series slowly rises."
"At the middle, the time series has no obvious trend direction."
"At the end, the time series goes down rapidly."
Number of Peaks: The number of local maxima counted using tsfresh. number -paake with parameter n=10. This feature generates description such as:
"At the heginning, there are peaks."
"At the middle, there are peaks." "
"At the end, there are peaks."
After extracting all features, each time series is represented as structured vector of indices, referred to as the attributes index. This vector includes:
Variable Index: [0,K-1] where K is the variable number.
Trend Attribute: {0.1) which indicates (upward, downward).
Seasonality Attribute: [0,9] which indicates the dominant periodicity.
Skewness Attribute: (0.1.2) which indicates (negative, positive. symmetrical).
Kurtosis Attribute: (0.1,2) which indicates (low, normal, high}.
The Text Caption for each time series constructed by combining the descriptions of global and local features. For example:
16

---

VerbalTS: Generating Time Series from Texts
"The distribution of the value in time series is shifted the negative and has high kurtosis. For the overall shape, the time series is going up rapidly. The main season cycle is around 2 pi."
Global description:
Local description: "At the beginning, there are 3 peaks, and the time series slowly rises. At the middle, there are peaks, and the time series has no obvious trend direction. At the end, there are 4 peaks, and the time series is going up rapidly."
A.2.2. ETTMl
Original Dataset
ETTml is obtained from the Electricity Transformer Dataset (ETDataset) (Zhou et al., 2021), a dataset specifically designed to address the challenges of long-sequence time series forecasting. It was collected through a real-world platform in collaboration with the research team and Beijing Guowang Fuda Science and Technology Development Company. The dataset covers a two-year period from July 2016 to July 2018.
ETTml, as a subset of ETDataset, focuses on minute-level data collected from a single transformer station. Each data point in ETTml consists of 7 numerical features: High Useful Load (HUFL.) High Useless Load (HULL), Middle Useful Load (MUFL), Middle Useless Load (MULL), Low Useful Load (LUFL), Low Useless Load (LULL), and the target variable Oil Temperature (OT).
Processed Dataset
The raw dataset, consisting of single long sequence with a total length of 69,680, is first split along the temporal dimension into training, validation, and test sets ratio of 8: 8:1:1, resulting in sample sizes of 55,744 and 6,968, and 6,968 across the original seven variables. Next, the seven-variate time series data are decomposed into univariate sequences. For each set, the long sequence is further divided into shorter sequences using a sliding window with a sequence length of 120 and stride of 30. This process yields the final samples split: (train: 13,013, valid: 1,631, test: 1,631).
Till now, we obtain the time series dataset for each time series feature variable of size (N, K, I.), where N is the number of samples, and N Nurin = 13, 013, Nvalid = 1, 631, Nus = 1, 631 for train, validation and test set. K denotes the number of time series variables, and K in our processed dataset. I. represents the length of the time series, and = 120 in this dataset.
Furthermore, apply z-score normalization each variable using the mean and standard deviation computed from the training set, which ensures that each variable has a mean of 0 and standard deviation of 1, eliminating differences in scale across variables. After that, we perform the feature extraction and text annotation to obtain the attributes and the text descriptions, with processing details given in Appendix 1.2.1.
The final text descriptions are of size N, where N the number of samples. Each sample contains string, corresponding to the concatenated global and local descriptions of a time series sequence. The statistics of token numbers for the text descriptions are summarized in Tab. 5.
Set
Average Tokens
Median Tokens
Max Tokens
Std. Dev.
Training
107.28
107
131
7.49
Validation
107.00
107
127
7.63
Test
106.63
107
128
7.39
Table 5. Summary of token number statistics for ETTml dataset.
A.2.3. TRAFFIC
Original Dataset
The Istanbul-Trafiic dataset (Leo, 2024) provides minute-level time series data on Istanbul's traffic index. It includes three time series features: the Traffic index Overall(TT), Traffic index of Asian side(T) An), and the Traffic index of European side(TI_Av). The dataset covers the time period from November 1, 2022, to June 16, 2024, with a sampling frequency of one minute and a weekly update frequency.
Processed Dataset
17

---

VerbalTS: Generating Time Series from Texts
The raw dataset, consisting of single long sequence with total length of 817,769, is first taken a sample every ten minutes and split along the temporal dimension into training, validation, and test sets in ratio of 8:1:1, resulting in sample sizes of 65421, 8178, and 8178 across the original 3 variables. Next, the three-variate time series data are decomposed into univariate sequences. For each set, the long sequence is further divided into shorter sequences using a sliding window with a sequence length of 144 and a stride of 24. This process yields the final sample split: (train: 8178, valid: 1023, test: 1023).
Till now, we obtain the time series dataset for each time series feature variable of size (N, K, L), where is the number of samples, Naainn 8178, Noalid = 1023, Nustt = 1023 for train, validation and test set. K denotes the number of time series variables, and K 1 in our processed dataset. represents the length of the time series, and L 144 in this dataset.
Furthermore, we apply z-score normalization to each variable using the mean and standard deviation computed from the training set. After that, we perform the feature extraction and text annotation to obtain the attributes and the text descriptions, with processing details given in Appendix 1.
The final text descriptions are of size N, where N is the number of samples. Each sample contains string, corresponding to the concatenated global and local deseriptions of a time series sequence. The statistics of token numbers for the text descriptions are summarized in Tab.6.
Set
Average Tokens
Median Tokens
Max Tokens
Std. Dev.
Training
102.96
103
130
6.83
Validation
103.70
104
124
6.95
Test
101.90
101
126
6.64
Table 6. Summary of token number statistics for Traffic dataset.
A.3. Real-world datasets
The real-world datasets represent the datasets that are composed of real-world time series and text description pairs, including the Blind and Weather.
Specifically, the original datasets contain paired time series and textual descriptions. We extract attributes from the original text using ChatGPT 3.5 (OpenAI, 2023) for baseline input, producing a processed dataset that combines three types of data: time series data, text descriptions, and extracted attributes.
1. BLINDWAYS DATASET
Original Dataset
The raw data is sourced from the benchmark dataset BlindWays (Kim et al., 2024). It includes 3D motion data from 10 blind individuals and 1 visually impaired participant, capturing actions performed while navigating along 8 carefully designed urban routes using a white cane or guide dog, along with corresponding rich textual descriptions.
The motion data comprises multivariate data from 1,029 motion segments, recorded through 18 IMU sensors worn by each participant, detailing the positions, angles, and trajectories of 24 joints in the human body. Each joint is represented and recorded using three variables, resulting in total of 72 variables.
Additionally, the dataset provides (2 * 1,029 = 2,058) detailed textual descriptions, which include high-level descriptions summarizing action intent and contextual background, and low-level descriptions that meticulously document action specifics, such as step count and cane usage, for each time series sample. We utilized the 1029 high-level descriptions as the textual descriptions paired with the time series in our work.
Processed Dataset
For the processed motion dataset, conducted random sampling the raw dataset to partition the raw data into training, validation, and test sets. The 1,029 motion segments were split into a training set containing 823 samples, a validation set with 103 samples, and test set also comprising 103 samples, resulting in a final distribution of (train: 823, validation: 103, test: 103) samples. Additionally, we extract attributes from the original text using ChatGPT 3.5 (OpenAI, 2023).
For the time series data, we maintain the same shape as the raw dataset. The shape of the time series data is represented as (N, K, I)), where N denotes the number of samples. Specifically, for the training, validation, and test sets, we have
18

---

VerbalTS: Generating Time Series from Texts
Muain 823, Maaid- 103, and Nust 103. The variable count K 72, while the time series length L 600. Furthermore, apply 7-score normalization to each variable using the mean and standard deviation computed from the training set.
For the text data, we utilize the high-level textual descriptions from the original BlindWays dataset. The text data has a shape of N, where .N indicates the number of samples, and we utilize the high-level textual descriptions in the original dataset. The statistic details of the number of tokens in the text data are given in Tab. 7.
Set
Average Tokens
Median Tokens
Max Tokens
Std. Dev.
Training
33.24
31
135.0
11.68
Validation
34.56
32
106.0
12.76
Test
32.51
32
71.0
10.23
Table 7. Summary of token number statistics for BlindWays datasct.
For the attributes, the shape is represented as A), where N indicates the number of samples and A = 2 represents the number of attributes variables. The two attributes are derived by classifying and extracting features from the original dataset's 's text using ChatGPT 3.5 (OpenAI, 2023): the Guide Method and Hand. Guide Method attribute indicates the method employed by the blind individuals, encompassing two values: [cane, guide dog] (i.e. using a white cane or a guide dog). Hand attribute indicates the hand used by the blind individuals, with three possible values: [left, right, unknown] (i.e. left hand, right hand, or unknown).
Attribute
Value
Guide Method
[Cane, Guide Dog] [Left, Right, Unknown]
Hand
Table Summary of attribute options for BlindWays dataset.
A.3.2.
DATASET
Original Dataset
The raw data is derived from the weather station WS Beutenberg, located at the Max Planck Institute for Biogeochemistry in Jena, Germany. The dataset includes climate time series spanning years from 2014 to 2022.
The time series data consist of 21 weather-related variables recorded 10-minute intervals throughout the year, with timestamps accurate to the second. The 21 weather-related variables include: atmospheric pressure (p. mbar); temperature (I, degC); potential temperature (Tpot, K); dew point temperature (Tdew, degC); relative humidity (rh, %); maximum vapor pressure (VPmax, mbar); actual vapor pressure (VPact, mbar); vapor pressure deficit (VPdef, mbar); specific humidity (sh, g/kg); water vapor concentration (H2OC, mmol/mol); air density (rho, g/m²); wind speed (wv, m/s); maximum wind speed (max. wv, m/s); wind direction (wd, deg); rainfall (rain, mm); rain duration (raining, s); shortwave radiation (SWDR, W/m)); photosynthetically active radiation (PAR, jmol//?/ss; maximum photosynthetically active radiation (max. PAR, pmol/m??/s); logarithmic temperature (Tlog. degC); and carbon dioxide concentration (CO2, ppm).
Textual descriptions are generated using weather forecasts obtained from publicly available platforms. These descriptions are created using the GPT-4 model, which generates weather-related descriptions independently of the time series data, relying solely on the weather forecast information.
Processed Dataset
For the processed dataset, we initially aggregate samples over 6-hour period and partition the aggregated samples in chronological order, using data from 2014 to 2020 for training. 2021 for validation, and 2022 for testing, resulting in 10, samples for training, 1,460 samples for validation, and 1,448 samples for testing. Each sample is pair of time series data and text data. Additionally, we extract attributes from the original textual descriptions to serve as input for the baseline.
For time series data, the shape is represented as (N, K, L), where N is the number of samples, K is the number of variables, and L is the sequence length. Specifically, for the training, validation, and test sets, we have Nirain = 10, 192, Nyalid = 1, 160, and Nest = 1, 448. The variable count K = 21, corresponding to the original weather variables. The sequence length is set to 36, representing 6 hours of data (1 (10-minute intervals, samples per hour, resulting in time series length of 6 6 = 36).
19

---

VerbalTS: Generating Time Series from Texts
To standardize the data, z-score normalization is applied to each variable using the mean and standard deviation computed from the training set.
For text data, each time series sample is associated with three textual descriptions, allowing for variability in phrasing while maintaining the same semantic meaning. In the processed dataset, all descriptions are retained, and during training, one description is randomly sampled for each time series instance. The textual data has a shape of N, where is the number of samples and we choose I textual description from 3 available textual descriptions each sample. The detailed statistics of the text length in tokens for the training, validation, and test sets are presented in Tab.
Set
Average Tokens
Median Tokens
Max Tokens
Std. Dev.
Training
246.42
244
397
30.04
Validation
262.91
260
423
33.16
Test
258.61
256
388
31.49
Table 9. Summary of token number statistics for Weather dataset.
For attribute data, the shape is represented as (N, where N the number of samples and A 7 the number of attributes. We use ChatGPT 3.5 (OpenAI, 2023) to classify and extract 7 attributes from the original textual descriptions, as detailed in Tab. 10:
Attribute
Values
Season
spring, summer, fall, winter
Time of Day
early morning, morning, afternoon, evening
Weather Condition
sunny, cloudy, rain, foggy, snowy, unknown increase, decrease, steady, unknown S. N, W. E, SW, SE, NW, NE, unknown low, average, high, unknown low, average, high, unknown
Temperature Trend
Wind Direction
Atmospheric Condition
Humidity Level
Table 10. Summary of attribute options for Weather dataset.
A.4. Transformation between structured and unstructured condition
In this section, we summarize the transformation between structured attribute conditions and unstructured textual description conditions in the three aforementioned types of datasets.
For synthetic datasets (Synth-M. Synth- U), the entire datasets are manually generated. First, primary and secondary attributes are predefined for multi-semantic characteristics. Then, textual descriptions are generated by substituting attribute values into the prompt templates. Thus, we generate text from attributes, as described in Appendix A.1.4
For augmented sal-world datasets (ETTm1, Traffic), the original datasets only contain time series data. First, the attributes are extracted from the original time series using the tsfresh. Subsequently, the textual descriptions are chosen from textual templates determined by the value of the attributes. Thus, we generate text from attributes, described in Appendix 1.2.1.
For real-world datasets (BlindWays, Weather), the original datasets already contain time series-text pairs. The attributes are directly generated from the textual descriptions in the original datasets using ChatGPT 3.5 (OpenAI, 2023). Thus, we generated attributes from text for these datasets.
B. Model Architecture
In this section, will show the architecture of method and some details about previously mentioned modules in Sec. 4.
Model overview. The VerbalTS model takes a pair of noisy time series x R and textual description E NM as inputs, and output the estimated noise E,. The noisy time series x; E R is patched into R resolutions and encoded into embedding P. E Kxx D via linear layers. TSA and SSA modules process P, to compute temporal and spatial attention, followed by a patch decoder and a multi-resolltion mixer, producing the estimated noise E, E kxt. as mentioned in Sec. 4.1. Textual descriptions e are encoded into embeddings E TR. MDD using a text encoder, followed by the multi-focal
20

---

VerbalTS: Generating Time Series from Texts
text processor that generates unified semantic representation Z E RSSXXXXD for the complete generation process, as mentioned in Sec. 4.2. For guiding conditional generation, the text representation Z is integrated with the time series representation Pt, E [] via the custom clement-wise multiplication and addition, as mentioned in Sec. 4.3.
B.1. TSA and SSA attention mechanisms
As mentioned in Sec. 4.1, introduce two attention mechanisms, Temporal Self-Attention (TSA) and Spatial Self- Attention (SSA), modeling the temporal and spatial dimensions of time series data, respectively. In TSA, we employ masking mechanisms to enforce intra-resolution attention for temporal modeling; While in SSA, the attention operation is applied across all variables, allowing every variable to interact with others.
Given a batch noisy time series input X IR KxL, multi-resolution patching and patch encoding are first performed, as mentioned in Sec. 4.1. Then, we obtain a multi- embedding P. IR. R x x K denotes the number of variables, N Σ N,, where N, denotes the patch number for resolution and D denotes the embedding dimension.
For simplicity and without causing confusion, we temporarily omit the notation t. The time series representation P is passed to the TSA, which is in the shape of (K x N x D), Temporal Transformers compute the self-attention along the temporal dimension N, and in parallel along dimension K. Specifically, the dimension K is regarded as the batch dimension, ensuring that each variable E [1, K] with its time series embedding of length N independent of others. This allows attention computation to be performed independently for each variable.
To restrict temporal self-attention within each resolution [1, R). we leverage an attention mask M N that restricts attention to patches within the same resolution, where R denotes the total number of resolutions. The mask is defined as:
M
0C
0
0
M
00
M
00
-00
MR
where each sub-matrix M, R.N,NN, with elements set to allowing attention within resolution v:
0
0
0
M,
and the residual values are set to negative infinity to mask nnsoruresolution attention. The temporal self-attention TSA output is computed
QKT
PiSA softmax
+M V.
VD
Where Q, K, V are query, key, and value matrices calculated from P. The output PISA E RKxNxD is subsequently passed into the SSA module.
To facilitate the spatial attention computation along the variable dimension K SSA module, PTSA permuted into the shape of (N x D). as the actual input of the Feature Transformer Layer to compute the self-attention along the spatial dimension K, and in parallel along dimension N. Similarly, the temporal dimension N is regarded as the batch dimension, ensuring that each patch index E [1, .]] with its feature embedding of dimension K is independent of others. This allows attention computation to be performed independently for each patch index, while each variable can have attention with every other variable effectively. The spatial self-attention output is computed as:
QK VD
PASA- = softmax
V
Where Q, K, V are query, key, and value matrices calculated from PISA The output PSSA E NxKxD subsequently passed into the FFN module.
21

---

VerbalTS: Generating Time Series from Texts
The complete process for temporal and spatial modeling in the j-th layer of the noise estimator be expressed as:
P p + FEN(SSA(TSA(PUU)) + e))),
where j E [] P(O) = C E RD is the embedding of the current denoising time step, and FFN represents the ford-rorward network. The final output is p(J) E RKXNXD.
B.2. Patch Decoder and Multi-resolution Mixer
After applying the TSA, SSA and FFN in residual layers, we obtain the time series representation p E RXXNXD, To generate multi-resolution noise estimates, two output modules are required to process these embeddings into the final noise estimates, as mentioned in Sec. 4.1.
First, Patch Decoder divides the multi-resolution representation P(J) into R resolutions and decodes them back into the original time series space, generating estimated noises {E") for R resolutions, where e" E RR
Next, Multi-resolution Mixer combines the estimated noise at different resolutions output from the Patch Decoder, into the final multi-resolution noise estimate. Specifically, we concatenate the estimated noise from different resolutions {<"),=m> and feed them into a Multi-Layer Perceptron (MLP) to generate the final noise estimate:
E = MLP ((=" .6,
where o denotes the concatenation operation, E E IR X denotes the final estimated noise.
B.3. Custom Element-wise Multiplication And Addition
As mentioned in Sec. 4.3, we propose the custom element-wise multiplication and addition, denoted @ and @, for applying the condition. In this section, we will introduce the details of these two operations. Suppose we have two matrices E RT*XXx x and B E S K Rx where T is the number of the diffusion step, K the variable number, N is the length of the patch sequence, S is the stage number, and R is the resolution number. Given each element Ar E IxxxxxxD in A with index l, k; n, we can find the corresponding index 8, k, r of element B. E RIxxxxxD in B through the following rules:
tx S T
b k,
arg min [E N,) 2 n).
where the first equation projects the given diffusion step t to the corresponding diffusion stage s; the second equation projects the given variable i to the same variable; and the third equation finds the corresponding resolution r covering the given multi-resolution patch index n, as described in Sec. 4.1.
Then the custom element-wis multiplication and addition can be expressed as:
(A O B)r = At,k.n X Ba,t.ry
(A B)r...n = Ar, Ba..r-
With the custom element- wise multiplication and addition, we can apply the multi-semantic text conditions to the corre- sponding time series components.
C. CTTP Model Details
As mentioned in Sec. 5.1, the Contrastive Text-Time Series Pretraining (CTTP) model is used to calculate the metrics which conceptually similar to the CLIP model (Radford et al., 2021). The purpose of the CTTP model is to train the time series encoder ths (x) and the text encoder elrea((C (c) by learning the alignment between a batch of time series data E R Bx xxxxxx X and its associated textual descriptions C E NBxM, where B is the batch size, paired samples x and are individual samples
22

---

VerbalTS: Generating Time Series from Texts
from X and C, respectively. We use PatchTST (Nie al., 2023) the time series encoder and a pre-trained text model of Long-clip (Zhang et al., 2024b) the text encoder. The output emheddings of the time series encoder and text encoder are Zx E Bxd and Ze E IR Bxd respectively. These embeddings are mapped to shared embedding space, enabling the alignment of time series and text data. The model is trained to minimize the cross-entropy loss between Zx and Ze. The pseudocode for the CTTP model is presented in Algorithm 1.
Algorithm 1 Pseudocode for the CTTP Model
Input:A batch time series and text pairs (X E BBXxL C CENBxM Output:Total cross-entropy loss Ceross
Extract embeddings of time series and text (joint embedding space)
2: Zx TimeeSerieSEncoder(X
DZx E Rxd time series embeddings.
3: Ze f TextEnnoder((C)
DZ E IR Bxd text embeddings.
4: Compute pairwise similarities
5: S < -imiim Zx, Ze)
bS BxB
similarity score matrix.
6: # Compute cross-entropy losses
7: Lx CrossEntropy(S.. axis 1)
Loss for time series alignment. Looss for text alignment.
8: Le f CrossEntropy((S axis 0)
9: # Total cross-entropy loss:
10: Leross f- LxtLe
Averaaee loss.
2
11: Return:
Leross
denotes the dimension of the joint embedding space, where both time series and text embeddings are projected. {0, 13 BxB the identity matrix.
The CTTP model effectively aligns time series with text using the cross-entropy loss, providing shared embedding space. We further use this model to evaluate the quality of the generated time series.
Following contrastive learning practices (Radford et al., 2021), we evaluate the CTTP model using a retrieval-based protocol. For each time series a random batch (B=32), we compute the top-1 accuracy of retrieving its paired text from among candidates. The results in Tab. 11 demonstrate the CTTP model's effective semantic alignment ability, providing evidence of the metries reliability.
Dataset
Synth-M
BlindWays
Weather
Synth-U
ETTml
Traffic
Accuracy
94.68%
15.53%
70.93%
97.55%
50.28%
36.56 %
Table 11. The evaluation results of CTTP models on different datascts. We report the accuracy of retrieving the most similar text description in 32 candidates given the time series.
D. Evaluation Metrics FID
The Frechet Inception Distance (FID) (Heusel et al., 2017) is a widely used metric for evaluating the fidelity of generated data. FID is based on the Frechet Distance, which measures the similarity between two multivariate Gaussian distributions. By comparing the feature distributions the real and generated data, it quantifies how closely the generated data resembles the real data.
23

---

VerbalTS: Generating Time Series from Texts
Specifically, the FID is defined as:
FID(X, X) - -,,
] represent the of real and generated data samples with the size of N, respectively. The terms Mı E Rd and Eg Rddd are the mean and covariance of the real data embedding distribution {z, ( while Ha E Rd and Σg E Raxd are the mean and covariance of the generated data embedding distribution {2; i==1 d is the dimensionality of the embedding. The embeddings Z;, 2, E I are constructed as:
where X [x: and X
2; = di (x;), 2, = tir (R:)
where (-) is the time series encoder of the CTTP Model, which processes time series data x; or &; to generate latent emheddings.
For the FID formula, the first term 1 fell measures the squared Euclidean distance between the mean vectors of the real and generated data distributions, reflecting the difference in their central tendencies. The second term, Tr(E, + Σg 2z) )), captures the discrepancy in the covariance structures of the two distributions, where Tr denotes the trace of matrix. A smaller FID value indicates higher similarity between the real and generated data distribution, suggesting that the generated data has a higher fidelity.
D.2. J-FTSD
The Joint Frechet Time Series Distance (J-FTSD) (Narasimhan et al., 2024) is metric specifically designed to evaluate conditional time series generation models. J-FTSD measures the similarity between the joint distributions of real and generated data, incorporating both the time series and their associated conditions, which corresponds to the text in our setting. By calculating the Frechet Distance in the joint embedding space, it provides a comprehensive evaluation of the fidelity of the generated results, considering both the time series and text.
Specifically, the J-FTSD is defined as:
J-FTSD(D, 112 + + 2
where D = [(x; e and D [{e, c2)) represent the joint distributions of real and generated data with size of N, respectively. Each sample consists of time series x, or 8; and its corresponding condition C, which is text in the CTTP model. The terms Pz Rhis and Σg E 1e x dpin are the mean and covariance of the joint embedding distribution of the real data {z; | while (1a E thin and Σg E Thaa x opa are the mean and covariance of the joint embedding distribution of the generated data {2.).-) djoin = du + diexs is the dimensionality of the joint embedding, where du and dext are the dimensionality of the time series embedding and the text embedding respectively. The embeddings 2,, 2, E TR Tgdhiiare constructed as:
(x;) (c,), 2; thr (St:) (hee((..)
where the (-) is the time series encoder of the CTTP Model, tha (-) is the text encoder in the CTTP Model, and denotes con- catenation. A smaller J-FTSD value indicates higher similarity between the real and generated data, which comprehensively evaluates the fidelity of conditional time series generation by considering both the temporal and contextual similarity in the joint embedding space.
D.3. CTTP Score
The CTTP score evaluates the sample-level alignment between the generated time series and the corresponding text description. It is calculated through the CTTP model, which aligns time series with text descriptions in a shared embedding space. Details of the CTTP model are provided in Appendix C.
Specifically, the CTTP model used to encode the time series x to the embedding x = v/ (x). and encode the text e to the embedding 2. (ces(C),, where v-a(.) and vres(()) are the time series encoder and the text encoder of the CTTP Model, respectively. The CTTP score is then defined the dot product between Zx and Ze. given by:
CTTP(x, c) 2x
where denotes the dot product. A higher CTTP score indicates better semantic alignment between the time series data and its corresponding textual description, reflecting the CTTP model's ability to capture meaningful relationships between these two modalities.
24

---

VerbalTS: Generating Time Series from Texts
E. Implement Details
E.1. Main Experiment
For all experiments, we set the number of diffusion steps as T= 50, embedding size for attributes and time series as 64. For training, we use Adam optimizer to train the model, the initial learning rate set to be le-4 with MultiSttpL_R scheduler for all datasets, the batch size is be 512 for Synth-M, Synth-U, Weather, ETTml and Traffic, 16 for BlindWays. For the hyperparameters of the multi-focal modeling, (R (3,3) 3) for all datasets. All experiments conducted three times running with different random seeds.
E.2. Effect of Multi-focal Text Processing
To evaluate how VERBALTS processes nuanced textual details (Sec. 5.4.3), analyze the averaged attention values of each token in the vocabulary. For each text description in the dataset, we first divide it into separate tokens and encode the tokens to embeddings independently. Then the token embeddings are used to calculate the attention values through Softmax QaKg vD in Eq. (5). For each token in the vocabulary, we calculate the average of its attention values across all sentences in which it appears. It's worth noticing that, in Fig. 6, we position semantically relevant tokens in the first half of the x-axis and place irrelevant tokens in the second half. The attention value distribution across the vocabulary further demonstrates that VERBALTS learns to focus on semantically relevant tokens.
F. Experimental Results
F.I. More T-SNE Visualization Results
We provide visualization results comparing the generation based unstructured and structured data 5.4.1.
1.0
1.0
Ground Truth
Ground Truth
0.8
VerbalTS
0.8
TimeWeaver
Dimension
Dimension
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.8
0.0
0.8
1.0
Dimension
Dimension
Figure Comparison generated distribution between the VERBALTS (left) and Time Weaver (right) on BlindWays dataset.
1.0
1.0
Ground Truth
Ground Truth
0.8
VerbalTS
0.8
TimeWeaver
2
2
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Dimension
Dimension
Figure 10. Comparison of generated data distribution between the VERBALTS (left) and Time Weaver (right) ETTml dataset.
25

---

VerbalTS: Generating Time Series from Texts
1.0
1.0
Ground Truth VerbaiTS
Ground Truth
TeavWe
0.8
Dimension
Dimension
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Dimension
Dimension
Figure 11. Comparison of generated data distribution between the VERBALTS (left) and Weaver (right) on Traffic dataset.
1.0
1.0
Ground Truth VerbalTS
Ground Truth
Tiaveeaver
0.8
0.8
Dimension
Dimension
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.2
0.6
0.8
1.0
0.0
0.2
0.6
0.8
1.0
Dimension
Dimension
Figure Comparison of generated data distribution between the VERBALTS (left) and Weaver (right) on Synth-U dataset.
F.2. Generation Case Study
In this section, we provide some visualization results (See. 5.2) to compare the generation ability of our method with the baselines. As shown in Fig. 13, VERBALTS better aligns with the semantic information, capturing nuanced details more accurately.
F.3. Intervention Case Study
In this section, we present additional visualization results from the intervention experiment (Sec. 5.4.2), which compares the effects of masking semantically relevant tokens versus irrelevant ones. The results demonstrate that VERBALTS effectively focuses semantically relevant information in the text achieve fine-grained control.
F.4. Editing Case Study
In this section, we provide more results of the editing experiment (Sec. 5.4.5) which compares the performance VERBALTS with TEdit.
26

---

VerbalTS: Generating Time Series from Texts
season cyc O. The trend is log down. The end part has sag, There high frequency component with a period 64.
Sample
10
100
50
100
100
There is 4 season cycle. The beginning part has a sing e peak. The high frequency component the time series has the period equal to 64. The trend type exp.The trend direction down.
Samale1
200
40
50
100
100
50
100
100
50
100
There cycle. The beginning single
time
series up
40
Myt
Sample
20
100
There is a high frequency companent with period of 64, There season of 4. The trend type is
log. The trend direction is up. The middle part has
10
2
Sample
50
100
50
100
50
100
50
100
50
100
The time linear down trend. There cycle. The high frequency component the time series the period equal 64. Deuble beginning area.A single peakat end
Sample
100 TimeVQVAE
So
100
50
100
50
100
So
100
DiffTime
TimeWeaver
TEdit
VerbalTS
Figure 13. The qualitative comparison between the VERBALTS and basclines. From left to right column, there are Time Dime Time Weaver, TEdit, and VerbalTS.
27

---

VerbalTS: Generating Time Series from Texts
The season cycle is 1. The high frequency component in the time series has the period equal to 64. The time series has log and down trend. The beginning part has double peaks.
Sample 0
CTTP:48.14
CTTP:46.94
CTTP:34.78
0
-2
100
(ls, to, The
double, peaks
The time series has linear and trend. The high frequency component in the time series has the period equal to 32. There is a season of 1.
Sample 1
www.erraa BMW
CTT9:99.89
-1
CTTP:50.06
CTTP:34.60
-2
50
100
50
100
50
100
<the, is, of)
{linear, up, 1>
The season cycle is 1. The time series has a high frequency signal of 16 frequencies. The time series has a linear and up trend. A sag at beginning area. The end part has double peaks.
Sample 2
2
CTTP:49.35
1
CTTP:55.35
CTTP:29.08
0
-1
50
100
50
100
0
50
100
iss The, and)
(linear, end}
There a sag beginning. The time series has a exp and down trend. The season cycle is 2.
Sample 3
-1
CTTP:49.64
-1
CTTP:49.67
CTTP:27.53
50
100
50
100
0
50
100
(The, has, is}
( (sag, beginning, exp)
Double peaks at middle grea.Double peaks end area. The season cycle is O. The trend is log and down. The time series has high frequency signal of 64 frequencies.
Sample
0
CTTP:50.90 90
CTTP:47.36 36
CTTP:36.30
-2
100
100
0
{at, area, of)
{double, end, log)
Figure 14. Comparison of generated data before masking (left column), after masking irrelevant tokens (middle column) and relevant tokens (right column). [Masked tokens) are shown below.
28

---

VerbalTS: Generating Time Series from Texts
Sample There is 1 season cycle. The trend is (log exp) and down. There is single peak at beginning. (A single peak Double peaks) middle There high frequency component with period of 64.
CTTP:39.08
CTTP:46.8M
CTTP:44.66
CTTP:51.81
Sample A at (middle beginning) area. area. There single peak at end. There high frequency component with period of 16. There is season of 4. The time series has and (up down) trend.
CTTP:27.76
CTTP:49.68
Sample The trend is quad and sag (beginning + end) area. There (0 + 2) season cydl.
CTTP:52.17
Sample 4: The time series has quad and down trend. The high frequency component the time series has period equal to 16. There is season of (4 - 0). The beginning part has (sag single peak).
CTTP:40.97
upLhs MPM
CTTP:24.23
Sample 5: The trend is (log - linear) and up. The time series has high frequency signal frequencies. There sag at beginning. (Double peaks - sag) at end area. The season cyde is
CTTP:50.08
Before editing
Edited through TEdit
Edited through VerbalTS
Figure 15. Illustration of editing task. Column 1: the raw time series before editing. Column 2: result edited by TEdit (Jing et al., 2024a); Column result edited by our VERBALTS; Column the condition prompts for editing, with (source - target) properties.
29


Reviews:
[{"review_id": "eyR2wOmXz2", "reviewer": "Reviewer_W3gc", "summary": "This paper presents a new task of generating time series data from unstructured text and introduces VERBALTS, a method that combines a multi-view time series noise estimator with a multi-focal text processor. Additionally, it establishes a new benchmark featuring multi-faceted time series datasets enriched with textual information.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments!\n\n> **W1**: The key aspect of this paper appears to be the alignment between text and time series data. However, the explanation of this process is unclear. It is not evident how different views and varying scales of time series data are aligned with the textual information. Additionally, the experimental analysis lacks an in-depth discussion on this critical aspect.\n\nIn our article, we propose aligning time series and text descriptions from three views: the temporal view, the spatial view, and the diffusion view. Below we briefly explain these aspects with (i) motivation, (ii) technical approach and (iii) experimental evidence to make it clearer.\n\n(i) **Motivation**. This design of multi-view modeling and generation is upon the findings from existing studies and empirical observations:\n+ From the temporal view, prior research [1] has shown that the influence of structured condition like attributes on time series generation spans multiple temporal scales as discussed in Section 4.1 in our paper. Similarly, in Section 5.4, we observe that the control of unstructured textual information over time series also operates across different time ranges and scales. For example, descriptions of trend direction influence the overall rise or fall of the time series, while descriptions of local peaks determine finer morphological details. \n+ From the spatial view, we analyze the impact of text on different variables within the time series and find their influence varies across variables, as illustrated in Section 5.4. For instance, in motion generation, a description of \"walking\" predominantly influences the motion trajectory related to leg joint variables. \n+ From the diffusion view, many studies like [2,3] have found that models at different diffusion steps capture information at varying granularities. For example, the early stages of denoising primarily shape the overall structure, while the later stages refine the details. We provide a comprehensive discussion of the motivation for our method in Sections 4.1 and 4.2.\n\nWe hope the above contents help understand the motivation of our study.\n\n(ii) **Approach**. Based on the motivation mentioned above, we propose to achieve alignment between time series and textual information across these three views:\n+ We first model the time series considering the three views. Specifically, we adopt a multi-resolution representation for the temporal view, employ spatial attention for the spatial view, and divide diffusion steps into multiple stages for the diffusion view. These are discussed in Section 4.1. \n+ Next, we reprogram text descriptions into representations that correspond to these different views using a multi-focal text processor, as detailed in Section 4.2. \n+ Finally, we align the multi-view text representations with the corresponding time series parts using a semantic adapter, as described in Section 4.3.\n\nWe will keep refining the description of our paper to reflect the above summary. \n\n(iii) **Experimental Evidence**. In Section 5, we proposed three research questions first and carried out experimental analysis accordingly. Our method tackles the challenges of text and time series alignment and has significantly improve the text conditional time series generation, as shown in Section 5 and detailed as below:\n+ In Section 5.3, the ablation study on the three views of text representation shows that incorporating multi-view alignment between time series and text descriptions enhances generation performance.\n+ In Section 5.4.1, we demonstrated that multi-view alignment effectively alleviates negative influences of incorporated noise from unstructured text condition.\n+ In Sections 5.4.2 and 5.4.3, we found that the multi-focal text processor mitigates the impact of noise in text, and the text processor assigns varying attention to tokens across different focuses.\n+ In Section 5.4.4, we also showcased our method's effectiveness of semantic control on time series editing.\n\nWe hope that the above summarization of experimental analysis has addressed your concerns about how our method tackles alignment between text and time series.\n\nIn conclusion, we believe that our proposed method is well-motivated, conceptually sound, and clearly articulated in the paper and it's also been recognized by other reviewers (Strengths from Reviewers Gv7f and 7E5h). Furthermore, we have conducted comprehensive experiments to study this novel task of text-based conditional time series generation. We'll continue refining the paper to enhance clarity and readability, and we hope the explanation above provides a better understanding of our work. We look forward to further valuable discussions with you.\n\n[1] Towards editing time series. NeurIPS 2024.\n\n[2] How Control Information Influences Multilingual Text Image Generation and Editing? NeurIPS 2024.\n\n[3] MG-TSD: Multi-Granularity Time Series Diffusion Models with Guided Learning Process. ICLR 2024.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "This paper does not contain any theoretical discussions and claims.", "experimental_designs_or_analyses": "The experimental designs are reasonable and comprehensive.", "relation_to_broader_scientific_literature": "The ideas presented could inspire further research into time series generation.", "essential_references_not_discussed": "I think this paper have included all essential references in this research area.", "other_strengths_and_weaknesses": "Strengths:\n1. This paper introduces an interesting task of generating time series data from unstructured text and provides a corresponding benchmark.\n2. Extensive experiments demonstrate that the proposed method achieves good performance.\n\nWeaknesses:\nThe key aspect of this paper appears to be the alignment between text and time series data. However, the explanation of this process is unclear. It is not evident how different views and varying scales of time series data are aligned with the textual information. Additionally, the experimental analysis lacks an in-depth discussion on this critical aspect.", "other_comments_or_suggestions": "please refer to weaknesses", "questions_for_authors": "please refer to weaknesses", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hYj7tuCegQ", "reviewer": "Reviewer_7E5h", "summary": "This paper studies time series generation, specifically generating time series from text. It proposes a method named verbalTS, which employs a multi-focus alignment and generation framework to effectively model the complex relationship between them.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments.\n\n> **Evaluation**: I think the main issue is that the authors omit several straightforward yet crucial baselines: directly using LLMs for time series generation, as well as employing visualizations for iterative revision. From my experience, the revision-based approach already achieves good generation performance. I suggest the authors incorporate multiple LLMs along with a simple iterative refinement mechanism as additional baselines.\n\nThank you for your suggestions and great idea! \n\n**On LLM utilization for time series generation**:\nIf our understanding is correct, your suggestion proposes leveraging large language models (LLMs) for time series generation with an iterative refinement mechanism, which is indeed an interesting idea. However, applying LLMs to time series generation presents several non-trivial challenges:\n+ Understanding time series with LLMs remains an open problem [1]. As discussed in [1], fundamental reconsiderations are required for both the technical foundations of time series modeling and the associated evaluation and benchmarking methodologies.\n+ Bridging the gap between text and time series modalities is challenging. LLMs are inherently designed for *discrete* text inputs and outputs, whereas time series data consists of *continuous* real-valued sequences. This discrepancy poses significant challenges in encoding and decoding time series using LLMs [2]. Our work introduces a novel approach to achieving fine-grained *semantic alignment* between text and time series, going beyond token-level reprogramming [2,3].\n+ Limited paired text–time series data hinders progress in this direction. Few existing works explore general time series generation conditioned on unstructured textual information due to the scarcity of paired datasets. However, our work proposes a novel method to augment existing time series data with textual descriptions, a contribution also acknowledged in Strength 3 by Reviewer Gv7f.\n\nWhile leveraging LLMs for time series generation is not straightforward, it remains a promising avenue for exploration. Extending LLMs to model, understand, and generate time series beyond their traditional text-processing capabilities is an exciting research direction.\n\n**On the iterative refinement mechanism**:\nYour insight regarding the iterative refinement mechanism in the generation process is highly valuable! In our method, the multiple denoising steps in the reverse diffusion process naturally align with the concept of iterative refinement. Specifically, as described in Equation (3) of our paper, the model progressively refines the generated time series by iteratively denoising it at each step. The process begins with pure Gaussian noise and gradually converges to the fully generated time series. This autoregressive denoising mechanism closely resembles iterative refinement. A detailed explanation can be found in Section 3.2 of our paper.\n\nOverall, your proposed idea is highly intriguing. Exploring how multiple LLMs could iteratively generate time series is a promising research direction, and we plan to conduct more in-depth investigations and experiments in the future.\n\n[1] Are Language Models Actually Useful for Time Series Forecasting? NeurIPS 2024.\n\n[2] S2IP-LLM: Semantic Space Informed Prompt Learning with LLM for  Time Series Forecasting. ICML 2024.\n\n[3] Time-LLM: Time Series Forecasting by Reprogramming Large Language Models. ICLR 2024.\n\n> **W1**: no code.\n\nThanks for your suggestion. You can find the details of our code at the following link: https://anonymous.4open.science/r/VerbalTS-E5BC\n\nAs stated in Section 5 of our paper, we plan to release all datasets and models upon the paper's acceptance.", "claims_and_evidence": "NA", "methods_and_evaluation_criteria": "I think the main issue is that the authors omit several straightforward yet crucial baselines: directly using LLMs for time series generation, as well as employing visualizations for iterative revision. From my experience, the revision-based approach already achieves good generation performance. I suggest the authors incorporate multiple LLMs along with a simple iterative refinement mechanism as additional baselines.", "theoretical_claims": "NA", "experimental_designs_or_analyses": "NA", "relation_to_broader_scientific_literature": "NA", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "No code.\n\nI appreciate the authors' effort, and I believe time series generation is an interesting yet underexplored problem.", "other_comments_or_suggestions": "NA", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m56CfowOCw", "reviewer": "Reviewer_aNaB", "summary": "This paper proposes VerbalTS, a novel framework for generating time series from unstructured textual descriptions. VerbalTS employs multi-focal alignment and generation framework that effectively models their complex relationships. Empirical evaluations demonstrate the benefits of VerbalTS in generation quality compared to existing baselines.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the comments and suggestions!\n\n> **Exp**: Extended Analysis covers the subset of the used datasets.\n\nThanks! The used datasets represent both synthetic and real-world datasets, showcasing diverse data coverage.\n\nPer your suggestion, we conducted additional experiments on other datasets, except those analysis requiring grounded generation function available only in synthetic datasets. The results (https://files.catbox.moe/dmmpmt.png) align with the findings in Sec. 5.4 of our paper. We'll revise the paper accordingly.\n\n> **Suggestion 1**: The ablation study could better use w/o $A^s$, w/o $A^t$ and w/o $A^d$.\n\nPer your request, we conducted an additional study by ablating each operation of the multi-focal text processing operators respectively, in https://files.catbox.moe/965rm7.png.\nThe results indicate that all three operations in our method contribute to performance improvement, though the functions of different modules exhibit a certain degree of overlap in some dataset. Combining with the ablation study shown in Table 2 of paper, each operation plays a significant role in our method.\nWe'll revise the paper accordingly.\n\n> **Suggestion 2**: Better show qualitative results or case studies.\n\nThanks! We show some case studies of different methods in https://files.catbox.moe/1910op.png and will add this in revision. They demonstrate that our VerbalTS better aligns with the semantic information, capturing nuanced details more accurately.\n\n> **Q1**: The ablation study shows even w/o $A^s$, $A^t$ and $A^d$, the generation still outperform baselines. Why?\n\nThanks for point.\nAs discussed in Fig. 1, Secs. 1 & 4.2 and Finding 1 in Sec. 5.4, this improvement comes from more flexbly expressed textual conditions with nuanced time series details, highlighting the overlooked advantage of text-based conditional time series generation, as most studies focus on structured conditions like attributes or class labels, e.g., [1,2].\nBesides, our method further enhances performance (as in Tabs. 1-2, Finding 2, Fig. 4), showing potential improvements via better leveraging unstructured text in generation, indicating a promising direction for multi-modal time series research.\n\n> **Q2.1**: Will the generation from text give more information than other conditions?\n\nYes, *unstructured* textual conditions may provide flexibly expressed and nuanced details that *structured* conditions cannot capture (discussed in Sec. 1).\nYet, we believe the comparison is fair, as almost all the existing works rely on structured conditions. To our knowledge, this is the first study highlighting the impact of unstructured conditions on time series generation (Finding 1) and formally addressing this problem, as also recognized by Reviewer Gv7f. Our approach also well handles the noise in text (Finding 2) with enhanced multi-modal semantic alignment (Findings 3–5).\n\n> **Q2.2**: If given enough textual descriptions that comes from the data itself, can it perfectly reconstruct the data? Should the paper compare with textual conditioned generation methods, if any?\n\nIt's a great point! However, as empirically observed, *perfectly* reconstructing a time series from texts remains challenging, as it's nearly impossible to describe time series perfectly. Yet, it's feasible to model the distributional alignment between text and time series modalities, which worth further exploration. \n\nTo our knowledge, our work is the first on generating time series from *unstructured* texts, few works have attempted to this direction. Thus, we only compared with *structured* information based (e.g., metadata or attribute) conditional generation methods.\n\n> **Q3**: In Tab. 1, ETTm1 and Traffic are multivariate. Why show only univariate setting?\n\nThis setting depends on dataset's textual conditioning property. For variate-specific textual descriptions (e.g., variate-specific text-augmented datasets ETTm1 and Traffic), we treat each variate independently to assess granular generation performance, following [1,2]. Contrastively, for texts describing multiple variates collectively (e.g., real-world datasets Weather and BlindWays), the multivariate time series is treated as a whole sample.\n\n> **Q4**: More practical textual time-series pairs like recently TimeMMD and ChatTime?\n\nThanks! Beside the used 4 real-world datasets, more diverse datasets could further strengthen our experiments.\nLimited by rebuttal time, we conducted experiments on the Environment dataset in TimeMMD, its largest subset. The results (https://files.catbox.moe/uf3vs2.png) align with our original conclusion.\nOur focus is on fine-grained time series generation, those containing dataset-level descriptions like that in ChatTime are omitted. \nWe'll add these to revise paper and continue incorporating more time series datasets.\n\n[1] Time Weaver: A Conditional Time Series Generation Model. ICML 2024.\n\n[2] Towards editing time series. NeurIPS 2024.", "claims_and_evidence": "Generally fine.\n\nOnly concern is the evaluation size to support the claims in the \"Extended Analysis\"", "methods_and_evaluation_criteria": "Generally fine.\n\nPlease see the concerns in the experimental design and question sections.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Checked all experiments; generally fine.\n\nFor the \"Extend Analysis\", the conclusion may not be solid enough, as the total evaluation datasets are not big, and these findings are based on the subset of the used datasets (e.g., Syn-M or Weather), which may not be strong enough to support the claim.\n\nMore concern please check question section,", "relation_to_broader_scientific_literature": "This work in time-series generation shows effectiveness in multiple scientific domains, as used in the evaluation.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "Minor suggestions:\n\n1. The ablation study could better use w/o $A^s$,  w/o $A^t$, and  w/o $A^d$, to understand the each factors contribute to the improvements.\n\n2. It is better to have some qualitative results or case studies that compare the generation difference between VerbalTS and baselines to understand the strength of the proposed method better.", "questions_for_authors": "1. The ablation study results show that even w/o $A^s$, $A^t$, and $A^d$, the generation results still outperform the baselines, e.g., on the Weather dataset. Could the author explain why this phenomenon, as it looks like the improvements are not exactly from the proposed model but the textual information.\n\n2. Based on my first equation, I would want to understand whether the evaluations are fair to the baselines. Will the generation from text give more information than other conditions, particularly worried given that the textual information itself is generated from the original time series. Does that mean if you give specific enough textual descriptions of the data that comes from the data itself, it can somehow perfectly reconstruct the data? And should the paper compare with textual conditioned generation methods, if any?\n\n3. In Table 1 results, ETTm1 and Traffic are known as multivariate time series datasets. Why do the evaluations on them show only univariate settings? It would be better to show both univariate and multivariate results on all four real-world datasets to show that the results are not cherry-picked.\n\n4. Instead of using the generated text of ETT and etc., why not consider using more practical textual time-series pairs, such as doctor notes and medical time series, or several recent multi-modal time-series datasets (TimeMMD, ChatTime, etc.)", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rHdOwLoGDZ", "reviewer": "Reviewer_Gv7f", "summary": "The paper introduces VERBALTS, a novel diffusion‐based framework that generates time series from unstructured textual descriptions rather than structured metadata. The authors argue that traditional time series synthesis is limited by the reliance on structured, expert‐annotated conditions and that text provides richer, sample‐specific control. VERBALTS leverages a multi‐focal text processor and multi‐view noise estimator to capture the hierarchical and multiresolution aspects of both text and time series. The approach is evaluated on two synthetic and (~four) real‐world datasets.\n\n## Update after Rebuttal \nI thank the authors again. The authors have mostly addressed my concerns. Unfortunately, the data pipeline is not included in the code, which is one of the key contributions of this work.\n\n> While directly adopting vision models is feasible like PatchTST in forecasting, later works [1,2] indicate large improvement space. To our knowledge, our VerbalTS is the first study on text-to-ts generation. We believe further exploration beyond adopting vision models could lead to significant advancements on this novel task, and we hope our work may shed some light on it\n\nThis response mostly satisfies my question. However, I’m not sure if it is truly the first study on text-to-time-series generation—especially given the substantial body of research focused on generating trajectories for human avatars (the trajectories/time series of the joints). While I did not mention this in the first round, upon further reflection, I believe it’s important to acknowledge this line of work. The authors should consider including this in an updated version of the paper.\n\nTevet, Guy, et al. \"Human motion diffusion model.\" ICLR 2023.\n\nShafir, Yonatan, et al. \"Human motion diffusion as a generative prior.\" ICLR 2024.\n\nI will maintain my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your recognition and valuable suggestions!\n\n> **Claims**: The paper's method contradicts the claim that can't simply use image diffusion‐based methods. The reviewer also think simpler adaptations might be feasible for text-to-ts (time series) generation.\n\nWe apologize for any ambiguity. Our statement in Sec. 1—\"generation framework based on diffusion models (Saharia et al., 2022) for text-to-ts generation\"—refers to the general diffusion framework (Sec. 3.2), not a specific vision model. Our method introduces many innovations upon vanilla diffusion model, including multi-view noise estimator, multi-focal text processor and multi-modality semantic alignment (Sec. 4) to address challenges in text-to-ts generation. We'll clarify this in the revised paper.\n\nWhile directly adopting vision models is feasible like PatchTST in forecasting, later works [1,2] indicate large improvement space. To our knowledge, our VerbalTS is the first study on text-to-ts generation. We believe further exploration beyond adopting vision models could lead to significant advancements on this novel task, and we hope our work may shed some light on it.\n\n[1] iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. ICLR 2024.\n\n[2] ElasTST: Towards Robust Varied-Horizon Forecasting with Elastic Time-Series Transformer. NeurIPS 2024.\n\n> **Evaluation**: More evidence of the CTTP's reliability are needed.\n\nThanks for suggestion! We recognize the importance of metric reliability. Following contrastive learning practices [1], we evaluate CTTP model using a retrieval-based protocol. For each time series in a random batch (B=32), we compute top-1 accuracy of retrieving its paired text from B candidates. Results (https://files.catbox.moe/zf1a6s.png) show effective semantic alignment ablity of CTTP model. We’ll include this in the revised paper.\n\n[1] Learning Transferable Visual Models From Natural Language Supervision. ICML 2021.\n\n> **Reference**: Missing related works: TS2Vec, TimeMixer, PromptCAST and AutoTimes.\n\nThanks for comments! \n+ TS2Vec and TimeMixer++ only capture multi-scale patterns in time series, while our work tackles that in both text and time series modalities with cross-modal interactions. \n+ Unlike LLM-based models (e.g., PromptCAST, AutoTimes) that use coarse-grained prompts to assist forecasting and QA, we leverages sample-specific text for fine-grained generation. \n\nWe’ll include these works in the revised paper.\n\n> **W1**: Fig. 2 is confusing.\n\nWe apologize for any confusion. To distinguish the three views and the information within them, we use different colors for variables, and color intensities for temporal resolutions. We'll follow other works like [1], simplify color scheme, clarify data flow for better readability.\n\n[1] Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting. ICLR 2023.\n\n> **W2**: The datasets and code are not provided.\n\nPlease check this link for codes: https://anonymous.4open.science/r/VerbalTS-E5BC.\n\nAs is stated in Sec. 5 of our paper, we plan to release all the datasets and models upon the acceptance of the paper.\n\n> **Suggestion**: Unified and clearly explained color scheme should be considered, e.g., Fig. 1.\n\nThanks! \nFor Fig. 1, since we highlighted two different advantages of text in time series generation, we used two colors in it.\nWe will refine Figs. 1 and 3-6 by unifying the legend style, marker sizes, and other visual elements to improve readability and consistency.\n\n\n> **Q1**: Any strategies for adapting VerbalTS to real-time or large-scale applications?\n\nIt's a great point! The auto-regressive generation of reverse diffusion process is an efficiency challenge in practice. To address this, we have utilized DDIM [1] using deterministic non-Markov diffusion process with significant acceleration.\n\nFurther improvements on diffusion efficiency include compressing the reverse steps via knowledge distillation [2] or performing diffusion in the latent space [3]. We plan to explore these to enhance our method for practice.\n\n[1] Denoising Diffusion Implicit Models. ICLR 2021.\n\n[2] Progressive Distillation for Fast Sampling of Diffusion Models. ICLR 2022.\n\n[3] High-Resolution Image Synthesis With Latent Diffusion Models. CVPR 2022.\n\n> **Q2**: How sensitive is VerbalTS to the specific parameters? Any situation where the performance greatly decreased?\n\nThanks for comment. We added sensitivity study on Synth-M dataset to evaluate the impact of the multi-resolution number $R$ and diffusion stage number $S$, with results shown in https://files.catbox.moe/n6v94k.png.\nIt suggests that, beyond certain threshold, the benefits of multi-resolution and multi-stage modeling become apparent. We plan to extend this analysis to more datasets and revise the paper accordingly.\n\n> **Q3**: More evidence for proposed CTTP metric.\n\nSee response to Evaluation.\n\n> **Q4**: Would you mind sharing the code?\n\nNot at all! See response to W2.", "claims_and_evidence": "The entire paper is based on the claim that we cannot simply use image diffusion‐based methods such as Song et al. (2021) because \"time series data exhibit multivariate characteristics and complex temporal dependencies (Torres et al., 2021; Wu et al., 2021), which fundamentally differ from the spatial structures typically encountered in image generation tasks.\" While I agree to a certain extent, later methods like PatchTST (Nie et al. ICLR 2023) have shown that one can essentially assume variate independence for time series‐based problems and utilize an off‐the‐shelf (vision) transformer; thus, simpler adaptations might be feasible. In addition, the authors use a text‐to‐image technique (Saharia et al. NIPS 2022) as a basis for their architecture, which partially contradicts their argument. If the authors persue further with this claim they need stronger evidence.\n\nOtherwise, i think the remaining claims and research questions are properly backed and linked with evidence. The quantitative performance is measured in FID, J-FTSD, and CTTP, along with ablation studies that underscore the benefits of their framework.", "methods_and_evaluation_criteria": "The proposed CTTP metric is an interesting approach for measuring the semantic alignment between generated time series and the input text.  Creating a new metric isn’t scientifically hard as long as the terms are clear that what’s being measured and you have external experts of domain aside from authors validating you. \nWhile the definition of the metric is clear, the paper does not provide sufficient external validation or experiments to ensure that CTTP truly captures the intended properties. More insight into its calibration would strengthen the evaluation.\n\nThe authors use 2 entirely synthetic and 2 real world and 2 augmented real-world datasets, which support their findings.", "theoretical_claims": "The authors used a custom tailored Element-wise Multiplication And Addition operation which is further discussed in Appendix B.3", "experimental_designs_or_analyses": "I think they experiments are in general sound and are valid, however as I mentioned before the experiments are partially based on CTTP which needs in my opinion additional evidence (at least in the appendix).", "relation_to_broader_scientific_literature": "They build upon generating time series in order to address data scarcity for certain situations\n(Narasimhan et al., 2024, Jing et al., 2024a). The architecture it self is based on an text2image method (Saharia et al. NIPS 2022).", "essential_references_not_discussed": "The authors missed relevant time series modeling work that utilizes hierarchical/multiresolution techniques in their method—specifically, TS2Vec (Yue et al., AAAI 2021) and TimeMixer++ (Wang et al., ICLR 2025). Furthermore, they overlooked recent multimodal work based on LLMs, such as PromptCAST (Xue et al., TKDE 2023) and AutoTimes (Liu et al., NIPS 2024). Including these references would provide a more complete picture.", "other_strengths_and_weaknesses": "## Strengths:\n- S1. The integration of unstructured text for fine-grained control is novel for time series generation, offering a exciting approach to bypass the limitations of structured metadata.\n- S2. The paper is well written, and the research questions are clearly stated and effectively linked to the experimental findings, which helps guide the reader through the contributions.\n- S3. The augmented datasets and the detailed data pipeline presented in the work are valuable contributions for the time series community, as they address real-world data scarcity issues and offer new benchmarks for evaluation.\n\n## Weaknesses:\n- W1. Figure 2 is confusing, the different colors and color mixtures as well as the spatial ordering is very tough to get through, or at the very least, not adequately explained. This makes it challenging to quickly grasp the key insights when skimming through the paper.\n- W2. Although the augmented datasets and data pipeline are significant contributions, a major drawback is that neither the model code nor the modified datasets are publicly shared. This limits reproducibility and hinders further research based on the work.", "other_comments_or_suggestions": "This is not really an issue, but rather a suggestion:\nThere is an inconsistency in the color scheme across different plots, which is distracting and hampers the overall readability. A unified and clearly explained color scheme would greatly improve the visual presentation.\nE.g. why is in Figure 1 the titles Blue as well as the time series, why look figure 3-6 look all different.", "questions_for_authors": "1. Given the slower reverse diffusion process, what strategies do you have in mind for adapting VERBALTS to real-time or large-scale applications?\n2. How sensitive is the performance of VERBALTS to the specific parameter choices, especially under noisy text conditions? Did you observe any situation where the performance greatly decreased?\n3. Could you provide more evidence for your proposed CTTP Metric?\n4. Would you mind sharing the code?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a new task of generating time series data from unstructured text, and introduces VERBALTS, a method that combines a multi-view time series noise estimator with a multi-focus text processor.

The paper received four reviews, initially consisting of three weak accepts and one weak reject. The authors submitted a rebuttal, and after the discussion, all reviewers converged on a weak accept.

As the AC, I have read the paper, the reviews, and the rebuttal. I find that the paper presents an interesting and novel task and offers a reasonable solution. I agree with the emerging consensus among the reviewers and recommend acceptance.
Decision: Accept (poster)