LAST SToP for Modeling Asynchronous Time Series
Authors: Shubham Gupta
Thibaut Durand
Graham W. Taylor
Lilian Bialokozowicz
Venue: ICML 2025 poster (2025)
Keywords: Large Language Models
Asynchronous Time Series
Time Series modeling
Deep Learning
Primary area: deep_learning->sequential_models_time_series
TL;DR: Using Large Language Models (LLMs) to model Asynchronous Time Series

Abstract: We present a novel prompt design for Large Language Models (LLMs) tailored to **Asynchronous Time Series**. Unlike regular time series, which assume values at evenly spaced time points, asynchronous time series consist of timestamped events occurring at irregular intervals, each described in natural language. Our approach effectively utilizes the rich natural language of event descriptions, allowing LLMs to benefit from their broad world knowledge for reasoning across different domains and tasks. This allows us to extend the scope of asynchronous time series analysis beyond forecasting to include tasks like anomaly detection and data imputation. We further introduce **Stochastic Soft Prompting**, a novel prompt-tuning mechanism that significantly improves model performance, outperforming existing finetuning methods such as QLORA. Through extensive experiments on real-world datasets, we demonstrate that our approach achieves state-of-the-art performance across different tasks and datasets.

LAST SToP For Modeling Asynchronous Time Series
Shubham Gupta 123* Thibaut Durand Graham W. Taylor* Lilian W. Bialokozowiez
Abstract
Unlike regular time series, which consist of values evenly spaced time intervals (like weather measurements), asyn- chronous time series consist of multiple types of discrete events occurring sporadically over time. For example, in the context of social media platforms like X (Twitter), user interactions (likes, comments, shares, and follows) happen sporadically and at irregular intervals (Zhao al., 2015). Each such type of interaction with a user's profile represents event type, and together with their timestamps, form an asynchronous time series (Xue et al., 2024). Modeling such asynchronous time series is challenging due to the irregu- lar timing and the diversity of event types, which contrasts with the uniformity and regularity of traditional time series data (Schirmer et al., 2022; Horn et al., 2020; Zhang et al.). Traditionally, to model asynchronous time series, events are grouped into a fixed, small number of categorical types (Xue et al., 2024). Separate stochastic processes- as Pois- son processes or Hawkes processes- are then modeled for each event type to predict which event will occur next and when (Mei et al., 2022; Hawkes, 1971). However, this ap- proach presents several significant drawbacks. Firstly, it inherently limits research to datasets with small number of event types because modeling each event type separately be- comes increasingly computationally intensive as the number of event types grows (Zuo et al., 2020). Secondly, events can vary widely and may not fit neatly into predefined categories. Thirdly, this method leads to the loss of meaningful natural language descriptions associated with the events. Fourthly, these methods treat each event type independently, ignor- ing any interactions between them - for example, likes and shares of a tweet are not independent events. Lastly, extending these methods to other tasks require significant theoretical development (Shchur et al., 2021).
We present novel prompt design for Large Lan- guage Models (LLMs) tailored to Asynchronous Time Series. Unlike regular time series, which assume values at evenly spaced time points, asyn- chronous time series consist of timestamped events occurring at irregular intervals, each de- scribed in natural language. Our approach effec- tively utilizes the rich natural language of event descriptions, allowing LLMs benefit from their broad world knowledge for reasoning across dif- ferent domains and tasks. This allows us to extend the scope of asynchronous time series analysis be- yond forecasting to include tasks like anomaly detection and data imputation. We further intro- duce Stochastic Soft Prompting, a novel prompt- tuning mechanism that significantly improves model performance, outperforming existing fine- tuning methods such as QLoRA. Through ex- tensive experiments on real-world datasets, we demonstrate that our approach achieves state-of- the-art performance across different tasks and datasets.
Introduction
An asynchronous time series (also named temporal event sequence or contimuous-lime event sequence) is a temporally ordered set of events that describe the progression of actions or occurrences. Asynchronous time series are ubiquitous in daily life. such as healthcare (Lorch al., 2018; Rizoiu et al., 2018), finance (Bacry al., 2015; Jin et al., 2020), e-commerce (Hernandez et al., 2017), and social media (Zhang et al.. 2022; Kong et al., 2023). In each of those domains, predicting the next events plays a crucial role.
Deep learning models have significantly revolutionized tech- niques for time series modeling, and even more so with the introduction of transformers (Vaswani et al., 2017). How- ever, there are often limitations due to the scarcity of training data, overfitting in specific domains, and the highly special- ized architectural designs. In response those challenges, Large Language Models (LLMs) have emerged as power- ful and promising direction to model time series data. For example, Gruver et al. (2023); Zhou et al. (2023); Xue & Salim (2023); Jin et al. (2024) have illustrated how LLMs can be used as time series forecasters when the input time series is encoded as a string of numeric digits, by casting the
"Work while the author was intern at RBC Borcalis 'Mila- Quebec AI Institute, Canada Universitéé Laval, Canada *RBC Borealis "Vector Institute, Canada 'Electronic Arts. Cor- respondence to: Shubham Gupta <shgupl @ uulavallca>>, Thibaut Durand <thibaut, durand @ borealisai. com>> Lilian W. Bialoko- zowiez witiiinniilloooowwize @ gmail.com>.
Proceedings of the 4nnd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s). Code available at http:///gthuhccco/oorraaassAiii

---

LAST SToP For Modeling Asynchronous Time Series
time series forecasting problem as next-token prediction
We introduce LASTS (Language-modeled Asyn- chronous Time Series), a novel framework that lever- ages Large Language Models (LLMs) to model asyn- chronous time series data, while effectively handling datasets with a large number of event types without the need for predefined categorical groupings. To the best of our knowledge, this the first work to explore the capabilities of LLMs to process textual asynchronous time series data across multiple tasks such as forecast- ing, anomaly detection, and data imputation.
in text, hence unlocking the use of powerful pre-trained models. LLMs have also been explored in other domains like action forecasting from videos (Zhao et al., 2024; Wang et al., 2024). However, these approaches focus on regular time series with evenly spaced numerical observations and cannot be directly applied to asynchronous time series due to their irregular intervals and diverse event types described in natural language. While LLMs have recently been explored for action recognition and action forecasting from videos (Zhao et al., 2024; Wang et al., 2024), applying I.L.s to textual asynchronous time series over multiple tasks (like anomaly detection and imputation) remains largely unex- plored.
We introduce Stochastic Soft Prompting (StoP) which is an novel prompt-tuning mechanism that serves as gurameter-efficien method to adapt LLMs to asyn- chronous time series data. StoP learns soft prompts that significantly improve model performance and gener- alizability by randomly truncating the prompts during training to learn more diverse representations.
"(guard, 0), (pass, 2.3), (dribble, 0.5)"
"(pass, 5.6)*
LLM
LASTS
We conduct comprehensive evaluations on real- world datasets across multiple tasks demonstrate the effectiveness of our proposed method. Our approach achieves sfate-of-the- art performance, outperforming existing methods, and highlights the potential of LLM- based models to effectively process and analyze asyn- chronous time series data.
"(guard. 0). (pass, 2.3). (shoot, 4.2). (pass, 5.6)"
"(shoat, 4.2)"
LLM
LASTS
"(guard, 0), (pass, 2.3), [MASK], (pass, 5.6)"
"(dribble, 0.5y"
LLM
LASTS
2 Related Work
Figure 1: We show that our LASTS framework can solve the following tasks asynchronous time series data: (a)Forecasting: (top) The model is given sequence of events, encoded as text, with the goal of predicting the next event. (b) Ab)mny detection: (middle) The model is given a sequence of events containing an incorrect event (bold) with the goal of finding the incorrect event. Impputttno::: (bottom) The model is given a sequence of events containing masked event, encoded as text, with the goal of predicting the masked event.
Temporal Point Processes (TPPs). TPPs (Hawkes, 1971; Daley & Vere-Jones, 2007) have emerged as the standard method to model asynchronous time series data. Over the last decade, a large number of neural temporal point pro- cesses have been proposed to capture complex dynamics of stochastic processes in time by using neural networks. Du et al. (2016); Mei & Eisner (2017) proposed to use models based on Recurrent Neural Networks (RNNs) to model the sequence of events. Then, more advanced models (Mchrasa et al., 2019; Lüdke et al., 2023) were proposed to better model uncertainty when predicting the future. Recently, sev- eral neural TPP models incorporate Transformers in order to improve performance by using attention to better model long-term dependencies. These include the Self-attentive Hawkes process (SAHP) (Zhang et al., 2020), Transformer Hawkes process (THP) (Zuo et al., 2020), and Attentive Neural Hawkes Process (Att-HHP) (Mei et al., 2022).
This paper presents LASTS (Language-modekle Asynchronous Time Series), novel prompting-based framework to adapt LLMs to asynchronous time series data while keeping the backbone model intact. To the best of our knowledge, this is the first work to explore the capabilities of LLMS to process textual asynchronous time series data and works on multiple tasks as shown in Figure 1. Our framework overcomes the drawbacks presented by traditional approaches for modeling asynchronous time series- it can handle datasets with numerous event types easily, it does not need to group events into predefined categorical bundles, it retains and utilizes the natural language descriptions of event types, and is able to leverage the rich interactions between different event types. Our contributions can be summarized as follows:
Transformers for Time Series. Transformers (Vaswani et al., 2017) have become popular to model regularly- sampled time series because of their ability to capture long- range dependencies and to extract semantic correlations among the elements of a long sequence. Informer (Zhou et al., 2021) introduced a novel self-attention architecture to reduce the quadratic complexity of the original self attention. Autoformer (Wu et al., 2021) used a novel decomposition
2

---

LAST SToP For Modeling Asynchronous Time Series
architecture with an auto-correlation mechanism to identify more reliable temporal patterns. Crossformer (Zhang & Yan, 2023) proposed novel architecture to model both the cross- time and cross disss-iinnioonn dependencies multivariate time series forecasting. PatchTST (Nie et al., 2023) tokenizes the time series in patches, and proposes a thaanee--ndeeendent patch time series Transformer to improve the long-term forecasting accuracy.
ner. LLMTime (Gruver al., 2023) showed that LLMs can zero-shot extrapolate time series if the numerical values of the time series are well represented. LLM Processes (Re- queima et al., 2024) explores various prompt configurations for using L.MSs for time series forecasting condiitoned on a textual context. We refer the reader to (Zhang et al., 2024) for a more detailed survey on the topic.
Due to space limitations, we only review some popular models and invite the reader to (Wen et al., 2023; Zeng et al., 2023) for a more complete literature reviews of Transformer models for regularly-samplle time series. Most of the time series Transformer models are designed for specific tasks, and cannot be easily extended to asynchronous time series data or other tasks like anomaly detection or imputation.
Vision Models for Time Series. Several works started to explore the use of FMs pretrained on images because of the better intrinsic similarities between images and time series such as trend, stationarity, seasonality/periidiittt, and sudden change. Zhou et al. (2023) tried to fine-tune BEiT (Bao et al., 2022) trained on images for time series forecasting, but it falls short of the leading text-based and time series-based FMs. Recently. VisionTS (Chen et al., 2024) proposes to use a vision Transformer pretrained on ImageNet to reduce the cross-domain gap or in-domain heterogeneity between time series and text.
Foundation Models (FMs) for Time Series. FMs (Bom- masani et al., 2021) are a family of deep models that are pre- trained on vast amounts of data, and have caused a paradigm shift due to their unprecedented capabilities for zero-shot and few-shot generalization. FMs have revolutionized nat- ural language processing (Brown et al., 2020; BigScience Workshop et al., 2023; Wu et al., 2024; Dubey et al., 2024) and computer vision (Radford et al., 2021; Kirillov al., 2023). The availability of large-scale time series datasets has opened the door to pretrain a large model on time series data. ForecastPFN (Dooley et al., 2024) proposed the first zero-shot forecasting method trained purely on synthetic data. Llag-Ilama (Rasul et al., 2023) introduced a univari- ate probabilistic forecasting model that was pretrained on large corpus of diverse time series data. TimeFM (Das et al., 2024) pretrained a decoder style attention model with input patching, using a large time series corpus comprising both real-world and synthetic datasets. Chronos (Ansari et al., 2024) introduced a framework for pretraining on tokenized time series data, achieving state-of-the-art zero-shot fore- casting performance and simplifying forecasting workflows. MOIRAI (Woo et al., 2024) is an enhanced Transformer architecture pretrained in the Large-scale Open Time Se- ries Archive that achieves competitive performance as a zero-shot forecaster.
Parameter Efficient Fine Tuning (PEFT). PEFT (Man- grulkar et al., 2022) is a paradigm adapt pretrained LLMs to various domains without fine-tuning all of model's pa- rameters, which can be costly and require large amounts of training data. LoRA (Hu et al., 2021) methods freeze the pretrained model weights and injects trainable rank de- composition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable pa- rameters for downstream tasks. QLoRA (Dettmers et al., 2023) advances finetuming by significantly reducing memory usage while preserving task performance.
Soft Prompt Tuning. Soft prompts have emerged compute efficient method for adapting pretrained L.MSs to new domains without altering their core architectures. Brown et al. (2020) were among the first to demonstrate the power of prompting for task adaption of pretrained lan- guage models, but automatically finding suitable sets of text prompts remains an open challenge. Li & Liang (2021); Qin & Eisner (2021) proposed the prefix tuning technique that preprends few task specific soft tokens to the input and hidden states of each Transformer layer. During train- ing, the parameters of soft prompts are updated by gradi- ent descent while the model parameters keep frozen. Liu et al. (2021) showed the prefix tuning technique could be effectively applied to natural language understanding with different scales of models. Lester al. (2021) simplified the prefix tuning technique such that it only adds soft prompts to the input layer and is now considered the standard soft prompt-tuning.
LLMs for Time Series LLMs pretrained on large amounts of text data have emerged as a promising direc- tion to model time series data. GPT4TS (Zhou et al., 2023), LLM4TS (Chang et al., 2023), and TEMPO (Cao et al., 2023) fine-tuned a pretrained GPT2 (Radford et al., 2019) on some time series downstream tasks to capture intrinsic dynamic properties. TimeLLM (Jin al., 2024) proposed reprogramming framework to repurpose LLMs for general time series forecasting with the backbone language models kept intact. PromptCast (Xue& & Salim, 2023) introduced a new prompt- based forecasting paradigm, where the numeri- cal input and output are transformed into prompts and the forecasting task framed in a -oreencee---entencee man-
3 Background
Notations. We observe n events over a fixed time interval (0, T), with each event being denoted as te, where e E
3

---

LAST SToP For Modeling Asynchronous Time Series
is the event type (or attributes) and & represents the space of event types. An asynchronous time series is sequence of events Iim 41), (e2,t2) (o),,,.).............. (en. (n) where & is an increasing sequence in [0. that does not necessarily observe any periodicity. A common alternative to the event time i is the inter-arrival time := f 1--1; they are considered isomorphic and often used interchangeably. In our work there is very little constraint on & and in princi- ple, our model still works even if & is infinite. We only need to be able to compute vectorial representation of the event type/attributes, which is achieved through the LLM's learned input embeddings in our work.
LASTS prompt
System Beader
7ask desoription
Oser
Tine sexies input
Reader
Beader
Figure 2: Components of LASTS prompt: A concise task description is included in the system prompt, while asynchronous time series is provided as input in the user prompt.
Language modeling, Language modeling is a widely used task to train LLMs where the goal is predicting the next word or character in a document. Language models are de- signed work on a sequence of tokens, where each token belongs to vocabulary. A tokenizer transforms the input text data into a sequence of tokens. The tokenization process is important and can impact performance significantly, for it directly influences how patterns form within tokenized sequences and the types of operations that language models can learn.
inter-arrival time, which is the time elapsed between event -;-1 and Z4.
In prior work on modeling asynchronous time series (Du et al., 2016; Mehrasa al., 2019; Zhang et al., 2020; Mei et al., 2022), events are typically reduced to categories from small set of options. In contrast, we retain the event types e, natural language descriptions. We introduce LASTS, which specifies how to input an asynchronous time series as part of a prompt to effectively leverage LLMs for various tasks on such data.
Tasks. We propose new approach to model asyn- chronous time series with LLMs, which solves three differ- ent tasks (Figure 1). Forecasting (also known as next event prediction): Given history of events 71:m from an asyn- chronous time series, the model is tasked with predicting the next event m1+ Data imputation: One of the events zj of the series is randomly chosen and masked, and the model is tasked with filling in the gap. Anomaly detection: One event 2, of the series is randomly chosen and its event type ej is replaced randomly by another event type e'. The model must identify the out-of-place element without knowing the position of the replaced element.
LASTS Prompt Structure The LASTS prompt consists of three parts that can be mapped to the system-user user- assistant structure when using an instruction fine-tuned L.M (see Figure 2). The system prompt introduces what an asyn- chronous time series is, provides description of the task to be performed, and includes details about the underlying dataset. The user prompt represents the input series as a comma-separated sequence of tuples (e,, 44), where e; is the textual description of the event type and f, is the anter-arrival time. The assistant prompt contains the correct event if performing LLM adaptation training, or is left to be gen- erated by the LLM during inference. More details about the exact prompts used in our experiments can be found in Appendix A.3.
To find the right recipe for the model to solve these tasks, we innovated in two major directions: first, we studied various representations of the asynchronous time series as inputs to LLMs (Section 4.1) for zero shot completion of these tasks; and secondly, we study different parameter efficient techniques to adapt an LLM backbone for working with asynchronous time series, while leveraging its knowledge of the world and its understanding of natural language (Section 4.2).
4.2 Parameter Efficient LLM Adaptation with LASTS representation
Having established a representation of asynchronous time series for use with LLMs via LASTS, we further enhance the model's adaptability to various tasks using three different adaptation techniques:
4 Proposed Method
Low Rank Adaption LoRA is a family of low-rank adap- tation techniques that reduce the number of trainable pa- rameters by learning low-rank updates to selected model weights, allowing for efficient fine-tuning of large models. We adapt the LLM backbone for our tasks by applying low- rank adaptations using the LASTS representation as inputs to encode both the task and the input asynchronous time series.
4.1 LASTS- Prompting LLMs with Asynchronous Time Series data
Unlike ordinary time series, often represented as sequences of numerical values (Gruver et al., 2023), asynchronous time series are represented as sequences of events I; = (e,, t,), where e, is the event type, and t; is a representation of the timestamp of this event. Normally, 1, is expressed as an
4

---

LAST SToP For Modeling Asynchronous Time Series
Soft Prompt training
Stochastic Soft Prompt training
P[]:50]
LASTS
e[1:40]
LASTS
batches
2
P12:501
LASTS
P11:BB]
2/ LLSTS
P [1]500]
WALTSS
P[1:32]
LASTS
Figure 3: Comparison of Soft Prompt (SP) and Stochastic Soft Prompt (StoP) training. For illustration, the soft prompt P is of length 50. In SP, the entire prompt is used during both training and inference. In StoP, random prefix of P is used per training batch, while the full prompt is used for inference. Fire marks the soft prompt, which is the trainable prompt portion, while snowflake represents the frozen LASTS text prompt.
Soft Prompting (SP) SP involves prepending a continu- ous prompt the LASTS representation, which is trained through gradients from next token prediction loss. This guides the model towards task-specific behavior without altering the model weights directly. (See Figure 3)
4.3 Experimental setup
Datasets. We perform experiments two different sets of datasets: three text-based action datasets and five stan- dard temporal point process datasets. The main difference is that actions are represented by words in the action datasets, whereas they are represented by indices in temporal point process datasets. The text-based action datasets are built from the action annotations of activity videos. Breakfast (Kuehne et al., 2014) contains 1712 videos with 177 action classes related to breakfast preparation. Each video has a sequence of events to prepare breakfast, with each event con- taining the timestamp and the action. EPIC-KITCHENS-IO0 (Damen et al., 2022) is a large-scale dataset in egocentric vision capturing daily activities in the kitchen over multiple days with total of 100 hours of recording. It presents more complex activity than the Breakfast dataset, with rich anno- tations of sequences of actions comprising 97 verb classes and 300 noun classes, with 20k unique narrations. Multi- THUMOS (Yeung et al., 2018) contains 400 videos with 65 action classes related to human activities. Each vidco has a sequence of human activity events, with each event containing the timestamp and the activity. For the temporal point process datasets, we use the five benchmarks intro- duced in (Xue et al., 2024): Amazon (Ni et al., 2019) where the goal is to predict the timestamp and category (among 16 categories) of the next reviewed product, Retweer (Zhou et al., 2013) where the goal is to predict the timestamp and category (among 3 categories) of the next user to retweet post, Taxi (Whong. 2014) where the goal to predict the timestamp and category (among 10 categories) of the next pick-up or drop-off of taxi driver, Taobao (Xue et al., 2022) where the goal is to predict the timestamp and cate- gory (among 20 categories) of the item clicked by a user, and StackOverflow' where the goal is to predict the times- tamp and category (among 22 categories) of the next badges assigned to a given user. We follow the same data prepro- cessing in (Xue al., 2024). For each of these datasets,
Stochastic Soft Prompting (StoP) We propose Stochas- tic Soft Prompts- an enhancement of SP which learns more robust prompts by imposing noarse-to-fing structure on the prompt tokens. (See subsection 4.5). Similar to SP, we prepend a continuous prompt to the LASTS representa- tion which is trained through gradients from next-token prediction loss. However, in SP, the entire soft prompt P of length L is used during training, while in StoP, we randomly select prefix of the prompt P for each train- ing batch. Specifically, for each batch, we sample a prefix length 1 from probability distribution p()), where s L. The soft prompt used for that batch is then represented by Phatch P(: with 1 ~ p()). In our experiments, we use uniform distribution as Both the forward pass and the backward pass are conducted using only the selected prefix P Prach.- During inference, we use the entire learned soft prompt of length T: Pinference = P(1 T.] See Figure 3 for more details. Our approach is inspired by techniques like dropout (Srivastava et al., 2014) and stochastic depth (Huang et al., 2016), as well as audio models like Sound- Stream (Zeghidour et al., 2021), where randomly selecting the first k codebooks during training enables better gener- alization. Similarly, we draw inspiration from Matryoshka Representations (Kusupati et al., 2022), which learn repre- sentations such that predefined prefix lengths remain valid representations.
These adaptation techniques enable an LLM backbone to handle a variety of asynchronous time series tasks, including forecasting, imputation, and anomaly detection, while main- taining parameter efficiency. Details on the exact prompt representation are provided in Appendix A.5.
hhttps: Pettps://snap.deee at ford, edu/data/
5

---

LAST SToP For Modeling Asynchronous Time Series
the semantic meaning of the event type is unknown, and only the index of the event type is available. We use the index of the event type as input to our model.
2023) and LLMPPrcceses (Requeima et al., 2024), baselines for zero-shot LASTS prompting asyn- chronous time series (Table 1, Figure 4).
Metries. Due to the bi-modal nature of the asynchronous time series, we report separate metrics for the event type and time. We report the Macro-F1 (M-F1) (Yang. 1999) for event type prediction as Macro-Fl is better suited for multi-class classification tasks with skewed class distribu- tions (Appendix A.2) than accuracy because Macro-Fl gives equal importance to all the classes. We also report accuracy numbers in Appendix A.13. We report either the Mean Ab- solute Error (MAF) or Root Mean Square Error (RMSE) for time prediction.
TPP models: We compare our model with state-of- the-art TPP models for asynchronous time series (Xue et al., 2024). We report the results for two popular RNN-based models: Recurrent marked temporal point process (RMTPP) (Du et al., 2016) and neural Hawkes Process (NHP) (Mei & Fisner, 2017). We also com- pare with three attention-based models: self-attentive Hawkes process (SAHP) (Zhang et al., 2020), Trans- former Hawkes process (THP) (Zuo et al., 2020), at- tentive neural Hawkes process (AtNHP) (Yang et al., 2022) (Table 2).
Implementation details We use Birma----B-Innttuuee (Dubey et al., 2024) as our LLM backbone. For zero-shot experiments, we disable sampling during response genera- tion, ensuring deterministic outputs. For LLM adaptation experiments, we use QLoRA as the low rank adaptation al- gorithm, Adam as the optimizer, and a constant learning rate 24 for QLoRA and 1 for prompt tuning. Following (Xue et al., 2024), we split our datasets into a train/valida- tion/test ratio of 70/10/20. Both SP and StoP training are conducted for the same number of epochs. We employ early stopping based on the Macro-F1 on the validation set. We report performance on the test set.
Results Our results on the the three tasks (forecast, im- putation, anomaly detection) and the three text datasets (Breakfast, MultiTHUMOS, EPIC-KITCHENS) sented in Table 1. Based on our results, we make five main observations. Firstly, LASTS proves to be an effective and robust representation for asynchronous time series data across multiple datasets. LASTS Zero Shot consistently outperforms the Time Series Foundation Model Chronos and LLM- methods (LLMTTmme and LLM Processes) in most evaluations, highlighting the advantage of using textual event descriptions enabled by LASTS. Secondly, our results demonstrate that the LASTS representation can be applied across multiple tasks without any investment needed in designing custom models for each task. Thirdly, LASTS work effectively with multiple LLM adaptation techniques without algorithm- specific modifications. Fourthly, we ob- serve that StoP an adaptation technique outperforms other techniques for most time prediction evaluations, and in all event type prediction evaluations. Finally, we highlight our results on the PITCHEIS dataset, which features very rich textual event descriptions (approximately 20,000). While traditional TPP modeling methods struggle to handle such a large set of classes, our approach effectively models various tasks on this complex dataset.
pre-
We use prompt length of 400 for prompt tuning in both SP and StoP experiments. This value was selected through hyperparameter tuning across all datasets and tasks, striking a balance between model capacity, performance, and the compute resources available to us. Given that Llama-3-8B- Instruct has a hidden dimension of 4096, this configuration results in approximately 1.6M trainable parameters, which corresponds to only 0.02% of the LLM parameters. For QLoRA, we use rank of 4, resulting in comparable number of trainable parameters (1.7M).
4.4 Experiment Results
Baselines We evaluate our methods using four sets of baselines. See Appendix A.6 for details.
We establish random baseline simulating random guesses to evaluate our methods on the three text-based datasets and tasks (Table 1, Figure 4).
Random baseline:
Foundation models for time series: We use state-of- the-art pretrained foundation model for time series fore- casting, Chronos (Ansari et al., 2024), baseline for forecasting and imputation tasks on asynchronous time series (Table 1).
Figure 4: Macro-F1 t. MAE 4, and Accuracy 1, averaged across all datasets for Forecast and Imputation for Zero Shot methods.
Comparison with TPP models. Table 2 shows experi- mental results that compare our model with existing TPP models on standard TPP datasets. TPP models are designed
LLM for time series: We adapt two LLM-based time series forecasting methods, LLMTTime (Gruver et al.,
6

---

LAST SToP For Modeling Asynchronous Time Series
Table I: Performance evaluation on three textual datasets for forecasting, imputation. and anomaly detection. Met- rics: Macro F1 (M-FI) and Mean Absolute Error (MAE) where applicable. Best results are bold, second-best are underlined. For anomaly detection, MAE is inapplicable, and Chronos/LLMProcesss are non-adaptable (see A. A indicates our method. Fhw-shot results use five examples (see A.11).
eling of the time distribution. It also shows that our model is performing well even when only the index of the event type is provided instead of its textual description, making it more generally applicable method (See Appendix A.6).
Comparison with Zero Shot Methods Figure 4 shows LASTS Zero Shot outperforms other zero shot techniques over all metrics when averaged over all tasks and datasets. See Appendix A.6 for details.
Foreeasting
Comparison with PEFT Techniques. As detailed in Ap- pendix A.9, Stochastic Soft Prompting provides a significant advantage, achieving an average Macro-F1 improvement of 12.69% over vanilla Soft Prompting and 13.55% over QLoRA across all tasks and datasets.
Model
Breakfast
MOliTHCMOS
EPIC-KITCHENS
M-F1 1
MAE
M-F1
MAE
M-FI
MAE 1
Rundom
0.0162
40.1513
0.0417
1.8803
0.0000
3,2001
Chronos
0.0011
43.0502
0.0265
1.9805
0.0000
3.5925
LLMTimm
0.0240
37,3902
0.1280
2.2060
0.0040
4.8948
L.MPPrcesses
0.0337
44.9856
0.1278
200471
0.0049
4.3843
LASTS Zero Shot"
0.0604
38.1630
0.1361
18868
0.0105
3.1366
LASTS Few Shor"
0.1518
35,5605
0.1676
1.8114
0.0144
3.3092
LASTS + QLORA*
0.2558
33.9737
0.3218
1.7281
0.0764
2 8964
LASTS SP"
0.23-11
32.8117
0,2707
16630
0.0750
LASTS + Stop"
0.2633
32,5464
0.3944
1,6003
0.0797
3,0318
Imputation
Model
Breakfast
HUMITHHMMS
RPIC-KITCHENS
M-FI
MAE
M.F1
MAE
M-F1 1
MAE 4
Random
0.0168
37,7029
0.0435
2.3622
0,0000
3,4269
Chronos
0.0013
38,1039
0.0294
2.3971
0.0000
3.6955
LLMTime
0.0137
35.9899
0.0968
2,6998
0,0005
3,6750
LLMProcesses
0.0156
34,7117
0.1123
2.5786
0.0008
4,2600
Adjecent Taken Cosine Smillarity
LASTS Zero Shot"'
0.0263
33,0097
0.0915
2.6696
0,0015
3.6527
LASTS Few Shot"
0.0520
33,3440
0.1013
2.3982
0.0023
3,2528
LASTS QLORA*
0.1688
28.5638
0,2132
2.2179
0.0378
3.1194
LASTS + SP
D.1581
28,8503
0.2014
2092
0.9423
3,1456
LASTS StoP"
0.2064
28.2251
0,2213
2.3445
0,0610
3,1116
Anomaly Delection
Model
Breakfast
MoliTHUMOS
EPIC-KITCHENS
M.FI f
MAE
M-FI MAE
M.F1
MAE І
Random
0.0349
0:0381
0.0238
L.L.Time
0.0240
0.0415
0.0048
LASTS Zero Shot"
0.0923
0,2755
0.0159
LASTS Few Shot"
0.0837
0.3535
0.0337
Token Index
LASTS + QLORA*
0.7011
0.6003
0.6520
LASTS + SP"
0.6520
0.5231
0.6159
Figure 5: Learned token representations of StoP and SP. The first two plots show t-SNE projections of 100 tokens from 400-length prompts (Breakfast dataset, forecasting) StoP tokens are more dispersed, while SP tokens cluster closely. The third plot shows lower adjacent token cosine similarity for StoP (blue) than SP (red), indicating greater diversity.
LASTS + Stop"
0.7198
0.6045
0.6603
for forecasting so we only show the results for the forecast- ing task. We observe that our model has competitive results w.r.t. TPP models, outperforming cxisting TPP models on 13 of the 18 evaluations, and is in the top-2 best models on 17 of the 18 evaluations. Our model has the best perfor- mance for all the event type evaluations, which shows that our model is more accurate to predict the next event type. On three of the eight datasets, our model is less accurate than TPP models to predict the time. We think that our model is not performing as well as the TPP models, because our model does not have an explicit prior about the time distribution whereas TPP models (e.g. Poisson process or Hawkes process) make strong assumptions about the time distribution. In the case of the Amazon dataset, the perfor- mance gap is more pronounced because this dataset groups a large number of diverse event types into single event cat- egory, making it harder to model anterl times. These results show that our model is able to outperform existing TPP models on most of the datasets without explicit mod-
4.5 Model analysis
Comparison of SP and StoP learned token represen- tations. Stochastic Soft Prompt (StoP) and Soft Prompt (SP) learn distinct token distributions due to differences in training. Figure 5 shows t-SNE projections of the first 100 tokens from 400-length prompts. We observe that the tokens learned through StoP training are more spread out, indicating greater diversity, while those learned through SP training tend to cluster more closely. StoP follows a coarse- to-fine approach, with early embeddings that are diverse and cover larger space. This difference is further highlighted in the last plot of Figure 5, where StoP tokens have lower adjacent cosine similarity than SP. As result, StoP outper- forms SP even when using only the first few tokens, with further improvements as more tokens are utilized (Figure 6).
7

---

LAST SToP For Modeling Asynchronous Time Series
Table 2: Performance of models on next-event's 's type and type prediction across five real datasets. The best result is shown in bold, and the second best result is underlined. OOM indicates an Out Of Memory error. A missing entry indicates the MuliTHUUOS (65 classes), Breakfast
model diverged. We tried optimizing these baselines for the three textual 117 classes), and EPIC- CITCCIIT (~ 20K classes) but these models either diverged, performed poorly, or ran out of
memory due the large number of classes.
Model
Retweet
Taxi
Tuohuo
StccKverllow
Breakfast
MoliTHUMMS
EPIC-KIICHENS
M-PL
RMSE
M-FI
RMSE 4
M-P1
RMSE L
M-FL
RMSE
M-FF
RMSE J M-PL
RMSE
M-FI
RMSE
M-P1
RMSE J
RMTPP
0.0088
0.4780
0.3110
16.5849
0.2969
0.3761
0..4955
0.1338
0.0277
1.3727
00M
GOM
NIP
0.1266
0.3667
0.3995
0.4287
0.0167
116.23
0.2861
SAMP
0.0846
0.5491
0.2772
16.6451
0.2780
0.3193
0.1816
0.1347
0.0322
L33266
0.0023
112.85
0.0
4,5908
00M
0OM
THP
0,1414
0.4911
0.2114
16.6440
0.3451
0.3736
0.2734
0.1340
0.0661
1.4054
00M
0OM
AUNHIP
0.1270
0.7054
0,4210
16.8278
0.2167
0.4072
0.1048
0.1350
0.0475
1.3661
0.0178
108.41
0.0809
5,2113
00M
00M
LASTS StoP
0.1520
0,0000
0,4299
16.4981
0.4174
0.3278
0.4633
0.1321
0
1.2596
0.2633
102.02
0.3947
3.6722
0.0797
7,3724
0.20
crucial role in StoP's performance gains (Appendix A.7).
x
Training speed. Another dimension on which to com- pare SP and StoP is the training speed. Due to differences in training paradigms, StoP trains significantly faster than SP for the same prompt length, as many training batches use only subset of the full prompt in StoP. In our experi- ments with 400 soft prompts, we observed that StoP trains approximately 25% faster than SP.
macro F]
0.15
StoP F1
0.10
SP F1
0.05
0.00
80
50
100
200
360
400
Prefix Length
Understanding StoP prompts through probing. While prior work such as (Lester et al., 2021) attempts to inter- pret learned prompts by mapping them to the closest input embeddings-offe often yielding incoherent results- we instead explore probing the LLM using the learned prompt. By ap- pending the learned prompt with simple instruction, such as "Tell me in as much detail possible what task you are supposed to do," we encourage the LLM to generate an out- put that reflects its understanding of the task. This approach allows us to gain some insight into what the model has sum- marized from the tasks and datasets it has been trained on. We present multiple model responses when probed like this in Appendix A.8.
45000
x
42500
40000
37500
X StoP MAE
35000
X SP MAE
32500
30000
27500
100 50
100
200
300
400
Prefix Length
Figure 6: StoP-trained prefixes function as standalone prompts, unlike SP. Testing 400-length prompts (Breakfast, imputation) shows StoP prefixes remain effective, while SP prefixes do not.
Average Macro 11 Armss Debasets Tasks
Average MAT Acrows Datanets and Tasts 12.00
32
All prefixes are valid prompts in StoP The training paradigm of StoP forces all prefixes of StoP to act as valid standalone prompts, as they are used as prompts during training for some batches (if trained for long enough). (see Figure 6). This further strengthens our belief that tokens in StoP are arranged from coarse, independent tokens at the be- ginning to tokens with tokens containing finer information towards the end. See Appendix A. for further discussion.
Disentangling Stochasticity and Prefix Picking in StoP. To highlight the impact of structured prefix picking in StoP, we compare it with an alternative approach where, instead of selecting a prefix, we randomly select tokens from the prompt per batch, with I drawn from a uniform distribution. We find that stochasticity alone is insufficient for learning effective soft prompts, and structured prefix picking plays a
Figure 7: Macro-F1 1 and MAE across all datasets and tasks for different model sizes.
Scaling Laws. We evaluate Stochastic Soft Prompts (StoP) across different LLM backbone sizes (1B, 3B, and 8B) and observe consistent performance gains with larger models, indicating that StoP benefits from improvements
8

---

LAST SToP For Modeling Asynchronous Time Series
in the underlying LLMs and is expected to scale accord- ingly (Figure 7, Appendix A.10).
BigScience Workshop et al. BLOOM: A Parameere Open-Access Multilingual Language Model. In arXiv 2211.05100, 2023.
Conclusion and Future Work
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosse- lut, A., Brunskill, E., et al. On the opportunities and risks of foundation models. arXiv 2108.07258, 2021.
We presented a novel approach to modeling asynchronous time series with an LLM, introducing a flexible alternative to traditional TPP methods. By encoding an asynchronous time series in prompt, our approach enables LLMs lever- age their world knowledge for various downstream tasks, including forecasting, anomaly detection, and imputation.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert- Messs, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J.. Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Additionally, we proposed Stochastic Soft Prompt (StoP), an efficient PEFT technique for adapting LLMs to asyn- chronous time series data. This approach not only im- proves adaptability but also suggests broader applicability to other data modalities such as image or natural language sequences.
Our findings highlight the potential of I.LM-based represen- tations for asynchronous time series and suggest new direc- tions for future research, including refining LLM adaptation strategies and exploring hybrid approaches that combine neural architectures with prompt-based modeling.
Cao, D., Jia, F, Arik, S. O., Pfister, T., Zheng, Y., Ye, W., and Liu, Y. TEMPO: Prompt-based Generative Pre- trained Transformer for Time Series Forecasting, In arXiv 23/0. 2310.04948, 2023.
Impact Statement
Chang. C., Wang, W.-Y., Peng, W.-C., and Chen, T.-F. LLM4TS: Aligning Pre-Traines LLMs Data-Efficient Time-Series Forecasters. In arXiv 2308 08469, 2023.
This work presents novel approach for applying Large Lan- guage Mods) to asynchronous time series analysis. Our LASTS framework democratizes time series analysis by leveraging natural language processing capabilities of LLMs, making sophisticated temporal modeling accessi- ble to practitioners without specialized domain expertise. The paramette--fffcciet Stochastic Soft Prompting technique reduces computational barriers, supporting broader adop- tion in resource-constraing environments. However, biases present in the underlying LLM could propagate to tempo- ral predictions, potentially affecting decision-making We encourage practitioners implement appropriate privacy safeguards, obtain necessary ethical approvals, and validate model outputs in high-stakes applications.
Chen, M., Shen, L., Li, Z., Wang, X. J., Sun, J., and Liu, C. VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters. In arXiv 2408. 17253, 2024.
Daley, D. and Vere-Jones, D. An Introduction to the The- ory of Point Processes: Volume II: General Theory and Structure. Probability and Its Applications. Springer New York, 2007. ISBN 9780387213378.
Damen, D., Doughty, H., Farinella, G. M., Furnari, A., Ma, J., Kazakos, E., Moltisanti, D., Munro, J., Per- rett, T., Price, W., and Wray, M. Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC- KITCHENS- International Journal of Computer Vision (IJCV), 2022.
References
Ansari, A. F., Stella, L., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Shchur, O., Rangapuram, S. S., Arango, S. P., Kapoor, S., et al. Chronos: Learning the language of time series. Transactions on Machine Learning Research 2024.
Das, A., Kong, W., Sen, R., and Zhou, Y. A decoder- only foundation model for time-series forecasting. In International Conference on Machine Learning (ICMI.), 2024.
Bacry, E., Mastromatteo, I., and Muzy. J.-F. Hawkes pro- cesses in finance. Market Microstructure and Liquidity, 2015.
Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: Efficient finetuning of quantized llms. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36. pp. 10088-10115 Cur- ran Associates, Inc., 2023.
Bao, H., Dong, L., Piao, S., and Wei, F. BETT: BERT pre- training of image transformers. International Conference on Learning Representations (ICIR), 2022.

---

LAST SToP For Modeling Asynchronous Time Series
Dooley, S., Khurana, S., Mohapatra, C., Naidu, S. V., and
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. A.C., Lo, W.-Y., et al. Segment anything. In IEEE International Conference on Computer Vision (ICCV), 2023.
White, C. ForecastPFN: Synthetically-trainee zero-shot forecasting, Advances in Neural Information Processing Systems (NeurIPS), 2024.
Du, N., Dai, H., Trivedi, R., Upadhyay, U., Gomez- Rodriguez, M., and Song, L. Recurrent marked temporal point processes: Embedding event history to vector. KDD '16, pp. 1555-1564, New York, NY, USA, 2016. Associa- tion for Computing Machinery. ISBN 9781450342322.
Kong, Q., Calderon, P., Ram, R., Boichak, O., and Rizoiu, M.-A. Interval-censored transformer Hawkes: Detecting information operations using the reaction of social sys- tems. In Proceedings of the ACM Web Conference 2023, 2023.
Kuehne, H., Arslan, A., and Serre, T. The language of actions: Recovering the syntax and semantics of goal- directed human activities. In IEEE Conference on Com- puter Vision and Puttern Recognition (CVPR), 2014.
Dubey et al. The Llama 3 Herd of Models. In arXiv 2407.21783, 2024.
Gruver, N., Finzi, M., Qiu, S., and Wilson, A. G. Large language models are sero-shot time series forecasters. Advances in Neural Information Processing Systems (NeurlPS), 2023.
Kusupati, A., Bhatt, G., Rege, A., Wallingford, M., Sinha, A., Ramanujan, V., Snyder,-Sydeer, W., Chen, K., Kakade, S., Jain, P., et al. Matryoshka representation learning. Advances in Neural Information Processing Systems, 35:30233-30249, 2022.
Gruver, N., Finzi, M.. Qiu, S., and Wilson, A. G. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36, 2024.
Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning. arXiv 2104.0869) 2021.
Hawkes, A. G. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 1971.
Li, X. L. and Liang, P. Prefix-tuning: Optimizing continuous prompts for generation. arXiv 2101.00190, 2021.
Hernandez, S., Alvarez, P., Fabra, J., and Ezpeleta, J. Analy- sis of users' behavior in structured e-commerce websites. IEEE Access, 2017.
Liu, X., Ji, K., Fu, Y., Tam, W. L.,, Du, Z., Yang, Z., and Tang, J. P-tuning v2: Prompt tuning can be comparable fine-tuning universally across scales and tasks. arXiv 21 10.07602, 2021.
Horn, M., Moor, M., Bock, C., Rieck, B., and Borgwardt, K. Set functions for time series. In International Conference on Machine Learning, pp. 4353-4363 PMLR, 2020.
Lorch, I.,, De, A., Bhatt, S., Trouleau, W., Upadhyay, U., and Gomez Rodrigue, M. Stochastic optimal con- trol of epidemic processes in networks. arXiv preprint arXiv: /8/0: 13043, 2018
Hu, E. J.. Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models, 2021. URL https: ///:///aa /arxiv. org/abs/ 210 0 6. 09685.
Lüdke, D., Biloš, M., Shehur, O., Lienen, M., and Günnemann, S. Add and thin: Diffusion for temporal point processes. In Oh, A., Naumann, T., Globerson, A., Saenko, K.. Hardt, M., and Levine, (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 56784-56801. Curran Associates, Inc., 2023.
Huang, G., Sun, Y., Liu, Z., Sedra, D.,, and Weinberger, K. Q. Deep networks with stochastic depth. In Computer Vision- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October l 1-14, 2016, Proceedings, Part IV 14, pp. 646-661. Springer, 2016.
Mangrulkar, S., Gugger, S., Debut, L.., Belkada, Y., Paul, S., and Bossan, B. Peft: State-of-the-an parameter- efficient fine-tuning methods. https:///ithhb com/ homm/huginngfaace setttfaaccpee.....ee 2022.
Jin, M., Wang, S. Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y. P. Liang, Y., Li, Y.-F., Pan, S., and Wen, Q. Time-LLM: Time series forecasting by reprogramming large language models. In International Conference on Learning Representations (ICLR), 2024.
Mehrasa, N., Iyothi, A. A., Durand, T., He, J., Sigal, I.., and Mori, G. variational auto-encoder model for stochastic point processes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Jin, Z., Guo, S., Chen, N., Weiskopf, D., Gotz, D., and Cao, N. Visual causality analysis of event sequence data.
IEEE transactions on visualization and computer graph- ics, 2020.
Mci, H. and Eisner, J. M. The neural hawkes process: A neurally eeff-modulatin multivariate point process. In
10

---

LAST SToP For Modeling Asynchronous Time Series
Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fer- gus, R., Vishwanathan, S., and Garnett, R. (eds.), Ad- vances in Neural Information Processing Systems, vol- ume 30. Curran Associates, Inc., 2017.
Shchur, O., Turkmen, A. C., Januschowski, T., Gasthaus, J., and Günnemann, S. Detecting anomalous event se- quences with temporal point processes. Advances in Neu- ral Information Processing Systems, 14:13419-13431, 2021.
Mei, H., Yang, C.,, and Eisner, J. Transformer Embeddings of Inregularly Spaced Events and Their Participants. In International Conference Learning Representations (ICLR), 2022.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-195: 2014.
Ni, J., Li, J., and McAuley, J. Justifying recommenda- tions using distantly-labeled reviews and fine-grained aspects. In Proceedings of the conference on empirical methods in natural language processing and the interna- tional joint conference on natural language processing SMMNLP-JJCLLP 2019.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is All you Need. Advances in Neural Information Processing Systems (NeurIPS), 2017.
Wang, Y., Yang, Y., and Ren, M. Lifelongmemory: Lever- aging IIms for answering queries long-form egocentric videos. arXiv preprint arXiv:2312.05269 2024.
Nie, Y., H. Nguyen, N., Sinthong, P., and Kalagnanam, J. A Time Series is Worth 64 Words: Long-term Forecast- ing with Transformers. International Conference on Learning Representations (ICLR), 2023.
Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., and Sun, L. Transformers in time series: survey. Interna- tional Joint Conference on Artificial Intelligenc((ICAAI), 2023.
Qin, G. and Eisner, J. Learning how to ask: Querying LMs with mixtures soft prompts. Association for Computational Linguisties. 2021.
Whong, C. Nye taxi open data, 2014. https://chriswhhh com/ /opnoooe---- data/ hoil_nye_taxi/
URL
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners, 2019.
Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D. Unified training of universal time series fore- casting transformers. In arXiv 2402.02592 2024.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021.
Wu, H., Xu, J., Wang. J., and Long, M. Autoformer: Decom- position transformers with cuto-corrllation for long-term series forecasting, Advances in Neural Information Pro- cessing Systems (NeurIPS), 2021.
Rasul, K., Ashok, A., Williams, A. R., Ghonia, H., Bhag- watkar, R., Khorasani, A., Bayazi, M. J. D., Adamopou- los, G., Riachi, R., Hassen, N., Bilo%, M., Garg, S., Schneider, A., Chapados, N., Drouin, A., Zantedeschi, V., Nevmyvaka, Y., and Rish, I. Lag-Llama: Towards Foun- dation Models for Probabilistic Time Series Forecasting. In arXiv 2310.08278, 2023.
Wu, Y., Sun, Z., Li, S., Welleck, S., and Yang, Y. An Empir- ical Analysis of Compute-Optimal Inference for Problem- Solving with Language Models, arXiv 2408.00724 2024.
Xu, Z., Liu, Z., Chen, B., Zhong, S., Tang, Jue, W., Zhou, K., Hu, X., and Shrivastava, A. Soft prompt recovers compressed Ilms, transferably. In Forty-first International Conference on Machine Learning.
Requeima, J., Bronskill, J. F., Choi, D., Turner, R. E., and Duvenaud, D. Llm processes: Numerical predictive distri- butions conditioned on natural language. In ICML 2024 Workshop on In- Context Learning, 2024.
Xue, H. and Salim, F. D. Promptcast: A new prompt- based learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering, 2023.
Rizoiu, M.-A., Mishra, S., Kong, Q., Carman, M., and Xie, L. Sir-Hawkes: on the relationship between epidemic models and Hawkes point processes. The Web Confernce, 2018.
Xue, S., Shi, X., Zhang, J., and Mei, H. Hypro: A hybridly normalized probabilistic model for long-horizon predic- tion of event sequences. Advances in Neural Information Processing Systems (NeurIPS), 2022.
Schirmer, M.. Eltayeb. M., Lessmann, S., and Rudolph, M. Modeling irregular time series with continuous recurrent units. In International conference on machine learning, pp. 19388 1188--1400 19405. PMLR, 2022.
Xue, S., Shi, X., Chu, Z., Wang, Y., Zhou, F., Hao, H., Jiang, C., Pan, C., Xu, Y., Zhang. J. Y., et al. EasyTPP: To- wards Open Benchmarking the Temporal Point Processes.

---

LAST SToP For Modeling Asynchronous Time Series
International Conference on Learning Representations (ICLR), 2024.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Conference on Artificial Intelligence (AAAI), 2021.
Yang. X., Chen, A., PourNeeatian, N., Shin, H. C., Smith, K. E.,, Parisien, C., Compas, C., Martin, C.,, Costa, A. B., Flores, M. G., et al. A large language model for electronic health records. NPJ digital medicine, 2022.
Zhou, K., Zha, H., and Song, Learning triggering kernels for multi-dimensional Hawkes processes. In International Conference on Machine Learning (ICML), 2013.
Yang, Y. An evaluation of statistical approaches to text categorization. Information retrieval, 1999.
Zhou, T., Niu, P., Sun, I.,, Jin, R., et al. One fits all: Power general time series analysis by pretrained Im. Advances in Neural Information Processing Systems (NeurIPS), 2023.
Yeung, S., Russakovsky, O., Jin, N., Andriluka, M., Mori, G., and Fei-Fei, L. Every moment counts: Dense detailed labeling of actions in complex videos. IEEE International Conference on Computer Vision 2018.
Zuo, S., Jiang. H., Li, Z.,, Zhao, T., and Zha, H. Transformer Hawkes process. In International Conference on Muchine Learning (ICML), 2020.
Zeghidour, N., Luebs, A., Omran, A., Skoglund, J., and Tagliasacchi, M. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495-507, 2021.
Zeng, A., Chen, M. Zhang. I.,, and Xu, Q. Are transformers effective for time series forecasting? In Conference on Artificial Intelligence (AAAI), 2023.
Zhang, Q. Lipani, A., Kirnap, O., and Yilmaz, E. Self- attentive Hawkes process. In International Conference on Machine Learning (ICML), 2020.
Zhang, W., Yin, C., Liu, H., Zhou, X., and Xiong, H. Irreg- ular multivariate time series forecasting: A transformable patching graph neural networks approach. In Forty-first International Conference on Machine Learning.
Zhang. X., Chowdhury, R. R., Gupta, R. K., and Shang, J. Large language models for time series: survey, arXiv preprint XXxx:2402.008800 2024.
Zhang, Y. and Yan, J. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representations (ICLR), 2023.
Zhang, Y., Cao, D., and Liu, Y. Counterfactual neural temporal point process for estimating causal influence of misinformation social media. Advances in Neural Information Processing Systems (NeurIPS), 2022.
Zhao. Q., Erdogdu, M. A., He, H. Y., Rajaraman, A., and Leskovec, J. Seismic: A self-exciting point process model for predicting tweet popularity. In Proceedings of the 2/th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1513-1522, 2015.
Zhao, Q. Wang, S., Zhang, C., Fu, C., Do, M. Q., Agarwal, N., Lee, K., and Sun, C. Antgpt: Can large language models help long-term action anticipation from videos? In The Twelfth International Conference on Learning Representations, 2024.
12

---

LAST SToP For Modeling Asynchronous Time Series
Appendix
A.1 Dataset Preparation
We remove any sequence in the dataset that is very small (< elements). We split the dataset random 70/10/20 train, validation and test split. Each sequence is expanded into multiple sequences based on the task:
Forecasting: We convert sequence into multiple prediction tasks. For each element of the series, the prediction task is to predict the element given the preceding elements. We impose a minimum and maximum length requirements on the number of preceding elements used.
Imputation: For every element in the series, we replace the element by a mask, and the imputation task is predict the masked element given the remaining sequence.
Anomaly Detection: For every element in the sequence, we replace the action by a random different action. the anomaly detection task is identify the element of the sequence that has been tampered with.
For the three test based datasets Breakfast, MultiTHUMOS and KITCHES, the event types are already represented text. The remaining 5 datasets from the temporal point processes domain lack textual component, and the event types represented by integers. For these datasets, simply treat each integer event type string, allowing the LLM to data.
process it similarly to
A.2 Dataset Class Imbalance
We observe significant class imbalance in our datasets, as shown in Figure for the Breakfast and MultiTHUMOS datasets. This imbalance motivates our choice of Macro-F1 as the primary metric, as it treats all classes equally, unlike Accuracy, which is heavily influenced by the dominant class.
breakfast
multihumos
Normalized Count (Sum 11
4.5
Figure 8: Normalized event counts (y-axis) vs. event types sorted by count (x-axis) for two datasets Breakfast and MultiTHUMOS, showing significant class imbalance.
A.3 LASTS representation of Asynchronous time series for Zero Shot
Here we present the LASTS prompt structure for use with LLMs for various tasks. The structure of the LASTS prompts is shown in Figure 2.
System Prompt system prompt used for Forecasting is:
The system prompt is very similar across tasks, except for the task specific portions of the prompt. The
You are helpful assistant. Your task is complete an asynchronous time series. dataset description. Each series is given in the format (inter_arri anteerrarrvvlltime action action_name) name). This indicates that the action_name _amie started inter_arrival milliseconds after the start of the previous action or the beginning of time if it's the first action. The allowable actions are: volad. Given the first few elements of an asynchronous time series, your task is to provide the next action with its inter arrival time as (inter_an aarivall_iime.. time, action action_name). You generate all your response as a single python tuple. Be sure to provide only that one python tuple and nothing else.
13

---

LAST SToP For Modeling Asynchronous Time Series
The system prompt used for Imputation is:
You are a helpful assistant. Your task is to find missing value in an asynchronous time series. dataset deeecrpttdsoiiitioonn Each series is given in the format (inter arrival time, action name). This indicates that the action name started anter_arrivaltimng milliseconds after the start the previous action or the beginning of time it's the first action. The allowable actions are: valid vocab. One of the elements in the series would be missing, marked by the word 'MISSING'. Provide your answer as single python tuple suver_arrival_inng action _ammo) which is your estimate of the missing element of the series. Be sure to give me that one missing python tuple your response and nothing else.
The system prompt for Anomaly Detection is:
You are a helpful assistant. Your task is find an anomolous value in an asynchronous time series, datasel description. Each series is given in the format (inter arrival time, action name). This indicates that the action_name started inter_ aiteraaevvalliiees milliseconds after the start of the previous action the beginning of time if it's the first action. The allowable actions are: pocidd One of the elements in the series is an anomaly, and your task is to identify this element which doesn't 't belong in the series. Provide your answer as a single python tuple (inter arrival time, action name) which is an element from the series you think an anomaly. Just give me that one anomolous python tuple from the series as your answer and nothing else.
Here, paatase_.dscripttin is short one line description of the underlying dataset, for example: "The underlying dataset is derived from tagged human actions while cooking/preparing meals". Also, palid_vocab a comma separated list of allowable action descriptions, if we choose provide this list and if this list is small.
User Prompt
The user prompt in all three tasks a comma separated string of sequence events, for example
(1,000nn howp:/arrra..o.aa,/20o0o bowl),
In case of imputation, there would be a missing element marked by the word MISSING, like so:
(0,
bowl), (41000 eggcarton),
Assistant Prompt This is empty for zero-shot, as it is filled by the LLM as its prediction for the task on the given sequence.
A.4 Evaluating LLM Interaction with LASTS Components
We considered various variants of framing the LASTS prompt and present few interesting ones here, evaluated on Breakfast dataset.
Testing LLMs use of world knowledge We want to test whether LLMs can understand a prompt like LASTS and provide meaningful response the task on the sequence using their world knowledge. To this end, we study a variant where each event description is replaced by uniquely mapped gibberish 4-letter string. This unique mapping ensures that while any semantic meaning in the descriptions is removed, the structure of the time series remains intact. Table 3 shows that all tracked metrics degrade considerably in the scrambled names variant. This confirms that LLMs not only understand LASTS properly but also leverage their world knowledge to perform the specified tasks.
Sequence Representation We probe about the right representation for the time series events should they be represented as (e,, f₂) or e;). Our results in Table 4 show that its better to have time first, followed by the event description. This is what we adopt in LASTS.
Time Representation We investigate if simplifying the series representation would improve LLM performance. For the Breakfast dataset, we replace inter-arrival times with durations, since we hypothesize that most actions occur contiguously for this dataset. We hypothesize that durations may be easier for the LLM to model rather than inter arrival. From the results in Table 5, we observe that while we have a favourable impact on forecast, both imputation and anomaly detection suffer from this change. This suggests that while durations help with forecasting, more precise anter-arrival times are crucial for more involved tasks like imputation and anomaly detection.
14

---

LAST SToP For Modeling Asynchronous Time Series
Forecast
M-F1 1
% A
Acc
% A
MAE
% A
Zero Shot 0.0432
0.0866
37.8030
Scrambled Names
0.0140
1 -67.633%
0.0397
-54.13%
38.0742
1.722%
Imputation
Zero Shot 0.0248
0.0338
33.7669
Scrambled Names
0.0100
-59.73%
0.0224
-33.733%
40.4918
1 19.92%
Anomaly Detection
Zero Shot
0.0760
0.0650
NA
Scrambled Names
0.0619
-18.55%
0.0469
-27.88%
NA
Table 3: Comparing LASTS Zero Shot with the Scrambled Names variant across Forecast, Imputation, and Anomaly Detection tasks. Higher values are better for M-FI and Acc, while lower values are better for MAE. Red indicates negative impact, while green indicates favorable impact.
Forecast
M-F1 f
% A
Ace t
% A
MAE 4
% A
Time First (t,, e,) Event First (ei, t4)
0.0432
0,0866
37.8030
0,0409
J. 5.38%
0.0726
+ 16.07%
37.5344
0.71%
Imputation
Time First (fi, Cs) Event First (c),t))
0.0248
0.0338
33.7669 31.8194
0.0071
4 -71.30%
0.0150
4 -55.56%
-5.77%
Anomaly Detection
Time First (t,, ex)
0.0760
0.0650
NA
Event First (e,, 4.)
0.0858
t 12.94%
0.0619
-4.81%
NA
Table 4: Comparison two ways to express events in an asynchronous time series event first or time first across Forecast, Imputation, and Anomaly Detection tasks. Higher values are better for M-FI and Acc, while lower values are better for MAE, Red indicates negative impact, while green indicates favorable impact.
A.5 LASTS representation used for LLM Adaptation
For experients LLM adaptation, keep the LASTS representation very similar shot experiments:
System prompt this case a very concise description of just the task. skip any dataset description as we expect the model learn that during the fine tuning process.
User prompt represented as a comma separated sequence of tuples of event description and inter arrival times.
Assistant prompt contains the expected prediction.
The exact system prompt used for each of the tasks are as follows:
Forecasting: "Predict the next element of this asynchronous time series where each element is of the form (in- ter jer_arrival_jime action_name).
Imputation: "Predict the element marked 'MISSING' in this asynchronous time series where each element is of the form (inter arrival time, action name)."
Anomaly Detection: "One of the element in this asynchronous time series is anomalous, find this element. Each element of the series is of the form (inter arrival time, action name)."
A.6 Baselines
Random Baseline To evaluate our methods on the three text-based datasets and the three tasks, we establish a random baseline simulating random guesses. For forecasting and imputation, given an input asynchronous time series, the baseline predicts the anter-arrival time as the average of all inter-arrival times in the sequence and selects a random event type from the valid event descriptions. For anomaly detection, randomly labels an event from the series as anomalous (see Table 10).
15

---

LAST SToP For Modeling Asynchronous Time Series
Forecast
M-F1 1
% A
Acc 1
% A
MAE 4
% A
Zero Shot
0.0432
0.0866
37,8030
Durations
0.0600
1 38.84%
0.0953
10.12%
33.781
10.62%
Imputation
Zero Shot 0,0248
0.0338
33.7669
Durations 0,0140
-43.56%
0.0288
4 14.88%%
29.6881
4 -1.00%%
Anomaly Detection
Zero Shot
0.0760
0,0650
NA
Durations 0.0767
1 0.96%
0.0532
4 -18.20%
NA
Table 5: Comparison of LASTS Zero Shot with the variant using durations instead of inter-arrival times across Forecast, Imputation, and Anomaly Detection tasks. Higher values are better for M-FI and Acc, while lower values are better for MAE. Red indicates negative impact, while green indicates favorable impact.
Foundation Models for Time Series Baseline We adapted Chronos (Ansari et al., 2024), state-of-the-ar foundation model designed for zero-shot forecasting on time series data, a baseline for forecasting and imputation tasks on asynchronous time series datasets. We use the largest model version (earzoon/chhonoo-5--laarg available which contains 710M model parameters. Since Chronos exclusively handles numerical data, we converted our event descriptions into categorical representations. Each asynchronous time series of length n was transformed into a sequence of 2n integers, alternating between inter-arrival times and event categories.
For forecasting, the task was framed as predicting the next two elements in this sequence given the historical context. Adapting Chronos for imputation, however, required additional considerations since is inherently designed for forecasting. We reformulated the imputation task as forecasting problem: if the prefix leading up to the missing element is longer than the suffix following it, we treated imputation as forecasting the missing element using the prefix as context. Conversely, if the suffix is longer, we reversed the suffix and used as context to forecast the missing element. This approach ensures the longest possible context is utilized for predicting the missing value.
It is worth noting that adapting Chronos for anomaly detection is not straightforward, anomaly detection involves identifying single anomalous event within the series, which does not align with Chronos" forecasting capabilities. Consequently, Chronos is provided as baseline exclusively for forecasting and imputation tasks.
LLMs for Time Series Baselines We adapted two LLM-based methods for time series: LLMTime (Gruver et al., 2024) and I.LMProcesses (Requeima et al., 2024), as baselines. Since both methods are designed for numerical time series, we converted textual event descriptions into categorical representations.
LLMTime In this method, each data point represented as a pair: (inter-arrival event-categorical) We modified the default next-token prediction behavior of the model using simple task-specific prompts:
Forecasting: Predict the next time and event.
Imputation: Find the element marked as 'MISSING.'
Anomaly Detection: Find the anomalous time and event.
LLMProcesses This method uses in-context learning with (x, y) examples derived from sequence, treating the sequence as a real-valued function on 2D space as domain. In this setup, represents point in 2D space 12), where æ1 denotes the sequence position, and 22 indicates the output type: 0 for inter-arrival time and for event categorical. For a given sequence, we crafted two distinct prompts: one for predicting the event categorical and another for predicting the inter-arrival time, based on the corresponding value of z. We followed the recommended settings from the original paper for prompt construction.
However, anomaly detection does not align with this framework, as it involves identifying a single anomalous time point where the function output is everywhere except at the anomaly. This makes it unsuitable for predicting function values at unseen points based on prior observations. Consequently, we adapted this approach exclusively for forecasting and imputation tasks.
16

---

LAST SToP For Modeling Asynchronous Time Series
TPP Models Baselines We compare our best fine-tuned model configuration, LASTS + StoP, against current state- of-the-art methods for forecasting on asynchronous time series. These methods are adapted from the benchmark study in (Xue et al., 2024). The evaluation spans eight datasets, five of which-Amazon, Retweet, Taxi, Taobao, and StackOverflow contain event categoricals without textual descriptions and are regarded as standard benchmarks for asynchronous time series analysis.
We benchmark the TPP models covered in the EasyTPP benchmark (Xue et al., 2024) on the three textual datasets considered in our work: Breakfast, MultiTHUMOS, and EPIC KITCHEN. Since these datasets represent events text and TPP models are not equipped to handle text directly, we converted the event names into event categoricals to make them compatible with these models.
Observations We summarize our comparison of various baselines with LASTS Zero Shot in Figure 9. We observe that Chronos performs the weakest among the baselines, yet it remains competitive. This is expected as Chronos, while being much smaller model compared to LLMs, is highly specialized for time series forecasting, which enables it to achieve decent performance. LLMTime and LLMProcesses also perform competitively, especially on the MultiTHUMOS dataset. We attribute this to the noisy nature of the MuliTHUMOS dataset, which includes non-standard event names (e.g.. "OneHandedCatch"" Toamerr"""""" etc) and repetitive, less meaningful patterns (e.g., "GolfSwing, Wait, GolfSwing, Wait..."). These characteristics may help weettcategoorical--seedd models like LLMTime and L.MProcesses. However, on the textual descriptions of events provide significant advantage,
the other two as evident from the comfortable margin by which LASTS Zero Shot outperforms LLMTime and LLMProcesses across all tasks.
Breakfast and
Furthermore, observed that existing TPP-based models struggled with datasets containing large number of unique event types, often performing poorly, failing to converge, or encountering out-of- out-of-memory errors. This highlights the challenges these models face in handling the diversity and complexity of such datasets.
Average M-F1
Average MAE
Average ACC
6.244
16
5.039
15.641
0.05
15
14,475
0.08
5:083
0.04
14-
33778
0.267
0.06
02 0.53
0.02
3.000
0.04
12
3.0
0.02
0:211
Random
Checero
LUK Proosss
Zero Shot
Random
Chronoo
LLM Processes
Shott
Chessoo
LLM Prooestes
LLM
LASTS
Figure 9: Comparison of performance metrics: Macro-F1 (M-F1), Mean Absolute Error (MAE), and Accuracy (ACC),
averaged across all datasets for Forecast and Imputation tasks. Higher values for M-F1 and ACC indicate better performance, while a lower value of MAE is preferred. is evident that LASTS Zero Shot (our method) achieves the highest average M-FI and average ACC among all the baselines and also produces the lowest MAE.
A.7 Disentangling Stochasticity and Prefix Picking in StoP
To further analyze the impact of prefix picking in StoP, compare it with alternative training paradigm where, instead of selecting structured prefix, we randomly select / tokens from the prompt during each batch, with drawn from a uniform distribution. This comparison isolates the effects of introducing stochasticity alone versus the structured prefix picking employed by StoP. Figure 10 presents a comparison of Macro-F1 and MAE metrics on the validation set as both prompts are trained for 10 epochs. These plots show that stochasticity alone is not sufficient for learning good soft prompts, and structured prefix picking is a key component of the StoP training.
17

---

LAST SToP For Modeling Asynchronous Time Series
Macro
MAE
4820
Macro F1
4499
4220
155
Epoch
Epoch
Figure 10: Comparison of Macro-F1 and MAE for StoP random token selection during training, evaluated validation data after 10 epochs. Results show that random token selection (shown in red) fails to learn effective prompts, while StoP's structured prefix selection (shown in blue) achieves significantly better performance.
A.8 StoP Prompt Interpretations Through Model Probing
Prior work such as (Lester et al., 2021) attempts to interpret learned prompts by mapping them to the closest input embeddings. However, this approach often produces sequences that are difficult to understand or lack meaningful content. For example, (Xu et al.) observed that the closest words to the learned embeddings are mostly meaningless, several tokens are mapped to the same word, and the cosine similarity between the tokens and their closest word embeddings almost always falls below 0.16. Instead, explore probing the LLM using the learned prompt. By appending the learned prompt with simple instruction, such as "Tell me in much detail possible what task you supposed to do," we encourage the LLM to generate output that reflects its understanding of the task. This approach allows us to gain meaningful insights into what the model has summarized from the tasks and datasets it has been trained on. Here we present few interesting interpretations of StoP prompts obtained by model probing.
StoP of Length 400, Breakfast Dataset, Forecast
We obtain a very nice summarization of the breakfast dataset:
The task appears be related preparing snack meal by using the facilities provided in the kitchen. Specifically, I was asked to perform series of actions to accomplish this goal. Here's what I've observed:1. **Opening the cabiner*s; need to open the cabinet to access the necessary utensils, cookware, ingredients.2. **rabbing the ingredients I will need to pick up the specified ingredients, such as a bowl, a cutting board, or a can opener, to prepare the desired dish.3. **Slicing the ingredients** will cut the ingredients into the re- quired proportions, shapes, or sizes depending on the recipe being followed.4. **Prcheating the cooking surface**; If needed, I will turn on the stove, oven, or other cooking device the desired temperature start cooking the ingredients.55 ** **Cooking the ingredients will proceed cook the ingredients according the required cooking method, such as boil- ing, saut 'ingg frying, baking.6. w* Adding condiments seasonings**, If necessary. will add the required condiments, spices, or seasonings to the cooked ingredients to enhance their flavor.7. **Combining the ingredients**: Finally, will combine the cooked ingredi- ents in the desired manner, whether it's mixing, blending, or assembling the final dish.Pleas let me know I'm correct in my understanding or if there's anything else can do to help!
StoP of Length 5, Breakfast Dataset, Forecast We obtain very cooking adjacent prompt interpretations from even a small prompt of length trained for forecasting on Breakfast. For example:
18

---

LAST SToP For Modeling Asynchronous Time Series
Here is my main task: "Reach for a tablespoon and move the direction of a plate where already placed some eggs and knife. Slowly pour one egg into the plate, take the knife, and carefully cut the egg in halves. Transfer both halves away from the plate, and then earry the remaining plate towards sink. Turn the tap on, wash the plate, and then move away from the sink, holding the plate still under rumning water. Remove the plate from the sink, carry it to a countertop, and put it down. Finally, reach for a knife and move towards bowl containing knife, carry the knife, and move towards a counter where I already placed a cutting board, where there is set of utensils lying on it. Take the knife, push towards the board, and reach for a knife. Move away from the cutting board.
StoP length 400, MultiTHUMOS Dataset, Imputation The interpretation seems have mapped the idea of an asynchronous time series another domain all together, while still keeping elements like walk, run, stand, from the original dataset:
I'm an Al designed to generate text based on the sequence of events occurring in the events diagram. Specifically, Im trained to analyze the sequence of events and identify the corresponding actions or actions sequences in the diagram, and then use that information to generate a textual description of what is happening in the diagram. The main tasks I was designed to perform are:1. **Identify sequence of * analyze the sequence of events the diagram, identifying the time-stamped data points and the transitions between them.2. ** M*tcch events with actions I match the identified sequence of events with the corresponding actions actions sequences. For example, if see sequence of time-stamped data points indicating person sitting, standing, and then walking, would match those events to the action of "walking".3 .3. **Generate textual description**: Using the matched events and actions, then generate a textual description of what is happening in the diagram. For example, match the sequence "sitting, standing, walking" to the action "walking". would generate a text describing that sequence as "walking" Additionally, also make use of some contextual information, such as the diagrams caption, to disambiguate and refine my generated textual descriptions. The final textual output is a natural language description of the content of the diagram.
StoP of length 400, MuliTTHMMSS Dataset, Anomaly Detection The interpretations center around games, which is expected as MuliTHUMOS is gaming dataset. Here are two sampled interpretations:
The task is to simulate a volleyball game. The events include:1. Slices or spikes the ball with the palm of hand (VolleyballSpiking)) 2. Blocks the ball with the palm the hand (VolleyballBlookk... Sets the ball with the palm of the hand (VolleyballSet). Pounds slams the ball with the fist (VolleyballSpikingg. 5. Jumps reaches for the ball VolleyballJumminng) 6. Moves sideways slides the ground (VolleyballSpikeRum)) important to remember that volleyball spiking, blocking, setting, and jumping involves movement and body language, which can't be reflected directly in a sequential diagram
The task you are supposed do is sports broadcasting, and specifically, you are golf broadcaster. As golfer, you supposed to perform a live broadcast of the golf match, providing commentary and analysis the golfers participate in the match. The situation is simulated, there no external stimuli or real golfers, you have to act as both commentator and golfer, making mock commentary and mimicking golf swings, task requires you to stay in fixed position, usually behind microphone, and maintain a semi-serious, professional tone. Your task is to articulate the events on the course, such as drives, putts, and other golf shots, as if you are actually seeing them happen. You should also describe the terrain, distance, and obstacles faced by the golfer, and offer your expert analysis and opinions on their technique, strategy, and chances of scoring. Your commentary should be smooth, clear, and engaging, drawing the audience into the game and painting vivid picture of the action the course. Your performance should exude confidence and authority, as if you many matches before. The audience should feel like they are right there on the course with you, experiencing the excitement and suspense of the tournament.
experienced commentator who has watched
StoP of Length 400, Taobao Dataset, Forecast Note that this dataset is not textual, so all the soft prompt has seen the task description and a sequence of pairs of numbers.
19

---

LAST SToP For Modeling Asynchronous Time Series
Prompt: Describe your task. Model Response:
Im an Al assistant, my task is to generate sequence based on rule that I'v learned from given inputs (or "train" sequences). The rule is inferred by analyzing the pattern of increasing or decreasing values of the input sequence. Each sequence is either called ("silenced") or activated, and if theres no change in the sequence, the "silence" used, unless the input sequence calls the "silence" itself. The task involves predicting the next output in sequence. given some input sequences that establish pattern.
A.9 Comparison of LASTS StoP with other PEFT techniques
In this section, we compare the performance of LASTS StoP with other PEFT techniques listed in Table 10. Table 6 highlights the percentage improvements observed in various metrics when using Stochastic Soft Prompting compared to standard Soft Prompting. We observe a significant advantage of Stochastic Soft Prompting across all datasets and tasks, with an overall average increase of 12.69% in Macro-F1 across all tasks and datasets. Similarly, Table 7 demonstrates an average increase of 13.55% in Macro-F1 when using Stochastic Soft Prompting instead of finetuning techniques like QLORA.
Breakfast
MultiTHMMSS
ЕPIC.KITCHEN
Task
M-F1
MAE
ACC
M-F1
MAE
ACC
M-F1
MAE
ACC
Forecast
11. 11.09%
0.91%
4.87%
6.08%
0.77%
0.04%
2.13%
-4.91%
3.52%
Imputation
23, 23.40%
2.22%
17.37%
7.64%
2.76%
10.95%
30.66%
1.09%
10.81%
Anomaly Detection
10.40%
12.82%
15.56%
10.97%
7.21%
6.06%
Avg Gain (Per Task)
14.96%
1.56%
11.69%
9.76%
1.76%
7.32%
13.33%
-1.91%
6.80%
Avg Gain (All Tasks, All Datasets)
M-F1: 12.69%
MAE: 0.47%
ACC: 8.60%
Table 6: Comparison of LASTS+StoP with LAS The table shows the percentage improvement in each metric achieved by using Stochastic Soft Prompting compared to standard Soft Prompting. Significant gains are observed across all datasets and tasks with Stochastic Soft Prompts. On average, across all datasets and tasks, Macro F1 increases by 12.69%.
Breakfast
MultiHUUMSS
EPIC.KITCHEN
Task
M-F1
MAE
ACC
M-F1
MAE
ACC
M-F1
MAE
ACC
Forecast
2.93%
4.39%
3.11%
22.65%
4.71%
10.31%
4.32%
-447%
6.39%
Imputation
22.27%
1.20%
9.60%
3,80%
-5.40%
3.46%
61.38%
0.25%
25.24%
Anomaly Detection
2.67%
3.40%
0.70%
1.65%
1.27%
0.70%
Avg Gain (Per Task)
9.29%
2.79%
5.37%
9.05%
-0.34%
5.14%
22.32%
-2.1 1.11%
10.78%
Avg Gain (All Tasks, All Datasets)
M-F1: 13.55%
MAE: 0.11%
ACC: 7.10%
Table Comparison LASTS+ with LASTS+QLORA. The table shows the percentage improvement each metric achieved by using Stochastic Soft Prompting compared to finetuning via QLORA. Significant gains are observed across all datasets and tasks with Stochastic Soft Prompts. On average, across all datasets and tasks, Macro-F] increases by 13.55%.
20

---

LAST SToP For Modeling Asynchronous Time Series
Breakfast
MultiThumos
EPIC KITCHEN
Params
Macro F1
MAE +
Macro FI f
MAE +
Macro F1
MAE
Forecast
1B
0.2292
33,9309
0.3210
1.8013
0.0574
3.0859
3B
0.2526
33.2541
0.3694
1.7259
0.0708
3.0169
8B
0.2633
32.5464
0.3947
1.6503
0.0797
3.0318
Imputation
1B
0.0256
31.1075
0.0907
2.4256
0.0102
3.2571
3B
0.0966
31.1597
0.1329
2.3963
0.0280
3.1445
8B
0.2064
28.2251
0.2213
2.3445
0.0610
3.1116
Anomaly Detection
1B
0.0688
0.0954
0.0318
3B
0.5726
0.4777
0.5793
8B
0.7198
0.6045
0.6603
Table 8: Comparison of Macro-FI and MAE across the Breakfast, MultiThumos, and KITCCKIS datasets for forecasting, imputation, and anomaly detection as the number of model parameters varies. The results show that Macro-F1 consistently improves with increasing model size across all datasets and tasks. In most cases, MAE decreases as model size increases, confirming that larger models generally lead to better performance.
A.10 Scaling to different LLM backhone sizes
We trained Stochastic Soft Prompts (StoP) across different backbone sizes of large language models and observed consistent improvements performance as the model size increased. Specifically, we conducted experiments using LLama3.2 models with 1B and 3B parameters, as well the LLama3-8B Instruct model. These improvements were clear across the Breakfast, MultiThumos and EPIC KITCHENS datasets and applied to all tasks forecasting, imputation, and anomaly detection.
Notably, Table 8 and Figure 11 show that macro-Fl scores consistently improve with larger model sizes across all datasets and tasks. Additionally, Mean Absolute Error (MAE) decreased in most cases as the model size increased, further confirming that larger models help Stochastic Soft Prompts perform better by utilizing their enhanced representational power. The performance difference between model sizes is smaller for forecasting tasks since these align with the next-token prediction that LLMS are trained on. However, for harder tasks like imputation and anomaly detection, the improvements are much larger as model size increases.
Average Macro F1 Across Datasets and Tasks
Average MAE Across Datasets and Tasks
12.501
12:450
12
11.918
0.287
10
Average MAE
Average
0.103
18
3B
18
38
8B
Model Size
Model Size
Figure 11: Comparison of average Macro Fl and MAE across datasets and tasks different model sizes. The left histogram shows the average Macro scores, while the right histogram depicts the average MAE values. We see clear trend of improvement both metries model sizes increase.
21

---

LAST SToP For Modeling Asynchronous Time Series
Few-Shot (k)
Breakfast
MultiTHUMOS
EPIC-KITCHENS
M-F1 f
MAE
ACC
M-FI1
MAE
ACC
M-FI
MAE
ACC1
Forecast
k 0
0.0604
38.1630
0.0969
0.1361
1.8868
0.1826
0.0105
3.1566
0.0920
1
0.1312
37.6239
0.1808
0.1393
1.7913
0.2381
0.0144
3.2606
0.1123
2
0.1257
36.4688
0.1870
0.1622
1.7960
0.2505
0.0151
3.2266
0.1180
k 5
0.1518
35.5605
0.2133
0.1676
1.8114
0.2581
0.0149
3.3092
0.1150
7
0.1491
35.6785
0.2107
0.1991
1.7810
0.2828
0.0138
3.2177
0.1002
= 10
0.1667
37.6084
0.2442
0.1807
1.7820
0.2397
0.0124
3.0904
0.0901
Imputation
= 0
0.0263
33.0097
0.0594
0.0915
2.6696
0.1210
0.0015
3.6527
0.0446
k 1
0.0419
33.1403
0.0738
0.1165
2.5106
0.1490
0.0018
3.6402
0.0569
k 2
0.0527
31.1138
0.0826
0.1102
2.3576
0.1486
0.0022
3.5375
0.0527
5
0.0520
33.3440
0.1001
0.1013
2.3982
0.1569
0.0023
3.2528
0.0547
7
0.0509
34.0198
0.0994
0.1001
2.4228
0.1462
0.0019
3.3447
0.0475
= 10
0.0474
31,2001
0.1069
0.1219
2.3771
0.1546
0.0015
3.2552
0.0406
Anomaly Detection
k 0
0.0923
0.0763
0.2755
0.1949
0.0159
0.0777
k 1
0.1002
0.0681
0.2809
0.1961
0.0172
0.0854
= 2
0.0739
0.0569
0.3361
0.2891
0.0213
0.1062
= 5
0.0837
0.0563
0.3535
0.2720
0.0337
0.1637
k 7
0.0705
0.0469
0.3436
0.2516
0.0278
0.1369
10
0.1026
0.0700
0.2340
0.1629
0.0222
0.1097
Table Comparison of performance metrics (M-F1, MAE, and ACC) across Breakfast, MuliTHUMMS and EPIC_KITCHEN datasets over forecast, imputation and anomaly detection tasks for different few-shot values l given in context examples. & () indicates Zero Shot. Higher M-FI and ACC values indicate better performance, while lower MAE values are better. MAE computation is not applicable for anomaly detection. Best metric values are indicated in bold.
A.11 LASTS Few Shot
We study the impact of varying the number of examples (k) in the few-shot setting to determine the optimal value of k for our method. Specifically, we evaluate the performance of LASTS Few Shot on all datasets and tasks using different values, ranging from 0 (Zero Shot) to 10. shown in Figure and detailed in Table 9, the performance metrics-Macro- MAE, and ACC-improve significantly
inereases from 0 to 5. However, further increases beyond do not consistently yield improvements and, in some cases, result in marginal performance degradation.
average, achieves the best balance across all metrics and datasets. Therefore, we adopt as the default value for LASTS Few Shot and include it as the entry for "LASTS Few Shot" in Table 10.
A.12 Further analysis on Stochastic Soft Prompts (StoP)
In this section, we comment on the structure learned by StoP prompts and discuss the practical benefits of Stochastic Soft Prompts.
Evidence for Coarse-to-Fine Structure of structured coarse-to-fine hierarchy. In this structure, the first few tokens appear to encode broader task-level information, while later tokens may refine predictions by adding more detailed nuances. Below, we provide observations that support this behavior:
The prompts learned through Stochastic Soft Prompts (StoP) suggest the presence
1. t-SNE Projections: Visualizations of t-SNE projections (see Figure 13) suggest that the first few tokens in StoP
22

---

LAST SToP For Modeling Asynchronous Time Series
Average Macro-F1 (Across All Dalasets and Tasks)
Average MAE (Across All Datasels and Tasks)
Average ACC (Across, |ll Dalasets and Tasks)
0.110
0.32
13.7
0300
0.44
13.6
0.14
0328
13.5
Hoas.1
0.300
0.2
13.4
0.005
0.685
(Pwge-Shot Examples)
[fhaeeSholl Exumples)
k (TwaeShot Exemples)
Figure 12: Average values of Macro-F1, MAE, and ACC across all datasets and tasks for different values of (number of few-shot examples). Higher values indicate better performance for Macro-F1 and ACC, while lower values indicate better performance for MAE, The results indicate that on an average, k 5 works best.
prompts may encode more diverse or independent representations, as indicated by their wider spread in the projection space. In contrast, the later tokens tend to cluster more closely together, potentially reflecting the refinement of previously encoded information.
2. Cosine Similarity: Adjacent tokens at the beginning of the StoP prompt tend exhibit lower cosine similarity compared to tokens later in the prompt (see Figure 13). This pattern suggests more diverse information being captured at the beginning of the prompt. Standard soft prompts, however, show uniformly high cosine similarities across all tokens, lacking this structure.
3. Prefix Validity: Figure 6 indicate that any prefix of StoP prompt serves as valid standalone prompt, with additional tokens refining the predictions. This behavior suggests that early tokens convey broad task-level information, while later tokens refine and add finer-grained details.
Practical Benefits of StoP
We observe that StoP offers many benefits over standard soft prompting:
1. Improved Generalization: StoP prompts achieve better generalization compared to standard soft prompts, with an average improvement of 12.69% % in Macro-FI across all datasets (Breakfast, MuliTHUMOS, and EPIC.KITCHENS) and tasks (Forecast, Imputation, Anomaly Detection) (see Table 6)
2. Faster Training:
The stochastic nature of StoP reduces training time by approximately 25%, making it more efficient than standard soft prompting.
3. Resource Efficiency:
StoP enables flexible deployment in resource-constraingg environments. Longer trained StoP prompts be truncated prefixes needed, allowing for adaptable inference without compromising performance.
A.13 Complete Evaluation on Textual Datasets
Here we reproduce the main table from our paper, along with accuracy numbers for the interested readers.
23

---

LAST SToP For Modeling Asynchronous Time Series
1m0
199
0.0
Figure 13: Left: t-SNE projections of Stochastic Soft Prompt (StoP) tokens with prompt length of 50 the Breakfast dataset for the forecasting task. Adjacent tokens are connected by line, and the color darkens as the token index increases. The presence of lighter tokens on the periphery and darker tokens in the center indicates that the initial tokens learn very diverse information, while this diversity diminishes as the token index increases. Right: Pairwise cosine similarity of the first 350 tokens stochastic soft prompt and soft prompt learned on the Breakfast dataset for forecasting. We observe that in StoP, the initial cosine similarities are smaller and increase as the token index increases, while no such variation by token index is present normal soft prompt.
24

---

LAST SToP For Modeling Asynchronous Time Series
Table 10: Performance of our models on three textual datasets for forecasting, imputation, and anomaly detection tusks. Metrics are macro FI, and accuracy (ACC) for event type prediction and MAE for event time prediction. The best result in each class is highlighted in bold, and the second-bes result underlined. Note that for anomaly detection, since the task involves identifying only the anomalous event, the MAE metric is not applicable and Chronos and LLMProcesses are not adaptable (see A.6). A indicates our method. We use 5 examples for few shot results (see A.11).
Model
Breakfast
MuliTHUMOS
EPIC-KITCHENS
M-FI (1)
MAE (L)
ACC
M-FI (t)
MAE (L)
ACC
M-FI (1) MAE ACC (t)
Forecast
Random
0.0162
40.1513
0.0201
0.0417
1.8003
0.0382
0.0000
3.2001
0.0001
Chronos
0.0011
43.0502
0.0021
0.0265
1.805
0.0279
0.0000
3.5925
0.0005
I.LTTime
0.0240
37.3902
0.0288
0.1280
2.2060
0.1235
0.0040
4.8948
0.0458
LLMProcesses
0.0337
44.9856
0.0845
0.1278
2.0471
0.0970
0.0049
4.3843
0,0703
LASTS Zero Shot"
0.0604
38.1630
0.0969
0.1361
1.8868
0.1826
0.0105
3.1566
0.0920
LASTS Few Shot"
0.1518
35.5605
0.2133
0.1676
1.8114
0.2581
0.0149
3.3092
0.1150
LASTS + QLORA"
0.2558
33,9737
0.3763
0.3218
1.7281
0,4337
0.0764
2.8964
0.2160
LASTS SP'
0.2341
32,8417
0.3691
0.3707
1.6630
0.4782
0.0780
2.8830
0.2217
LASTS + StoP"
0.2633
32,5464
0,3880
0.3947
1.6503
0.4784
0.0797
3.0318
0.2298
Imputation
Random
0.0168
37.7029
0.0214
0.0435
2.3622
0.0416
0.0000
3.4269
0.0001
Chronos
0.0013
38.4039
0.0044
0.0294
2.3971
0.0312
0.0000
3.6955
0,0000
ILMTTme
0.0137
35.9899
0.0381
0.0968
2.6998
0.1330
0.0005
3.6750
0.0314
L.MPrrocesss
0.0156
34.7117
0.0488
0.1123
2.3786
0.1430
0.0008
4.2600
0.0371
LASTS Zero Shot"
0.0263
33.0097
0.0594
0.0915
2.6696
0.1210
0.0015
3.6527
0,0446
LASTS Few Shot"
0.0520
33.3440
0.1001
0.1013
2,3982
0.1569
0.0023
3.2528
0.0547
LASTS QLORA"
0.1688
28.5638
0.2500
0.2132
2.2179
0.2744
0.0378
3.1194
0.1137
LASTS + SP"
0.1581
28.8503
0.2264
0.2044
2,4092
0.2528
0.0423
3.1456
0.1270
LASTS StoP"
0.2064
28.2251
0.2740
0.2213
2.3445
0.2839
0.0610
3.1116
0.1424
Anomaly Detection
Random
0.0349
0.0396
0.0381
0.0552
0.0238
0.0307
LLMTime
0.0240
0,0288
0,0415
0.0639
0.0048
0,0650
LASTS Zero Shot
0.0923
0.0763
0.2755
0.1949
0.0159
0.0777
LASTS Few Shot
0.0837
0.0563
0.3535
0.2720
0.0337
0.1637
LASTS + QLORA"
0.7011
0.6478
0.6003
0.5084
0.6520
0.6988
LASTS SP"
0.6520
0.5937
0,5231
0.4657
0.6159
0.6635
LASTS StoP"
0.7198
0.6698
0.6045
0.5168
0.6603
0.7037
25


Reviews:
[{"review_id": "vxzDkShBnF", "reviewer": "Reviewer_fec8", "summary": "This paper presents LASTS, a novel framework that uses large language models (LLMs) to model asynchronous time series—sequences of events that occur at irregular intervals and are described in natural language. Unlike traditional methods that rely on fixed time intervals and predefined event categories, LASTS leverages the semantic richness of event descriptions and irregular timing to enable LLMs to perform tasks such as forecasting, anomaly detection, and data imputation. The authors also propose Stochastic Soft Prompting (StoP), a new prompt tuning technique that improves model performance by randomly truncating soft prompts during training, resulting in more diverse and generalizable representations. Through extensive experiments on real-world datasets, including action recognition and temporal point process benchmarks, LASTS consistently outperforms existing methods and demonstrates strong adaptability across different tasks. The approach offers a flexible and efficient alternative to conventional time series models and highlights the potential of LLM-based solutions for complex temporal reasoning.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their detailed review. We appreciate the recognition of our method's novelty, the thoroughness of our experimental evaluation, and the relevance of our chosen tasks and benchmarks. We also thank the reviewer for highlighting how our work aligns with broader scientific trends—adapting LLMs to new domains, advancing parameter-efficient fine-tuning techniques, and challenging the reliance on specialized architectures.\n\nWe acknowledge the two specific areas of improvement identified and address them below:\n1. **Relatively weaker time prediction performance compared to TPP models**:\nWe agree with the reviewer, however, our design prioritizes simplicity, general applicability across tasks (forecasting, imputation, and anomaly detection), and effective use of diverse natural language event descriptions—without explicitly modeling time. This leads to our model ranking as best on 13 and in the top 2 on 17 out of 18 evaluations in Table 2. Introducing explicit time modeling is a valuable next step, which we are actively exploring as future work.\n\n\n2. We answer this in two parts:\n- **Anomaly detection baselines:**  This task remains underexplored  in the context of asynchronous time series. While we have adopted someTime Series methods (e.g., Chronos, LLM Processes) to the asynchronous setting for forecasting and imputation, they cannot be easily extended to anomaly detection as they are very forecasting focused and anomaly detection is not easily recast as a forecasting problem. We see our work as an early step toward bridging this gap.\n\n- **Inclusion of industrial datasets:** We used publicly available datasets from standard benchmarks widely adopted in the literature to ensure fair comparison with both traditional TPP models and modern time series forecasting methods. We agree that incorporating additional real-world industrial datasets could further strengthen the evaluation.\n\nWe thank the reviewer again for their insightful feedback, and are happy to answer any additional questions.", "claims_and_evidence": "Most claims in the submission are supported by clear and convincing evidence. The authors provide comprehensive experimental results across multiple datasets and tasks (forecasting, imputation, anomaly detection) to demonstrate the effectiveness of their method. They also include strong baselines for comparison, such as traditional TPP models, foundation models, and other LLM-based approaches. The performance gains of the proposed LASTS framework and Stochastic Soft Prompting are consistently shown. However, one potential limitation is the relatively weaker time prediction performance compared to some TPP models, which the authors acknowledge but do not fully address with additional modeling strategies.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation criteria are appropriate for the problem of modeling asynchronous time series. The use of natural language-based prompts aligns well with the irregular and semantically rich nature of such data. The selected tasks—forecasting, imputation, and anomaly detection—are relevant and practical. The chosen datasets, including both real-world temporal point process and action recognition data, provide a comprehensive benchmark. However, incorporating more diverse anomaly detection baselines or real-world industrial datasets could further strengthen the evaluation.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experimental design appears generally sound, with a reasonable choice of tasks, datasets, and evaluation metrics. The use of multiple baselines and both zero-shot and fine-tuned settings adds credibility to the results. At a glance, there are no obvious flaws, but a deeper look would be needed to verify the robustness of all experimental components.", "relation_to_broader_scientific_literature": "The key contributions of the paper build on and extend several existing directions in the broader scientific literature. First, it advances the emerging line of work that explores using large language models for time series tasks by adapting them to irregular, event-based sequences rather than traditional regularly sampled data. Second, it connects to research on prompt-based learning and parameter-efficient fine-tuning, introducing a novel variant—Stochastic Soft Prompting—that aligns with broader trends in reducing adaptation cost for large models. Finally, the work challenges the conventional reliance on specialized architectures like temporal point processes by demonstrating that general-purpose language models, when properly prompted, can handle a wide range of temporal reasoning tasks, contributing to the growing movement toward more unified, flexible modeling approaches across domains.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pB4RY7sRN4", "reviewer": "Reviewer_aqLD", "summary": "This paper introduces a novel framework for modeling asynchronous time series data using Large Language Models (LLMs). Unlike regular time series with evenly spaced time points, asynchronous time series consist of timestamped events occurring at irregular intervals, each described in natural language. This work demonstrates the potential of LLM-based approaches for asynchronous time series analysis across multiple tasks and domains, offering a flexible alternative to traditional methods while leveraging the world knowledge embedded in LLMs.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the creative integration of concepts, practical applicability, innovation in training techniques, and comprehensive analysis in our work. Responses to key points raised:\n\n**Chronos: Limited analysis; inclusion of TEMPO:** Chronos performs poorly as expected, given its reliance on time series–specific augmentations and synthetic data, which make it ill-suited for asynchronous time series due to their fundamental differences (L046–L020). We included Chronos  as a representative general-purpose TS model to highlight how such assumptions hinder performance, unlike models like LLMTime and LLMProcesses that incorporate fewer biases. which likely contributes to their stronger performance.\n\nThanks for pointing us to TEMPO; it faces similar limitations due to its reliance on seasonal and trend decomposition—concepts not meaningful for asynchronous sequences. These comparisons underscore the need for purpose-built approaches like LASTS. We will revise the manuscript to clarify this discussion.\n\n**NeuralODEs, LipCDE:**\nThanks for bringing up NeuralODE-based approaches like LipCDE. NeuralODEs are generally ill-suited for modeling asynchronous time series due to two key limitations:\n1. In ODE systems, the future trajectory is entirely determined by the initial state, which implies that modeling long asynchronous time series would require the initial state to encode all future observations—an unrealistic assumption for most real-world data.\n2. The continuous trajectories assumed by NeuralODEs make them ill-suited for capturing abrupt changes or irregular time gaps, which are common in asynchronous time series with discrete, sudden shifts.\n\nThus, NeuralODEs are only appropriate when underlying dynamics are deterministic and continuous, which is rarely true in practice. To our knowledge, no NeuralODE-based models have been evaluated on the datasets used in this paper.\n\n**Weaknesses**\n1. **Domain-Specific Analysis:**\nWe agree that domain-specific analysis can offer additional insights. In our work, we chose to focus on generalizability across 8 diverse datasets spanning 7 distinct domains given the broad ICML audience which makes it difficult to derive domain-specific conclusions. However, we make following observations to be included in teh camera-ready version:\n-  Online Shopping (amazon): As discussed in L371–373, the Amazon dataset includes a mix of unrelated event types grouped under one label,  which possibly hurts time prediction. \n- Cooking: Datasets like Breakfast and EPIC-KITCHENS show strong performance, as our model benefits from rich natural language descriptions and meaningful event sequences.\n- Sports (MultiTHUMOS): This dataset features more Markovian event transitions, making forecasting relatively easier but anomaly detection harder.\n\n2. We answer this in two parts:\n- **Theoretically, why SToP is better than SP:** Our focus in this work is to demonstrate empirically, across multiple tasks and datasets, that SToP outperforms soft prompting—both in performance (Table 1) and in the structure of learned tokens (Section 4.5). We agree that a theoretical foundation would be valuable, but given the complexity, we are exploring this as future work.\n- **Coarse-Fine Structure Emergence:** This emergence is a direct consequence of random prefix length selection during training which encourages early tokens to capture general patterns, while later tokens refine the representation. Similar behaviours are observed in prior work discussed in our manuscript (L254-260) :\n  - **Soundstream's** residual vector quantization (RVQ), where selecting a random number of quantizers during training leads to coarse-to-fine audio reconstruction, and \n  - **Matryoshka Representations**, which explicitly optimize for informative prefixes. \n\n  We will incorporate this discussion in the final version of the paper.\n\n3. **Practical Implementation Details:**  We're happy to clarify any remaining implementation details, revise the manuscript as needed, and will release the code upon publication.\n\n**Other Comments: Examples and  Visualization:** Thank you for the suggestion. Due to space constraints, we cannot include this analysis in the rebuttal but will incorporate it in the camera-ready version.\n\n**Questions**\n1. **What do we learn form Fig 5 & 13:** We hypothesize the emergence of a coarse-to-fine structure in SToP, where earlier tokens capture diverse high-level features, and later tokens refine them:\n- **Fig5**  the t-SNE projection shows that the first 100 tokens in SToP are more spread out compared to the clustered tokens in standard soft prompting. This indicates that SToP learns more diverse representations in earlier tokens. \n-  **Fig13:** shows that the cosine similarity between adjacent tokens is lower at the beginning of the SToP prompt and gradually increases, consistent with a coarse-to-fine pattern. No such organization is observed in standard soft prompting.\n\nThank you for your thoughtful review.", "claims_and_evidence": "1. LASTS as an effective framework for asynchronous time series modeling\n- Evidence: Comprehensive evaluations across multiple datasets (Breakfast, MultiTHUMOS, EPIC-KITCHENS and five standard TPP datasets) show consistent performance improvements.\n- The zero-shot performance of LASTS exceeds other zero-shot baselines (Tables 1, 2, Figure 4).\n- Comparison with specialized TPP models shows competitive or superior performance (Table 2).\n2. Stochastic Soft Prompting (SToP) outperforms other PEFT methods\n- Evidence: Detailed comparative results show SToP consistently outperforming SP and QLoRA across datasets and tasks (Table 1).\n- Appendix Tables 6-7 quantify the performance gains (average 12.69% M-F1 improvement over SP and 13.55% over QLoRA).\n- Training efficiency measurements show ~25% faster training than standard soft prompting.\n3. Multi-task capability without task-specific designs\n- Evidence: The same LASTS representation is successfully applied to forecasting, imputation, and anomaly detection without architectural changes (Table 1).\n- Performance on all three tasks significantly exceeds baselines when using the adapted models.\n4. Ability to handle large event spaces\n- Evidence: Successful modeling of EPIC-KITCHENS dataset with ~20,000 unique event descriptions.\n- Table 2 indicates traditional TPP methods encounter OOM errors on this dataset", "methods_and_evaluation_criteria": "The methods are well-designed for the problem, and the evaluation criteria are appropriate and comprehensive, covering diverse datasets, tasks, and comparison points. The authors have made sensible choices in metrics and baseline comparisons that allow for a fair assessment of their contributions.", "theoretical_claims": "There is no theoretical contribution.", "experimental_designs_or_analyses": "1. The Chronos's performance is bad based on the results shown in Table1. It should be more analysis or more TS foundations, such as TEMPO, involved in the experiments to understand the fundamental results.\n\n2. Can the NeuralODE-based solution such as LipCDE[1] be used to address such irregular time series task?\n\n[1] Cao, D., Enouen, J., Wang, Y., Song, X., Meng, C., Niu, H., & Liu, Y. (2023, June). Estimating treatment effects from irregular time series observations with hidden confounders. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 37, No. 6, pp. 6897-6905).", "relation_to_broader_scientific_literature": "The key contributions of LAST SToP relate to the broader scientific literature by: (1) addressing limitations of traditional Temporal Point Processes that struggle with large event spaces and natural language descriptions; (2) bridging the gap between foundation models for time series (like Chronos) and asynchronous time series modeling; (3) extending Parameter-Efficient Fine-Tuning approaches like soft prompts with a novel stochastic training method that parallels techniques from other domains like dropout and Matryoshka Representations; and (4) demonstrating that LLMs can effectively model complex temporal data beyond their traditional text domain, complementing efforts like TimeLLM and LLMTime which focused on regular time series.", "essential_references_not_discussed": "As mentioned in the previous answers, there could be more NeuralODE-based related works.", "other_strengths_and_weaknesses": "1. Creative Integration of Concepts\n\n- The paper creatively combines soft prompting approaches with stochastic training techniques, drawing inspiration from areas like dropout and Matryoshka representations to create a novel adaptation mechanism.\n- The authors' approach to viewing asynchronous time series as natural language data is an elegant shift in perspective that leverages the strengths of LLMs.\n\n2. Practical Applicability\n\n- The method addresses real-world challenges in handling asynchronous time series data, which appear in numerous important domains (healthcare, finance, e-commerce, social media).\n- The parameter-efficient nature of SToP (using only 0.02% of model parameters) makes it practical for deployment in resource-constrained environments.\n\n3. Technical Innovation in Training\n\n- The stochastic prefix selection during training is a simple yet effective innovation that produces measurable benefits in representation quality and training speed.\n- The observed coarse-to-fine structure in learned prompts suggests an interesting emergent property that could have broader applications in prompt tuning.\n\n4. Comprehensive Analysis\n\n- The paper provides thorough analyses (t-SNE visualizations, cosine similarity measurements, model probing) that give insights into why their method works.\n- The scaling experiments across different model sizes (1B, 3B, 8B) demonstrate the approach's robustness and future potential.\n\nWeaknesses\n\n1. Limited Analysis of Domain-Specific Performance\n\n- While the paper tests on datasets from different domains, there's limited analysis of how performance varies across domains and why certain domains might benefit more from the approach.\n- A deeper exploration of domain-specific challenges and how the method addresses them would strengthen the paper.\n\n2. Theoretical Foundations\n\n- The paper lacks theoretical grounding for why SToP works better than standard soft prompting. While empirical results are strong, a more formal analysis would strengthen the contribution.\n- The connection between the stochastic training procedure and the emergence of coarse-to-fine structure could be better explained.\nPractical Implementation Details", "other_comments_or_suggestions": "- The paper would be strengthened by including some concrete examples of model predictions compared to ground truth, especially for cases where the model performs particularly well or poorly.\n\n- Visualizing how predictions differ across methods could provide intuition about the advantages of LASTS.", "questions_for_authors": "What can we exactly learn from the Figure 5 and Figure 13?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "trze7ge1Ap", "reviewer": "Reviewer_L5nG", "summary": "This paper presents LASTS (Language-modeled Asynchronous Time Series), a novel framework for modeling asynchronous time series data using Large Language Models (LLMs). The approach addresses the challenges of irregular timing and diverse event types by representing asynchronous time series as natural language prompts, allowing LLMs to leverage their broad world knowledge for reasoning across different domains and tasks.\nThe authors introduce Stochastic Soft Prompting (StoP), a parameter-efficient adaptation technique that significantly improves model performance. Unlike traditional soft prompting, StoP randomly selects prefixes of the prompt during training, encouraging the learning of diverse representations and improving generalizability.\nThrough extensive experiments on real-world datasets, the paper demonstrates that LASTS achieves state-of-the-art performance across forecasting, anomaly detection, and data imputation tasks. The framework outperforms existing methods including temporal point process models, foundation models for time series, and other LLM-based approaches.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their thoughtful summary and generous assessment of our work. We are glad that the core contributions—LASTS and Stochastic Soft Prompting (SToP)—were found meaningful and well-supported. Below, we address the specific questions raised:\n\n**Train/Validation/Test Splits**\n\nWe follow the standard protocol for each dataset as adopted in prior work (e.g., [Xue et al., 2024]). The splitting is done at the sequence level—given a dataset of N independent sequences, we use 80% for training, and 10% each for validation and testing, sampled independently from a shared distribution. The model thus learns patterns governing sequences of events and their interarrival times from the training set, and generalizes to unseen sequences in the validation and test sets. We use the standard protocol because it enables us to compare our results with those already published.\n\nWe acknowledge that traditional time series tasks require chronological splits to account for drift. However, in our datasets (e.g., EPIC-KITCHENS, MultiTHUMOS), the sequences are short, self-contained, and typically do not exhibit long-term temporal drift. This mitigates the need for chronological splitting. Nevertheless, we appreciate the reviewer’s concern and will make our data split methodology more explicit in the final version of the paper.\n\n**Comparison to Other Prompt-Based Adaptation Methods**\n\nThank you for this suggestion. Many adapters have been proposed recently, and it would not be feasible to compare all of them. Therefore, we decided to focus our comparison on **widely-used PEFT techniques**: Soft Prompting and QLoRA. QLoRA is the standard adapter-based method and strong baseline, and we highlight that LASTS is compatible with such adapter techniques. Although we have not yet tried prefix tuning, there are no methodological limitations that would prevent its use with **stochastic training strategy (SToP)**, and we plan to investigate this in future work. \n\nOnce again, thank you for your encouraging and constructive review. We are grateful for the thoughtful feedback.", "claims_and_evidence": "Yes, I think most claims in the paper are well-supported.\n\n1. LASTS effectively leverages LLMs for asynchronous time series analysis\n\nThis is supported by extensive experiments across multiple datasets and tasks (forecasting, anomaly detection, imputation) and demonstrated through comparisons with traditional temporal point process models and other LLM-based approaches.\n\n2. Stochastic Soft Prompting (StoP) improves performance over traditional soft prompting\n\nThis is supported by quantitative results showing improvements in Macro-F1 scores across dataset and visualized through t-SNE projections demonstrating more diverse token representations in StoP.\n\n3. LASTS is parameter-efficient\n\nThis is supported by implementation details showing only 1.6M trainable parameters for prompt tuning.\n\n4. LASTS outperforms existing methods\n\nThis is supported by comprehensive comparisons with temporal point process models, foundation models for time series, and other LLM-based approaches.", "methods_and_evaluation_criteria": "Yes I do think they make sense for the problem studied.\n\nFor the LASTS framework, I think the approach of representing asynchronous time series as natural language prompts makes sense given the irregular timing and diverse event types characteristic of such data.\n\nFor the StoP technique, I think its modification to traditional soft prompting addresses the need for more diverse representations in prompt-based adaptation and the coarse-to-fine structure learned by StoP appears suitable for capturing both general task information and specific details in asynchronous time series.", "theoretical_claims": "There are no theoretical claims in the paper.", "experimental_designs_or_analyses": "I checked the soundess of the experimental designs and analyses of the paper. I think they are overall sound.\n\n1. Experimental designs: The authors select diverse approaches as the baselines: including random baselines, foundation models for time series (Chronos), LLM-based approaches (LLMTime, LLMProcesses) and TPP models. The authors cover three text-based action datasets and five standard TTP datasets in the experiments. As for the evaluation metrics, the authors use M-F1, MAE and RMSE to evaluate the performance of models.\n\n2. Analyses: The paper includes ablation studies comparing different prompt representations (time first vs. event first) and different time representations (inter-arrival times vs. durations). These analyses help establish the effectiveness of the chosen representations.\nThe comparison between StoP and traditional soft prompting is thorough, with both quantitative results and qualitative analysis of learned representations. The analysis of training speed differences between StoP and traditional soft prompting adds practical value. The evaluation of few-shot learning with varying numbers of examples (k=0 to k=10) is well-designed and provides useful insights into how many examples are needed for optimal performance. The identification of k=5 as the optimal few-shot setting is justified by the results.", "relation_to_broader_scientific_literature": "The paper's contributions represent meaningful advancements while building on established foundations in the field. The authors have successfully connected their work to prior literature, demonstrating how LASTS and StoP address limitations in existing approaches and extend the capabilities of LLMs to new types of data and tasks.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "None", "other_comments_or_suggestions": "None", "questions_for_authors": "1. Did you perform chronological splitting of the datasets for train/validation/test, or was it done randomly? This is particularly important for time series data to avoid data leakage and better reflect real-world deployment scenarios. If random splitting was used, how might this affect the validity of your results compared to chronological splitting?\n\n2. Beyond comparing with traditional soft prompting and QLoRA, have you considered comparing with other prompt-based adaptation methods like prefix tuning or adapter layers? How might these comparisons affect your claims about the superiority of StoP?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4vpQBQddM5", "reviewer": "Reviewer_KG4G", "summary": "The authors propose a method for modeling temporal event sequences by finetuning pretrained language models. The paper shows that by using a novel prompt tuning method they are able to outperform several baselines and ablations.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's positive evaluation, thoughtful suggestions, and recognition of our contributions, including the LASTS representation and the Stochastic Soft Prompting (SToP) mechanism. We respond to each comment below:\n\n**Random Initialization:** Thank you for highlighting this relevant literature. We evaluated pretraining on the Breakfast dataset's forecasting task and observed clear benefits: fine-tuning a randomly initialized model yielded F1-score 0.14 vs 0.26, Accuracy 0.21 vs 0.39, and MAE 39.29 vs 32.55. This highlights the value of text pretraining in our asynchronous setting, given the rich natural language input. We’re happy to include this as an additional baseline in the main table of the camera-ready version if the reviewer recommends it.\n\n**Redundant Notation (Lines 210–215):** Thank you for pointing this out. We will remove the redundant notation.\n\n**Clarification on LORA:** Our goal in this section was to show that LASTS integrates easily with existing PEFT methods like LoRA and Soft Prompting for adapting an LLM backbone. We acknowledge the wording may have caused confusion and will revise it to clarify that we use the original LoRA implementation from [1]\n\n[1] https://github.com/huggingface/peft\n\n**Questions**\n\n1. We answer this in two parts:\n- **SToP Generality:**  We agree that SToP may have broader applicability, but the focus of our current work is specifically on asynchronous time series. Exploring SToP in other NLP domains is beyond the scope of this paper and is left for future work.\n- **Comparison to Other Prompt Tuning Techniques:** We include the *most widely-used PEFT methods*—Soft Prompting and QLoRA—as strong representatives for LLM adaptation; please see our response to Reviewer L5nG titled \"Comparison to Other Prompt-Based Adaptation Methods\" for further details.\n\n2. **Clarification on use of semantic information:** We appreciate the reviewer’s careful reading\n- The three datasets in Table 1—Breakfast, MultiTHUMOS, and EPIC-KITCHENS—contain semantic information that is available to the model and used in our experiments. (L270-274)\n-  For the five TPP datasets shown in Table 2, event names are replaced by categorical indices, and neither our model nor any TPP baseline uses semantic information.  (L264-267)\n- The experiments in Appendix A.4 using gibberish and textual descriptions are controlled ablations designed to isolate the effect of semantic content. \n\nThese experiments demonstrate that our model is flexible and effectively utilizes semantic information when it is available. We will make this clarification more prominent in the camera version of the manuscript.\n\n3. **Finetuning LLMTime, LLM Processes:** We focused on using these models in their zero-shot capacity, as they were specifically proposed and pretrained for that purpose. Since these methods are compared against  LASTS in the same zero-shot setting, we believe our comparisons are fair and meaningful, especially to assess generalization without additional tuning. We also highlight this zero-shot comparison in our main paper (L381–384), Figure 5, and Appendix A.6. We agree that fine-tuning LLMTime and LLMProcesses could be interesting baselines, but it may require additional effort to determine the optimal fine-tuning recipe and ensure a fair evaluation between models. Therefore, we will consider it as part of our future work..\n\n4. **QLoRA vs Full Finetune:** We chose QLoRA because it allows parameter-efficient adaptation with low memory cost, aligning with our goal of scalable deployment. As Table 1 and Appendix A.9 show, QLoRA performs competitively, and SToP improves upon it while using only 0.02% of model parameters. While full fine-tuning could potentially yield slightly improved performance, it would be computationally impractical given the limitations of our current hardware. \n\n5. **Simpler non neural baselines:** We thank the reviewer for this suggestion. Our primary baselines were state-of-the-art TPP models and recent LLM/PEFT techniques. Most recent literature on asynchronous time series modeling has shifted toward neural approaches, and we follow this trend to maintain consistency with comparable works. In non-neural TPPs, the features and functional forms used to model event intensities, dependencies, and histories often need to be manually designed or based on assumptions. This restricts the model's ability to generalize to complex datasets. In contrast, neural models offer greater flexibility in modeling the intensity function, enabling them to capture intricate relationships within the data. This adaptability makes neural TPPs better suited to handle diverse and complex datasets, improving their generalization and performance.\n\nOnce again, thank you for your encouraging and constructive review. We are grateful for the thoughtful feedback.", "claims_and_evidence": "On the whole, yes. I am generally skeptical of methods that adopt language models for time series forecasting, as prior work has shown that they are outperformed by simple linear methods \\[1\\]. However, the authors of this paper do compare to relevant TPP methods (Table 2), conduct strong ablations, and compare to a random baseline. \n\nOne other result from \\[1\\] is that randomly initialized language models (i.e. those without any pretraining at all) surprisingly perform just as well as pretrained models. The paper would be strengthened by showing that this is not the case for this task. \n\n\\[1\\] Tan, Mingtian, Mike A. Merrill, Vinayak Gupta, Tim Althoff, and Thomas Hartvigsen. 2024. “Are Language Models Actually Useful for Time Series Forecasting?” arXiv. [http://arxiv.org/abs/2406.16964](http://arxiv.org/abs/2406.16964).", "methods_and_evaluation_criteria": "Yes, the evaluation methods and datasets are standard and appropriate.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "I have a few questions about the experiments that I've listed below. On the whole the experiments (including the \"bonus\" experiments in the appendix) are interesting and well thought out.", "relation_to_broader_scientific_literature": "The contributions of this paper will be interesting to anyone working in the field of language models for time series, which is an active area of research. The StoP method is potentially of broad interest to the NLP community, although this is not the primary focus of the paper.", "essential_references_not_discussed": "See \"Claims and Evidence\"", "other_strengths_and_weaknesses": "The paper is very well written and contains exhaustive experiments.", "other_comments_or_suggestions": "- No need to reintroduce the notation in lines 210-215, would help paper flow\n- Your description of LoRA in Section 4.2 is confusing. My understanding is that you're simply applying the original technique, but the description reads as though you're making a novel contribution. This should be clarified.", "questions_for_authors": "- How does StoP compare to other prompt tuning techniques? It would appear that it is applicable to all kinds of NLP tasks, not just those involving time series. Your experiments show that it's really doing something different (and apparently better) than SP. Similarly, how do other techniques perform on this task? \n- \"For each of these datasets, the semantic meaning of the event type is unknown, and only the index of the event type is available.\" However, in Appendix A.4 you go on to to test the case where the event description is replaced with gibberish. This leaves me a little confused - in the main experiments (e.g. Table 1) does the model have access to semantic information about the events? \n- If you tune LLMTime and LLMProcess rather than doing zero shot is the comparison still as favorable? I know that these models are originally proposed as zero-shot, but it seems like it would be a fairer comparison if you tuned them as well. \n- Why are you using QLoRA rather than finetuning the whole model? \n- How does your method compare to simpler, non-neural baselines?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper models event sequences by fine-tuning pretrained language models. They proposes STOP, a stochastic soft prompting method that adapts the trained model to downstream tasks, and it is able to outperform several baseline methods. The paper is interesting to the subfield at the intersection of LLMs and time series.

All the reviewers give positive feedback to this paper, recognizing its technical design, experiment design, and interesting results.

I recommend to accept this paper.
Decision: Accept (poster)