Get Rid of Isolation: A Continuous Multi-task Spatio-Temporal Learning Framework
Authors: Zhongchao Yi
Zhengyang Zhou
Qihe Huang
Yanjiang Chen
Liheng Yu
Xu Wang
Yang Wang
Venue: NeurIPS 2024 oral (2024)
Keywords: continuous multi-task learning
spatio-temporal forecasting
urban intelligence
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: Breaking free from isolation, this work presents an innovative multi-task spatio-temporal modeling approach, fostering interconnectedness among diverse data sources for enhanced prediction accuracy and adaptability in urban forecasting

Abstract: Spatiotemporal learning has become a pivotal technique to enable urban intelligence. Traditional spatiotemporal models mostly focus on a specific task by assuming a same distribution between training and testing sets. However, given that urban systems are usually dynamic, multi-sourced with imbalanced data distributions, current specific task-specific models fail to generalize to new urban conditions and adapt to new domains without explicitly modeling interdependencies across various dimensions and types of urban data. To this end, we argue that there is an essential to propose a Continuous Multi-task Spatio-Temporal learning framework (CMuST) to empower  collective urban intelligence, which  reforms the urban spatiotemporal learning from single-domain  to cooperatively multi-dimensional and multi-task learning. Specifically, CMuST proposes a new multi-dimensional spatiotemporal interaction network (MSTI) to allow cross-interactions between context and main observations as well as  self-interactions within spatial and temporal aspects  to be  exposed, which is also the core for capturing task-level commonality and personalization. To ensure continuous task learning, a novel Rolling Adaptation training scheme (RoAda) is devised, which not only preserves task uniqueness by constructing data summarization-driven task prompts, but also harnesses correlated patterns among tasks  by iterative model behavior modeling. We further establish a benchmark of three cities for multi-task spatiotemporal learning, and empirically demonstrate the superiority of CMuST via extensive evaluations on these datasets. The impressive improvements on both few-shot streaming data and new domain tasks against existing SOAT methods are achieved. Code is available at https://github.com/DILab-USTCSZ/CMuST.

Get Rid of Isolation: A Continuous Multi-task Spatio-Temporal Learning Framework
Zhongchao Yi', Zhengyang Zhou 1.2.,,, Qihe Huang', Yanjiang Chen', Liheng Yu', Xu Wang 12, Yang Wang
Uniiversity of Science and Technology of China (USTC), Hefei, China
Suzhou Institute for Advanced Research, USTC, Suzhou, China
State Key Laboratory of Resources and Environmental Information System, Beijing, China haaeeemmannall @mail. uut.cstcee edu. cn, {wx309, zzy0929; angyan" Justc.. edu.
(zhongchaoyi, hqh, yjchen, yuliheng)
Abstract
Spatiotemporal learning has become a pivotal technique to enable urban intelli- gence. Traditional spatiotemporal models mostly focus on a specific task by assum- ing same distribution between training and testing sets. However, given that urban systems are usually dynamic, multi-sourced with imbalanced data distributions, current specific task-specific models fail to generalize to new urban conditions and adapt to new domains without explicitly modeling interdependencies across various dimensions and types of urban data. To this end, we argue that there is essential to propose a Continuous Multi-task Opatio-Tempone empower collective urban intelligence, which reforms the urban spatiotemporal learning from single-domain to cooperatively multi-dimensional and multi-task learning. Specifically, CMuST proposes ral interaction network (MSTI) to allow cross-interactions between context and main observations well self-interactions within spatial and temporal aspects to be exposed, which is also the core for capturing task-level commonality and personalization. To ensure continuous task learning, a novel Rolling Adaptation training scheme (RoAda) is devised, which not only preserves task uniqueness by constructing data summarizationndriven task prompts, but also harnesses cor- related patterns among tasks by iterative model behavior modeling. We further establish a benchmark of three cities for multi-task spatiotemporal learning, and empirically demonstrate the superiority of CMuST via extensive evaluations on these datasets. The impressive improvements on both few-shot streaming data and new domain tasks against existing SOAT methods are achieved. Code is available at aijttggg//gghbb...c////aaaaaLLL
learning framework (CMuST)
multi-dimensional spatiotempo-
Introduction
Spatiotemporal learning has become pivotal technique enable smart and convenient urban lives, benefiting diverse urban applications from intra-city travelling, environment controlling to location- based POI recommendation, and injecting the vitality into urban economics. Existing spatiotemporal
learning solutions [19] 9 38 40
8) focus on improving performances a task-specific model independently where these methods devise various spatial learning blocks [44 g 43 and
51
42
37
10
temporal dependency extraction modules 40 D to model the spatiotemporal heterogeneity.
Actually, urban spatiotemporal systems are usually highly dynamic with emerging new data modality. leading to serious generalization issue on both data pattern and task adaptation. As illustrated in
Yang Wang and Zhengyang Zhou are corresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Figure 1 the traffic volume patterns can evolve with urban expansion and establishment of new POIs. Concurrently, with increasing attention on road safety, traffic accident prediction has become a new task in intelligent transportation that inevitably suffers from the cold-start issue. Unfortunately, traditional task-specific models usually assume that data on a single task follows independent and identical distribution and are intensively available where such assumption directly leads to failures on data sparsity scenarios and generalization to new tasks. In fact, given diverse datasets, separately training single task-specific spatiotemporal models is cost sensitive and will trap the models into isolation. To this end, we argue that a continuous multi-task spatiotemporal learning framework is highly desirable to facilitate the task-level cooperation. It is even more interesting and exciting to jointly model multi-domain datasets with multi-task learning, which empowers understanding spatiotemporal system in holistic perspective and reinforce each individual task by exploiting the collective intelligence from diverse data domains.
The key towards exploiting task-wise correlations for mutual improvement is to capture the common interdependencies across data dimensions and domains. Current multi-task learning schemes either investigate the regularization effects between auxiliary and main tasks [31 [7], or devise loss objectives to constrain the consistency between each task Actually, given spatiotemporal domain, there must be common interdependencies across dif- ferent data types and domains, which are valu- able for cooperated learning. Even prosperity of multi-task learning and spatiotemporal fore- casting, there are never a systematic solution on how various sourced data from different tasks reinforce a specific task with multi-task learning. More specifically, interrelations among obser- vations in a system can be decomposed into multi-dimension interactions, i.e., from contextual environment to respective spatial relations and temporal evolutions, spatial-temporal level interactions and relations across different data domains. Considering the multi-level semantic correlations, learn- ing the system in a holistic perspective is required and further pose two challenges to continuous multi-task spatiotemporal learning framework, i.c.. 1) How to disentangle complex associations between data dimensions and domains, and capture such dependencies in an adaptive manner to improve spatiotemporal representation, hence facilitating the extraction of common patterns for mutual enhancement. 2) How to exploit task-level commonality and personality to jointly model the multi-task datasets, and exploit such extracted task-level commonality and diversity to reinforce respective task for getting rid of task isolation.
Irafie CHRANANEE VAPAPERORRL
eooeenncca center
New
(a) Traflic salara patam cvrkazie whar
Mo cold start kiee shea Envodreed
Figure 1: Illustration of evolution on data patterns and learning tasks.
In our work, a Continuous Multi-task SpatioTempore learning framework, CMuST, is proposed to jointly model multiple datasets in an integrated urban system thus reinforcing respective learn- Temporl Interaction Network (MSTI) is first
ing tasks. Specifically, a Multi multi-dimensional devised dissect interactions across data dimensions. including context-spatial, context-temporal and self-interaction within spatial and temporal dimensions. MSTI enables improved spatiotemporal representation with interactions, and also provides disentangled patterns to support commonality extraction. After that, a Rolling Adaptation training scheme, RoAda, which iteratively captures the task-wise consistency and musk-specific diversity, proposed. In RoAda, to maintain task characteri- zation, a task-specific prompt is constructed to preserve unique patterns distinguishing from other tasks by compressing data patterns via an Autoenc To capture the commonality across tasks, we propose a weight behavior modeling strategy to iteratively highlight the minimized variations of learnable weights, i.c., stable interactions during continuous training. which encapsulates crucial task-level commonalities. This approach not only stabilizes learning through continuous task rolling, but alleviates the cold-start over new tasks with shared patterns. Finally, a sask-specific refinement is devised to leverage commonality and fine-grained adaptation specific tasks.
The contributions of this work can be fhree-fold. 1) The first continuous multi-task spatiotemporal learning framework, CMuST to jointly model learning tasks in same spatiotemporal domain, which not only reinforces individual correlated learning task collective perspective, but also help understand the cooperative mechanism of dynamic spatiotemporal systems. 2) Technically, two learning modules, MSTI and RoAda are proposed to dissect the impacts and interactions over multi- dimensions, and iteratively update the task-wise commonality and generate individual personalization to continuous task adaptation in multi-task learning. 3) We construct benchmark datasets in each of
2

---

three cities, where two of them consists of at least 3 types of observations within same spatiotemporal domain. The extensive experiments demonstrate the superiority on enhancement of each individual task with limited data and the interpretation of task-wise continuous learning.
2 Related Work
Spatiotemporal forecasting is an emerging technique to capture the dynamic spatial and temporal evolution for diverse urban predictions, where the methods can be divided into machine learning- based and deep learning-basse simulate the dynamics including ARIMA [23), SVR { and matrix-factorization learning [28] for
Conventional solutions rely on complex mathematical tools to
capturing spatial correlations. With deep learning solutions flourishing, Convolution Neural Networks (CNNs) [40 II 46) are exploited to imitate the temporal dependencies and GNNs [43 13 36
are utilized to imitate spatial propagation. Meanwhile, by taking the advantage of the flexibility and interpretability of attention, Spatial-Tempora Attention (9), and Vision Transformer (ViT) [ are introduced to improve spatiotemporal representation. Also there are also many methods for temporal periodicity capturing [12 20 More specifically, DG2RNN
4 designs -raph
convolutional module to capture local spatial dependencies from both road distance and adaptive correlation perspectives. PDFormer [15] designs a spatial self-attention and introduces two graph masking matrices to highlight the spatial dependencies of short- and long-range views. TESTAM [88] uses time-enhanced ST attention by mixture-of experts and modeling both static and dynamic graphs. Even so, most solutions focus on single-task intelligence, fail to deal with complex interactions between data dimensions and never extract task-level commonality patterns, resulting in inferior performances on exploiting collective intelligence over multiple tasks. In contrast, we merge the gap by disentangling learnable interaction patterns and exploring rolling task adaptations.
Multi-task learning. Plenty of efforts have been made on multi task learning (MTL) and MTL can be elaborated by two-fold, i.c., feature-based and crrammter-based Feature-based MTL 2 learns a common feature representation for different tasks, but it may be easily affected by outlier tasks. To this end, parameter- based MTL (30 16 devised to exploit model parameters to relate different tasks, which is expected to learn robust parameters. Majority of these MTL schemes either concentrate on the diversity design and regularization effects of auxiliary tasks to main task 330 2 or construct loss functions to ensure task-wise consistency [47.. A pioneering work investigates gradient-drive task grouping to realize multi-task learning 41 where the focuses are text and images using pre- rreitrined CLIP model. For spatiotemporal learning. RiskOracle (52) and RiskSeq [53] are proposed to simultancously predict multi multi-grained risks and auxiliary traffic elements. More recently, UniST 445 and UniTime [22] construct a unified model for spatiotemporal and time-series prediction. However, all researches on ST learning still never dissect task-wise correlations, especially capture explicit consistency and diversity among tasks and investigate how each task reinforce the core task, which is of great significance for performance and interpretability in MTL.
Continuous learning and task continuous learning. Continuous learning (CL) usually keeps long-term and important information while updates model memories with newly arrived instances [29
3.5 For spatiotemporal forecasting, Chen, et, al. [5] proposes historical-date replay strategy,
TraffiSStream to update the neural network with all nodes feeding, while PECPM (35) manages
pattern bank with conflict nodes, which reduces the memory storage burdens. Most existing CL solutions are designed for homogeneous sourced data. Then there are very few research investigating task-level continuous learning where task can be converted from one to another. A pioncering work, CLS-ER [ realizes class-level and domain-level continuous learning with a dual memory to respectively store instances and build long-short term memories. Even so, the data input to CL.S-ER is with same image-like like property and scales, which is still far away from task diversity. We argue that learning continuously on task levels can increase the data tolerance over low-quality data and also generalization capacity on new tasks. In this work, by taking bonus of continuous learning, we propose iteratively assimilate the commonality and extract personalization reinforce each learning task and realize continuous multi-task spatiotemporal intelligence.
3 Preliminary
Definition 1 (Spatiotemporal features.) Spatiotemporal features refer to the data points collected by sensors deployed in urban environments, such as traffic dynamics on roads. We define spatiotemporal
3

---

All samples of spatio temporal series
Multi-dimensional
Task summarizziionn scheme
spatio-temporal interaction
nodes
cross interacttttnn
Spatinl-context
Spatial self-uttention
Various internctions
Tats protegt
Weight behavior modeling
Task
Forward
Data representation
and integration
LagetNocn
Taskespecific refinement
Observation field
Temporal encoding
Spatial enonding
&
Temporal-context
Temporal self-attention
Load bolk
cros--nteraccion
Task prompt
Rolling adaptation
Coice
Figure 2: Framework overview of CMuST.
feature at specific time and location vector X E R,,, where C represents the number of attributes (e.g., vehicle flow, speed) recorded by the sensor. We generalize this to define spatiotemporal data collected over period as X E T xC where 7 denotes the number of discrete time intervals, and N denotes the number of spatial nodes corresponding to sensor locations.
Definition (Spatiotemporal prediction.) Spatiotemporal prediction involves forecasting future values of spatiotemporal data based on historical observations. Specifically, given historical dataset X = IX.-,,.... X X.] E RT'xNxC' the objective is to predict future observations Ý = , X E TT'xXXC Here, T' represents the prediction horizon, the number of future time steps seek to forecast, which for simplicity is equal to T in this paper.
Definition 3 (Multi-task spatiotemporal learning.) Given set of spatiotemporal tasks in an integrated urban system T = (T1, T2..... Tk} where each task Ti is associated with a dataset Dr = (X,, Y.). Here, X; X represents the input features collected over T, time steps, across N; spatial locations, and C; feature dimensions, and Y; E xxxxC; represents the corresponding targets. Multi-task spatiotemporal learning aims to learn a function f Xo, XI,..., X Yo. ......, Y - that optimally predicts all target Y; from their respective inputs X,, leveraging shared and unique patterns across tasks to improve generalization performance.
Methodology
4.1 Framework Overview
The CMuST is crafted advance urban intelligence through synergistic integration of three components in Figure 2 We processes and standardizes diverse urban data into a harmonized format, and propose the MSTI to intricately dissect the complex interactions within spatiotemporal data, and devise a RoAda to dynamically fine-tune the model via continuous and careful updating, ensuring robust adaptability and consistent performance across fluctuating urban environments.
4.2 Data Representation and Integration
To harness various data domains within urban spatiotemporal systems well data interactions between diverse dimensions, the first task is to appropriately process main ST observations, spatial indicator, and temporal indicator create comprehensive and integrated data representation tailored for multi-task ST learning, enabling further interactive modeling between them.
To be specific, the main observation data, namely target of interest of urban datasets, are denoted as Xabs E 1 xC, and then mapped into a spatiotemporal representation Egbs E T xddas via an MLP ObsMLP l.b.). Similarly, the spatial indicator X, R consisting of longitude and latitude coordinates, is applied with a linear layer SpatialMLP(XX H,) to produce the spatial representation E, E TT'xNxx Temporal indicator comprises day-of week, time-of-day Xioos Xiod E RN'N and timestamps X E [B NxX6 which further compressed into hidden

---

representation, i.e.. E, = TemporalMLP | |tod: 0r) E R enabling the model to capture the periodicity and sequential features temporal data. P, task-specific prompt to ensure task-awar,ssss, and is integrated into the final embedding. Given the task Tk.
(T)) Eobs E, ((T))
where || denotes vector concatenation, combining spatial, temporal, and observational embeddings with the task prompt into comprehensive representation H(Tr) E RTxNxd, We will use H as representation for a specific task in following sections.
4.3 Multi-dimensional Spatio-Temporal Interaction
Spatiotemporal observations are usually complex with multiple-level interactions where such interac- tions and correlations play vital roles in enhancing commonality learning through different learning domains. To this end, we devise an MSTI, to intricately dissect and disentangle interactions within spatiotemporal data from spatial-temporal indicators to main observations by inheriting nice property of attention mechanisms, which all utilize transformed slices from the integrated representation H.
Spatial-context cross-interaction. To quantitatively investigate how spatial indicator interact with main observations, we devise a multi-head cross-attention [14] architecture (MHCA) where spatial and observational components are alternately used queries (Q) and key-value (KV) pairs:
head
MHCA (x.
CrossAttention (a.0) Wo
(2)
A=1
CrossAttention (a.6) (a.))())
Attention Q
K
(( h
Q K VD
Attention (Qi, K₁, Vo.) softmax
V
The symbol || denotes the concatenation of multiple attention heads, and Wo is the projection matrix that aligns the output dimensions with those of H. Here, we let variables and b indicate spatial (s) indicator and main observation (o), i.e., = 8 and b = o. Then the queries, keys, and values are generated through following transformations,
% Hi.., slice (1) (Q.)
{ H slice ww(k))
(K))
V - slice (0) W (Vo)
(5)
where W transforms input data into dimension D attention space, slice and ( denote the respective slices of H for spatial and observational features. After computing attention scores, the embeddings are taken into ferdffowarrd network (FFN) to enhance the learning capabilities, where FFN(X) max(0, XW, b, W2 + b2. The final attention outputs are then normalized by,
H (a.b) [... slice (b) LN(FFN(L((MHCA (o.b) (H) H (
(6)
+ LN(MHCA (a.b) (H) + H (b)),
where LN is layer normalization, the resulting matrices (s.00) and (o.s) are then concatenated back to H at their respective dimensions as (SCCI) e Nxxx, enriching the original representation with refined features that encapsulate intricate cross-dimensional relationships.
Temporal-context cross-interaction. To facilitate the attention computation with respect to temporal dimension, the representation first transposed as (SCC))) NxTxd, by denoting T as the sequence length for subsequent attention calculations. Then the step-wise positional encoding (refer to Appendix B is introduced allow our attention aware of specific temporal evolution.
The subsequent steps closely follow those of the spatial-context cross-attention mechanism, so the temporal- context cross-interaction (TCCI) can be performed as (CT) = (CCI(E)()CC))), and final representation (CI) e NxTxd, becomes the outcome of cross interactions between spatial and temporal dimensions.
Self-interactions within Spatial and Temporal Aspects. We proceed to apply self-interaction across different dimensions of the representations using self-attentions [21] Specifically, begin with
We introduce task summarization the prompt (refer Section 4.4 generate the distinct prompts.
5

---

temporal dimension, as the sequence length of representation dimension has naturally been aligned as T. This setup allows direct computation of the required self-attention,
head
MHA(X)
Attention (Qn, Kr, Va. Wo
h=1
In this multi-head attention (MHA) configuration, the attention calculation (referred to in Equation 4
involves queries, keys, and values, which is derived from the entire representation H H(C)) rather than individual slices. Then the output undergoes further processing through a FFN and non-linear transformations and LN to stabilize and enrich the feature representations similar Equation 6 The resulting (TST)) in RNx xxxx,, signifies the outcome of temporal self- ceff-interactions (TSI). Then the tensor is transposed to H T The final spatial self- inffrinterattion (SSI) computation is analogous to the temporal version, which refines spatial interactions and aggregates features across spatial nodes, as H = (SI(H)))))) The resulting tensor from this computation is H E T xds, which represents the outcome of comprehensive nulti-dimensional interactions.
We then adaptively integrate the interactions by a fusion strategy and Huber loss is adopted to ensure the robustness to outliers in spatiotemporal samples. Details can be found in Appendix B.3 Our MSTI allows extracting diverse interactions, including spatial-temporal domain interactions via designing cross attentions respective indicators, and self interactions within respective dimensions, enhancing the data relation learning and supporting commonality extraction across task domains.
4.4 Rolling Adaptation over Continuous Multi-task Spatio-Temporal Learning
To ensure continuous task learning, we propose a Rolling adaptation scheme, RoAda, to model the distinction and commonality among task domains. Our RoAda is composed of two stages with warm-up for commonality extraction and a task-specific refinement. Before the task rolling, we construct prompts to distinguish personalization of each task. Thus, the commonality and diversity can be leveraged to boost individual task adaptation.
Task summarization as prompts. To capture the task distinction, we devise task summarization by a Sampling- Sampling-Autoonoon Autooncoding scheme from each task. Consider task Tk. main observation becomes
(Tx)
R.
XNxC Such data is sampled by averaging observations over equivalent times of Tax (Te)
day, yielding periodic sample representation X (Tx) R² xNxC where L((T)) denotes the samp
number of time slots for task & within a day. Since neural networks tend to fit any data regularity, the sampled features are fed into autoencoder for extracting the compressed and distinguished data patterns. Given the encoding and decoding processes, i.e., Φ Xaurmp + S and ab S - X" Kaamp S encapsulates the summarized core characteristics of the task. The decoding phase maps S back to reconstructed X samp' by minimizing the mean squared reconstruction error,
Φ : S sigmoid (W, (W.X Xasapp + bz), O tb = arg min| XXamp (vb Φ))
X sampl 2
(8)
where We is the weight, and b, is the bias. Following the encoder, the summary features S are transformed into the k-th task prompt ((T)) (S(T)) 0p) with dimension alignment.
Weight behavior modeling. The first warm-up stage designed via weight behavior modeling, which assimilates the regularity from task to task. This process not only adapts the model to new tasks but also solidifies its ability in generalizing across scenarios by capturing task-wise common relations with modeling of model weight behaviors.
We begin with the task Tn via independently training the model until its performance stabilizes. By denoting M as the model learned by MSTI, the training phase can be formally described as,
(T.)
prompt
M,
Train(M(D(
train) Winit))
uotil
load
convergence
where W inst are the initialized weights. (TTi) is the training dataset of task Ti. and WT!T) are the weights when model converges. After that, our model transitions learning task from Ti to To by loading the corresponding task prompt ((T)) and dataset D(72) This transition involves a critical step where the evolution behavior of model weights W are carefully stored,
(( (2) traini W (TTT)))
for
W 0
(22) (T2)
..... W
(T)) e
(2)
(10)
each opoch

---

To capture common patterns, we reflect the task-level stability and variations by the evolution behavior of weights in M, i.e. W(T5) (w! w w w! The weight set W(Ta) is deliberately incorporated with finalized weight of task TI and evolution of T2, which explicitly captures the weight transition between tasks. We then introduce a collective variance o to
capture such stability, and employ threshold ò to disentangle the stable and dynamic weights along the learning process, i.e.,
( W stable
[wi E W(TT), Var(w:;) < 8), {wig E MM(Ta); Var(u,,) 2 53
(11)
dynamic
where Var(w;;) represents the element-wise variance of the across different training iterations from W to (! indicating the fluctuation degree of the weight values. A lower variance indicates higher stability, suggesting minimal change in weights across updates. After that, stable weights
W
stable are then frozen, and the model transitions to the next task, Ts, using the stabilized weights as the initialization for further training,
Train (MMinnMMD(tn W(TT) frozen W (Ta) traini
W )))
for
((
W(Ta)
(12)
each spoch
Similar to Equation 12 this process is repeated until the completion of task Tk. resulting in the collec- tion V(TT)) = {
w (T&) ( }. The Equation 11 is similarly implemented dynamic Since Ti is not involved with the commonality extraction, subsequently load Ti with W(T.), frozen w stable to achieve the complete rolling process. The dynamic serves as W can ulti-
Te-1)
W
to derive W(T))
(Ti)
stable || W (Tx)
stabilized weights across continuous tasks (T)) = w
stable || W mately result in a robust multi- task learning parameters encapsulated with well-extracted common patterns via iterative stable weight selection, and can also be served a collective intelligence by exploiting multiple tasks, thus enhancing the generalization for subsequent learning.
Task-specife refinement phase. This phase aims to merge the gap between task-level commonality and specificity, where the process continuously train the model from the prior phase to the next one, as W stable is frozen to maintain overall model stability and remaining weights are iteratively update with task-specific prompts. This process allows CMuST to insert the individual intelligence into integrated model and adjust itself to better suit the unique pattern of each task, i.e.,
Ti) (a) FineTuning W ((T))
(13)
where (T)) is tuned to maximize performance on the task at hand, which represents the task-specific submodel after refinement. The FineTuning denotes adjustment process of model parameters.
Summary. Our RoAda not only ensures the preservation of commonalities across tasks, but each model is also optimally tuned for its respective task with compressed task-level patterns, providing CMuST with opportunity to achieve peak performance with both collective and individual intelligence. More details of our methodology can be found in Appendix.
5
Experiment
5.1 Datasets and Experiment Setup
Data description. Given the emerging multi-task ST learning, we collect and process three real- world datasets for evaluation: 1) NYC Includes three months of crowd flow and taxi hailing from Manhattan and its surrounding areas in New York City, encompassing four tasks: Crowd In. Crowd Out, Taxi Pick, and Taxi Drop. 2) SIP: Contains records of Traffic Flow and Traffic Speed within Suzhou Industrial Park over period of three months. 3) Chicagd") Comprises of traffic data collected in the second half of 2023 from Chicago, including three tasks: Taxi Pick, Taxi Drop, and Risk.
Baselines and metrics. Our CMuST model is evaluated on a widely-used baselines spatiotemporal prediction, including RNN-based models (DCRNN [19], AGCRN 44) STGNNs (GWNET (40). STGCN (44) for single task, and MTGNN (39,, STEP 33
PromptST
48 for multiple tasks.
7

---

Table 1: Performance comparison on three datasets. Best results are bold and the second best are
underlined.
Datesets
NYC
SIP
Chicago
Methods
Metrics
Crowd In Crowd Out
Taxi Pick
Taxi Drop
Traftic Flow
Traffic Speed
Taxi Pick
Taxi Drop
Risk
MAE
17,5289
19.5667
10,8188
9.6142
12,5326
0,7044
3.0624
2,5793
1.1174
DCRNN
MAPE
0.5939
0.5695
0,4330
0,4818
0.2455
0.2686
0.4237
0.4816
0.2504
AGCRN
MAE
11,5135
13.1569
7.0675
6,0066
15,8319
0.6924
2.3542
2,0884
1.1183
MAPE
0.5094
0.4733
0,3753
0,3665
0.2926
0,2744
0,4092
0.4046
0.2505
MAE
11,4420
13.2992
7.0701
6.1171
13.0529
0.6900
2.3671
2.0434
1.1197
GWNET
MАPE
0.4778
0.6171
0,3713
0,3514
0.2483
0,2655
0.3912
0.4044
0.2514
STGCN
MAE
11.3766
13.3522
7.1259
5,9268
15.3501
0.7111
2.3781
2.1427
1.1184
МАРE
0.5018
0,4318
0.3234
0.3339
0.3041
0.2660
0.40774
0.4331
0.2507
MAE
11.3414
13.3
7,0662
6.0912
13,0368
0,6952
2,3663
2.0316
1.1182
GMAN
MAPE
0.4782
0.6065
0,3652
0.3468
0.2464
0126678
0.3953
0.4036
0.2516
ASTGCN
MAE
14.2847
17.1582
9.1430
7.7063
16,4896
0.6980
2,5091
2.1520
1.1175
MАPE
0.6396
0.5922
0.4607
0.4524
0.3104
0,2682
0.4593
0.4413
0,2502
STTN
MAE
12.1994
19,1966
7.6716
6,3816
15.1751
0.6939
2.2996
2.0355
1.1214
МАPE
0.4757
0.4744
0,3600
0,3763
0,2881
0,2625
0.3893
0.4133
0.2518
MTGNN
MAE
11,4350
13,3072
7.0736
6.1162
13.0486
0.6989
2.3692
2.0361
1.1201
MAPE
0.4785
0.6185
0,3782
0,3502
0,2475
0.2687
0.3979
0,4073
0.2578
STEP
MAE
11.2328
13.1043
6,9619
5.9101
12,0032
0.6970
2.3592
2.0168
1.1190
MAPE
0.4537
0.4361
0,3248
0.3379
0,2391
0.2638
0.3914
0,4019
0,2507
MAE
11.00,66
13.0237
6.8711
S.8797
11.8620
0.6921
2.3576
2.0065
1.1186
PromptST
MAPE
0.4465
0.4358
0,3265
0,3382
0.2375
0,2632
0.3913
0.4012
0.2511
CMuST
MAE
11.1533
12,9088
6.7581
5,8546
11.5811
0.6843
2.3264
2.0034
1.1172
МАРE
0,4384
0,4265
0.3118
0,3375
0.2279
0,2585
0,3672
0.4009
0.2503
as well as attention- based models of (GMAN (50. ASTGCN 9.. STTN 42)). The performance metrics are mean absolute error (MAE), and mean absolute percentage error (MAPE), where lower values indicate higher predictive performance.
Implementation details. We partitioned datasets into training, validation, and testing sets with 7:1:2 ratio. CMuST forecasts observations of next 12 time steps based on previous 12 steps, as T= T = 12. All data were normalized to zero mean and unit variance, and predictions were denormalized to normal values for evaluation. For the MSTI, embedding dimensions were date = 24, d. 12, d2 60, and the prompt dimension was 72. Dimensions for self-attention and cross- attention respectively were 168 and 24, with each attention having 4 heads and FFN's hidden dimension was 256. The Adam optimizer is adopted with an initialized learning rate of 1 x 10 and weight decay of 3 x 10-1, where the early-stop was applied. For RoAda, the threshold 5 = 10-6. Our model was implemented with PyTTorch on Linux system equipped with Tesla V100 16GB.
5.2 Performance Comparison
1) Performance comparison among baselines. In Table [ we compared the predictive perfor- mance of our CMuST with other methods across various city datasets. Obviously, our CMuST significantly outperforms other basclines on all datasets across most of metrics and the multitask- based methods improve performance by an average of 8.51% over the singletask- based methods. This result underscores the effective- ness of the cross- atteettttntionn mechanism for decoupling multidimensional de- pendencies, which not only enhances spatiotemporal representational capac- ity, but also enables easy extraction of common correlations among tasks and thus empowering each individual task to benefit from well-extracted common patterns. 2) Robustness in data-scarce scenarios within multi-task framework. We constructed scenarios of data scarcity for specific tasks, i.c., we reduce part of the spatial nodes in prediction of Crowd In, and also reduce number of samples in tempo- ral dimension by expanding time intervals on NYC, to study the robustness of CMuST under the challenging scenario of limited (reduced) data. Results shown in Table inddicate that assimilating common information from other tasks can help better prediction in a single task even though it is with limited data on either spatial and temporal dimension. This demonstrates that multi-task prediction relax the requirement of individual tasks on both data volumes and distributions, where the shared commonality effectively captures and delivers the consistency and diversity among tasks.
Table Performance against data sparsity.
NYC for Crowd In
model
25% nodes
50% nodes
2 times Interval
timess Interval
MAF
MAPE MAE
MAPE
MAE
MAPE
MAE
MAPE
GWNET
13.7648
0.4825
12.4637
20.2547
0.4465
20.6487
0.4958
STEP
13.1827
0.4772
12,2393 0.4612
20,1936
0.4436
20.1465
0.4915
PrompiST
128362 0.4719 12.0361
19.8465
19,5238
0.4872
CMuST
12.1611 0.4506 11.2864 0.4470 18.2925 0.4279 18,4084
0.4797
8

---

5.3 Ablation Study
To assess the effectiveness of each module in CMuST and its capability for multi-task learning. we designed set of variants 1) w/o context-data interaction: remove the spatial-context and -emporal-context cross-interactions in the MSTI module, 2) w/o consistency maintainer: omit the separation and freezing of stable weights during the RoAda phase, instead using all weights for rolling training. 3) w/o task-specific preserver: eliminate the task-specific prompts, thus re- moving the task-specific diversity preservation. Figure 3 supports our hypothesis that CMuST cohesive and integral system, where the results of "w/o context-data interaction" deteriorate, potentially indicating multidimensional interac- tions from context environments to spatial rela- tions and temporal evolution exactly make sense for prediction. The results on "w/o consistency maintainer" highlights the validity of capturing task consistency and commonality to facilitate task learning. Lastly, performances of "w/o task- specific preserver" show that removing unique patterns distinctive from other tasks makes infe- rior results.
3.5
aeo maittain
wircented dute entnuiiinn
2.5
2.0
1.5
0.5
20
Taxi Taxi Drop
Pisk
Taxt
Taxi Drop
Risk
Figure 3: Ablation studies of CMuST on Chicago.
5.4 Case Study
1) Visualizing attention across training phases. In Figure 4(a) we visualize the changes of attention weights CMuST during the stage of RoAda. It is observed that as tasks are learning continuously, the relationships and interactions across various dimensions is becoming distinctive and going stable, demonstrating the consolidation process of dimension-level relations. By modeling weight behavior, such consolidated relations and interactions between context and observations can further enable the extraction of consistency in spatiotemporal interactions across tasks. 2) Performance variation along with task increasing. Figure 4(b) shows the performance of individual tasks on NYC as the number of tasks increases. The performance of each task improving with addition more tasks, which indicates that tasks are no longer isolated but gain the collective intelligence via assimilating common representations and interactive information.
Spa Setiene aodtomncii
39.8
11.30
33.2
NAPONN
11.00
crond Dut
MAPINN
6.0
With Rocda phrase progresing
Tral
(a) Visualizing attention across training phases.
(b) Performance variation along with task in- creasing.
Figure 4: Case studies of our proposed CMuST on NYC.
5.5 Parameter sensitivity analysis
We varied the dimension of the task prompt d, as {18, 36, 72, 144), the number of attention heads in
MSTI as {1. 2, 4,8, 16}, and the threshold 5 for RoAda among {10-4, 4 ,1005 100 6, 1.0-7.. Results shown Figure 6 indicate that the optimal settings were d, 72, head and 6 10- Details of parameter analysis and fine-tuning can be found in our Appendix C.6
Conclusion
In this work, we enable spatiotemporal learning to get rid of isolation with proposed CMuST, which consists of two major components. In CMuST, the MSTI devised to dissect complex multi- dimension data correlations, to reveal disentangled patterns. To extract the task-wise consistency

---

and task-specific diversity, we propose rolling learning scheme RoAda, where it simultaneously models weight behavior to enable collective intelligence, and constructs task-specific prompts by compressing domain data with AutoEncoding to empower task-specific refinement for enhancement. We believe our CMuST can not only help better understand the collective regularity and intelligence in urban systems, but significantly reduce repeated training and improve data exploitation, which is progressively approaching green computing in future cities. For future work, we will further investigate the collective intelligence in open urban systems, which can potentially generalize to wider domains such as energy and environment for human-centered computing.
Acknowledgement
This paper is partially supported by the National Natural Science Foundation of China (No. NN..122770 12227901, No. No.62072427), Natural Science Foundation Jiangsu Province 20240404,, the Project Stable Support for Youth Team in Basic Research Field, CAS (No. YSBR-O05))) and the grant from State Key Laboratory of Resources and Environmental Information System.
References
[1] Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general contin- ual learning method based on complementary learning system. arXiv preprint arXiv:2201.12661 2022.
[2] Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil. Convex multi-task feature learning. Machine learning, 73:243-272. 2008
[3] Muhammad Tayyab Asif, Justin Dauwels, Chong Yang Goh, Ali Oran, Esmail Fathi, Muye Xu, Menoth Mohan Dhanya, Nikola Mitrovic, and Patrick Jaillet. Spatiotemporal patterns in large-scale traffic speed prediction. IEEE Transactions on Intelligent Transportation Systems, 15(2):794-888 804, 2013.
[4] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting, 2020.
[5] Xu Chen, Junshan Wang, and Kunqing Xie. Trafficstream: streaming traffic flow fore- casting framework based on graph neural networks and continual learning. arXiv preprint 1xXv:22006.0623,, 2021.
[6] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1 /929, 2020.
[7] Ting Gong, Tyler Lee, Cory Stephenson, Venkata Renduchintala, Suchismita Padhy, Anthony Ndirango, Gokce Keskin, and Oguz Elibol. A comparison of loss weighting strategies for multi task learning in deep neural networks. JEEE Access, 7:141627-141632, 2019.
[8] Kan Guo, Yongli Hu, Yanfeng Sun, Sean Qian, Junbin Gao, and Baocai Yin. Hierarchical graph convolution network for traffic forecasting. In Proceedings the AAAI conference on artificial intelligence, volume 35, pages 151-159, 2021.
[9] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song. and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages
2019.
[10] Shengnan Guo, Youfang Lin, Huaiyu Wan, Xiucheng Li, and Gao Cong. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions Knowledge and Data Engineering, 34(1 (4(11):5415-5425 2021.
[11] Pradeep Hewage, Ardhendu Behera, Marcello Trovati, Ella Pereira, Morteza Ghahremani, Francesco Palmieri, and Yonghuai Liu. Temporal convolutional neural (ten) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing, 24:16453-16682, 2020.
10

---

[12] Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shouhong Ding, Zhengyang Zhou, and Yang Wang. Hdmixer: Hierarchical dependency with extendable patch for multivariate time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38,
pages 12608-12616, 2024.
[13] Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refine- 2023.
ment. Advances in Neural Information Processing Systems,
[14) Qihe Huang, Zhengyang Zhou, Kuo Yang. Gengyu Lin, Zhongchao Yi, and Yang Wang. Leret: Language-empowered retentive network for time series forecasting. In Proceedings of the Thirty-Thire International Joint Conference on Artificial Intelligence, IJCAI-24, 2024.
[15] Jiawei Jiang, Chengkai Han, Wayne Xin Zhao, and Jingyuan Wang. Pdformer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 4365 4373, 2023.
[16] Zhuoliang Kang, Kristen Grauman, and Fei Sha. Learning with whom to share in multi-task feature learning. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 521 521-528, 2011.
[17] Po-Nien Kung, Sheng- SainggSiang&Paa Yin, Yi-Cheng Chen, Tse-Hsuan Yang, and Yun-Nung Chen. Effi- cient multi-task auxiliary learning: selecting auxiliary data by feature similarity. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 416-428, 2021.
[18] Hyunwook Lee and Sungahn Ko. Testam: time-enhanced spatio-temporal attention model with mixture of experts. arXiv preprint arXiv:2403.02600 2024.
[19] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv: 2017.
[20] Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. Sparsetsf: Modeling long-term time series forecasting with Ik parameters. In Forty-first International Conference on Machine Learning, 2024.
[21] Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, and Xuan Song. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Pruceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 4125-4122 29, 2023.
[22] Xu Liu, Junfeng Hu, Yuan Li, Shizhe Diao, Yuxuan Liang, Bryan Hooi, and Roger Zimmermann. Unitime: A language-empowered unified model for cross-domain time series forecasting. arXiv preprint arXiv:2310.09751, 2023.
[23] Xu Liu, Yutong Xia, Yuxuan Liang, Junfeng Hu, Yiwei Wang, Lei Bai, Chao Huang. Zhenguang Liu, Bryan Hooi, and Roger Zimmermann. Largest: benchmark dataset for large-scale traffic forecasting, In Advances in Neural Information Processing Systems, 2023.
[24] Aurelie C Lozano and Grzegorz Swirszez. Multi-level lasso for sparse multi-task regression. In Proceedings of the 29th International Coference International Conference Machine Learning, pages 595-602, 2012.
[25] Spyros Makridakis and Michele Hibon. Arma models and the box-jenkins methodology. Journal of forecasting. 16(3):147-163, 1997.
[26] Gregory P Meyer. An alternative probabilistic interpretation of the huber loss. In Proceedings of the ieeelevf conference on computer vision and pattern recognition, pages 5261-5269, 2021.
[27] Guillaume Obozinski, Ben Taskar, and Michael I Jordan. Joint covariate selection and joint subspace selection for multiple classification problems. Statistics and Computing, 20:231-252. 2010.
11

---

[28] Zheyi Pan, Zhaoyuan Wang, Weifeng Wang, Yong Yu, Junbo Zhang, and Yu Zheng. Matrix factorization for spatio-temporal neural networks with applications to urban flow prediction. In Proceedings of the 28th ACM international conference on information and knowledge 2019.
management, pages
[29] German I Parisi, Jun Tani, Comnelius Weber, and Stefan Wermter. Lifelong learning of spatiotem- poral representations with dual-memory recurrent self-organization. Frontiers neurorobotics, 12:78, 2018
[30] Ting Kei Pong, Paul Tseng, Shuiwang Ji, and Jieping Ye. Trace norm regularization: Reformu- lations, algorithms, and multi-task learning. SIAM Journal on Optimization, 20(6):3465-35 3489, 2010.
[31] Fynn Schröder and Chris Biemann. Estimating the influence of auxiliary tasks for multi-task learning of sequence tagging tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2971-2985, 2020.
[32] Ozan Sener and Vladlen Koltun, Multi-task learning as multi-ohjective optimization. Advances in neural information processing systems, 2018.
[33] Zezhi Shao, Zhao Zhang. Fei Wang, and Yongjun Xu. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1567-1577, 2022.
[34] Xiaoming Shi, Heng Qi, Yanming Shen, Genze Wu, and Baocai Yin. A attention approach for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 22(8):4909-4918. 2020.
[35] Binwu Wang. Yudong Zhang, Xu Wang, Pengkun Wang, Zhengyang Zhou, Lei Bai, and Yang Wang. Pattern expansion and consolidation on evolving graphs for continual traffic prediction. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2232- 2023.
[36] Kun Wang, Guohao Li, Shilong Wang, Guibin Zhang, Kai Wang. Yang You, Xiaojiang Peng, Yuxuan Liang, and Yang Wang. The snowflake hypothesis: Training deep gnn with one node one receptive field. arXiv preprint arXiv:2308. 10051, 2023.
[37] Kun Wang, Hao Wu, Yifan Duan, Guibin Zhang, Kai Wang, Xiaojiang Peng. Yu Zheng, Yuxuan Liang, and Yang Wang. Nuwadynamics: Discovering and updating in causal spatio-temporal modeling. In The Twelfth International Conference on Learning Representutions, 2024.
[38] Kun Wang, Hao Wa, Guibin Zhang, Junfeng Fang, Yuxuan Liang, Yuankai Wu, Roger Zim- mermann, and Yang Wang. Modeling spatio-temporal dynamical systems with neural discrete learning and levels-of-experts. IEEE Transactions on Knowledge and Data Engineering, 2024.
[39] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Xiaojun Chang. Connecting the dots: Multivariate time series forecasting with graph neural networks, 2020.
[40] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. arXiv preprint arXiv: 001200 2019.
[41] Yi Xin, Junlong Du, Qiang Wang, Ke Yan, and Shouhong Ding- Mmap: Multi-modal alignment prompt for cross-domain multi-task learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16076-16084, 2024.
[42] Mingxing Xu, Wenrui Dai, Chunmiao Liu, Xing Gao, Weiyao Lin, Guo-Jun Qi, and Hongkai Xiong. Spatial-temporal transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.00 002900 2020.
[43] Junchen Ye, Leilei Sun, Bowen Du, Yanjie Fu, and Hui Xiong. Coupled layer-wise graph convolution for transportation demand prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 4617-4625 2021.
12

---

[44] Bing Yu. Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: deep learning framework for traffic forecasting. In Proceedings of the Twenty-Seventh International Joint Conference Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, 2018.
[45] Yuan Yuan, Jingtao Ding. Jie Feng, Depeng Jin, and Yong Li. Unist: A prompt-empowered universal model for urban spatio-temporal prediction. arXiv preprint arXiv:2402.11838, 2024.
[46] Mingyang Zhang, Yong Li, Funing Sun, Diansheng Guo, and Pan Hui. Adaptive spatio-temporal convolutional network for traffic prediction. In 2021 IEEE International Conference on Data Mining (ICDM), pages 1475-1480. IEEE, 2021.
[47] Yu Zhang and Din Yeung. A convex formulation for learning task relationships in multi-task learning. arXiv preprint arXiv::203.88.. 3536, 2012.
[48] Zijian Zhang, Xiangyu Zhao, Qidong Liu, Chunxu Zhang, and Qian Ma. Promptst: Prompt- enhanced spatio-temporal multi-attribute prediction, 2023.
[49] Jie Zhao, Chao Chen, Chengwu Liao, Hongyu Huang, Jie Ma, Huayan Pu, Jun Luo, Tao Zhu, and Shilong Wang. 2f-tp: Learning flexible spatiotemporal dependency for flexible traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 24(12):5379-11531, 2022.
[50] Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. Gman: A graph multi- attention network for traffic prediction, 2019.
[51] Zheng yang Zhou, Hao Liu, Kun Wang, Peng kun Wang, Xu Wang, and Yang Wang. A teacher- student spatiotemporal semi-super method for urban event forecasting (in chinese). Acta Electronicu Sinica, 51 51(12):5557-357) 2023.
[52] Zhengyang Zhou, Yang Wang, Xike Xie, Lianliang Chen, and Hengchang Liu. Riskoracle: minute-level citywide traffic accident forecasting framework. Proceedings of the AAAI conference artificial intelligence, volume 34, pages 1258-1265, 2020.
[53] Zhengyang Zhou, Yang Wang, Xike Xie, Lianliang Chen, and Chaochao Zhu. Foresee urban sparse traffic accidents: spatiotemporal multi- granuuartttuut perspective. IEEE Transactions on Knowledge and Data Engineering, 34(8):3786-3799, 2020.
13

---

A Explanation of Relevant Concepts
A.1 The Concept, Definition and Scope of Multi-task Learning
Actually, our study, various domains correspond to different urban elements collected with different manners in a given city. For instance, in an integrated urban system, it includes taxi demands, crowd flow, traffic speed and accidents. We collect and organize various domain data (urban elements) in city into one integrated dataset. The goal of our work is to explore the integrated intelligence from various domains and enhance learning of each individual urban element. To this end, the concept of multi-task here is to forecast various elements from different domains in an integrated model. Therefore, our work does not target at unifying regression or classification problems, but proposes an integrated model to iteratively establish the common intelligence among different elements and improve generalization for each element learning in succession, thus getting rid of task isolation. Noted that our experiments performed with regression tasks, but it can easily generalize to classification task with shared representations.
A.2 Continuous & Continual learning
In this work, "continuous" is equivalent to 'continual'. The uniqueness of our work refers to a novel continuous task learning in ST community, which collects the integrated intelligence and benefits each individual learning task.
B Methodology Details
B.1 Illustration of Data Representation and Integration
Raw data c 1 THNSt
Observation field
Spatial encoding
prompt
Longindde
Eobe TXW
P PERTANKG
nodes
Laiitide
Temporal encoding
Ei B Tskide
Timestamp
Integrate dimensional representation
Edow
BR PuMMdaw
Day of Week
Data representation and Integration
of Day
Figure 5: This figure illustrates the process of Data Representation and Integration within the CMuST framework. and N denote the length of the time series and the number geographic nodes, respectively. The diagram the right details the encoding of information across different dimensions, with the bottom right corner aggregating the results of all encodings into a unified multidimensional representation.
B.2 Positional Encoding for Temporal-context Cross-interacte intteretttattnnn
Given the D dimensions in total, we define the positional encoding as,
{
(0s().24) = sin (t/100002</D) (0. cos (t/ (t/1000024/D)
for d = 0,... D/2 -
(14)
where and are the indexes of time slots and feature dimensions, which alternately uses sine and cosine to provide unique representation for each time slot.
14

---

B.3 Fusion & Regression
Predictions for future steps are influenced by representations affected by culti-dimensional inter- actions, where the impact of each dimension may vary. Therefore, we integrate information across various aspects effectively by a parameter matrix-bar based fusion strategy,
W. H(..., slice!o)) + W. H(...,, slice!*)] + W. HI..H,,,,liee. slice!")
(15)
where * denotes the convolution operation, employing x I convolution kernels. W., W., and W are the parameters within these kernels, tailored to adjust the influence of observational data, spatial locations, and temporal information on the prediction targets. respectively. We further integrate the task- prompt to culminate in the final prediction Y FC, (ZW= + H slicep) Wp: fy),
slicep/Wpr. the Huber Loss [26] is utilized as the optimization function, which is less sensitive to outliers compared to squared error loss, defined as:
C -
YY YY, 5 - 10),
-
(16)
otherwise
where Y is the ground truth, controls the sensitivity to outliers, which enhances the accuracy by integrating diverse data dimensions and improves robustness against data variability and anomalies.
B.4 Details of Rolling Adaptation
The detailed algorithmic procedure for the RoAda phase can be found in Algorithm
C
Additional Experiment Details
C.1
Dataset Details
We provide detailed information about the dataset, including the number of records in the original data, the number of regions into which the data was divided, and the time intervals. These details are presented in Table 3 The specific data preprocessing is as follows:
We collect yellow taxi trip data from January March 2016 from the NYC Open Data website. Each trip record includes information such as pickup and dropout times, locations, and the number of passengers. We filter out records with abnormal longitude and latitude values or missing data. Then we select data within Manhattan and surrounding areas, divided into 30x15 grids, and counted trips per grid, selecting those with total trips greater than or equal to 1000, resulting in 206 grids. Each grid's data is aggregated into 30-minute intervals, yielding taxi pickup counts, taxi dropout counts, and crowd in/out flows. We also include time day (tod) and day of week (dow) as context, resulting in four tasks with input features [value, tod, dow].
NYC:
SIP:
We collect traffic data from Suzhou Industrial Park from Jamuary to March 2017, comprising tens of thousands of records. The area is divided into nodes, and data is aggregated into 5-minute intervals. After filtering out grids with sparse data, we obtain 108 nodes, each containing traffic speed and traffic flow. We include time of day and day of week as input context, resulting in two tasks: traffic flow and traffic speed, with input [value, tod, dow].
Chicago: We collect taxi trip and accident data from the Chicago Open Data platform for June to December 2023. The taxi data includes trip start, end times and locations. We divide the area into 30x20 grids and select grids with total trips greater than 100, resulting in 220 grids. Similar to the NYC dataset, data is aggregated into 30-minute intervals, yielding taxi pickup and dropout counts, resulting in two tasks with input features [value, tod, dow]. The accident data includes incident locations, times, casualty numbers, and injury severity of each casualty. We then obtain the risk score by weighting it according to each casualty and injury, mapped it to the 220 grids, and aggregated the risk score over time intervals, resulting risk task with input features [risk score, tod, dow).
C.2 Implementation Details and Fairness-aware Experimental Evaluation
To verify whether the multi-task learning in urban systems can compete single-task learning scheme and further show the superiority of our continuous multi-task learning. our experiments are designed
15

---

Algorithm Rolling Adaptation Process
Require: Model M, tasks [T,,T,,.. T2...., TeJ prompts (P(7i) p(T)) ..... P(Tk),, train data
(D(Ti) D ..... D J. learning rate y, threshold d
1: Warm-up Phase:
2: For all parameters in M. the parameters corresponding to their respective names
3: Initialize: Whistories f 0 > Dictionary with keys as names and values as parameters list
4: for each task T, in {T1, T2,... ..,,............................ Tx] do
5: if i == then
6:
Train task for initializing model Load prompt for task Ti: MM:M4 P
7:
((T))
8:
Set learning rate as y
Train M on ((Ti)) until convergence to obtain W (Ti) (!T)) train
9:
10:
Store weights:
Whistories
append
(
parammere
11:
else
12:
Rolling training for task 2 to k
Load prompt for task T;: .M P(T)
14:
Set learning rate as y x 0.01
b Prevent forget during rolling
Train M D(7.) with initial weights w!h 1)
15:
16:
for each epoch j do
17:
Record weights: Whistories
append
namc.ppraameeess
After training, obtain the weights of M as ( > Calculate variances and freeze stable weights
18:
19:
20:
for each parameters' name in M do V. < CALCULLTE_VARRAACEMMM.. VARIANCE [name))
21:
22:
if v Ō then
23:
Freeze the parameter corresponding to the name
Reset Whistories with (!T.)
24:
25: b Train task 1 again since it's not involved with the commonality extraction
26: .M
((T)
27: The remaining steps are similar to lines 14-22
28: Finally, the weights of M are saved as W
29: Task-Specific Refinement Phase:
30: for each task Ti in (Ti ..,..., Tk. do
31: Load weights and prompt M +-
P(T)
32: Set learning rate as 7
33: Fine-tune M on train and save model
into both single-task learning and multi-task learning. We will further clarify how to ensure the fair evaluation and comparisons across different models.
For single-task learning, we take different datasets as individual ones to respectively train and test the model [23] where compared models and our CMuST are respectively and repeatedly trained for times with different random seeds. The average results are reported in our Table. T
For multi-task learning, since existing models have not designed the task-level continuous learning scheme, we align the features of different types of data in the same urban system into a same urban graph, where data features are concatenated for training the model. The learning objectives are followed by corresponding statement each paper. For our CMuST, we implement the continuous and iterative model update described in Sec. 4.4] The fairesss is well-incorpprrted by ensuring the all the data input to each model is equivalent.
The experimental results find that almost all multi-task learning can compete against single-task learning, and our continuous multi-task learning is also superior to other multi-task learning without explicitly capturing the task commonality and diversity.
16

---

Table 3: The detailed information of the three datasets.
City
Task
#Records
Time Span #Regions #Time Steps Time Interval
Taxi Drop
Taxi Pick
01/01/2016-
NYC
30,245k
206
4368
30mins
Crowd
03/31/2016
Crowd Out
Traffic Flow
1,237k
01/01/2017-
SIP
108
25920
5mins
Traffic Speed
307k
03/31/2017
Taxi Drop
3,291k
06/01/2023-
Chicago
Taxi Pick
220
10272
30mins
12/31/2023
Risk
61k
C.3 Avoiding Catastrophic Forgetting
To avoid catastrophic forgetting, we implement several strategies during the RoAda phase. Firstly, we set the learning rate for each task to lc-5. This helps retain more knowledge from previous tasks and prevents the model from over-adjusting to new tasks. By maintaining a low learning rate, the model incrementally learn new information while preserving the stability of previously learned tasks. Additionally, we use task-specific weights for each task, such as task prompts. This method allows to absorb the common features across all tasks while independently preserving and updating task-specific parameters. This approach ensures that when learning new tasks, the model does not forget the knowledge gained from previous tasks, thereby preventing catastrophic forgetting and avoiding overfitting to new tasks.
C.4 Comparison Experiments with Unified Models
We compare with unified spatiotemporal/tiee series learning, such UniST 45 and UniTime [2]. to better show the generalization ability of our model for multi-task learning in the same urban system. The results of these [5
Table 4: Explanation of table symbols of Table and Table 0
The datasets are numbered by D23 respectively, and the specific tasks the dataset are numbered by Roman numerals III and so on.
Dataset
Task
Notation(Dataase(task)
NYC
Crowd
NYC
Crowd Out
O/II
NYC
Taxi Pick
I/III
NYC
Taxi Drop
D/IV
SIP
Traffic Flow
2/1
SIP
Traffic Speed
2/II
Chicago
Taxi Pick
3/I
Chicago
Taxi Drop
3/II
Chicago
Risk
3/III
Table 5:
The results of unified model.
Model/Dataset
//
1/11
//111
D/IV
2/1
2/111
//
3/111
/I11
URiST/MAE
11.3865
13.0762
6.8942
5.8804
11.7461
0.6985
2,3551
2.0134
1.1186
UniST/MAPE
0.4610
0.4478
0.3261
0.3490
0.2465
0.2661
0.3916
0.4162
0.2508
UniTime/MAE
12.2874
14.9120
7.4723
6.4641
13,9172
0.6993
2.4564
2.0341
1.1292
UniTime/MAPE
0.4721
0.4760
0.3671
0.3719
0.2965
0.2713
0.3987
0.4254
0.2511
17

---

C.5 Experiments for Cold-start
We have designed the experiment of cold start. Specifically, for NYC dataset, we selected three of the four tasks of Crowd In, Crowd Out, Taxi Pick and Taxi Drop in turn for training, and calculated the adaptation time and results for the remaining one task on this basis, comparing with training single task alone. A similar design is applied for SIP and Chicago datasets. The results are shown in Table 6 which show that, both in terms of effect and time, it performs better than single task, indicating that our model adapts to the newly arrived task more quickly and well, which is conducive to solving the problem of cold start of urban prediction.
Table 6: Experiments for cold start.
Type/MetrieDDtasse
D/I
O/III
D/IIII
D/IV
2
2/II
/
VII
/IIII
Single Task/MAE
11.2457
13.1284
6.9357
6.0122
11,8684
0.6912
2.3317
2.0223
1.1175
Single Task/MAPE
0.4623
0.4782
0.3453
0.3531
0.2758
0.2613
0.3983
0.4023
0.2506
Adaptation Time (s)
2.132
1,344
1540
1,545
4,272
4,525
3,301
4,504
4,313
Cold Start/MAF
11.1611
13.0027
6.8032
5,9834
11.7832
0.6901
2.3278
2.0089
1.1173
Cold Start/MAPE
0.4407
0.4323
0.3225
0,3463
0.2469
0.2598
0.3906
0.4013
0.2504
Adaptation Time (s)
1,571
1,150
1,322
1,441
3,019
4,053
2,886
2,736
4,198
C.6 Parameter Sensitivity Analysis
To study the impact of hyperparameters on model performance, we varied the dimension of the task prompt dp as {18, 36, 72, 144), the number attention heads in MSTI as {1, 2, 4. 8, 16). and the threshold for RoAda among (10-4, 10-5 10
10 10-7). The experimental results, displayed in Figure are based on analysis of three datasets. 1) Initially, as the dimension of the prompt increases, both MAE and MAPE decrease, indicating that within a lower range, increasing dp benefits the model by capturing more task-specific information. However, performance significantly deteriorates after dp exceeds 72, suggesting that additional dimensions may carry redundant and non-informative elements that introduce noise and prevent the model from effectively learning useful information. 2) On the other hand, as the number of heads increases, model performance initially improves, allowing the attention mechanism heads. Yet, this positive effect reverses after exceeding four heads, indicating that too many heads can lead to information overlap, reducing efficiency and adding unnecessary complexity to the model, thereby degrading performance. 3) During the rolling adaptation phase, both excessively high and low values for the threshold lead performance degradation. When 5 is set too low, it results in fewer stable weights being retained, which may fail to preserve essential information critical for learning stability. Conversely, setting ô too high results in more weights being frozen, which restricts the model's ' flexibility and adaptability, locking in potentially dynamic information and preventing the model from adapting novel and evolving data patterns effectively.
capture more diverse and dimensional information through different
D Others
D.1 Limitation
Our current work confined to single urban system and our experiments are solely within the transportation sector. The exploration into other domains within the same city, such as electricity, transportation, and pollution multi-task learning still limited. Addressing these limitations will form the basis of our future research.
18

---

MAE Promet Dimersion
MAPEINI Pronept Dimension
HAE vs Attertion Heads
HAPEING NU vs Attertion Heads
MAC Detaa
MAPEINI Deta
Croud
Crand
42.1
Drop
Crowdl Out
Cried but
Croed Out
Crowd out
Cread Out
Drop
144
Parameter sensitivity NYC.
MAF Proreet Dinemiion
Dimension Speed
Atteettion
Aftertionn
Trlficc
Traffic
buffic Speed
Trafic Speed
bults Speet
2
Vokce Poongt Dinemson
Value Pronngt Dimersiee
Number noads
of Heads
Values
Detaa Values
(b) Parameter sensitivity on SIP.
Promet Dimension
MAPEINI Pronppt Dimension
Attention Heads
MAPEEN) Attertion Heeds
Delta
MAPEUNI
4.0
Mak
Roak
Posk
3-
2.0
3.5
25.0
144
Premet Dimenaie
Number
1011
15
(c) Parameter sensitivity on Chicago.
Figure 6: Parameter sensitivity analysis of CMuST.
19

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: In the introduction (Section we summarized the contributions of this paper, which include the technical development of two modules: MSTI and RoAda, and the construction of benchmark datasets for three cities. Through extensive experimentation, we demonstrated the advantages of our CMuST framework in multi-task learning under conditions of limited data.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discussed the limitations of our paper and the areas requiring further improvement in Appendix D.I
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
20

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: This paper does not include theoretical assumptions and results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the suppiemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: We provided detailed description of our model architecture, datasets, and implementation methods. At the same time, in order to facilitate reproduction, we also provided source code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
21

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
We added the source code to an anonymous repository and provided the source of the dataset.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/nips. wwt.cc/pcccpcpcccceooooooop gwwcc//guuee///eee /wweddeeeooooebbmicccceee setbbmmsssinnn swinnnossiiiyccyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental settings are presented in Section 5.1 Full details for fairness-
aware experimental evaluation are presented in Appendix C.2
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification; For all experimental results in this paper we provided the mean the numerous experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
22

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτor rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provided basic information about our deployment platform and computing power in Section S.I
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://///nnnppp..ppppppt//ttttt
Answer: [Yes]
Justification: We make sure our research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., - if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed the potential positive societal impacts of our paper in Section 6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
23

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There are no such risks of the datasets this paper.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have properly cited the original papers and web pages that produced the code package and dataset.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
24

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our code has been provided in anonymous link.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
25

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "qFDtIU4dML", "reviewer": "Reviewer_YiFX", "summary": "This work proposes a multi-task spatiotemporal learning framework that helps the model understand the relationships between multiple tasks. The specific contributions lie in proposing MSTI to model the multidimensional spatiotemporal data and RoAda to capture the commonality and personalization among multiple tasks.", "questions": "1.\tIs the author using \"continuous learning\" to replace the commonly used \"continual learning\" in the community to indicate the uniqueness of this work?\n2.\tDid the author train three models on three datasets, or train only one model and continuously update it on three datasets? In other words, the author provided definitions for temporal increment, spatial increment, and feature increment in Definition 3, but in the actual work, was the increase in spatial nodes ignored?", "limitations": "The limitations have been addressed.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1.\tThe author attempts to construct a spatiotemporal model for continuous multi-tasks, which is an attractive motivation with development potential.\n2.\tRoAda provides an executable technical solution for spatiotemporal continuous learning.", "weaknesses": "1.\tThe author mentions multi-task and multi-domain problems several times in the introduction, but these concepts are not intuitively introduced in the paper. Multi-task and multi-domain do not always have a unified consensus in the ST community. For example, does multi-task include regression tasks and classification tasks? Does multi-domain refer to different cities or different modes of transportation? The author should provide more specific scopes for these terms in the introduction.\n2.\tThe author's method of handling ST dependencies is not novel. Using cross-attention techniques to model from the perspectives of temporal dimension, spatial dimension, and spatiotemporal relationships separately is a common processing paradigm in the ST community. The contribution of MSTI can be considered overstated.\n3.\tThe research on ST prediction and continual learning in the related work section is insufficient, lacking analysis of advanced ST prediction models and continual learning models in recent years.\n4.\tThe author argues that continual learning can help spatiotemporal models enhance generalization ability. The theory and experiments in the paper are insufficient to support this argument.\n5.\tThe author neglects experiments on cold start problems.\n6.\tThe comparative experiments did not achieve the best results on all tasks; reasons for this should be analyzed.\n\nOverall, I believe this work proposes a very attractive challenge, but in the end with the old problem, it only solves a standard problem, i.e. the ST multi-task problem. The author proposes Rolling Adaptation to solve this problem, which is a contribution that cannot be ignored. However, this work is incomplete for many issues mentioned in the introduction are not addressed, the definition of tasks is confusing at the beginning, and there is a lack of a pseudocode algorithm to help readers understand the proposed method more accurately.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer YiFX,\n\nThank you for your detailed and insightful feedback. We have carefully addressed each concern below.\n\n**W1. Concept, definition and scope of Multi-task learning.** Various domains correspond to different urban elements in a given city, and the concept of multi-task is to forecast various urban elements in a same neural network. (More details in **Common issue 1)**.\n\n**W2. Contribution of MSTI.** Previous attention-based spatiotemporal learners often process spatial and temporal aspects respectively. Different from those, MSTI designs cross-dimension attention, allowing flexible decomposition and spatial-temporal cross interactions. (More details in **Common issue 2**)\n\n**W3. Related work review.** TrafficStream [1], PECPM [2] on ST learning, and CLS-ER [3] on task-level continuous learning are investigated in our paper.  We supplement some more related works as below.\n\n1. *ST learning:* DG2RNN [4] designs a dual-graph convolutional module to capture local spatial dependencies from both road distance and adaptive correlation perspectives. PDFormer [5] designs a spatial self-attention and introduces two graph masking matrices to highlight the spatial dependencies of short- and long-range views. TESTAM [6] uses time-enhanced ST attention by mixture-of-experts and modeling both static and dynamic graphs. These solutions focus on accuracy and generalization but neglect significance of continuous learning and fail to capture commonalities across different dynamic elements for collective intelligence.\n2. *Detailed continuous learners:* C-LoRA [7] is low-rank adaptive by continuous self-regularization in cross-attention layer with stable diffusion. Kang et al. [8] develop a customized dirty label backdoor for online settings while maintaining high accuracy. COPAL [9] continuously adapts new data through a pruning approach guided by sensitivity analysis. These researches predominantly centers on the same task. Even CLS-ER addresses different tasks, it still focuses on image classification, trapping into little exploration on task-level streaming data. To model various aspects of commonality, MSTI and RoAda are proposed respectively. Therefore, the proposed task and solution are both novel to ST learning community.\n\n[1] TrafficStream, IJCAI'21\n\n[2] PECPM, KDD'23\n\n[3] Learning Fast, Learning Slow, ICLR'22\n\n[4] DG2RNN, TITS'24\n\n[5] PDFormer, AAAI'24\n\n[6] TESTAM, ICLR'24\n\n[7] C-LoRA, TMLR'24\n\n[8] Poisoning Generative Replay in Continual Learning ...., ICML'23\n\n[9] COPAL, ICML'24\n\n**W4. Generalization experiments and theory.**\n\n1. *Experimental Evidence:* We have conducted generalization tests in Tab. 2 of Sec.5.2 and Fig. 4(b) of Sec. 5.4. Tab. 2 shows performance variations when node number reduced on NYC, indicating the robustness of CMuST. Fig. 4(b) shows  performance changing with number of input  tasks, indicating that task learning benefits from collective intelligence through assimilating common representations and interactive information, supporting the enhanced generalization in continual learning. Additional task-level cold start experiments are added to validate CMuST (**Common issue 3**).\n2. *Theoretical Perspective:* It can be analyzed from uncertainty and information theory. First, introducing more diverse  samples and iteratively repeating model training can reduce the epistemic uncertainty and increase the experience of models [10,11]. From information-theory aspect, continual learning allows the model to maintain useful common information and dynamically updates the model with new data, increasing the mutual information across task-level observations [12,13]. By learning multiple related tasks, the perceived knowledge of model is expanded via increasing patterns and dependencies, leading to enhanced generalization [14,15].\n\n[10] Aleatoric and epistemic uncertainty in machine learning, MachLearn'21\n\n[11] SDE-Net, ICML'20\n\n[12] A Comprehensive Survey of Continual Learning, TPAMI'24\n\n[13] Graph information bottleneck, NeurIPS'20\n\n[14] Improving robustness to model inversion attacks via ..., AAAI'21\n\n[15] Incorporating neuro-inspired adaptability for continual learning ..., NMI'23\n\n**W5. Cold-start issue.** Generalization capacities have been empirically validated in Sec 5.2 and Sec. 5.4, where Tab. 2 can be viewed as imitating the cold-start issue on spatial dimension. (Task-level cold-start experiments are added in **Common issue 3**)\n\n**W6. Analysis of comparative experiments.**  1) CMuST achieves overall good performances with most best results, and only 4 out 18 achieve the second best, showing the superiority against all baselines. 2) Baseline models are usually designed from specific tasks and datasets, then they tend to be tailored and tuned for the specific data and tasks, e.g., PromptST is designed on NYC, thus obtaining best performances on NYC. To this end, baseline model tends to individually achieve best while CMuST achieves overall best results. We will incorporate such discussions into our manuscript.\n\n**Q1. Continuous & continual learning.**  'Continuous' is equivalent to 'continual'. The uniqueness of our work refers to a novel continuous task learning in ST community, which collects the integrated intelligence and benefits each individual learning task.\n\n**Q2. Model training and working details.** Given each dataset with different urban elements, we trained separate models for each city (dataset). Regarding the increment on spatial domain, we have conducted the generalization experiments on Tab.2 by node masking. The results suggest that CMuST can ease the data requirements of single task by capturing and exploiting commonalities and diversity among tasks.\n\n**Other. The pseudocode of RoAda.** We have added pseudocode of RoAda to global response **PDF**.\n\nBased on your suggestions, we are revising the manuscript to satisfy the high standards of the NeurIPS community. If you have further questions, please feel free to discuss with us.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FM80PvM78t", "reviewer": "Reviewer_SHU9", "summary": "This paper proposes a Continuous Multi-task Spatio-Temporal learning framework (CMuST) to enhance urban intelligence. CMuST introduces a Multi-dimensional Spatio-Temporal Interaction network (MSTI) for capturing complex data interactions and a Rolling Adaptation training scheme (RoAda) to iteratively update the model, simultaneously maintaining task uniqueness and leveraging shared patterns across tasks. The framework is validated through extensive experiments on datasets from three cities, demonstrating superior performance against existing methods.", "questions": "1. In your CMuST, how can you avoid catastrophic forgetting during task rolling adaptation?\n\n2. Do more comparison experiments with SOTA baselines will be better.", "limitations": "The limitations are discussed.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "S1. Well presentation. This paper is well-presented and well-organized, providing a clear and comprehensive overview of the proposed methods and their implications.\n\nS2. Good significance. The  proposed CMuST can jointly model different tasks of spatiotemporal forecasting  within the same spatiotemporal domain. This approach not only reinforces individual correlated tasks from a collective perspective, but also helps  understand the cooperative mechanism within the dynamic spatiotemporal system. \n\nS3. Sufficient and qualified technical contribution. The contribution and innovation of MSTI network lies in that effectively dissecting interactions across multiple data dimensions for improved spatiotemporal representation and commonality extraction, and RoAda training scheme  for ensuring model  to adapt to new tasks and continuously learn commonality and personalized patterns. The coupling of these two major components can well contribute to the ST learning field. \n\nS4. New benchmark construction and good experiment designs. The construction of benchmark datasets for three cities enriches the research field and provides a solid foundation for evaluating the framework performance. Extensive experiment designs including robustness in data-scarce scenarios, visualized attention scores, and performance variation with task increasing, demonstrate the framework's superiority in enhancing individual tasks with limited data and providing insights into task-wise continuous learning.", "weaknesses": "1. In Section 4.4, there is missing detailed description on how to avoid catastrophic forgetting during task rolling adaptation. It would be beneficial if the authors could provide more experimental details in this regard.\n\n2. Lacking comparison baselines. More baselines which are argued for unified spatiotemporal/time series learning, such as UniST, UniTime should be added for comparisons.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer SHU9,\n\nThank you for your valuable feedback and for recognizing the contributions of our work. Your insights are greatly appreciated and will help us further improve the quality of our manuscript.\n\n**W1&Q1. Avoiding catastrophic forgetting.** Actually, to avoid catastrophic forgetting, we implement several strategies during the Rolling Adaptation (RoAda) phase. Firstly, we set the learning rate for each task to 1e-5. This helps to retain more knowledge from previous tasks and prevents the model from over-adjusting to new tasks. By maintaining a low learning rate, the model can incrementally learn new information while preserving the stability of previously learned tasks. Additionally, we use task-specific weights for each task, such as task prompts. This method allows us to absorb the common features across all tasks while independently preserving and updating task-specific parameters. This approach ensures that when learning new tasks, the model does not forget the knowledge gained from previous tasks, thereby preventing catastrophic forgetting and avoiding overfitting to new tasks.\n\n**W2&Q2. More comparison baselines.** We appreciate your feedback and agree that comparing against unified spatiotemporal/time series learning (such as UniST [1] and UniTime [2]) will better showcase the generalization capabilities of our model for multi-task learning within same urban system. In response to your suggestion, we have conducted additional experiments comparing our CMuST model with UniST and UniTime. The results of these comparisons are as follows:\n\n| **Model/Dataset** | **①/Ⅰ** | **①/Ⅱ** | **①/Ⅲ** | **①/Ⅳ** | **②/Ⅰ** | **②/Ⅱ** | **③/Ⅰ** | **③/Ⅱ** | **③/Ⅲ** |\n| ----------------- | ------- | ------- | ------- | ------- | ------- | ------- | ------- | ------- | ------- |\n| UniST/MAE         | 11.3865 | 13.0762 | 6.8942  | 5.8804  | 11.7461 | 0.6985  | 2.3551  | 2.0134  | 1.1186  |\n| UniST/MAPE        | 0.4610  | 0.4478  | 0.3261  | 0.3490  | 0.2465  | 0.2661  | 0.3916  | 0.4162  | 0.2508  |\n| UniTime/MAE       | 12.2874 | 14.9120 | 7.4723  | 6.4641  | 13.9172 | 0.6993  | 2.4564  | 2.0341  | 1.1292  |\n| UniTime/MAPE      | 0.4721  | 0.4760  | 0.3671  | 0.3719  | 0.2965  | 0.2713  | 0.3987  | 0.4254  | 0.2511  |\n\n(The symbols in the table are explained in the attached **PDF** of the global response.)\n\nThe results of these additional comparisons will be incorporated into the next version of our manuscript, providing a thorough evaluation and demonstrating the practical benefits and advancements introduced by our approach.\n\nThank you once again for your valuable feedback. Your suggestions and comments will significantly enhance the quality of our manuscript, and we are committed to making the necessary improvements to meet the high standards of the NeurIPS community.\n\n[1] UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction, SIGKDD, 2024\n\n[2] UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting, WWW, 2024\n\nAuthors of Paper 2077", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hEEthveaF4", "reviewer": "Reviewer_GaXf", "summary": "This paper proposes a Continuous Multi-task Spatio-Temporal Learning Framework CMuST to facilitate the task-level cooperation in spatiotemporal predictions (mainly for traffic related tasks). The model is composed of three components . Data representation and integration module processes and standardizes diverse urban data into a harmonized format. MSTI modules reduce the complex interactions within spatiotemporal data. RoAda modules iteratively captures the task-wise consistency and task-specific diversity.\n\nThis study is generally fine with a relatively novel method for spatiotemporal multi-task learning through prompting (although prompting studies in this field is being rapidly developed). The main contribution is how the prompting is handled. The main problem I found is that the text is not that easy to follow with many jargons, coined (complicated) phrases, e.g., continuous multi-task spatiotemporal learning is a bit confusing -  is it something related to continual (lifelong) learning?", "questions": "Q1: The definition of \"domain\" in the paper appears somewhat ambiguous. Please clarify whether \"domain\" refers to different types of tasks (such as \"pick up\" vs. \"drop off\") or different geographical regions (such as \"NYC\" vs. \"SIP\").\nQ2: Please explain the distinction between the symbols \"H[. . . ,slice(s)]\" and \"Es\". Currently, these two parts appear to belong to the same content.\nQ3. For different tasks in the same city, are the input features the same?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "S1: The paper provides open access to data and code.\nS2: The paper introduces the first continuous multi-task spatio-temporal learning framework for joint modeling of tasks within the same spatio-temporal domain, which is generally a novel method.\nS3: The paper validates the proposed model on multiple datasets and tasks, demonstrating its generalization ability.", "weaknesses": "W1: During the model construction process, the paper does not clearly address the inconsistency issue of feature C across different task data. It is recommended to provide detailed explanations either in the data preprocessing stage or within the \"Data representation and integration module\" of the model.\nW2: For continuous task rolling, specific operational details of each task model from training to convergence (such as the number of epochs) are not mentioned in the paper. \nW3: Typos such as \"Compressedd\" (line 235).\nW4: Modify proprietary terms that lack detailed explanations, such as \"PECPM\" on line 124.\n\nOverall, the paper falls short in its presentation. Hope that it could be revised to ease the understanding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer GaXf,\n\nThank you for your thoughtful review and acknowledging the potential and contribution of our work. We appreciate your insightful comments, which have provided us with an opportunity to refine our manuscript and address critical aspects that will enhance the clarity and impact of our research.\n\n**W1. Feature inconsistency across Tasks.** Actually, the tasks we collected have consistent features ($C=3$: numerical value, time of day, day of week).  The investigated space and interval units are standardized, and context factors are mapped into same dimension with an MLP to aviod the diverse dimensions of raw contexts. The detailed data preprocessing process can be referred to the **Common issue 4** of the global response. On the other hand, if another task with inconsistent features, we will use task-specific MLP for observation encoding and a task-specific prediction head to transform features into consistent spatiotemporal interaction (MSTI), which is corresponding to unique prompt for each task. This approach ensures the unique features of each task to be handled  uniformly and effectively, thus the shared spatial and temporal encoders can enhance spatiotemporal representation during rolling training.\n\n**W2.** **Task rolling details.** For the rolling training of each task, we set a maximum of 100 epochs, as this number is based on observations from our training logs, where most tasks typically converge around 90 epochs. Additionally, we maintain a learning rate of 1e-5 during rolling training to prevent catastrophic forgetting. We will include these specific operational details in the revised manuscript to provide a clearer understanding of this process.\n\n**W3 & W4. Presentation issue.** Appreciate your careful reading, we will check thoroughly and correct the typos in the whole paper and modify the proprietary terms with detailed explanation.\n\nSpecifically, PECPM refers to Pattern Expansion and Consolidation on Evolving Graphs for Continual Traffic Prediction [1] proposed in SIGKDD 2023.\n\n[1] Pattern Expansion and Consolidation on Evolving Graphs for Continual Traffic Prediction, SIGKDD, 2023\n\n**Q1. Definition of domain.** Actually, in our study, various domains correspond to different urban elements collected with different manners in a given city. For instance, in an urban system, it includes diverse elements such as taxi demands, crowd flow, traffic speed and accidents. We collect and organize various urban elements in a city into one integrated dataset. The goal of our work is to explore the integrated intelligence from various domains and enhance learning of each individual urban element. To this end, the concept of multi-task here is to forecast various elements from different domains in an integrated model. \n\n**Q2. Explanation of formalization.** Initially, $E_s$ represents the spatial embeddings at the beginning of the model's operations, and $H[..., slice(s)]$ refers to the spatial segment within the tensor $H$ that initially matches $E_s$. As the model processes, $H'[..., slice(s)]$ evolves to reflect these updates, making it distinct from the initial embeddings $E_s$ , so we adopt a uniform slice representation. We will clarify this issue and the dynamic nature of these embeddings in our revised manuscript.\n\n**Q3. Input features across tasks.** For different tasks within the same city, the input features can vary specific to each task, though they share the same dimensionality $C$. Currently, the tasks we have collected features with an input dimension $C=3$, which includes numerical values, time of day, and day of week as answered in **W1**. In our future work, we plan to collect more diverse datasets and conduct further experiments with various types of input features with different dimensions $C$ to accommodate broader ranges of urban tasks.\n\nWe again show our great appreciation of your valuable efforts on our work. We will comprehensively take you and all other reviewers comments into consideration and try our best to polish our manuscript for satisfying the high-level requirement of NeurIPS community.\n\nAuthors of Paper 2077", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hImQevKziR", "reviewer": "Reviewer_xkVJ", "summary": "This work proposed a novel spatiotemporal learning framework CMuST. In CMuST, MSTI is devised to dissect complex multi-dimension data correlations, to reveal disentangled patterns. And RoAda is proposed to extract the task-wise consistency and task-specific diversity. In addition, this paper introduce a benchmark of three cities for multi-task spatiotemporal learning, and empirically demonstrate the superiority of CMuST via extensive evaluations on these datasets.", "questions": "Please explain each weaknesses.", "limitations": "This paper adequately addressed the limitations.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 1, "strengths": "1. The paper achieves the best or second-best results in most experiments, validating the feasibility of the proposed method.\n\n2. The paper proposes a continuous learning mechanism that enables the model to continuously learn from tasks. As claimed by the paper, it is \"the first continuous multi-task spatiotemporal learning framework, CMuST, to jointly model learning tasks in the same spatiotemporal domain.\"", "weaknesses": "1. The paper considers one of its major contributions to be the proposal of a benchmark. However, after reviewing the code, it seems that the authors did not provide details on how they processed the data.\n\n2. The proposed MTSI module largely uses the Attention module from Transformers, but the authors did not provide any references to Transformers. Additionally, using the attention mechanism to capture relationships is a relatively straightforward design and lacks significant innovation.\n\n3. One of the main problems this paper addresses is the cold start problem for new tasks. However, the paper still involves task-specific refinement, i.e., training is still required. Perhaps the superiority of the proposed module can be validated by comparing the adaptation time to new tasks.\n\n4. In the experiments conducted by the authors, it can be observed that when the number of tasks is one, the model's performance is not superior to many models. This might indicate that the proposed MTSI module is not sufficiently effective.\n\n5. The authors use a simple layer to obtain task summarization $S$. Compared to common designs in many MAEs, using just a linear layer with an activation function seems somewhat simplistic for MAE.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer xkVJ,\n\nThank you for your meticulous review and insightful comments, which are invaluable in refining our approach and enhancing the quality of our manuscript.\n\n**W1. Data preprocessing.** Thank you for your reminder. We have included the detailed data processing code in our anonymous repository, please check it available, and we explain the preprocessing steps in the **Common Issue 4** of our global response. All datasets and the processed results will be made publicly available on a cloud storage platform after the paper is accepted.\n\n**W2.** **Contribution of MSTI and Transformers references.** In MSTI, our innovation not only lies in designing a new attention mechanism but in how we use it to capture and refine interactions across different dimensions, such as 'spatial aspect - main observation' and 'temporal aspect - main observation' correlations. By coupling with the RoAda process, we enhance capturing  these correlations across multiple tasks, leading to more effective and enriched encoding of spatiotemporal representation over main observations. This multidimensional interaction is specifically designed to exploit the inherent complexities and dependencies in multiple urban elements that standard spatiotemporal models may not fully capture. Additionally, we explain our specific design and the differences from other traditional ST with attention in the **Common issue 2**. Regarding the specific references to the foundational work on Transformers and the attention mechanism, we now have rectified this by including pertinent references to the original works on Transformers by Vaswani et al. [1], as well as other seminal papers that have shaped the use of attention in spatiotemporal learning [2-4], which provides a clearer context for our contributions.\n\n[1] Attention Is All You Need, NeurIPS'17\n\n[2] Learning Dynamics and Heterogeneity of Spatial-Temporal Graph Data for Traffic Forecasting, TKDE'22\n\n[3] STAEformer, CIKM' 23\n\n[4] PDFormer, AAAI'24\n\n**W3. Cold-start problem.** Actually, generalization capacities have been empirically validated in Sec 5.2 and Sec. 5.4, where experiments on Tab. 2 can be viewed as imitating the cold-start issue on spatial dimension. Thanks for your good suggestion, we further add the cold-start experiments concerning tasks, i.e., training on two tasks and testing on another task. The details and results can be found in the **Common issue 3**.\n\n**W4. Performance of single task.** Thanks for your careful reading, we also confirmed this result and found not seriously inferior to other single-task baselines (The comparison  with the baselines have been made over single-task in Tab. 1 of Sec 5.2). Moreover, since in the experiment of Fig. 4(b), the hyperparameters and experimental settings we adopted are based on multi-tasks, investigation over the influence of the task number are also following such multi-task settings to guarantee the fairness of comparisons. This ensures that the only variable is the number of tasks.  Considering learning over individual task, we also believe our backbone, MSTI, can outperform other baselines. To confirm this intuition, we have conducted additional experiments following the individual task setting, (one model for one task), and the testing MAE and MAPE are as follows, \n\n| **Metrics/Dataset** | **①/Ⅰ** | **①/Ⅱ** | **①/Ⅲ** | **①/Ⅳ** | **②/Ⅰ** | **②/Ⅱ** | **③/Ⅰ** | **③/Ⅱ** | **③/Ⅲ** |\n| ------------------- | ------- | ------- | ------- | ------- | ------- | ------- | ------- | ------- | ------- |\n| MAE                 | 11.2457 | 13.1284 | 6.9357  | 6.0122  | 11.8684 | 0.6912  | 2.3317  | 2.0223  | 1.1175  |\n| MAPE                | 0.4623  | 0.4782  | 0.3453  | 0.3531  | 0.2758  | 0.2613  | 0.3983  | 0.4023  | 0.2506  |\n\n(The symbols in the table are explained in the attached **PDF** of global response.)\n\nEven in single-task setting,  comparison with single-task baseline results in Tab. 1, it is observed that  performance of MSTI  is still better than most other single-task models, showing the effectiveness of MSTI. Additionally, the goal of our task is to collective the intelligence across different tasks, and enhance the individual learning, especially improving resilience on extreme scenarios. For detailed evaluation, it include experiments of Sec 5.2, experiments in Sec 5.4, and additional cold-start challenge investigated in our global rebuttal (Results in **Common issue 3**). All these results have demonstrated the success of coupling RoAda and MSTI,  as well as the well-obtained goal of our work.  We will also incorporate these discussion and new results into our revised manuscript. \n\n**W5. Justify the data summarization module and potential improvement.** For this module, we are aimed to extract a snapshot of the data that captures the representative characteristics and typical pattern summarization of the data, so  that it can play the role of task prompt. We utilized a straightforward yet effective MLP with an activation function to capture data snapshots, efficiently reflecting the intrinsic properties of the data. This approach has been empirically validated through ablation studies, demonstrating its capability to capture essential data features effectively. Thanks for your suggestion, and it inspire us for a potential improvement, i.e., we can further introduce contrastive loss, and build a constraint mechanism that is similar among tasks and different across tasks for the abstract representations of different tasks, so as to generate a higher quality task description learner. We are committed to further exploring this improvement and will include the results in revisions of our manuscript.\n\nThank you once again for your thoughtful feedback. We are committed to addressing these issues thoroughly and improving our manuscript accordingly, ensuring it meets the standards expected by the NeurIPS community.\n\nAuthors of Paper 2077", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

Thanks to all reviewers for your meticulous review and valuable feedback. We collated several common questions that identified multiple reviewers and  have compiled detailed explanations and responses to these concerns as follows:

**Common issue 1.(Reviewer YiFX, GaXf)** **The concept, definition and scope of Multi-task learning.** Actually, in our study, various domains correspond to different urban elements collected with different manners in a given city. For instance, in an integrated urban system, it includes taxi demands, crowd flow, traffic speed and accidents. We collect and organize various domain data (urban elements) in a city into one integrated dataset. The goal of our work is to explore the integrated intelligence from various domains and enhance learning of each individual urban element. To this end, the concept of multi-task here is to forecast various elements from different domains in an integrated model. Therefore, our work does not target at unifying regression or classification problems, but proposes an integrated model to iteratively establish the common intelligence among different elements and improve generalization for each element learning in succession, thus getting rid of task isolation. Noted that our experiments are performed with regression tasks, but it can easily generalize to classification task with shared representations.

**Common issue 2.(Reviewer YiFX, xkVJ) The design and technical contribution of MSTI.** Conventional attention-based spatiotemporal learners often process spatial and temporal aspects respectively [1-3]. Different from those, our MSTI designs a cross-dimension attention mechanism, where the dimension indicates the data representation on spatial aspect or temporal aspect. Our MSTI not only considers the self-correlation within spatial dimension, temporal dimension and main observations (e.g., taxi demands, flows), but also the interactions from main observation to spatial representation and main to temporal representation. This design allows flexible decomposition and capturing spatial-temporal cross interactions. Coupling with RoAda, we can flexibly capture the various commonality between spatial-temporal dimensions thus enhancing the continuous learning over each task. We believe this strategy is less-explored, especially for continuous multi-task ST learning.

[1] Learning Dynamics and Heterogeneity of Spatial-Temporal Graph Data for Traffic Forecasting, TKDE'22

[2] STAEformer, CIKM' 23

[3] PDFormer, AAAI'24

**Common issue 3.(Reviewer YiFX, xkVJ)** **Experiments for cold-start and generalization.** We have designed the experiment of cold start. Specifically, for NYC dataset, we selected three of the four tasks of Crowd In, Crowd Out, Taxi Pick and Taxi Drop in turn for training, and calculated the adaptation time and results for the remaining one task on this basis, comparing with training a single task alone. A similar design is applied for SIP and Chicago datasets. The results are shown in Table 2 in the attached **PDF**.

The results show that, both in terms of effect and time, it performs better than single task, indicating that our model adapts to the newly arrived task more quickly and well, which is conducive to solving the problem of cold start of urban prediction.

**Common issue 4.(Reviewer GaXf, xkVJ)** **Preparation of datasets and data processing.**

We construct datasets with multiple tasks for each city. For a given city, we first collect the intensive main ST data and filter out the in-range geolocation indicators (e.g., GPS). Different urban elements  are then aggregated to corresponding valid geographical ranges. 
The space and interval units are standardized. The contexts are mapped into same dimension with an MLP to aviod the diverse dimension of raw contexts.

1. **NYC**: We collect yellow taxi trip data from January to March 2016 from the NYC Open Data website. Each trip record includes information such as pickup and dropout times, locations, and the number of passengers. We filter out records with abnormal longitude and latitude values or missing data. Then we select data within Manhattan and surrounding areas, divided into 30x15 grids, and counted trips per grid, selecting those with total trips greater than or equal to 1000, resulting in 206 grids. Each grid's data is aggregated into 30-minute intervals, yielding taxi pickup counts, taxi dropout counts, and crowd in/out flows. We also include time of day (tod) and day of week (dow) as context, resulting in four tasks with input features [value, tod, dow].
2. **SIP**: We collect traffic data from Suzhou Industrial Park from January to March 2017, comprising tens of thousands of records. The area is divided into nodes, and data is aggregated into 5-minute intervals. After filtering out grids with sparse data, we obtain 108 nodes, each containing traffic speed and traffic flow. We include time of day and day of week as input context, resulting in two tasks: traffic flow and traffic speed, with input [value, tod, dow].
3. **Chicago**: We collect taxi trip and accident data from the Chicago Open Data platform for June to December 2023. The taxi data includes trip start, end times and locations. We divide the area into 30x20 grids and select grids with total trips greater than 100, resulting in 220 grids. Similar to the NYC dataset, data is aggregated into 30-minute intervals, yielding taxi pickup and dropout counts, resulting in two tasks with input features [value, tod, dow]. The accident data includes incident locations, times, casualty numbers, and injury severity of each casualty. We then obtain the risk score by weighting it according to each casualty and injury, mapped it to the 220 grids, and aggregated the risk score over time intervals, resulting in a risk task with input features [risk score, tod, dow].

The detailed process can be found in the code implementation in the anonymous repository.

Authors of Paper 2077

Meta-review: The paper presents a Continuous Multi-task Spatio-Temporal learning framework (CMuST) that introduces a Multi-dimensional Spatio-Temporal Interaction network (MSTI) for capturing complex data interactions and a Rolling Adaptation training scheme (RoAda) to iteratively update the model, simultaneously maintaining task uniqueness and leveraging shared patterns across tasks.The main goal of the work (as summarized by the authors in their rebuttal) is to explore the integrated intelligence from various domains and enhance learning of each individual urban element. To this end, the concept of multi-task here is to forecast various elements from different domains in an integrated model. The framework is validated on data from 3 cities to enhance urban intelligence. 

The paper presents novel technical contributions as summarized by the various reviewers. The benchmark construction and experimental designs are also novel. Also, all reviewers have agreed that the authors have addressed their concerns. 

As the paper presents a novel framework, with a novel continual learning method for spatio-temporal data sets aimed at enhancing urban intelligence, it will be of interest to a larger audience in the domain.
Decision: Accept (oral)