Equivariant Graph Learning for High-density Crowd Trajectories Modeling
Authors: Yang Liu
Zinan Zheng
Yu Rong
Jia Li
Venue: Accepted by TMLR ()

Abstract: Understanding the high-density crowd dynamics of urbanization plays an important role in architectural design and urban planning, preventing the occurrence of crowd crush. Most traditional methods rely on formulas designed based on expert knowledge, which are inflexible and incomplete to model complex real-world crowd trajectories. To address the issue, recent studies propose to simulate crowds via data-driven models. However, these models fail to learn the inherent symmetry of high-density crowd trajectories, leading to insufficient generalization ability. For example, existing models can not predict left-to-right trajectories by learning right-to-left trajectories, even though they share similar patterns. In this work, we propose a novel Equivariant Graph Learning framework for high-density crowd dynamic modeling, called CrowdEGL. It utilizes an additional objective to encourage models to predict the transformed output given the input under the same transformation. We summarize three types of transformation groups, which are determined by the symmetry of environments. To explicitly incorporate these augmented data, a multi-channel GNN is employed to learn the latent graph embedding of pedestrian patterns. Finally, to model dense crowd interactions, future positions of original and transformed inputs are obtained by multiple independent graph decoders. Extensive experiments on 8 datasets from 5 different environments show that CrowdEGL outperforms existing models by a large margin.

Published in Transactions on Machine Learning Research (09/2024)
Equivariant Graph Learning for High-density Crowd Trajec- tories Modeling
Yang Liu
yluukj@@commedt..s..kkk
Hong Kong University of Science and Technology Hong Kong University of Science and Technology (Guangzhou)
Zinan Zheng
shheng@78@ccommeet..uus---s-- edd.cr
Hong Kong University of Science and Technology (Guangzhou)
Yu Rong
you. yyu.rong@hotan com
Alibabo DAMO Academy
Jia Li
jillee
Hong Kong University of Science and Technology (Guangzhou)
Reviewed on OpenReview: https: // operreview. net/ put/forum? id= TeQRze2Zj0
Abstract
Understanding the high-density crowd dynamics of urbanization plays an important role in architectural design and urban planning, preventing the occurrence of crowd crush. Most traditional methods rely on formulas designed based on expert knowledge, which are in- flexible and incomplete to model complex real-world crowd trajectories. To address the issue, recent studies propose to simulate crowds via data-driven models. However, these models fail to learn the inherent symmetry of high-densily crowd trajectories, leading to insufficient generalization ability. For example, existing models can not predict left-to-right trajectorics by lcarning right-to-left trajectories, even though they share similar patterns. In this work, we propose a novel Equivariant Graph Learning framework for high-density crowd dynamic modeling, called CrowdEGT. Tt utilizes an additional objective to encourage models to predict the transformed output given the input under the same transformation. We summarize three types of transformation groups, which determined by the symme- try of enviromments. To explicitly incorporate these augmented data, multi-channel GNN is employed to learn the latent graph embedding of pedestrian patterns. Finally, to model dense crowd interactions, future positions of original and transformed inputs are obtained by multiple independent graph decoders. Extensive experiments on 8 datasets from 5 different environments show that CrowdEGL outperforms existing models by a large margin.
1 Introduction
The increasing population and resulting urbanization have led to a significant rise in crowd density, par- ticularly in urban areas such metro stations, stadiuns, and exhibition centers (Sindagi Patel] 2018). Therefore, understanding high-density crowd dynamics is essential for applications in public safety (Johans- son et al. 2008). Specifically, given the initial states (locations, velocities, etc.) of crowds, to forecast their future state by modeling interactions among pedestrians and the impact of environments. Such forecasting is crucial for crowd management (Xu et al. 2014), architectural design, and urban planning, Otherwise, without proper and sufficient safety measures, the gathering of large crowds in confined spaces can lead stampede incidents, causing injurics and fatalitics. For example, Scoul Halloween crowd crush 2022 killed 159 people and 196 others were injured (contributors/ (2023): In 2023 Freedom City Mall crush, crowds rushing to firework display got stuck in a corridor and killed 10 people (Atuhaire 2023
objective is

---

Published in Transactions on Machine Learning Research (09/2024)
aa) High-density Crowds
(b) Trajectory Equivariance Rotation
Figure 1: (a) Illustratonn of high-density crowds at a 90'crossing, which contains dense interactions; (b) Rotation equivariance of pedestrian trajectory. They enter through the four entrances to reach the opposite exits. Trajectories starting from different entrances are indicated by different colors.
Farly crowd modeling approaches (Yan et al. 2020 rely on formulas designed based on human prior knowledge. Among them, the Boids algorithm (Reynolds 1987) representative model, which simulates crowd dynamics through three fundamental rules: separalion, alignment, and cohesion. Another widely used method is the social force model (Helbing & Molnar 1995] which characterizes interactions among individuals, obstacles, and destinations as different types of forces (repulsive or attractive) and utilizes Newtonian mechanics to update their behaviors. However, these assumptions are incomplete and overly restrictive to model real-world crowd dynamics, which are inherently complex and heterogeneous (Moussaid
et al. 2011). Therefore, recent works (Shi et al. 2023 Zhang et al., 2022; Yu et al., 2023) have employed data-driver models to obtain more realistic crowd trajectorics and demonstrate promising performance. In particular, they treat individuals, obstacles, and destinations their distance. Graph Neural Networks (GNNs) GGimer et al. 2017) are then employed to encode their interactions, and Multilayer Perceptron (MLP) decoder is commonly used to predict the future states of the crowd.
nodes and construct graphs based on
Nevertheless, current GNNs Shi et al. 2023 ficient to capture internal symmetries of the physical world (Cohen Welling 2016 Walters et al.) 2021 Xu et al. 2023). thus they fail to generalize in different directions. Figure displays high-density crowd trajectories in 90% crossing. Pedestrians enter the crossing from four entrances and aim to exit from the opposite exits. In such highly crowded situations, specific rotational symmetries. For instance, if the initial states of pedestrians are rotated by 180", the crowd trajectories would rotate in an analogous way, known as rotation equivariance. With reduced individual free- dom of movement, the interactions among pedestrians, as well as the interactions between pedestrians and environments, are nearly invariant to such rotations. Regardless of this inductive bias, GNNs are inadequate to learn the real crowd dynamics and tend to overfit the observed trajectories (See Figure
Sanchez-Gonzalez et al. 2020 Gilmer et al. 2017 are insuf-
can observe that pedestrian trajectories demonstrate
Although multiple GNNs (Satorras et al. 2021 Han et al. 2022a. Huang et al. 2022 have been proposed to predict physical dynamics with strictly equivariant under a given group action, it remains non-trivial to design symmetric crowd dynamic models because of the following challenges: (1) Dense interactions. In high- density circumstances, crowd interactions occur with great frequency, so that pedestrians are often forced to take detours or even change their intended destinations (Cao et al. 2017); (2) Imperfectly symmetric. Due to the effects of varying human psychology and physiology, they might not react completely the same to ncarby pedestrians and the environment, breaking the symmetry of crowd trajectorics to a certain degree; (3) Finite group actions. Existing equivariant GNNs (Han el. al., 2022b) leverage the complete symmetry of the Euclidean space, ensuring their outputs will rotate/ sitaateetrassatteeeee rrffeett in the same way as the inputs. Such constraint is too strong for crowd dynamic modeling (Dangovski et al.) 2022 Wang et al. 2022). For instance, in Figure I r00'rotation of crowds would violate the boundary condition and thus be invalid.
In this work, we propose a novel Equivariant Graph Learning framework called CrowdEGL, encouraging GNNs to learn more practical imperfect equivariance property. Specifically, it achieves this objective via an equivariance loss that is constructed based on data augmentation. It works as regularization term so that we can control the degree of equivariance. The augmented strategies are defined by cyclic groups (Suzuki
2

---

Published in Transactions on Machine Learning Research (09/2024)
1986). We summarize them for various environments such as corridors and crossings. For example, for the crowd at the crossing, we use the cyclic group of 900rotations to describe its rotation equivariance. Furthermore, to incorporate crowd trajectories that exhibit a high degree of similarity in behavior patterns, we explicitly treat augmented data as imput and propose a multi-channel message- message-passing GNN to learn latent graph representations. Finally, to caplure dense crowd interactions and allow the model to behave distinctly in different directions, multiple independent GNNs are utilized as decoders. Extensive experiments on five different crowded environments demonstrate that CrowdEGL has a better generalization ability over state- of- ahet models and is beneficial for learning the symmetry of crowd dynamics. Our main contributions are summarized as follows:
We emphasize the significance of learning high-density crowd dynamics, which challenging and crucial for applications in human activity and society. We highlight the importance of symmetry for high-density crowd trajectory prediction. To our knowledge, it is the first time equivariance is considered in data-driven crowd simulation.
We propose a novel and flexible equivariant learning framework to incorporate imperfect equivariance in practical crowd trajectories to backbone GNNs, and design a multi-channel GNN to model common pedestrian behavior palterns in different directions.
We conduct extensive experiments to evaluate the performance of CrowdEGL eight datasets of five crowded environments. Experimental results show that it achieves significantly better generalization abil- ity over the state-of-the-ar models. Ablation studies demonstrate the effectiveness of our model designs. To facilitate the research on high-density crowd modeling, we make our datasets and implementation available at Supplementary Material.
2 Preliminary
2.1 Problem Definition
Different from existing studies Alaai et al. 2016 Gupta et al. 2018 Kothari et al. 2022 Shi et al. 2021 Yu et al. 2020 on urban open space (e.g., parks), in this work, we study trajectories in various bounded and narrow environments that have extremely dense interactions. Therefore, we consider a challenging problem setting that focuses on learning interactions of initial crowd states and forccasting their positions after a fixed time interval. Compared to existing works (Shi et al. 2023: Zhang et al.) 2022), we omit historical states and destinations, which are employed to reflect pedestrian intentions but not always available in practice.
Specifically, as shown in Figure 1 we employ discretized obstacles to represent the environments (e.g. walls). Then pedestrians and obstacles can treated as nodes in a graph. At time t, each node i represented by: (1) geometric features including the position vector ax!") E IR and the velocity vector vit) RR; (2) non- geomeeeeetric features such as the node type (pedestrian or obstacle), denoted by us (3) spatial connection with others, where an edge Cij is constructed via distance cutoff and the edge attributes (c.g., node distances) are denoted by aıj. For simplicity, we denote (xx()), v(()) and (u, e = {eij,, a = {an})) as dynamic crowd states and graph features correspondingly. Formally, the crowd trajectory forecasting problem is defined follows:
Definition 2.1 (Crowd Trajectory Forecasting) Given the initial crowd states (ax(t), wl!)) at time t and graph (u,e, e, a), the objeclive is lo predict the subsequend posilion xf (+(+))) where AL is the largel time interval.
2.2 Graph Neural Networks
GNNs Giilmer et al. 2017 Kipf & Welling 2017 Xu et al. 2019 Zhao et al. 2024 Sun et al. 2023: Tang et al.) (022) are neural models specifically designed to learn graph-structured data with complex relationships and dependencies. Recently, they have shown great potential in simulating physical systems
such fluid (Sanchez- et. al. 2020: Li & Farimani
2022 and molecular dynamics (Huang et al.
3

---

Published in Transactions on Machine Learning Research (09/2024)
2022 They achieve this through iteratively propagating and updating information through the nodes and edges. Let ( be the 1-th layer embedding of node and mi denotes the I-th layer message embedding between node and j. The 1-th GNN layer computes:
m; 1 Agg((m () )JEN)), h. =Combinc(h]) (t-1), m
(1)
where Mt collects the neighbors of node i. The common choices of Agg function are Sum or Mean, while that of Combine function is Multi-layer Perceptrons (MLPs). The final prediction is obtained by applying several iterations of message passing. Most existing Equivariant GNNs (e.g., EGNN and GMN) and Graph Neural Simulators such as GNS and CrowdSim are based on such a framework.
2.3 Equivariance
To improve model generalization, we force them to be equivariant, which refers the property where the output of a system changes the same way as the input changes. Formally, equivariance is defined on a specific group (Suzuki 1986):
Definition 2.2 (Group) A group is a set of operations that satisfy: closure, associativity, the eristence of an identity element, and the eristence of inverse elements for each element in the set.
For instance, a widely studied group the Euclidean group, which includes translation, rotation, and reflection. Given group G, the dcfinition of G Equivariance is:
Definition 2.3 (g-Equivariance) A function f X - Y, where X, C R2, is equicariont to group 9, if for any transformation g E 9,
f(goz) = g f(z), I X.
(2)
The symmetry of the real world is not always continuous. We focus on discrete groups within finite elements.
3 Methodology
3.1
Overview
Figure 2 illustrates the overall framework of CrowdEGL. The high-level idca is to learn a latent graph rep- resentation that is sensitive to set of symmetric transformations. To achieve this property, CrowdEGL is additionally trained to predict symmetric transformations of input. As shown in Figure 2 for crowd trajec- tories of corrosinngs, we consider three more proxy tasks that predict 90% 009/180" /180000000000/ 2200/2200rroion of the original input. Although CorwdEGL can be flexibly adapted to various latent GNNs, provide sufficient global environmental information, input. Finally, to distinguish information learned from the original augmented tasks, each task has independent graph decoder to forecast the crowd positions of the next state.
employed a multi-channel GNN to incorporate original and all augmented
3.2
Equivariant Graph Learning
In this section, we introduce how CrowdEGL constructs augmented symmetrical data and learns from them.
Equivariant data augmentation. Data augmentation strategies are the core of learning an equivariant model. In our work, these strategics are described by a group, which represents the symmetry of target environments. Particularly, we focus on eyclic groups, whose definition is follows:
Definition 3.1 (Cyelic Group) A cyclic group is group that can be generaled by repealedly applying single element.
Note that the group is closure (the result of two elements within a group is still an element of the group). For example, the cyclic group of r0'rotation consists of four elements identity and 90º 10//100//// rotttrttt.s.sss...... o Al- though different environments typically have distinct symmetry groups, they can be summarized as following three types:
4

---

Published in Transactions on Machine Learning Research (09/2024)
Input States
Graph Construction
Target Positions
Prediction
Latent
Embedding
Multi-Channel Graph Encoder
Rot. 90
Rot, 90'
Aggregation
Message Passing
Equivarian auivaeiance
180
Rot. 180*
Equivariance
RoL. 270
Rot. 270"
Equlvariance
A Augmented Input
A Augmented Targets
Figure 2: Overview of CrowdEGL framework. Input states and target positions are firstly augmented via group transformations. Then multi-channel graph encoder is employed aggregate multi-channel features and perform message passing. Finally, CrowdEGI. utilizes multiple independent graph decoders to forecast future positions. The prediction loss and equivariance losses are calculated for the model predictions of original and augmented inputs, respectively.
Rotation is used to augment crowd trajectories in environments with central symmetry, including a0a crossingg.
Reflection is used construct augmented data for axisymmetric environments. A common example is crowd dynamics in narrow corridor T-junction, where we employ cyclic group of reflections over the x- y-axis. The intuition is crowd trajectory from left to right would be similar to that of the same crowd from right to left.
Combination of rotation and reflection is used generate augmented data for locations like right- angled corner. In this case, have to combine reflection and rotation obtain the reversed trajectory from the exit to the entrance.
Model training. The objective function of CrowdEGL consists of a prediction loss and an equivariance loss. Given the group G, the model parameters are optimized by minimizing the following loss L:
L L +
Σ Lg
(3)
gEG
where Lp and Lg denote the prediction loss and equivariance loss of transformation |G] is the size of group G. À is a hyper-parameter to control the strength of equivariance loss, larger À leads to a more strict equivariant model. Since our goal is to forecast the future positions of crowds, models are trained to minimize the discrepancy between the exact and approximated positions. Specifically, Let f(;00)) be the backbone GNN with parameters 0g and 0p) be the decoder for the prediction loss. Then Ip is formulated as:
Lp (x) (0;00);;p) (.(0)::0))(((++000 (++20))
where (t+Att) the groundtruth. Here we omit the static input u, e, for simplicity. For equivariance loss, let Mg E 2 be the representation (c.g., rotation or reflection matrix) of the group element g E G, the equivariance loss T. is computed by:

---

Published in Transactions on Machine Learning Research (09/2024)
= [p[(<( Mgs w(t) M2;00);;0 (t+ Mall2-
Note that, depending on the backbone GNN, decoder networks for prediction loss can be shared with that of equivariance losses. During experiments, we found that multiple independent decoders achieve better performance.
3.3 Backbone GNN
To design a backbone under the CrowdEGT, framework, common solution utilize a shared graph encoder and decoder (e.g., GNS Sanchez-Gonzzler al. 2020)) for all original and transformed input. However, the crowds may not be exactly symmetric due to the complex human interactions. Simply treating the augmented inputs original imputs may lead to suboptimal performance. Instead of using hard parameter-sharing strategy, we design a flexible multi-channel pipeline to share partial parameters across tasks and learn equivariance from data. Therefore, if the augmented trajectories are exactly equivariant, the CrowdEGL can degrade to the hard parameter shraringr-sharing strategies, where the parameters of each channel are the same. Specifically, we first construct a graph and initialize the multi-channel node embeddings based on the defined group. Then multiple steps of message passing are performed obtain latent graph embeddings. The final predictions computed via independent graph decoders. We elaborate on the model designs follows.
Graph construction. For personal safety and comfort, pedestrians adjust their trajectories in high-density crowds to avoid collisions with others or obstacles (Yu et al. 2023). Consequently, when they approach each other, pedestrians often instinctively adjust their movement velocities and alter their directions. To model these interactions, we treat all pedestrians and obstacles as nodes and build a radius graph. That is, nodes are connected if the distance between them is smaller than predefined radius r.
Multi-channel node embedding. This subsection investigates how to attain desirable initialization for node embeddings that involve 2D symmetry information. To integrate the global environment information, given the rotation matrix reprosentations My of group element g G, The initial embodding of node í is derived by:
(6) where hig represents the features under the transformation 9, which includes both the original (i.e., identity transformation) and augmented imputs. Φ is a linear embedding layer. is the node type embedding, including the pedestrian and obstacle. I;, v, are position and velocity features, respectively. The velocities of obstacles are set to zero. Then the initial multi-channel node embeddings h obtained by aggregating all the above features:
hig (g, Mg u:),
h (0) Agg((
It is worth noting that the aggregation function should not be permutation invariant (Zaheer et al. 2017) such as Mean or Sun. These functions result in G-invariant GNN and latent embeddings. That is. their outputs remain invariant under imput transformations of group G. Consequently, the model can not distinguish the original from augmented inputs and would map all original and augmented inputs to the same position, instead a Geequivariant model. For CrowdEGL, we use the following aggregation:
n:00 = (hhtgleea))
(8)
where da is an MLP. [higlees the concatenated feature that follows a fixed order of group elements.
Message passing. In high-density crowds, the interactions among pedestrians, and that between pedes- trians and obstacles, are extremely dense and significantly influence their trajectories. Thus the quality of approximating these interactions determines the model performance. We adopt the message-passing frame- work, which has been proven effective in learning such dynamics. Our Message Passing Layer (MPL. takes

---

Published in Transactions on Machine Learning Research (09/2024)
Table 1:
Dataset statisties and symmetry property.
Dataset
#Sample Mean # Pedestrian Max # Pedestrian Target At
Cyclic group
# Elements
Corner
2813
13
31
4
90" Hotation and Reflection
2
T-Junction
5927
34
70
4
Reflection
2
Crossing-90 90"
6581
129
224
20
90" Rotation
4
Crossing-120 120"
10680
132
222
10
120" Rotation
3
Corridor- Uni
17231
81
160
6
Reflection
2
Corridor-Bi Bi.
10405
88
143
10
Reflection
Stadium Gate-Up
7461
97
197
10
Reflection
Stadium Gate-Low
7047
103
198
10
Reflection
the set of node embeddings h9 (h( is initialized via Eq. 8 and edges as input and outputs a transfor-
mation on (141), Concisely, h(+1) MPL(h e). The definition of message embeddings is as follows:
(1-1) m = om (n!-- n
(1-1) a.y),
(9)
where om denotes the MLP. ai are edge attributes that are the distance ||a; æ,112 between node and j. Then the nodes aggregate the message embeddings and update their embeddings follows:
h (1+1) dhn (h ( Σ
m h
(10)
JEN.
Here the node embedding updated via the skip connection. obtain the latent graph embeddings h (L)
After L ilerations of message passing, we
Graph decoder There are two main designs to predict future positions from h(4) First, most existing models Shi et al. (2023) rely on simple MLP as the decoder, ignoring crucial interactions in high-density crowds. Hence, (1) decoded by several message passing layers:
x (t+
x MPL(... (M (M)..(t)))..........))) .)),...)ee)
(11)
The decoder outputs the change of crowd positions and the skip connection of the decoder's last layer is removed due to the dimension. Second, to increase model capacity and allow models to perform differently for prediction and equivariant tasks, we use independent decoders for each task instead of shared decoders.
4 Experiments
4.1 Settings
Methods are evaluated on public pedestrian dynamic datao] that built up by the Institute for Advanced Simulation 7:Civil Safety Research of Forschungszentrum Jülich (Cao et al. 2017). The density of these data is extremely high and their scenarios are specially designed for observing crowd dynamics. Dataset statistics are shown in Table The target time interval is determined by the space size and crowd speed. For all datasets, 70%, 10%, and 20% are randomly split over time for training, validation, and testing, respectively. We use the following five environments: (1) Crossing: crowds start from all entrances and go through a 90 1209rrsssing: (2) Corridor: a crowd goes through a corridor unidirectionally or bidirectionally; (3)Stadium Gate: large number of pedestrians enter or leave the stadium through a gate; (4) Corner: large number of pedestrians move from one side of the corner to the other side; (5)T- Junction: two groups of pedestrians entered from both sides of the T-junction and exited from the middle. Note that our work focuses modeling high-density crowd trajectories where the pedestrian space is severely constrained by here and obstacles. Thus, we do not test our model on traditional human trajectory datasets
Datasets
such as ETH/UCY (Pellegrini et al. 2009 Lerner et al. 2007) and GC Zhou et al. 2012), where crowds
are sparse and more random.

---

Published in Transactions on Machine Learning Research (09/2024)
Table 2:
MSE (in decimeters) of all models. Bold font. indicates the best result and Underline is the
strongest bascline. We report both mean and standard deviation that are computed over runs. Abs. Improv. denoles absolute improvement.
Crossing
Corridor
Stadium Gate
Model
Corner
T-Junction
90 Degree
120 Degree
Unidirectional
Bedirectional
Euter
Leave
MLP
16.359
0.001
14.224 0.001
3.283+ 0.001
5,984 0.001
3,939
0.001
5.622 0.001
1,133+ 0.001
0.962+ 0.001
S-LSTM
13.690+600067
12.451++0.000
2.961+0.003
5.116++0.016
3.148++0.008
4.585++0039
0.943+0.013
0.804+0.004
TransF
12,682+ 0.088
12.352= 0.030
3.061: 0.064
5.123t 0.009
3.106= 0.026
4.515= 0.011
0.924: 0.005
0.785: 0.005
GNS
1.609+ 0.007
2.254: 0.093
0.191 0.009
0.443:: 0.013
0.391 0.001
0.427 0.011
0.100 0.002
0.140:: 0.001
EGNN
3.548+ 0,951
2.900+ 0.284
0.248+ 0.054
0.899+ 0.015
0.496- 0.030
0.612- 0.049
1,258+ 0.008
0.3234 0.001
GMN
3.6684 0.240
3.9574
0.070
0.652: 0.129
0.7891 0.310
0.779= 0.108
0.866= 0.018
0.545:: 0.004
0.2462 0.008
CrowdSim
1.4214 0.043
18115: 0.043
0.167 0.004
0.380 0.019
0.314 0.007
0.032 0.413
0.107: 0.007
0.1133 0.001
SEGNO
2.279=0.839
3.78915.311
0.69::::2234
0 229
0:331=:0.210
0.11520.016
0.513+0.012
0.261
CrowdEGL
0.530#
0.024 0.891
0.638+
0.044
0.056+
0.001
0.121 0.008
0.119
0.010
0.161#
0.009
0.073#
0.001
0.063 0.001
Abs. Improv.
1.177
0.111
0.259
0.195
0.252
0.034
0.050
Basclines We compared CrowdEGL against various basclines including (1) basic MLP; (2) trajectory
forecasting models: S-LSTM (Alahi et al., 2016) and TransF (Giuliari al. 2021); (3) fundamental GNNs:
GNS Sanchez-Gonzalee et al.) 2020 and CrowdSim (Shi et al.
2023); (3) equivariant GNNs: EGNN (Sator-
ras et al. 2021). GMN Huang al. 2022), and state-of-the-art SEGNO
(Liu et al.
2024
Following previous studies (Shi et al. Error (MSE) between model predictions and groundtruth to measure the single-step prediction performance. For multi-step predictions, Average Displacement Error (ADE) and Final Displacement Error (FDE) are utilized as metrics. They are the l2 distance of the predicted whole trajectory trajectory/endpoint /endpoint to the ground truth of the whole jejjectory/endpoiinn
Evaluation metric
2023 Satorras et al. 2021 we use Mean Square
Parameter settings We empirically find that the following hyperparameters generally work well, and use them for all datasets: Adam optimizer with learning rate 0.0005, batch size 100, the hidden dimension 64, weight decay x 10-10, the message passing layer number and the decoder layer number 2. All models are trained for 5000 epochs with an early stopping strategy of 100. For CrowdEGT, the strength À of equivariance loss is turned from 0.1 to with a step size of 0.1. All models are implemented based on Pytorch and PyG library (Fey & Lenssen 2019), trained on GeForce RTX 4090 GPU. The cyclic group used in CrowdEGL can be found in Table
4.2 Performance Comparison
In this section, we compare our model with baselines.
Overall performance Table displays have the following observations:
displays the performance of CrowdEGL and all baselines. From them, we
GNNs significantly purperfor non-GNN models (i.e., MLP, S-LSTM, TransF), indicating the importance of modeling interactions in high- crowd dynamics. Although EGNN, GMN, and SEGNO consider equivariance property as well, they underperform GNS and CrowdSim. These results show that strict Euclidean group equivariant models are too strong to model the crowd dynamics and even have a negative effect. For example, EGNN underperforms MT.P the Corner dataset.
CrowdEGL consistently oulperforms all baselines under all circumstances in a large gap. In particular, CrowdEGL achieves 1.177 and 0.891 lower MSE than the runner-up model CrowdSim. Such improvement verifies the effectiveness of learning a soft cquivariance via data augmentation and multi-channel GNN backbone.
prediction performance We additionally evaluate model performance on multi-step pre- dictions well. Following the previous work Xu et al. 2023 the decoder is slightly modified to perform
Multi-step
8

---

Published in Transactions on Machine Learning Research (09/2024)
1.0
MLP
S-LSTM
TransF
GNS
EGNN
GMN
CrowdSim
SEGNO
CrowdEGL
ADE
0.6
0.4
0.2
Crossing-90
Crossing- 120" Unidirectional
Bidirectional
Enter
Leave
Corner
T-junc
2.0
MLP
S-LSTM
TransF
GNS
EGNN
GMN
CrowdSim
SEGNO
CrowdEGL
1.6
FDE
0.8
Crossing-909 900 Crossing- 120" Unidirectional Ridirectional
Enter
Leave
Corner
T-junc
Figure 3: MSE (in decimeters) of multi-step predictions. CrowdEGL achieves the best, performance.
Observed
CrowdSim
CrowdEGL
Crossing 901
CHNN
Crenchia
SEGNO
MSE
Crossing 1.30
Crossing
Corridor
Figure 5: Generalization experiments. 9rosssiin models are trained tra- jectories from the right and bottom en- trances and tested on those from the left and top. Corridor: models are trained on left-to-right trajectories and lested the reversed direction.
Figure 4: Visualization of CrowdSim and CrowdEGL predic- tion of high-density crowds. Our method produces a crowd iistribution that closer to the groundtruth.
multistep predictions. We set the timestep At to and aim to predict the future 10 steps. The results are displayed in Figure 3 We observe that CrowdEGL consistently outperforms baselines with the lowest ADE and FDE, demonstrating its effectiveness.
Visualization. To gain more insights into the superior performance of CrowdEGL, the density of predicted crowd positions CrowdSim and CrowdEGL are visualized Figure g We choose samples that have a high density. Particularly, the pedestrian numbers are 199 and 217 in the case of Crossing a0 120". According the visualization results, we can conclude that CrowdEGL accurately predicts the distribution of all directions, while CrowdSim only captures the patterns in partial directions. For example, the x-axis of CrowdSim output is similar to the ground truth, but it has obvious difference the y-axis. Such results again demonstrate the effectiveness of our cquivariant training framework.
Generalization ability To further investigate whether the proposed model learns the symmetry of crowd dynamics, we conduct generalization experiment Corridor and 90*Crossing environments. For Corridor, models are trained on left- to-right trajectorics and tested in the reversed direction. Similarly, in 90*Crossing, models are trained on trajectories from the right and bottom entrances and tested on those from the left and top. The results are shown in Figure 5 where the results MLP are omitted since they are too high. Under such a situation, the performance of all basclines decreases by a large margin. In contrast, the test error of CrowdEGL still relatively low level.
Running time To compare the efficiency, we report the running time all methods in the largest two datasets (i.e., Crossing-120 120 and (nndiretttii Table Results are run on GeForce RIX 4090 GPU. From the table, we can find that CrowdEGL has comparable efficiency with CrowdSim.

---

Published in Transactions on Machine Learning Research (09/2024)
Ablation Studies (in decimeters) on model designs. Improv. denotes relative improvement.
Table 3:
Results are computed over runs.
Rel.
Crossing
Corridor
Stadium Gate
Model
90 Degroe
120 Degroe
Unidirectioal
Bidircctional
Enter
Leave
Corner
T-Junction
Ouly data angmentation
2.123:10.222
1.989_0.126
0.195.10.018 0.155.10.00
0.30100.009
0.385.10.018
0.163100001 0.15611.001
Shared decoder
1.491 0.088
1311 0.031
0,101= 0.006
0.1924 0.011
0.2014
0.000
0.301 0.021
0.114 0.003
0.0984 0.001
MLP decoder
0.946+ 0.001
1010+ 0.105
0.080= 0.004
0.201
0.024
0.1984
0.000
0.207= 0.006
0.007 0.001
0.089+ 0.001
w/o Multi-channel
0.713# 0.001
0.743+
0.030
0.061= 0.003
0.160# 0.070
0.1474
0.007
0.199
0.011
0.080+ 0.001
0.000+ 0.001
0
0.583+ 0.086
0.730+
0.026
0.063
0.005
0.130+
0.019
0.1244
0.004
0.186-
0.027
0.081+
0.001
0.065+ 0.001
Default
0.530+ 0.024
0.638=
0.044
0.056
0.001
0.1214
0.008
0.119=
0.010
0.161
0.009
0.073:
0.001
0.057= 0.001
Rel. Improv.
9.09%
12.60%
8.20%
6.92%
4.03%
13.44%
9.88%
12.31%
Table 4: Running time (in seconds) comparison on the largest two datasets. Results are run on GeForce RTX 4090 GPU. Uni. is short for Unidirectional.
Dataset
MLP S-LSTM TransF GNS EGNN GMN CrowdSim SEGNO CrowdEGL
Crossing-122 1 1.493
2.190
3.150
4.608
5.513
4.534
3.331
3.136
3.643
Corridor-Uni. 2.000
2.583
3.217
3.220
3.886
4.004
3.060
3.428
3.280
4.3 Ablation Study
Model design To evaluate the benefits of our model designs, we compare CrowdEGL with five model variants: (1) Only data augmentation: simply treat augmented data as observed trajectories to train the GNN (without multi-channel and independent graph decoder); (2) Shared decoder: use the same decoder for both prediction and equivariance tasks; (3) MLP decoder: replace graph decoders with MLPs; (4) w/o Multi-channel: removes multi-channel inputs and only employ the original input features as initialize node embedding; (5) À O: remove equivariance loss by setting the hyperparameter À to zero. Table g shows the results of all models. We have the following findings:
The straightforward solution, only employing data augmentation, underperforms all model variants, indi- cating it is insufficient to model the challenging crowd dynamics. Utilizing a shared decoder in CrowdEGL leads a significant drop in performance, mainly because the models fail distinguish the observod or augmented data, reducing the quality of learned latent graph embeddings.
MLP decoders underperform graph decoders. These observations suggest that modeling interactions are essential in not only the encoding but also decoding processes, especially in the high-density case where interactions are frequent. and dense. Removing multi-channel information results in a local model and enhances model errors, demonstrating that explicitly integrating the symmetric features in the encoder can learn better latent representation.
Incorporating EGL largely improves the model performance in various environments by encoding equiv- ariance in latent representations. Specifically, EGL achieves 12.60%, 13.44%, and 12.31% relative im- provement on Crossing, Stadium Gate, and T-Junction datasets, respectively. Such results demonstrate its offectivencss.
Performance w.r.t. crowd density To investigate the model performance on crowd density, we divide the testing sample into equal intervals according to the number of pedestrians. The results are shown in Table 5 From the results, can observe that the MSE decreases the density increases, validating our motivation.
Model performance w.r.t. X It is crucial to control the degree of equivariance for real-world crowd dynamics modeling. The sensitivity of X on Crossing and Corridor datasets is illustrated in Figure 6 According to the results, we can find that its values largely influence the model performance. larger value first leads a lower error, and then enhances the error, indicating strict equivariance does not achieve better performance for real-world crowd dynamics.
10

---

Published in Transactions on Machine Learning Research (09/2024)
Table 5:
Mean square errors w.r.t. crowd density. Testing samples are divided into 4 equal intervals according to the number of podestrians. Uni. and Bi. denote Unidirectional and Bidircctional.
Crossing-90"
Crossing- 120"
Corridor-Unl.
Corridor-Bi.
Density
42-95 96-149 150-206 207-262
126-179
180-233
234-287
288-313
49-88 89-128 129-170 171-212 41-74 75-108 109-143
144-179
MSE
0.710
0.570
6.009
1.138
0.343
0.184
0.053
0.149
0.102
Crossing
Corridor
Crossing
6.14
Corridor
Equivariance Errr
1.30
Equivariance Error
2.0
0.75
1.25
0.70
1.20
Crossing 50
M5
MSE
0.10
Unitirectional Eddireciiona
1.15
Crossing-120"
0.60
1:10
1.2
6.55
Undirectoooll Biotrectional
Croading
Crosdin-1207
6.08
9.0
0.2
0.4
0.5
0.8
06
0.2
066
0.8
0.0
0.2
0.4
0.6
0.5
1.0
0.0
0.2
0.4
0.6
0.8
Figure 7: Equivariance error w.r.t. À on Crossing and Corridor. We scale the equivariance error of the Corridor-Unidhe Unidircctionll by factor of 10.
Figure Model performance w.r.t. on Crossing and Corridor.
Crossing
Corridor
Multi-channel aggregation functions The choice of multi-channel aggregation functions is the basis for constructing an equivariant model. We re- place the non-linear transformation of multi-channel inputs in E by Mean and Sum pooling and show their results on Crossing and Corridor in Figure g The key finding that these permutation invari- ant functions will significantly reduce model accu- racy since they make the model invariant to group transformation. Such invariance contradicts the equivariant nature of crowd dynamics, leading to poor performance.
MSE
Mean
0.015
0.2
Default
0.005
Sum
Default
90 Degree
Degree
Eidrrctioonll
Unidirectional
Figure 8: Performance w.r.t. the multi-channel aggre- gation function.
Equivariance error w.r.t. X computer the equivariance error group as:
To provide insights into the imperfect equivariance of models, we can
Leg = - [()))l
(12)
A smaller error indicates the model is more equivariant. If the model is exactly equivariant, the equivariance error will be zero. We compute the average model equivariance errors of considered groups and display the results in Figure 7 From the figure, equivariance errors. A larger À generally leads to lower equivariance error in Crossing-909 and Corridor- Unidirectional, while in Crossing-120 and at around 0.4.
find that employing oquivariant training generally achieves lower
the lowest equivariance errors are achieved
5
Related Work
5.1 Modeling crowd dynamics
Traditional methods simulate pedestrian behaviors, especially collision avoidance, based on expert. knowl- edge. Typical methods include rule-based (Reynolds) 1987), force- based (Helbing & Molnar 1995 Saboia & Goldenstein, 2012), and velocity-based models (Fiorini Shiller 1998). Nevertheless, the designed for-
mulas are insufficient to model complex and uncertain real-world dynamics. With the development of deep
learning, recent studies (Shi et al. 2023 Yu et al. 2023 Zhang et al. 2022 seek neural solutions to improve
11

---

Published in Transactions on Machine Learning Research (09/2024)
simulation performance. Another similar research line is pedestrian trajectory prediction (Alahi et al. 2016 Gupta et al. 2018), whose goal is to predict pedestrians' future positions based their historical trajecto- ries. However, they generally study the pedestrian trajectories in urban open spaces where interactions are relatively sparse. Thus, they focus on capturing their sequential dependency via time-sorics models, such as recurrent models (Alahi et al., 2016 Feng et al. 2018) and Transformer (Yu et al. 2020). Different from the above studies, our target is to predict the future position of high-density crowds based on current states. In this task, interactions are extremely dense and significantly affect crowd trajectories. Thus, we focus on modeling such interactions via GNNs.
5.2 Graph Neural Netwoks
Equivariant graph neural networks Recently, to model the dynamics of particle systems (Huang al.,
researchers have proposed equivariant GNNs that. meet specific symmetry constraints. They de- provably equivariant message passing layers, which could be broadly classified into two types (Han 2022ah. First, leverage irreducible representation to denote higher-order geometric fcatures, such as Thomas et al. 2018: Fuchs et al. 2020 and SEGNN (Brandstetler et al., 2022). Second, construct an invariant scalarization message embedding like the inner product (Satorras et al. 2021 Huang et al. 2022: Zheng et al. 2024). However, they can not be directly extended to pedestrian dynamics. Because of human psychology and physiology, crowd trajectories might not be exactly the same after transformation. Further- more, due to the boundary conditions, not all rotation aeatiiin//efleeee /reeleetiioon///ee /reflection/raannlation /translation are valid in environments. Tn this work, we propose to encourage the equivariance of backbone GNNs via self-supervised training, which can flexibly control the degree of equivariance during the learning procedure. In contrast to the above works, we study equivariant GNNs of finite group.
2022
sign
ct al.
TFN
Graph network-based simulators Due to the simplicity and effectiveness of graph networks, plenty of studies SSanchez-Gonzalle et al., 22200] Shi al., 2023 Li et al. [019 Allen et al. 2022 Pfaff al.) 2021) have adopted them to simulate complex physical dynamics, including fluids, meshes, and rigid objects (Li et al. 2019). They follow mncoder-proosso--eeoodde
framework. The core idea is to use a particle reprosentation of objects, then dynamically construct interaction graphs among them and perform multiple steps of information propagation lo model their interactive forces. Despite their promising empirical results, these works have not adequately considered the symmetry of real-world crowd dynamics. As a result, they fail to achieve satisfactory generalization ability to forecast crowd trajectorics.
6 Conclusion and Future Work
In this work, we propose a novel CrowdEGI. for modeling the trajectories of high-density crowds in various environments. It learns a soft equivariant model via data augmentation and an additional equivariance loss. We further design an advanced backbone to incorporate multi-channel inputs and employ independent graph decoders distinguish original and augmented tasks. Extensive experiments conducted on high- density crowd trajectorics in various environments demonstrate that CrowdEGL outperforms all competing methods. Ablation sludies have further substantiated the elfectiveness of our model designs. For future work, For future work, we will enhance CrowdEGT by designing multi-channel embedding without specific input orders and generalize it to multi-environment scenarios instead of learning on a single environment.
Acknowledgments
This work was supported by NSFC Grant No. 62206067, HKUST- HKUUST-HKUST(GZ) 20 for 20 Cross campus Collaborative Research Scheme C019 and (G) Joint Funding Scheme 2023A03J0673.
References
Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robiequet, Li Fei-Fei, and Silvio Savarese. Social LSTM: human trajectory prediction in crowded spaces. In CVPR, pp. 961-971, 2016.
12

---

Published in Transactions on Machine Learning Research (09/2024)
Kelsey R. Allen, Tatiana Lopez-Cuevara, Yulia Rubanova, Kimberly L. Stachenfeld, Alvaro Sanchez- Gonzalez, Peter W. Battaglia, and Tobias Pfaff. Graph network simulators can learn discontinuous, rigid contact dynamics. In CoRL, pp. 1157-1167, 2022.
Patience Atuhaire. Uganda arrest deadly New Year Freedom City mall crush. 2023. URL https:
//www.bbc.com/// btt.:./////eeee/ooooorraaaataaaa
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik J. Bekkers, and Max Welling. Goometric and physical quantities improve E(3) equivariant message passing, In ICLR, 2022.
Shuchao Cao, Armin Seyfried, Jun Zhang, Stofan Holl, and Weiguo Song. Fundamental diagrams for mul- tidirectional pedestrian flows. Journal of Statistical Mechanics: Theory and Experiment, 2037(3) 2017.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In ICML, pp. 2990 2999, 2016.
Wikipedia contributors. Scoul Halloween crowd crush, 11 2023. URL https://ee..ikkipeete wtt.:kkwpppaaaaa...iiioo. org/wiki/
Seoul Halloween crowd crush
Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljacic. Equivariant self-supervised learning: Encouraging equivariance in repre- sentations. In ICLR, 2022.
Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. Deepmove: Pre- dieting human mobility with attentional recurrent networks. In WWW, pp. 1459 1468, 2018.
Matthias Fey and Jan Eric Lenssen. Fast graph reprosentation learning with pytorch geometric. arXiv preprint Xrv: 02428, 2019.
Paolo Fiorini and Zvi Shiller. Motion planning in dynamic environments using velocity obstacles. The international journal of robotics research, 17(7):760-772 1998.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se t3a-srffsformors:: 3d roto-translation equivariant attention networks. NeurIPS, pp. 1970-1988 2020.
Justin Gilmer, Samuel S. Schoenholz, Patrick Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In ICML, volume 70, pp. 2263-122.. PMLR, 2017.
Francesco Giuliari, Irtiza Hasan, Marco Cristani, and Fabio Galasso. Transformer networks for trajectory forecasting, In ICPR, pp. 10335 10342. IEEE, 2021.
Agrim Gupta, Justin Johnson, Li Fei-Fei, Silvio Savarese, and Alexandre Alahi. Social GAN: socially acceptable trajectories with generative adversarial networks. In CVPR, pp. 2255 2264, 2018.
Jiaqi Han, Wenbing Huang, Hengbo Ma, Jiachen Li, Josh Tenenbaum, and Chuang Gan. Learning physical dynamics with subequivariant graph neural networks. In NewrIPS, 2022a.
Jiaqi Han, Yu Rong, Tingyang Xu, and Wenbing Huang. Geometrically equivariant graph neural networks: A survey. CoRR, abs/ abs/2202.07230, 2022b.
Dirk Helbing and Peter Molnar. Social force model for pedestrian dynamics. Physical review E, (5)):4288. 1995.
Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, and Junzhou Huang. Equivariant graph mechanics networks with constraints. In ICLR, 2022.
Anders Johansson, Dirk Helbing, Habib Z Al-Abideen, and Salim Al-Bosta. From crowd dynamics to crowd safely: video-based analysis. Advances in Complex Systems, 11(04):497 4977 2008.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ICLR, 2017.
13

---

Published in Transactions on Machine Learning Research (09/2024)
Parth Kothari, Sven Kreiss, and Alexandre Alahi. Human trajectory forecasting in crowds: A deep learning perspective. IEEE Trans. Intell. Transp. Syst., (10):))8---400 2022.
Alon Lerner, Yiorgos Chrysanthou, and Dani Lischinski. Crowds by example. In Computer graphics forua, volume 26, pp. 664, 2007.
Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, and Antonio Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. In ICLR, 2019.
Zijie Li and Amir Barati Farimani. Graph neural netwoork-acceleratedd lagrangian fluid simulation. Computers & Graphics, 103:201 211, 2022.
Yang Liu, Jiashun Cheng, Haihong Zhao, Tingyang Xu, Peilin Zhao, Fugee Tsung, Jia Li, and Yu Rong. Improving generalization in equivariant graph neural networks with physical inductive biases. ICLR, 2024.
Mehdi Moussaid, Dirk Helbing, and Guy Theraulaz. How simple rules determine pedestrian behavior and crowd disasters. PNAS, 108(17):6884 6888, 2011.
Stefano Pellegrini, Andreas Ess, Konrad Schindler, and Luc Van Gool. You'll never walk alone: Modeling social behavior for multi-target tracking. In ICCV, pp. 261-2666 268, 2009.
Tobias Pfaff, Meire Fortunato, Alvaro Sanzaez, and Peter W. Battaglia. Learning mesh-based simulation with graph networks. In ICLR, 2021.
Craig W Reynolds. Flocks, herds and schools: A distributed behavioral model. SIGGRAPH, pp. 25 34, 1987.
Priscila Saboia and Siome Goldenstein. Crowd simulation: applying mobile grids to the social force model. The Visual Computer, 28:1039 1048, 2012.
Alvaro Banchez-Gonzile Jonathan Godwin, Tobias Pfaff, Rex Ying. Jure Leskovee, and Peter W. Battaglia. Learning to simulate complex physics with graph networks. In ICML, volume 119, pp. 8459 8468, 2020.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling, E (n) equivariant graph neural networks. In ICML, pp. 9323-932-009 9332, 2021.
Hongzhi Shi, Quanming Yao, and Yong Li. Learning to simulate crowd trajectories with graph networks. Tn WWW, pp. 4200-4 42092,20,00000000 2023.
Liushuai Shi, Le Wang, Chengjiang Long, Sanping Zhou, Mo Zhou, Zhenxing Niu, and Gang Hua. Sgen: Sparse graph convolution network for pedestrian trajectory prediction. In CVPR, pp. 8994 9003, 2021.
Vishwanath A Sindagi and Vishal M Patel. A survey of recent advances in cnn-based single image crowd counting and densily estimation. Pallern Recognition Lellers, 107:3 16, 2018.
Xiangguo Sun, Jiawen Zhang, Xixi Wu, Hong Cheng, Yun Xiong, and Jia Li. Graph prompt learning: A comprehensive survey and beyond. 2023.
Michio Suzuki. Group theory II. Springer, 1986.
Jianheng Tang, Jiajin Li, Ziqi Gao, and Jia Li. Rethinking graph neural networks for anomaly detection. TCML, 2022.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and transsation-qquiiariatt neural networks for 3d point clouds. arXiv preprint arXiv:1802.08 08219, 2018.
Robin Walters, Jinxi Li, and Rose Yu. Trajectory prediction using equivariant continuous convolution. ICLR, 2021.
14

---

Published in Transactions on Machine Learning Research (09/2024)
Rui Wang, Robin Walters, and Rose Yu. Approximately equivariant networks for imperfectly symmetric dynamics. In ICML, volume 162, pp. 23078-22222 23091, 2022.
Chenxin Xu, Robby Tan, Yuhong Tan, Siheng Chen, Yu Guang Wang. Xinchao Wang, and Yanfeng Wang. Eqmotion: Equivariant multi-agent motion prediction with invariant. interaction reasoning. In CVPR, pp- 1410-1420, 2023.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka, How powerful are graph neural networks?
ICLR, 2019.
Ming-Liang Xu, Hao Jiang, Xiao-Gang Jin, and Zhigang Deng. Crowd simulation and its applications: Recent advances. Journal of Compuler Science and Technology, 29(5):799 811, 2014.
Shanwen Yang, Tianrui Li, Xun Gong, Bo Peng, and Jie Hu. A review on crowd simulation and modeling. Graph. Model., 111:101081, 2020.
Cunjun Yu, Xiao Ma, Jiawei Ren, Haiyu Zhao, and Shuai Yi. Spatio-temporal graph transformer networks for pedestrian trajectory prediction. In ECCV, pp. 507- 523, 2020.
Zihan Yu, Guozhen Zhang, Yong Li, and Depeng Jin. Understanding and modeling collision avoidance behavior for realistic crowd simulation. In CIKM, pp. 3052-3061, 2023.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R. Salakhutdinov, and Alexan- der J Smola. Deep sets. NeurIPS, 30, 2017.
Guozhen Zhang, Zihan Yu, Depeng Jin, and Yong Li. Plysics-infused machine learning for crowd simulation. In KDD, pp. 2439-2449, 2022.
Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li. All in one and one for all: A simple yet elfective method towards cross-domain graph pretraining. In KDD, pp. 4443 4454, 2024.
Zinan Zheng, Yang Liu, Jia Li, Jianhua Yao, and Yu Rong. Relaxing continuous constraints of equivariant graph neural networks for broad physical dynamics learming. In KDD, pp. 1548-4558, 2024.
Bolei Zhou, Xiaogang Wang, and Xiaoou Tang. Understanding collective crowd behaviors: Learning mixture model of dynamic pedestrian-agents In CVPR, pp. 2871-2878, 2012.
15


Reviews:
[{"review_id": "Sztn59jzKM", "reviewer": "Reviewer_YJ7y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The work shows strong improvements with a fairly simple idea. The ablation studies are well-designed and answer the questions a reader may have about the different subparts of the system. The paper is a nice progress. \n\nOne small weakness is that it would've been nice to see a comparison of the extra compute required at training time for the data augmentation / equivariant setup vs the vanilla GNN baseline.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The work aims to predict the dynamics of high-density crowds for applications in urban planning and architectural design. In particular, this work proposes to use an equivariant approach to exploit (imperfect) symmetries that the crowd dynamics may obey such as e.g. left-to-right vs right-to-left symmetry and rotational symmetry. This is done via data augmentation combined with a weighted symmetry-aware loss. The work shows that this outperforms non-GNN, non-symmetry-aware and data-augmentation-only baselines. It also studies the various contribution through ablation studies.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- \"For simplicity, we denote (x(t), v(t)) and (u, e = {eij}, a = {aij}) as dynamic and static state information of the entire graph correspondingly. \" is it not clear to me why position would be dynamic, but the distance between nodes would be static, if the position changes, so would the distance and the graph connectivity?\n\n- \"The symmetry of real world is generally not continuous.\" this statement is clearly not true, large parts of physics are based on continuous symmetry such as the entire universe around us obeying E(3) symmetry. \n\n- eqn 5 should be L_g instead of L_p\n\n- \" First, most existing models Shi et al. (2023) rely on a simple MLP as the decoder, ignoring crucial interactions in high-density crowds. \" --> it's not really clear why going from a simple MLP to the iterated MLP shown in the eqn would make such a big difference and why a simple MLP on the already learned, highly nonlinear latents would \"ignore crucial interactions\"?\n\n- how are the data split, if they are temporal are they split s.t. earlier segments are train, later are val/test or are they split randomly over time?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AAAQAH7V18", "reviewer": "Reviewer_vwt6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "From my perspective, the strong aspects of this paper include:\n1. Novelty in the question set. As mentioned in the paper, considering equivariance in the combined field of crowd trajectory prediction and GNNs is relatively new.\n2. Better performance compared to previous methods.\n\nWeaknesses include:\n1. Limited novelty in methods. The data augmentation techniques mentioned, such as rotation and reflection, are straightforward and common in machine learning. While the multi-channel network structure is more novel, it requires more detailed explanation and clarification on the types of problems this network structure may address. The intuition behind this structure is not clearly explained in the paper.\n2. The evaluation metrics used in the paper, focusing on the MSE of pedestrians over a longer horizon, are similar to the missing rate used in other trajectory prediction papers. However, solely using the missing rate is insufficient for evaluating a prediction model. Given the principal aim of this work is to model interactions among pedestrians and between them and the environment, as stated in the conclusion, outputting a whole trajectory with smaller time steps and evaluating the results using various metrics such as minADE, Missing Rate, minFDE, etc., would provide a better assessment of how well interactions are modeled.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents an interesting perspective that the equivalence of crowds should be explicitly addressed and introduces the algorithm named CrowdEGL to enhance the generalization ability of GNNs. In my opinion, the core contributions of this paper include:\n1. Emphasizing the importance of explicitly handling equivalence in the field of crowd trajectory prediction using GNNs.\n2. Utilizing data augmentation combined with a multi-channel network structure for training.\n3. Conducting extensive experiments to validate their methods across different datasets, especially in high-density crowd scenarios.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. I think Fig. 2 should provide more insights about the multi-channel network structure.\n2. It is not necessary to emphasize that designing a backbone GNN is \"non-trivial\" at the beginning of section 3.3.\n3. Please add more experiments and insights on the \"imperfectness\" of equivariance. For example, how can it be measured in datasets and how to tune it during training? Since the authors agree that it is an important factor influencing results, as illustrated in Figure 6, more detailed analysis would be beneficial.", "broader_impact_concerns": "Not applied.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "i0wSVtBw7S", "reviewer": "Reviewer_SVBZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper introduces CrowdEGL, a method incorporating imperfect equivalence for high-density crowd trajectory prediction.\n2. CrowdEGL significantly outperforms state-of-the-art models in generalization and performance on diverse datasets.\n\nWeakness:\n1. Motivation: \n- The trajectory evolution rules for various crowded trajectories are similar. Hence, the original trajectories and transformed trajectories should adhere to the same trajectory evolution rule. Considering this observation, I believe that a model trained with data augmentation can acquire the shared trajectory evolution rule from crowd trajectories that possess inherent symmetry. It is difficult to comprehend what is learned in CrowdEGL within this paper. If the graph encoder in CrowdEGL can grasp the common trajectory evolution rule, then so can the GNN trained with data augmentation.\n\n2. Method: \n- The paper only focuses on a limited set of input transformations, such as rotations of 90, 180, and 270 degrees. I believe that inputs rotated at other angles are also significant.\n\n3. Experiments: \n- The robustness of (8) needs further verification. For instance, it is unclear how the performance will be affected when treating the 90/180/270 degree rotated input as the original input. There is a potential risk that CorwdEGL may be overfitted to a specific input order in (8).\n- How to measure the density of the crowd, hows the performance changes w.r.t density of the crowd?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel Equivariant Graph Learning method, called CrowdEGL, for modeling high-density crowd trajectories in urban environments. Unlike traditional methods that rely on rigid formulas, CrowdEGL uses data-driven models and incorporates symmetry transformations to enhance prediction accuracy. The framework employs a multi-channel GNN to learn latent pedestrian patterns and uses multiple independent graph decoders to predict future positions. Extensive experiments demonstrate that CrowdEGL significantly outperforms existing models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weakness above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a method for learning equivariant graphs using GNNs. It is applied for modeling crowd trajectories, and its superior performance is shown across 8 datasets and 5 different environments.

Reviewers vwt6 and YJ7y recommend acceptance. Reviewer SVBZ found that a more realistic evaluation is required (see comments above). Despite that, the paper's claims seem robust and supported by the current experiments.
Decision: Accept (poster)