LocoVR: Multiuser Indoor Locomotion Dataset in Virtual Reality
Authors: Kojiro Takeyama
Yimeng Liu
Misha Sra
Venue: ICLR 2025 Poster (2025)
Keywords: Dataset
Human trajectory
Indoor locomotion
Virtual reality
Social motion behavior
Primary area: datasets and benchmarks
TL;DR: We present a dataset of two-person trajectories across 130+ home environments, capturing geometrically and socially-aware motion behaviors.

Abstract: Understanding human locomotion is crucial for AI agents such as robots, particularly in complex indoor home environments. Modeling human trajectories in these spaces requires insight into how individuals maneuver around physical obstacles and manage social navigation dynamics. These dynamics include subtle behaviors influenced by proxemics - the social use of space, such as stepping aside to allow others to pass or choosing longer routes to avoid collisions. Previous research has developed datasets of human motion in indoor scenes, but these are often limited in scale and lack the nuanced social navigation dynamics common in home environments. 
To address this, we present LocoVR, a dataset of 7000+ two-person trajectories captured in virtual reality from over 130 different indoor home environments. LocoVR provides accurate trajectory and precise spatial information, along with rich examples of socially-motivated movement behaviors. 
For example, the dataset captures instances of individuals navigating around each other in narrow spaces, adjusting paths to respect personal boundaries in living areas, and coordinating movements in high-traffic zones like entryways and kitchens. Our evaluation shows that LocoVR significantly enhances model performance in three practical indoor tasks utilizing human trajectories, and demonstrates predicting socially-aware navigation patterns in home environments.

Published as a conference paper at ICLR 2025
LoCOVR: MULTIUSER INDOOR DATASET IN VIRTUAL REALITY
LOCOMOTION
Kojiro Takeyama Yimeng Liu', Misha Sra!
I: University California Santa Barbara, 2: Toyota Motor North America {t (takeyama, yimeng 1iu, sra) edu
ABSTRACT
Understanding human locomotion is crucial for Al agents such as robots, par- ticularly in complex indoor home environments. Modeling human trajectories in these spaces requires insight into how individuals maneuver around physical obstacles and manage social navigation dynamics. These dynamics include subtle behaviors influenced by proxemics the social use of space, such as stepping aside to allow others to pass choosing longer routes to avoid collisions. Previous research has developed datasets of human motion in indoor scenes, but these are often limited in scale and lack the nuanced social navigation dynamies common in home environments. To address this, we present Loco a dataset of 7000+ two-person trajectories captured in virtual reality from over 130 different indoor home environments. Loco provides accurate trajectory data and precise spatial information, along with rich examples of socially-motivatee movement behaviors. For example, the dataset captures instances of individuals navigating around each other narrow spaces, adjusting paths to respect personal boundaries living ar- eas, and coordinating movements in high-traffic zones like entryways and kitchens. Our evaluation shows that LocoVR significantly enhances model performance three practical indoor tasks utilizing human trajectories, and demonstrates predict- ing socially-aware navigation patterns in home environments. The dataset and evaluation code are available at https://github.com////it hatthhubbbbboomt22222ahoo/oooo
INTRODUCTION
Predicting human trajectories is crucial for Al systems like home robots. While many outdoor pedestrian trajectory datasets exist, they are not applicable to indoor settings due to differences in geometric complexity, scale, and movement patterns. An ideal indoor dataset would include diverse scenes and trajectories, but creating such a dataset at scale challenging. Camera- camera-based collection methods often fail due obstructions, while advanced 3D scanning methods are limited by high costs and time constraints. Consequently, a comprehensive dataset of human locomotion in varied indoor environments remains elusive, hindering the development of Al systems that can effectively navigate and assist in home settings.
To overcome data collection challenges, we propose LocoVR, dataset captured virtual reality (VR) that efficiently captures detailed spatial information, human-scene interactions, and human- human social motion behaviors across diverse indoor environments. LocoVR captures task-focused movements of two people in over 130 home settings, including their trajectories, head orientations, and precise spatial data. Crucially, LRcoVR captures motion proxemics the social use of space, such yielding in narrow spaces, maintaining personal distances in shared areas, and coordinating movements high-traffic considering the Interpersonal Adaptation Theory BBrggoo et al. (1955). These proxemics brasemics-based motion behaviors, often missing in current datasets, serve as form of non-verbal communication, and are influenced by factors such as the relationship between individuals and their cultural backgrounds Hall (1963); Watson 2014). Human social dynamics can provide valuable insights for home robots to navigate domestic spaces more naturally while adhering to implicit social norms.
Our goal is to understand and predict human trajectories in complex indoor environments by consid- ering both geometric constraints and social proxemics. Geometrically, we aim to model how people

---

Published as a conference paper at ICLR 2025
avoid obstacles and find efficient paths. Socially, want to capture how individuals anticipate and react to other people's 's movements, adjusting their trajectory to avoid collisions, maintaining personal space, and minimizing path interference.
We demonstrate our dataset through three tasks: global path prediction, trajectory prediction, and goal area prediction (Figure[) The first tasks showcase the dataset's capability facilitate geometrically and socially aware path predictions, while the last task demonstrates its versatility in supporting broad spectrum of applications. Our key contributions are outlined as follows.
Developing VR system for the efficient and accurate collection of two-person trajectories across diverse indoor environments.
2. Building the first large-scale indoor trajectory dataset featuring two-person motions, which enhances task performance in unseen indoor scenes from both geometric and social perspectives.
Showcasing enhanced model performance trained on our dataset across three practical indoor tasks, demonstrating geometrically and socially aware navigation patterns in home environments.
Data collection system
Loco VR dataset
Ego views
Real space
Virtual space
Synchremized
Large scalo, multi person, full-- body 3D motion, head direction geometry. and sementica
Applications
Global path prediction
Trajectory prediction
Goal prediction
Figure 1: Overview of the data collection VR system, snippets of the LocoVR dataset, and the dataset applications. We collected multi-person trajectories in 131 complex indoor scenes, including trajectory with head orientation, precise geometry. In the dataset applications, predicted and past trajectories are shown in red and blue, respectively.
RELATED WORK
2.1 DATASETS FOR SOCIAL NAVIGATION
Datasets for social robot navigation can be categorized based on their scene types and the nature of the robot or human agent involved. Studies such as Rudenko et al, (2020); Schreiter et al. (2024); Finean al. (202)); Manso et al. (2000) provide datasets collected in controlled lab settings where robots coexist with humans. While these datasets offer valuable insights into human-robot interactions in indoor settings, their scene structure simplistic, and the number of unique scenes is limited, which reduces their capability on diverse real-world indoor settings. On the other hand, datasets like SIT Bae et al. 2024)) SACSON Hirose et al. (0023), JRDB Vendrow et al. (202)), and Socnav Manso al. (2020) capture interactions in crowded public spaces by robots or human equipped with cameras and Lidar. Despite their richness in capturing diverse interactions, the open and unstructured nature of these scenes poses challenges for generalizing the findings to indoor settings. Furthermore, the partial observation of scene structures limits their application for indoor navigation tasks, such as global path planning. Datasets such those proposed by Zhou al. Zhou et al. (2012). Robicquet et al.
2

---

Published as a conference paper at ICLR 2025
Robicquet al. (2016), Ess al. EEss et al. (2007). and Kothari et al. Kothari et al. (2021) used aerial cameras to capture human trajectories and surrounding scene images. They provide complementary data human trajectories and scene geometry across wide range of environments, enabling the study of crowd dynamics. However, their applicability to indoor scenes is limited due to differences in scene structure, spatial constraints, and interaction dynamics within enclosed spaces.
2.2 DATASETS FOR HUMAN MOTION SYNTHESIS AND GENERATION
Understanding human motion is important for various research problems, such as synthesizing motion for 3D environments. Prior work has studied the 3D human motion problem and built datasets to sup- port research in this domain. One line of research has explored human-scene interactions and focused on understanding how environmental constraints affect human behavior, such as GIMO (Zheng et al. 2022 Guzov et al.) [02T Zhang al.. 2022). A challenge faced by human-scene interaction datasets is capturing scene variation is difficult and expensive. To address this challenge, prior research Wang (2022) Cao et al. (2020) has worked on synthesizing human motions in virtual environments and game engines or generating human motions with generative AI models. However, these synthesized motions do not necessarily follow human behavior principles. especially when social motion behav- iors are involved. Additionally, the existing datasets focus mainly on single-person pereen motions though understanding multi-pen pertin social motion behavior is critical for many practical applications, such as human-robo collaboration. Moreover, existing human motion datasets provide limited locomotion data needed to understand room-scale human motion dynamics, as they primarily focus on capturing fine-grained details of individual actions, such as opening fridge. To fill these research gaps, we introduce Loco VR, a novel dataset comprising two-person trajectories with social motion behaviors, collected using VR to allow large number of trajectories across diverse indoor scene variations.
2.3 VR SYSTEMS FOR HUMAN MOTION ANALYSIS
VR technologies including motion tracking systems have been extensively used to study human-
interactions and behavior across various domainsLee et al. (2023): Takahashi et al. (2021) Simeone
et al. (207); Brookes et al. (2020), including sports strategy evaluation Wang et al. (2024b). road
crossing studies Gallo et al. (024). and navigation crowded environments Yun et al. (2024). These applications have demonstrated their versatility in replicating real-world scenarios while ensuring safety and repeatability. Datasets specifically developed for VR-based studies focus on distinct scenarios, such as road crossing Wu et al.) (2023) and object-approaching tasks Araújo al. (2023). While these datasets facilitate task-specific behavioral analyses, they lack representation of complex indoor locomotion scenarios involving intricate geometry and multi-person interactions. To address this gap, we introduce a VR system that delivers an immersive locomotion experience across diverse indoor scenes, offering novel dataset and valuable insights to advance research human locomotion and interactions in complex indoor environments.
3 LOcOVR DATASET
3.1 OVERVIEW
Table I summarizes the statistics of existing human trajectory and motion datasets and Loco VR. Our dataset contains 2500K frames of human trajectories in 131 scenes (see Figure fggureffor examples). The number of scenes surpasses all the real human motion datasets. We collected two-person trajectories that are geometrically and socially aware, which is not included in most of the compared datasets. The number of trajectories is 7071 in total (see Appendix [ for detailed statistics of LocoVR). LocoVR facilitates the enhancement of task performances from both geometric and social perspectives in unseen, complex, and confined indoor environments. Also, it includes body tracker data on wead/wais//handeee as auxiliary information. These additional observations could facilitate a deeper understanding of human locomotion and enhance model performance.
3.2 DATA COLLECTION
Figure T shows our data collection system. The data collection experiment has been approved by the IRB at our institution under protocol number 14-22-0646. During the experiment, two people
3

---

Published as a conference paper at ICLR 2025
Table 1: Statistics of existing human motion datasets and our Loco dataset.
Stome
Sukject
Frome
Count
Geometry
Location
Pos/Pose
Malti
Motion"
Torget - action
HPS (Guzov et al.
3021
300K
8
V3D mesh)
Outlindoee
3D
Keal
Daily actions
EguBody
Zhang etal. 2023
153K
15
v3DD mech)
Indour
3D
Real
Daily actions
100K
12
mesh)
Indoor
3D
Real
actions
Zheng
129K
19
(3DD mesh)
Indoor
3D,
Real
Daily actions
Cimod) Station 7Thru al
50K
1
(Aerial image)
Oundoor
2D
Real
Trajectory
SDD
[Robiquue et al. 2016
920K
6
(Aerial image)
Oundor
2D
V
Real
Trijectory
FTH Fas 0t
2007]
50K
2
(Aerial image)
Dusdoor
2D
Real
Trajectory
THOOR (Rudenko al.
2000
340
3
V3D point cloud)
Indoor
2D
Real
Trsjectory
JRDB Vendrow
6368
30
(3D point cloud)
Outlindooe
3D
Real
Trajectory
CCo
2020
1000
v(3D
Induor
3D
Symtheri
Trijectory
HUMANISE Wang et
1200X
643
v(DD mesh)
Indoor
3D>
Symtheric
Daily actions
CIRCLE Araūjo et 2023
4300X
9
V3DD mesh)
Indoor
3D
Real
Daily actions
THOR-MAGNI (Screiitr 2t al.
2014
1260K
4
(3D mesh)
Indoor
3D, Gaze
V
Reall
Tejeetory, Daily actions
LR (Ours
2500K
131
v/(3D mesh)
Indoor
3D, Head
Real
Trajectory, Social motion
"Real" relers to real motions walking behaviors cuptured video or motion capture; "Syntheihic" refers to synthesized
motions and behaviors via animation techniques.
Yecture mup
Height map
Semantic map
Figure 2: LocoVR includes 131 scenes with detailed spatial information, like photorealistic textures, 3D geometry, and semantics. Blue and red curves show two people's 's trajectories in one session.
wore VR headsets to see a shared virtual environment where they could interact with each other and perform tasks that required walking. The participants' movements motion capture and mapped onto virtual avatars that move in the same way as the participants to help them keep social awareness. The advantages of our system include: (1) scene variation is fast and easy by switching the virtual scenes such that human trajectory data in a large variety of scenes can be collected; (2) accurate spatial information can be obtained by recording the spatial data in digital format; (3) participants can walk naturally and produce locomotion data recorded in VR as the virtual room has been well aligned with the physical space; (4) geometrically and socially aware motion behaviors can be accurately and easily controlled and replicated in virtual space.
recorded in real-time by
The data collection was conducted in a 10m by 10m open indoor room in the physical space, where participants walked within similarly sized virtual rooms. In the experiment, each person was assigned a unique goal, represented by a virtual marker that only they could see. Once they reached their goal, new goal appeared in different location, starting new round of the task. The new goal appeared at a random location, which was at least 2m away from the previous goal position, to prevent short trajectories. This process repeated around minutes per scene to allow the participants to fully explore the virtual environment and add variations to the collected trajectory data. See Appendix for more details of the data collection setup.
4

---

Published as a conference paper at ICLR 2025
4 EVALUATION
We evaluate our dataset by demonstrating its usage in the following trajectory-based indoor tasks:
1. Global path prediction:
This task estimates a static global path from a start to a goal, applicable to goal-conditioned human path prediction or global path planning for robots. The task demonstrates the capability of our dataset to learn physically and socially plausible human trajectory. including social motion behaviors such maintaining social distance when passing or choosing longer routes to avoid others. The input includes the past trajectories of two people, P) and p2 (length=1 .Os, interval=0.067s)) past heading directions of p1 and p2 (length=1.0s, .Os, interval=0.067s), scene map, and goal position. The output is a static path from the start to the goal.
This task predicts short-term future trajectories based on past movement data and is used to develop policies for robots to avoid collisions with other humans. By training on our dataset, trajectory prediction models can consider the movements of others in indoor environments where the space is small and where obstacles have significant impact on path choice. The input contains the past trajectories of p1 and P2 (length=1.0s 1.0s, interval=0.067s) past heading directions of Pl and p2 (length= 1.0ss interval=0.067s) and the scene map. The output is time-series future trajectories of pı and p2 (length=3.0s 330s, interval=0.067s)
Trajectory prediction:
This task predicts the goal position based on past trajectory and can be used in applications where robots AI predict potential goals as people begin to move toward their next task location. We demonstrate that the goal prediction model trained on our dataset effectively narrows down the goal candidates by considering scene geometry and past trajectory. The input are the past trajectory and heading directions of p1 (length=6.06 Os, interval=0.0677 and the scene map. The output is predicted goal position. Note that the arrival time the goal is not given.
Goal prediction:
In these tasks, it is crucial to consider how human trajectories are influenced by goal positions, the movements of other people, and scene geometry. Particularly, geometry is a dominant factor affecting human trajectories in complex indoor environments. Although there geometry-aware trajectory prediction models like NSP Yue et al. (2022), Goal-GAN Dendorfer et al. (2020), and SoPhie [Sadehiian et al. (2019), they often compress geometric features into small sets, losing the detailed structure of the entire scene. As a result, these models struggle to learn how humans move in complex indoor geometries. To address this, we employed U-Net-based models (a simple U-Net and Ynet(Mangaalam et al.) 2021)) to preserve the scene geometry. Details of the benchmark models for each task are described in the following sections. Additionally, in the tasks, we incorporated both the trajectories and heading directions (front direction of head poses) of individuals to maximize prediction performance, as head direction data are available in all benchmark datasets, including LocoVR, GIMO, and THOR-MAGNI.
4.1 DATASETS
4.1.1 TRAINING DATASETS
We evaluated Loco VR and two existing datasets as a benchmark. Given the absence of datasets specifically focusing on locomotion in complex indoor scenes, we chose GIMO and THOR-MAGNI as training data because they closely align with LocoVR and are suitable for our tasks.
LocoVR: LocoVR is our main contribution, and it was collected using our VR system. The dataset includes over 7000 trajectories in 131 indoor environments. We split it into training (85%) and validation sets (15%).
GIMO Zheng et al. )2022): GIMO is an indoor daily activity dataset containing trajectory data with heading information in real complex indoor environments, while has limited scene variations and only single-person data. We extracted the locomotion data and excluded trajectories that were too short (< 2s), resulting in 187 trajectories in 19 scenes. We divided the dataset into training (85%) and validation sets (15%).
THOR-MAGNI Schreiter et al. (2024);
is an indoor multi-person trajectory dataset that contains number of trajectories comparable to dataset, including heading information. However, it includes only four types of scene maps, which are similar to each other. To align the data format with our test data, we extracted trajectory segments between goals and then picked up pairs of
5

---

Published as a conference paper at ICLR 2025
trajectories from 2s) and trajectory pairs that included a time jump in either trajectory. As a result, we obtained around 10,000 trajectories in 4 scenes and divided them into training (85%) and validation sets (15%).
trajectories within the same scene. We excluded short trajectories (<
4.1.2 TESTING DATASET
LocoReal: To test the models on real-world data, we built human trajectory dataset collected in a physical room space. To collect this dataset, we invited two participants to walk in a room that contained furniture. Each participant's s movements were tracked and recorded by motion capture system. They were then given a list of goals and asked to reach each goal in sequence, one after another. This allowed us record the full trajectories of their motion including social behaviors necessary to navigate around the furniture and in 4 tight spaces. See Appendix G for the details.
4.2 IMPLEMENTATION DETAILS
In our evaluation, all the positional information, such trajectory or goal position, is handled in 2D images. We use binary maps as scene maps, with indicating the area between -0.3m to 0.3m height above the floor level and for all the other areas. The map size is 256 pixels by 256 pixels in the image and 10m by 10m in the physical world. All the training data are augmented by horizontal flipping and rotation with 90, 180, and 270 degrees, increasing the number of training data by eight-fold. We also augmented the data in time-series direction to obtain different sets of past trajectories and ground truth future trajectories. We define fixed length time window and slide it with a certain interval to obtain sets of augmented trajectories. See Appendix for the details.
4.3
GLOBAL PATH PREDICTION
4.3.1 1
BENCHMARK MODEL.
A# (Hart et al. 1968) U-Net Ronneberger et al. (2015): A is an algorithm that finds the optimal path using a cost map. minimizing the total cost from the start the goal. While it commonly used for robots to find the shortest path to the goal, we use it to find a human-like path by incorporating the cost map created based on probabilistic path distributions derived from model trained on the datasets. Specifically, trained the model with simple U-Net structure using the training datasets (Loco GIMO, THOR-MAGNE
The model's s input and output have been explained in Section [ the cost map, first take the reciprocal of the model's output and then multiply it by the obstacle map to incorporate obstacle information. Additionally, we used two types of non-learning- based cost maps as benchmarks: MAP and DISTMAP. MAP is based on the scene map, where the cost is in obstacle areas and O elsewhere. DISTMAP is based on the distance from the nearest obstacle, defining the cost 1/(1 + d), where d is the distance from the obstacle in pixels (with maximum value of d 10).
To obtain
Ynet (Mangalam et al., 2021): Ynet is a state-of-the-art technique for goal-conditioned human trajectory prediction. can generate long-term trajectories considering complex scene geometry based on multiple U-Net framework. While Ynet predicts a dynamic trajectory to the goal, we convert into a static global path by projecting the trajectory onto an image to match the benchmark's output.
4.3.2 METRICS
We adopted an evaluation metric based on Chamfer distance assess the differences between the predicted and ground -ruth paths. This metric calculates the distance from each pixel in the predicted path to the nearest pixel in the ground-truth path. We employed the mean and maximum of these distances across all pixels in the path and converted them to meters.
4.3.3 RESULTS
Quantitative results: presensss the evaluation results based on two metrics. A* with the U-Net trained with LocoVR demonstrated significantly superior performance compared to the benchmarks that include models trained on GIMO and -MGNI. It is also shown that Ynet trained on LocoVR presents better accuracy than the other two datasets. This is mainly due to the difference in scalability of the datasets, including a number of trajectories and scenes. In GIMO, both the number of trajectories and scenes limited, whereas
has many trajectories but includes

---

Published as a conference paper at ICLR 2025
only 4 scenes, leading low generalization ability to new scenes not present in the training data. On the other hand, LocoVR ensures high performance even in unseen scenes owing to its high scalability in data amount and scene variation. See Table S in Appendix fioor details.
Table 2: Mean and Max Chamfer distance between predicted and ground truth paths grouped by distance to the goal. The table reports averaged value over three trials + SD.
Mean
Mar
Method
Om < d < 3m
3m < d < Gm
Guz < d
Om < d < 3m
3m < d < Gra
Gen < d
Ynet (GIMO)
a.08
0.22g0012
0.51ga0m
0.17:
0.46
1.1 111ms
Ynet (THOR-MAGNI)
0. 100000
0,30
0.65
0.19
0,56
1 1.29
Ynet (L)coVR)
0.09
0.18
0.42
0.1800002
a.37
0.92:am0
A MAP
0.10g0
0.27 20000
0.40g
0.22:1000
0.58
0.89
A + DISTMAP
0.102
0.18 0.0000000
0.26
a.24
0,46
0. 0.66g0000
A U-Net (GIMO)
0.09
0.23
0.36+0011
0.20
0.53
0.84
A+ U-Net (THOR MAGNI)
0.07
0.21 200000
0. 0.3020005
0.1700001
0.45
0.71
A U-Net (LocoVR)
0.06=4.011
0.12:4900
0.1920m3
0.15:
0.29::940
0.50
Qualitative results: Figure 3 shows the result of global path prediction by A* with U-Net trained on each dataset, in four different scenes. In each image, the yellow distribution indicates lower values in the cost map that guides the global path prediction. The green and blue lines represent the past trajectories of pi and p2, respectively. The orange circle indicates pi 's goal position. The light green and red lines denote the groundtruth and predicted global path of p1-
While the cost maps of learning-based methods emphasize expected future paths regions, those in GIMO and THOR-MAGNI are not clear and continuous, hindering the prediction of smooth, human- like paths. This limitation stems from the restricted scene variations in GIMO and THOR-MAGNI, leading to poor performance in unseen environments. In contrast, the Loco dataset, with its large-scale diversity, enables the prediction of geometry-aware smooth paths, even in complex and previously unseen environments.
Furthermore, LocoVR also demonstrates its ability predict social motion behaviors. In scene and 2 where P2 is walking on the p1 's shortest route to the goal, only the model with LocoVR accurately predicts detour route avoid interrupting P2- In contrast, other models predict shorter routes that lead to potential collisions with p2. In scenes 3 and 4, where pi and P2 pass each other in a narrow space, the model trained LocoVR predicts paths that maintain social distance from p2 potential trajectory, closely matching the ground truth. Specifically, in scene 3, curves closer to the wall to keep distance from P2. while scene 4, P1 steps aside create space for P2 to pass. This attributed to LocoVR's is capability to learn social motion behaviors across diverse scenes.
4.4
TRAJECTORY PREDICTION
4.4.1 BENCHMARKS
U-Net Ronneberger et al. (2015): We evaluated simple model with a U-Net structure trained using the datasets (LocoVR, GIMO, THOR-MAGND, Past trajectories and heading directions of pi and p2 and the scene map are concatenated and fed to the model, then the U-Net structured encode-decoder outputs probabilistic distribution of dynamic trajectory for p1 and P2 represented by images.
Ynet (Mangalam et al., 2021): Here, we evaluate performance on dynamic trajectory prediction using Ynet trained on the datasets (LocoVR, GIMO, THOR-MAGND), Ynet is a single-person trajectory predictor; we use the past trajectory of pı and the scene map as the input. The output is the probabilistic distribution of dynamic trajectory for P1 represented by images.
4.4.2 METRICS
We use ADE (Average Displacement Error), a commonly used metric, to evaluate the performance of trajectory synthesis. ADE refers to the mean squared error over all the time correspondence points on predicted and ground-truth trajectories. The ADE scale is represented in meter units.
7

---

Published as a conference paper at ICLR 2025
Scene (detour)
Scene (detour)
Scene 3 (social distance)
Scene (social distance)
goal
artt
goo
geal
+ U-Net (GIMO)
goal
stat
start
goal
Start
goal
A U-Net (THOR- MAGNI)
start
gooll
star
goal
tart
goal
goal
Social distance
Social distance
A U-Net
(LocoVR)
starn
Avoiding interruption
goal
Avoiding interruption
star
Past trajectory (p1)
Predicted global path (p1)
Cost map for A" (pt)
Walkable area
(higher propartion of yellow corresponding to lower costs, guiding the global path)
Past trajectory (p2)
Groundtruth global path (p1)
Non-walkable area (obstacie)
Figure 3: Predicted global paths with cost maps (intense yellow represents low-cost areas). A* generates the optimal path that minimizes the cost along the way. With LocoVR, the cost map concretely guides human-like paths (red line) that capable of avoiding collision with obstacles and other people's paths, which align with the groundtruth paths (green line).
4.4.3 RESULTS
Quantitative results: Table 3 reports the performance of trajectory prediction over time. As can be seen, both Ynet and U-Net trained on LocoVR outperform those trained other datasets. This is mainly due to the difference in scalability as described in the global path prediction section: fewer scene variations the limited number of trajectories with GIMO and THOR-MAGNI result in lack of generalization performance on new scenes. Additionally, social motion behavior, which is contained in GIMO, is factor that affects performance in the two-person setting. See Table 6 in Appendix C for details.
Table 3: ADE (Average Displacement Error) between predicted and ground-trute truth time-serics trajecto- ries. The table reports averaged error [m] in all of the trajectories over three trials + SD.
Method
Os S (S 1s
1s 1s51
2s
251
3s
Ynet (GIMO)
0,28aumu
0,53goon
0.814n013
Ynet (THOR-MAGNI)
0.62=axz6
0.89:0052
0.88#0042
Ynet (LocoVR)
0.40
0.61
U-Net (GIMO)
0. 0.192m09
0.34g00m
0.55gnm:
U-Net (THOR-MAGND
0.59gmm0
0.92#0011
1.14a001s
UNet (LocoVR)
0.11:aam0
0.24:4.00
0.44:0010
Qualitative results: Figure shows the result trajectory prediction with U-Net. The model trained on LocoVR is able to predict trajectory, taking into account both the obstacles and the other person's movement. In contrast, predicted trajectory distribution with GIMO is spread to multiple directions, resulting in collisions with other people since GIMO does not include multi-person data. With THOR-M THOR-MAGNI, the predicted trajectory becomes stuck along the way due to its unstable performance in unseen scenes.
8

---

Published as a conference paper at ICLR 2025
U-Net (GIMO)
U-Net (THOR-MAGNI)
U-Net (LocoVR)
Past trajectory (p1)
Potential collision
Post trajectoy (64)
Collision
Groundrruh trajctory
Predicted Prajstoe (p1)
trajectory (p1)
Avoiding collision
iredicted distributiion of trajectory (p2)
Figure 4: Predicted trajectory and probabilistic distribution using U-Net. U-Net trained on LocoVR predicts p1 's trajectory that smoothly proceeds with collision with obstacles or other people.
4.5
GOAL PREDICTION
4.5.1
BENCHMARK MODELS
U-Net Ronneberger et al. (2015): We applied a simple U-Net model to predict goals. Inputs are past trajectories of P1 and the scene map; the output is the probabilistic distribution of p1 's goal position.
We evaluated two types random sampling of goals according to the metric defined below. One method randomly determines the goal position assess goal position accuracy, while the other randomly selects goal objects to evaluate object prediction accuracy.
RANDOM:
NEAREST: This benchmark determines the goal position based person's 'ss current position, using two different methods according to the metrics. One samples the goal within 1.5m from the person's current position, while the other selects goal objects based their distance from the current position.
4.5.2 METRICS
Goal position error: It is defined as the distance between the true goal position and the predicted goal position used to measure the basic performance of goal prediction.
Object prediction accuracy: In the testing dataset (LocoReal), the goal position is on one of the 20+ objects in the scene map. so evaluate the rate predicting the correct goal object. We sampled the best three objects based on confidence and evaluated the rate it includes the true goal object.
4.5.3 RESULTS
Quantitative results: Table presents the performance the models evaluated on the two metrics. Compared to RANDOM and NEAREST, the models trained each dataset exhibit better perfor- mance. Notably, the model trained with Loco VR significantly outperforms those trained on other datasets owing to the dataset scale. See Table 7 in Appendix flo details. Performance along the distance to the goal tends to improve as the distance decreases. Note that the narrowing of the goal area cannot be attributed to the time duration the goal, as the arrival time is not provided in this task. It is assumed that proximity to the goal correlating with longer trajectories offer additional clues for narrowing down the goal location.
Table 4: Performance on goal position prediction and goal object prediction. The table reports averaged performance in all of the trajectories over three trials + SD.
Goul position error
Object prediction accuracy
Method
Om & d 3m
3m & d'g 6m
6m & d
0m & d 3m 3m & s 6m
6m S
d
RANDOM
3.70#am
3.75=am
3.76
15.5+10
16.1
15.3
NEAREST
1.7620m
4.73:10.0
42.7400
U-Net (GIMO)
1,58
2.47aum
3.35
49.2467
17.8420
3.920
U-Net (THOR-MAGND)
1.82
3.292am
4.231000
40.1
18.9296
9,521A
U-Net (LocoVR)
0,83+nm
1.89
3,45
72.2416
40.1
13.5+27

---

Published as a conference paper at ICLR 2025
Qualitative results: F shows the results of object prediction. In LocoVR, as the trajectory progresses, the probability distribution of the goal area narrows down near the true goal object. This is due to the model learning from the dataset a policy that narrows down the goal area based on the areas already passed and the current heading direction. On the other hand, GIMO and THOR-MAGNI do not include sufficient number of trajectories scenes to learn policy applicable to unseen scenes, resulting in the probability distribution of the goal area not being appropriately narrowed down.
GIMO
THOR-MAGNI
LocoVR
Goel position prediction
Goal object prediction
Goal posítion prediction
Goal object prediction
Goal position
Goal object prediction
prediction
Past trajedtry
Groundrruth goal posiiiie
2001
predecion
3--11
goel
dt
dt
dt
dt
dt
dt
goal position
Pradicted
objest (fo)
Prediced
goal
paoducieg
object
dt
dt
dt
dt
dt
Predidted goal object (hr))
Wallkabl area
jobstasiie)
poal
presiction
Figure 5: Predicted goal positions and objects. In the left column with LocoVR, the predicted goal area distribution (yellow color) is narrowed down as the trajectory proceeds. As result, candidates of the goal object are accurately predicted (right column).
LIMITATIONS AND FUTURE WORK
Although we have evaluated the prediction models through the data collected real space (LocoReal), the gaps could affect detailed human behaviors, such as walking speed or interpretation of non-verbal communication through facial expressions. Investigating these impacts and comparing LocoVR to fully synthetic datasets are important research questions for future work.
While Loco VR is designed to facilitate research on indoor human trajectories, serves as founda- tional resource for exploring the relationships between motion patterns, goal positions, and indoor scene geometrics. Beyond its core purpose, LocoVR shows promise for extended tasks such as inferring indoor layouts from trajectory data predicting human actions by incorporating trajectory Takeyama et al. 2024). Furthermore, it has potential applications diverse fields, including robot navigation, Al agents in VR, avatar control in both real- aal-wollSSchwaz ments Wang et al. (2024a), and human action prediction Takeyamm et al. (2024). With its versatility, LocoVR is poised drive innovation across these domains and beyond.
al. (2023) and VR environ-
CONCLUSION
To model geometrically and socially aware human trajectories in complex indoor environments, introduced the LocoVR dataset, which captures two-person social motion behaviors across 131 home environments, including accurate trajectory and detailed spatial information. we introduced three indoor tasks that utilize human trajectory: global path prediction, trajectory prediction, and goal prediction. Experimental results showed that the models trained with LocoVR outperformed other prior indoor datasets evaluated on the real-world test data. This indicates that our dataset facilitates adaptation to unseen indoor environments with complex geometries and social motion behaviors across variety of tasks. Furthermore, these findings demonstrate the potential of virtual environments for training models that generalize well to real-worlh applications. We envision the data collection method to expand the variety of indoor scenes used for training and propose the experiments as a standard benchmark for future research on human motion and trajectory analysis in indoor settings.
the experiments,
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This research was funded by Toyota Motor North America and conducted as a collaborative effort between its members and those from UCSB. We sincerely appreciate the members of the Human-AI Experience (HAX) Lab and other students for their participation in data collection experiments and their valuable feedback. We also extend our gratitude the members of Toyota Motor North America, Toyota Motor Corporation, and Toyota Central Research and Development Labs Inc. for their insightful discussions and contributions. Additionally, the human subject study conducted in this research was reviewed and approved by the UCSB Human Subjects Committee under protocol number 14-22-0646.
REFERENCES
Joao Pedro Araújo, Jiaman Li, Karthik Vetrivel, Rishi Agarwal, Jiajun Wu, Deepak Gopinath, Alexander William Clegg, and Karen Liu. Circle: Capture in rich contextual environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
21211 21221,, 2023.
Jong Wook Bae, Jungho Kim, Junyong Yun, Changwon Kang. Jeongseon Choi, Chanhyeok Kim, Junho Lee, Jungwook Choi, and Jun Won Choi. Sit dataset: socially interactive pedestrian trajectory dataset for social navigation robots. Advances Neural Information Processing Systems, 36, 2024.
Jack Brookes, Matthew Warburton, Mshari Alghadier, Mark Mon-Wer Williams, and Faisal Mushtaq. Studying human behavior with virtual reality: The unity experiment framework. Behavior research methods, 52:455 463, 2020.
Judee Burgoon and AS Ebesu Hubbard. Cross-cultural and intercultural applications of expectancy violations theory and interaction adaptation theory. Theorizing about intercultural communication, pp. 149-171, 2005.
Judee K Burgoon, Lesa A Stem,, and Leesa Dillman. Interpersonal adaptution: Dyadic interuction pattern.. Cambridge University Press, 1995.
Zhe Cao, Hang Gao, Karttikeya Mangalam, Qi-Zhi Cai, Minh Vo, and Jitendra Malik. Long-term human motion prediction with scene context. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pp. 387-404. Springer, 2020.
Patrick Dendorfer, Aljosa Osep, and Laura Taixé Goal-gan: Multimodal trajectory prediction based on goal position estimation. In Proceedings of the Asian Conference Computer Vision, 2020.
Andreas Ess, Bastian Leibe, and Luc Van Gool. Depth and appearance for mobile scene analysis. In 2007 IEEE 11th international conference computer vision, pp. 1-8. IEEE, 2007.
Mark Nicholas Finean, Luka Petrović, Wolfgang Merkt, Ivan Marković, and Ioannis Havoutis. Motion planning in dynamic environments using context-aware human trajectory prediction. Robotics and autonomous systems, 166: 104450. 2023.
Franz Franco Gallo, Hui-Yin Wu, and Lucile Sassatelli. Divr: incorporating context from diverse vr scenes for human trajectory prediction. arXiv preprint 24ti:22111008009 2024.
Vladimir Guzov, Aymen Mir, Torsten Sattler, and Gerard Pons-Moll. Human poseitioning system (hps): 3d human pose estimation and self-localizztion in large scenes from body-mounted sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
4318-4329, 2021.
Edward T Hall. A system for the notation of proxemic behavior. American anthropologist, 65(5): 1003-1026, 1963.
Peter E Hart, Nils Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum paths. IEEE transactions on Systems Science and Cybernetics, 4(2):100-107, 1968.
11

---

Published as a conference paper at ICLR 2025
Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J. Black. Resolving 3D human pose ambiguities with 3D scene constraints. International Conference on Computer Vision, pp. 2282-2292, October 2019. TRtps:ss: is. uue mpg de
Noriaki Hirose, Dhruv Shah, Ajay Sridhar, and Sergey Levine. Sacson: Scalable autonomous control for social navigation. IEEE Robotics and Automation Letters, 2023.
Valentin Holzwarth, Joy Gisler, Christian Hirt, and Andreas Kunz. Comparing the accuracy and precision of steamvr tracking 2.0 and oculus quest in a room scale setup. In 2021 the Sth International conference virtual and augmented reality simulations, pp. 42-464,4- 46, 2021.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Parth Kothari, Sven Kreiss, and Alexandre Alahi. Human trajectory forecasting in crowds: A deep learning perspective. IEEE Transactions on Intelligent Transportation Systems, 23(7):7386-7400, 2021.
Sunmin Lee, Sebastian Starke, Yuting Ye. Jungdam Won, and Alexander Winkler. Questenvsim: Environment-aware simulated motion tracking from sparse sensors. In ACM SIGGRAPH 2023 Conference Proceedings, pp. 1-9, 2023.
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6): 248:1-248 1-248:166, 16, October 2015.
Karttikeya Mangalam, Yang An, Harshayu Girase, and Jitendra Malik. From goals, waypoints & paths to long term human trajectory forecasting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15233-15242, 2021.
Luis Manso, Pedro Nuñez, Luis V Calderita, Diego R Faria, and Pilar Bachiller. Socnavl: A dataset to benchmark and learn social navigation conventions. Data, 5(1):7, 2020.
Matterport. Matterport: High-fidelity capture of your manufacturing facilities. matterporr. com/ 2023.
https://
Santhosh Kumar Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alexan- der Clegg, John M Turner, Eric Undersander, Wojeiech Galuba, Andrew Westbury, Angel X Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-matterpont 3d dataset (HM3d): 1000 large-scale 3d environments for embodied AI. In Thirty-ffth Conference Neural Information Processing Systems Datasets and Benchmarks Track, 2021. MtLtttttp///xxiv /bs/ 08238
Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese. Learning social eti- quette: Human trajectory understanding in crowded scenes. In European Conference Computer Vision, 2016. URL http:::://///
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention-MICCAl 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part IIl 18, pp. 234 244.241.555 Springer, 2015.
Root Motion. Root motion: Professional inverse kinematics animation tools. com/ 2024. Accessed: 2024-08-17.
http://
Andrey Rudenko, Tomasz Kucner, Chittaranjan S Swaminathan, Ravi T Chadalavada, Kai O Arras, and Achim Lilienthal. Thör: Human-robot navigation data collection and accurate motion trajectories dataset. IEEE Robotics and Automation Letters, 5(2):676-682, 2020.
Amir Sadeghian, Vineet Kosaraju, Ali Sadeghian, Noriaki Hirose, Hamid Rezatofighi, and Silvio Savarese. Sophie: An attentive gan for predicting paths compliant to social and physical constraints. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1349- 1358, 2019.
12

---

Published as a conference paper at ICLR 2025
Tim Schreiter, Tiago Rodrigues de Almeida, Yufei Zhu, Eduardo Gutierrez Maestro, Lucas Morillo- Mendez, Andrey Rudenko, Luigi Palmieri, Tomasz P. Kucner, Martin Magnusson, and Achim J. Lilienthal. Thör-magni: A large-scale indoor motion capture recording of human movement and robot interaction, 2024.
Max Schwarz, Christian Lenz, Raphael Memmesheimer, Bastian Pätzold, Andre Rochow, Michael Schreiber, and Sven Behnke. Robust immersive telepresence and mobile telemanipulation: Nimbro wins ana avatar xprize finals. 2023 REE--AS 22nd International Conference on Humanoid Robots (Humanoids), pp. 1-8. IEEE, 2023.
Adalberto I. Simeone, Ifigeneia Mavridou, and Wendy Powell. Altering user movement behaviour in virtual environments. IEEE transactions visualization and computer graphics, 2314)1131
2017.
Nanami Takahashi, Tetsumari Inamura, Yoshiaki Mizuchi, and YongWoon Choi. Evaluation of the difference of human behavior between vr and real environments in searching and manipulating objects in domestic environment. In 2021 30th IEEE International Conference Robot & Human Interactive Communication (RO-MAN), pp. 454-460. IEFF, 2021.
Kojiro Takeyama, Yimeng Liu, and Misha Sra. Tr-llm: Integrating trajectory data for scene-aware llm-based human action prediction. arXiv preprint arXiv:2410.03993 2024.
Edward Vendrow, Duy Tho Le, Jianfei Cai, and Hamid Rezatofighi. Jrdb-pose: A large-scale dataset for multi-person pose estimation and tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 481 4820, 2023.
VIVE. Htc vive. https:
vi www. uu/n 2023.
Cheng Yao Wang, Eyal Ofek, Hyunju Kim, Payod Panda, Andrea Stevenson Won, and Mar Gonzalez Franco. Avatarpilot: Decoupling one-to-one motions from their semantics with weighted inter- polations. In 2024 International Symposium on Mixed and Augmented Reality. IEEE, October 2024a.
Jiashun Wang, Jessica Hodgins, and Jungdam Won. Strategy and skill learning for physics-based table tennis animation. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024b.
Zan Wang. Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Humanise: Language- conditioned human motion generation in 3d scenes. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
O Michael Watson. Proxemic behavior: A cross-cultural study, volume Walter de Gruyter GmbH & Co KG, 2014.
Hui-Yin Wu, Florent Alain Sauveur Robert, Franz Franco Gallo, Kateryna Pirkovets, Clément Quere, Johanna Delachambre, Stephen Ramanoël, Auriane Gros, Marco Winckler, Lucile Sassatelli, et al. Exploring, walking, and interacting in virtual reality with simulated low vision: living contextual dataset. 2023.
Jiangbei Yue, Dinesh Manocha, and He Wang. Human trajectory prediction via neural social physics. In European Conference on Computer Vision, pp. 376-394. Springer, 2022.
Haoran Yun, Jose Luis Ponton, Alejandro Beacco, Carlos Andujar, and Nuria Pelechano. Exploring the role of expected collision feedhack in crowded virtual environments. In 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pp. 472-481 481. IEEE, 2024.
Siwei Zhang, Qianli Ma, Yan Zhang, Zhiyin Qian, Taein Kwon, Marc Pollefeys, Federica Bogo, and Siyu Tang. Egobody: Human body shape and motion of interacting people from head- -ounted devices. In European Conference on Computer Vision, pp. 180-200. Springer, 2022.
Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, Karen Liu, and Leonidas J Guibas. Gimo: Gaze-informed human motion prediction in context. In European Conference Computer Vision, pp. 676-694. Springer, 2022.
Bolei Zhou, Xiaogang Wang, and Xiaoou Tang. Understanding collective crowd behaviors: Learning a mixture model dynamic redestrian-agents In 2012 IEÉE Conference on Computer Vision and
Pattern Recognition, pp. 2871-2878 IEFE, 2012.
13

---

Published as a conference paper at ICLR 2025
A ETHICAL IMPLICATIONS
While our dataset provides valuable insights into adult locomotion patterns, it lacks sufficient diversity in age groups and motor abilities. This homogeneity restricts the model's generalizability to individuals outside the able-bodied adult demographic. To address this limitation, future work should focus on collecting data that encompasses a broader spectrum of ages and motor capabilities, such as children, elderly individuals, and people with mobility impairments. This will allow the model to develop a more comprehensive understanding of human movement and improve its ability predict trajectories across a wider range of scenarios.
B EXPERIMENTAL DETAILS
We use the Adam optimizer (Kingma & Ba 2014) to train the U-Net models used in the experiments. The learning rate 5e-5, and the batch size is 16. Each model trained for up to 100 epochs on single NVIDIA RTX 4080 graphics card with 8G memory.
In the U-Net models, time-series trajectory is handled in a multi-channel image format. Specifically, the 2D coordinate of position trajectory is plotted on blank image (256 by 256 pixels) with Gaussian distribution, and the time-series data is contained in multi-channels. Similarly, the goal position is encoded into the model.
image and concatenated with the multi-channel trajectory image when fed
Further details of U-Net models described follows.
Global path planning
Input: (62xHxW)
Past trajectory of Pl for 15 epochs (15xHxW)
Past trajectory of P2 for 15 epochs (15xHxW)
Past heading directions of Pı for 15 epochs (15xHxW W)
Past heading directions of p2 for 15 epochs (15xHxW)
Goal position of Pı (IxHxW)
Binary scene map (1)HxW)
Output: (2xHxW
Pi static future global path (goal conditioned) (1xHxW) W)
- P2's s static future global path (non-goal conditioned) (I (1xHxW)
Groundtruth: (2xHx W)
pi 's static future global path (IxHxW)
P2's s static future global path (1xHxW HxW) W)
Loss: BCELoss between the output and ground-truth
U-Net channels:
encoder: 128, 128, 256, 256, 256
decoder: 256, 256, 256, 128, 128
Calculation time for training: 10-12 hours Loco VR
Trajectory prediction
Input: (61xHxW)
Past trajectory of Pi for 15 epochs (15xHxW) W)
Past trajectory of P2 for 15 epochs (15xHxW)
Past heading directions of P1 for 15 epochs x15xHxW)
Past heading directions of p2 for 15 epochs (15xHxW)
Binary scene map xXx
14

---

Published as a conference paper at ICLR 2025
Output: (90xHxW)
's future trajectory (45xHxW)
P2 s future trajectory (45xHxW)
Groundtruth: ())
pi future trajectory (45xHxW)
P2 's future trajectory (45xHx) W)
Loss: BCELoss between the output and ground-truth
U-Net channels:
encoder: 128, 128, 256, 256, 256
decoder: 256, 256, 256, 128, 128
Calculation time for training: 20-22 hours on Loco VR
Goal prediction
Input: (18IxXxW)
Past trajectory of Pt for 90 epochs (90xXxW) W)
Past heading directions of P1 for 90 epochs (90xHxW) Binary scene map (1 (1xHxW) W)
Output: (1 W))
Pi 's goal position (IxHxW)
Groundtruth: (IxHxW) W)
Pi 's goal position (IxHxW)
Loss: BCELoss between the output and ground-truth
U-Net channels:
encoder: 256, 256, 512, 512, 512
decoder: 512, 512, 512, 256, 256
Calculation time for training: 30-35 hours Loco VR
The scene map was sampled from the 3D room dataset (HM3D) by manually cutting the predefined area and projecting the height map onto the aerial-view image that covers area of 10m by 10m. The height map is then thresholded at 0.2m and converted into the binary map representing the walkable area.
C ABLATION STUDY
We conducted ablation study Loco VR analyze the factors that leverage the strengths of the dataset. Specifically, we imposed constraints on data scale, multi-person data usage, and heading direction usage in this study.
To investigate the impact of dataset scale, we created two types of scaled-down datasets by removing data from LocoVR, data-size-G and data-size-T. data-size-G is a dataset simulated to match the scale of GIMO, with the number of scenes and trajectories reduced to 19 and 190, respectively. by randomly selecting and removing data. data-size-T is simulated dataset modeled after THOR- MAGNI, containing 658 trajectories across 4 scenes. This dataset has fewer trajectories than the actual THOR-MAGNI dataset because Loco does not have as many trajectories per scene (See fig [0 in Appendix M). However, attempted to maximize the number of trajectories within the constraints of four scenes. addition, we evaluated the impact on the performance by considering the other person's 's movement (wo/p2) and heading direction (wo/head).
Table 5 presents the result of the ablation study on global path prediction. Due to the constraints, performance has deteriorated compared to the original LocoVR dataset. Notably, the reduction in dataset scale has a significant impact on performance, underscoring the importance of dataset scale in enhancing performance, which is key strength of our dataset.
15

---

Published as a conference paper at ICLR 2025
Table 6 shows the ablation study the trajectory prediction. The performance of LocoVR dete- riorated when constraints were applied to the original LocoVR, highlighting the strengths of its features.
Table represents a result of the ablation study on the goal prediction. Similar to the two tasks above, the performance improvement due LocoVR's features is also demonstrated in this table. In data-size-G, the object prediction accuracy within d< 3m is comparable to that of the original LocoVR, whereas it falls significantly short other metrics. This is because the model trained on data-size-G tends to rely on the current location due to the lack of training data, resulting in higher accuracy when the goal close to the current position.
Table 5: Global path prediction Mean and Max Chamfer distance between predicted and ground- truth paths grouped by distance to the goal.
Method
Orn d 3m
3m S d 6m
6m S
On
3m
3m
S 6m
6m
A + U-Net RLocoVR data-size-G)
0.077
0.194
0,326
0,180
0.438
0.752
A# +
U-Net (LocoVR data-size-T)
0.076
0.163
0.242
0,186
0.417
0.627
A+ U-Net (1 (LocoVR wi/ps)
0.061
0.122
0.205
0.147
0.297
0.537
U-Net (LocoVR worhead)
0.063
0.134
0.238
0.150
0.321
0.614
A U-Net (LecoVR)
0.060
0.119
0.192
0.145
0.290
0.501
Table 6: Trajectory prediction ADE (Average Displacement Error) between predicted and ground-
truth time-series trajectories.
Method
Om S S 3m
3m S d
6m
6m S
U-Net (LocoVR data-size-G)
0.274
0.496
0.775
U-Net (LocoVR data-size-T)
0.144
0.297
0.505
U-Net (LocoVR wolps)
0.113
0.254
0.470
U-Ner (LocoVR wo/head)
0.122
0.254
0.446
U-Net (LocoVR)
0.111
0.238
0.441
Table 7: Goal prediction Goal position error and goal prediction accuracy.
Goal position error
Object prediction accuracy
Method
Om S d 3m
3m S S 6m
6m S
Om S d. 3m
3m S d' 6m
6m S
U-Net (LocoVR data-size-G)
0.923
2.165
3.791
73.5
28.9
7.0
U-Net (LocoVR data-size-T)
1,552
2.479
4.118
63.6
34.8
6.8
U-Net (LocoVR wothead)
L.055
2.151
3,403
62.9
25.4
U-Net (LocoVR)
0.83
1.89
3.45
72.2
D ADDITIONAL EXPERIMENTS
D.1 TESTING ON GIMO
While the main paper utilized real-world trajectory data (LocoReal) as the test data, we conducted an additional experiment further validate the contribution of LocoVR using an alternative test dataset. Given that THOR-MAGN has fewer variational scenes, we selected GIMO as the test data, In this experiment, GIMO was divided into 70% for training, 15% for validation, and 15% for testing, ensuring that the scenes in the training, validation, and test sets were mutually exclusive. Additionally, to mitigate potential bias the test data, we performed five random splits to generate five different datasets and averaged the results. Tables through 10 illustrate the evaluation results for global path planning, trajectory prediction, and goal prediction. respectively. The evaluation results demonstrate that the model trained with LR consistently outperforms those trained other datasets. This superior performance is attributed to the enhanced generalization capabilities provided by Loco extensive coverage of scenes and trajectories.
16

---

Published as a conference paper at ICLR 2025
Table 8: Global path prediction Mean and Max Chamfer distance between predicted and ground- truth paths grouped by distance to the goal.
Mean
Mox
Method
Om S d
3m S
6m
6m S
Om d'S 3m
3m S 6m
6m E
U-Net (GIMO)
0.161
0.180=00026
0.277=00026
0,399
0.517#6.733
A N-t (THOR MAGNI)
0.151
0.19220001e
0.277::0:04
0.389
0.515
A + U-Net (LocoVR)
0.129
0,150=a0113
0.166=00.294
0.277
0.384
0.485
Table 9: Trajectory prediction - ADE (Average Displacement Error) between predicted and ground- truth time-series trajectories.
Method
S S 3m
3m d <
6m d
U-Net (GIMO)
0.21 2000000
0.382:0089
0.673
U-Net (THOR-MAGNI)
0.795
1.565=mm111
2.356 1800
U-Net (LocoVR)
0.140
20000
0.356 1005
Table 10: Goal prediction Goal position error.
Gool position
Method
0m S d 3m
3m d S 6m
6m d
U-Net (GIMO)
199
2,40320.5609
4.446204m
U-Net (THOR-MAGNI)
1.766
3.065:0,207
n.332
U-Net (LocoVR)
1.054
2.100
3.206
D.2 INFLUENCE OF SCENE INFORMATION BINARY OBSTACLE HEIGHT SEMANTICS
To explore the potential utility of the semantic and height information included in the scene map, we conducted a small experiment to evaluate how replacing binary obstacle maps with 3D height maps and semantic maps affects performance. Table I presents the results of the global path prediction task using the UNet+A* model. Each model was trained and tested on LocoVR with binary maps, height maps, and semantic maps, height and semantic maps clearly outperformed those trained with binary maps. Although we do not yet have detailed analysis of these findings, they potentially suggest that human trajectories could be influenced by object attributes inferred from height and semantic information. For instance, participants might unconsciously maintain a distance from movable objects, such as chairs or doors, or adjust their trajectories based on the visual clearance provided by different object types. For example, walls, kitchen counters, and low tables offer varying degrees of vision clearance, with lower clearance potentially exerting subtle psychological pressure on trajectory planning. A detailed analysis on influence of variational scene information on the human trajectories could provide valuable insights from the perspectives of cognitive and behavioral sciences.
three trials. As shown in the table, the models trained with
Semantic map
Binary map
Figure 6: Examples of binary (obstacle) map, height map, and semantic map
17

---

Published as a conference paper at ICLR 2025
Table 11: Global path prediction with binary/semantic/hhightt maps
ADE (Average Displacement
Error) between predicted and ground-truth time-series trajectories. The table reports averaged
performance in all of the trajectories over three trials + SD
Method
On S d 3m 3m S g 6m
6m S d
Binary map
0. 0.187220004
0.286200013
Semantic map
1.137
1.7010.006
0.21610:078
Height map
0.136:00001
0.165:20008
0.201
E SOCIAL MOTION BEHAVIOR
Typically. person's 's trajectory is influenced by the movements of others who are close by, as people naturally consider how their motion behavior might impact others in close proximity and modify their own motion behavior to accommodate others. Even when people are bit farther apart, social motion behaviors can still occur, such as navigating around each other to respect personal spaceBurgoon & Hubbard 2005) choosing less direct route avoid collision.
Our Loco VR dataset offers a unique perspective on social navigation dynamics within home environ- ments by focusing on how people navigate shared spaces. Nearly half of the trajectories in our dataset involve individuals coming within 1.5 meters of each other (as seen in Figure [6]. capturing a range of direct and indirect interpersonal space interactions. By analyzing the overlap of personal space volumes, we can identify moments of close proximity that require mutual awareness and behavioral adjustments. These interactions. while not as overt handshakes or object exchanges, reveal subtle yet crucial aspects of cohabitation. They showcase how individuals modify their movements in response to another's s presence - slowing down, altering paths, maintaining respectful distances, or yielding path. This focus on spatial negotiation provides valuable insights into the unspoken choreography of daily life that occurs when sharing living quarters.
7 illustrate three types of social navigation dynamics between two individuals (pl and p2) in the real-world test dataset LocoReal. Each figure exemplifies different types of social motion behaviors, including maintaining social distance, stepping aside to allow others pass, and choosing which side of an object to pass on to avoid crossing paths. Also, We demonstrate our model's capability to perform social navigation by considering the trajectories of others. Each shows the results with and without using P2's past trajectories as input, while each column represents a different sample scene. We used the model A ++U-Ne((LocoVR)) presented in Section4.3.1 throughout this experiment.
The dark green, light green, and red lines represent pl's past trajectory, true future path, and the predicted path by our model, respectively. The orange circle marks pl's goal position. Additionally, the blue line and light blue arrow indicate P2's past trajectory and direction of movement, respectively.
Scene to 3 depict scenes where pl and p2 are about to cross paths. The groundtruth future path (light green) shows that pl maintains an appropriate social distance from p2's path. Focusing the predicted future trajectories by our model (red), the upper row of the figure shows that the model predicts a path that overlaps with p2's heading direction since the model is not able to obtain p2's movement at all. In contrast, the lower row demonstrates that the model generates path that maintains a certain distance from p2's s heading direction, similar to the ground truth. This indicates that the model has effectively learned social motion behavior from LR dataset.
Scene 4 to 6 illustrate situations where pl chooses longer paths to avoid interference with p2. The ground truth future path of pl demonstrates choice that avoids potential proximity to p2 by selecting a route where p2 is absent. In the upper row, where the model does not consider P2's trajectory, the predicted path generally follows the geometrically efficient route to the goal, which could potentially overlap with p2's movement area. In contrast, the lower row shows that the model prioritizes avoiding potential proximity to p2, indicating that it has learned to account for social motion behavior.
18

---

Published as a conference paper at ICLR 2025
Scene 1 (social distance)
Scene 2 (social distance)
Scene 3 (social distance)
LocoVR single-person
trajectory learned
LocoVR
two-persan
trajectory learned
Scene 4 (detour)
Scene 5 (detour)
Scene 6 (detour)
LocoVR single-person
trajectory learned
LocoVR
two-person
trajectory
tearned
trajectory (p1)
trajectory
Walkable area
Predicted future global path (pf)
Heading direction
Non-walkable area (obetacle)
Groundtrath future global path (p1)
map ((x))
Groundiuuh
thigher proportion yellow comeaponding lover couts, puidingthe global
Figure 7: Social motion behavior
F VR SYSTEM FOR DATA COLLECTION
F. SYSTEM STRUCTURE
The system receives real-time tracking data from the motion trackers worn by each participant. This data used update the avatars in the virtual environment they accurately reflect the poses and movements of the participants. Each virtual avatar is represented by SMPL mesh (Loper et al. 2015), calibrated to match the proportions each person's 's body using FINAL-IK (Root Motion 2024). To encourage movement and social motion related behaviors between the two participants, system generates goal object that each person needs to reach as part the data collection task. This gives the participants reason to move around and interact with each other in the virtual space, resulting in social behaviors, such as waiting for someone to pass before proceeding or backtracking for oncoming person if the path is too narrow to allow them to cross.
F.2 VR HARDWARE
Using the HTC VIVE system (VIVE, 2023), we track the movements two people as they explore virtual space. Each person wears VR headset, holds controller in each hand, and has three motion trackers (VIVE pucks) on their body two on the ankles and one on the torso, for total of six tracked points. This setup allows to capture the full range of body movements as the participants interact with the virtual environment and with each other. The HTC VIVE's outside-in tracking system uses the six tracked points each person's body caleulate their absolute pose and position with high
19

---

Published as a conference paper at ICLR 2025
degree of accuracy. With tracking frequency of 90 Hz and accuracy within few millimeters (Holzwarth et al. 202T), the system can track small movements and gestures in real-time to provide a highly responsive and immersive experience, thereby eliciting natural walking and social behaviors necessary for collecting realistic data.
F.3 3D VIRTUAL ROOM DATA
We use the Habitat-matterpon 3D dataset (Ramakrishnan et al. 2021 to create the virtual scenes. The dataset includes more than 1K 3D indoor spaces, which are captures of actual rooms using the Matterport 3D scanner [Matterport 2023). Some of the scenes have semantic information added that Matterport has manually labeled. We used 131 scenes with full 3D scene geometry and semantic labels to collect our data.
F.4 ALIGNMENT OF VR AND THE REAL SPACE
The data collection conducted in VR lab (10m 10m), which was larger than every virtual room we used. Firstly, we aligned the centers the virtual and the physical rooms so that the virtual room was totally contained in the physical room. During data collection, goal positions were controlled to appear in the predefined virtual room to make the participants walk safely without getting close to the physical walls. If a participant got close the physical walls, a virtual guardian appeared, indicating to the participant that they were too close to the boundary of the virtual space. This is built-in safety feature of the VR headset we used.
F.5 AVATAR CALIBRATION
We employed inverse kinematics software, FINAL-IK reconstruct avatar motion from tracked body motions on the hand/wais/hands/feeet. Due to the performance of FINAL-IK, the avatar motions may occasionally display unnatural joint movements. However, these slight inaccuracies in body motion do not impact the contribution of our experiment, rather than fine- ree--grined body movements. Our dataset currently includes raw data from sparse motion trackers, which is highly accurate (within few millimeters). For users requiring precise avatar motion, applying state-of-the-art IK algorithms to the raw tracker data would reconstruct more accurate avatar movements than those displayed in our video.
focus is room-scale human dynamics
G LOCOREAL: A DATASET FOR TESTING IN THE REAL WORLD
Although LocoVR is collected in highly realistic virtual environments and useful for learning human trajectory considering the surrounding environment, it is general concern that there might be difference in human perception between the physical and virtual space that results in performance degradation when transferring from the virtual the real world. To address the concern, we built LocoReal, human trajectory dataset in the physical space, which can be used as test data to show that the model trained with LocoVR can be utilized in the real environment.
Collecting real-world human trajectory data was done in an empty room in a campus building. Two participants walked conduct a task in the room where several pieces of furniture were placed, and their 3D motions and trajectories were captured by motion capture system. The experiment was conducted in different layouts with participants, resulting in 450 collected trajectories. Figure illustrates the binary maps of the 4 scenes collected in LocoReal.
H DATA AUGMENTATION
All the training data are augmented by horizontal flipping and rotation with 90, 180, and 270 degrees, increasing the number of training data by eight-fold. We also augmented the data in time-series direction to obtain different sets of past trajectories and ground-truth future trajectories. We define a fixed length time window and slide it with certain interval to obtain sets of augmented trajectories. Fig[Issee shows an overview of the data augmentation strategy. Each task has a different strategy in data augmentation.
20

---

Published as a conference paper at ICLR 2025
Scene
Scene
Scene 3
Scene 4
Semantic map (with trajectories)
map
Obstacle map (binary)
Figure 8: Maps data of LocoReal dataset. We collected the dataset with 5 participants performing tasks in the shown 4 different layouts of physical room.
For the global path prediction, the time window with a length of 10.0s (1.0s for past trajectory and 9.0s for future trajectory) slides with interval of 0.67s to extract trajectory segments from the raw trajectory. The duration of 10.0s is chosen to encompass the majority of trajectories in the dataset (as shown in Figure [3), while a time step of 0.67 seconds sufficient capture human movement at speeds of up to 3.0 m/s. A part of the trajectory segment that is out of the raw trajectory is padded with the last value observed in the raw trajectory.
For the trajectory prediction, the time window with length of 4.0s (1.0s for past trajectory and 3.0s for future trajectory) slides with an interval of 0.67s to extract trajectory segments from the raw trajectory. We assume that 3.0s is appropriate time range for future trajectory prediction, this task
and affected by rapidly increasing positional uncertainty over time, Note that all the trajectory segments within the trajectory because the task focuses trajectory prediction local areas on the way to the goal.
For the goal prediction, the time window with length of 6.0s (for past trajectory) slides with an interval of 0.67s to extract trajectory segments from the raw trajectory. We set 6.0 seconds as the maximum length for past observations, the primary objective is predict the goal before it is reached, making long observation periods unnecessary for evaluating goal prediction performance. A part of the trajectory segment that is out of the trajectory is padded with the initial value observed in the raw trajectory.
Global path prediction
Trajectory prediction
Rowe Ingietery Oream stort to grad)
Part
Putare erpectery
Pedded with
Trijectory (1)) Trijectry (3)
csjectsris
Goal prediction
traeetory from goal)
Gool poottios
irgectettes
Podded
divervation
Figure 9: Data augmentations in the time direction
21

---

Published as a conference paper at ICLR 2025
I LocoVR DATASET STATISTICS
In this section, we describe the statistics of the trajectory data contained LocoVR. We collected trajectory data from 32 participants in total, resulting in 7071 trajectory sequences after data pre- processing. Since we collected trajectories from two participants simultaneously, each participant's trajectory was counted separately. We removed short trajectories (less than 2m or 2s) and poor motion tracking data in the data preprocessing phase.
1.1 STATISTICS ON THE DATA
Figure 10 shows the number of trajectories collected each scene. The average number and standard deviation over 131 scenes are 54.0 and 32.0, respectively. The number of trajectories differs across scenes, resulting from the following factors. We collected a large number of trajectories in scenes where human interactions occur frequently (e.g., paths with bottleneck). Also, the number of trajectories is affected by the speed preferences of the participants. Given the same amount of time, participants walking fast gave us more trajectories than participants who walked slowly. Further, the stability of the motion- motion-tracking performance also affected the number of trajectories since trajectories with large tracking errors are removed in the data preprocessing.
Figure I1 Path efficiency is metric used to evaluate the complexity of trajectory, defined as the ratio of the straight-ling distance the goal to the actual traveled distance. The mean path efficiency in LocoVR is 0.81, comparable to that reported in THOR-MAGNI (Schreiter et al., (024). Notably, the data in THOR-MAGNI were collected in single, controlled experimental setting designed to emphasize the influence of surrounding objects or other pedestrians. In contrast, our dataset was collected diverse, naturally occurring home environments that were not controlled by experimenters.
Figure 1 shows the distance distributions the trajectories. The figure shows that more than half of the trajectories longer than 4m. Since virtual rooms usually smaller than 7m by 7m, the distribution reasonable to assume daily movements in a room. Figure [3 shows the travel time distribution of the trajectories. It shows that more than half of the trajectories are longer than 5s, which would be enough to learn locomotion in a single room.
Figure 14 shows the speed distribution of the trajectories. The mean value is around 0.8, indicating the walking speed is relatively low compared to the open outdoor space since the scale of home environments is smaller.
Figure 15 depicts the number of peaks observed in the speed history of individual trajectories. It shows that more than half of the trajectories exhibit two more peaks, indicating that participants frequently change their speed on their way to the goal. This behavior is likely influenced by interactions with other participants or the narrow and complex indoor environment.
Figure shows the minimum distance between two participants in each trajectory. It is shown that approximately 25% of the trajectories are within Im of the other participant, and more than 70% are within 2m (See Figure 18). b. It indicates that many of the trajectories could be influenced by the trajectories of the other participants when they are in close proximity, as people typically consider how their behaviors might affect others when they are located close to other people. In the rest of the cases where the participants were at least 2m away from each other, there could still be social motion behaviors that involve passing through each other at distance to respect other people's personal space or taking less direct route to the goal to avoid the risk of physical conflict with the other.
Figure 17 illustrates the distribution of closing speeds between participants. The relatively high closing speeds, considering the room scale, suggest that participants need to remain attentive to the movements of others.
22

---

Published as a conference paper at ICLR 2025
Number ofsceres
Path off: ency
Figure 10: Number of trajectories in each scene
Figure 11: Path efficiency (straight-line distance to the goal/traveled distance) in each trajectory.
Taue stance In!
Pavd Sine (second)
Figure 12: Travel distance in each trajectory
Figure 13: Travel time in each trajectory
6.58
1.2
3.00
Number speeed each
Figure 14: Speed of participants.
Figure 15: Number of peaks in speed history in each trajectory
1.2 STATISTICS ON THE PARTICIPANTS
There were 32 participants total, comprising males and females, with ages ranging from 18 to 42. From this pool, pairs were formed to conduct 25 experiments, each involving a unique pair [2). The experiments included various combinations male-male, female-female, and male- female pairs, well as pairs of friends and nonfriends, as shown in the Table As reactions between pairs in close proximity could influenced by attributes and interpersonal relationships, further data analysis may provide insights into the relationship between these attributes, relationships, and behavioral patterns. could be an intriguing study from the perspective of cognitive and behavioral sciences.
23

---

Published as a conference paper at ICLR 2025
Mitimum distance benween 2i and p2 [m]
Ocdig speed between. n. and (mvid
Figure 16: Minimum distance between partici- Figure 17: Closing speed between participants. pants in each trajectory.
Figure 18: Scene participants approaching each other
Table 12: Participants demographics
Numberr of porticipants
Age
Male
Female
Under 20
Over 30
Table 13: Diversity of pairs
Number of parficipants
Relationship
Male
Male
Female Female
Mole Female
Friends
2
2
Non friends
9
1
I.3 WIYY TWO PERSON ? MOTIVATION OF TWO-PERSON EXPERIMENT
In contrast to conventional studies that focus on crowd dynamics in open public spaces, our research emphasizes room-scale social motion behaviors between two individuals, particularly in confined indoor spaces. For example, interpersonal behaviors such as proxemics, trajectory negotiation, and mutual space adaptation are affected and induced by the narrow geometry. We consider this is fundamental study on the room-scale social motion behaviors, and believe this focus does not limit the dataset's utility. Researchers can build upon it to study two-person interactions isolation or as basis for modeling interactions in more complex, multi-person environments. In addition, more than two-person interaction could be considered as a rare scenario in home environments. For instance, recent census data indicates that in the US, 60% of households consist of two people fewer, and 80% consist of three people or fewer. Also, indoor home spaces are generally smaller than open public spaces, and activities in such settings often individualized. As result, residents
24

---

Published as a conference paper at ICLR 2025
typically move independently rather than engaging in collaborative coordinated movements within the same space. This relevance underscores the practical utility of our dataset for studying interaction dynamics that are directly applicable to real-world scenarios.
J INFLUENCE OF GAP BETWEEN VR AND THE REAL
We consider the virtual avatar effective tool for addressing the WRrreal-world gap. Additionally, we introduced a filter to exclude data exhibiting inappropriate hehaviors. In the following sections, discuss the impact of this gap.
(1) In our experiment, participants were aware that the virtual avatars were synchronized with real humans sharing the same physical space. Also, the avatar enables participants to percept relative position between their body and surrounding objects. These awareness discouraged socially or physically inappropriate behavior, mitigating the potential impact of the VR/real gap. as demonstrated in recent studies on VR locomotior Yun et al. ) (Simeone et al. (2017). In addition, we have introduced a filter to detect instances of users passing through virtual objects to remove such data from the dataset.
(2) Our evaluation used locomotion data collected physical spaces as test data. Models trained on the LocoVR dataset outperformed those trained on other physically collected datasets (GIMO/THOR- MAGNI), demonstrating that VR-collected data is effective when applied to real-world scenarios.
For the futurework, more expressive avatar to facilitate non-verbal communication during walk, or user interaction cues such as haptic devices could further reduce the gap between VR/Real, We anticipate that future advancements in VR technologies will further contribute to bridging this gap.
25


Reviews:
[{"review_id": "O0mTQPCCBd", "reviewer": "Reviewer_AaG4", "summary": "This paper introduces a dataset for multi-user indoor navigation collected in a Virtual Reality (VR) environment. The dataset includes 7,071 trajectories with 2.5 million frames across 131 scenes. Baseline methods, such as A* and U-Net, are used to demonstrate and analyze the proposed dataset.", "questions": "See above.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+ The dataset is based on real human subject studies, and the use of VR environments facilitates data collection efforts.\n\n+ This open-source dataset could benefit the research community focused on social navigation.", "weaknesses": "- Novelty and contributions are key concerns. Although the human subject studies require significant time and the dataset a reasonable sample size, the dataset has not been demonstrated, for example, to train state-of-the-art neural network models. The demonstrated methods are relatively simple. The dataset is also limited to two-person navigation scenarios.\n\n- Given that the objective of the dataset is to estimate user trajectories and goal positions, without addressing the estimation of human body motions, why is VR more advantageous than an overhead camera with a bird-eye view?\n\n- How are real human full-body motions in the physical world synchronized with the virtual environment? \n\n-  Research (e.g., in human-robot interaction) has shown that humans respond differently to virtual agents compared to physical agents. The authors are encouraged to provide a study and analysis on whether this difference exists and, if so, how significant it is.\n\n- The work mentions robotics as a motivation and application scenario; however, related research on social robot navigation is not well reviewed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xE57dp8pmU", "reviewer": "Reviewer_6iZX", "summary": "The paper introduces LocoVR, a virtual reality dataset that captures approximately 7,000 two-person trajectories across 130 indoor home scenes. The authors demonstrate the utility of LocoVR through three applications: global path prediction, trajectory prediction, and goal area prediction. The model trained on this dataset exhibits socially and geometrically aware navigation patterns within indoor scenes.", "questions": "The paper seems to focus on locomotion. Without interactions like sitting on a sofa or standing up from a chair, does the goal prediction remain compelling?\nWith many figures presented in 2D planes, would a bird's eye view semantic map provide enough information for the prediction tasks? What's the importance of 3D geometry?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The extensive collection of two-person human trajectory (with motion capture) data in indoor scenes, is a valuable resource for the community.\n2. Utilizing virtual reality for data collection is a promising approach, allowing more diverse 3D scenes when resources are limited.\n3. The data and code will be released.", "weaknesses": "1. While using VR to collect human trajectory data is helpful, this paper would benefit from a discussion in the related works section about VR and human motion. For instance, referencing works like \"QuestEnvSim: Environment-aware Simulated Motion Tracking from Sparse Data\" in SIGGRAPH 2023 which uses VR for motion tracking and \"Strategy and Skill Learning for Physics-based Table Tennis Animation\" in SIGGRAPH 2024 which involves interaction between human and humanoid agents.\n2. I notice authors utilize motion capture to provide whole body motion, and I wonder the reason to consider only experiments of path, trajectory and goal prediction. The occasionally unnatural motion observed in the video could be explained.\n3. The use of A* baselines seems inappropriate for two-person interaction scenarios. I notice this dataset mainly focuses on obstacle avoidance. There appears to be a lack of interactive behaviors between the two persons. It may not be enough if two persons just operate independently and avoid the other person within the same space. I think it doesn't reflect scenarios often seen in real life. Can the authors provide more information on the distribution of action types within the dataset? Given this is a dataset paper, more statistics and descriptions would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V1pBtC3tSi", "reviewer": "Reviewer_1K21", "summary": "1) LocoVR introduces a large dataset with 7000 two-person interactions across 130 diverse indoor environments in VR -- includes full body pose data and spatial information. \n3) LocoVR improves model performance in 3 indoor tasks including human trajectories and predicting socially aware navigation patterns in home environments.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1) LocoVR’s large, indoor locomotion dataset records two-person interactions across 130 diverse indoor environments \n2) Introduction of motion proxemics in a large-scale dataset which is useful for downstream tasks such as studying human-human interactions and potentially human-robot interactions\n3) Rigorously quantitatively evaluated against strong baselines; baseline configurations and settings are fair and well-documented \n4) Data of two-person trajectories is useful to study motion proxemics \n\nThe authors introduce an exciting/ relevant problem. The problem is well-motivated while the execution and evaluations are strong. I see many potential downstream applications and I believe that this will make a huge impact in the robotics field.", "weaknesses": "1) Potential Overfitting to VR-Specific Biases:\nI am curious what the authors have done to further minimize the gap between real-world scenes vs VR scenes. Are there any obstacle perception features e.g., vibrational feedback when participants bump into objects in the scene? \n\n2) *Qualitative results: ‘as the trajectory progresses, the probability distribution of the goal area narrows down near the true goal object’*\nI think that it is reasonable to assume that humans narrow the probability distribution in LocoVR closer to the true goal object. However, I would like to understand exactly why this observation arises from LocoVR’s dataset. Is it merely the size of the dataset or is it the difference in data collected in LocoVR vs the other datasets? Are the authors claiming that this is a strength as it mimics the probability distribution of human trajectories better? If so, I would like to see analysis of this phenomenon in the different datasets. \n\n3) Cultural/ societal biases of motion proxemics:\nSince motion proxemics is influenced by cultural norms, authors should show aggregated user demographics. It would also be interesting to see motion proxemics based on the aggregated clusters of people.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iuVpmN08QK", "reviewer": "Reviewer_Fpz9", "summary": "LocoVR is a virtual reality-based dataset aimed at improving the modeling of human locomotion in complex indoor environments. This dataset specifically focuses on multi-user indoor navigation, capturing over 7,000 two-person trajectories within more than 130 different home-like scenes. The main goal of the LocoVR dataset is to enhance the ability of AI systems, like home robots, to understand and predict human movement patterns that incorporate both spatial and social navigation dynamics.", "questions": "I would like future work to discuss how this type of work can be merged with other approaches like weighted interpolations to define trajectories of avatars indoors: https://www.microsoft.com/en-us/research/publication/avatarpilot-decoupling-one-to-one-motions-from-their-semantics-with-weighted-interpolations/", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "By using VR, the dataset captures detailed spatial data and full-body motion in diverse home environments. The VR setup enables controlled capture of social navigation behaviors, such as maintaining personal space and avoiding collisions in shared spaces like entryways.\n\nThe dataset is well evaluated. The paper evaluates the dataset on three trajectory-based tasks—global path prediction, trajectory prediction, and goal prediction. The results show that models trained on LocoVR outperform those trained on other datasets, particularly in predicting realistic, socially aware navigation paths in complex environments.\n\nI can see implications of this work not just for virtual agents, games etc, but also as we move to further robotic presence in our homes this type of dataset can help train their trajectories.", "weaknesses": "I m not sure about the Non-Verbal Social Cues: such as gaze direction or facial expressions—that influence social navigation.\n\nThe two agent approach perhaps limits also its scope in multi-user indoor scenarios common in real homes with multiple occupants.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9Hu4oRjDhC", "reviewer": "Reviewer_PhCB", "summary": "This paper focuses on understanding human locomotion behaviors in indoor environments, with the specific scenario of two persons walking two separate goals in an indoor room. The main contribution is a new dataset LocoVR capturing sequences of two persons walking to their goal locations in 3D indoor rooms. To overcome the high cost of capturing locomotion in physical scenes, this paper proposes a VR-based solution where the subjects wear a VR device and navigate through a virtual 3D room displayed in VR. The human locomotion trajectories are tracked using VR devices. This VR capture solution scales up locomotion capture to 130+ scenes. The authors then conducted experiments validating that the collected LocoVR dataset outperforms existing datasets in three tasks: global path prediction, trajectory prediction, and goal prediction.", "questions": "1. In appendix H, why are time windows and intervals set as the presented numbers? Are there any motivation or empirical study?\n\n2.L860, figures should be tables?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed VR capture solution eliminates the high cost of physically setting up indoor scenes and capturing human movements, which facilitates scaling up locomotion capture to many more scenes compared to previous datasets.\n\n2. This paper captures two-person goal-reaching motions which include the social navigation behaviors such as adjusting the path to respect personal boundaries or side steps to give way to another person. Such social navigation behavior is not covered in most previous datasets and is important for understanding multi-human social navigation and potential human-robot interactions.\n\n3. Experiments on three navigation-related tasks show that models trained on the LocoVR dataset consistently outperform models trained on existing datasets when tested on a real-world two-person locomotion test set.", "weaknesses": "1. Although this paper claims to provide full body pose data (L20), the human motion capture is far from realistic according to the supplementary video (0:00-0:30). If aiming for capturing full body poses, it may be necessary to change from the HTC VIVE tracking to marker-based motion capture as in CIRCLE (Araujo et al., 2023). With the current presented results, I recommend removing the claims of full body pose data since all experiments only use trajectory data.\n\n2. The VR capture system is limited to simple behaviors assuming a flat floor scene and no contact-based close object interactions. It can only capture locomotion or reaching behaviors as in CIRCLE (Araujo et al., 2023). The VR capture system can not work for behaviors like lying on a sofa or walking up stairs. When the humans try to do such interactions in VR, they are actually interacting with air in the real world and will fall. This virutal-real inconsistency can also cause the subjects to slightly walk into obstacles as discussed in paper.\n\n3. This paper only focus on a very simple social navigation scenario of two persons avoiding each other. However, the social navigation behaviors can be much more complex. For example, humans do not only avoid each other but also collaborates and coordinates, consider the cases where one person is leading the way and other persons follow the leading one, and two persons walk to each other to talk. It is also necessary to include social scenarios with more than two persons.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)