# Toward phygital agent-based interactive simulations to support urban planning

Gary Perelman, Frédéric Amblard, Emmanuel Dubois, Benoit Gaudou, Elsy Kaddoum, Marc Mequignon, Cédric Sanza, Marcos Serrano, Antonio Serpa, Nicolas Verstaevel

<figure><img src="image_1.png" /></figure>

## To cite this version:

Gary Perelman, Frédéric Amblard, Emmanuel Dubois, Benoit Gaudou, Elsy Kaddoum, et al. Toward phygital agent-based interactive simulations to support urban planning. 6th International Workshop on Agent-Based Modelling of Urban Systems (ABMUS @ AAMAS 2021), May 2021, London (Virtual), United Kingdom. pp.36-40. hal-03226686

HAL Id: hal-03226686

https://hal.science/hal-03226686

Submitted on 15 May 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
Toward phygital agent-based interactive
simulations to support urban planning

Gary Perelman<fnref n="1" />[0000-0002-7649-2692], Frédéric Amblard<fnref n="2" />[0000-0002-2653-0857],
Emmanuel Dubois<fnref n="1" />[0000-0002-0479-1036], Benoit Gaudou<fnref n="2" />[0000-0002-9005-3004],
Elsy Kaddoum<fnref n="1" />[0000-0003-4610-4174], Marc Mequignon<fnref n="3" />[0000-0002-5284-6476],
Cédric Sanza<fnref n="1" />[0000-0001-9325-1113], Marcos Serrano<fnref n="1" />[0000-0001-8431-0601],
Antonio Serpa<fnref n="1" />[0000-0003-2873-4543], and Nicolas
Verstaevel<fnref n="2" />[0000-0002-7879-6681]

Contact Author: gary.perelman@irit.fr

**Abstract.** To support urban planners in assessing and comparing various alternative scenarios, this paper provides a novel methodological framework for phygital agent-based simulation, i.e. simulations with which the user can interact through the usage of a physical 3D model and visualize in augmented reality (digital).

**Keywords:** urban planning · agent-based model · phygital interaction · 3D animation

Urban planning practices have changed deeply over the last 50 years requiring now to consider the whole complexity of the urban area before planning a new development [5]. Indeed, planning the construction of a new building is not limited to ensuring the possible access to the building for people, vehicles, electricity, water... The city needs to be considered as a complex system and many other factors (material or immaterial, qualitative or quantitative...) and feedback loops need to be taken into account and understood. In addition, evolution of the individual behaviors (e.g. in terms of mobility mode) and institution regulation (e.g. the Local Urban Plan (PLU)) are key factors that can influence the impact of an urban planning. Finally making all these aspects more easily exploitable by different stakeholders is crucial.

Agent-based modeling and simulation has become a tool of choice to simulate such complex socio-environmental systems [6]: it is suitable to take into account very diverse dynamics from various fields and their feedback loops. Thanks to its expressivity, an agent-based model can become a very powerful virtual laboratory to experiment and understand the effects of alternative urban plans. A key challenge is now to make such tools available and usable by any user to help understanding and thus improve the acceptability of urban plans, but also by planners to provide insights and support their decision. This requires innovative approaches in terms of visualization of simulation and interaction with them.

<footnote n="1">¹ University of Toulouse, France</footnote>

<footnote n="2">² UMR 5505 IRIT, Université Toulouse 1 Capitole, France</footnote>

<footnote n="3">³ University of Toulouse, UPS, LERASS-ERPURS</footnote>
Much recent effort has focused on extending the limited input/output inter-
action capabilities of desktop computers through the use of interactive tabletops,
augmented reality [2] or tangible interaction [1]. In our context, we explore an
original interaction mode that we called "phygital interaction": it depicts the
use of a physical 3D model, representing the physical location to which a data
set is related. The use of 3D physical referent 1) provides support for tangible
interaction, which ensures a better memorization through proprioception, and
2) allows for a more situated data analysis, which provides a physical anchor
for data facilitating large data sets exploration and understanding [4]. Render-
ing of the data is then performed using a Head-Worn Display (HWD), such as
the Hololens, which provides a large immersive display area to augment the 3D
physical model.

Along with the physical 3D model, there is a need for a digital 3D model which will hold all the simulation information. This model would be superimposed on the physical 3D model to enhance the user's understanding of the impact of urban planning on ecology or traffic through the visualization of pollution clouds or pedestrians movement. Furthermore, urban planners use 3D representations to design new infrastructures. These models often consist in a static description of the different parts of the buildings, using classical file formats like CityGML or IFC. Such a single visualisation, even with high level of details needs to be integrated into a huge area to have a more precise vision of the project. In our approach, we propose to visualize not only the 3D model of the project but the environment that is directly impacted. The next step is then the animation of all the movable elements like pedestrians, cars, buses.

The objective of this work is to provide a methodological framework for
**phygital agent-based simulation**, i.e. simulations with which the user can
interact through the usage of a **physical 3D model** and visualize in augmented
reality (**digital**). Through different scenarios, we aim to test the impact of the
urban project on road traffic and more. In the following, we introduce the three
main components of the framework (Simulation, Interaction and 3D animation)
and the way they interact.

**Simulation component.** The *Simulation* component is in charge of computing
evolutions of the system and providing information about the current simula-
tion state to the Interaction and 3D animation components. Four simulation use
cases have been identified to provide an optimal phygital simulation experience:
(i) step-by-step simulation allowing to visualise and interact at runtime with the
simulation, (ii) visualisation of recorded simulations, in this case interactions
will be limited to query the 3D model to display information (no modification
is possible), (iii) comparison between 2 (or N) alternative simulations, and (iv)
interaction with the simulation at its initial state, to modify deeply the environ-
ment (e.g. create a new building). Visualisation and interactions with simulations
in real-time (cases (i) and (iii)) require fast (and thus reactive) simulations. In
these cases, the model or some of its dynamics will be simplified to preserve this
execution time constraint. The designed model should thus be modular enough
to adapt these constraints. The GAMA platform [6], an agent-based modeling
and simulation platform providing powerful tools to develop spatial models, has been selected to implement the models and run the simulations. GAMA provides a multi-simulation feature, allowing to run several synchronized simulations. It has already been used in several projects coupling simulation and tangible interface, in particular to study urban planning and the impact of new mobility [3].

**Phygital interaction component.** Two major interaction forms has been identified: exploring the simulation results and tuning the intrinsic simulation parameters. To explore the simulation dataset, users' inputs on the physical 3D model trigger the visualization of the data charts associated with the physical referent (building, street, red light). The chart is displayed above the model using the HWD and can be configured using different physical features (point, edge, surfaces) of the physical referent, e.g. a touch on the edges of a building allow to configure the axes of the 3D chart. The adjustment of the intrinsic simulation parameters corresponds to a modification of the physical environment's topology, or a digital modification of a parameter of the simulation engine. For example, a physical floor can be added to the desired building, using a 3D printed pluggable floor, to analyze the impact of higher population density on traffic. Another example is when the speed of a red light requires adjustments to streamline the traffic. To do so, the user may select the red light using a touch on the physical 3D model, then slide on its pole to change the light switching speed.

**3D animation component.** The main role of this component is to run pre-computed simulations in 3D realistic environments. Visualization is the last step of validation for urban planners. Each element of the scene (building, road, tree, street furniture) is rebuilt from open source data (OpenStreetMap, IGN) providing information about coordinates, height, name, function... We here focus on the buildings which is the most complicated part. To generate realistic buildings, we need accurate data. The first step is preprocessing to recompute the coordinates to make right angles and parallel sides. Then the set of GPS coordinates defines a polygon (footprint) that is extruded regarding the number of floors. The second step is for missing data, we use default values, hierarchically organized by areas, to provide two main characteristics: texture and roof. Then the roof is added and the textures are applied for each wall. Similar algorithms are used for each type of object (road, vegetation...). Finally, we can add 3D models proposed by urban planners.

**Overview of the interactions between the three components.** The 3D animation component is in charge of reading the input OSM data and to augment them with additional metadata (e.g. obligations and constraints from the PLU). This augmented file is then used for visualisation, to initialize the Simulation environment or, once transformed into an augmented STL file for the Interaction component. The Simulation initialisation is completed by demographic, synthetic data. During its run, the Simulation will transmit to the two other components updates of the simulation states (e.g. location, state...), to allow
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Workflow of our phygital simulation approach. (*Interaction* will use Hololens by Microsoft, image credit: https://www.visartech.com/blog/)</figcaption></figure>

them to visualise the simulation. When an interaction occurs, either the envi-
ronment is modified (e.g. add a new building), in which case the 3D *animation*
will regenerate a new 3D model for the 2 other components, or the parame-
ters of the simulation or agents' states are updated and only transmitted to the
*Simulation*.

Coupling agent-based simulations with a tangible interface has demonstrated
its benefits in terms of communication, but the improvement of the interactions
is mandatory to let it become part of the urban planner toolbox. The project
goal is thus to apply this approach to support Toulouse Metropole in its planning
of a new ward.

***Acknowledgements.*** This work was supported by the French Government in the framework of the Major Investment Plan (PIA3 - Territoire d'Innovation) and by Toulouse Metropole.

References

1. Duc, P.M., Chapuis, K., Drogoul, A., Gaudou, B., Grignard, A., Marilleau, N., Tri, N.H.: HoanKiemAir: simulating impacts of urban management practices on traffic and air pollution using a tangible agent-based model. In: RIVF. IEEE (2020)

2. Fonnet, A., Prié, Y.: Survey of immersive analytics. IEEE TVCG (2019)

3. Grignard, A., Alonso, L., Taillandier, P., Gaudou, B., Nguyen-Huu, T., Gruel, W., Larson, K.: The impact of new mobility modes on a city: A generic approach using ABM. In: International Conference on Complex Systems. Springer (2018)

4. Jansen, Y., Dragicevic, P., Isenberg, P., Alexander, J., Karnik, A., Kildal, J., Sub-ramanian, S., Hornbæk, K.: Opportunities and challenges for data physicalization. In: Proceedings of the 33rd Annual ACM CHI. pp. 3227–3236 (2015)

5. Perez, P., Banos, A., Pettit, C.: Agent-based modelling for urban planning current limitations and future trends. In: ABMUS. pp. 60–69. Springer (2016)

6. Taillandier, P., Gaudou, B., Grignard, A., Huynh, Q.N., Marilleau, N., Caillou, P., Philippon, D., Drogoul, A.: Building, composing and experimenting complex spatial models with the GAMA platform. GeoInformatica **23**(2) (2019)