# Development of Multimodal Physical and Virtual Traffic Reality Simulation System
## 1 Introduction  
As traffic demand continues to rise, the need for multimodal transportation solutions and advancements in automated vehicle technologies has become increasingly evident. To support the development and implementation of these innovations, modeling tools play a crucial role in evaluating transportation policies, infrastructure designs, and human factors. Microscopic traffic simulation tools, such as PTV Vissim [1], SUMO [2], and Aimsun [3], are widely used to assess the network-wide effects of emerging transportation technologies. However, beyond traffic flow analysis, it is essential to consider how individual road users interact with vehicles and other modes of transportation. Traditional survey-based methods provide insight into public perception, but they fail to capture real-time behavioral responses to dynamic traffic scenarios [4]. Driving simulators addressthis gap by offering controlled, immersive environments where human subjects interact with virtual road conditions [5]. While driving simulators have been extensively used to study driver behavior and advanced driver assistance systems (ADAS), their application to bicyclist and pedestrian interactions remains limited [6]. Bicyclists and pedestrians, as vulnerable road users, experience unique interactions with motorized vehicles, making it critical to analyze their behavioral and physiological responses in dynamic traffic environments [7; 8; 9]. However, existing simulation studies often rely on pre-programmed agents to represent bicyclists and pedestrians, limiting their ability to capture real-time human reactions. Moreover, these simulators typically lack realistic background traffic models, as most rely on simplified vehicle movements rather than the calibrated car-following and lane-changing models found in microscopic traffic simulations. The absence of adaptive traffic control mechanisms further reduces their ability to replicate complex urban environments. To address these limitations, researchers have developed co-simulation platforms that integrate driving simulators with microscopic traffic simulation tools [10]. These platforms allow for real-time data exchange, where microscopic simulation provides dynamic, calibrated background traffic, while the driving simulator enables human interaction with live traffic conditions. By enhancing the realism of multimodal interactions, co-simulation facilitates more accurate studies on bicycle-vehicle and pedestrian-vehicle dynamics. An additional gap in transportation simulation research is the limited integration of physiological sensors to capture real-time human responses to traffic interactions. Many studies have emphasized the significance of physiological measures such as heart rate (HR), electrodermal activity (EDA), and brain activity (electroencephalography (EEG)) in assessing stress, cognitive workload, and emotional responses in traffic environments [11; 12; 13]. However, these data sources are often collected separately from simulation events, leading to challenges in synchronization and interpretation. To overcome this issue, we integrated multiple physiological sensors with the simulation system to provide a fully synchronized dataset, ensuring that biometric responses are aligned with real-time simulation events [14]. The ability to analyze physiological responses in conjunction with behavioral data offers deeper insights into the human factors involved in transportation. In this study, we present a multimodal and multi-participant simulation system, the Physical and Virtual Traffic Reality System, which integrates a driving simulator, a bicycle simulator, and a pedestrian simulator into a shared virtual environment. The system is designed to study real-time human interactions between different road users, overcoming the limitations of studies that rely on pre-programmed agents for bicycles and pedestrians.