# Development of Multimodal Physical and Virtual Traffic Reality Simulation System
## 4 Integration of Co-Simulation Components
### Design Challenges and Solutions of CARLA--Physiological Sensors Integration  
The integration of physiological sensors with CARLA is essential for studying human factors in transportation research, particularly for assessing driver, cyclist, and pedestrian responses to traffic interactions. One of the main challenges in this integration is ensuring precise synchronization between sensor data and simulation events. To address this, we employed the Lab Streaming Layer (LSL), an open-source framework designed for real-time data acquisition and synchronization.  
LSL provides a standardized protocol for streaming, recording, and processing physiological signals from multiple sensors simultaneously. It allows for the seamless integration of various biosensors while maintaining precise temporal alignment with external data sources, such as simulation events in CARLA. LSL ensures that all recorded signals share a common global timestamp, facilitating accurate analysis of physiological responses to specific traffic scenarios.  
To collect and manage sensor data, we used LabRecorder, an LSL-compatible software tool that records streams from multiple sensors into a synchronized dataset. LabRecorder enables real-time visualization of incoming physiological signals while ensuring that all data streams are properly timestamped and stored in a structured format. By integrating CARLA with LSL, simulation events, vehicle trajectories, and physiological responses can be recorded within a single synchronized dataset, allowing for a comprehensive analysis of human reactions to different driving conditions.  
Our setup was tested with multiple physiological sensors, demonstrating the flexibility of this integration. The Emotiv Epoc is a wireless EEG headset designed for measuring brain activity. It provides 14-channel EEG data and is commonly used in cognitive workload and neurophysiological studies related to driving and pedestrian behavior. By integrating Emotiv Epoc with CARLA through LSL, it is possible to capture real-time changes in brain activity during traffic interactions, enabling a deeper understanding of cognitive stress and attention levels.  
The Bitalino EDA sensor is used for measuring electrodermal activity (EDA), which reflects physiological arousal and emotional responses. This sensor provides valuable insights into how individuals react to different traffic scenarios, such as close vehicle interactions, sudden braking events, or high-speed passing maneuvers. The integration of Bitalino with CARLA through LSL allows for a continuous and time-synchronized recording of EDA signals, enabling researchers to assess emotional responses under controlled experimental conditions.  
The Polar H10 is a chest strap heart rate monitor capable of providing high-precision heart rate variability (HRV) data. Heart rate changes are widely used as indicators of stress, anxiety, and cognitive load in driving studies. The integration of the Polar H10 with CARLA enables the measurement of physiological stress responses during various driving and pedestrian scenarios. By recording heart rate data alongside simulation events, researchers can analyze how traffic conditions, vehicle proximity, and environmental factors influence physiological states.  
The combination of these sensors with LSL and CARLA creates a fully synchronized data collection framework that can accommodate a wide range of human factors research applications. This integration allows researchers to investigate physiological and cognitive responses in real-time, providing deeper insights into human behavior in complex traffic environments. The flexibility of the system also allows for additional sensors to be incorporated, enabling further expansion into areas such as eye-tracking, muscle activity, and respiration monitoring. Through this approach, the developed setup offers a comprehensive and adaptable platform for studying multimodal interactions in transportation research.