第 13 回計測自動制御学会制御部門大会
回計測自動制御学会制御部門大会 (2013)

Development of a driving simulator for vehicle control andITS research
Yoann PENCREACH*1*1

Forum 8 Co., Ltd. (www.forum8.co.jp) VR Development Group

Gakuen Kibanadai-nishi 2-1-1, Miyazaki-ken Miyazaki City, 889-2155 Japan

Abstract: In this paper, we discuss how we addressed some of the synchronization issues in HILS
system and why solving synchronization can impact on the latency, then how with a simple
technique we could improve both latency and synchronization. The studied driving simulator uses areal time vehicle dynamics module connected to our virtual reality application, where the simulationof the environment is FPS dependent.

Key Words: Driving Simulator, ITS, Vehicle control

1

Introduction on testing ECUs of avehicle in a virtual environment

Since the 80s the number of ECU has
incredibly increased in our vehicles in order toimprove their efficiency, safety and comfort.

Virtual reality is a tool widely used forMany different types of ECU exit such as thesimulation where the interaction with an actual

Powertrain Control Module (PCM), Brake

person is necessary. It is used for instance inControl Module (BCM), Transmission Controlsystems for training on emergency situation,

Module (TCM), Body Control Module (BCM),

flight and drive simulators are also well known

Suspension Control Module (SCM), etc… Eachas efficient means to train or test pilots andof them controlling a specific part of thedrivers. But virtual reality is also used for thevehicle according to information acquired

reverse purpose: test an actual part of a systemthrough various types of sensors. The number

under real usage conditions. A good example isof ECU on a vehicle can exceed 50 nowadays,given by the ECU or Electronic Control Units;

and the heavy research effort in vehicle makes

as their complexity increases, the use ofcomplexity of the overall system increase

simulation where the actual ECU hardware is

rapidly. Programming an ECU has becomes a

integrated (HILS) becomes necessary.

great challenge for the engineers increasing theneed in simulation, validation and experiments.

第 13 回制御部門大会（2013 年 3 月 5 日～ 8 日・福岡）

SY0002/13/0000-0545 © 2013 SICEIn 2000 we (Forum 8 Co., Ltd.) released thedesigns. UC-win/Road included from its first

first version of UC-win/Road*1 a commercial

version an extensive set of tools to model invirtual reality software application. At that time

three dimensions the infrastructures of thewe already had a long expertise in development

roads. From that point the product wasofandextended to allow seamless simulation of theUC-win/Road started as a tool for consensus

traffic and later driving simulation in the 3D

and review of road and city planning andvirtual environment.

softwareforcivilengineers

Fig. 1: Example of driving simulator using UC-win/Road.
Since then we participated at many projects

used in ECU is information coming from other

involving technologies like HILS and biometryparts of the vehicle itself. But the information

instruments for the driver. As many equipments

can be also about the driver, the road condition,and modules have to operate together latency

surrounding vehicles or pedestrians. Moreandrecentlysynchronization

issuesbecomevery

important for the comfort of the user but also towiththeincreasing

interest

inIntelligent Transport Systems (ITS) the use ofallow the hardware to work correctly

Driverin the simulation.
Pedestrians

As shown in Figure 2 the ECU
outputs control signals directed toVehicle

ECU
Other vehicles

other parts of the vehicle while theinput can come from various sourcesof information. The most common
and the first king of information*1

Road infrastructure
Fig. 2: An ECU and its environment

UC-win/Road official web site: http://vr.forum8.co.jp/

real-time data transmitted from other vehicles,engine. Each of them should be highlypedestrians or infrastructure equipments hascustomizable to allow an easy integration of

become also very important.

other subsystems needed for the research.

In order to test and develop an ECU in aIf we consider the entire simulator system, thesimulation environment it is essential toECU is not the only part that requires a

reproduce all the input signals that the ECU

simulation for its input but the driver also needs

will receive in a real environment and also be

his stimuli in order to carry out the experiment.

able to execute many different scenarios that anThis aspect of the system will not be discussed

ECU may encounter in a real driving situation.

in detail in this document but is also a subject

For that the simulator must be able to simulateof great interest covering may fields like traffic

the data coming from the vehicle itself, fromsimulation, artificial intelligence and behavior

the driver and the traffic and other aspect of themodeling, visualization, sound generation, 3Denvironment. A standard set of tools would be amodel generation, force feedback and motion

software module for the vehicle physicalcueing etc…

simulation, a virtual reality system connected toA typical driving simulator system usinga driving simulator so a real person can drive

UC-win/Road and integrated with an ECU canthe simulated vehicle and a traffic simulation

be represented as shown in Figure 3.

ECU

UC-win/Road VR
Scenario generator

Vehicle dynamics simulation

Road surface information
Traffic, environment simulation

Simulator Cabin andProjection, sound system

Visual and sound generation

DriverFig. 3: UC-win/Road in a HILS environment.
This kind of configuration is also called aECU needs are sent from the VDS and theHardware In the Loop Simulation (HILS)ECU can also control back the vehicle

because a real ECU (hardware) is connected tosimulated in the VDS.

the simulator and plays its role in thesimulation. The main elements are as follow:(2) The Simulator cabin and the driver whoserole is to control the VDS.
(3) The traffic simulator has two roles: show a(1) ECU connected to the Vehicle Dynamic

realistic traffic condition to the driver andSimulation (VDS). Usually all the data thesend real time information to the ECU via

the VDS.

cannot ensure the availability of the(4) The road condition including mainly theroad

surfacegeometry

friction

2. UC-win/Road provides tools for the user tocoefficient used to feed the VDS with thebuild his driving environment. Therefore

parameters

vehiclethe application allows the user to create 3D

simulation. This can be done offline before

environment that may not be able to sustain

starting the simulation or in real time fora constant image refresh rate also calledchanging conditions. This information isframe per second (FPS).

neededinandresources at a given time for an application.

thealso used for visualization and is shared

Regarding the real time issue, we could

between the virtual reality engine and theimprove the system with techniques such asVDS.

using material interruption and virtualization to(5) Virtual reality (VR) engine: its role is toalmost force a constant simulation time step.

provide as much as possible a convicting

However, because the amount of data to beexperience to the driver in terms of visual

handled by the visualization is variable andand audio queue.

because we want to keep that flexibility in ourapplication the FPS is potentially variable.

2How to deal with a variable frame
rate in a HILS environment?

In any case the result of the VDS must be
somehow synchronized with the actual frame
displayed on the screen. We will explain later

2.1

Context

what would happen without synchronization. In
this paper we will study the case where theIn the test environment the ECU must receive

traffic simulation is calculated in UC-win/Road

the same data, at the same rate as in a real

at a variable time step and the driver’s vehicle

environment. The VDS module is responsible

simulation handled by the VDS at a constantfor sending the data of the simulated vehicle intime step.

real time to the ECU. In order to ensure a realtime simulation and data transfer at a constant

2.2

Description of our solution

rate, a real time operating system (RT OS) isused.

The traffic and environment simulation in

UC-win/Road is connected to the VDS and runs®

®

UC-win/Road are calculated to follow theon a standard Microsoft Windows operating

actual measured FPS as shown on Figure 4. Thesystem. In this environment a constant data

simulation runs in a separate thread from thetransfer cannot be ensured mainly for twodrawing process to improve the overallreasons:

performances of the system. However before

1. The operating system is not designed tostarting the drawing process of a frame bothprovide a strictly constant timer and also

threads are synchronized. The drawing threadwaits for the calculation of the current step of

the simulation is chosen equal to the actual time

the simulation to be completed. As the timeinterval measured between the two previous

interval between steps is not constant, theframes. This ensures the total time of theduration of the time interval to be simulated atsimulation is equal to the actual time of thea particular step can only be estimated. Incomputer.

UC-win/Road the time interval of one step ofFn

Fn+1

Fn+2

Fn+3

Frame number

tn

tn+1

tn+2

tn+3

Time

Calc(dtn)

Calc(dtn+2)

Drawing(tn+2)

Calculation thread
Synchronization

Drawing(tn+1)

Synchronization

Synchronization

Drawing(tn)Calc(dtn+1)

Drawing thread

dtn+1

dtn+2

dtn+3

Fig. 4: Simulation and drawing synchronization in UC-win/Road.
This creates a discrepancy as the actual time

As shown in Figure 5, if we consider a simple

interval between the current frame (Fn) and thesystem where the latest information coming

previous frame (Fn-1) may not be equal to thefrom the VDS is integrated in the displayedinterval between the two previous frames (Fn-1

frame without processing, we realize that theand Fn-2). Between two images separated inperceived time of the traffic and environment istime of [tn - tn-1] seconds the simulation

not the same as the one of the VDS. Although aprogressed of [tn-1 - tn-2] seconds. These twoconstant difference may create the impression

durations can be different because of thethe traffic does not respond in a realistic

varying FPS and this difference could induce areaction time to the actions of the user, thiswrong instant speed perception for the user. It

difference in time is also fluctuating as the FPS

would be interesting to study what levels ofdoes. The “wrong” reaction time of the traffic isvariation are acceptable for the user; however

actually very difficult for the user to perceivewe will consider for now that except for large

when the difference is small and can be

fluctuation of the FPS this issue can be ignored.

adjusted in the traffic algorithm according to
the average difference or delay between both

The next synchronization issue we studied issimulations. But the fluctuation creates a real

between the VDS and the VR application.

discomfort for the driver and prevents fromcarrying out a driving simulation when abecome unstable. Concretely, the user seesrealistic traffic situation is important. Because

other vehicles as if they were capable ofof this variation, the relative positions between

constantly doing sudden decelerations andthe user’s vehicle and other moving objects

accelerations.

Time difference between traffic and VDS

VDS

Latest VDS data

Actual drawing time

Actual drawing time

Projected drawing time

Latest VDS data

VR System

Projected drawing time

Fig. 5: Basic synchronization issue with HILSTo address that issue we had to improve theenvironment simulation, 2. drawing thread andestimation method for the time interval in the3. VDS. The first improved mechanism istraffic

represented in Figure 6.

simulation

module.

We

also

implemented a simple buffer used whensynchronizing the three processes: 1. traffic andVDS

Case A

Case B

Data buffering
Data actually usedfor drawing

VR System
Actual drawing time

Ready to draw

Wait
The VR system andProjected drawing time

Projected drawing time

11 x VDS loop time

14 x VDS loop time

HILS are synchronized
but an undesiredlatency is introduced

Introduced latency

Fig. 6: VDS and visual synchronization at a variable FPS.

However because we cannot control theCase A, some undesired latency is introduced in

combined time taken for the traffic simulation

the visual system as we do not use the mostand drawing processes, it may differ and berecent data coming from the VDS. As long as

longer or shorter than the projected interval. Yet

Case B does not occur the latency will not be

the calculation is finished and the data from thereduced to its minimum value.

VDS must be used to represent the user’s

Typically, after a drop in FPS an additional

vehicle in the virtual environment.

latency is artificially added for the needs of theIn Case A, if the data for the time T has been

synchronization and as long as the FPS is notsent already from the VDS to the VR system

restored to its original value, the latency stays

and is stored in the buffer, the buffer data isin the system.

used and older data is deleted from the buffer.

This issue can be simply improved by adding

Case A occurs when the FPS drops.

the introduced latency time in the next time

In Case B, the drawing thread waits for theinterval of the traffic calculation.

VDS to send the data corresponding to the timeIn Case B, it is the reverse case, there is noT. Case B occurs when the FPS increases.

impact on the latency but as we force the visual

We understand now why the time interval issystem to wait for future VDS data, the systemrounded to the closest multiple of the VDS loop

runs at lower FPS than its maximum capacity.

time. If not, the VDS may never be able to send

To resynchronize the systems, the wait time isthe data for the time T, as T is never simulated.

subtracted from the next time interval of thetraffic calculation.

This first improvement ensures that the data ofFigure 7 shows the final improved mechanism

the user’s vehicle and traffic simulation areafter a drop in FPS.

perfectly synchronized. However we see that inVDS
Data buffering
Data actually used for drawing

VR System
Actual drawing time

Actual drawing time

Projected drawing time

Projected drawing according
to last FPS + added latency

Introduced latency

Latency compensation

Fig. 7: Latency compensation in VDS and visual synchronization.

3Conclusions

As the complexity of our vehicle increases, the need of test and experiment increases. To carry out
such experiments safely but also as close as possible to a real situation, synchronization and latency
issues have to be taken in account in the design of virtual reality applications. In driving simulators
various synchronization issues exist, such as the synchronization of data collection from several
independent measurement equipment, audio-visual synchronization, motion cues and visual cues
etc… This time we saw that while allowing a maximum flexibility for the user about the content of
his experiment, it is possible to implement a robust connection between a real time system andvisualization at a variable frame rate.
As future development we plan to extend our system to enable low latency synchronization between
multiple driving simulators. We believe that it will be an efficient tool for studies on human factors
where various driving situations needs to be generated quickly and easily by the researchers andengineers.

References(1) Forum 8 Co.,Ltd. “UC-win/Road reference manual”
ftp://ftp.forum8.co.jp/forum8lib/ucwin/road/UCwinRoadV80000ManualJpn.exe(2) Michael Meehan (Stanford University), Sharif Razzaque, Frederick P. Brooks, Jr. Mary C.
Whitton (UNC-Chapel Hill) “Effect of Latency on Presence in Stressful Virtual Environments”(3) Rohit Goyal (Ohio State University) “Simulation Experiments with Guaranteed Frame Rate for
TCP/IP Traffic” (1997)(4) Ing. Petr Bouchner (Czech Technical University) “Driving Simulators for HMUI Research”(5) Chris Schwarz, Yefei He, Andrew Veit (National Advanced Driving Simulator, Iowa)[N11-001] “Eye Tracking in a Cots PC-Based Driving Simulator: Implementation andApplications” (Image 2011 Conference)(6) Yefei He (National Advanced Driving Simulator, Iowa) [N06-025] “NADS MiniSim Driving
Simulator” (Technical Report 2006)