# Performance Evaluation of a Real Vehicular Delay-Tolerant Network Testbed

Authors: Maicke C. G. Paula; Joel J. P. C. Rodrigues; João A. Dias; João N. Isento; Alexey Vinel
Journal: International Journal of Distributed Sensor Networks
 (2015)

---

## Abstract

Vehicular Delay-Tolerant Networks (VDTNs) are a breakthrough based DTN solution used to provide vehicular communications under challenging scenarios, characterized by long delays and sporadic connections. Using the store-carry-and-forward paradigm, this technology allows in-transit bundles to asynchronously reach the destination hop by hop over traveling vehicles equipped with short-range wireless devices. The VDTN architecture assumes out-of-band signaling with control and data planes separation and follows an IP over VDTN approach. This paper presents a real-world VDTN prototype evaluated through a safety application and a Traffic Jam Information Service. It also demonstrates the real deployment of this new vehicular communication approach. The real testbed is an important contribution since some complex issues presented in vehicular communication systems can be studied more accurately in real-world environments than in a laboratory approach. The results confirm that real deployment of VDTNs is doable and can be seen as a very promising technology for vehicular communications, although it requires appropriated technologies for outline interferences and quality of service support.


---

## Body

## 1. Introduction

Vehicular networking emerges as an important solution to promote road safety improvements [1], low-cost communication for developing regions [2], and telematics and infotainment applications, which enhance drivers’ experience [3]. These networks have been largely influenced by government investments in the last years. Nowadays, there is a strong commitment on economic progress, social development, and sustainability issues. It is clear that investments in vehicular communications present many advantages since they offer a large return in terms of mobility, decreasing road fatalities, and further improving the ecology and efficiency of traffic road. Thus, investing on Intelligent Transport Systems (ITS) design is a promising and challenging issue.Safety applications represent the driving force for enhancing vehicular communications. Unfortunately, thousands of death and injuries happen on the world’s roads due to distracted drivers (e.g., sending SMS messages, eating food, talking on cell phones, etc.), driver’s behavior (e.g., driving with excessive speed), road conditions, and roadway design or equipment failures. In order to solve the above-mentioned issues, low-cost emergency systems based on cooperative ITS [4] emerged as a technological solution to reduce accidents or anticipate the driver reaction.Currently, social development is one of the most discussed topics. Although technological advances are appearing in first world countries, the same behavior is not happening in developing countries. The penetration of technology in such countries is not uniform around the world and, unfortunately, there are still many countries economically backward with digital divide. Vehicular communications may be seen as an innovative and low-cost solution since, in such countries, they may be deployed without the need of a network infrastructure. In this case, vehicular networks act as the physical data carrier over the network, enabling communications.Commercial applications (e.g., [5, 6]) represent another potentially attractive area for vehicular networks. Not only are cars considered a kind of transport but passengers are also interested in comfort, security, and happiness while driving. Infotainment and telematics systems are being developed to provide entertainment, multimedia services, interactivity, advertisements, or Web access to users and represent a cost-effective solution to automakers. However, vehicular networks have considerable challenges for large-scale applications. These challenges arise from their unique characteristics, such as high vehicle mobility, dynamic scenarios, low number of vehicles, short and limited contact durations, scalability, disruption or intermittent connectivity, and strict requirements for latency. Traditional routing schemes proposed to other wireless networks, such as Mobile Ad Hoc (MANETs) 2 [7], are not suitable to be deployed on Vehicle Ad Hoc Networks (VANETs) [8] are also not suitable for them. The proposed routing protocols [9] do not present a satisfactory performance because sporadic connections with excessive packet dropping [10] are not assumed. For this reason, new solutions for data carrying are required. Recent studies [11, 12] have shown thatstore-carry-and-forward paradigm used on Vehicular Delay-Tolerant Networks (VDTNs) overcomes some of the above-mentioned problems. This new approach allows incoming packets to be assembled into large data packets, called bundles. VDTNs [13–15] also allow in-transit bundles (VDTN data units) to reach their final destination, even in the absence of an end-to-end path. To outline asynchronously opportunistic communications, bundles are stored and forwarded hop by hop till reaching their destination. A VDTN network considers the following three types of nodes: terminal, relay, and mobile nodes. Terminal nodes can be fixed or mobile devices (vehicles) that may be the origin or destination of data. Once having Internet access, they can act as access points (APs) or servers, providing useful information about road and weather conditions (e.g., traffic jams, entertainment, etc.). Relay nodes are stationary nodes placed at crossroads, endowed with store-and-forward capabilities. They are used to enhance the network connectivity increasing the contact opportunities and, consequently, increase the bundle delivery probability. Mobile nodes (e.g., bikes, cars, buses, trucks, etc.) are responsible for physically store-carry-and-forward bundles from the source to the destination nodes.This paper studies the performance of VDTNs through a real testbed, including its design and construction. This contribution extends the main features of a previous laboratory testbed [13, 16–18] applying them into a real deployment (an embedded VDTN testbed). Simulations [19] and real experiments [20] contribute for modeling and analysis of vehicular architectures. Simulations are important to evaluate more realistic results and study detailed behaviors on time scale. However, real experiments testify the accuracy of the theoretical results and can verify if the simulation captures all the necessary fundamental characteristics.Two safety applications (called Warning Message and Traffic Jam Information Services) are created to demonstrate the robustness of the VDTN architecture and attest, in a real scenario, all the main concepts of VDTNs. The real deployment of VDTNs presented in this paper includes the design and assembly of a specific embedded hardware with suitable software for managing vehicular communications in a real environment. A Traffic Jam Information service was created to demonstrate the feasibility of this technology. Basically, after obtaining traffic congestion information, cars should disseminate an alert message through the network (other cars or road side units) to allow drivers saving time and select alternative routes. The performance of the real prototype was analyzed in order to verify the efficiency of the proposed solution.The main contributions of this paper can be summarized as follows:(i)
presentation of VDTNs and related literature review about real vehicular networks testbeds;(ii)
design and construction of a real VDTN prototype;(iii)
proposal and demonstration of two new safety services and corresponding applications (called Warning Message and Traffic Jam Information);(iv)
demonstration of VDTN architecture, services, protocols, and applications in real scenarios.The remainder of the paper is organized as follows. Related works overviewing the most relevant real vehicular networks with all their services and applications that may contribute to the development of a real VDTN testbed are considered on Section 2. Section 3 presents the system requirements, design, and construction of the proposed real VDTN prototype while Section 4 analyses its network performance. Finally, Section 5 presents the conclusions and points some guidelines for future works.


## 2. Related Work

Real vehicular network prototypes have been developed to evaluate and demonstrate new architectures, protocols, services, and applications for vehicular communications. The design and construction of the proposed real VDTN testbed gathered several contributions from these proposals. Then, in order to highlight the most relevant applications and services, these approaches are described below.DRIVE [21] is a reconfigurable testbed developed to evaluate advanced services and communications. Based on the standard AUTOSAR, this is an open architecture allowing multiplatform deployment. Currently, DRIVE supports phone-like communication (dual WI-FI/3G communications), infotainment services, Web access, Google maps based GPS navigation, and multimodal HMI (graphical or voice). The communication is executed in-vehicle (e.g., distributed sensors) or externally through vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2I). Each vehicle includes a carPC hardware, wireless sensors, and rooftop antennas, which integrate WI-FI, UMTS, and Bluetooth technologies.A testbed to study the performance of the Vehicular Data Transfer Protocol (VDTP) in a real urban VANET is presented in [22]. A file transfer application developed in JAVA is executed between two cars, moving at different speeds (urban low speed and urban high-speed experiments) through open roads in Málaga downtown, Spain. Several data files of different types and lengths are exchanged in order to evaluate the quality of service (QoS) of different VDTP configurations. Each car is equipped with a notebook, an 802.11b/g device, a Wi-Fi transceiver, and a 7dBi omnidirectional antenna.An experimental platform to evaluate different applications and protocols was created for the testbed presented in [23]. In order to minimize the random aspects found in a VANET scenario, experiments are conducted in each car at the same time through multiple virtual machines (running its own protocol and application) to reproduce the same environment condition. The hardware platform is based on an Intel Core 2 Duo CPU, 2 GB of RAM, 120 GB hard drive, an Atheros 802.11 device, a GPS receiver, and an antenna with 8 dBi.Cabernet [24] is a real-world taxi testbed developed to evaluate data delivery from moving vehicles. The communication is performed through Wi-Fi access points that provide opportunistic communication. Cabernet testbed was deployed in Boston using 25 taxis equipped with a Soekris 4801 computer, an Atheros chipset 802.11b/g, a GPS receiver, and a small 3 dBi omnidirectional antenna. To reduce the connection setup time with APs, the QuickWi-Fi was created. QuickWi-Fi is an optimized and integrated collection of tools. This system has proven that, even with the presence of several experiencing problems like intermittent connectivity, it is possible to transfer megabytes of data at broadband speeds when APs are available.The SAFESPOT [25], GeoNet [26], and NoW [27] are projects designed to attend several needs of cooperative systems for safety road, using intelligent vehicles to detect critical situations in advance. With positioning information, traffic efficiency, and infotainment, among others, vehicles equipped with IEEE 802.11p radios execute a notification system to deliver safety messages, which can prevent road accidents, fatalities, and traffic jams.TATPA [28] is a testbed developed by the University of Bologna used to evaluate the architecture and advanced transport protocols. Using a powerful and flexible emulation tool, this testbed evaluates the performance of new TCP variants and new transport architecture (e.g., DTN) over heterogeneous networks including satellite links. It is composed of Linux PCs and networking tools like NistNet and Iperf. The system is accessed remotely through a Web user’s interface that allows easy management of the testbed features.The Shanghai urban vehicular network, SUVnet [29], is a real testbed developed to study the characteristics of a large-scale VANET. The real trace collected from the network topology of Shanghai allowed the construction of a mobility model to evaluate and design new routing protocols. Using the DTN store-and-forward concept, a new epidemic protocol called DAER (Distance Aware Epidemic Routing) was proposed to improve the data delivery ratio. SUVnet uses more than 4000 taxis equipped with Wi-Fi and GPS devices.A Smart Road testbed used to investigate IEEE 802.11 handoff in vehicular environment is described in [30]. This testbed was developed to ensure seamless connectivity in V2I communications. In order to improve the ability to quickly switch an association with a new AP, a prior knowledge about the channels of the neighbor access points is furnished for vehicles. For quantifying Wi-Fi network performance, this testbed uses the IxChariot software to simulate network traffic and the hrPING software to calculate the round trip delay. The system considers four fixed Wi-Fi nodes on the roadway containing omnidirectional antennas of 8 dBi and cars equipped with laptops running applications. The access points use a 2.4 GHz radio for connecting to wireless clients and 5 GHz radios for backhaul operation.A testbed for verification of vehicle safety communications applications is demonstrated in [31]. This testbed was created to validate a rapid-prototyping model, which is used to help catching bugs and errors in early stages of development. Cars exchange safety messages that are recorded, analyzed, and verified through a data acquiring system. Tools like CANape, MATLAB, and Simulink are used to access the vehicle bus and to manipulate data. The results obtained are used for further tuning and development of the application. Each car is equipped with CANape and hardware interface to vehicle bus (CAN of FlexRay).C-VET [32] is an interesting testbed platform developed at UCLA, USA. This testbed integrates concepts of VANETs with wireless mesh networks to provide vehicular communication and Web access. The proposal includes access points installed in strategic locations as traffic lights, which are responsible for furnishing services and applications. This testbed allows the evaluation and implementation of several mobility patterns and protocols. Each vehicle is equipped with Wi-Fi and Bluetooth interfaces which provide information like weather conditions, among others.In [33, 34] authors present the VanLAN testbed. This testbed aims to investigate the characteristics of WiFi-based connectivity in urban scenarios. It was also used to evaluate how the raw connectivity varies as the vehicle moves and whether it is stable across traversals of the same location. VanLAN is composed of 11 base stations and 2 vehicles.The possibility of exploring open Wi-Fi networks in urban scenarios is studied in [35]. In this study, authors try to evaluate the impact of collecting data from open access points in order to upload it to Internet servers. Performance studies were conducted in Boston metropolitan area, using 9 distinct cars during a period of 290 hours of driving.CarTorrent [36] is a real vehicular ad hoc network that allows peer-to-peer file to share applications. Two vehicles and an access point compose this testbed. Another testbed composed of two vehicles is presented in [37]. This testbed aims to study critical factors that affect the quality of a video transmission over a VANET in different scenarios.A vehicular testbed to evaluate telematics application is presented in [38]. Vehicles are equipped with an open embedded vehicular Android/OSGi platform that transfers digital data from smartphone nodes to central servers on the Internet. Among the telematics applications, stand-out remote management, proprietary vehicular applications, and easy management of APIs are considered.Cartel [39, 40] and Drive-thru [41, 42] are examples of real-world testbeds that were developed for supporting research and development of delay-tolerant networking techniques in vehicular communications. Cartel exploits opportunistic wireless networks and shields applications from the underlying details and network disruptions. The Fleet testbed [43] was deployed with 27 cars, each one running a Cartel embedded platform. This platform communicates with several sensors in the car, in order to collect data and deliver it to an Internet server.The Drive-thru Internet project [41, 42] studies how mobile users may have access to Internet in moving vehicles (trains, cars, etc.), using WLAN hot spots deployed along the roads, in rest areas, or at gas stations. This project allows applications to deal with intermittent connectivity and evaluated the communication characteristics when TCP or UDP standard protocols were used.All of these aforementioned projects have contributed to the design and construction of the proposed real VDTN testbed. The proposal and construction of the different node types also collect contributions from all the above-described projects. A real vehicular testbed deployment is an important research method for performance evaluation and demonstration, since it gives more reliable results in more realistic scenarios. However, a testbed suffers from limited scalability and flexibility due to the higher cost of construction.


## 3. Construction of a Real VDTN Testbed

This section presents the system requirements, specifications, design, and construction of the proposed real VDTN testbed. All efforts were made to faithfully represent the state of the art of the vehicular communications. The VDTN testbed considers two main platforms: a dedicated embedded hardware and software system. These platforms are responsible for executing the control and data planes, managing all the communication among nodes. Nodes interaction occurs when nodes are in the same Wi-Fi coverage area. In this situation, the out-of-band signaling process is initialized through a communication request for bundle transfer (exchanging control data) and ends through the data transfer, when a contact is valid.Different to the laboratory proposal [16] that used Wi-Fi only for the data plane, this real proposal uses Wi-Fi communication for both planes since real vehicular communication requires a wide area of coverage. However, it is still possible to conserve energy, once the data plane is executed only in critical situations or when vehicle is turned on with a battery recharge process.This section considers two subsections as follows. The first subsection presents the embedded hardware platform created for mobile nodes with all the components and used network entities. The second subsection presents the structure of the software platform responsible for managing the hardware components and controlling the vehicular communication.


### 3.1. Embedded Hardware Platform

All the components involved on the hardware platform were chosen in order to represent a real deployment. Embedded computers, laptops, Wi-Fi and Global Positioning System (GPS) interfaces, external antennas, and cars were used. Cars that act as mobile nodes were equipped with a PFM-945C single board [44] with an 8 GB CompactFlash disk. This is a PC/104 CPU module built upon the onboard Intel AtomTM N270 1.6 GHz Processor and Intel 945GSE + ICH7M chipset with up to 1 GB of onboard DDRII 400/533 system memory. The PFM-945C offers one 10/100 Base-TX Ethernet RJ-45 port as well as 4 USB 2.0, 4 COM, and PCI/104-Express interfaces. The PFM-945C was designed to operate in special situations requiring low power consumption and antivibration strategies. The storage needs are met by both SATA and CompactFlash interfaces. The integration with the vehicle solution was made powering the PFM-945C by an M2-ATX-HV Intelligent Automotive supply (140 W) 12 V DC-DC ATX PC [45]. This power supply is very useful since it was designed to provide power and to control the ON/OFF switch of the PFM-945C. The M2-ATX is a 6–24 V input ATX PSU capable of surviving tough engine cranks (down to 6 V) as well as transient overvoltage situations.For executing the most possible applications and services, the real testbed requires a GPS module that furnishes the vehicle positioning in real time. In the proposed testbed, a Ublox EVK-5H evaluation kit [46] plugged into a USB port of the PFM-945C was used. This GPS device was chosen because it does not require an external power supply; it is easy to use and allows data transfer (NMEA string) over USB. The antenna is mounted on the vehicle panel with direct vision for the windshield.The real VDTN testbed also uses an Encore’s ENUWI-XAN3 wireless N-150 adapter [47] compatible with IEEE 802.11b/g/n and operation frequency of 2.4 Ghz. This device is also plugged into a USB port of the PFM-945C. The 802.11 configuration parameters to use ad hoc mode and SSID were set. One of the advantages of this wireless network adapter comes from the fact that antenna is detachable and can be replaced for other powerful antennas. The real testbed experiments were conducted with an omnidirectional antenna mounted on the vehicle rooftop. This antenna is available in frequencies ranging from 2400 to 2500 MHz and has a gain of 9 dBi. For displaying vehicle communication, a Lilliput FA801-NP/C/T 8′′ touch screen VGA LCD monitor [48] was fitted inside the vehicle. It represents the user interface used to interact with the VDTN system. Once the VDTN applications are directed to safety information, the monitor operates with an interactive human machine interface able to provide accurate, dynamic, and real-time information. The touchscreen functionality allows the driver to set the parameters easily.Figure1 shows the final version of the in-vehicle VDTN testbed implementation. It presented the final integration of the testbed on vehicles. The main hardware (PFM-945C single board computer and the M2-ATX-HV Intelligent Automotive supply) is installed in a PC-104 rack cabinet. All the other communication interfaces (e.g., Wi-Fi devices or the monitor) are connected to the ECU through USB ports. This definitive implementation acts as a central processing unit and is responsible for managing all the vehicular communication. For convenience, easy maintenance, and technical requirements (e.g., humidity, temperature below 60°C, etc.), the rack was installed on vehicle trunk. The Wi-Fi omnidirectional antenna has a magnetic stand; however, it was necessary to improve the fixation with adhesive tapes, due to high-speed mobility of vehicles. The cable length (approximately 4 meters) was enough to connect the antenna with the wireless device installed via USB on the ECU.Figure 1
Photos of in-vehicle implementations of VDTNs.The real construction of terminal nodes and relay nodes may be seen in Figure2. The terminal node was installed in an internal building with Internet access. It served as an AP and is responsible for providing updated information about safety messages, which were used in the services proposed in the paper. An external antenna (with the same characteristics of the antenna used on vehicles) was installed in a window outside the building. The relay node was placed on a road intersection in order to download, store, and upload data from/to vehicles and increase the number of contact opportunities or delivery ratios. Even though it is recommended for sparse networks with low density, this testbed has demonstrated the relay node efficiency, since it provides more contact opportunities. Laptops with Wi-Fi interfaces and storage capabilities are used to emulate this type of VDTN nodes. These fixed network entities are always active and available to V2I communications.Figure 2
Photos with the (a) indoor, (b) outdoor implementation of the terminal node, and (c) relay node.
(a)
(b)
(c)


### 3.2. Software Platform Structure

To allow the system development, integration, and management of diverse hardware components, a software platform was developed. Various software modules were created in C++/C# under Microsoft .Net Framework 3.5 and deployed in the network nodes. The hardware platform runs on Windows 7 operation system. Microsoft Visual Studio 2010 IDE was used as development environment. In addition to the implementation of the VDTN components, the software modules are interactive allowing the emulation of network resource constraints (e.g., storage capacity, battery consumption, bandwidth, Wi-Fi signal range, etc.). These characteristics allow the emulation of network applications in several situations. Different routing protocols approaches (First Contact [49], Epidemic [50], and Spray and Wait [51]) and scheduling and dropping policies (e.g., FIFO, Remaining Lifetime, and Random [18]) are also supported. At the application level, in order to help the communication management, Human Machine Interaction (HMI) is provided by the software platform.


## 3.1. Embedded Hardware Platform

All the components involved on the hardware platform were chosen in order to represent a real deployment. Embedded computers, laptops, Wi-Fi and Global Positioning System (GPS) interfaces, external antennas, and cars were used. Cars that act as mobile nodes were equipped with a PFM-945C single board [44] with an 8 GB CompactFlash disk. This is a PC/104 CPU module built upon the onboard Intel AtomTM N270 1.6 GHz Processor and Intel 945GSE + ICH7M chipset with up to 1 GB of onboard DDRII 400/533 system memory. The PFM-945C offers one 10/100 Base-TX Ethernet RJ-45 port as well as 4 USB 2.0, 4 COM, and PCI/104-Express interfaces. The PFM-945C was designed to operate in special situations requiring low power consumption and antivibration strategies. The storage needs are met by both SATA and CompactFlash interfaces. The integration with the vehicle solution was made powering the PFM-945C by an M2-ATX-HV Intelligent Automotive supply (140 W) 12 V DC-DC ATX PC [45]. This power supply is very useful since it was designed to provide power and to control the ON/OFF switch of the PFM-945C. The M2-ATX is a 6–24 V input ATX PSU capable of surviving tough engine cranks (down to 6 V) as well as transient overvoltage situations.For executing the most possible applications and services, the real testbed requires a GPS module that furnishes the vehicle positioning in real time. In the proposed testbed, a Ublox EVK-5H evaluation kit [46] plugged into a USB port of the PFM-945C was used. This GPS device was chosen because it does not require an external power supply; it is easy to use and allows data transfer (NMEA string) over USB. The antenna is mounted on the vehicle panel with direct vision for the windshield.The real VDTN testbed also uses an Encore’s ENUWI-XAN3 wireless N-150 adapter [47] compatible with IEEE 802.11b/g/n and operation frequency of 2.4 Ghz. This device is also plugged into a USB port of the PFM-945C. The 802.11 configuration parameters to use ad hoc mode and SSID were set. One of the advantages of this wireless network adapter comes from the fact that antenna is detachable and can be replaced for other powerful antennas. The real testbed experiments were conducted with an omnidirectional antenna mounted on the vehicle rooftop. This antenna is available in frequencies ranging from 2400 to 2500 MHz and has a gain of 9 dBi. For displaying vehicle communication, a Lilliput FA801-NP/C/T 8′′ touch screen VGA LCD monitor [48] was fitted inside the vehicle. It represents the user interface used to interact with the VDTN system. Once the VDTN applications are directed to safety information, the monitor operates with an interactive human machine interface able to provide accurate, dynamic, and real-time information. The touchscreen functionality allows the driver to set the parameters easily.Figure1 shows the final version of the in-vehicle VDTN testbed implementation. It presented the final integration of the testbed on vehicles. The main hardware (PFM-945C single board computer and the M2-ATX-HV Intelligent Automotive supply) is installed in a PC-104 rack cabinet. All the other communication interfaces (e.g., Wi-Fi devices or the monitor) are connected to the ECU through USB ports. This definitive implementation acts as a central processing unit and is responsible for managing all the vehicular communication. For convenience, easy maintenance, and technical requirements (e.g., humidity, temperature below 60°C, etc.), the rack was installed on vehicle trunk. The Wi-Fi omnidirectional antenna has a magnetic stand; however, it was necessary to improve the fixation with adhesive tapes, due to high-speed mobility of vehicles. The cable length (approximately 4 meters) was enough to connect the antenna with the wireless device installed via USB on the ECU.Figure 1
Photos of in-vehicle implementations of VDTNs.The real construction of terminal nodes and relay nodes may be seen in Figure2. The terminal node was installed in an internal building with Internet access. It served as an AP and is responsible for providing updated information about safety messages, which were used in the services proposed in the paper. An external antenna (with the same characteristics of the antenna used on vehicles) was installed in a window outside the building. The relay node was placed on a road intersection in order to download, store, and upload data from/to vehicles and increase the number of contact opportunities or delivery ratios. Even though it is recommended for sparse networks with low density, this testbed has demonstrated the relay node efficiency, since it provides more contact opportunities. Laptops with Wi-Fi interfaces and storage capabilities are used to emulate this type of VDTN nodes. These fixed network entities are always active and available to V2I communications.Figure 2
Photos with the (a) indoor, (b) outdoor implementation of the terminal node, and (c) relay node.
(a)
(b)
(c)


## 3.2. Software Platform Structure

To allow the system development, integration, and management of diverse hardware components, a software platform was developed. Various software modules were created in C++/C# under Microsoft .Net Framework 3.5 and deployed in the network nodes. The hardware platform runs on Windows 7 operation system. Microsoft Visual Studio 2010 IDE was used as development environment. In addition to the implementation of the VDTN components, the software modules are interactive allowing the emulation of network resource constraints (e.g., storage capacity, battery consumption, bandwidth, Wi-Fi signal range, etc.). These characteristics allow the emulation of network applications in several situations. Different routing protocols approaches (First Contact [49], Epidemic [50], and Spray and Wait [51]) and scheduling and dropping policies (e.g., FIFO, Remaining Lifetime, and Random [18]) are also supported. At the application level, in order to help the communication management, Human Machine Interaction (HMI) is provided by the software platform.


## 4. Performance Evaluation and Results

A real-world performance analysis was realized in order to prove the feasibility of the proposed services in a real VDTN environment. The objective includes the evaluation of the transmission capacity between vehicles using the standard IEEE 802.11b/g. Based on this, a file transfer application was executed where the amount of data of different sizes that can be transferred between VDTN network nodes was measured. Across the conducted experiments, mobile node moves at different speeds and theIperf version 2.0.2 was used as a traffic source. This tool is open source software commonly used for measuring the throughput of a network. It is composed of client and server components, each one presented in the two edges of the network, creating TCP and UDP data streams to perform throughput measurements between them. Although the VDTNs do not perform the transport layer, theIperf tool forces to set TCP or UDP as protocol. The UDP was chosen taking into account its better performance [46] for vehicle applications that present loss links. The average and the total number of data received are presented graphically. The results also served to analyze the behavior of the standard IEEE 802.11b/g.The remainder of this section is organized as follows. The first subsection describes the two services created to demonstrate the real VDTN testbed. The scenarios used for the performance evaluation considering the network transmission capacity are presented in the second subsection while the third presents the performance analysis and the obtained results.


### 4.1. VDTN Testbed Demonstration

The experiments were carried out at the internal streets of Brazilian Fiat Automobile manufacturing plant, MG, Brazil. In order to evaluate if VDTNs support the communication under a real application, an urban scenario was defined considering two safety services: Warning Message Service and Traffic Jam Information Service.A Warning Message Service was used to demonstrate a safety alert system as well as the behavior of different network nodes through a real VDTN testbed. In this scenario, to prevent accidents, damaged vehicles send a safety notification alert, with their GPS position, to other vehicles, which make the diffusion. The experiment scenario for this service can be described as follows. Three vehicles and a relay node located at road intersection are considered. A vehicle was responsible for receiving the alert and forwarding it to a relay node while another vehicle was responsible for forwarding the message for the third car.A Traffic Jam Information Service was proposed to evaluate another safety service based on alerts about traffic jam situations. Three cars, a relay node, and a terminal node were used. A vehicle was responsible for receiving the traffic alert from a vehicle in a traffic jam situation and forwarding it to a relay node while another vehicle was responsible for forwarding the message until it reaches the terminal node. Figure3 shows the Graphical User Interface (GUI) of the two safety services available in this real testbed. Figure 3(a) shows the application created for Warning Message Service with information about the vehicle position. In background, the application tries to establish connection with other vehicles. In Figure 3(b), the application used for Traffic Jam Service is shown. It is composed of a map, similar to a GPS device, that indicates where traffic jams occur. The traffic jam points appear on the map when vehicles receive this information from other cars (network nodes) allowing users to choose other alternative routes.Figure 3
Graphical User Interface illustrating the (a) Warning Message and (b) Traffic Jam Information Service.
(a)
(b)


### 4.2. Experimental Scenarios

Two different scenarios were considered to evaluate the performance of the wireless communication, through the standard IEEE 802.11b/g. In the first scenario, the amount of data exchanged between a mobile node and a terminal node was measured while, in the second, the data transfer between two mobile nodes was evaluated. Experiments were performed inside internal streets of Fiat-Brazil park, surrounded by buildings, under an urban scenario with a significant presence of traffic and interferences of other IEEE 802.11 networks. Figure4 illustrates the considered scenario for all experiments.Figure 4
Real scenario illustration representing the position of terminal and relay nodes.The first scenario was realized on an internal 250 m long street of Fiat-Brazil with the terminal node and the mobile node located in each far-end extremity. Although the experiments have been conducted in an urban scenario, the traffic of vehicles was moderate. In the early experiments, the terminal node and the vehicle are out of range. The vehicle speed was changed so that it could cross the terminal node at exactly 20, 40, and 60 Km/h (using the cruise control). The vehicle starts the route executingIperfas client. Once both the vehicle and the terminal node are in range,Iperf data are transferred between them. The terminal node executesIperfas a server and counts the data received every second. 30 experiments (at different speeds) were performed for each bundle size with 1 Kbyte, 20 Kbytes, and 60 Kbytes.The second scenario was performed over an internal 500 m long street of Fiat-Brazil with severe situations. The experiments were performed during a peak hour of traffic jam containing trucks, buses, and tractors. The weather was wet and it had rained a little. In the early experiments, vehicles were located in each extremity and out of range coverage. Vehicles intersect each other at exactly 20, 40, and 60 Km/h. One of the vehicles serves as anIperf client, whereas the other is the server, which is responsible for counting the data received every second. 30 trials (at different speeds) were also performed for each message size with 1 Kbyte, 20 Kbytes, and 60 Kbytes. The vehicles’ speeds were obtained using cruise control systems.


### 4.3. Performance Analysis

This section presents the performance evaluation regarding the links communication capacity of the two above-mentioned application scenarios. The quality of the network is measured by the amount of data that can be transferred within the opportunistic contact that is formed between the network nodes.Figures5, 6, and 7 present the average data received (goodput) in both scenarios, respectively. In the first scenario, the terminal node runs theIperfserver while the second scenario assumes this function is performed by a mobile node. As noted in Figure 5, in the first scenario with vehicles running at 20 Km/h, a greater volume of data is transferred for larger packets and the average contact time is about 24 seconds. However, the results in the second scenario are different and even worse than the results found in the first scenario. Although some inconsistent peaks of goodput were founded, the peak goodput is about approximately 0.25 MB/s, obtained with 60 Kbytes packets when vehicles are side by side with the terminal or the mobile node.Figure 5
Average UDP data transferred at 20 Km/h for the (a) mobile-to-terminal scenario and for the (b) mobile-to-mobile scenario.
(a)
(b)Figure 6
Average UDP data transferred at 40 Km/h for the (a) mobile-to-terminal scenario and for the (b) mobile-to-mobile scenario.
(a)
(b)Figure 7
Average UDP data transferred at 60 Km/h for the (a) mobile-to-terminal scenario and for the (b) mobile-to-mobile scenario.
(a)
(b)When the speed of the vehicles is increased to 40 Km/h, a different behavior for both scenarios is observed (Figure6). In the first one (mobile-to-terminal), using the packet size of 60 Kbytes the system still allows transferring more data during the crossing, but the contact time is about 20 seconds and the peak goodput is approximately 0.16 MB/s. Based on these results, for this first scenario, it is concluded that a commitment between speed and packet size exists, especially for speeds above 40 Km/h and larger packet sizes. However, in the mobile-to-mobile node scenario, when the vehicles speed is increased to 40 Km/h, using the packet size of more than 40 Kbytes, the system cannot transfer more data during the cars crossing process, because the contact duration decreases to about 11 seconds. Based on such fact, it is concluded that, with two vehicles running at different speeds, the relationship between speed and packet size is more severe. In other words, at higher speed, shorter contacts duration exists and, consequently, the network transfer capacity decreases. In this scenario, at 40 Km/h, the number of data transferred reduces to approximately 28% for packets of 60 Kbytes.The same experiments with a vehicle moving at 60 Km/h were also performed, as may also be seen in Figure7. The presented results confirm the above-mentioned observations, although it appears that higher speeds directly influence the amount of data transferred. In this specific case, at 60 Km/h, the number of data transferred reduces for mobile-to-terminal node scenario to approximately 12% for packets of 60 Kbytes. The contact duration was reduced and the network capacity also decreases. The results found in mobile-to-mobile scenario can occur due to several factors [47–49] that influence the radio signal attenuation. Among these factors physical obstructions caused for other large transportation vehicles (e.g., trucks, tractors, buses, etc.) stand out which often obstruct the signal Line-of-Sight (LoS) and the high velocity of the vehicles resulting in a hidden station scenario. The number of other IEEE 802.11 units operating in the same bandwidth as the testbed radio can also cause a huge volume of noise.


## 4.1. VDTN Testbed Demonstration

The experiments were carried out at the internal streets of Brazilian Fiat Automobile manufacturing plant, MG, Brazil. In order to evaluate if VDTNs support the communication under a real application, an urban scenario was defined considering two safety services: Warning Message Service and Traffic Jam Information Service.A Warning Message Service was used to demonstrate a safety alert system as well as the behavior of different network nodes through a real VDTN testbed. In this scenario, to prevent accidents, damaged vehicles send a safety notification alert, with their GPS position, to other vehicles, which make the diffusion. The experiment scenario for this service can be described as follows. Three vehicles and a relay node located at road intersection are considered. A vehicle was responsible for receiving the alert and forwarding it to a relay node while another vehicle was responsible for forwarding the message for the third car.A Traffic Jam Information Service was proposed to evaluate another safety service based on alerts about traffic jam situations. Three cars, a relay node, and a terminal node were used. A vehicle was responsible for receiving the traffic alert from a vehicle in a traffic jam situation and forwarding it to a relay node while another vehicle was responsible for forwarding the message until it reaches the terminal node. Figure3 shows the Graphical User Interface (GUI) of the two safety services available in this real testbed. Figure 3(a) shows the application created for Warning Message Service with information about the vehicle position. In background, the application tries to establish connection with other vehicles. In Figure 3(b), the application used for Traffic Jam Service is shown. It is composed of a map, similar to a GPS device, that indicates where traffic jams occur. The traffic jam points appear on the map when vehicles receive this information from other cars (network nodes) allowing users to choose other alternative routes.Figure 3
Graphical User Interface illustrating the (a) Warning Message and (b) Traffic Jam Information Service.
(a)
(b)


## 4.2. Experimental Scenarios

Two different scenarios were considered to evaluate the performance of the wireless communication, through the standard IEEE 802.11b/g. In the first scenario, the amount of data exchanged between a mobile node and a terminal node was measured while, in the second, the data transfer between two mobile nodes was evaluated. Experiments were performed inside internal streets of Fiat-Brazil park, surrounded by buildings, under an urban scenario with a significant presence of traffic and interferences of other IEEE 802.11 networks. Figure4 illustrates the considered scenario for all experiments.Figure 4
Real scenario illustration representing the position of terminal and relay nodes.The first scenario was realized on an internal 250 m long street of Fiat-Brazil with the terminal node and the mobile node located in each far-end extremity. Although the experiments have been conducted in an urban scenario, the traffic of vehicles was moderate. In the early experiments, the terminal node and the vehicle are out of range. The vehicle speed was changed so that it could cross the terminal node at exactly 20, 40, and 60 Km/h (using the cruise control). The vehicle starts the route executingIperfas client. Once both the vehicle and the terminal node are in range,Iperf data are transferred between them. The terminal node executesIperfas a server and counts the data received every second. 30 experiments (at different speeds) were performed for each bundle size with 1 Kbyte, 20 Kbytes, and 60 Kbytes.The second scenario was performed over an internal 500 m long street of Fiat-Brazil with severe situations. The experiments were performed during a peak hour of traffic jam containing trucks, buses, and tractors. The weather was wet and it had rained a little. In the early experiments, vehicles were located in each extremity and out of range coverage. Vehicles intersect each other at exactly 20, 40, and 60 Km/h. One of the vehicles serves as anIperf client, whereas the other is the server, which is responsible for counting the data received every second. 30 trials (at different speeds) were also performed for each message size with 1 Kbyte, 20 Kbytes, and 60 Kbytes. The vehicles’ speeds were obtained using cruise control systems.


## 4.3. Performance Analysis

This section presents the performance evaluation regarding the links communication capacity of the two above-mentioned application scenarios. The quality of the network is measured by the amount of data that can be transferred within the opportunistic contact that is formed between the network nodes.Figures5, 6, and 7 present the average data received (goodput) in both scenarios, respectively. In the first scenario, the terminal node runs theIperfserver while the second scenario assumes this function is performed by a mobile node. As noted in Figure 5, in the first scenario with vehicles running at 20 Km/h, a greater volume of data is transferred for larger packets and the average contact time is about 24 seconds. However, the results in the second scenario are different and even worse than the results found in the first scenario. Although some inconsistent peaks of goodput were founded, the peak goodput is about approximately 0.25 MB/s, obtained with 60 Kbytes packets when vehicles are side by side with the terminal or the mobile node.Figure 5
Average UDP data transferred at 20 Km/h for the (a) mobile-to-terminal scenario and for the (b) mobile-to-mobile scenario.
(a)
(b)Figure 6
Average UDP data transferred at 40 Km/h for the (a) mobile-to-terminal scenario and for the (b) mobile-to-mobile scenario.
(a)
(b)Figure 7
Average UDP data transferred at 60 Km/h for the (a) mobile-to-terminal scenario and for the (b) mobile-to-mobile scenario.
(a)
(b)When the speed of the vehicles is increased to 40 Km/h, a different behavior for both scenarios is observed (Figure6). In the first one (mobile-to-terminal), using the packet size of 60 Kbytes the system still allows transferring more data during the crossing, but the contact time is about 20 seconds and the peak goodput is approximately 0.16 MB/s. Based on these results, for this first scenario, it is concluded that a commitment between speed and packet size exists, especially for speeds above 40 Km/h and larger packet sizes. However, in the mobile-to-mobile node scenario, when the vehicles speed is increased to 40 Km/h, using the packet size of more than 40 Kbytes, the system cannot transfer more data during the cars crossing process, because the contact duration decreases to about 11 seconds. Based on such fact, it is concluded that, with two vehicles running at different speeds, the relationship between speed and packet size is more severe. In other words, at higher speed, shorter contacts duration exists and, consequently, the network transfer capacity decreases. In this scenario, at 40 Km/h, the number of data transferred reduces to approximately 28% for packets of 60 Kbytes.The same experiments with a vehicle moving at 60 Km/h were also performed, as may also be seen in Figure7. The presented results confirm the above-mentioned observations, although it appears that higher speeds directly influence the amount of data transferred. In this specific case, at 60 Km/h, the number of data transferred reduces for mobile-to-terminal node scenario to approximately 12% for packets of 60 Kbytes. The contact duration was reduced and the network capacity also decreases. The results found in mobile-to-mobile scenario can occur due to several factors [47–49] that influence the radio signal attenuation. Among these factors physical obstructions caused for other large transportation vehicles (e.g., trucks, tractors, buses, etc.) stand out which often obstruct the signal Line-of-Sight (LoS) and the high velocity of the vehicles resulting in a hidden station scenario. The number of other IEEE 802.11 units operating in the same bandwidth as the testbed radio can also cause a huge volume of noise.


## 5. Conclusion and Future Work

This paper proposed the design and construction of a real-world testbed for Vehicular Delay-Tolerant Networks (VDTNs) created to demonstrate the real deployment of this architecture and evaluate the network performance with a safety application and a traffic jam system. The paper tried to demonstrate and validate several VDTN deployments, in a real scenario, and study the behavior of VDTN network nodes and communication modules used in a preliminary laboratory testbed [16]. A real prototype is very important because some complex issues presented in vehicular communication systems cannot be treated in a laboratory environment or through simulation.The results demonstrated that VDTN proposal is really feasible and extremely promising. All the main concepts of the VDTN architecture like IP over VDTN approach and out-of-band signaling with the separation between control and data planes could be verified and validated in practice. It confirms that it is possible to outline critical connectivity issues in a real environment.Designing a real system that involves vehicles is not an easy task because there are many relevant factors that interfere with it. Equipment deployment in vehicles requires high investment and especially time to prepare the network setup and experiments validation. Equipment integration requires careful analysis, since it does not wish to exert a harmful influence (e.g., EMC) on each other’s equipment [52]. Nevertheless, the integration of the VDTN architecture with the design of hardware and software platforms that best apply to the embedded scenario was a success, although it has brought challenges and required large amount of time in the setup phase. Different to the preliminary laboratorial testbed that used Bluetooth for the control plane, to allow the fact that communication could be made over a wide area, the real deployment required the use of the Wi-Fi communication for both planes. The energy conservation was maintained, since the data plane is active only when the vehicle is turned on (with a battery recharge process) or in critical situations.The results obtained in Section5 deserve attention. The experimental measurements performed on a real urban scenario demonstrated through a file transfer application that real VDTN testbed offers a good solution for vehicular communications. Packets with 60 Kbytes can be transferred at relatively high speeds although the safety services required at most packets about 2 Kbytes. However, it is important to highlight the encountered communication problems. Obstructions, fading, and interference of other IEEE 802.11 networks in operation interfered on QoS parameters. While obstructions by other vehicles or buildings can influence the radio signal attenuation, the existence of several other Wi-Fi networks forces a competition for the communication medium, which reduces the overall network performance. Maybe the use of the directional antennas [53] can reduce the interference. Some studies suggest that a better placement of the antenna [54] can improve the signal.Further research directions based on current work are suggested. The proposal of more safety services (e.g., other hazardous situations, emergency vehicle warning or diagnosis warnings, etc.) and the use of more cars on the network allowing other statistics to evaluate quality of service (QoS) may be considered. The performance evaluation and analysis of the standard IEEE 802.11p for short vehicular communications should also be considered for future research work since it was observed that IEEE 802.11b/g may suffer from several interferences and is not very appropriated for nodes with high mobility. The performance evaluation of the real VDTN behavior with different routing protocols (e.g., Epidemic [49], Spray and Wait Source [50], Spray and Wait Binary, Prophet [55], and MaxProp [56]) and different scheduling and dropping policies (FIFO, Remaining Lifetime, and Random [18]) created for the laboratory testbed should also be considered. The performance evaluation of the system behavior using different antennas with different gains and a greater integration of VDTN with telematics and infotainment applications (running on clusters, navigators, etc.) is also important to give a more real aspect to the testbed.



---

*Source: 219641-2015-03-22.xml*