Multivariate analysis of the cross-layer interaction in wireless network simulations

While there exist many papers that compare the performances of different routing protocols for wireless ad-hoc networks, these analysis are often realized using tools from descriptive statistics (curves drawing, means and variances computation, etc.). In this paper we propose the use of multivariate statistics to unveil and characterize the interaction between the input variables of a wireless network simulation. The ANalysis Of VAriance (ANOVA) tool helps us in getting the impact of the four variables that we have studied: routing algorithm, propagation conditions, nodes density, and mobility scheme. Using our methodology we are able to show that not only a single layer of the protocol stack can affect the network operation but also the interactions between two or more inputs of the simution. An important implication of the study is that the efficiency of the routing algorithm is strongly correlated with the environment (indoor/outdoor) and that the performance analysis of the lower levels of the OSI stack should be conducted by focusing on a single layer. In the same way, wireless networks simulators cannot be considered accurate if they neglect a realistic-enough implementation of the physical layer (i.e., propagation, interference and modulation) I. MOTIVATION AND CONTRIBUTION W IRELESS ad-hoc network simulation is among one of the most hot topics of these ten last years. Since it is difficult to conduct experiments with real equipment and a large number of nodes, computer-based simulations have been used from the beginning in order to investigate and to validate new routing schemes as to conduct performance analysis on future wireless network[14] [15]. However, in [12], the authors report that the use of too simplistic models for the particular aspects of wireless networks (physical layer model, interference, modulation schemes, etc.) are more and more lowering the credibility of the studies based on simulations. In a companion paper [1] we have shown that the choice of physical layer instance (e.g., shadowing, free space, or ray-tracing) has a significant influence on the performance of the network. Moreover, it can lead to radically different conclusions on the protocol efficiency if the conlusions are subsequent to the analysis of some metrics like routing overhead or packet delivery fraction. We conducted a simulation-based experimental analysis to characterize the cross-layer interaction. Our research has been motivated by earlier work by Barrett et al. [10] who has set up a statistical analysis of the intraction between routing and MAC, Takai et al. [7] and, results of Royer and al. that emphasized in [8] ". the critical need for studying interaction between protocol layers. ". In this paper, we introduce the use of multivariate statistical analysis in order to reveal the existence of complex interactions between different factors of indoor wireless networks. To our best knowledge, only one paper [10] has used this statistical approach. However, this paper was more focused on the interaction between RTR, MAC and injection rate in a more general context. We present here a study on the interaction between the routing algorithm, the deployment scenario, the nodes density, and the propagation model in an indoor environment. II. EXPERIMENTAL SETUP The statistical analysis using ANOVA requires a set of input variables with a discrete range of values and allows to conduct analysis on ouput variables with continuous values. Apart from the physical layer model, we use as input variables the routing algorithm, the number of nodes and the deployment scenario. Our evaluation criteria consists of the following widely-used metrics [9]: 1) the routing overhead, defined as the ratio of the number of packets spent for routing purposes on the total number of packets sent in the entire network 2) the packet delivery fraction which is the ratio of packets successfully delivered to the destination 3) the end-to-end delay which is the mean time spent for the delivery of the packets that were successfully received. In our study, we used the NS-2, a discrete event simulator used by the networking research community [13]. It implements several routing algorithms and some of the most widelyused MAC algorihms and propagation models. As a previous study has already focused on the interaction between the MAC and RTR, we have used here the same 802.11 DCF MAC for each simulation. 1. Deployment: medium-sized office floor (40m× 40m) in a concrete and brick building. 2. Connections and Traffic: 7 connections between 5 different nodes over the whole simulation. CBR traffic with an injection interval of 0.001s. The transport agent is UDP. The connection and traffic schemes are the same for all of the simulations. 3. Routing Protocols: Two reactive (AODV, TORA) and one active (DSDV) routing protocols. The three levels of this discrete variable are denoted as {RTRi} = {AODV, DSDV, TORA}. 4. Nodes Density: 10, 20 and 30 nodes were randomly deployed all over the office floor. The associated levels are denoted as {NODESi} = {Nodes10, Nodes20, Nodes30}. Each scenario has a different random deployment. 5. Scenarios: 12 different scenarios modeling nodes deployment and mobility. For each scenario, the nodes are spread uniformely accross the 2D floor. Nodes are moving along straight lines connecting halls and office. Pause time is limited to one minute between the arrival and the departure from a point. 6. Simulator Used: ns-2.27 compiled, validated and running on MacOSX. The simulation time was 600 seconds of simulated operation. 7. Propagation: channel frequency set to 2.4 Ghz with 11 Mbp/s data rate. The other parameters, such as the carrier threshold and the signal strength are left to their default values (Lucent-class card). Three propagation models are refered as {PROPAi} = {FreeSpace, Shadowing,Raytracing}. 8. MAC Protocol: 802.11 DCF. 9. PHY layer: we have written a subclass of NS-2’s WirelessPhy class in order to take into account interferences and collision. This feature is not fully implemented in the current distribution of NS-2. 10. Modulation: BPSK. Once again we had to implement it as it is not present in the simulator. Fig. 1. Summary of variables used in the ANOVA statistics