the channel are modeled, using Markov chains. For the lower layers, the authors used the characteristics of the CC2420 transceiver \cite{11}. They show that the standard specified MAC can be accurately modeled as non-persistent CSMA. They also demonstrate that letting the radio in a shutdown state between the different transmissions is a very effective means of reducing the average power consumption for a very wide range of traffic rates. Finally, they propose to initialize the contention window length to 1, in order to significantly improve the throughput and reduce the energy consumption when MAC-level acknowledgements are not used.

Misic et al. have modeled in \cite{9} the operation of the IEEE 802.15.4 MAC layer in the beacon-enabled mode through discrete time Markov chains. They identify the downlink queue stability at the PAN coordinator as the tightest criterion for the network. Consequently, they assume that the number of nodes and their traffic load should be chosen to avoid the saturation point of the network.

In \cite{10}, the authors provide an analytic performance model using Markov chains in order to compute the saturation throughput of the network in a star topology. The model is validated through simulations with NS2 in a network composed of a maximum of 50 nodes. One of the main conclusions is that the aggregated throughput is never higher than 70Kbps, whatever the number of nodes and the total load.

In this paper, we propose an original study consisting in evaluating several MAC protocols (BMAC and TKN15.4), using a real 802.15.4 testbed with two different environments (indoor and outdoor). We use the throughput, the RSSI and the loss ratio as evaluation metrics. The goal of this comparative study is to illustrate the impact of the MAC protocols which observe the IEEE 802.15.4 standard (such as TKN15.4) and those which do not (such as BMAC). The second contribution of this paper is to present and compare the results given by three major network simulators (NS2, OPNET, QualNet) with the testbed results. The added value of this contribution is to present and analyze the testbed and the simulation results in order to point out their divergences and the causes of these divergences.

\section{Evaluation of B-MAC and TKN15.4 over Imote2}

\subsection{Context}

In this section, we point out the significant impact of the environment on the wireless communications. We distinguish two main environments: the indoor and outdoor environments. The indoor environment is more realistic, for instance the use of wireless access in companies, offices and at home. The outdoor environment represents a free space area without any physical obstacle, such as emergency deployments. It is thus much less frequent, especially in a WPAN context. Nevertheless, many performance simulation studies on WPAN use the free space model, which is not very realistic. In the simulator, we can easily reproduce an outdoor environment, using a free space propagation model. However, this model does not take into account the floor reflection signals. On the opposite, the Two-ray ground model does consider these signals from a certain distance threshold between the transmitter and receiver nodes.

For our experiment, we selected scenarios taking place in two realistic environments. The first one is an indoor environment, represented by our laboratory, made up of 15--30m2 offices located along a 50m corridor. This scenario illustrates a usual office context. The nodes are located both in the corridor and an office, in order to take into account the fading effects. Nowadays, the WiFi technology is used in all buildings. Thus, we kept the existing WiFi communication on, in order to get results as close to the reality as possible (the 802.11 and 802.15.4 technologies use the same frequency band: 2.4 GHz). When the transmitter sensor wants to send a packet, it selects an available channel. The second one is an outdoor environment, also called free space. This environment is represented by our campus park without any obstacle. In both scenarios, we used one CBR connection with maximum rate in order to reach the limit of the channel capacity (1KHz). The selected frame size is set to its maximum value (127 Bytes). The distance between the transmitter and receiver nodes fluctuates between 0 and 65m, which is its maximum value. Table~\ref{tab:table_1} shows the default parameters of CC2420 technology.

\begin{table}[h]
\centering
\caption{CC2420 Characteristics}
\label{tab:table_1}
\begin{tabular}{ll}
Frequency Band (ISM) & 2400.0 -- 2483.5 MHz \\
Data Rate & 250 kb/s \\
Tx Power & -24 -- 0 dBm \\
Rx Sensitivity & -94 dBm \\
Range (line of sight) & \textasciitilde 30 m
\end{tabular}
\end{table}

For our experiment, we used the Crossbow sensor technology \cite{13}, particularly Imote2 sensors. These sensors are equipped with CC2420 radio transceiver \cite{11}, a 13--416MHz processor, a 256kB SRAM and a 32MB SDRAM. BMAC is the native MAC protocol in Imote2 sensors. We know that this protocol does not observe IEEE 802.15.4 standard and many capabilities are not implemented (such as the beacon-enabled mode). In order to compare the native BMAC protocol and other protocols which observe IEEE 802.15.4 standard, we have implemented an adapted TKN-15.4 MAC protocol to the Imote2 platform. The TKN-15.4 protocol is provided by the TinyOS 15.4 working group. In this study, we only focus on the non-beacon-enabled mode, in order to provide a fair comparison between TKN-15.4 and BMAC. Indeed, BMAC protocol does not take into account the beacon-enabled mode. Table~\ref{tab:table_2} shows the overhead attributed to BMAC and TKN-15.4 protocols.

\begin{table}[h]
\centering
\caption{BMAC and TKN-15.4 Characteristics}
\label{tab:table_2}
\begin{tabular}{lll}
 & BMAC & TKN-15.4 \\
PHY & 5 bytes & 5 bytes \\
MAC & 11 bytes & 11 bytes \\
NWK & 2 bytes & 0 byte \\
DATA & 113 bytes & 113 bytes
\end{tabular}
\end{table}

The selected metrics for this study are the throughput, the RSSI (Received Signal Strength Indication) and the packet loss ratio in order to evaluate the performance of the transmission and the quality of the channel.