# Experimental Evaluation of Unicast and Multicast CoAP Group Communication
## 5 Evaluation
### Performance Evaluation on a Wireless Sensor Testbed
#### 5.2.2 Reliability  
Reliability is a key performance indicator. In this subsection, we experimentally evaluate the reliability of both unicast and multicast CoAP group communication in the presence of Wi-Fi interference. To generate this interference, we send UDP traffic from one Wi-Fi node to the other at a constant bandwidth by using the iperf tool. We have setup the Wi-Fi communication to use Wi-Fi Channel 13, which completely overlaps with IEEE 802.15.4 Channels 25 and 26 that we use inside the WSN. Since we are using CSMA as the Media Access Control (MAC), the sensor nodes will back off when Wi-Fi is sending. However this is not true for the other direction. Typically, Wi-Fi MAC will not detect that wireless sensors are sending and will not back off. To measure the reliability, we used the same experiment setup shown in Figure 6 to communicate with a group of 10, 20 and 30 members. We gradually increased the Wi-Fi interference in the network in steps of 5 Mb/s and measured the reliability of getting responses to the respective requests. We repeated the same experiment for our group communication solution and for multicasts. We run each experiment 50 times and show the averages of the member reliability (i.e., reliability of the communication with individual group members) in Figure 10. Multicasts are not transported reliably, and thus, the reliability of the network decreases as soon as there is a packet loss due to the Wi-Fi interference in the network. When using our unicast group communication solution, CoAP confirmable messages are used. For the group of 10 members, the reliability of individual resources remains always 100%, even when the Wi-Fi nodes were transmitting as fast as they could (28.5 Mb/s). The reliability of individual resources for the group of 20 nodes dropped a bit to 99.9% under maximum Wi-Fi interference. For the 30-member group, the reliability is further reduced to 99.5% (compared to 94.6% in the case of multicasts). Figure 10 also shows that the reliability of individual members decreases with an increasing group size, both for unicast and multicast communication. This is due to two reasons. First, \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Number of Nodes**} & \multicolumn{2}{c}{**Hop Count (\(h\))**} \\ \cline{2-3}  & **Average** & **Maximum** \\ \hline
5 & 1 & 1 \\
10 & 1.4 & 2 \\
20 & 1.7 & 3 \\
29 & 1.8 & 3 \\
40 & 2.4 & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Characteristics of the WSNs used in congestion control experiments on w-iLab.t Zwijnaarde wireless sensor testbed. larger groups are denser and, thus, have a higher chance of collision between the group members. Second, and maybe with a higher impact on the reliability, bigger groups have a larger average hop count. This means that every message (both request and reply) between a client and a server has an additional chance of getting dropped at each hop on the way to its destination. Nevertheless, in our 20- and 30-member groups, 100% reliability was maintained for unicast communication until a Wi-Fi transmission rate of 25 Mb/s, with one single exception for the 30-member group at 5 Mb/s, where one message was lost and resulted in a reliability of 99.9%. In many group communication use cases, it is desirable to get answers from all members of the group. A complete group communication is considered successful when communication to all members in the group is successful. Figure 11 shows the effect of packet loss on the reliability of the complete group for our 10-, 20- and 30-member groups. Certainly, the reliability of a complete group is less than the reliability of its individual members, since the loss of a message to or from a single member renders the complete group request unsuccessful. In these cases, the use of multicasts does not provide good results. Already at 15 Mb/s Wi-Fi traffic, the reliability of 20- and 30-member groups drops to about 80%.