# Experimental Evaluation of Unicast and Multicast CoAP Group Communication
## 5 Evaluation
### Evaluation in a Real-Life Setup  
The evaluations we have conducted so far were performed in a more or less controlled environment in an open-room testbed. This allowed us to analyze the behavior of the different group communication approaches in a clean environment with very limited external interferences. It also allowed us to inject controlled Wi-Fi interfering traffic and to study its effect of the WSN and the used group communication approaches. By doing so, we were able to compare the obtained results with the simulation results, which were also conducted in a comparable controlled environment. In some use cases, the radio environment might be clean, such as is the case inside a shielded room, such as our testbed. However, in many use cases, and in our building automation use case in particular, it is expected that much interference will be present that will affect the WSN. Typical WSNs operate in the Industrial, Scientific and Medical (ISM) radio bands. These radio bands are used by many other devices, not just WSNs and Wi-Fi, e.g., microwave ovens, cordless phones and Bluetooth. In order to evaluate the behavior of our solution and that of the standard multicast solution in an environment that is more realistic than the open-room testbed, we have conducted a series of tests on the w-iLab.t office testbed. This wireless sensor testbed contains about 200 nodes spread over three floors (15 by 90 m each) of an operational office building in Ghent, Belgium [33]. In our experiments, we used 10 nodes spread over parts of the third floor during office hours (Figure 15). The circles represent the location of the nodes on the third floor of the office building. The filled circles represent the nodes used in the experiment. The other nodes were idle. As can be seen in the figure, the nodes are located in different rooms and in the hallways. In this setup, we have no control over the amount of interference that occurs during the experiments, since the sources of interference are diverse and are beyond our control; as is typically the case when using the Industrial, Scientific and Medical (ISM) radio band. A main source of interference in this part of the building is Wi-Fi, which was operating on Wi-Fi Channel 13. As explained before, this channel completely overlaps IEEE 802.15.4 Channel 26, which we used inside the WSN. Other sources of interference are also present (cordless phones, Bluetooth headphones and a microwave oven), but those are not always used. We conducted two sets of experiments at two different times of the day in order to get different conditions for the tests. The first set of experiments was conducted when a few people were present at the office, and the other set was conducted when almost everybody was present and working. In each set of experiments, we queried the group of ten nodes 50 times using our unicast-based group communication solution and another 50 times using multicast. Similar to the experiments in the open-room testbed, we have measured the reliability of the communication and the response time. In Table 8, we summarize the results of these experiments. These results confirm the results  
Figure 15: Experimental setup at the w-iLab.t office. The circles represent the location of the nodes on the third floor of the office building. The filled circles represent the nodes used in the experiment. The other nodes were idle. we obtained from the experiments on the open-room testbed. Under normal network environments, multicasts reliability is not very encouraging. Replies were received from the members with a success rate of 91.2%, resulting in a poor 58% reliability for the complete group. On the other hand, the reliability of our solution was maintained at 100% since the CoAP retransmission mechanism was able to handle all errors. Of course, this leads to higher response times for the unicast groups, which increased from 1.7 s to 7.21 s on average between the two experiments. As expected, the response time for multicasts was not much affected.