## 5 Robustness Results and Analysis
### Varying the Message Generation Rate  
Most of the parameters for the scenarios are equal to the ones described in Table 2 of Section 4.2. Note that the \"Base\" scenario corresponds to the exact settings of Table 2 and serves as a comparison for the other scenarios. In the \"Less messages\" scenario, a node creates a message every 35-70 s. In the \"More messages\" scenario, each node generates messages in an interval from 9 to 18 s. The \"Bigger transmission rate\" scenario maintains the message generation rate from the base scenario, but the transmission rate from the nodes' interfaces increases to double. This is summarized in Table 3.  
First, looking at the node classification metrics, it would be expected that, for the \"Less messages\" scenario, fewer black-hole nodes would be identified, and for the \"More messages\" and \"Bigger transmission rate\" scenarios, more. For the \"Less messages\" scenario, fewer messages would be created. Therefore, fewer messages circulate in the network, and fewer opportunities exist for nodes to prove themselves as good. The exact opposite happens with the \"More messages\" scenario. In the \"Bigger transmission rate\" scenario, the message transmission rate is double, allowing nodes to exchange more messages and having more and faster information to classify nodes as good or as bad. The results for the Prophet are presented in Figure 9 for 20% of malicious nodes. The 95% confidence intervals are very small, showing great trust in the results.  
Analyzing the results, the expected is confirmed. The detection ratio for good nodes increases in all scenarios, ranging from 70% to 90%, depending on the scenario. After 8 h, when the action phase starts and messages from black-hole nodes are dropped, traffic in the network is reduced, so the detection rate increases more slowly. The false-positive ratio decreases to zero for most of the simulation before 4 h. For the routing protocol metrics, it was decided to test the reputation system with various percentages of malicious nodes  
\begin{table}
\begin{tabular}{c c c c c} \hline
**Scenario Name** & **Base** & **Less Messages** & **More Messages** & **Bigger Transmission Rate** \\ \hline Message generation interval & 25–35 s & 35–70 s & 9–18 s & 25–35 s \\ Interfaces’ data rate & 2 Mbps & 2 Mbps & 2 Mbps & 4 Mbps \\ \hline \end{tabular}
\end{table}
Table 3: Settings assigned to different simulation scenarios to assess message generation impact.  
to test how BiRep would respond. The results for the same sets of simulations used for Figure 9 are presented in Figure 10. Again, the 95% confidence intervals are small, showing good confidence in the results.  
Without a reputation scheme, the delivery ratio, average latency, and overhead ratio for good nodes performance decrease as the percentage of black-hole nodes increases. Having more black-hole nodes in the network makes it harder for good nodes to deliver messages, resulting in a lower delivery ratio. When they deliver the messages, it takes longer, so more latency and, in general, more hops. The overhead ratio is also bigger because, with the increase in malicious nodes in the network, the same number of messages are relayed, but fewer are delivered using more hops. It is desired that all scenarios have a better performance than if no reputation scheme was used.  
Prophet, for all scenarios, shows better results for the delivery ratio for good nodes than if no reputation scheme was used. This proves the usefulness of the reputation scheme. Furthermore, the overall performance of the other metrics also improves.  
Figure 9: Comparison of detection ratio for good nodes and false-positive ratio for good nodes for Prophet, with 20% of malicious nodes to test the robustness of scenarios with different message generation and transmission settings.