## 6 How to Evaluate Defense Systems
### [PERSON] et al. [46] set up a hybrid Cloud, using Amazon EC2 as a public Cloud and simulating a private Cloud in their lab. The private Cloud consists of two Linux machines, one of which hosting DaMask and the network controller while the other emulates a virtual network to extend the private Cloud. They wanted to measure the communication costs as well as the computation overhead of DaMask. They began by computing the network bandwidth between the private and the public Cloud, with and without DaMask. For the communication overhead, caused by the traffic being examined by DaMask, they showed the overhead was a constant if the link status of network remained stable. As for the computation overhead caused by the detection algorithm, they ended up with an interference time of 80 ms, which they consider quite efficient.
With the defense system, the CPU load was only around 10% for 20 s, while blocking the attack, and without additional memory usage. The mechanism also successfully handled a coercive parsing attack, an oversize encryption attack and a spoofing attack. To evaluate the response time of their system, they sent 200 SOAP requests at a rate of two requests per second and then 10,000 SOAP requests at 50 requests per second, both with and without the filter in place. In the first load, introducing the defense system made the processing time of the web service go from 3 to 5 ms, whereas in the high load, it went from 3 to 9 ms. Those results outperform the current existing solutions (for example, the Cloud Protector mentioned above). [PERSON] et al. [53] conducted four experiments. First, they evaluated how the framework could protect against flash crowd attacks, then against high rate DDoS attacks. Third, they studied the ability to validate clients and trace the true IP source of the attack. Eventually, they evaluated the proper blocking of the attacking IP address as close as possible to the network entrance. They used the following parameters to make the evaluations: the number of incoming requests and number of detected and prevented attacks against the web application (conducted by way of the Apache log, based on the response code number to the incoming request--an error code means the attack was detected). Finally, 420,000 incoming requests were generated to cause a DDoS. The AntiDDoS_Shield system detected and prevented all high rate HTTP-based DoS/DDoS attacks: 369,726 out of 420,000 flash crowd (FC) attacks, at the edge router. AntiDDoS_Shield system succeeded in validating and tracing back 369,726 out of 420,000 incoming requests. In summary, apart from defense architectures that have no evaluations whatsoever, most defenses came with experiments on design and performance, whether theoretical or practical. By using several sets of data and carefully choosing the parameters of interest, the authors were able to study the response of their system under different workloads, while reproducing real-life situations. They discussed the possibility of adopting their system regarding the cost, scenario (real time), best practices (how to correctly set up the system for maximal performance), overhead, precision (rate of false positives or negatives), effectiveness to detect and filter attack messages and ability to detect new attacks. [PERSON] et al. [55] propose a survey and a taxonomy to evaluate the defense against DoS and DDoS attacks in Cloud Computing.