## 5 Methodology
### Performance Evaluation Framework  
The methodology for evaluating CNI performance within a Kubernetes environment is an automated, iterative approach that evaluates network performance metrics such as latency, bandwidth, and packet handling. The iterative used by this methodology starts with Kubernetes installation, followed by these phases:  
* Automated CNI deployment, start containers with performance evaluation tools built-in (iperf for bandwidth evaluation, netperf for latency evaluations), start evaluations;
* Iterate through tuned optimizations--six built-in tuned profiles, two developed for this, and the paper referenced [57], based on previous evaluations;
* Iterate through 1 Gbit/s, 10 Gbit/s, 10 Gbit/s LAG, 10 Gbit/s fiber-to-fiber, local net-work evaluations (configurable network configurations);
* Iterate through packet size and frame configurations;
* Loop through TCP bandwidth/latency (five times), UDP bandwidth/latency evaluations (two times)--number of runs is configurable;
* Results for all runs are parsed and uploaded to the MySQL database;
* Automated CNI de-installation, followed by the next CNI installation, iterate through the whole process again. During the development phase of this methodology, manual checks were made many times to check if the results were meaningful. Several manual results evaluations were performed during the methodology's development phase. This was an added layer of human-based checks to verify that the Ansible code works as designed. The core design is intended to provide an efficient way for the high-performance computing community to evaluate CNI plugins and configurations under varying conditions. It utilizes tools like perf and netperf for precise measurements. The tests are deployed using Ansible, while the results are stored in MySQL and visualized through Grafana. Grafana offers complete customizability regarding displaying data read from the data source--in the case of this methodology, MySQL. An example of the Grafana dashboard for TCP bandwidth is shown in Figure 1 (the green color is for Antrea, the red color for Cilium, the blue color for Calico, and the orange color for Flannel):  
Figure 1: An example Grafana dashboard with CNI performance overhead when using NIC optimizations tuned profile, compared to the evaluation of the physical network using the same tuned profile. The value of 0 equals the performance of the same network setup without a CNI, describing the difference between the physical network without a CNI plugin and the four covered CNI plugins. The more negative the value, the lower the performance of a CNI for a given tuned profile. Also, an example that can be used to display the results of TCP latency evaluations is shown in Figure 2 (the green color is for Antrea, the red color for Cilium, the blue color for Calico, and the orange color for Flannel):  
In this example, the latency graph displays the opposite of the bandwidth graph. The value 0 is the relative latency performance evaluated on the physical network. The more positive the evaluated latency is, the more latent a CNI plugin is. However, this is one of the infinite ways data can be displayed in Grafana, as it offers endless customization options for all visualizations. The system uses a set of Ansible playbooks executed by a script to manage and control the testing environment. The framework includes several key components:  
1. [noitemsep,topsep=0 pt]
2. Ansible Automation: Manages configuration deployment, runs tests, and collects data across different test scenarios. Ansible scripts ensure that all tests follow a consistent process, from initiating network configurations to collecting performance metrics;
3. CNI Plugins: The performance of multiple CNI plugins--Antrea, Calico, Cilium, and Flannel--is evaluated. The system automatically deploys and removes CNI plugins using HELM charts, allowing for streamlined testing of multiple configurations without manual intervention;
4.