# A Secure Link-Layer Connectivity Platform for Multi-Site NFV Services
## 4 Implementation and Validation
### Performance Evaluation
#### 4.2.1 Practical Experiments and Results  
In our experiments, we considered different configuration options for the L2S VNFs, aiming at evaluating the impact on performance of security mechanisms and the available CPU resources. For each configuration under test, we used the _iPerf_ tool to measure the maximum average throughput between the communication endpoints, repeating each throughput measurement 40 times to calculate the corresponding average value. Figure 5 collects the results obtained from the different configurations of the L2S VNFs. In a first experiment, we disabled the security mechanisms at the L2S VNFs. This way, data traffic was exchanged unprotected between communication endpoints, using the VXLAN tunnel established between the L2S VNFs. Using 1 virtual CPU at each L2S VNF results in a maximum average throughput of 1.13 Gb/s. This value is determined by the overhead of VXLAN tunneling operations (our preliminary tests indicate a baseline throughput higher than 15 Gb/s if VXLAN and IPsec procedures are disabled). In a second experiment, we enabled IPsec transport mode at both L2S VNFs. This caused a noticeable reduction of the achievable throughput, with an average value of 556.67 Mb/s. The use of VXLAN interfaces reduces the MTU on L2S VLANs to 1450 bytes. This is due to VXLAN tunneling procedures, which encapsulate the MAC frames received from the communication endpoints into IP packets, appending an additional overhead to the MAC frames in the form of outer headers [18]. These packets can then be forwarded out the Ethernet interface of the L2S VNF towards the edge router, which presents a MTU of 1500 bytes (i.e., the default Ethernet MTU). Hence, VXLAN interfaces support a reduced  
Figure 4: Performance evaluation scenario. Figure 5: Performance of the L2S platform implementation. MTU value of 1450 bytes. As each VXLAN interface is connected to an access port of its corresponding L2S VNF through the _Open vSwitch_ software, the MTU of all the access ports of an L2S VNF will be 1450 bytes. Consequently, this will be the MTU of any virtual link that is available for inter-site endpoint communications. We want to note that using IPsec causes fragmentation of data packets at L2S VNFs. As commented, a communication endpoint will generate IP packets with a size of 1450 bytes. However, after appending the IPsec protocol overhead to VXLAN encapsulated packets, these packets will exceed the MTU of the outgoing interface at the L2S VNFs, i.e., 1500 bytes. The Ethernet interface of the L2S VNF will split each data packet produced by IPsec into two fragments, which will be delivered to the other VXLAN tunnel endpoint, i.e., the other L2S VNF. These fragments will need to be reassembled at the receiving VXLAN tunnel endpoint. This process of fragmentation and reassembly requires computation at the L2S VNFs [26], causing a negative impact on performance. An alternative to address this well-known issue is to fragment IP packets before they are encapsulated [27], configuring an appropriate MTU value in the upstream data path, such that: (1) fragmentation happens before IPsec processes; and (2) the VXLAN and IPsec protocol overheads do not make IP fragments exceed the MTU of the outgoing link at the L2S VNFs. This presents the additional benefit that reassembly operations are not needed at the L2S VNFs (i.e., the VXLAN tunnel endpoints). Instead, they will be performed by the destination of the fragmented packets (the communication endpoints in our experiments). In our solution, this can be done by simply decreasing the MTU on the L2S VLANs to account for the IPsec protocol overhead. To this purpose, in a third experiment, we have set this MTU value to 1400 bytes. With this, we observed a maximum average throughput of 616.5 Mb/s, i.e., an increase of approximately 10.7% with respect to the case where the MTU was 1450 bytes and fragmentation and reassembly was performed at the L2S VNFs.