# Variable Neighborhood Search for the Two-Echelon Electric Vehicle Routing Problem with Time Windows
## 5 Computational Experiments
### Numerical Results
In this latter case, CPLEX could only provide valid solutions (without being able to prove optimality) in seven out of 12 instances. Nevertheless, for one instance (C106_15), CPLEX produced a better solution than both VNS variants. Both VNS variants performed comparably on small problem instances with 5 and 10 customers. They were able to find solutions with the same objective function values as those of CPLEX. However, the performance of the two VNS variants starts to differ on the  
instances with 15 customers. While VNS\({}_{\text{full}}\) provides results at least as good as CPLEX for all instances except for C106_C15, VNS\({}_{\text{red}}\) only does so in seven out of 12 cases. Considering those instances for which CPLEX was able to obtain a solution, both VNS variants improved the solution quality of CPLEX, on average, by 0.55% (VNS\({}_{\text{red}}\) ) and 6.86% (VNS\({}_{\text{full}}\) ). In fact, VNS\({}_{\text{full}}\) outperforms VNS\({}_{\text{red}}\) both in terms of best-performance (column 'dist') and in terms of average-performance (column 'avg'). Note also that the running times of both VNS\({}_{\text{full}}\) and VNS\({}_{\text{red}}\) are in the order of seconds. While the superiority of both VNS\({}_{\text{full}}\) and VNS\({}_{\text{red}}\) over CPLEX in terms of CPU time is more significant for the instances with 10 and 15 customers, see Tables 6 and 7, only VNS\({}_{\text{red}}\) provides better CPU times for the instances with 5 customers. Finally, note that the results of the C&W savings heuristic are, in the context of these small problem instances, approx. 20% worse than the best results obtained. This is, however, achieved in very low computation times of a fraction of a second, which shows that our C&W savings heuristic is a good candidate for producing the initial solutions of VNS. Next, we analyze the results of the four approaches when applied to the large problem instances of our benchmark set. These results are shown in Tables 8-10. The structure of these tables is slightly different to the one of the previous result tables. First, results of CPLEX are not provided, because CPLEX was not able to generate a single valid solution within 2 h of computation time. Second, the additional column with heading 'imp(%)' provides the improvement (in percent) of the VNS variants over the results of the C&W savings heuristic. In addition to the tables we also provide _critical difference_ (CD) plots [62] as a statistical tool for assisting the interpretation of the obtained results. First, the Friedman test was used to compare the three approaches simultaneously. As a consequence of the rejection of the hypothesis that the techniques perform equally, the corresponding pairwise comparisons were performed using the Nemenyi post hoc test [63]. The obtained results are graphically shown by means of the above-mentioned CD plots in Figure 8. Note that each considered algorithm variant is placed on the horizontal axis according to its average ranking for the considered subset of problem instances. The performances of those algorithm variants that are below the critical difference threshold (computed with a significance level of 0.05) are considered as statistically equivalent; see the horizontal bars joining the markers of the respective algorithm variants. The following observations can be made. For the large clustered instances (Table 8) and large random instances (Table 9), VNS\({}_{\text{full}}\) significantly outperforms VNS\({}_{\text{red}}\), both in terms of best-performance and average-performance. This is also shown in Figure 8b,c. However, the opposite is generally the case in the context of random-clustered instances, as shown in Figure 8d. This means that the removal/destroy operators have a rather negative impact on the performance of VNS in these cases. This is most probably due to their elevated computation time requirements. Nevertheless, Figure 8d also shows that this difference is not statistically significant.