# Target State Optimization: Drivability Improvement for Vehicles with Dual Clutch Transmissions
## 3 Benchmark: Self-Learning Algorithms  
To solve the optimization problem of Section 2, several self-learning algorithms have been applied to the problem. For comparing these algorithms, the virtual vehicle of Section 2.4 is used to ensure reproducible results. The behavior of the vehicle is therefore only influenced by varying parameters of the TCU (Section 2.5) to calibrate the launch behavior. These parameters are set with the following algorithms: the well-known GA NSGA-II and the RL approaches: Deep Deterministic Policy Gradient (DDPG) [54], Proximal Policy Optimization (PPO) [55], Advantage Actor-Critic (A2C) [56] and Soft Actor-Critic (SAC) [57]. To validate the algorithms, vehicle launches are performed within the software in the loop environment and evaluated regarding the optimization objectives of Sections 2.1 and 2.2. Each algorithm is tested within five test runs to neglect the influence of having favorable or unfavorable starting conditions. The starting conditions can vary since, usually in optimization algorithms, the first iterations are performed with a set of randomly chosen parameters. [PERSON] [24] introduced for this purpose a design of experiments to identify a starting population, which gains a wide spread of results within the search space and, therefore, gathering the most possible information out of one iteration. The benefit of having a greater knowledge about the search space is that the evaluation time is shortened [24]. The drawbacks of the method are that domain knowledge is required to set up such a design of experiments and the method is difficult to generalize. Therefore, for this study, such a design of experiments is not intended, instead, the aim is to identify methods which generalize for different optimization problems. Thus, the five test runs are introduced to compare the algorithms while minimizing the influence of starting conditions. A single test run contains 1000 consecutive iterations (1000 vehicle launches within the software in the loop environment). The number of successful iterations (as explained in Section 2.3 a launch is successful if the reward is zero) and the required iterations until the first successful iteration occurs is determined in each test run. The optimization usually stops if the first successful result is observed in a test vehicle. During these tests the optimization continues until 1000 iterations are reached to ensure a sustainable quality of the results and to eliminate the chance of having a random hit. To eliminate outliers the results of these five test runs are averaged. The results of the five test runs are shown in Table 3. The column \(\overline{x}_{S}\) is the average of the successful iterations, \(\sigma_{S}\) denotes to the standard deviation and the columns \(min_{S}\) and \(max_{S}\) indicates the least respectively the highest number of successful iterations out of the five test runs. The corresponding average of the first success (reward is zero for the first time) is illustrated in column \(\overline{x}_{F}\) and the standard deviation is illustrated in column \(\sigma_{F}\). From the results of Table 3 it is observable that the drawbacks of Section 1 are proved and that the GA outperforms the RL algorithms. The standard deviation indicates that the results are reproducible. The only RL algorithm with some promising results for this single step problem is the SAC algorithm. The fact that one test run contained 1000 iterations only is a probable cause of the lack of performance of the RL algorithms since RL needs many iterations until promising results are available (and 1000 might be not enough) [37; 41].