To address the issues with optimization performance, the researchers first considered the appropriate choice of the control parameter values for different type of objective function landscapes. However, it has been shown that single fixed control parameter value chosen for the certain problem is not adequate for the whole optimization process. Therefore, researchers developed several adaptive and self-adaptive strategies for changing the value of control parameters based on some information from the optimization process [34]. Furthermore, to successfully deal with MOO problems researchers proposed numerous variants of DE algorithm, which include external archive where the successfully non-dominated solutions found during the evolutionary process are stored and lately, during the evolution process, employed to direct the individuals towards global optimum [11; 35]. Another widely applied algorithm from the group of EAs is the PSO algorithm. This method is inspired by the intelligence of the swarm of birds searching for food [25]. Initially, it has been proposed for solving unconstrained single-objective optimization problems in the continuous domain. In the last few decades, PSO algorithm has been applied to several practical engineering optimization problems, where it has shown good performance. However, the conventional PSO algorithm has some drawbacks in solving large scale complex optimization problems, such as premature convergence and trapping into the local optima [36]. As with other EAs, the optimization performance of PSO algorithm dependslargely on the proper choice of the control parameters such as social \(c_{1}\) and cognitive \(c_{2}\) acceleration coefficients and population size \(N_{P}\). In this way, to deal with above-mentioned drawbacks there has been several papers dealing with the appropriate choice of these parameters, among which the time-varying acceleration coefficients have been widely applied [36; 37]. To improve the performance in solving the MOO problems researchers have introduced the archiving strategies into the PSO algorithm, where the best individuals are selected from a non-dominated external archive [11; 38]. Furthermore, researchers studied the impact of different PSO topologies on the outcome of the optimization process [22]. To further enhance the optimization performance of PSO algorithm in solving MOO problems, in this paper an improved hybrid PSO and DE algorithm, called Multi-Objective Hybrid Particle Swarm Optimization Differential Evolution (MHPSODE) algorithm, has been proposed to deal with the complex MOO of planetary gearbox design. To address the issues that appear during the optimization of the conventional DE and PSO algorithms, in this paper a hybrid algorithm has been proposed which successfully combines the traits of each algorithm and overcomes the disadvantages. The hybridization has been performed by introducing the mutation operators DE/rand/1 and DE/current-to-best/1 from DE algorithm into the velocity update equation of the PSO algorithm, with the adaptive normalized population spacing parameter employed to select the appropriate mutation operator for the current optimization condition. The experimental results have demonstrated that the proposed MHPSODE algorithm is able to achieve better balance between convergence and diversity compared to well-known existing algorithms, such as NSGA-II and DEMO by adopting the proposed adaptive mutation mechanism. In summary, the main contributions of this paper can be outlined as follows:  
* The methodology for the MOO of the single stage planetary gearbox optimization, along with a comprehensive number of conflicting objective functions and several critical design constraints has been formulated in this paper. * The theoretical formulation for the calculation of the planetary gearbox power efficiency has been outlined, as a significant objective in the formulated MOO problem. To evaluate the efficiency of the planetary gearbox and perform numerical analysis the appropriate numerical procedure has been presented.