## 3 Literature Survey and Critical Analysis
### Taxonomies and Literature Analysis
#### 3.2.3 Taxonomy 3: Multi-Objective Optimizer  
The analysis of the literature arrives at the third criterion followed to perform optimization in a principled manner: the algorithmic procedure by which every JSP under consideration has been optimized. In this regard, we used four criteria [36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49; 50; 51; 52; 53; 54; 55; 56; 57; 58; 59; 60; 61; 62; 63; 64; 65; 66; 67; 68; 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89; 90; 91; 92; 93; 94; 95] to orchestrate the literature in the taxonomy shown in Figure 5: (1) specific algorithm utilized; (2) the algorithm(s) to which the proposal has been compared; (3) crossover parameters; (4) mutation parameters. In what follows, we discuss the general design and methodological patterns stemming from this third systematic analysis. We begin the discussion by considering the specific multi-objective algorithm used in every contribution. This classification is focused on the multi-objective algorithm used to optimize the problem. A priori, here, we expected to find widely known multi-objective evolutionary algorithms, such as NSGA-II, MOEA/D, or SPEA-2. However, after carrying out the review, most of the papers analyzed referred to new optimization techniques that attempt to improve the performance of the aforementioned algorithms. This is why this feature does not account for the specific algorithm in use, but rather ascertains whether a new algorithm has been developed for the study or not. Surprisingly, we found that 88.5% of the articles included in the review developed new algorithms or improvements of existing ones, whereas only in 9.8% of the cases, no new algorithm was proposed. This fact links to one of the critical points elaborated later in Section 3.3. We turn the scope of the discussion to the algorithm for comparison included in benchmarks. For this criterion, we considered those techniques that the authors of every contribution used to compare their performance to that elicited by their own proposed algorithms. As expected, NSGA-II prevailed in the area as a comparison counterpart, with 32.47% of the papers featuring it in their benchmarks. This multi-objective evolutionary algorithm is followed by SPEA-2, which was used in 10.39% of the cases. MOEA/D, another popular multi-objective evolutionary algorithm, appeared in 9.09% of the analyzed literature, while the remainder included comparisons with other algorithms such as NSGA-III, MOGWO, SPEAR, etc. Another fact to take into account in this category is that commercial software for optimization such as CPLEX is quite frequently employed in this area. This statement is buttressed by Figure 6, where we can observe that the three aforementioned algorithms predominated over the rest. Figure 5: Taxonomies by multi-objective optimizer. Table 2 provides a complete list, ordered by publication year, of the algorithms used, the algorithm taken as a starting point to develop the newly proposed approach, and the algorithms against which it was compared. We can clearly observe that most of the papers focused their efforts on developing their own algorithm. It is also worth noting that none of the newly developed metaheuristics over the years makes a comparison with previously defined solvers in the table. This indicates that research works reported in this area mainly attempted beating generic algorithms (e.g., NSGA-II, ILS, MOGA) and did not include in the comparison algorithms specifically designed to solve JSP instances. Since most of the metaheuristics used to solve multi-objective formulations of the JSP are genetic algorithms (i.e., evolutionary algorithms using crossover and mutation operators in the variation step), we now analyze the crossover operators in use and, in particular, the value of the crossover probability parameter (see again Figure 5).