# Complementing Solutions to Optimization Problems via Crowdsourcing on Video Game Plays
## 4 Results and Discussion
### Scheduling Problems
#### 4.1.3 Solving OSS, Using Genetic Algorithm (GA), GBF and Crowdsourcing (CS)  
Figure 6 displays the output results JSS after solving with diverse approaches for m = 5 (see Figure 5(a) and for m = 7 (see Figure 5(b). Each whisker box represents the output solutions for the following approaches, namely, genetic algorithm (GA), second the crowdsourcing approach, third the GBF algorithm, and finally the fourth and fifth whiskers boxes illustrate the hybrid approach CS-GA and CS-GBF. The first whiskers box from Figure 5(a),b, shows the output results obtained using a traditional genetic algorithm. Clearly the genetic algorithm approach outperforms all other approaches, for bigger instances, m = 7 it cannot be complemented with those output solutions obtained using crowdsourcing, as Figure 5(b) illustrates. Nevertheless, users have yielded as good enough solutions as the genetic algorithm, for some cases, as Figure 5(a) illustrates. Yet, inserting crowdsourcing-based solutions to a genetic algorithm is not a straight forward task; consider the following assumptions, namely: first, we must have knowledge of exactly which moves the player made; second within the genetic algorithm we need to find the exact intermediary state where to insert the solution; third, clearly identify non-dominated solutions so it is worth injecting them into the genetic algorithm, which already is an NP-hard problem. On the other hand, Figure 5(b) (for m = 7), we can argue that due to the robustness of the genetic algorithm, as we expected the resulting output solutions of the genetic algorithm yielded better results than those yielded by the crowdsourcing approach. However, genetic algorithm does many iterations to reach an objective function, while crowdsourcing participants have only played once in order to obtain an equivalent solution, justifying why genetic algorithm obtains better solutions, as we fixed the number of iterations to 100. To improve our crowdsourcing-based approach and the output solutions we would have to train the players; implying each participant plays a greater number of games and spends a greater overall time to learn optimization techniques. In contrast when complementing a not so robust approach, as Figure 5(a),b illustrates the crowdsourcing (CS)-based approach has obtained better scores than some of the GBF algorithm. The (GBF) has difficulties to find solutions because this algorithm is less likely to sacrifice an immediate gain for a more significant long-term gain, in contrast to the behavior of the human players (even though stochastic) who identified these situations and have followed the path that led them to a better overall score (Figure 5(a)), without having to explore all possible moves. Figure 6: Performance of genetic algorithms (GA), crowdsourcing (CS), greedy best-first (GBF) search algorithm, and the merged approaches (CS-GA and CS-GBF) for solving the Open Shop Scheduling (OSS) for (**a**) m = 5, (**b**) m = 7 machines. Similar to the previous figures, we have added a fifth whisker plot for GBF-CS in Figure 6a,b, we have obtained these results by selecting the best output performance values from GBF search algorithm and from crowdsourcing-based approach. For instance, for m = 5 in Figure 6a, we have considered four solutions from GBF and 16 solutions from crowdsourcing. The range of solutions is smaller than that for crowdsourcing (CS) but greater than that for the GBF. This finding does not imply that the GBF solutions are better than the other solution sets, as more than 75% of the GBF-CS solutions have a lower makespan than the GBF solutions (second box) and 100% of the GBF-CS solutions have a lower makespan than 75% of the crowdsourcing (CS) solutions (second whisker box). In addition, we built the fifth whisker plot in Figure 6b for m = 7, using 11 GBF and nine crowdsourcing values respectively. We can observe that the variability of the solutions is significantly lower than that for the crowdsourcing and GBF solutions.