# Complementing Solutions to Optimization Problems via Crowdsourcing on Video Game Plays
## 4 Results and Discussion
### Flp
#### 4.2.2 Results for the London Dataset  
We performed two other experiments for solving the FLP using a larger dataset (London dataset). Figure 10a shows the Pareto set distribution found using genetic algorithms. Likewise, Figure 10b shows the Pareto set distribution found by crowdsourcing (CS) the problem to users. Again, the problem was solved for two conflicting objective functions. We have selected the best answers computed from 20 different players (shown in Figure 10b), and 20 different executions of the genetic algorithms (shown in Figure 10a). Figure 10c, illustrates how the solutions of players complement those found using genetic algorithms. From the data we obtained shown in Figure 10a,b we found that users obtained better solutions in many cases, as shown in Figure 10c, for example, in finding the best locations to place facilities for \(n=5\) up until \(n=27\). Sometimes, the CS approaches find a solution, whereas genetic algorithms do  
Figure 8: The Pareto set after solving the facility location problem for the Swain dataset using: genetic algorithms (**a**), crowdsourcing (**b**) and finally the genetic algorithm complemented by crowdsourcing solutions (**c**). Figure 9: The Pareto set after solving the facility location problem for the Swain dataset using: genetic algorithms, crowdsourcing and cubic splines. not, for example, when \(n=2,3,4\) or the genetic algorithms give no value, as for \(n=28\) to \(n=42\). Finally, the genetic algorithms present gaps from \(n=98\) to \(n=119\). Note that the total distance should always decrease as the number of facilities increases (because the best solution for \(n\) open facilities, when augmented with any other open facility, will yield a lower total distance for the \(n+1\)). However, in the crowd-sourced data, the graph presents regions in which the total distance increases because those solutions were independently encountered and were found separately in the raw data. Through some necessary processing, statistical analysis resulted in an upper bound to the total distance of the solution located to the right for any given case. Figure 10 also shows that the players give better solutions when placing a large number of facilities or when asked to identify the best places for facilities. For instance, for \(n=120\) to \(n=155\); the crowdsourcing (CS) approach finds a better solution than the genetic algorithm. Figure 11 shows the genetic algorithm solutions for the London dataset case, where we show how genetic algorithms can be complemented by two different methods, namely, crowdsourcing and interpolation based on cubic splines. In this figure, analogous to Figure 9, we illustrate that in most cases, when complementing the missing solutions for the genetic algorithms, the crowdsourcing solutions are better than those obtained by interpolation. We chose one of the best cubic splines tools for interpolating by polynomials; however, once again, the crowdsourcing solutions are the best approximations. From Figures 8 and 10 we can conclude that users can complement the Pareto search space; in fact, in many cases, the solutions obtained by the crowdsourcing approach are better than those obtained with the genetic algorithms. It would be interesting to explore a hybrid combination of  
Figure 11: The Pareto set after solving the facility location problem for the London dataset using: genetic algorithms, crowdsourcing and cubic splines. Figure 10: The Pareto set after solving the facility location problem for the London dataset using: genetic algorithms (**a**), crowdsourcing (**b**) and the genetic algorithm complemented by crowdsourcing solutions (**c**). these approaches because genetic algorithms present gaps, and our empirical study has shown that crowdsourcing can provide a sufficient number of solutions for filling those gaps. Likewise, with the help of machine learning techniques, it may be possible to analyze the heuristics that users follow or implement to solve computationally hard problems.