## 5 Dynamic Optimization of the Crowdsourcing Platform
### Dynamic Optimization Engine Based on Metric Indices
On the other hand, the _Index_ approach reports a lower number of optimization actions but each one consumes more time than in the case of the _DQL_. The result of accumulating the CPU time of the optimization actions of each algorithm is 23.86 s for the metric index-based approach and 11.73 ms for the optimization with _DQL_. However, the execution time of the simulation with both approaches tends to be similar, close to 108 s. Notice that the time of the crowdsourcing campaign is higher than 90 h; therefore, the execution times of both dynamic optimization approaches are negligible when operating on a real system. Regarding the time required to set up the index-based dynamic optimization engine, it is important to notice that the database and the index are only built once, and it depends on the size of the database, but generally it takes a couple of hours at most. When using the _DQL_-based approach, we also have to build the configuration vectors (databases) from previous simulations and then we have to train the model. The training process can take several days, and it must be repeated for each trained model until the desired result is obtained. So, there is a higher uncertainty in the amount of time required to fine-tune the accuracy of the _DQL_-based approach. ### Scalability

Figure 12 shows the percentage of consensus obtained with the dynamic optimization engine as we increase the value of \(nQ\). The black bars represent the variation. The results show that with a larger number of images the approaches tend to present a lower consensus. This is more evident for the _DQL_-_O_, _DQL_-_Act_, and _DQL_-_O_-_Act_ approaches, while the percentage of consensus remains high--above 80%--for _IndexF_, _IndexLimit_, _Index_, and _DQL_. Figure 11: Execution time reported by the _Index_ and _DQL_ optimization approaches when processing 10,000 images. We set the Distributed algorithm as the initial routing algorithm with an initial parameter \(H=50\). Figure 13 shows the processing time (_workTime_) obtained for all the dynamic optimization approaches with different numbers of \(nQ\). Again, results show that the _IndexLimit_, _Index_ and _DQL_ report the best results. The _IndexF_, presents a slightly lower performance. More importantly, the processing time reported by the dynamic optimization approaches for the case of 15,000 images is 42% lower than the approach without optimization (None). ### Size of the Control Window

This section discusses the impact of using different time intervals (called control intervals or control windows) to run the dynamic optimization engine. That is, as the crowdsourcing platform processes images, the dynamic optimization algorithm is activated every \(\Delta_{t}\) units of time. We show the results obtained with \(H=30\) and using the Baseline routing algorithm at the beginning of the experiment. In Figure 14, we show the ISP utilization obtained with an interval of \(\Delta_{t}=\{2,4,8,16,32\}\) hours. Figure 15 shows the average consensus obtained and Figure 16 shows the processing time (_workTime_). We show the results for the _IndexLimit_, _Index_, and _DQL_ approaches, since similar trends have been observed for the other variants. Figure 14: Average ISP utilization obtained with different values of the control interval. Figure 12: Percentage of consensus obtained with the dynamic optimization approaches. We set \(H=50\) and the platform received 5000, 10,000, and 15,000 images. Figure 13: Processing time reported by the dynamic optimization approaches. We set the initial value of \(H=50\) and the platform receives 5000, 10,000 and 15,000 images. The results show that a larger \(\Delta_{t}\) tends to increase the IPS utilization (Figure 14). This is mainly because as we increase the time in which the changes are made in the parameters, it takes more time to correctly adjust the values of the parameters.