## 5 Dynamic Optimization of the Crowdsourcing Platform
### Dynamic Optimization Engine Based on Metric Indices
\[D1(q,u)=\sum_{p\in P}(|d(q,p)-d(u,p)|) \tag{1}\]
```
1:Procedure SEARCH_NN (Input: _Index_, _q_n_pivots,k_)
2:D_q_P[n_pivots] = \(\O\)
3:candidates[K] = \(\O\)
4:D1_q_U[Index_n_vectors] = \(\O\)
5:forp \(\in\) Index_piv do
6:D_q_P = D(q, p);
7:Insert_candidate (candidates, p, D_q_P)
8:endfor
9:foru \(\in\) (Index.obj \(-\) Index.piv) do
10:D1 = 0
11:forp \(\in\) Index.piv do
12:D1 = D1 + 1D(q, p) \(-\) Index.Table[u,p] |
13:endfor
14:D1_q_U[u] = D1
15:endfor
16:sort_ascending (D1_q_U)
17:Dmax
18:foru \(\in\) D1_q_U do
19:Dmax = \(D_{\infty}(q,u)\)
20:if (not candidates.complete) or (Dmax \(<\) lastElement(candidates).D) then
21:D_q_u = D(q, u)
22:Insert_candidate (candidates, u, D_q_u)
23:endif
24:endfor
```

**Algorithm 4** Top-\(k\) search algorithm for a pivot-based index. The estimated distances \(D1(q,u)\) and the vectors \(u\) are stored in the array \(D1\_q\_U\) (line 14), and then the array is sorted in ascending order according to the distances estimated (line 16). Finally, for all index objects that are not pivots, we calculate \(D_{\infty}(q,u)\) according to Equation (2) in line 18. \[D_{\infty}(q,u)=\arg\max_{p}|d(q,p)-d(u,p)| \tag{2}\]

If the candidate array has less than \(k\) elements, we insert the new vector as a candidate. If the candidate array is complete, we evaluate whether the estimated distance \(D_{\infty}(q,u)\) is greater than the distance from the farthest candidate to \(q\). If so, we calculate the distance between the input vector \(q\) and \(u\) and the vector \(u\) is inserted as a candidate in the corresponding position (line 22). At the end of the algorithm, the _candidates_ array has the \(k\) nearest neighbors to \(q\). ## 6 Experimental Settings

We simulated the crowdsourcing platform using the LibCppSim library [44]. We set the image size as the average taken from aerial images in DOTA [45] and NWPU VHR-10 [46] and the natural disaster Twitter databases CrisisMMD [47]. The arrival rate distribution was modeled with an envelope described by a Weibull distribution (similar to [47; 48; 49; 50; 51]). It includes the working intervals of the volunteers and also intervals during which user activity decreases, related to night-time hours. Figure 7 shows an example of the distribution. The \(x\)-axis shows the time advance in hours and the \(y\)-axis shows the number of incoming images. We implemented the distribution described in [52] to represent the processing time of each volunteer. According to [52], the average classification time of each volunteer is 1.76 s. Similar results were presented by [53] to classify tweets. We ran experiments with \(nQ=10,000\) incoming images to be processed. We set the number of volunteers receiving an image as \(H=\{30,50\}\) and the network size was set to 500 peers, which is an intermediate community size between those proposed in [52; 54]. We modeled the network latency and network transfer rate according to [55]. Thus, we set the average communication delay inside the P2P network to 228.2 ms, the average communication delay between the P2P and the server to 441.4 ms, the average transfer rate inside the P2P network to 8.67 MBps, and the average transfer rate between the server and the P2P network to 5.3 MBps. In the following experiments, we simulate the execution of the platform using experts and non-expert volunteers. Expert volunteers are those with skills obtained in previous crowdsourcing campaigns and who have been trained to correctly classify the images. In this work, we set to 50% the ability of non-expert volunteers to correctly classify the images and that to 90% for experts. A similar setting was used in [56].