## 5 Dynamic Optimization of the Crowdsourcing Platform
### Dynamic Optimization Engine Based on Metric Indices
Alternative Dynamic Optimization Approaches

In the following experiments, we compared the results obtained by our proposed metric space index-based dynamic optimization engine, named Index, with the following approaches:

* IndexLimit: Based on the proposed Index optimization algorithm, but between successive optimization actions the parameter \(H\) changes at most 5 units, the parameter \(TTL\) can vary 24 units, and the parameter \(C\) can vary at most by 5%. * IndexF: Similar to Index, but during situations without stress, the new configuration vector is applied if the metrics do not match a stress situation. * Analytic Ad Hoc Controller: This uses Equation (3) to adjust the value of \(H\). This equation is based on the difference between the average utilization of the ISP (\(ISP\_Util\)) and the utilization requested by the data center engineer (e.g., 40%). \[H=H-30\times(ISP\_Util-0.4) \tag{3}\]

We also compared the effectiveness of the proposed dynamic optimization engine with a deep-reinforcement-learning-based approach. To this end, we trained a deep reinforcement learning model implementing a Deep Q-Learning (DQL) agent. DQL uses a neural network to approximate the state perceived by the agent, thus allowing one to control environments with combinations of complex states and address non-tabular problems [57]. We implemented the optimization technique using a wrapper from the OpenAI Gymnasium library, based on Python. In particular, we implemented the following versions:

* DQL: Deep Q-Learning (DQL) model with Experience Replay, trained with a minimum of 1500 episodes. The model input variables are the same as those presented in Figure 4.

Figure 4.0 Distribution of the arrival of images used in the experiments. * DQL-O: This is similar to DQL, but we added to the state vector the original values of the controllable parameters set by the data center engineer. * DQL-Act: We included in the state vector of the \(DQL\) model the size of the queue of active tasks on the server. In this way, we evaluated the hypothesis that knowing the number of active tasks can allow the agent to more adequately estimate the server workload. * DQL-O-Act: This combines the \(DQL\)-\(O\) and \(DQL\)-\(Act\) approaches. The state vector is built with the original values of the controllable parameters and with the size of the list of active tasks of the server. We defined the agent's reward according to Equations (4) and (5). PC is the percentage of consensus achieved. \(\Delta\) is the difference between the current and the original values of the parameters. For the \(H\) parameter, \(\Delta=|H_{o}-H_{c}|\) is the difference between the original value of \(H\) (\(H_{o}\)) and its current value (\(H_{c}\)). The same is applied for the \(TTL\) and the consensus threshold \(C\). For the routing algorithm, we used the values \(0=\) Baseline, \(1=\) Centralized, and \(2=\) Distributed. One the one hand, under stressful situations, the reward receives a penalty equal to one negative unit, and also receives a reward equal to the percentage of consensus obtained. In this way, the agent is induced to escape the stress situation, also favoring a high level of consensus. On the other hand, in situations without stress, the agent is granted a reward equal to the percentage of consensus obtained and a penalty proportional to the difference that the controllable parameters have with respect to the values that were initially configured by the data center engineer. Thus, the agent will try to bring the platform to an operating point similar to the one that was initially configured, but will try to keep the percentage of consensus high. Figure 4.1

Figure 4.2

## 7 Results

### Effectiveness Evaluation

First, we evaluated the impact of the controllable parameters on the different index-based dynamic optimization approaches. Table 1 shows the values of the fixed parameters, that is, the fixed parameter values which were set while varying the value of a specific parameter.