## 6 Numerical Evaluation
### Comparative Evaluation  
In this section, a comprehensive comparative evaluation is conducted following the baseline scenario in terms of the number of crowdsourcers and crowdworkers for the proposed CROWDMATCH mechanism in comparison to three alternative models: (i) Highest Reward--wherein crowdworkers select crowdsourcers offering the highest rewards until their capacity is reached, subsequently moving to the next most rewarding crowdsourcer [35]; (ii) Random Matching--entailing a random allocation of crowdworkers to crowdsourcers; and (iii) Stochastic Learning Automata (SLA)--where the crowdworkers select a crowdsourcer following a reinforcement learning approach, based on the following probabilistic rules:  
\[Pr_{\mathtt{n,m}}(ite+1)=Pr_{\mathtt{n,m}}(ite)+b\cdot r_{n}^{m(ite)}\cdot(1 -Pr_{\mathtt{n,m}}(ite)),\quad m^{(ite+1)}=m^{(ite)} \tag{7a}\] \[Pr_{\mathtt{n,m}}(ite+1)=Pr_{\mathtt{n,m}}(ite)-b\cdot r_{n}^{m(ite)}\cdot Pr _{\mathtt{n,m}}(ite),\quad m^{(ite+1)}\
eq m^{(ite)} \tag{7b}\]  
Figure 4: Scalability analysis for an increasing number of crowdworkers: (**a**) execution time, and (**b**) average crowdworkers’ utility under the Approximate and Accurate CROWDMATCH algorithms.  
Figure 5: Scalability analysis for an increasing number of crowdsourcers: (**a**) execution time, and (**b**) average crowdworkers’ utility under the Approximate and Accurate CROWDMATCH Algorithms.  
where \(t_{n}^{m(ite)}=\frac{r_{n}^{m(ite)}}{\max\{r_{n}^{m(ite)}\}}\) denotes the normalized reward, where  
\[r_{n}^{m(ite)}=\begin{cases}\frac{P_{n}}{N_{m}^{(ite-1)}C_{m}}&\text{if }N_{m}^{(ite-1)}<N_{m}^{max}\\ 0&\text{if }N_{m}^{(ite-1)}=N_{m}^{max}\end{cases} \tag{8}\]  
denotes the reward of crowdworker \(n\) after selecting crowdsourcer \(m\), \(b\in(0,1)\) (in this simulation, \(b=0.8\)) denotes the learning rate and \(ite\) is the SLA algorithm's iteration index [36; 37]. Figure 6a,b depicts the utility of crowdsourcers based on their ID as well as their average utility and the average utility of crowdworkers across various matching mechanisms, respectively. The results indicate that, under the Highest Reward model, the crowdworkers tend to favor crowdsourcers with the highest rewards, resulting in diminished utility for crowdsourcers offering lower rewards. Moreover, the Random Matching model yields inferior results compared to the CROWDMATCH mechanism, attributable to its simplistic approach in matching crowdworkers with crowdsourcers. The SLA scenario achieves better results for the crowdsourcers compared to the Highest Reward and Random Matching models; however, it still performs worse than the CROWDMATCH mechanism.  
When examining the utility achieved by crowdworkers, the CROWDMATCH mechanism outperforms the Highest Reward, Random Matching, and SLA models. This is derived from the observation that, under the Highest Reward model, the crowdworkers with greater information availability disproportionately select crowdsourcers that offer the highest rewards, thereby leaving other crowdworkers with suboptimal utility. Conversely, the Random Matching model achieves a mix of selections from crowdworkers with varying information availability, resulting in better outcomes compared to the Highest Reward model. In contrast, the proposed CROWDMATCH mechanism achieves superior average utility for crowdworkers, even compared to the SLA reinforcement learning-based model, as it strategically considers both crowdworkers' and crowdsourcers' characteristics. The CROWDMATCH approach enables a more effective and balanced allocation, enhancing the overall utility for both parties involved. Considering the complexity of the proposed models, the execution time of the CROWDMATCH mechanism was on the order of magnitude of 56 ms, while the execution time of the SLA model was double. The Highest Reward and Random Matching models had negligible execution time due to their non-iterative nature.