# A System Design Perspective for Business Growth in a Crowdsourced Data Labeling Practice
## 3 Payment Mechanism
### Credit Function Payment
#### 3.2.3 Solution to RQ3 (Preliminary)  
Addressing the third concern C3 involves tuning the shape parameter (\(T\)), particularly when facing large-scale datasets. Table 3 displays the trends displayed by the credit values of small- and large-scale instances under different configurations. The data are sorted in terms of correctly responding to a single, half, and the whole number of golden items, absent of any incorrect label (\(g_{Incorrect}^{+,-}=0\)). Consider the example of [10], who set \(T=0.5\), for a total of 10 questions (\(N=10\)) where 30% are golden (\(Q=3\)). Assume now that the maximum and minimum credits equal 80 (\(\mu_{\text{max}}=80\)) and 0 (\(\mu_{\text{min}}=0\)) units, respectively. Within this framework, the credit obtained by users would equal 20, 40, and 80 after answering 1, 2, and 3 golden questions correctly, respectively. Next, consider a larger dataset with 1000 questions. Applying the same proportions of the previous example, the parameters would be given by \(Q=300\), \(\mu_{\text{max}}=8000\), and \(\mu_{\text{min}}=0\). However, selecting a proportional shape parameter would violate the maximum threshold value of 1. By keeping the shape parameter value unchanged (e.g., \(T=0.5\)), the income obtained by users when correctly submitting 1 and 150 golden items is negligible. There is also a substantial income jump when answering the last golden question correctly. The resulting credit distribution would lessen the reliability of the system from the perspective of users. Most individuals would dismiss the slow credit increments obtained despite responding correctly to a considerable number of questions. An increase in the shape parameter from 0.5 to 0.95 would still deliver credit values that remain insufficient to stimulate users. The output of the credit function becomes more meaningful as \(T\) rises to 0.98. After this value, even marginal decimal increments (e.g., 0.985) would significantly increase the outputs. The shape parameter of the formula requires tuning via sensitivity analysis to yield a work \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Parameter Setting**} & \multicolumn{8}{c}{\(\boldsymbol{Q=3}\), \(\boldsymbol{\mu_{\text{max}}=80}\), \(\boldsymbol{Q=300}\), \(\boldsymbol{\mu_{\text{max}}=8000}\)} \\ \cline{2-10}  & \multicolumn{2}{c}{\(\boldsymbol{T=0.5}\)} & \multicolumn{2}{c}{\(\boldsymbol{T=0.5}\)} & \multicolumn{2}{c}{\(\boldsymbol{T=0.95}\)} & \multicolumn{2}{c}{\(\boldsymbol{T=0.98}\)} \\ \hline \multirow{4}{*}{\(g_{Incorrect}^{+,-}=0\)} & \multirow{2}{*}{\(\boldsymbol{S}_{Correct}^{+,-}\)} & \multirow{2}{*}{**Value**} & \multirow{2}{*}{\(g_{Correct}^{+,-}\)} & \multirow{2}{*}{**Value**} & \multirow{2}{*}{\(g_{Correct}^{+,-}\)} & \multirow{2}{*}{**Value**} & \multirow{2}{*}{\(g_{Correct}^{+,-}\)} & \multirow{2}{*}{**Value**} \\  & & & & & & & & & \\ \cline{1-1} \cline{6-10}  & \(\boldsymbol{1}\) & 20 & \(\boldsymbol{1}\) & \(7.85\times 10^{-87}\) & \(\boldsymbol{1}\) & 0.002 & \(\boldsymbol{1}\) & 19.04 & \(\boldsymbol{1}\) & 87.20 \\ \cline{1-1}  & \(\boldsymbol{2}\) & 40 & \(\boldsymbol{150}\) & \(5.61\times 10^{-42}\) & \(\boldsymbol{150}\) & 3.64 & \(\boldsymbol{150}\) & 386.37 & \(\boldsymbol{150}\) & 828.94 \\ \cline{1-1}  & \(\boldsymbol{3}\) & 80 & \(\boldsymbol{300}\) & 8000 & \(\boldsymbol{300}\) & 8000 & \(\boldsymbol{300}\) & 8000 & \(\boldsymbol{300}\) & 8000 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Credit values of small- and large-scale instances. However, as the size of the dataset and the number of golden items increases, sensitivity analysis is insufficient to manage the payments in a real labeling system. Let \(N=1000\), \(Q=300\), \(\mu_{\max}=8000\), and \(T=0.985\). Consider the case with 1000 potential contributors. Assume now that users label a single question and leave the system after collecting the credit. How much will the system pay to this type of user? Any of these users facing a golden item by chance will receive 87.2 units when answering correctly.