# Success Prediction of Crowdsourced Projects for Competitive Crowdsourced Software Development
## 3 Methods
### Evaluation
#### 3.6.5 Results and Discussions
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Re-Sampling** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline
**No (Default)** & 93.75 & 93.76 & 93.75 & 93.75 \\
**Yes (Under-Sampling)** & 90.26 & 90.27 & 90.26 & 91.28 \\
**Yes (Over-Sampling)** & 96.80 & 96.97 & 96.80 & 96.81 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Importance of resampling (in %). For the LSTM model, we employ a sequential layer and a LSTM layer with 64 units. Dropout is applied with values up to 0.2 and 12 regularization is used. The final output neuron utilized sigmoid activation. The model is trained and tested on the BERT embedding matrix, employing an 80-20% data splitting ratio. It underwent 10 epochs, had a batch size of 32, and a validation split of 0.1. The CNN model consisted of a sequential layer followed by two conventional layers (with filter sizes of 32 and 64), a MaxPooling1D layer (with a pool size of 2), and a dense layer with 64 units. Dropout with a rate of 0.2 is applied. The dense units employed the Rectified Linear Unit (ReLU) activation function, and the final output neuron utilized sigmoid activation. Like the LSTM model, the CNN model is trained and tested on the BERT embedding matrix with an 80-20% data splitting ratio. It underwent 10 epochs, had a batch size of 32, and a validation split of 0.1. The XGC is trained and validated with 1000 estimators, a learning rate of 0.05, and the following settings: max-depth = 6, min-child-weight = 1, gamma = 0, subsample = 0.8, closample-bytree = 0.8, objective = 'binary: logistic', nthread = 4, scale-pos-weight = 1, and seed = 27. The GBC is trained and tested with 300 estimators, a learning rate of 0.05, a max-depth of 5, and a random state of 42. To compare the proposed approach with the machine learning approach, we train and test XGBost and GradientBoost models using an 80-20% data-splitting ratio. The evaluation results of BERT, LSTM, CNN, XGC, and GBC classifiers are summarized in Table 4. Each row represents a specific classifier, while columns 2-5 present the performance metrics of accuracy, precision, recall, and F1 score, respectively. The average accuracy, precision, recall, and F1 score of each classifier (BERT, LSTM, CNN, GBC, and XGC) are (93.75% accuracy, 93.76% precision, 93.75% recall, and 93.75% F1 score), (86.62% accuracy, 86.54% precision, 99.93% recall, and 92.75% F1 score), (86.56% accuracy, 86.53% precision, 99.86% recall, and 92.72% F1 score), (86.5% accuracy, 86.54% precision, 99.76% recall, and 92.68% F1 score), and (86.21% accuracy, 87.91% precision, 97.27% recall, and 92.36% F1 score), respectively. From Table 4 and Figure 6, we draw the following conclusions:  
* The BERT outperforms LSTM, CNN, XGC, and GBC. The possible reason for this performance improvement is that it uses pre-training on a large amount of data, which allows it to learn more meaningful and generalized representations of text data. Additionally, BERT uses attention mechanisms to capture long-range dependencies and contextual information, making it more effective in natural language understanding and processing. * DL models, i.e., BERT, LSTM, and CNN outperform ML models, i.e., XGC and GBC. The possible reason for this improvement is that DL models can learn complex and hierarchical representations of data, leading to improved performance in tasks involving natural language understanding and processing. Additionally, DL models can handle large amounts of data and automatically learn relevant features from the data, eliminating the need for feature engineering. * Recall of BERT is lower than other models. This is because feature-based M/DL models have a higher false positive rate that can also lead to a higher recall rate, where these models may identify some negative cases as positive. One of the key explanations for this behavior is that feature-based models, which depend on pre-defined and engineered features, may encounter challenges in effectively capturing intricate