# Success Prediction of Crowdsourced Projects for Competitive Crowdsourced Software Development
## 3 Methods
### Evaluation
#### 3.6.5 Results and Discussions
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model Name** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline BERT & 93.75 & 93.76 & 93.75 & 93.75 \\ LSTM & 86.62 & 86.54 & 99.93 & 92.75 \\ CNN & 86.56 & 86.53 & 99.86 & 92.72 \\ GradientBoost & 86.5 & 86.54 & 99.76 & 92.68 \\ XGBoost & 86.21 & 87.91 & 97.27 & 92.36 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of different classifiers (%). contextual relationships and nuanced patterns [53]. Often depending on handcrafted features, these models might misclassify instances not fitting well within their predefined feature space [53]. For instance, in sentiment analysis, such models might base sentiment polarity on simple features like word presence and miss contextual nuances where words convey varied sentiments based on their surrounding context. This oversight could lead to mislabeling instances, impacting the model's recall. On the other hand, BERT, being a contextual language model, considers the entire context of words and sentences due to its bidirectional nature and deep contextual embeddings [32]. It can capture intricate relationships between words and their context, resulting in more accurate classifications and potentially a lower recall than feature-based models. As shown in Figure 6 LSTM and CNN are predicting 92.7%(450) of _unsuccessful (485)_ projects as _successful_. Whereas, BERT is predicting 21.6%(525) of _unsuccessful (2429)_ projects as _successful_. * XGC achieves higher precision than LSTM, CNN, and GBC as shown in Figure 7. This is because it has the ability to handle imbalanced datasets, minimize false positives, and prevent overfitting using regularization techniques. Its gradient-boosting algorithm assigns higher weights to misclassified instances, reducing false positives and improving precision. Impact of Non-Textual Inputs  
To answer the fourth research question, we evaluate the outcomes of the proposed approach by turning off non-textual inputs, e.g., _project ID_ and _prize_. Table 5 illustrates the evaluation outcomes. The first column of the table indicates input settings, while columns 2 to 5 display the accuracy, precision, recall, and F1 score performance metrics. The average precision, recall, and F1 score for the proposed approach with textual and non-textual inputs are 93.75%, 93.76%, 93.75%, and 93.75%, respectively. On the other hand, the average  
Figure 6: Confusion matrices of BERT, LSTM, and CNN. Figure 7: Precision comparison of LSTM, CNN, GBC, and XGC. precision, recall, and F1 scores of the proposed approach by turning off non-textual inputs are 93.66%, 93.67%, 93.66%, and 93.67%, respectively. From Table 5, we draw the following observations:  
* Disabling non-textual features decreases accuracy, precision, recall, and F1 score by 0.10%, 0.10%, 0.10%, and 0.09%, respectively. * Disabling non-text features in the proposed approach resulted in slightly lower scores across all metrics compared to the default settings. This outcome is attributed to BERT's ability to learn meaningful patterns and relationships in numerical and textual data, thanks to its pre-training on diverse data types. Unlike traditional machine learning models, BERT's deep neural network architecture allows it to handle complex relationships. It performs well on a combination of alpha-numeric and textual data. Still, it is unsuitable for handling only numerical data, as it requires a mix of text and non-textual data for optimal performance.