Reward Crowdfunding Success Forecasting: An Empirical Evaluation of Machine Learning Algorithms

Reward-based crowdfunding by entrepreneurs and small businesses to raise for their creative whose is central to this We offer new empirical evidence from machine learning (ML) models, based on variables related to sentiment on media campaign profile, and geography of the cities where the campaigns are launched. We evaluated fifteen ML models and in particular multiple classifier systems (MCSs), a promising category of algorithms that combine different models. Over 4,000 campaigns hosted on the largest reward crowdfunding platform in one of the ten largest economies in the world are used in the experimental study. Our results show that Meta-DES, which performs dynamic selection, obtains the best overall results among the evaluated models such as artificial neural networks, decision trees, random forest and XGBoost. ABSTRACT Reward-based crowdfunding has been increasingly used by entrepreneurs and small businesses to raise capital for their creative projects, whose success is central to this industry. We oﬀer new empirical evidence from machine learning (ML) models, based on variables related to sentiment on media campaign proﬁle, and geography of the cities where the campaigns are launched. We evaluated ﬁfteen ML models and in particular multiple classiﬁer systems (MCSs), a promising category of algorithms that combine diﬀerent models. Over 4,000 campaigns hosted on the largest reward crowdfunding platform in one of the ten largest economies in the world are used in the experimental study. Our results show that Meta-DES, which performs dynamic selection, obtains the best overall results among the evaluated models such as artiﬁcial neural networks, decision trees, random forest and XGBoost. Furthermore, since usually interpreting the output of ML models is considered to be very diﬃcult due to their complex “black box” architecture, we also use Shapley additive explanations to interpret the outputs of forecasting. Among variables evaluated in our models, including the textual sentiment of the mass media, the number of pledges and the target amount of the campaign deserve to be highlighted when predicting the campaign’s success. crowdfunding campaign success multiple classiﬁer systems that dynamically select the best subset of models for each campaign under analysis. Thus, we compare the performance of single models with static and dynamic multiple classiﬁers systems in the context of predicting crowdfunding campaign success. We perform a comprehensive set of experiments on the Brazilian reward crowdfunding platform Catarse dataset composed of 4,193 campaigns. Fifteen machine learning models are evaluated using three diﬀerent metrics: Accuracy, Area Under the Receiver Operating Characteristic (ROC) Curve, and F1-Score. Additionally, we show that the results obtained by the Meta-DES model (R. M. O. Cruz, Sabourin, Cavalcanti, & Ren, 2015), which is a framework that employs a dynamic selection of the models per test instance, compare favorably to other models. After analyzing the inﬂuence of variables using SHAP, a method to explain the model’s predictions, we have observed that campaigns that attract fewer pledges and set higher revenue targets tend to have more diﬃculty obtaining success. These two variables turn out to be the main predictors of campaign success, although there are other relevant variables such as the city’s gross domestic product per capita, population, and also media sentiment on the campaign launch day.