# Machine Learning for Bankruptcy Prediction in the American Stock Market: Dataset and Benchmarks
## 3 Dataset  
Since most of the bankruptcy models are evaluated on private datasets or small publicly available ones, we provide a novel dataset for bankruptcy prediction related to the public companies in the American stock market (New York Stock Exchange and NASDAQ). We collected accounting data from 8262 different companies in the period between 1999 and 2018. The stock market is dynamic with new companies becoming public every year, changing properties and names, or being removed or suspended from the market as a result of acquisitions or regulatory action. For this reason, we consider the same companies used in [6; 37] since this set of firms has been proved to be a fair approximation of the American stock market for each year in that time interval. According to the Security Exchange Commission (SEC), a company in the American market is considered bankrupt in two cases:  
* If the firm's management files Chapter 11 of the Bankruptcy Code to \"reorganize\" its business: management continues to run the day-to-day business operations but all significant business decisions must be approved by a bankruptcy court. * If the firm's management files Chapter 7 of the Bankruptcy Code: the company stops all operations and goes completely out of business. When these events occur we label the fiscal year before the chapter filing as \"bankruptcy\" (1) for the next year. Otherwise, the company is considered healthy (0). In light of this, our dataset enables learning how to predict bankruptcy at least one year before it happens and, as a consequence, it is possible to deal with the default prediction task using time series and also to deal with the survival probability task looking ahead. Figure 1 shows the percentage of companies' default for each year in the dataset. This value can be underestimated due to the exclusion of some companies in the past because of their small market capitalization, but it appears to agree with the literature that usually reports that only a percentage below 1% of the available firms in the market is likely to default every year under normal conditions. However, in some periods, bankruptcy rates have been higher than usual, for example during the Dot-com Bubble in the early 2000s and the Great Recession in 2007-2008. Our dataset distribution reflects this condition, see Table 1.  
For all the companies and for each year, we selected 18 accounting and financial variables. Features were selected according to the most frequently used ratios, and accounting information to which the literature refers [21; 23; 38]. The dataset has no missing values or synthetic and imputed added values. Finally, the resulting dataset of 78,682 firm-year observations is divided into three subsets according to the period of time: a training set, a validation set, and a test set. We used data from 1999 until 2011 for training, data from 2012 until 2014 for validation and model comparison, and the remaining years from 2015 to 2018 as a test-set to prove the ability of the models to predict bankruptcy in real never seen cases and macroeconomic conditions. Table 2 shows the full list of the 18 features available in the dataset and their description. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Year** & **Total Firms** & **Bankrupt Firms** & **Year** & **Total Firms** & **Bankrupt Firms** \\ \hline
2000 & 5308 & 3 & 2010 & 3743 & 23 \\
2001 & 5226 & 7 & 2011 & 3625 & 35 \\
2002 & 4897 & 10 & 2012 & 3513 & 25 \\
2003 & 4651 & 17 & 2013 & 3485 & 26 \\
2004 & 4417 & 29 & 2014 & 3484 & 28 \\
2005 & 4348 & 46 & 2015 & 3504 & 33 \\
2006 & 4205 & 40 & 2016 & 3354 & 33 \\
2007 & 4128 & 51 & 2017 & 3191 & 29 \\
2008 & 4009 & 59 & 2018 & 3014 & 21 \\
2009 & 3857 & 58 & 2019 & 2723 & 36 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The table provides the firm distribution by year in the dataset. Figure 1: Rate of bankruptcy in the dataset (2000–2019) with financial variables in the period (1999–2018). The next subdivision in training, validation, and test is highlighted with different colors.