# A Multi-Head LSTM Architecture for Bankruptcy Prediction with Time Series Accounting Data
## 11 Conclusions  
In this paper, we proposed a multi-head LSTM neural network to assess corporate bankruptcy. According to the experimental analysis on the test set, this model outperformed single-input LSTM with the same hyper-parameters and architecture, as well as the other traditional ML models. The better forecasting performance of multi-head LSTM also proved that modeling each accounting time series independently with an Attention head contributes to better-identifying companies that are likely to face default events (highest recall and lowest type II error). This was also evident in the analysis of false positives presented in the experimental section. Moreover, we can finally argue that using accounting data for the four most recent fiscal years leads to better performance when predicting the likelihood of corporate distress. Experiments were conducted on a dataset composed of accounting variables from 8262 different American companies over the period 1999-2018 for a total of 78,682 firm-year observations. This dataset has been made public so that it can be used as a benchmark for future studies. Future developments will involve exploiting textual disclosures from financial reports in conjunction with this model. Moreover, it should be possible to predict defaults only in specific sectors by adding macroeconomic variables such as sustainability, interest rates, sovereign risk, and credit spread. Furthermore, our models provide predictions over a single period, not the survival probabilities over time. In future work, multi-period models can be incorporated to also reach this goal. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Division**} & \multicolumn{2}{c}{**Test Set**} & \multicolumn{2}{c}{**LSTM (False Positive)**} & \multicolumn{2}{c}{**Multi-Head LSTM (False Positive)**} \\ \cline{2-7}  & **WL = 4** & **WL = 5** & **WL = 4** & **WL = 5** & **WL = 4** & **WL = 5** \\ \hline A & 17 & 15 & 12 (70\%) & 9 (60\%) & 4 (23\%) & 3 (20\%) \\ B & 152 & 142 & 84 (55\%) & 79 (52\%) & 34 (22\%) & 29 (20\%) \\ C & 30 & 28 & 13 (43\%) & 14 (50\%) & 5 (17\%) & 6 (21\%) \\ D & 1476 & 1382 & 824 (55\%) & 790 (57\%) & 326 (22\%) & 284 (21\%) \\ E & 254 & 248 & 107 (42\%) & 139 (56\%) & 77 (30\%) & 44 (18\%) \\ F & 106 & 99 & 73 (69\%) & 71 (72\%) & 34 (32\%) & 28 (28\%) \\ G & 213 & 202 & 134 (62\%) & 131 (65\%) & 104 (49\%) & 95 (47\%) \\ H & 66 & 63 & 34 (52\%) & 32 (51\%) & 12 (6\%) & 5 (8\%) \\ I & 606 & 556 & 310 (51\%) & 308 (55\%) & 70 (11\%) & 65 (12\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Division distribution in the test set for each model. This table provides an overview of the distribution of companies across various divisions within the test set, including the total number of companies belonging to each category. However, for LSTM and multi-head LSTM models, only instances of false positives are reported, along with their distribution among different divisions. Additionally, percentages are presented to express the ratio of false positives in relation to the total data present in the test set. **Author Contributions:** Conceptualization & Software, M.P. ; Conceptualization & Software, G.L. ; Methodology, G.A. ; Supervision, P.M.P. and A.P. ; Writing-review & editing, S.C. All authors have read and agreed to the published version of the manuscript. **Funding:** This research received no external funding. **Data Availability Statement:** The data presented in this study are publicly available on GitHub under the CC-BY-4.0 license. Source code: [[https://github.com/sowide/Multi-head_LSTM_for_bankruptcy-prediction](https://github.com/sowide/Multi-head_LSTM_for_bankruptcy-prediction)]([https://github.com/sowide/Multi-head_LSTM_for_bankruptcy-prediction](https://github.com/sowide/Multi-head_LSTM_for_bankruptcy-prediction)), accessed on 1 February 2024.