# A Multi-Head LSTM Architecture for Bankruptcy Prediction with Time Series Accounting Data
## 10 Final Analysis on the Test Set
### False-Positive Analysis  
In bankruptcy prediction, an error has a different cost depending on the class that has been incorrectly predicted. The cost of predicting a company going into default as healthy (FN) is much higher than the cost of predicting a healthy one as bankrupt (FP). Both networks achieved a small number of false negatives but a considerable number of false positives that affected the performance. Moreover, in light of the evidence presented in the previous section about the different levels of overlap between bankrupt and healthy firms achieved by the multi-head and single-input LSTM models, we decided to further analyze the latent representation (embedding) of the false positives achieved by both networks to better prove the benefits introduced by the multi-head LSTM architecture.  
For this analysis, we again leveraged T-SNE dimensionality reduction to display the distribution of false positives. The results are depicted in Figure 11 for window lengths of 4 and 5 years.  
Examining the embedding obtained from both models, one may observe how the false positives were close to the bankruptcy clusters presented in the previous section, presenting a considerably smaller overlap ratio with healthy firms in the case of multi-head LSTM.  
Furthermore, to complete our analysis, we considered how the false positives were distributed across the financial industries since economic conditions and market dynamics can significantly impact the behavior of businesses within a particular industry. With this aim, we leveraged the SIC codes presented in Section 4 to understand if industry-specific factors can contribute to misclassification.  
Table 7 presents the false positives for both models across different SIC divisions under different test set conditions (WL = 4 and WL = 5), as well as the respective percentages. We could make the following observations:  
Figure 11: Visualization of false positives using T-SNE: false-positive distribution in a high-dimensional feature space visualized using T-SNE. Subfigure (**a**) represents the embeddings from single-input LSTM, and (**b**) shows the embeddings from multi-head LSTM.  
* Across all divisions, the multi-head model outperformed LSTM for both window lengths in terms of the number of false positives. This shows how much better multi-head LSTM was in classifying and identifying alive and bankrupt firms.
* Examining the variation in false positives with an emphasis on window length revealed an interesting pattern. When considering multi-head LSTM, the greater the window length, the lower the percentage of false positives. This can be explained by multi-head LSTM's capacity to identify structures and patterns that could be missed when the window was set to 4 years. Conversely, since single-input LSTM's performance had more variance, no definitive conclusions could be drawn about it, making it difficult to understand how the false positives changed across various time windows. This is because there were equal numbers of positive and negative variations.  
These observations regarding the sectors and spatial distribution of false positives provide important guidance for improving model parameters, maximizing predictive precision, and eventually improving our models' applicability in bankruptcy prediction tasks.