Corporate Bankruptcy Prediction with Domain-Adapted BERT (2021)

Abstract: This study performs BERT-based analysis, which is a representative contextualized language model, on corporate disclosure data to predict impending bankruptcies. Prior literature on bankruptcy prediction mainly focuses on developing more sophisticated prediction methodologies with financial variables. However, in our study, we focus on improving the quality of input dataset. Specifically, we employ BERT model to perform sentiment analysis on MD&A disclosures. We show that BERT outperforms dictionary-based predictions and Word2Vec-based predictions in terms of adjusted R-square in logistic regression, k-nearest neighbor (kNN-5), and linear kernel support vector machine (SVM). Further, instead of pre-training the BERT model from scratch, we apply self-learning with confidence-based filtering to corporate disclosure data (10-K). We achieve the accuracy rate of 91.56% and demonstrate that the domain adaptation procedure brings a significant improvement in prediction accuracy.

## introduction predicting imminent corporate bankruptcies has been of great importance both in academia and in industry. early studies on bankruptcy prediction focuses on identifying financial variables that precede impending insolvencies. altman (1968) finds out that z-score, a composite measure of several financial variables, predicts imminent insolvencies. since then, numerous papers document additional financial variables that seem to predict bankruptcies @xcite @xcite . among 39 distinct financial variables, @xcite choose seven key variables that effectively predict bankruptcies within 12 months by lasso. however, in contrast to the fact that the majority of * equal contribution. ## conclusion in our study, we examine whether context-specific textual sentiment analysis (bert) improves the accuracy of corporate bankruptcy prediction. we utilize five financial variables calculated from the stock market and annual reports that are known to precede impending insolvencies. further, we collect and examine a large sample of md&a narrative disclosures from 1995 to 2020 to test whether textual sentiment is helpful in predicting financial distress. we find that textual sentiment has additional predictive ability to well-known financial variables. most importantly, we show that bertbased analysis outperforms dictionary-based analysis suggested by @xcite and word2vec-based analysis combined with convolutional neural network. further, we acknowledge the domain shifting problem of current bert model. to assuage such a limitation, we perform domain-adaptation to the existing financial bert model. this approach reduces computational costs compared to pre-training the bert model with a new corpus and, at the same time, significantly improves the prediction accuracy.