News Risk Alerting System ( NRAS ): A Data-Driven LLM Approach to Proactive Credit Risk Monitoring (2024)

Abstract: Credit risk monitoring is an essential process for financial institutions to evaluate the creditworthiness of borrowing entities and minimize potential losses. Traditionally, this involves the periodic assessment of news regarding client companies to identify events which can impact their financial standing. This process can prove arduous and delay a timely response to credit impacting events. The News Risk Alerting System (NRAS) proactively identifies credit-relevant news related to clients and alerts the relevant Credit Officer (CO). This production system has been deployed for nearly three years and has alerted COs to over 2700 credit-relevant events with an estimated precision of 77%.

## introduction credit risk management is a systemic process to evaluate and monitor the solvency of borrowing entities, allowing financial institutions to understand and detect emerging risks. previous algorithmic approaches for credit risk have focused on financial assessment, generally using a company's financial statements @xcite . however, these financial reports are produced relatively infrequently and often lack wider commercial context, so monitoring credit-impacting news events is a necessary part of effective credit risk management. as financial institutions often have credit portfolios that contain a multitude of clients, this manual analysis can be labor intensive. leveraging natural language processing (nlp) and machine learning (ml) can help expedite this process. ## system overview nras is comprised of two main components: text analysis and alert generation. this section explains the design decisions and structure of these components. an overview of the system architecture of nras can be seen in figure 2 . ## system development nras has two alerting subsystems, one for sentiment and another for m&a. these subsystems utilize the same underlying architecture, with minimal changes to hyperparameters. this architectural configuration was initially used for the sentiment alerting stream, but proved flexible enough to add a new m&a alerting stream with minimal modifications. this section discusses the experiments performed for the parameter selection of each subsystem. parameters were selected to prioritize precision, but with a secondary consideration for the number of alerts which was used as a proxy for real-world recall. ## real-world evaluation each alert produced by nras is reviewed by cos, who assess the relevance of the alert to the credit rating and risk review process. the production system incorporates this ongoing performance monitoring and inbuilt end-user feedback to enable continuous system improvements. alerts are categorized using the following five class typology: ## conclusion this paper presents nras, a system to proactively identify and alert for credit adverse news events. it is designed to process thousands of news articles daily, and proactively generate realtime actionable alerts. in the three years that nras has been in production, it has generated 1946 and 781 credit-relevant alerts for sentiment and m&a respectively, of which 509 and 301 were marked as new information. nras automates the news monitoring process, enhancing credit risk management by analyzing large news volumes from a diverse set of sources, including smaller publications. this enables the evaluation of smaller client entities who might not have as much media coverage in major financial publications. the custom dynamic threshold allows for each client entity to be assessed based on their distinct news volume history, which means that entities with lower average news coverage are just as likely to have alerts raised. this paper presents a detailed account of the system's development and evaluation processes, addressing the general lack of public information on how commercial systems are built and assessed. nras is designed to scale to multiple alerting streams without requiring changes to the overall architecture, while its modular design allows for seamless integration of additional components such as filtering of templated headlines, deduplication, and clustering. the individual components of nras, such as sentiment analysis and m&a classification, are based on established techniques, however, the novelty of our work lies in their holistic integration and deployment within a real-world, scalable system. this integration and the system's ability to dynamically adjust thresholds to prevent spurious or duplicative alerts are key innovations that enhance its practical applicability and effectiveness. while nras has been developed for credit risk management, the underlying framework is versatile and adaptable to other domains requiring real-time news monitoring and alerting. this adaptability can be achieved by integrating models which recognize relevant news events in other domains. the system can also extend monitored entities to include individuals or countries, in addition to corporate entities. for instance, specifying the target entity to be a person or location allows nras to generate alerts based on their news volume while utilizing the same underlying mechanisms. this flexibility demonstrates the system's broader applicability beyond just company monitoring for credit risk. concept drift may be a limitation of nras due to an evolved understanding of the problem over time. for example, new types of events may become relevant to cos, and the underlying model should reflect that change. this can only be identified through discussions with smes as with the current real-world evaluation setup it would not be detected. possible future enhancements include: the continuous evaluation and retraining of the underlying models with expanded datasets; increasing the number of alerting streams; as well as integrating new components which perform detailed information extraction. for example, identifying the buyers, sellers, and deal size of an m&a transaction to measure its material impact on the parties involved. additionally, the information presented to cos can be improved by leveraging recent summarization advancements with genai, which can provide more fine-grained information about the cause of an alert. 7 appendices 7.1 text analysis model selection as demonstrated in the tables, the highest performing model for targeted sentiment was bert with 0.136 mae and 0.0312 mse and therefore was our model of choice. similarly, for m&a classification deberta had the highest mean f1 score of 92.9, with a standard deviation of 2.47 over 10 runs.