AMA - LSTM : Pioneering Robust and Fair Financial Audio Analysis for Stock Volatility Prediction (2024)

Abstract: Stock volatility prediction is an important task in the financial industry. Recent multimodal methods have shown advanced results by combining text and audio information, such as earnings calls. However, these multimodal methods have faced two drawbacks. First, they often fail to yield reliable models and overfit the data due to their absorption of stochastic information from the stock market. Moreover, using multimodal models to predict stock volatility suffers from gender bias and lacks an efficient way to eliminate such bias. To address these aforementioned problems, we use adversarial training to generate perturbations that simulate the inherent stochasticity and bias, by creating areas resistant to random information around the input space to improve model robustness and fairness. Our comprehensive experiments on two real-world financial audio datasets reveal that this method exceeds the performance of current state-of-the-art solution. This confirms the value of adversarial training in reducing stochasticity and bias for stock volatility prediction tasks.

## introduction in the stock market, predicting the exact price of a stock is deemed impossible @xcite , yet it is widely accepted within the financial industry that one can forecast a stock's volatility level using publicly available information @xcite . stock price volatility, defined as the standard deviation of a stock's returns over a specific period, serves as a commonly used indicator of financial risk. in the past, research efforts have focused on employing time-series models, using 1 earnings calls are public available and often involve the management team of a public company and interested parties to discuss the company's earnings. 2 code are available at: https://github.com/hao1zhao/ ama-lstm historical stock prices, to predict future volatility @xcite . in recent years, advancements in natural language processing (nlp) and speaker recognition (sr) have opened up new possibilities for the task. for example, researchers leverage novel sources of textual and audio data, including financial news @xcite , financial reports @xcite , social media @xcite , earnings call @xcite and merger and acquisition (m&a) call @xcite to predict stock volatility. these diverse data sources provide a richer, more nuanced understanding of market dynamics and investor sentiment, significantly enhancing the prediction of stock volatility. however, the inherent nature of financial data presents two unique challenges. first, textual data is discrete and stochastic @xcite , and audio signals possess a high temporal resolution @xcite . this characteristic makes both text and audio vulnerable to adversarial examples, which can effortlessly mislead human evaluators @xcite . second, the financial data is biased. research @xcite has delved into various biases within this field, with gender imbalance being a particularly prominent issue. @xcite found that female ceos, despite having comparable credibility, are perceived as less capable of attracting capital. in the context of earnings calls, @xcite observed a significant underrepresentation of female executives. this biased financial audio data is further exacerbated by deep learning models, which tend to amplify variations in audio features due to the scarcity of female training examples, leading to gender bias @xcite . to address the aforementioned issues, we use adversarial training to enhance the robustness and fairness of financial data interpretation models. adversarial training was initially used in computer vision tasks @xcite , and then expanded to nlp @xcite and sr @xcite fields. our method differs from the feature space adversarial training @xcite . it enhances model robustness and reduces output bias by introducing adversarial examples directly from input embeddings, making the approach better aligned with the privacy and proprietary constraints of financial modeling. we have applied our adversarial training method to a multimodal attentive lstm model, which efficiently processes information from two modalities. by introducing perturbations into the input space and dynamically optimizing these perturbations to maximize their impact on the model's output, we enhance the model's ability to perform well on both clean and perturbed data. the adversarial training method effectively guarantees stable training and robust performance of the model when dealing with stochastic and biased financial data. this training approach can also apply to the transformer, but we've found the attentive lstm to be more effective for our scenario. this is because the transformer struggles with temporal information and lacks enough data to learn its many parameters effectively. to the best of our knowledge, this work is the first one to explore the potential of adversarial training in multimodal learning financial audio task. the main contributions of this paper are summarized as: our work is closely related with the following three lines of research: stock volatility with multimedia data : traditional stock volatility prediction method relies on historical pricing data and typically employs both continuous and discrete time-series models @xcite . in addition to stock prices, financial news, analyst reports, earnings reports, and social media have been proven to significantly enhance stock prediction tasks @xcite @xcite @xcite . furthermore, recent multimodal models that employ lstm @xcite , gcn @xcite and transformer @xcite to extract features from earnings calls audio and combine it with textual transcript, have achieved stateof-the-art results in stock volatility prediction. bias in financial audio data : financial audio features can indicate a speaker's emotional and psychological state @xcite . previous research has demonstrated that audio features, such as pitch and intensity, differ significantly across genders @xcite . especially in earnings calls, female executives are highly underrepresented, suresh and guttag (2019) noted that only 5% of fortune-500 ceos are women. moreover, under identical conditions, men are often perceived as more charismatic than female executives @xcite . this disparity in audio features becomes more pronounced in deep learning models due to the scarcity of female training examples, leading to the manifestation of gender bias. ## problem formulation the stock volatility prediction problem is formulated following @xcite . for a given stock with an adjusted closing price p i on trading day i, we calculate the average log volatility over n days following the earnings call as follows: ## adversarial multimodal attentive lstm architecture our adversarial multimodal attentive lstm (ama-lstm) has two parts. we first introduce the architecture of the attentive multimodal model, which operates without adversarial training. then, we specify the attentive lstm block, which contains four components: feature mapping layer, lstm layer, attention layer, and prediction layer. secondly, we thoroughly explain how adversarial training helps multimodal models improve robustness and fairness. ## conclusion our research has shown that neural networks used for predicting stock market volatility often face challenges in robustness, particularly in handling the stochasticity and bias inherent in financial audio data. to tackle this, we introduced an innovative solution: the adversarial multimodal attentive lstm. this method employs adversarial training to more effectively simulate the market noise and biases during model training. the experiments on two benchmark datasets not only validated the effectiveness of our approach but also underscored the importance of considering the stochasticity and bias in financial data for stock volatility prediction tasks. the results further revealed that adversarial training significantly enhances the robustness and fairness of the prediction models.