Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis (2023)

Abstract: Monetary policy pronouncements by Federal Open Market Committee (FOMC) are a major driver of financial market returns. We construct the largest tokenized and annotated dataset of FOMC speeches, meeting minutes, and press conference transcripts in order to understand how monetary policy influences financial markets. In this study, we develop a novel task of hawkish-dovish classification and benchmark various pre-trained language models on the proposed dataset. Using the best-performing model (RoBERTa-large), we construct a measure of monetary policy stance for the FOMC document release days. To evaluate the constructed measure, we study its impact on the treasury market, stock market, and macroeconomic indicators. Our dataset, models, and code are publicly available on Huggingface and GitHub under CC BY-NC 4.0 license.

## introduction on august 26th, 2022, fomc chair jerome h. powell gave an 8-minute long speech at jackson hole which immediately resulted in an almost @xmath06 trillion usd loss in equity valuation over the next 3 days. drastic market shifts to the fed's pronouncements indicate just how important the fomc communications have become and highlight the need for a model which can capture the policy stance from fed-related text. the federal open market committee (fomc) is a federal organization responsible for controlling u.s.'s open market operations and setting interest rates. it tries to achieve its two main objectives of price stability and maximum employment by controlling the money supply in the market. given correspondence to agam shah {ashah482@gatech.edu} 1 the fine-tuned model and data are available on the huggingface. the code is available on fintech lab github. the market condition (employment rate and inflation), the fed either increases (dovish), decreases (hawkish), or maintains the money supply foot_0 (neutral). to understand the influence the fomc has on the different financial markets, we need to extract its monetary policy stance and the corresponding magnitude from official communications. utilizing the traditional sentiment analysis model, which classifies text into positive vs negative, one can't extract policy stance. a sentence that has the word "increase" could either be dovish or hawkish without a clear negative connotation. for example, the word "increase" with the word "employment" means the economy is doing well, but the word "increase" with the word "inflation" is negative for the economy. current sota finance domain-specific language models @xcite trained for sentiment analysis find both cases to be positive, which is inaccurate. the performance analysis for finbert @xcite model is provided in appendix a. this problem creates a need to develop a new task for hawkish vs dovish classification accompanied by high-quality annotated data. given the lack of annotated data, computational linguistic work related to fomc text in the literature @xcite @xcite @xcite @xcite @xcite @xcite @xcite so far has been limited to unsupervised and rule-based models. these rule-based models don't perform well on the hawkish-dovish classification task, which we will use as a baseline in performance analysis. additionally, we conduct a benchmark of the zeroshot chatgpt model using the annotated dataset to gain insights into the significance of fine-tuning on such data. in this work, we collect text data (speech transcripts, press conference transcripts, and meeting minutes) from the fomc over the period 1996-2022 and annotate a sample of each data type. we not only create new datasets and tackle the task of building a hawkish-dovish classifier, but also test the performance of various models starting from rule-based to fine-tuned large plms. as sentences presented in fomc text sometimes have two sub-sentences that have counterfactual information to tone down the stance, we employ a simple sentence-splitting scheme as well. we also construct the aggregate monetary policy stance and show its validity by looking at its performance in predicting various financial market variables. through our work, we contribute to the literature in the following way: the majority of the research takes advantage of news articles @xcite , sec filings @xcite @xcite , or earnings conference calls @xcite @xcite . development of finance domainspecific language models @xcite @xcite have pushed the current benchmarks further. recent work of @xcite proposes a set of heterogeneous benchmarks for the financial domain and shows sota performance using their proposed language model, but it doesn't include macroeconomics-based tasks. fomc and text analysis a study on communications from the central banks of the czech republic, hungary, and poland by @xcite suggests that words from central banks affect the market but the effect varies based on communication style. other various studies @xcite @xcite @xcite @xcite @xcite also point to a similar conclusion that the communication from the central banks moves the market, but they don't leverage the power of the transformer-based model available at their disposal. many articles in the literature use lda to analyze various texts released by fed. @xcite uses lda on the fomc meeting statements to predict the fed fund rate and long-term treasury rate. @xcite use an ldabased topic modeling on fomc-released text to understand how forward guidance affects the market and economic variables. in their study, they only used statements released post-meeting and suggest that the use of meeting minutes and speeches may offer greater insight. @xcite also uses lda to analyze meeting minutes. they suggest that even though meeting minutes are released a few weeks after the actual meeting, the minutes still carry pertinent market-moving information. in recent work by @xcite , they created a multimodal dataset (monopoly) from video press conferences for multimodal financial forecasting. the monopoly dataset is comprehensive and not only covers text but also utilizes audio and video features. yet, it misses two critical economic downturn periods of the last two decades: the dotcom bubble burst of @xcite @xcite @xcite used word embedding to extract semantic changes in the monetary policy documents. @xcite used the graph clustering method to generate the hawkish-dovish stance of monetary policy due to the dearth of annotated data. frunza (2020) developed an unsupervised methodology to extract various information from fomc postmeeting statements. ## results and analysis in this section, we evaluate and benchmark different nlp models on the hawkish vs dovish classification task that we created. for all models and datasets, we used training and testing data based on an 80:20 split. upon this split, we institute another 80:20 split on the training data to generate our final training and validation data. we use the best-performing model (roberta) to generate a document (event) level measure of hawkish tone. we then validate the generated measure by looking at its relation with the inflation indicators and the us treasury. we also look at the performance of a simple trading strategy based on the generated measure. ## conclusion our work contributes a new cleaned, tokenized, and labeled open-source dataset for fomc text analysis of various data categories (meeting minutes, speeches, and press conferences). we also propose a new sequence classification task to classify sentences into different monetary policy stances (hawkish, dovish, and neutral). we show the application of this task by generating a measure from the trained model. we validate the measure by studying its relation with cpi, ppi, and treasury yield. we also propose a simple trading strategy that outperforms the high benchmark set by the qqq index over the last decade. we release our models, code, and benchmark data on hugging face and github. we also note that the trained model for monetary policy stance classification can be used on other fomc-related texts.