EDGAR - CORPUS : Billions of Tokens Make The World Go Round (2021)

Abstract: We release EDGAR-CORPUS, a novel corpus comprising annual reports from all the publicly traded companies in the US spanning a period of more than 25 years. To the best of our knowledge, EDGAR-CORPUS is the largest financial NLP corpus available to date. All the reports are downloaded, split into their corresponding items (sections), and provided in a clean, easy-to-use JSON format. We use EDGAR-CORPUS to train and release EDGAR-W2V, which are WORD2VEC embeddings for the financial domain. We employ these embeddings in a battery of financial NLP tasks and showcase their superiority over generic GloVe embeddings and other existing financial word embeddings. We also open-source EDGAR-CRAWLER, a toolkit that facilitates downloading and extracting future annual reports.

## introduction natural language processing (nlp) for economics and finance is a rapidly developing research area (hahn et al., 2018, 2019; chen et al., 2020; el-haj et al., 2020). while financial data are usually reported in tables, much valuable information also lies in text. a prominent source of such textual data is the electronic data gathering, analysis, and retrieval system (edgar) from the us securities and exchange (sec) website that hosts filings of publicly traded companies. @xcite in order to maintain transparency and regulate exchanges, sec requires all public companies to periodically upload various reports, describing their financial status, as well as important events like acquisitions and bankruptcy. financial documents from edgar have been useful in a variety of tasks such as stock price prediction (lee et al., 2014), risk analysis (kogan et al., 2009), financial distress prediction (gandhi et al., correspondence: eleftherios.loukas@ey.com, eleftheriosloukas@aueb.gr in this paper, we release edgar-corpus, a novel financial corpus containing all the us annual reports (10-k filings) from 1993 to 2020. foot_0 each report is provided in an easy-to-use json format containing all 20 sections and subsections (items) of a sec annual report; different items provide useful information for different tasks in financial nlp. to the best of our knowledge, edgar-corpus is the largest publicly available financial corpus ( ## experiments on financial nlp tasks we also compare edgar-w2v embeddings against generic glove (pennington et al., 2014) embeddings 9 and the financial embeddings of tsai et al. (2016) in three financial nlp tasks. for each task, we use the same model and we only alter the embeddings component. in addition, we use the same pre-processing during the creation of the vocabulary of the embeddings in each case. financial tagging (fint) is an in-house sequence labeling problem for financial documents. the task is to annotate financial reports with word-level tags from an accounting taxonomy. to tackle the problem, we use a bilstm encoder operating over word embeddings with a shared multinomial logistic regression that predicts the correct tag for each word from the corresponding bilstm state. we report the f1 score micro-averaged across all tags. for this problem, we employ a bilstm encoder which operates over word embeddings, and a linear regressor operating over the last hidden state of the @xcite we use the 200-dimensional glove embeddings from https://nlp.stanford.edu/data/glove.6b.zip . 10 https://spec.edmcouncil.org/fibo/ . ## conclusions and future work we introduced and released edgar-corpus, a novel nlp corpus for the financial domain. to the best of our knowledge, edgar-corpus is the largest financial corpus available. it contains textual data from annual reports published in edgar, the repository for all us publicly traded companies, covering a period of more than 25 years. all the reports are split into their corresponding items (sections) and are provided in a clean, easy-to-use json format. we also released edgar-crawler, a toolkit for downloading and extracting the reports. to showcase the impact of edgar-corpus, we used it to train and release edgar-w2v, which are financial word2vec embeddings. after illustrating the quality of edgar-w2v embeddings, we also showed their usefulness in three financial nlp tasks, where they outperformed generic glove embeddings and other financial embeddings. in future work, we plan to extend edgarcrawler to support additional types of documents (e.g., 10-q, 8-k) and to leverage edgarcorpus to explore transfer learning for the financial domain, which is vastly understudied.