Dynamic Graph Transformer for Implicit Tag Recognition (2021)

Abstract: Textual information extraction is a typical research topic in the NLP community. Several NLP tasks such as named entity recognition and relation extraction between entities have been well-studied in previous work. However, few works pay their attention to the implicit information. For example, a financial news article mentioned “Apple Inc.” may be also related to Samsung, even though Samsung is not explicitly mentioned in this article. This work presents a novel dynamic graph transformer that distills the textual information and the entity relations on the fly. Experimental results confirm the effectiveness of our approach to implicit tag recognition.

## introduction documents on the web deliver and spread lots of most recent information to people worldwide. in order to automatically update the real-world information, textual information extraction is a fundamental issue for nlp researchers. many downstream tasks such as fake news detection @xcite and stock movement prediction @xcite can benefit from the extracted information. however, most of the previous works @xcite focus on using explicit information in articles and do not consider the implicit information. for example, two companies, sprint and t-mobile, are explicitly mentioned companies in the news article in figure 1 . however, the stock price of a non-mentioned company, softbank, may be influenced since softbank owns shares of sprint. although this kind of inference is intuitive for professional analysts, few previous works take such implicit information into consideration. in this paper, we aim to increase the sense of machines toward this kind of implicit information. transformer-based @xcite neural networks achieve state-of-the-art performances in title: the judge nodded! sprint rise 70% in after-hour trading. ## related work extracting and using the information in articles is one of the focuses in the nlp community. some works @xcite @xcite adopt the extracted information for stock market prediction. some of them @xcite use the information in news articles to construct socio-economic indicators. most of previous works only focus on explicit information in the articles. in this way, the implicit entities in the articles may be under looked. in this paper, we aim to extract the non-mentioned but related entities from a document. recently, gnn has become popular for modeling relationships among multiple entities. kipf and welling (2016) use the convolution neural network to learn the node representation by aggregating the features of neighboring nodes. @xcite employ the attention mechanism to improve the gnn architecture. recent studies @xcite @xcite also show the effectiveness of graph neural networks in various tasks. inspired by these works, we present a new blend of graph attention network (gat) @xcite with transformer @xcite and bert @xcite for extracting the implicitly related entities to a given article. ## conclusion and future work this paper presents a novel dynamic graph transformer model and a pre-training task for extracting the implicit entities in articles. experimental re-sults show the usefulness of the proposed methods. we also discuss what kinds of features our model captures. in our previous work @xcite , we apply the proposed task to accelerate the working process of the journalists and show that using the extracted entities could be useful for downstream tasks such as news aggregation and stock movement prediction. in the future, we plan to apply the proposed approach to datasets with both graphical knowledge and textual content.