###### Abstract
The biggest highlight of GPT is that it is different from ELMo's bidirectional LSTM model, but uses a Transformer encoder [11] that can capture long-distance information. When we use GPT, the model can be supervised and fine-tuned according to specific downstream tasks. Coincidentally, the BERT model proposed by Google combines the advantages of ELMo and GPT, uses the Transformer encoder while adding a bidirectional structure, and proposes two new natural language processing target tasks, MLM (Masked Language Model) and the task of predicting the next sentence. These pre-trained language models refreshed the list of natural language processing tasks as soon as they were proposed. After fine-tuning the pre-trained language models, researchers can adapt to their respective downstream tasks. Of course, the pre-trained language model also has some shortcomings. During pre-training, due to the need to grasp the global semantic information, the training overhead is usually the square of the training text length. Therefore, the aforementioned pre-trained language models usually limit the text length to 512. Many researchers use news text to construct news knowledge graphs and use graphs to judge news development trends. At the same time, other researchers use pre-trained language models to fine-tune to achieve news text classification and development trend prediction. The method of using news knowledge graphs can handle long texts, but the effect is relatively poor; the method of using pre-trained language models has good results, but it is incapable of processing long texts. What's more, few researchers study the development trend of Chinese news texts. The lack of a fully-labelled database and technical obstacles make it an unsolved problem for researchers to use Chinese news texts to predict news development trends. To better solve the above problems, we have made two efforts. We first used web crawler technology to build a database of Chinese news text trends. This database contains nearly 9000 labelled news data, including news time, news headlines, news sources, news text, and news trend labels, which fill the gaps in the current analysis and trend prediction database of Chinese news texts. Secondly, we designed a news text trend prediction method that uses knowledge extraction technology, graph convolutional neural networks, and pre-trained language models. We use the open-source code on GitHub named TextGrapher [12] to generate a homogenous knowledge graph of news text. The open-source code uses the TF-IDF algorithm to extract keywords and high-frequency words of the news text and uses named entity recognition and knowledge extraction to extract ternary relationship groups and build news knowledge graphs. At the same time, we use the Chinese pre-trained language model of HFL (Joint Laboratory of Harbin Institute of Technology and iFLYTEK Research) [13] to obtain the embedding vector of the named entity as the initial value of the node of the graph convolutional neural network and use the graph convolutional neural network to extract the entire graph feature and design the graph-level task. At the same time, the graph pooling algorithm is improved to make the graph embedding vector better retain the hierarchical features, and the convolved graph embedding vector can be used for comparison and prediction of news trends. The experimental results show that the effect of this method compared with the baseline graph classification algorithm is improved, and it also solves the shortcomings of the pre-trained language model, which cannot handle very long texts.