[PERSON]  
1 Key Laboratory in Media Convergence of Yunnan Province, Kunming 650504, China; [EMAIL_ADDRESS] (Z.Z. ); [EMAIL_ADDRESS] (L.X.) School of Software, Yunnan University, Kunming 650504, China; [EMAIL_ADDRESS] (Z.W. ); 138887@[EMAIL_ADDRESS] (J.W. ); [EMAIL_ADDRESS] (Y.C. ); [EMAIL_ADDRESS] (D.Z.) Correspondence: [EMAIL_ADDRESS] These authors contributed equally to this work. [PERSON]  
1 Key Laboratory in Media Convergence of Yunnan Province, Kunming 650504, China; [EMAIL_ADDRESS] (Z.Z. ); [EMAIL_ADDRESS] (L.X.) School of Software, Yunnan University, Kunming 650504, China; [EMAIL_ADDRESS] (Z.W. ); 1388887@[EMAIL_ADDRESS] (J.W. ); [EMAIL_ADDRESS] (Y.C. ); [EMAIL_ADDRESS] (D.Z.) Correspondence: [EMAIL_ADDRESS] These authors contributed equally to this work. [PERSON]  
1 Key Laboratory in Media Convergence of Yunnan Province, Kunming 650504, China; [EMAIL_ADDRESS] (Z.Z. ); [EMAIL_ADDRESS] (L.X.) School of Software, Yunnan University, Kunming 650504, China; [EMAIL_ADDRESS] (Z.W. ); 138887@[EMAIL_ADDRESS] (J.W. ); [EMAIL_ADDRESS] (Y.C. ); [EMAIL_ADDRESS] (D.Z.) Correspondence: [EMAIL_ADDRESS] These authors contributed equally to this work. [PERSON]  
1 Key Laboratory in Media Convergence of Yunnan Province, Kunming 650504, China; zhuchaoyang@stu. Many modern methods leverage deep learning techniques to learn news features and user interests to address the cold start problem. Among them, numerous research efforts have attempted end-to-end learning of news representations, such as embedding words into low-dimensional vectors and applying popular network architectures like CNNs and attention mechanisms to learn hidden news representations, enhancing recommendation accuracy. Although these neural news recommendation models have shown improved performance, there is still a need for further exploration on how to better simulate the complex correlations between user interests and news. For example, [PERSON] et al. (2017) [9] proposed learning news representation from news organizations using autoencoders and from user-accessed news using GRUs. However, GRUs are computationally expensive and they fail to capture word contexts. [PERSON] et al. [10] used a news representation model that learns news representations from titles using a CNN and employs user ID embeddings as queries for word-level and news-level attention networks. [PERSON] et al. (2018) introduced a deep-knowledge-aware network for news recommendations (DKN) [8], which preprocesses news using a knowledge graph, learns news representations from news titles using a three-channel CNN, and calculates relevance by aggregating candidate articles with browsed articles. However, the CNN fails to capture long-range word contexts and, thus, it cannot model relationships between accessed news items [11]. In the neural news recommendations with multi-head self-attention (NRMS) model [11], although multi-head attention mechanisms are utilized to process news, the single layer of multi-head attention is too simplistic to extract news features effectively. The work in this paper is inspired by many classical papers in the field of news recommendations, as well as the format of news headlines in real life. From studying various news recommendation models, we have found that the most important action is the processing of information in news headlines. For textual data, the key is to enable the model to better understand the relationships between words and between sentences. For example, in the preprocessing of text-related tasks in the BERT [12] model, two basic modules are used to help the model better understand the relationships between words and between sentences. However, in real life, most news headlines consist of only one sentence. Therefore, as news titles can accurately summarize news content, we believe that if we can effectively handle the relationships between words and improve feature extraction in this context, this will likely have a positive impact on the final recommendation results.