# BTCAM: Attention-Based BiLSTM for Imbalanced Classification of Political Inquiry Messages
## 1 Introduction  
With the development of information technology, political inquiry platforms have emerged as a crucial channel for citizen-government department communication, particularly during the COVID-19 pandemic. Citizens typically submit inquiry messages through various information services. These messages are then collected, classified, and forwarded to the relevant government departments for response [1]. However, due to the sheer volume and complexity of inquiry messages, manual classification based on expertise is inefficient and error-prone [2]. This will inevitably lead to a decrease in public satisfaction, which is harmful to the establishment of a positive government image. Therefore, the message-processing efficiency must be improved. With the emergence of natural language processing (NLP) technology, it has become feasible to automate the classification of these political inquiry messages, enabling the advancement towards building a smart government. Machine learning algorithms can automatically learn features and patterns by training large amounts of text data, thereby building a model that can process natural language. Therefore, more and more researchers have begun to introduce these algorithms into government affairs management to promote the construction of smart government affairsand improve service levels. For example, rule-based algorithms were first proposed to predict governance indexes or classify emails [3; 4]. [PERSON] et al. [5] utilized text-mining technology to analyze extensive data on civil complaints to identify social problems and develop solutions automatically. Considering each document covers a different topic, [PERSON] [6] utilized the latent Dirichlet allocation (LDA) method to automatically identify the topics of electronic petitions, demonstrating that the discovered topics were closely related to social events. [PERSON] et al. [7] analyzed data on citizen petitions and looked into the variety of these requests by assessing the relevance of bus-related keywords. LDA performs well in some text analysis and clustering tasks, but it may encounter challenges in classification tasks. Therefore, [PERSON] and [PERSON] [8] selected support vector machines (SVMs) to improve the classification performance on official messages. Subsequently, the introduction and use of convolutional neural networks (CNNs) designed for text processing quickly outperformed the benchmarks. For example, [PERSON] et al. [9] introduced Topic-RNN, which can more accurately identify topics hidden in official documents. In response to the need to capture contextual information over time, researchers have utilized LSTM [10; 11; 12] in government affairs text processing with time-series analysis, resulting in remarkable advancements. [PERSON] et al. [13] proposed a cyberbullying detection method based on the BERT pre-trained model and achieved excellent performance, confirming the strong generalization ability of BERT in complex social media text classification tasks. However, Chinese has multiple meanings for one word and BERT cannot adapt well to these characteristics using word-level embedding. Despite the effectiveness of machine learning algorithms in dealing with government affairs, there is limited research on the automatic classification of government inquiry messages. The automatic classification of political inquiry messages still faces two serious challenges:  
* As illustrated in Figure 1, the length of political inquiry messages is usually short, with most messages less than 100 words. Due to the limitation of text length, the association and contextual information between different elements are incomplete, resulting in possible ambiguity during the classification. For example, for a certain message, such as _\"There is construction at night near the community. The sound is very loud and affects rest. I hope it can be solved.\"_, the message may involve categories such as _\"Urban and rural construction\"_ and _\"Transportation\"_.