System Report for CCL 23-Eval Task 6: A Method For Telecom Network Fraud Case Classification Based on Two-stage Training Framework and Within-task Pretraining (2023)

Abstract: “Domain-specific text classification often needs more external knowledge, and fraud cases havefewer descriptions. Existing methods usually utilize single-stage deep models to extract semanticfeatures, which is less reusable. To tackle this issue, we propose a two-stage training frameworkbased on within-task pretraining and multi-dimensional semantic enhancement for CCL23-EvalTask 6 (Telecom Network Fraud Case Classification, FCC). Our training framework is dividedinto two stages. First, we pre-train using the training corpus to obtain specific BERT. The seman-tic mining ability of the model is enhanced from the feature space perspective by introducing ad-versarial training and multiple random sampling. The pseudo-labeled data is generated throughthe test data above a certain threshold. Second, pseudo-labeled samples are added to the trainingset for semantic enhancement based on the sample space dimension. We utilize the same back-bone for prediction to obtain the results. Experimental results show that our proposed methodoutperforms the single-stage benchmarks and achieves competitive performance with 0.859259F1. It also performs better in the few-shot patent classification task with 65.160% F1, whichindicates robustness.”

## introduction the official implementation of law of the people's republic of china on anti-telecom and network fraud demonstrates china's determination to combat telecom and network fraud in our society. as an essential part of the fight against telecom and network fraud crimes, accurate classification of fraud cases facilitates the public security services to grasp the distribution characteristics of current fraud cases. it assists them in making targeted measures, such as prevention, supervision, suppression, and detection. currently, there are two problems with the task: (1) the categories of fraud cases are more finely grained. to cope with these problems, we introduce the two-stage training to enhance the differentiation among samples and to motivate the model to distinguish fine-grained case samples and thus achieve effective classification. in the first stage, the model extracts semantic features in fraud cases to obtain abovethreshold prediction samples and use them as prior knowledge. the model then fuses the obtained samples into the second stage to enhance the representation capability of the model. in the second stage, the model uses bert @xcite based fine-tuning to improve the semantic feature mining capability for a few descriptive texts of the cases and finally predicts the labels. in the training process, we introduce a multi-dimensional semantic enhancement method to improve the contextual modeling capability of the model. it contains two dimensions: feature space dimension and sample space dimension. the former uses a combination of adversarial training and multi-sampling. at the same time, the latter introduces semi-supervised learning that uses a model fusion-based approach to generate pseudo-labeled data. we implement the model fusion-based approach by summing the predicted probabilities. in addition, to enhance the model's fitness, we introduce the within-task pretraining approach to maximize the use of data from the dataset and further improve the classification effect without introducing external knowledge. in this paper, we proposed a two-stage training framework, and here are our main contributions: ## conclusion in this paper, we propose a two-stage framework based on within-task pretraining and multi-dimensional semantic enhancement for ccl23-eval task 6 (fcc). it enhances the model's ability to represent the case text in feature and sample space dimensions. experiments show that our proposed architecture achieves good results in this review, outperforming baseline models. in future work, we plan to use the framework extensively for domain text classification tasks in other scenarios. inspired by model pruning and knowledge distillation, we try to refine the sub-stage to improve our architecture.