Application of Entity Classification Model Based on Different Position Embedding in C hinese Frame Semantic Parsing (2024)

Abstract: “This paper addresses three subtasks of Chinese Frame Semantic Parsing based on the BERT and RoBERTa pre-trained models: Frame Identification, Argument Identification, and Role Identification. In the Frame Identification task, we utilize the BERT PLM with Rotary Positional Encoding for the semantic frame classification task. For the Argument Identification task, we employ the RoBERTa PLM with T5 position encoding for extraction tasks. In the Role Identification task, we use the RoBERTa PLM with ALiBi position encoding for the classification task. Ultimately, our approach achieved a score of 71.41 in the closed track of the B leaderboard, securing fourth place and validating the effectiveness of our method.”

## introduction the chinese framenet (cf n ) is a chinese lexical semantic knowledge base for computer use, based on fillmore's frame semantics theory and referencing framenet from the university of california, berkeley, with chinese real corpus as the basis @xcite . frame semantic parsing involves a deep understanding of the entities or meanings involved in a sentence, which is of great significance in downstream tasks such as text summarization, relation extraction and reading comprehension. the basic structure of frame semantic parsing is as follows: as shown in figure 1 , in the example sentence "从 海 拔2800米 的 仁 青 岗 村 到 海 拔4600多 米 的 詹 娘 舍 哨 所" ("from renqinggang village at an altitude of 2800 meters to zhanniangshe outpost at an altitude of over 4600 meters"), the phrase "从. . . . . . 到. . . . . . " ("from . . . . . . to . . . . . . ") can activate a "moving path" frame, where "海 拔2800米" ("at an altitude of 2800 meters"), "仁青岗村" ("renqinggang village"), "海拔4600多米" ("at an altitude of over 4600 meters"), and "詹娘舍哨所" ("zhanniangshe outpost") are the four arguments of the example sentence. the attributes of the arguments are respectively "feature, starting point, feature, end point". chinese frame semantic parsing is a semantic parsing task based on chinese frame semantic resources. this task consists of the following three subtasks: frame identification, argument identification, and role identification. the frame identification task involves identifying the corresponding frame category from candidate frame categories based on the target word given in the sentence. the argument identification task determines the boundaries of the arguments in the frame based on the target word in the sentence. the role identification task determines the names of the frame elements corresponding to each argument based on the results of argument identification. therefore, this paper categorizes the frame identification task and role identification task as classification problems, and the argument identification task as an extraction task. target words arguments 从 moving path feature starting point feature end point 海拔2800米 的 仁青岗村 到 海拔4600多米 的 詹娘舍哨所 :entity categories :entity attributes figure 1: training data examples ## related work in the task of chinese frame semantic parsing, traditional machine learning algorithms were initially used, such as maximum entropy @xcite , conditional random fields @xcite , "obi data type" annotation, and greedy algorithms @xcite . however, with the continuous development of deep learning algorithms, zhao et al. @xcite used dnn for frame identification, suhaifeng et al. @xcite utilized frame relationships and definitions for frame identification, zhou et al. @xcite identified frame elements and frames together in end-to-end tasks, and utilized the relationship between target words and frame elements to assist in frame identification during decoding. wang et al. @xcite proposed a chinese frame semantic role labeling method based on self-attention mechanism to obtain long-distance information from sentences. the chinese framenet (cfn) contains extensive resources and applications that are significant for natural language understanding tasks such as frame disambiguation and semantic role labeling. cfn mainly focus on common knowledge within contexts @xcite , utilizing a combination of topdown, bottom-up, and expert manual curation methods for its creation. @xcite used different end-to-end frameworks for parsing and employing data augmentation and voting methods to further improve prediction accuracy. @xcite propose an entity classification model based on rotary position embedding (rope). @xcite used a multi-task pipeline strategy and pre-trained language models to address the problem of chinese frame semantic parsing. ## conclusion our article first acknowledges a drawback of traditional transformer models regarding position encoding, namely the "lack of relative positional information." it proposes addressing this issue by incorporating directional information between tokens into entity classification models through the use of "t 5" and "alibi" position encodings. additionally, it considers scenarios involving multiple target words and enhances the performance of the model in the context of framework identification tasks through aggregation. through experimentation, comparisons were made regarding various hyperparameters such as the composition of bert's hidden layers, linear warm-up parameters, adversarial training parameters, and various model architectures. the best combination on the validation set was identified, resulting in achieving the fourth position on the closed-track b leaderboard. our article also has some shortcomings. for instance, in the framework identification task, we were unable to find downstream model architectures that could effectively leverage roberta. as a result, we had to settle for using bert only, which led to a performance gap compared to the first-place result. improving the framework identification model to obtain more accurate and comprehensive framework semantic parsing results is a direction worthy of further research.