# An Entity-Relation Extraction Method Based on the Mixture-of-Experts Model and Dependency Parsing
## 1 Introduction  
The goal of entity-relation extraction is to simultaneously identify entity pairs and their corresponding relationships from unstructured texts. It is a core task in knowledge extraction and is crucial for downstream tasks, such as knowledge graph construction and question-answering systems. Therefore, exploring effective methods for entity-relation extraction from unstructured texts is of significant importance. The entity-relation extraction task consists of two key subtasks: named-entity recognition (NER) [1] and relation extraction (RE) [2]. The former identifies various entities within sentences, while the latter extracts the relationship between any two entities in a sentence. The development of entity-relation extraction techniques mirrors the history of artificial intelligence, having undergone three iterations: rule-based and dictionary-based methods, statistical machine learning-based methods, and deep learning-based methods [3]. Entity-relation extraction methods based on deep learning can be broadly categorized into two approaches: the pipeline extraction method and the joint extraction method. Thepipeline approach treats NER and RE as two independent tasks, which leads to a lack of interaction between the subtasks. As a result, textual information is not fully utilized, causing issues such as redundant entities that can affect the quality of subsequent relation extraction results. In recent years, joint extractions have garnered increasing attention from researchers, which unify the two subtasks of NER and RE using a joint framework to extract relation triplets. Although joint methods have demonstrated promising performance, several challenges persist. First, there remains the issue of insufficient semantic representation. Pre-trained language models, such as BERT, are often used as encoders due to their excellent performance. However, these models cannot explicitly model syntactic structures and are limited in their ability to fully capture semantic information, which restricts their accuracy in understanding complex linguistic phenomena. Some studies incorporate syntactic information as additional features into the model but typically treat dependency syntactic information and semantic representations as separate steps or only concatenate them without fully leveraging the syntactic structure to enhance semantic understanding. Second, a text often contains complex relationship types, and the distribution of different relationship types is imbalanced, which hinders the model's ability to comprehensively understand the relationships, making triplet extraction more challenging. Figure 1 illustrates the distribution of relationship frequencies in the training set of the Chinese public dataset DUIE, which contains 48 types of relationships, with a highly skewed distribution of instances across these relationships. This is a common phenomenon in textual data. Addressing these challenges can enhance the accuracy of entity-relation extraction tasks, which in turn has a significant impact on related fields. For instance, knowledge graphs build relationship maps between entities, providing rich structured data for information retrieval, reasoning, and decision-making. High-quality entity-relation extraction ensures the accuracy and completeness of knowledge graphs, thereby improving the capability of knowledge reasoning. In question-answering systems, accurately understanding the entities and their interrelationships in a query is crucial for providing precise answers and improving the reasoning abilities for complex questions. Therefore, to address these challenges, we propose a novel joint entity and relation extraction model. For challenge 1, we incorporated dependency syntax information at the encoding layer, encoding it separately in a single path, and applied an additive attention mechanism to fuse word-level information with BERT's character-level representations. This enables the model to recognize syntactic dependencies between words, enhancing its semantic understanding.