Do LLM s Know and Understand Domain Conceptual Knowledge? (2025)

Abstract: This paper focuses on the task of generating concept sememe trees to study whether Large Language Models (LLMs) can understand and generate domain conceptual knowledge. Concept sememe tree is a hierarchical structure that represents lexical meaning by combining sememes and their relationships.To this end, we introduce the Neighbor Semantic Structure (NSS) and Chain-of-Thought (CoT) prompting method to evaluate the effectiveness of various LLMs in generating accurate and comprehensive sememe trees across different domains. The NSS, guided by conceptual metaphors, identifies terms that exhibit significant external systematicity within a hierarchical relational network and incorporates them as examples in the learning process of LLMs. Meanwhile, the CoT prompting method guides LLMs through a systematic analysis of a term’s intrinsic core concepts, essential attributes, and semantic relationships, enabling the generation of concept sememe trees.We conduct experiments using datasets drawn from four authoritative terminology manuals and evaluate different LLMs. The experimental results indicate that LLMs possess the capability to capture and represent the conceptual knowledge aspects of domain-specific terms. Moreover, the integration of NSS examples with a structured CoT process allows LLMs to explore domain conceptual knowledge more profoundly, leading to the generation of highly accurate concept sememe trees.

## introduction large language models (llms) are regarded as versatile tools for various tasks, such as recommendations, language learning, and writing @xcite . previous * sijia shen and feiyan jiang contributed equally to this work. ## method our method is divided into six stages, as shown in figure 2 . firstly, prepare the data, terms and term definitions; secondly, three datastores are constructed to obtain the key-value pairs with the highest cosine similarity. by summarizing the terms and counting their frequencies, we identify and select the top five terms with the highest occurrence times, and used these terms as nss. thirdly, cot is added to provide a thinking process for llms. finally, post-processing is carried out to obtain the sememe tree. ## related work a sememe tree represents a hierarchical structure designed to express lexical meaning through the combination of semantic primitives (the smallest semantic units) and the relationships among them. hownet has been widely applied to various tasks, including word embeddings, word sense disambiguation, language modeling, and reverse dictionary construction. in recent years, research has increasingly focused on the automatic generation of semantic primitive trees for specific terms. @xcite developed a semantic primitive knowledge base for aviation-related terms using manual and semi-automatic methods. @xcite proposed a transformer-based model for generating sememe trees. while this method can determine whether a relationship exists between semantic primitives, it does not specify the type of relationship. @xcite introduced a term def generation model based on nnp-tdgm to address two key challenges: insufficient decoder training on low-frequency samples and limited encoding capacity. however, existing methods still have limitations. for instance, they rely heavily on labeled data and face challenges in leveraging large volumes of unlabeled data. additionally, they exhibit substantial domain specificity. in contrast, large models offer improvements in these areas through large-scale pretraining and enhanced semantic understanding. llms capabilities in lexical semantics and ontology construction have attracted significant scholarly interest. @xcite proposed a zero-shot classification-based induction method to extract hypernym relations from llms, demonstrating that prompt-based guidance can effectively capture hierarchical relationships. this demonstrates that llms possess the ability to infer ontological knowledge, thereby facilitating term structuring. @xcite assessed the performance of llms in understanding financial terminology. using definition modeling, they demonstrated the models' ability to generate precise domain-specific definitions through zero-shot and few-shot learning. @xcite analyzed the performance of llama-2 and mistral in classification learning tasks, revealing their capacity to acquire hierarchical knowledge and in-fer conceptual relationships. although existing research predominantly examines word semantics and ontology tasks, term definition modeling remains underexplored. this paper explores whether llms can comprehend terms and articulate their underlying conceptual semantics, offering valuable insights for term definition modeling. ## conclusion in this study, we comprehensively investigate whether llms can effectively acquire domainspecific conceptual knowledge, moving beyond mere surface-level recognition to attain a deeper semantic understanding. our experiments demonstrate that llms possess a measurable ability to generate semantic concepts associated with terms. grounded in the theory of term metaphor, we introduce examples of semantically related structures and employ a chain-of-thought approach. this method guides the model to systematically analyze the core concepts, attributes, and relationships of terms in a sequential manner, thereby enhancing its ability to uncover underlying meanings.