Benchmarking H indi Term Extraction in Education: A Dataset and Analysis (2025)

Abstract: This paper introduces the HTEC HindiTerm Extraction Dataset 2.0, a resourcedesigned to support terminology extractionand classification tasks within the education domain. HTEC 2.0 has been developed with the objective of providing a high-quality benchmark dataset for the evaluation of term recognition and classification methodologies in Hindi educationaldiscourse. The dataset consists of 97 documents sourced from Hindi Wikipedia, covering a diverse range of topics relevant tothe education sector. Within these documents, 1,702 terms have been manuallyannotated where each term is defined as asingle-word or multi-word expression thatconveys a domain-specific meaning. Theannotated terms in HTEC 2.0 are systematically categorized into seven distinct classes.Furthermore, this paper outlines the development of annotation guidelines, detailingthe criteria used to determine term boundaries and category assignments. By offeringa structured dataset with clearly definedterm classifications, HTEC 2.0 serves as avaluable resource for researchers workingon terminology extraction, domain-specificnamed entity recognition, and text classification in Hindi.

## introduction terminology extraction techniques are essential in various computational applications that involve processing domain-specific language. these techniques focus on identifying and extracting specialized lexical units from text, which can be useful for structuring information @xcite , improving knowledge organization @xcite , and supporting automated text analysis @xcite . the extracted terms serve as key components in various natural language processing tasks, including text classification @xcite , information 1 https://tinyurl.com/6jcr5umc retrieval @xcite , and domain-specific knowledge modeling @xcite . a term is defined as a lexical unit that conveys a precise meaning within a specific field @xcite . various approaches have been proposed for extracting terms, differing in methodology, scope, and intended application. count-based methods such as tf-idf @xcite and cvalue @xcite have traditionally been used to extract terms. although these methods are computationally efficient, they have been outperformed by data-driven term extraction techniques. particularly, deep learning based methods based on language models have established state-of-the-art benchmarks on this task @xcite . the development of term extraction systems for low-resource languages has remained an open challenge due to the lack of high-quality annotated datasets and standardized evaluation frameworks. in this paper, we introduce a dataset specifically designed for term extraction in hindi, aiming to address this gap. additionally, with the increasing availability of synthetic data generated using generative language models, this dataset can also serve as a gold standard for evaluating term extraction systems. as discussed by @xcite evaluation frameworks for term extraction typically consist of two essential components. the first component is a gold-standard dataset, which is a collection of manually annotated texts that serve as a benchmark for comparison. the second component involves performance metrics such as precision, recall, and the f1-score. these metrics allow for a systematic assessment by comparing the outputs of extraction methods against the annotations in the gold standard. by providing a reliable benchmark, this resource facilitates the development and assessment of extraction methodologies tailored for hindi and other low-resource languages. to support this goal, the dataset has been carefully curated with enhanced annotation quality. two annotators were engaged during the initial rounds, allowing for iterative refinement of the guidelines to improve consistency and reliability before proceeding with the final annotation process. additionally, terms are classified into fine-grained semantic categories, enabling detailed analysis and supporting a range of terminology extraction and classification tasks. furthermore, detailed annotation guidelines were developed to standardize the annotation process. these guidelines evolved over multiple annotation rounds, incorporating feedback and refinements to enhance clarity and consistency. this iterative approach ensured that the annotated terms adhered to a well-defined framework, reducing subjectivity and improving overall dataset quality. the structure of this paper is as follows: section 2 discusses related work on term extraction and the development of term-annotated datasets. section 3 presents the dataset statistics and details the process of creating the annotation guidelines as well as the inter-annotator agreement. section 4 discusses the experimental setup and the experiments. section 5 discussed the results. finally, the paper concludes in section 6. ## dataset this section outlines the annotation guidelines established to ensure consistency in the annotation process and provides an overview of the dataset statistics. ## experimental setup to establish benchmark performance on this dataset, we conducted experiments using both unsupervised and supervised term extraction methods. the objective of these experiments is to evaluate the effectiveness of various methodologies in extracting domain-specific terms and to provide a baseline for future research. ## results the results demonstrated in ## conclusion we introduced htec 2.0, a hindi term extraction dataset for education, supporting term extraction and classification. it features annotated terms with improved consistency and a category for ambiguous cases. evaluations show xlm-roberta outperforms statistical methods, demonstrating the need for context-aware models. results highlight limitations of unsupervised approaches in low-resource languages like hindi. round 2: refinement of term selection rules and handling ambiguity • the selection criteria for multi-word terms were refined to ensure annotators consistently selected the longest meaningful phrase. round 3: introduction of semantic classification and color coding • annotators classified terms into predefined semantic categories, improving clarity in classification.