Des Prompts aux Profils: Evaluation de la qualité des données générées par LLM pour la classification des soft skills (2025)

Abstract: L’extraction automatique des soft skills à partir de CV constitue un enjeu central du Traitement Automatique du Langage Naturel (TALN) pour les ressources humaines. Toutefois, le manque de données annotées et les contraintes de confidentialité limitent le développement de modèles robustes. Cette étude préliminaire explore le potentiel des Grands Modèles de Langage (LLMs) pour générer des CV synthétiques dédiés à la classification des soft skills. Deux corpus sont proposés, un jeu de données de référence généré à partir de prompts explicites, et un corpus de CV complets produits selon une structure réaliste. Un cadre d’évaluation combinant des métriques avec et sans référence est mis en place, afin de mesurer la diversité, la redondance et la fidélité sémantique. Les résultats révèlent des compromis importants entre diversité lexicale et réalisme contextuel, apportant des pistes pour guider la génération future de données synthétiques pour la classification des compétences comportementales.

## introduction in the contemporary job market, soft skills -such as communication, teamwork, and adaptabilityplay a significant role in hiring decisions. the automatic extraction of soft skills from textual documents represents a task that has been widely explored in the fields of natural language processing (nlp) and information retrieval. while most previous efforts frame this as a sequence labeling or information extraction problem @xcite @xcite , our study approaches it as a classification task. given a pre-defined taxonomy of soft skills, the objective is to assign the correct labels to a given cv. this formulation allows us to apply standard classification techniques, but it also presents a critical dependency : the need for a well-labeled dataset annotated according to our specific taxonomy. the absence of such a resource in the current literature and the specificity of our task motivated us to explore different, newer alternatives. to face this challenge, we turn to large language models (llms) for synthetic data generation. llms, such as gpt @xcite , have demonstrated high proficiency in producing human-like text, offering a promising solution when real data is scarce. moreover, using real cvs raises significant challenges related to data privacy and anonymization. synthetic cvs generated by llms bypass this issue, offering a privacy-preserving alternative that eliminates the need for manual anonymization while still providing realistic training data. however, the efficacy of llm-generated synthetic data is contingent upon its quality and representativeness. past studies have highlighted the importance of evaluating synthetic data to ensure its reliability for nlp downstream nlp applications @xcite . in this context, our paper presents a short preliminary study on using llms to generate synthetic data for soft skill classification. we make three contributions : ## methodology in the context of our research, the automatic extraction of soft skills from cvs is challenged by the lack of annotated data. recent work has explored both extraction models and synthetic data generation using llms, and our study places itself in the line of research that looks into the evaluation of the quality of such synthetic data in context-rich domains like cvs. our work assumes prompt quality and focuses instead on assessing the resulting data itself-measuring its semantic similarity, diversity, and redundancy to inform downstream use in soft skill classification. ## results and discussion we applied our evaluation framework to two datasets : the baseline dataset, generated via direct prompting with soft skills names, and the cv dataset, generated using more naturalistic prompts simulating full resumes. below, we discuss the main findings based on diversity, redundancy, semantic similarity, and alignment with real cvs. ## limitations and future perspectives while our study provides valuable insights into llm-generated data for soft skill classification, several limitations remain. first, the behavior of llms is model-dependent, and the results obtained with one model, such as gpt-3.5, may differ significantly from other models. this study may be expanded in the future to compare various llms. second, we focused on evaluating the data itself but did not systematically assess or compare the prompts, which are known to strongly influence generation quality. third, domain drifts are a valid concern. llms, primarily trained on broad internet corpora, might introduce hallucinated elements not representative of actual cvs. future work should explore domain-adaptive fine-tuning, retrieval-augmented generation, and hallucination detection to mitigate such risks. integrating hallucination detection metrics would allow us to quantify factual inaccuracies in generated cvs, and, more importantly, detect internal contradictions. fourth, while we computed a wide range of textual measures, which enable relative comparison between data generation strategies, we did not benchmark these metrics against existing datasets. this absence of external reference points makes it difficult to contextualize our results in relation to real-world or human-authored datasets. although no publicly available benchmark datasets for soft skill cv classification currently exist to our knowledge, this remains an important direction for future research. moreover, while our evaluation focused on textual properties such as diversity and semantic similarity, it did not directly assess the datasets' effectiveness in downstream classification tasks. a definitive assessment of dataset utility requires training classification models exclusively on synthetic data and evaluating their generalization performance on real-world cvs. the best dataset ultimately depends on the model used, highlighting a trade-off between content quality and task-specific performance. looking ahead, future work should explore more sophisticated prompt engineering and integrate evaluation metrics into the generation process. incorporating human-in-the-loop validation, where synthetic outputs are lightly reviewed and corrected, could significantly enhance data quality. we also plan to refine our generation strategy by focusing on key cv sections that better capture soft skills, balancing generation cost with quality. finally, mixing real and synthetic data, and testing across models, will be crucial for building robust and generalizable systems.