Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings (2024)

Abstract: Recent years have brought significant advances to Natural Language Processing (NLP), which enabled fast progress in the field of computational job market analysis. Core tasks in this application domain are skill extraction and classification from job postings. Because of its quick growth and its interdisciplinary nature, there is no exhaustive assessment of this field. This survey aims to fill this gap by providing a comprehensive overview of deep learning methodologies, datasets, and terminologies specific to NLP-driven skill extraction. Our comprehensive cataloging of publicly available datasets addresses the lack of consolidated information on dataset creation and characteristics. Finally, the focus on terminology addresses the current lack of consistent definitions for important concepts, such as hard and soft skills, and terms relating to skill extraction and classification.

## introduction skill extraction and classification has recently been the subject of an increased amount of interest @xcite , which shows in a high number of publications, driven by the advances in natural language processing (nlp) technology. for instance, through large language models (llms) the low resource tasks of skill extraction can be approached by using synthetic training data @xcite . surveys regarding skill extraction are emerging @xcite , nevertheless, a comprehensive overview from an nlp perspective is still lacking-a gap we aim to fill in this survey. our contributions are: while prior surveys exists, they focus typically on skill count and topic modeling methods for extracting skills. skill count is performed manually or by matching n-grams with a skill base. topic modeling is an unsupervised method utilizing word distributions to identify underlying topics in documents. due to primary statistical basis and lack of defined skill spans or labels, topic modeling, as well as skill count, methods are not covered in this survey. for further details on skill count, see @xcite and @xcite , and for topic modeling, please refer to @xcite , @xcite and @xcite . ## skill-related terminology the terms skill extraction, identification @xcite , detection @xcite , standardization @xcite and classification are used differently, sometimes interchangeably, and describe the same or different tasks. we provide the following definition (see an example in given these definitions, the skill extraction step can happen at different levels of granularity (of the input). some works extract skills per jp (e jp , the overall document), per sentence (e sentence ) or per n-gram (e n-gram ). a skill span (e span )is a continuous n-gram sequence that capture a skill. a skill base (b) is a knowledge base containing skill entities and terminology. a taxonomy is a hierarchically structured skill base, while ontologies provide a structure via relationships between concepts @xcite . several works use the term "skill dictionary" for a skill base, most often referring to an unstructured skill base or a list of skills @xcite . two popular publicly-available skill bases, created by domain experts, and are frequently used and maintained are the european skills, competences, qualifications and occupations (esco; le vrang et al., 2014) taxonomy and the us occupational information network (o*net; @xcite . we refer to @xcite for more examples of skill bases. ## what are skills? on skill definitions understanding the concept of a skill is pivotal in the field of skill extraction. in this section, we investigate several definitions of skills by various publications and institutions, aiming to identify commonalities and distinctions across different sources, which is crucial for establishing a common ground in this emerging field. the concept of skill can be seen as one broad concept @xcite @xcite or split into subclasses, with multiple possibilities for the split. in the latest version of the esco taxonomy the "skill pillar" is divided into four categories: "transversal skills", "skills"," knowledge" and "language skills and knowledge". foot_0 o*net is structured in six domains @xcite , the domain most fitting for skill extraction from jp is "worker requirements". this domain entails four subcategories: basic skills, cross-functional skills, knowledge, and education. 2 but publications considered in this survey that define skills, mainly distinguish between hard and soft skills @xcite @xcite , which is therefore also the separation used in this survey. @xcite delineate hard skills as professional competencies, activities, or knowledge pertinent to organizational functions, processes, and roles, essential for the successful completion of specific tasks. this definition emphasizes the practicality and functionality of hard skills within a professional setting. aligning with this, the study by @xcite views hard skills as task-oriented technical competencies, drawing upon lyu and @xcite to define them as formal technical abilities for performing certain tasks. furthermore, gugnani and misra (2020) expand on this perspective by incorporating technological terminologies for skill identification and therefore integrating knowledge as a fundamental component of hard skills. ## operationalization of skill definitions in this section, we explore various methodologies for operationalizing skill definitions in skill extraction and classification research. using a skill base by using a given skill base, a pre-defined definition of the concept of skills is provided by the authors of the skill base. numerous studies employ established skill bases such as the esco taxonomy @xcite @xcite @xcite or o*net (gugnani and misra, 2020). however, it is often ambiguous whether these studies use all or only specific subcategories @xcite @xcite . some papers mention explicitly the use of all subclasses @xcite other times it can be inferred from the number of skill spans used @xcite . however, one should note that the interpretations of esco definitions differ based on the esco version and authors' perspective. @xcite used esco version 1.0 with a different soft skill category than discussed in section 4 and implemented two labels: "knowledge" aligns with esco's "knowledge" category, and "skills" as a fusion of the hard and soft skills. in contrast, @xcite using the same esco version, but treat soft skills separate from hard skills. most of the publications used all subcategories as skills without differentiating @xcite @xcite . beyond these, there are other skill bases, such as the russian professional standard in @xcite or the chinese occupation classification grand dictionary used in @xcite ; @xcite . additionally, non-official skill bases exist, like the list of 1k soft skills in @xcite or linkedin's in-house taxonomy for skill extraction @xcite . in general, for transparency and reproducibility, it is helpful to state which subset of fine-grained labels l of the skill base (b) and which skill base version is used. leveraging automated tools some studies leverage automated tools like autophrase @xcite or microsoft azure analytics service for ner for initial skill term detection, followed by manual verification and refinement @xcite . also @xcite extract parts of their training data using an automated tool, while others are taken from a skill base. 7 lastly, gugnani and misra (2020) employ an ibm tool for skill identification, which forms a part of a larger skill identification framework. 8 while some previous work did not apply manual verification @xcite , we recommend it to reduce automation bias from the tool impacting the data. definition through labeling domain experts play a crucial role for labeling data and therefore impact how the definition of skills is put into work @xcite @xcite . @xcite additionally provide a codebook with skill definitions to address ambiguities. @xcite used next skills identified by hiring experts and skills common among successful applicants as training data. the study by @xcite treat the companies filing the jps as domain experts by using their labels (see also section 6). besides domain experts, crowd workers and the people writing the guidelines for the workers oftentimes determine which terms are skills. some studies do not mention who labels the data @xcite @xcite . we suggest being clear about the labeling process and guidelines, making them public for transparency and re-use/standardization, and using domain experts if possible for accurate labeling. ## data in this section, we provide a comprehensive description of publicly available datasets, with an overview in sayfullina by @xcite is a dataset derived from a publicly available kaggle dataset, containing jps from within the uk and representing a variety of sectors. 9 the authors retrieved soft skill spans by exact matching with a list of 1,072 soft skills. each identified span is accompanied by up to 10 surrounding words. crowdsourcing was used to determine whether the highlighted skill belongs to a job applicant. to ensure reliability, the workers were tested on a small set of jps and each snippet was evaluated by at least three workers. this process led to a dataset with high class imbalance due to more positive examples. to counter this, additional skill spans were added, including those usually not describing candidates (marked as negative) and those consistently labeled positive. green by @xcite uses the same kaggle dataset as sayfullina. the labeling was done via crowdsourcing, they did not use experts but only workers who passed a test were included, and encouraged to follow the guidelines. apart from the "skill" label capturing hard and soft skills, the labels "occupation", "domain", "experience", "qualification", and "none" are used in a bio scheme. the authors reduced errors by label aggregation with a preference towards labels from higher-performing workers. additionally, they reclassified specific "experience" spans, as "skill" spans, and manually split multi-term spans into separate spans. fijo by @xcite was created in partnership with canadian insurance companies, and consists of cleaned and de-identified french jps published between 2009 and 2020. the dataset focus on soft skills and includes 867 jps with 47 annotated jps, selected and annotated by a domain expert. the annotated spans are unevenly distributed across four classes: "thoughts", "results", "relational", and "personal". skillspan by @xcite consists of the anonymized raw data and annotations of skill and knowledge spans from three jp datasets, one of which cannot be made publicly available due to its license. the available datasets are: the development of the publicly available annotation guidelines involved an iterative process, starting with a few jps and progressing through several rounds of annotation and refinement by three domain experts. kompetencer by @xcite consists of danish jps with annotated skill and knowledge spans, see decorte by @xcite is a variant of the skillspan dataset with annotated esco labels. they used the identified skill without the skill and knowledge labels, but they can be recreated by matching the dataset with skillspan, see ## methods in this section, we survey methods for skill extraction and classification. as in section 3 the goal of the extraction is to identify skill spans with (e c ) or without coarse labels (i). the classification section covers direct classification methods (c d ) and classification methods with extraction (c e ), both aim to retrieve fine-grained skill labels. ## conclusions and future directions recent publications indicate two emerging trends in skill extraction. firstly, extracting skills against skill bases like esco is gaining popularity, facilitating cross-industry and regional comparisons. secondly, llms are increasingly applied in skill extraction and classification, proving particularly advantageous due to the scarcity of training data in this domain. future research in skill extraction and classification could focus on emerging skills and the extraction of implicit skills. methods like those by @xcite and @xcite update skill bases with emerging technologies and frequently used keywords, but evaluating these remains difficult without a standard benchmark. the challenge of extracting implicit skills, not directly stated in job postings, is also gaining attention. techniques include prompting llms to generate training data with implied skills @xcite and using complete sentences to encompass both explicit and implicit skills @xcite . however, these methods need thorough evaluation, presenting an open field for future exploration.