J ob XMLC : EX treme Multi-Label Classification of Job Skills with Graph Neural Networks (2023)

Abstract: Writing a good job description is an important step in the online recruitment process to hire the best candidates. Most recruiters forget to include some relevant skills in the job description. These missing skills affect the performance of recruitment tasks such as job suggestions, job search, candidate recommendations, etc. Existing approaches are limited to contextual modelling, do not exploit inter-relational structures like job-job and job-skill relationships, and are not scalable. In this paper, we exploit these structural relationships using a graph-based approach. We propose a novel skill prediction framework called JobXMLC, which uses graph neural networks with skill attention to predict missing skills using job descriptions. JobXMLC enables joint learning over a job-skill graph consisting of 22.8K entities (jobs and skills) and 650K relationships. We experiment with real-world recruitment datasets to evaluate our proposed approach. We train JobXMLC on 20,298 job descriptions and 2,548 skills within 30 minutes on a single GPU machine. JobXMLC outperforms the state-of-the-art approaches by 6% in precision and 3% in recall. JobXMLC is 18X faster for training task and up to 634X faster in skill prediction on benchmark datasets enabling JobXMLC to scale up on larger datasets.

## introduction online recruitment platforms such as linkedin and glassdoor are extensively used to post jobs, find relevant candidates, and match resumes to the jobs posted. recruiters create job positions mentioning skills, roles, and responsibilities to reach potential candidates. among all these required fields, skills are crucial parameters to determine whether or not a candidate is suitable for a job position @xcite . recruiters often miss adding relevant and crucial skills required for the job due to a communication gap between the domain experts and recruiters. according to statistics, 65% of the job descriptions (jds) do not include relevant and popular skills, and 40% of jds miss listing 20% or more explicitly-stated skills in the prose description @xcite . it reduces the number of relevant applications for the job posting and affects the performance of major recruitment tasks such as job-to-resume matching. therefore, it is imperative to recommend such missing skills to improve the quality of job postings. figure 1 shows the sample (fictitious) job posted over the recruitment platform where some skills are missing from the textual jd. prior works @xcite explored missing skill recommendation task using large-scale pre-trained language models. document embedding and graph-based systems @xcite are used for skill extraction and recommendations. however, these approaches have a few shortcomings-(a) they do not exploit the structural relationships between jobs and skills across the whole dataset. for example, assume jobs j 1 and j 2 share a common skill s 1 . if there is another skill s 2 relevant to j 2 and other similar jobs, we can infer that s 2 might also be relevant to j 1 . such transitive cues can be extremely useful for identifying missing skills. current deep extreme classifiers @xcite find it hard to model such implicit relationships unless the training set explicitly contains a pair (j 1 , s 2 ), (b) they give equal importance to every skill corresponding to the job. however, each skill in the skill label set has different weights based on the frequency of their occurrence in job descriptions, (c) language models bring high computational costs at massive scales as a task not only involves predicting multiple missing skills but also requires to precisely organize the most relevant skills specific to the job posting. graphs are naturally suitable to make the relationships explicit such as job-skill networks. job title job description required skills market analyst assist the manager in sourcing the food industry and in conducting product research and analysis. facilitate effective communication between the analytics and user experience teams. evaluates customers' online behaviour and provide insights and recommendations for further enhancements to the guest experience. strong research, data analysis and communication skills. communication visualization tableau data analysis python explicit skills implicit skills excel figure 1: an example of a fictitious job posted over a recruitment platform. the job description does not include implicit and job-specific skills such as 'tableau', 'visualization', 'python', and 'excel'. two nodes (jobs) are likely to have common neighbors (skills) if the jobs have overlapping skills. to model these structural relationships between nodes (jobs and skills), a graph neural network (gnn) is a well-known architecture for representing the knowledge and additional information @xcite . missing skills applications also face extreme skill label sparsity; using label co-occurrence alone without graphs yields fractured correlations. to this end, we propose a framework called jobxmlc, which uses a gnn with label attention to jointly model the jobs and skills in the same space. through the use of ubiquitous job description data, we aim to predict the missing skills using collaborative learning of jobs and skills. therefore, we model our problem as an extreme multi-label classification (xmlc) task as there is no existing dataset with manually-annotated labels from textual jds, making it sub-optimal to train a sequence labeling task. the contributions of this work are summarized as follows: recently, several works @xcite have been done for skill prediction using extreme multi-label classification. xmlc refers to the classification of text where the number of the set of labels is large, i.e., thousands or millions. one-vs-all (ova) is a well-known method for text classification tasks with high accuracy @xcite . the ova approach is computationally efficient for the xmlc task for modest-sized label sets (up to a few thousand labels). these methods are broadly classified as (a) deep learning-based models, (b) graph-based, and (c) domain-specific methods. ## problem formulation a job j @xmath0. the skill set is represented as, ## jobxmlc: extreme multi-label classification of job skills in this section, we introduce jobxmlc as shown in figure 2 . the architecture is inspired by the models proposed in @xcite . the architecture comprises of three major components: 1) job-skill graph 2) graph neural network that learns multi-hop embeddings with neighbourhood selection approach on the job-skill graph 3) a scalable mechanism of extreme classifiers to predict skill labels in cold and warm-start scenarios. ## results and analysis ## ablation study initial embeddings: jobxmlc shortlisting criteria offered much better recall if we use the initial fasttext embeddings to create shortlists. we observe that fasttext worked best for our recruitment domain dataset in comparison to other recent pre-trained language representation models (see appendix b) including bert @xcite , distilbert @xcite , and paraphrase-mini-lm-l6 @xcite . for example, on the mycareersfuture.sg dataset, the recall for the top 100 labels shortlisted using the initial fasttext and bert embeddings are around 85.20% and 56.90%, respectively. based on an average of 20.61 skills per job, about 4 skills were derived within the top 5 and 19 within the top 100 of derived skills. warm and cold start scenarios: job description required skills (ground truth) minimum 5 7 years experience information technology software development must 3 4 yeras experience dot net development experience asp.net c, .net xml experience, language query update etc knowledge pc networking require dot net developer mnc client singapore typre position long term contract initial degree information technology require minimum 5 7 years experience information technology software development must 3 4 years experience dot net development experience asp.net c net xml etcknowledge pc networking good communication skills software development javascript .net java jquery xml bert-xmlc+cab jobxmlc web applications software development software development java java .net javascript .net jquery jquery xml xml web applications php python c linux software engineering asp.net sdlc asp.net sdlc integration guage. our approach can handle missing skills which are part of our skill vocabulary, but it cannot infer new emerging skills from job descriptions, i.e., out-of-vocabulary. we will consider domain knowledge and the popularity of job skills to generalize our approach for job-candidate mapping applications for future work. we wish to expand our work to other recruitment domain applications with resumes and candidate profiles. ## ethical considerations the paper investigates the missing skills problem with the help of a graph-based framework by incorporating word-based embeddings that can be insightful for other researchers in academia and industry. any biases present in the dataset or embedding model can creep into the proposed approach.