IDENTIFYING SKILL ADJACENCIES AND SKILL GAPS FOR GENERATING RESKILLING RECOMMENDATIONS AND EXPLAINABILITY

An embodiment for identifying skill adjacencies and skill gaps to generate reskilling recommendations. The embodiment may receive input from a user including candidate details and a job description. The embodiment may automatically extract a first set of skill keywords from the candidate description and a second set of skill keywords from the job description. The embodiment may automatically input the first and second set of skill keywords into a first type of word embedding model and a second type of word embedding model to automatically generate word embeddings. The embodiment may automatically compare the generated word embeddings and calculate cosine similarity scores for the first and second set of skill keywords. The embodiment may automatically identify skill overlaps and skill gaps using the calculated similarity scores, and automatically generating and outputting corresponding explainability statements to the user, and generate and output corresponding reskilling recommendations for the identified skill gaps.

BACKGROUND

The present application relates generally to identifying skill adjacencies, and more particularly, to identifying skill adjacencies and skill gaps to generate reskilling recommendations by using multiple word embedding models.

Today's fast changing workplace frequently necessitates reskilling of the workforce across a variety of industries and workplace settings. A growing number of jobs require specialized skillsets including constantly changing and expanding definitions of new and evolving skill sets. An automated method to determine skill adjacencies and skill gaps for efficiently reskilling workers is therefore desirable.

SUMMARY

According to one embodiment, a method, computer system, and computer program product for identifying skill adjacencies and skill gaps to generate reskilling recommendations by using multiple word embedding models is provided. The embodiment may include receiving input from a user including candidate details and a job description. The embodiment may also include automatically extracting a first set of skill keywords from the candidate description and a second set of skill keywords from the job description, and separating the first and second set of skill keywords. The embodiment may also include automatically inputting the first and second set of skill keywords into a first type of word embedding model and a second type of word embedding model to automatically generate word embeddings corresponding to the first and second set of skill keywords. The embodiment may further include automatically comparing the generated word embeddings and calculating cosine similarity scores for the first and second set of skill keywords. The embodiment may also include automatically identify skill overlaps and skill gaps using the calculated similarity scores, and automatically generating and outputting corresponding explainability statements to the user. The embodiment may also include automatically generating and outputting reskilling recommendations to the user based on the identified skill gaps.

DETAILED DESCRIPTION

Embodiments of the present application relate generally to identifying skill adjacencies, and more particularly, to identifying skill adjacencies and skill gaps to generate reskilling recommendations by using multiple word embedding models. The following described exemplary embodiments provide a system, method, and program product to, among other things, receive input from a user including candidate details and a job description automatically extract a first set of skill keywords from the candidate description and a second set of skill keywords from the job description, automatically input the first and second set of skill keywords into a first type of word embedding model, and a second type of word embedding model to automatically generate word embeddings corresponding to the first and second set of skill keywords, and automatically compare the generated word embeddings to calculate cosine similarity scores for the first and second set of skill keywords to identify skill overlaps and skill gaps for generating and outputting explainability statements and reskilling recommendations to the user. Therefore, the present embodiment has the capacity to improve approaches to reskilling that require manual labor by providing an automated system for identifying skill adjacencies and skill gaps to generate reskilling recommendations. The present embodiment further improves approaches to reskilling by providing a system that utilizes multiple word embedding models to more accurately generate vectors and calculate similarity scores for extracted sets of skill keywords. The present embodiment further improves approaches to reskilling by providing a system that automatically provides explainability statements and reskilling recommendations to the user based on the calculated similarity scores.

As previously described, today's fast changing workplace frequently necessitates reskilling of the workforce across a variety of industries and workplace settings. A growing number of jobs require specialized skillsets including constantly changing and expanding definitions of new and evolving skill sets. Current approaches to reskilling depend on manual logic, which can be time-consuming and expensive due to their dependence on manual labor requiring an expert to explicitly define relationships between skills. Manual approaches require constant updating of underlying manually-defined skill relationships to accommodate new skills for a given workplace. Manual approaches may be too expensive and therefore inaccessible to non-profit organizations or smaller corporations. Current approaches to reskilling typically do not provide sufficient explainability or remedial actions that may be taken to address an identified skill gap between a potential employee and a given job, preventing candidates from being able to reskill as fast as possible to get back into the staffing pool in a competitive and growing job market. Illustrative embodiments described herein, provide for an improved automated system that more accurately identifies skill adjacencies and skill gaps by using multiple word embedding models, and ultimately generates and outputs reskilling recommendations to a user, thereby providing a more accurate, cheaper approach to reskilling that provides a user with reskilling recommendations and explainability to assist the user in reskilling quickly.

According to at least one embodiment of a computer system capable of employing methods in accordance with the present invention to identify skill adjacencies and skill gaps to generate reskilling recommendations, the method, system, computer program product may receive input from a user including candidate details and a job description. The method, system, computer program product may then automatically extract a first set of skill keywords from the candidate description and a second set of skill keywords from the job description, and separate the first and second set of skill keywords. Next, the method, system, computer program product may automatically input the first and second set of skill keywords into a first type of word embedding model, and a second type of word embedding model to automatically generate word embeddings corresponding to the first and second set of skill keywords. According to one embodiment, the method, system, computer program product may then automatically compare the generated word embeddings and calculate cosine similarity scores for the first and second set of skill keywords. Next, the method, system, computer program product may automatically identify skill overlaps and skill gaps using the calculated similarity scores, and automatically generate and outputting corresponding explainability statements to the user. The method, system, computer program product may then automatically generate and output reskilling recommendations to the user based on the identified skill gaps. In turn, the method, system, computer program product has provided an improved automated system that more accurately identifies skill adjacencies and skill gaps by using multiple word embedding models, and ultimately generates and outputs reskilling recommendations to the user.

The following described exemplary embodiments provide a system, method, and program product for identifying skill adjacencies and skill gaps to generate reskilling recommendations.

Referring toFIG.1, an exemplary networked computer environment100is depicted, according to at least one embodiment. The networked computer environment100may include client computing device102, a server112, and Internet of Things (IoT) Device118interconnected via a communication network114. According to at least one implementation, the networked computer environment100may include a plurality of client computing devices102and servers112, of which only one of each is shown for illustrative brevity.

Client computing device102may include a processor104and a data storage device106that is enabled to host and run a software program108and a reskilling recommendation program110A and communicate with the server112and IoT Device118via the communication network114, in accordance with one embodiment of the present disclosure. Client computing device102may be, for example, a mobile device, a telephone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of running a program and accessing a network. As will be discussed with reference toFIG.4, the client computing device102may include internal components402aand external components404a, respectively.

The server computer112may be a laptop computer, netbook computer, personal computer (PC), a desktop computer, or any programmable electronic device or any network of programmable electronic devices capable of hosting and running a reskilling recommendation program110B and a database116and communicating with the client computing device102and IoT Device118via the communication network114, in accordance with embodiments of the present disclosure. As will be discussed with reference toFIG.4, the server computer112may include internal components402band external components404b, respectively. The server112may also operate in a cloud computing service model, such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). The server112may also be located in a cloud computing deployment model, such as a private cloud, community cloud, public cloud, or hybrid cloud.

IoT Device118may be a mobile device, a voice-controlled personal assistant, and/or any other IoT Device118known in the art for receiving queries that is capable of connecting to the communication network114, and transmitting and receiving data with the client computing device102and the server112.

According to the present embodiment, the reskilling recommendation program110A,110B may be a program capable of receiving input from a user including candidate details and a job description. Reskilling recommendation program110A,110B may then automatically extract a first set of skill keywords from the candidate description and a second set of skill keywords from the job description and separate the first and second set of skill keywords. Next, reskilling recommendation program110A,110B may then automatically input the first and second set of skill keywords into a first type of word embedding model and a second type of word embedding model to automatically generate word embeddings corresponding to the first and second set of skill keywords. Reskilling recommendation program110A,110B may then automatically compare the generated word embeddings and calculate cosine similarity scores for the first and second set of skill keywords. Next, reskilling recommendation program110A,110B may then automatically identify skill overlaps and skill gaps using the calculated similarity scores, and automatically generating and outputting corresponding explainability statements to the user. Finally, reskilling recommendation program110A,110B may automatically generate and output reskilling recommendations to the user based on the identified skill gaps. In turn, reskilling recommendation program110A,110B has provided for an improved automated system that more accurately identifies skill adjacencies and skill by using multiple word embedding models, and ultimately generates and outputs reskilling recommendations to a user, thereby providing a more accurate, cheaper approach to reskilling that provides a user with reskilling recommendations and explainability to assist the user in reskilling quickly.

Referring now toFIG.2, an operational flowchart depicting a process200for identifying skill adjacencies and skill gaps to generate reskilling recommendations according to at least one embodiment is provided. At202, the reskilling recommendation program110A,110B receives input from a user including candidate details and a job description. Candidate details may include a variety of data for a given candidate in need of reskilling or restaffing. For example, candidate details may include a resume, job role, skill specialty, historical certifications, and employee ID. In instances where a resume is deemed insufficient, the additional information sources listed above may be sourced from an organization's HR system. The job description may include a written description of the selected job, or a description of a given project, role, or desired specialty. Job description data may also be sourced from an organization's HR system or other stored talent acquisition data. In one illustrative example, reskilling recommendation program110A,110B may receive an input from a user including candidate details for a Candidate A, and a job description for a Job B that is related to a job opening for a software engineer.

At204, the reskilling recommendation program110A,110B automatically extracts a first set of skill keywords from the candidate details and a second set of skill keywords from the job description, and separates the first and second set of skill keywords. In other words, reskilling recommendation program110A,110B will separate skill keywords from the candidate details and the job description into two separate groups. For example, reskilling recommendation program110A,110B may extract a first set of skill keywords from the candidate details for an exemplary Candidate A including the following skill keywords: ‘Neural_networks’, ‘nlp’, ‘excel’, ‘machine_learning’, and ‘c++’ and sort this first set of skill keywords into a first group. Reskilling recommendation program110A,110B may then extract a second set of skill keywords from the job description for an exemplary Job B including the following skill keywords: ‘python’, ‘machine_learning’, ‘communication’, and ‘consulting’ and sort this second set of skill keywords into a second group. Reskilling recommendation program110A,110B may include a variety of skills-related datasets to function as a skills corpus or dictionary of relevant technical and non-technical skills. These skill-related datasets may be obtained through available job listings or through analysis of external job markets that are domain-specific or generalized. An exemplary skills corpus may be further enhanced with skills, tools, and technology related keywords sourced from a third-party website that includes a large number of job postings and candidate details therein. Once a skills corpus is established, skill keywords may be extracted from free text (resumes, job descriptions). Keyword extraction may be performed with any suitable extraction module, such as, for example, flashtext python library's KeywordProcessor module with case sensitivity, to extract keywords from text.

Next, at206, reskilling recommendation program110A,110B automatically inputs the first and second set of skill keywords into a first type of word embedding model, and a second type of word embedding model to automatically generate word embeddings corresponding to the first and second set of skill keywords.

In an exemplary described embodiment, the first type of word embedding model may be a Word2Vec model. The Word2Vec model may be custom-trained on the skills corpus discussed above. For the training of the Word2Vec model, the chosen parameters may include skipgram vs CB OW, hierarchical softmax vs negative sampling, embedding size, and evaluation method. In embodiments, the skipgram method is considered, negative sampling is considered, the embedding size may be 100, and the window size may be 5. Python library gensim's Word2Vec implementation may be used to train the model on the selected datasets, and then the genism python module's skip-gram technique may be used to generate the word embeddings. The corpus may be passed through a data cleaning pipeline to tokenize the keywords to ensure effective training. Once the Word2Vec model training is completed, the trained model file may be saved so that vectors can be loaded later for leveraging word embeddings in computing cosine similarity scores for skill keywords calculated at208, described in more detail below.

The second word embedding model may be a DistilBERT model. The DistilBERT model may be used to produce dense skill keyword vector representations. A vector representation for a single skill keyword may be obtained by feeding a phrase or skill keyword into the DistilBERT model and taking the mean of the subword token vectors the model produces. In embodiments, the DistilBERT model may include further pretraining on a domain specific corpus for a given job description. Entire paragraphs of a given job description may be used for pre-training. Alternatively, skill keywords may be extracted using the methods described above, which then may be used for further pre-training. Once pre-training language is selected, a masked language modelling approach, for example masking 15% of all tokens in the pretraining text, may be used for training. The number of epochs of pre-training may range from 20 to 100 using increments of 20.

The generation of word embeddings for a given set of skill keywords is ultimately used to calculate similarity scores for determining and output reskilling recommendations. As such, the accuracy of the word embeddings for a given set of skill keywords is important. In an exemplary embodiment, the extracted skill keywords may be run through both a Word2Vec model and DistilBERT model. This is because it was determined that each model has its own strengths for generating accurate embeddings for certain types of skill keywords. For example, the DistilBERT model provides more accurate results for non-technical skill keywords that contain no acronyms, as DistilBERT relies on a rich language understanding provided through extensive pre-training. However, for technical skills, skill names may not align with their literal meanings. For example, “Beautiful Soup”, a python web scraping library, has a misalignment between the literal meaning of the words and the meaning of the skill. Accordingly, DistilBERT would generate a less accurate word embedding than the Word2Vec model which relies on co-occurrence frequency of the skills and would therefore be unaffected by the misalignment. There is similar, if not more pronounced, misalignment for skill keywords containing acronyms. Accordingly, illustrative embodiments in accordance with this disclosure run the skill keywords through both models to generate word embeddings with improved accuracy, thereby improving the accuracy of the similarity scores and the generated reskilling recommendations. Accordingly, in embodiments, reskilling recommendation program110A,110B may be configured to include an ensemble (composite) model including both the Word2Vec and DistilBERT model to best handle a variety of extracted skill keywords. For example, in a describe embodiment, a Word2Vec model and a DistilBERT model may be trained using the same corpus of data. Then, manual annotations of a given list of skill keyword pairs may be made by unbiased volunteers. These manual annotations may be used as a standard against which each model's performance may be evaluated. This allows for determination and validation of which models more accurately characterize which types of skill keyword pairs, ultimately allowing for the use of an ensemble model approach using a composite model containing both the Word2Vec model and the DistilBERT model to process the most appropriate skill keyword pairings. It is envisioned that reskilling recommendation program110A,110B may be configured to utilize other combinations of word embedding models to form ensemble or composite models using two separate word embedding models to characterize a variety of extracted skill keywords more accurately.

At208, reskilling recommendation program110A,110B automatically compares the word embeddings generated at206and calculates cosine similarity scores for the first and second set of skill keywords. Cosine similarity is a measure of similarity between two non-zero vectors of an inner product space that measures the cosine angle between them. A cosine similarity score of 1.0 indicates that two skill keywords are identical, and a score of 0 indicates no similarity. The closer the calculated cosine similarity score is to 1.0, the more similar the skill keywords being compared are, and the more likely there is a skill overlap. The lower the calculated cosine score is, the more likely there is a skill gap.

Referring now toFIG.3, using the same example discussed above depicting an example process300, reskilling recommendation program110A,110B may, for example, extract ‘Neural_networks’, ‘nlp’, ‘excel’, ‘machine_learning’, and ‘c++’ as skill keywords310from Candidate A, and ‘python’, ‘machine_learning’, ‘communication’, and ‘consulting’ as keywords from Job Description B. Then reskilling recommendation program110A,110B may use the word embeddings generated at206for these illustrative sets of skill keywords to calculate cosine similarity scores320for each of the skill keywords. For example, reskilling recommendation program110A,110B may calculate and output the following similarity scores: [(‘machine_learning’, 1.0), (‘python’, 0.7144307), (‘consulting’, 0.4087105), (communication’, 0.2165209)]. In this example, reskilling recommendation program110A,110B calculated a cosine similarity score of 1.0 for the skill keyword ‘machine_learning’ because there was an exact match of skill keywords in the candidate details, and the job description. Reskilling recommendation program110A,110B then calculated a cosine similarity score of 0.7144307 for ‘Python’ because although the candidate details did not explicitly include ‘Python’, reskilling recommendation program110A,110B valued the remaining skill keywords in the candidate details as being similar enough to generate a similarity score between 0.5 and 1.0. Lastly, reskilling recommendation program110A,110B calculated a similarity score for ‘communication’ of 0.2165209 because the skill keywords in the candidate details were not determined to be similar to the skill keywords from the job description.

Next, at210, reskilling recommendation program110A,110B automatically identifies skill overlaps and skill gaps using the calculated cosine similarity scores, and automatically generates and output explainability statements to the user. A user may define a suitable skill satisfaction threshold that reskilling recommendation program110A,110B may be configured to recognize. If a skill keyword in the job description has a corresponding vector embedding that has a cosine similarity score (calculated at208) to the candidate detail skill keyword vector that is higher than the skill satisfaction threshold, then this skill keyword would be identified as being associated with a skill overlap. The remaining skill keywords that fall below the skill satisfaction threshold would be identified as being associated with skill gaps. Next, reskilling recommendation program110A,110B may automatically generate and output explainability statements330to the user identifying which skill keywords were categorized as similar skills, and which were identified as skill gaps. For example, using the same example as above, reskilling recommendation program110A,110B may generate and output the following (as illustrated inFIG.3) to a user if using a threshold of 0.6: “similar_skills: [(‘machinelearning’, 1.0), (‘python’, 0.7144307)] Skills_gap: [(‘communicaiton’, 0.21659209)]”

Finally, at212, reskilling recommendation program110A,110B automatically generates and outputs reskilling recommendations to the user based on identified skill gaps. In embodiments, the reskilling recommendations may include, for example, recommended courses, assignments, or alternative open jobs. In the above example, reskilling recommendation program110A,110B would generate and output a reskilling recommendation to the user for ‘communications’ because it was identified as a skill keyword that was associated with a skill gap. Thus, reskilling recommendation program110A,110B may output to the user the name of a training course or program for addressing the identified skill gap. In another example in which the skill gap was for the skill keyword “Python”, reskilling recommendation program may, for example, output to the user a recommendation including “Introduction to Python Programming” or “Advanced Concepts in Python Programming” or other courses for which the “python” skill keyword is a core skill. In embodiments, reskilling recommendations may be domain-specific, and added to reskilling recommendation program110A,110B via manual annotations. In embodiments, reskilling recommendation program110A,110B may use a machine learning approach to recommend reskilling plans based on the identified skill gaps.

In the context of this disclosure, machine learning broadly describes a function of a system that learns from data. Machine learning is a branch of artificial intelligence that relates to mathematical models that can learn from, categorize, and make predictions about data. Such mathematical models, which can be referred to as machine-learning models, can classify input data among two or more classes; cluster input data among two or more groups; predict a result based on input data; identify patterns or trends in input data; identify a distribution of input data in a space; or any combination of these. Examples of machine-learning models can include (i) neural networks; (ii) decision trees, such as classification trees and regression trees; (iii) classifiers, such as Naïve bias classifiers, logistic regression classifiers, ridge regression classifiers, random forest classifiers, least absolute shrinkage and selector (LASSO) classifiers, and support vector machines; (iv) clusterers, such as k-means clusterers, mean-shift clusterers, and spectral clusterers; (v) factorizers, such as factorization machines, principal component analyzers and kernel principal component analyzers; and (vi) ensembles or other combinations of machine-learning models. In some examples, neural networks can include deep neural networks, feed-forward neural networks, recurrent neural networks, convolutional neural networks, radial basis function (RBF) neural networks, echo state neural networks, long short-term memory neural networks, bi-directional recurrent neural networks, gated neural networks, hierarchical recurrent neural networks, stochastic neural networks, modular neural networks, spiking neural networks, dynamic neural networks, cascading neural networks, neuro-fuzzy neural networks, or any combination of these.

Machine-learning models can be constructed through an at least partially automated (e.g., with little or no human involvement) process called training. During training, input data can be iteratively supplied to a machine-learning model to enable the machine-learning model to identify patterns related to the input data or to identify relationships between the input data and output data. With training, the machine-learning model can be transformed from an untrained state to a trained state. Input data can be split into one or more training sets and one or more validation sets, and the training process may be repeated multiple times. The splitting may follow a k-fold cross-validation rule, a leave-one-out-rule, a leave-p-out rule, or a holdout rule.

It may be appreciated thatFIGS.2and3provide only illustrations of an exemplary implementation and do not imply any limitations with regard to how different embodiments may be implemented. Many modifications to the depicted environments may be made based on design and implementation requirements.

FIG.4is a block diagram400of internal and external components of the client computing device102and the server112depicted inFIG.1in accordance with an embodiment of the present disclosure. It should be appreciated thatFIG.4provides only an illustration of one implementation and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made based on design and implementation requirements.

The client computing device102and the server112may include respective sets of internal components402a,band external components404a,billustrated inFIG.4. Each of the sets of internal components402include one or more processors420, one or more computer-readable RAMs422, and one or more computer-readable ROMs424on one or more buses426, and one or more operating systems428and one or more computer-readable tangible storage devices430. The one or more operating systems428, the software program108and the reskilling recommendation program110A in the client computing device102and the reskilling recommendation program110B in the server112are stored on one or more of the respective computer-readable tangible storage devices430for execution by one or more of the respective processors420via one or more of the respective RAMs422(which typically include cache memory). In the embodiment illustrated inFIG.4, each of the computer-readable tangible storage devices430is a magnetic disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices430is a semiconductor storage device such as ROM424, EPROM, flash memory or any other computer-readable tangible storage device that can store a computer program and digital information.

Each set of internal components402a,balso includes a RAY drive or interface432to read from and write to one or more portable computer-readable tangible storage devices438such as a CD-ROM, DVD, memory stick, magnetic tape, magnetic disk, optical disk or semiconductor storage device. A software program, such as the reskilling recommendation program110A,110B, can be stored on one or more of the respective portable computer-readable tangible storage devices438, read via the respective R/W drive or interface432, and loaded into the respective hard drive430.

Each set of internal components402a,balso includes network adapters or interfaces436such as a TCP/IP adapter cards, wireless Wi-Fi interface cards, or 3G or 4G wireless interface cards or other wired or wireless communication links. The software program108and the reskilling recommendation program110A in the client computing device102and the reskilling recommendation program110B in the server112can be downloaded to the client computing device102and the server112from an external computer via a network (for example, the Internet, a local area network or other, wide area network) and respective network adapters or interfaces436. From the network adapters or interfaces436, the software program108and the reskilling recommendation program110A in the client computing device102and the reskilling recommendation program110B in the server112are loaded into the respective hard drive430. The network may comprise copper wires, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.

Each of the sets of external components404a,bcan include a computer display monitor444, a keyboard442, and a computer mouse434. External components404a,bcan also include touch screens, virtual keyboards, touch pads, pointing devices, and other human interface devices. Each of the sets of internal components402a,balso includes device drivers440to interface to computer display monitor444, keyboard442, and computer mouse434. The device drivers440, R/W drive or interface432, and network adapter or interface436include hardware and software (stored in storage device430and/or ROM424).

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

Workloads layer90provides examples of functionality for which the cloud computing environment may be utilized. Examples of workloads and functions which may be provided from this layer include: mapping and navigation91; software development and lifecycle management92; virtual classroom education delivery93; data analytics processing94; transaction processing95; reskilling recommendations96. Reskilling recommendations96may relate to automatically identifying skill adjacencies and skill gaps to generate reskilling recommendations using multiple word embedding models.