Title: LinkedIn Jobs Recommendation Systems - PyImageSearch

URL Source: https://pyimagesearch.com/2023/08/07/linkedin-jobs-recommendation-systems/

Published Time: 2023-08-07T13:00:00+00:00

Markdown Content:
* * *

**Table of Contents**
---------------------

*   [LinkedIn Jobs Recommendation Systems](#h2BPTitle)

*   [Feed Recommendations](#h3Feed)

*   [LinkedIn Feed](#h4Feed)
*   [First Pass Rankers](#h4FPR)
*   [Second Pass Ranker](#h4SPR)
*   [Evaluation](#h4Evaluation)
*   [Feature Representation](#h4FeatureRepresentation)

*   [Recruiter Search Recommendations](#h3Recruiter)

*   [LinkedIn Talent Search and Recommendations](#h4TalentSearch)
*   [Unsupervised Approach](#h4UnsupervisedApproach)

*   [LinkedIn’s Economic Graph](#h5EconomicGraph)
*   [First-Order Embeddings](#h5First-OrderEmbeddings)
*   [Second-Order Embeddings](#h5Second-OrderEmbeddings)
*   [Computing Relevance](#h5ComputingRelevance)

*   [Supervised Approach](#h4SupervisedApproach)

*   [Course Recommendations](#h3CourseRecommendations)

*   [LinkedIn Learning](#h4LinkedInLearning)
*   [Collaborative Filtering](#h4CollaborativeFiltering)
*   [Response Prediction](#h4ResponsePrediction)

*   [Summary](#h2Summary)

*   [Citation Information](#h3Citation)

* * *

In this tutorial, you will learn about LinkedIn Jobs recommendation systems.

[![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20393'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/08/linkedin-job-rec-sys-featured.png)

LinkedIn has made it easier for professionals to connect and build their network, look for new job opportunities, get professional advice from others in the same field, and build connections to help them advance in their careers.

LinkedIn has also become a popular platform for recruiters to find new talent. It allows recruiters to search for candidates based on their skills, experience, and other criteria. This has made it easier for companies to find candidates for their job openings. In addition, LinkedIn provides a great platform to recruit top-notch talent, especially for startups, which don’t have enough visibility compared to multinational corporations.

Apart from making job and talent hunting easier, LinkedIn now offers a variety of resources for professionals to upgrade their skills and advance their careers. This includes online courses, certifications, and webinars that are up-to-date with industry standards. Furthermore, it helps professionals build their brands by helping them establish themselves as thought leaders in their fields and making it easier for others to find them online.

Behind the scenes are LinkedIn’s amazing recommendation systems (**Figure 1**) that provide professionals or recruiters with a personalized feed, jobs they are interested in, connections and courses that can help them advance their careers, the best candidates for their job description (for recruiters and companies), etc.

[![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)](https://lh4.googleusercontent.com/rX8uf-aoifOx0FTPIBdlNijyKxG6UqLTUEsduAmkT-VfWya3gTU7lkASX5FolEI_GAyFtwFW81B49tv7utxXk9VnWs7AO76DTiuAuUX_Pf9jx1OsXKhLf3PzTFPWNc3N6lpstl_GZpbGIEO58xbGYug)

**Figure 1:** LinkedIn jobs recommender systems (source: [Quora](https://www.quora.com/How-does-LinkedIns-recommendation-system-work)).

In this lesson, we will cover several aspects of LinkedIn recommendations (e.g., feed recommendations, talent recommendations, course recommendations) and how they work behind the scenes.

This lesson is the last in a 3-part series on **Deep Dive into Popular Recommendation Engines 101**:

1.  [_Fundamentals of Recommendation Systems_](https://pyimg.co/9kf03)
2.  [_Netflix Movies and Series Recommendation Systems_](https://pyimg.co/sqd7x)
3.  [**_LinkedIn Jobs Recommendation Systems_**](https://pyimg.co/9ip27) **(this tutorial)**

**To learn how LinkedIn recommendations work,** **_just keep reading._**

* * *

[**LinkedIn Jobs Recommendation Systems**](#TOCh2BPTitle)
---------------------------------------------------------

Most of the LinkedIn recommendations are powered by artificial intelligence (AI) in some way or another. For example, LinkedIn uses AI in ways its members experience daily, like recommending the right job opportunities, encouraging them to connect with someone (“People You May Know” feature), providing relevant content on their feed, providing course suggestions to upgrade their skills, suggesting recruiters with the best talent for their job openings, etc.

In this lesson, we will discuss LinkedIn’s three major recommendation systems: Feed, recruiter search, and course recommendations.

* * *

### [**Feed Recommendations**](#TOCh3Feed)

* * *

#### [**LinkedIn Feed**](#TOCh4Feed)

LinkedIn feed (**Figure 2**) is a sorted list of updates displayed to its members when they log in. It comprises a heterogeneous inventory of updates (e.g., articles shared by connected members, job recommendations, news recommendations, people you may know recommendations, stories mentioning companies the member follows, etc.).

The feed aims to keep LinkedIn members updated and informed in their professional world. Feed recommendation estimates the utility of an update for a particular member. It is similar to a recommendation system that suggests movies but with a few twists.

Most members interact heavily with the feed by scrolling down. Therefore, displaying where the member will likely engage in interactive sessions rather than the latest feed is very important. Furthermore, the feedback collected is based on members’ implicit actions (e.g., time spent on a feed, clicks, comments, likes, etc.) rather than explicit actions (e.g., rating the feed on a scale of 1-5 as done for movies).

Incorporating LinkedIn’s social graph in recommendations is another important aspect, as members often find updates shared, liked, or commented on by their connections relevant. Moreover, members are incentivized to update their profiles as it attracts other people and companies. This allows LinkedIn’s feed recommendation system to access high-quality data about members’ interests, skills, and backgrounds.

* * *

#### [**First Pass Rankers**](#TOCh4FPR)

The LinkedIn feed recommendation system (**Figure 3**) comprises two engines: First Pass Rankers (FPRs) and Second Pass Rankers (SPRs). First Pass Rankers are responsible for separately ranking items according to user relevance for each inventory (i.e., articles, news, jobs, connections, etc.). Top-k items from each inventory (ranked by FPRs) are then passed through Second Pass Rankers (SPRs), which combine all these outputs from FPRs to create a single personalized ranked list. After SPR, the ranked list is passed to a re-ranker that modifies the output of SPR and creates the final list of feeds to be displayed on the homepage.

**Figure 3** shows the general architecture of the feed recommendation system. First, multiple FPRs (in the left part of the figure) feed the top-k items from different inventories into SPRs that combine all these ranked lists into a single ranked list which is sent to the front.

Next, **Figure 4** focuses on the FPRs, which pass top-k items from different inventories to the feed mixer (second pass ranker + re-ranker). One of the important FPRs is “activities from your network,” which includes shares from people and companies that members follow. This source comprises 50% of updates shown in the LinkedIn feed.

The reason for having multiple FPRs is that different teams having different domain expertise can own different FPRs allowing them to work efficiently. Furthermore, having a second pass maker will enable LinkedIn to use additional features and complex ranking rules, which otherwise would be costly if done for each FPR.

* * *

#### [**Second Pass Ranker**](#TOCh4SPR)

The core of SPR (**Figure 5**) is a logistic regression model that takes the user profile (e.g., skills, interests, background), the feed candidate, and its FPR score as inputs and computes the probability that the member will interact with that update (e.g., like, comment, share, etc.).

LinkedIn uses several metrics to measure the engagement level of a member with a feed item (e.g., number of clicks, likes, comments, follows, connects, re-shares, and whether the user scrolled down to view at least 10 updates, etc.). These metrics can then calculate the overall interaction per user, known as Click Through Rate (CTR), which is the probability that a member will click/like/share/comment on a feed item shown to them.

Metrics that capture immediate reaction to the feed are called upstream metrics (e.g., interactions or CTR). In contrast, downstream metrics capture the delayed reaction of the user to the feed (e.g., engaged feed sessions). In general, it is much easier to predict and optimize upstream metrics than downstream metrics.

The logistic regression model estimates the CTR given user features and item features. Let ![Image 3: \hat{y}_{it}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2016%2016'%3E%3C/svg%3E) be a binary variable to represent the interaction between user ![Image 4: i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%205%2012'%3E%3C/svg%3E) and update ![Image 5: t](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2010'%3E%3C/svg%3E) (![Image 6: 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2013'%3E%3C/svg%3E) if there is an iteration, ![Image 7: -1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2019%2013'%3E%3C/svg%3E) otherwise). Assuming ![Image 8: X_{it}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2021%2014'%3E%3C/svg%3E) be the features corresponding to user ![Image 9: i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%205%2012'%3E%3C/svg%3E) and item ![Image 10: t](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2010'%3E%3C/svg%3E), the logistic regression model can be written as:

![Image 11: \log\left(\displaystyle\frac{P(y_{it}=1 | \text{user}=i, \text{update}=t)}{1-P(y_{it}=1 | \text{user}=i, \text{update}=t)}\right) = \sum_j \beta_j [X_{it}]_j,](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20403%2041'%3E%3C/svg%3E)

where ![Image 12: \beta_j](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%2016'%3E%3C/svg%3E) are the model parameters. The model is trained over fixed data collected over some time. For some ![Image 13: K](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2011'%3E%3C/svg%3E) updates shown to a member, the ones where the member interacted are considered as positives (![Image 14: y_{it} =1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2046%2015'%3E%3C/svg%3E)), and others are considered as negatives (![Image 15: y_{it} =0](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2047%2015'%3E%3C/svg%3E)).

* * *

#### [**Evaluation**](#TOCh4Evaluation)

Offline evaluation is important before deploying the model in an online setting. LinkedIn uses two metrics for offline evaluation. The first one is the O/E ratio which calculates the ratio of observed (predicted by the logistic regression model) and the expected probabilities (ground truth). If O/E is less than ![Image 16: 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2013'%3E%3C/svg%3E), the CTR model overestimates pCTR probabilities. If it is greater than ![Image 17: 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2013'%3E%3C/svg%3E), the CTR model underestimates pCTR probabilities.

The second metric is “Replay,” which runs the trained logistic regression model on random historical data and records when the new model recommends updates that the user interacted with.

* * *

#### [**Feature Representation**](#TOCh4FeatureRepresentation)

Each feed activity (e.g., connection, like, share, update, etc.) is represented as a triplet (actor ![Image 18: j](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2015'%3E%3C/svg%3E), verb ![Image 19: v](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%207'%3E%3C/svg%3E), and object ![Image 20: o](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%207'%3E%3C/svg%3E)). So, for example, a member (actor) shared (verb) an update (object), a member (actor) updated (verb) his profile (object), etc. **Table 1** shows the taxonomy of different feed activity triplets.

Let’s understand it more clearly by taking an example of activity ![Image 21: t](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2010'%3E%3C/svg%3E) about a job opening (for a Java developer) shared by Microsoft. Here actor ![Image 22: j](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2015'%3E%3C/svg%3E) is company Microsoft, verb ![Image 23: o](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%207'%3E%3C/svg%3E) is sharing of jobs, and object ![Image 24: o](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%207'%3E%3C/svg%3E) is the job description for Java developers.

Now, for a user ![Image 25: i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%205%2012'%3E%3C/svg%3E) and the activity ![Image 26: t](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2010'%3E%3C/svg%3E) (job posting from Microsoft), the feature ![Image 27: X_{it}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2021%2014'%3E%3C/svg%3E) used as input for the logistic regression model comprises the following features:

*   User-only feature, ![Image 28: X_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2017%2014'%3E%3C/svg%3E), represents user profile, interests, skills, etc. (e.g., it will include features like if the user has expertise and experience in Java).
*   Activity-only feature, ![Image 29: X_t](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2017%2014'%3E%3C/svg%3E), representing activity type (![Image 30: k](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%2011'%3E%3C/svg%3E)), actor type (![Image 31: j](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2015'%3E%3C/svg%3E)), verb type (![Image 32: v](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%207'%3E%3C/svg%3E)) and creation time, FPR score, activity impression time, popularity, etc. (e.g., how many likes, comments, and shares the job posting received).
*   User-actor feature, ![Image 33: X_{ij}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2022%2016'%3E%3C/svg%3E), representing the interaction between user ![Image 34: i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%205%2012'%3E%3C/svg%3E) and actor ![Image 35: j](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2015'%3E%3C/svg%3E) (of update ![Image 36: t](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2010'%3E%3C/svg%3E)) (e.g., whether the user follows Microsoft or not).
*   User-activity feature, ![Image 37: X_{ik}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2023%2014'%3E%3C/svg%3E), captures the interaction between user ![Image 38: i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%205%2012'%3E%3C/svg%3E) and activity type ![Image 39: k](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%2011'%3E%3C/svg%3E) (e.g., whether the user is looking for a job or has recently applied to any job).
*   User-actor-activity feature, ![Image 40: X_{ijk}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2029%2016'%3E%3C/svg%3E), captures the interaction between user, actor, and activity type (e.g., whether the user applied to a job posting by Microsoft in the past or recently).
*   User-object feature, ![Image 41: X_{io}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2023%2014'%3E%3C/svg%3E), capturing the interaction between user and object (e.g., whether the user is apt for the job posting).

The feature ![Image 42: X_{it}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2021%2014'%3E%3C/svg%3E) is thus the concatenation of the above features, that is,

![Image 43: X_{it} = (X_i, X_t, x_{ij}, X_{ik}, X_{ijk}, X_{io})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20226%2019'%3E%3C/svg%3E)

* * *

### [**Recruiter Search Recommendations**](#TOCh3Recruiter)

Recruiter Talent Search and Recommendations is one of the major offerings by LinkedIn that contributes to around 65% of its annual revenue. It provides a marketplace to connect job seekers with job providers by enabling recruiters and companies to reach out to potential candidates for their job openings.

* * *

#### [**LinkedIn Talent Search and Recommendations**](#TOCh4TalentSearch)

LinkedIn talent search recommendation system (**Figure 6**) provides a ranked list of potential candidates corresponding to a search query, which can be a text query, job description, etc. Given a search request (e.g., a software developer with a machine learning background ready to join in California), the candidates are ranked based on their experience with machine learning and expertise as a software developer, similarity of their work, living in California, and likelihood that they will respond to the job description.

For each result shown by the recommender system, the recruiter can then perform three actions:

*   View the candidate’s profile
*   Bookmark or save the candidate’s profile for further evaluation
*   Send an InMail to the candidate

All these actions tell us that the recruiter finds the recommended candidate relevant for the job opening. However, suggesting a list of potential candidates that match the search query is inefficient if most are not interested in applying. Therefore, recommending talent to recruiters requires mutual interest between the recruiter and the candidate.

Therefore, a new feature, “InMail Accept,” is provided by LinkedIn to record the event when the candidate positively responds to the InMail request sent by the recruiter. Thus a candidate recommendation is said to be relevant when the recruiter sends an InMail and candidates positively respond to it.

The nature of search queries is usually complex, involving a combination of several structured fields (e.g., location, experience, etc.) and unstructured fields (e.g., free text). This makes the association between search queries and interested candidates complex and non-linear. Thus, LinkedIn leverages neural networks that can handle complex queries and capture the non-linear relationship between the query and the candidates.

Given a query ![Image 44: q](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2010'%3E%3C/svg%3E) by a recruiter, we want to rank a list of LinkedIn members ![Image 45: m_1, m_2,\dots, m_d](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20105%2011'%3E%3C/svg%3E) in order of decreasing relevance. In other words, we want to learn a function that assigns scores to each candidate based on their relevance to the query provided by the recruiter. To achieve this, LinkedIn knows the dense vector representations of question and member in a common subspace and then performs the final scoring afterward (e.g., cosine similarity between both vectors). There are two approaches to learning these representations: unsupervised and supervised.

* * *

#### [**Unsupervised Approach**](#TOCh4UnsupervisedApproach)

LinkedIn entities like skill, title, company, institution or school, and location (mentioned in query and member profile) are categorical and suffer from sparsity because of the large search space. This makes learning dense embeddings for these entities difficult to use traditional approaches like [Word2vec](https://pyimagesearch.com/2022/07/11/word2vec-a-study-of-embeddings-in-nlp/). LinkedIn thus uses its economic graph to learn dense representations for these entities.

* * *

##### [**LinkedIn’s Economic Graph**](#TOCh5EconomicGraph)

[LinkedIn’s Economic Graph](https://economicgraph.linkedin.com/) is a digital representation of its economy based on data from over 500 million members, tens of thousands of standardized skills, employers and open jobs, and tens of thousands of educational institutions, along with the relationships between these entities. Vertices represent the network members, and edges represent the intensity of interaction (e.g., clicks, likes, shares, social actions, etc.) between the two members.

However, the economic graph is large and sparse, which results in intractable models and noisy embeddings. Hence, LinkedIn uses an entity graph, a sub-network of the economic graph representing only entities (skill, title, company) whose dense representations need to be learned.

**Figure 7** shows the entity graph representing companies. Each vertex represents a company, and an edge between two companies represents the LinkedIn members who have worked in both companies. In this example, the aim will be to calculate the dense representation of each company. Similar graphs can be constructed for other entity types (e.g., skills and schools).

* * *

##### [**First-Order Embeddings**](#TOCh5First-OrderEmbeddings)

First-order embeddings are generated based on the observation that vertices connected by an edge are similar. We define a joint probability distribution between vertices ![Image 46: v_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2010'%3E%3C/svg%3E) and ![Image 47: v_j](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2012'%3E%3C/svg%3E) connected by an edge of weight ![Image 48: w_{ij}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2021%2012'%3E%3C/svg%3E) as follows:

![Image 49: p_1(v_i, v_j) = \displaystyle\frac{\sigma(\langle u_i, u_j \rangle)}{\sum_{(v_m, v_n) \in E} \sigma(\langle u_m, u_n \rangle)},](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20250%2045'%3E%3C/svg%3E)

where ![Image 50: u_i \in \Re^d](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2052%2017'%3E%3C/svg%3E) is the ![Image 51: d](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%2011'%3E%3C/svg%3E)\-dimensional embedding of vertex ![Image 52: v_i, \langle .,. \rangle](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2048%2018'%3E%3C/svg%3E) represents the dot product between two vectors, and ![Image 53: \sigma(x)  = \frac{1}{1+\exp(-x)}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20113%2023'%3E%3C/svg%3E) is the sigmoid function. Set ![Image 54: E](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2011'%3E%3C/svg%3E) consists of all edges of the entity graph. Next, we can define an empirical joint distribution between vertices ![Image 55: v_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2010'%3E%3C/svg%3E) and ![Image 56: v_j](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2012'%3E%3C/svg%3E) as follows:

![Image 57: \hat{p}_1(v_i, v_j) = \displaystyle\frac{w_{ij}}{\sum_{(v_m, v_n) \in E} w_{mn}}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20195%2038'%3E%3C/svg%3E)

Finally, to learn embeddings ![Image 58: u_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2010'%3E%3C/svg%3E), we minimize the Kullback-Leibler (KL) divergence between ![Image 59: p_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%2011'%3E%3C/svg%3E) and ![Image 60: \hat{p}_1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%2017'%3E%3C/svg%3E) with respect to embeddings to preserve the first-order proximity:

![Image 61: O_1 = \sum_{(v_i, v_j) \in E}  \hat{p}_1(v_i, v_j) \log \displaystyle\frac{p_1(v_i, v_j) }{\hat{p}_1(v_i, v_j)}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20264%2041'%3E%3C/svg%3E)

* * *

##### [**Second-Order Embeddings**](#TOCh5Second-OrderEmbeddings)

Second-order embeddings are generated based on the observation that vertices having shared neighbors are similar. In this case, a vertex plays two roles: the vertex itself and the specific context for the other vertex. To represent this, we define the probability of a context vertex ![Image 62: v_j](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2012'%3E%3C/svg%3E) being generated by a vertex ![Image 63: v_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2010'%3E%3C/svg%3E) as follows:

![Image 64: p_2(v_j | v_i) = \displaystyle\frac{\exp(\langle u^\prime_j, u_i \rangle)}{\sum_{k=1}^\vert V \vert \exp(\langle u^\prime_k, u_i \rangle)},](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20240%2047'%3E%3C/svg%3E)

where ![Image 65: u^\prime_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2017'%3E%3C/svg%3E) represents the second-order embedding of vertex ![Image 66: v_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2010'%3E%3C/svg%3E) used when it is treated as a context, and ![Image 67: \vert V \vert](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2020%2017'%3E%3C/svg%3E) is the total number of vertices in the entity graph.

The empirical probability can be written as follows:

![Image 68: \hat{p}_2(v_j | v_i) = \displaystyle\frac{w_{ij}}{\sum_{v_k: (v_i, v_k) \in E} w_{ik}}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20194%2038'%3E%3C/svg%3E)

Finally, to learn embeddings ![Image 69: u_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2010'%3E%3C/svg%3E) and ![Image 70: u^\prime_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2017'%3E%3C/svg%3E), we minimize the KL divergence between ![Image 71: p_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2010'%3E%3C/svg%3E) and ![Image 72: \hat{p}_2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2016'%3E%3C/svg%3E) with respect to embeddings to preserve the second-order proximity:

![Image 73: O_2 = \sum_{v_i \in V} \lambda_i \cdot \sum_{v_j: (v_i, v_j) \in E} \hat{p}_2(v_j | v_i) \log \displaystyle\frac{p_2(v_j | v_i) }{\hat{p}_2(v_j | v_i)},](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20352%2041'%3E%3C/svg%3E)

where ![Image 74: \lambda_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2014'%3E%3C/svg%3E) is the degree of vertex ![Image 75: v_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2010'%3E%3C/svg%3E) representing its importance in the entity graph.

* * *

##### [**Computing Relevance**](#TOCh5ComputingRelevance)

After obtaining the first- and second-order embeddings ![Image 76: u_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2010'%3E%3C/svg%3E) and ![Image 77: u^\prime_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2017'%3E%3C/svg%3E) for a vertex ![Image 78: v_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2010'%3E%3C/svg%3E), each entity (the company in our case) can now be represented as a single vector by concatenating its first- and second-order embeddings ![Image 79: [u_i, u^\prime_i]](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2042%2018'%3E%3C/svg%3E).

Each query and member can be represented by a bag of companies (i.e., a query can contain multiple companies referenced in the search terms, and a member could have worked at multiple companies, which is manifested on the profile).

Thus, we can represent each query and member as a point on vector space with a simple pooling operation (max-pooling or mean-pooling) over the bag companies. A similarity function between two vector representations can be used as a feature ranking.

* * *

#### [**Supervised Approach**](#TOCh4SupervisedApproach)

The supervised approach (**Figure 8**) to learning dense representations for query and member involves training three neural networks:

*   A query network ![Image 80: \phi_q(.)](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2032%2019'%3E%3C/svg%3E) that embeds a given query to a fixed dimensional latent space.
*   A member network ![Image 81: \phi_m(.)](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2037%2018'%3E%3C/svg%3E) embeds a given member to a fixed dimensional latent space.
*   A cross-network ![Image 82: f(.)](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2026%2018'%3E%3C/svg%3E) that takes a query and member embeddings and computes their relevance score.

[![Image 83](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20603%20500'%3E%3C/svg%3E)](https://lh4.googleusercontent.com/PpAXCNwCKepewML8weIYMT1I8KvBDQG49Fw5zxbuWnl4w54_KEvis8nXDW8Wt-HrHAm-jnMhMJW7xw6ZoV-yEw1Go_n4RPsI7y6vwOSnSgOp5QFZnwHFdOjDt8U01pzbIY1vKWUFI4D2BGGYmiiJSRI)

**Figure 8:** Supervised approach for learning entity representations (source: [Ramanath et al.](https://arxiv.org/pdf/1809.06473.pdf)).

In other words, the predicted relevance score ![Image 84: \hat{y}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2016'%3E%3C/svg%3E) for a query ![Image 85: q](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2010'%3E%3C/svg%3E) and member ![Image 86: m](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2014%207'%3E%3C/svg%3E) is computed as:

![Image 87: \hat{y} = f(\phi_q(q), \phi_q(m))](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20137%2019'%3E%3C/svg%3E)

The cross-network can be a neural network of a similarity function such as cosine similarity.

These three models are trained over historically collected triplets ![Image 88: (q_i, m_i, y_i)](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2068%2018'%3E%3C/svg%3E), where ![Image 89: q_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2011%2010'%3E%3C/svg%3E) is the query provided by the recruiter, ![Image 90: m_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2010'%3E%3C/svg%3E) is a member profile, and ![Image 91: y_i](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2010'%3E%3C/svg%3E) is a binary label which is ![Image 92: 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2013'%3E%3C/svg%3E) if the member was relevant to query and also accepted recruiter InMail request and ![Image 93: 0](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2012'%3E%3C/svg%3E) otherwise (either candidate is not relevant to the query or s/he didn’t accept the InMail).

* * *

### [**Course Recommendations**](#TOCh3CourseRecommendations)

* * *

#### [**LinkedIn Learning**](#TOCh4LinkedInLearning)

LinkedIn Learning (**Figure 9**) is a platform that allows members to develop relevant skills through courses to advance their professional careers. Over the last few years, LinkedIn has developed a course recommendation system that recommends relevant courses to its members based on their interests and learning aspirations.

The goal of the course recommendation system is to provide personalized courses to its members so that it helps them advance their professional careers. The recommendation system (**Figure 10**) consists of three major blocks:

*   **Collaborative filtering:** This block uses collaborative filtering algorithms to recommend courses to a member based on preferences from many users.
*   **Response prediction:** This system predicts the member-course relevance using the learner’s profile features (e.g., skills and industry) and course metadata (e.g., course difficulty, category, and skills).
*   **Blending:** This block combines the recommendations from collaborative filtering and response prediction to determine every learner’s final set of courses. This is an online component, so the offers from each model are fetched upon request.

* * *

#### [**Collaborative Filtering**](#TOCh4CollaborativeFiltering)

Typically there are three collaborative filtering algorithms: user-item-based utility, matrix factorization technique, and deep neural network-based approach. However, due to the popularity and efficacy of deep learning algorithms, LinkedIn adopts a neural collaborative filtering approach (**Figure 11**) consisting of three networks: course, learner, and output.

Learner and course networks input course and learner features and generate learner and course embeddings. The output network then computes the ranking score between the learner and course embeddings, which creates a list of course recommendations for each learner.

The input to the learner’s network is a sparse vector of all courses watched by that learner in the past year (**Figure 12**). For example, for a learner who has watched two courses, the learner input vector will have non-zero values for those courses and zero for others. The input to the course’s network is a vector representing the similarity of that course with other courses (**Figure 12**). These similarities are computed beforehand based on their co-watching patterns.

LinkedIn generates binary labels for all learner-course pairs based on historical learner watch history to train the collaborative filtering model. A label of ![Image 94: 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2013'%3E%3C/svg%3E) denotes that the course is relevant to the learner, whereas a label of ![Image 95: 0](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2012'%3E%3C/svg%3E) denotes the course is not relevant to the learner.

A course is considered relevant for the learner if the learner has watched the course up to a certain duration. If a learner has only watched the first 10 seconds of a course, that engagement does not impact the model like viewing a full course session.

* * *

#### [**Response Prediction**](#TOCh4ResponsePrediction)

Response prediction aims to estimate the relevance of a course for a learner by using learner-level features (e.g., skills, industry, etc.) and course-level features (e.g., course difficulty, course category, and course skills, etc.). LinkedIn uses the Generalized Linear Mixture Model (GLMix) (**Figure 13**) for response prediction.

Apart from learning fixed or global model coefficients, GLMix also knows learner-level coefficients based on their engagements with courses and course-level coefficients based on engagement actions of a course. Learning learner and course-level coefficients are important for personalization as they capture the learner’s unique interests.

The predicted score in GLMix is the sum of the three components: global, learner-level, and course-level models.

* * *

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

* * *

[**Summary**](#TOCh2Summary)
----------------------------

In this lesson, we discussed LinkedIn recommendation systems. LinkedIn uses AI in ways its members see every day:

*   Recommending the right job opportunities
*   Encouraging them to connect with someone (“People You May Know” feature)
*   Providing relevant content on their feed
*   Providing course suggestions to upgrade their skills
*   Suggesting recruiters with the best talent for their job openings

We discussed how recommendation systems are used for LinkedIn feeds. The feed is a sorted list of heterogeneous updates (e.g., news, articles, jobs, connections, etc.) displayed to its members when they log in.

The LinkedIn feed recommendation system comprises two engines: First Pass Rankers (FPRs) and Second Pass Rankers (SPRs).

First Pass Rankers are responsible for separately ranking items according to user relevance for each inventory (e.g., articles, news, jobs, connections, etc.). Top-k items from each inventory (ranked by FPRs) are then passed through the Second Pass Ranker (SPR), which combines all these outputs from FPRs to create a single personalized ranked list.

Next, we discussed LinkedIn talent search and recommendations that provide a marketplace to connect job seekers with job providers by enabling recruiters and companies to reach out to potential candidates for their job openings. The talent search and recommendation system learns a function that assigns scores to each candidate based on their relevance to the query provided by the recruiter.

To achieve this, LinkedIn learns dense vector representations of query and member in a common subspace and then performs final scoring afterward. The unsupervised approach involves leveraging the entity graph, a sub-network of the economic graph representing only entities (skill, title, company) whose dense representations need to be learned.

The supervised approach involves learning dense representations for query and member by training neural networks over historically collected query-member pairs and their corresponding binary label (a ![Image 96: 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%206%2013'%3E%3C/svg%3E) for member was relevant to the query and also accepted recruiter InMail request and ![Image 97: 0](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2012'%3E%3C/svg%3E) otherwise).

Lastly, we discussed LinkedIn course recommendations that allow members to develop relevant skills through courses to advance their professional careers. The recommendation system consists of two algorithms: collaborative filtering and response prediction.

In collaborative filtering, LinkedIn adopts a neural collaborative filtering approach that takes course and learner features as input and computes the ranking score between the learner and course embeddings, which is used to create a list of course recommendations for each learner.

In response prediction, it uses the Generalized Linear Mixture Model (GLMix) to estimate the relevance of a course for a learner by using learner-level features (e.g., skills, industry, etc.) and course-level features (e.g., course difficulty, course category, and course skills, etc.).

However, this is just the tip of the iceberg, as there is much more to LinkedIn recommendation systems.

Stay tuned for the next sequence of this course series, where we will discuss recommendation systems used on Amazon, YouTube, and Spotify.

* * *

### [**Citation Information**](#TOCh3Citation)

**Mangla, P.** “LinkedIn Jobs Recommendation Systems,” _PyImageSearch_, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, and R. Raha, eds., 2023, [https://pyimg.co/9ip27](https://pyimg.co/9ip27)

@incollection{Mangla\_2023\_LinkedInJobs,
  author = {Puneet Mangla},
  title = {LinkedIn Jobs Recommendation Systems},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha},
  year = {2023},
  url = {https://pyimg.co/9ip27},
}

* * *

![Image 98: Featured Image](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20276'%3E%3C/svg%3E)

### Unleash the potential of computer vision with Roboflow - Free!

*   Step into the realm of the future by [signing up or logging into your Roboflow account](https://roboflow.com/?ref=pyimagesearch). Unlock a wealth of innovative dataset libraries and revolutionize your computer vision operations.
*   Jumpstart your journey by choosing from our broad array of datasets, or benefit from PyimageSearch’s comprehensive library, crafted to cater to a wide range of requirements.
*   Transfer your data to Roboflow in any of the 40+ compatible formats. Leverage cutting-edge model architectures for training, and deploy seamlessly across diverse platforms, including API, NVIDIA, browser, iOS, and beyond. Integrate our platform effortlessly with your applications or your favorite third-party tools.
*   Equip yourself with the ability to train a potent computer vision model in a mere afternoon. With a few images, you can import data from any source via API, annotate images using our superior cloud-hosted tool, kickstart model training with a single click, and deploy the model via a hosted API endpoint. Tailor your process by opting for a code-centric approach, leveraging our intuitive, cloud-based UI, or combining both to fit your unique needs.
*   Embark on your journey today with absolutely no credit card required. Step into the future with Roboflow.

[Join Roboflow Now](https://roboflow.com/?ref=pyimagesearch)

* * *

![Image 99](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to **join the PyImageSearch Newsletter** and **download my FREE 17-page Resource Guide PDF** on Computer Vision, OpenCV, and Deep Learning.