Machine Learning for Fake Job Detection
By: Kafaru Simmie
Tags: NLP, Machine Learning, Data Science, Text Analytics, Classification

There are many use cases for Machine Learning Models. Text analytics, a branch of Natural Language Processing provides ways of allowing Machine Learning algorithms to be applied to textual data for classification models. Patterns exist in data in ways we can not ordinarily detect all the time but using analytical tools we are able to detect these existing patterns. These patterns have even been made easier and scalable using Machine Learning and a typical example we probably interface with every day is the spam classifier.

This article takes you through how you can use represent textual data into encodings that can then be fed into your Machine Learning model. The data for this can be downloaded here;

The first procedure is to do a slight exploratory data analysis. This is to familiarize us with the dataset. We load in our dataset and call the needed libraries;

library(tidytext)

library(tidyverse)

library(tidygraph)

library(tm)

library(SnowballC)

library(randomForest)



#Read in the dataset

posting <- read.csv("fake_job_postings.csv")



A snippet of the data

Once the data has been loaded, we will visualize the number of job posts across each industry. Note that 0 in the fraudulent column represents real jobs and 1 represents fake jobs.

Visualizing the top industries for job postings in both fake and real jobs according to the fraudulent column;

posting %>%

group_by(fraudulent) %>%

count(industry) %>%

arrange(desc(n)) %>%

top_n(10) %>%

mutate_if(is.character, list(~na_if(.,""))) %>%

mutate(total = paste(round(100*n/sum(n), 1), "%", sep = "")) %>%

filter(fraudulent == 1)%>%

ggplot(., aes(reorder(industry, n), n, fill = industry)) +

geom_bar(stat = "identity", show.legend = F) +

coord_flip() +

geom_text(aes(label = total), color = "black", size = 4) +

labs(title = "Fake Posts", subtitle = "Count by Industry",

x = 'Industry') + theme_classic()

Prevalent industries in Fake Job Posts

Visualizing top industries for real posts;

Prevalent industries in Real Job Posts

We can see that for both cases, a larger percentage of the jobs do not specify their industries and thus it is not a good metric to judge is a job notification is real or fake.

Taking a look at the job requirement for each job post type i.e. real or fake, we dig in to find some information;

#Job Requirements

posting %>%

group_by(fraudulent) %>%

count(required_experience) %>%

mutate(percentage = paste(round(100*n/sum(n), 1), "%", sep = "")) %>%

mutate_if(is.character, list(~na_if(.,""))) %>%

ggplot(., aes(reorder(required_experience, n), log(n), fill = factor(fraudulent))) +

geom_bar(stat = "identity") +

geom_text(aes(label = percentage)) +

labs(x = "Job Requirement", title = "Jobs Requirement") +

scale_fill_discrete(name = "Dose", labels = c("Not Fake", "Fake")) +

coord_flip() +

theme_classic()

A logged value was taken so as to better visualize the axis. From here we can also see that a larger percent of real jobs do not specify their educational requirements.

Examining the job posts categories where companies had logos. Interestingly, you find that most of the companies with actual job posts have no logos while it is opposite for fake posts. Perhaps to make sure they look genuine? I mean, any shady business would want to cover their tracks well, thus, the need for the logo.

From here on we move on to the actual preprocessing procedure, so we can feed the data into a machine learning algorithm.

Data Preprocessing/Cleaning

We will remove stop words, punctuations, hyperlinks, and other unwanted words first.

#Data Cleaning

corpus <- Corpus(VectorSource(posting$description)) inspect(corpus[1:5]) #Convert to lower cast

corpus <- tm_map(corpus, tolower) #Remove Punctuationa and inspect

corpus <- tm_map(corpus, removePunctuation) inspect(corpus[1:5]) #Remove stopwords

corpus <- tm_map(corpus, removeWords, stopwords(kind = "en")) #Stem document

corpus <- tm_map(corpus, stemDocument)

We will create a term-document matrix of the cleaned data which is a major preprocessing stage

#Create document frequency

freq <- DocumentTermMatrix(corpus)

freq #remove sparsity

freq_sparsed <- removeSparseTerms(freq, sparse = 0.995) freq_sparsed #Convert cleaned document to a df

df <- as.data.frame(as.matrix(freq_sparsed)) #Give unique names to colnames

colnames(df) <- make.names(colnames(df)) #Add the fraudelent colum

df$fradulent <- posting$fraudulent #Remove duplicate column names

colnames(df) <- make.unique(colnames(df), sep = "_")

Term Document Matrix

The image above is what your new dataframe should look like when you take perform the Term-Document Frequency operation.

Splitting for Train and Test Set

We will use the split component on the caTools library and caret for our ML model;

#Create train and test dataset

library(caTools)

library(caret) #Convert target variable to factor type

df$fradulent <- as.factor(df$fradulent) #Split

set.seed(2020, sample.kind = "Rounding")

test_index <- createDataPartition(y = df$fradulent, times = 1, p = 0.1, list= FALSE)

train_set <- df[-test_index, ]

validation <- df[test_index, ] #Check the split ratio of the target variable

table(train_set$fradulent)

table(validation$fradulent)

Because of the computational time and also the presence of a huge class imbalance, we will just use 1000 random samples of the data, you can however decide to train on the entire dataset if that suits you.

library(caret)

#Get a sample number

n <- 1000 #Cross validation contro;

control<- trainControl(method = "cv", number = 5, verboseIter = TRUE) #Number of trees for each train

grid <-data.frame(mtry = c(1, 5, 10, 25, 50, 100)) #create a sample from n number from dataset

index <- sample(nrow(train_set), n) #subset data

rf_train_data <- train_set[index, ] #Train random forest model

subset_train_rf <- train(fradulent ~ ., method = "rf", data = rf_train_data, ntree = 150, trControl = control, tuneGrid = grid) #Accuracy on the sample train data

prediction <- predict(subset_train_rf, rf_train_data) #Confusion matrix

confusionMatrix(prediction, rf_train_data$fradulent)

Accuracy from Confusion Matrix on the train data

A 97.6% on the train data looks good but we need to see how well it performs on the earlier split of the validation set (Although we used a small set of 1000 rows to train the data).

#predict on the train set

prediction01 <- predict(subset_train_rf, validation) #confusion matrix

confusionMatrix(validation$fradulent, prediction01)

Accuracy from Confusion Matrix on Validation data

Interestingly, the model still performs well with the validation set of over 1800 sample size.

You can also go ahead and use other models but in my model, I used the K-fold cross-validation to handle overfitting largely possible due to class imbalance. Lastly, the grid search was used to select the optimum hyperparameters for the model.

Conclusion

Notice that only the description column for the features alongside the target column which is the fraudulent column are the only columns used. However, you can decide to use all the other columns either by encoding them. For example columns such as has_company_logo and telecommuting can just be appended to the data frame after creating the Term-Document and thus included. Other columns such as industry can simply be encoded and appended.

You should however consider that you’re creating more features and thus increase in computational time.

Gain Access to Expert View — Subscribe to DDI Intel