# Automatic Detection of Online Recruitment Frauds: Characteristics, Methods, and a Public Dataset
## 6 Analysis and Experimentation
### Machine Learning Analysis of the Empirical Ruleset  
For the second experiment, we transformed each record of the balanced dataset to a vector of binary features. As shown in Table 6, each feature derives from the rules presented in Section 6.2. For example, corporate logo presence shown in Figure 5a was transformed into the has-no-company-logo binary feature for the new model. The new model is more compact and can scale better on large datasets as it requires significantly lower space and computational resources in comparison to a bow model. The model was tested against the same six classifiers. The dataset was partitioned into training and cross-validation subsets using the k-fold cross-validation strategy (k = 10). The results are summarized in Tables 7 and 8. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Category** & **Name** & **Description** \\ \hline \multirow{4}{*}{Linguistic} & contains-spamwords & Set if a spamword such as “easy money” is present in job text. \\  & has-consecutive-punctuation & Set when consecutive punctuation such as “\(\sharp\)!!” is spotted. \\  & contains-money-in-title & Set when money symbols are detected in job title. \\  & contains-money-in-description & Set when money symbols are detected in job description. \\ \hline \multirow{4}{*}{Contextual} & has-no-company-profile & Set when company profile field is empty. \\  & has-short-company-profile & Set if company profile is less than 10 words. \\  & has-no-long-company-profile & Set if company profile is not short and is less than 100 words. \\  & has-short-description & Set if job description is less than 10 words. \\  & has-short-requirements & Set if job requirements are less than 10 words. \\  & contains-email-link & Set when email links are present in job text. \\  & prompts-for-external-application & Set when phrases such as “apply at” or “send resume” are detected. \\  & addresses-lower-education & Set when phrases such as “High School or equivalent” are detected. \\  & has-incomplete-extra-attributes & Set when designated job attributes are empty. \\ \hline \multirow{4}{*}{Metadata} & located-in-us & Set if job is located in the US. \\  & is-telecommuting & Set when the job is marked as a telecommuting job. \\  & has-no-company-logo & Set if company logo is missing. \\  & has-no-questions & Set if screening questions are missing. \\ \cline{1-1}  & has-emphasized-description & Set when high emphasized word ratio in job description is higher than 0.5. \\ \cline{1-1}  & has-emphasized-requirements & Set when high emphasized word ratio in job requirements is higher than 0.5. \\ \cline{1-1}  & has-emphasized-benefits & Set when high emphasized word ratio in job benefits is higher than 0.5. \\ \cline{1-1}  & has-no-html-lists-in-requirements & Set when job requirements do not contain HTML lists. \\ \cline{1-1}  & has-no-html-lists-in-benefits & Set when job benefits do not contain HTML lists. \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Ruleset based binary features. Figure 5: Binary rules from non text attributes and their distribution among the classes. (**a**) Corporate logo; (**b**) Screening questions; (**c**) Salary mentions; (**d**) Fulfilled designated job attributes; (**e**) Prompts external application; (**f**) Contains email links. In comparison to bow modeling presented in Section 6.1, the second experiment showed increased performance with higher accuracy for all but one of the tested classifiers. In further detail, all classifiers significantly increased their achieved accuracy by a margin of 2%-13%. Only RF presents a small decrease of 0.5% which could be product of the sample chosen. Also, notice that ZeroR is not a real classifier as it naively assigns labels to the majority class (in this case the first class) and was included to provide a baseline. In order to evaluate in detail the effectiveness of each feature deriving from the empirical ruleset, we've also performed Pearson's correlation feature analysis using Weka for the random forest classifier.