SlowGuess/NAACL_2022_OCR huggingface dataset:
2014, volume 32 of JMLR Workshop and Conference Proceedings, pages 1188-1196. JMLR.org.
Junhua Liu, Yung Chuen Ng, Kristin L Wood, and Kwan Hui Lim. 2019a. Ipod: An industrial and professional occupations dataset and its applications to occupational data mining and analysis. *ArXiv* preprint, abs/1910.10495.
Mengshu Liu, Jingya Wang, Kareem Abdelfatah, and Mohammed Korayem. 2019b. Tripartite vector representations for better job recommendation. ArXiv preprint, abs/1907.12379.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European semantic web conference, pages 593-607. Springer.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net.
Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al. 2019a. Deep graph library: A graph-centric, highly-performant package for graph neural networks. ArXiv preprint, abs/1909.01315.
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019b. Heterogeneous graph attention network. In The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019, pages 2022-2032. ACM.
Han Xiao. 2018. bert-as-service. https://github.com/hanxiao/bert-as-service.
Denghui Zhang, Junming Liu, Hengshu Zhu, Yanchi Liu, Lichen Wang, Pengyang Wang, and Hui Xiong. 2019. Job2vec: Job title benchmarking with collective multi-view representation learning. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019, pages 2763-2771. ACM.
Le Zhang, Ding Zhou, Hengshu Zhu, Tong Xu, Rui Zha, Enhong Chen, and Hui Xiong. 2021. Attentive heterogeneous graph embedding for job mobility prediction. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 2192-2201.
Jun Zhu, Gautier Viaud, and Céline Hudelot. 2021. Improving next-application prediction with deep personalized-attention neural network. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 1615-1622. IEEE.
A.1 Job Title Label Assignment
Job titles are not pre-labeled in the original working experience dataset provided by CareerBuilder12. Therefore, for the job title classification task, we use an online third-party API O*Net-SOC AutoCoder to assign a Standard Occupation Classification code (SOC) 2018 to each job title, as well as a match score (i.e., scores above 70 means that the correct code is accurately predicted at least  $70\%$  of the time). SOC 2018 is a four-level taxonomy structure, including MajorGroup (23), MinorGroup (98), BroadGroup (459) and DetailedOccupation (867). For example, O*Net-SOC AutoCoder assigns the code 11-2022 (Sales Managers) for the title "sales director", which belongs to the level of DetailedOccupation. 11-2020 (Marketing and Sales Managers) is BroadGroup level, 11-2000 (Advertising, Marketing, Promotions, Public Relations, and Sales Managers) is MinorGroup level, and 11-0000 (Management Occupations) is MajorGroup level. In this work, we categorize job titles into MajorGroup. We have annotated a total of 30,000 job titles. The developer guarantees that the code assigned to the title plus description has an accuracy rate of  $85\%$ . However, only the job title is provided in our experiments, so the SOC 2018 code may be incorrectly assigned. For this reason, we filtered out job titles with scores below 70. Therefore, 12,908 unique job titles remain.
A.2 Randstad Data Description
Figure 3 shows an example of parsed resume in Randstad dataset. We build graphs from EmploymentHistory, which contains a JobTitle, and its corresponding occupation labels (i.e., JobCode, JobGroup and JobClass). The hierarchical taxonomy structure used in the Randstad dataset has a three-level hierarchy, where JobCodes are leaf classes, and each internal class (i.e., JobGroup) or root class (i.e., JobClass) is the aggregation of all its descendant classes. There are 25 JobClasses, 295 JobGroups and 4,443 JobCodes, respectively. In this work, we categorize job titles into JobClass.
A.3 Tag Generation
For both datasets, we first tokenize titles into tokens and remove stopwords, numbers, and punctuation. The word frequency distribution of words
Figure 3: An example of parsed resume in Randstad.
in two datasets are shown respectively in Figure 4, which are subject to the long-tail distribution, similar to the observation in (Zhang et al., 2019). Most words appear only once, i.e.,  $53.55\%$  of words only appear one time in CB12 dataset, and this ratio is  $56.55\%$  in Randstad dataset. Figure 4 further shows the top ten and last ten frequent words in each dataset. Obviously, high-frequency words like "manager" and "sales" describe the responsibility or functionality of the job title, while low-frequency words are usually noise or person-specific words. Based on the domain-specific NE tags (i.e., RESponsibility, FUNction) proposed in IPOD (Liu et al., 2019a), we then select the Top-200 tokens that appear most frequently and appear in the IPOD NE tag set as tags for each dataset.
(a) CB12.
(b) Randstad.
Figure 4: Word frequency distribution, where the red line represents the average value. Top-10 words are sorted by frequency, and Last-10 are randomly selected from the words with a frequency of 2.
A.4 Baseline Description
We explore the following network embedding methods on our proposed graphs to learn job title representation. According to the type of graph, the network embedding methods are naturally divided into Homogeneous and Heterogeneous. Then, we further categorize each category into Unsupervised and Semi-Supervised according to whether node
http://www.onetsocautocoder.com/plus/onetmatch
labels are provided for learning.
Homogeneous&Unsupervised
- Node2Vec (N2V) (Grover and Leskovec, 2016): is an extension of DeepWalk with a biased random walk process for neighborhood exploration.
Homogeneous&Semi-supervised:
- GCN (Kipf and Welling, 2017): is a semi-supervised Graph Neural Network (GNN) that generalizes the convolutional operation to homogeneous graphs.
- GAT (Velickovic et al., 2018): uses a self-attention strategy to learn the importance between a node and its neighbors.
Heterogeneous&Unsupervised:
- Metapath2Vec (M2V) (Dong et al., 2017): performs meta-path-guided walks and utilizes SkipGram to embed heterogeneous graphs.
Heterogeneous & Semi-supervised:
-  $R G C N$  (Schlichtkrull et al., 2018): is an extension of  $G C N$  on heterogeneous graphs, introducing relation-specific transformations based on the type of edges.
- HAN (Wang et al., 2019b): proposes a hierarchical attention mechanism, i.e., node-level and semantic-level for heterogeneous graphs.
In addition to the comparison between network embedding methods, we will also compare the representation learned through graphs with the representation obtained by semantic-based methods.
Semantic-based:
- Word2Vec (W2V) (Le and Mikolov, 2014): the representation of a job title is obtained by averaging word vectors in it. We use word vectors trained on Google News  $^{7}$  for CB12, and a pretrained French embedding model (Fauconnier, 2015) for Randstad.
- BERT (Devlin et al., 2019): the job title representations are obtained by using the bert-asservice package (Xiao, 2018), a sentence encoding service for mapping variable-length sentences to fixed-length vectors. We default to using the pre-trained BERT models provided by the package, i.e., BERT-Base-Uncased is used for CB12, and BERT-Base-Multilingual-Based (New) for Randstad.
A.5 Parameter Settings
Our implementation is based on the PyTorch version of the DGL package (Wang et al., 2019a). For job title classification, we randomly split the data into training/validation/test sets with a ratio of  $60\% / 20\% / 20\%$ . We keep the same split ratio on positive/negative edges for next-job prediction, where negative edges are randomly picked from unconnected node pairs (i.e., the same size as positive edges). To ensure fairness, we keep the same data split for all methods, and we set the node embedding to 128 for all methods, except for W2V.
In the job title classification and next-job prediction tasks, for unsupervised methods, node representations are learned from the entire dataset. The logistic regression classifier is then trained on both the training and validation sets. Each semi-supervised model is trained on the training set, and the parameters are optimized on the validation set. The final performance is evaluated on the test set. Models are optimized with the Adam (Kingma and Ba, 2015) with a learning rate of 1e-3, and we apply  $L_{2}$  regularization with value 5e-4. We use an early stop with a patience of 100, i.e., if the validation loss does not decrease in 100 consecutive epochs, we stop training. For models applying the attention mechanism, the dropout rate of attention is set to 0.2. For random-walk-based methods, including  $N2V$  and  $M2V$ , we set the window size to 5, walk length to 10, walks per node to 50, the number of negative samples to 5. For  $M2V$ , we test all metapaths and report the best performance. For next-job prediction task, edge features are represented by applying binary operators (Grover and Leskovec, 2016) on pairs of nodes, and then the best operator is chosen based on the validation set, while the dot product is used for all semi-supervised methods.
We repeat each prediction experiment ten times and report the average performance scores (i.e., Macro-F1 and Micro-F1 for job title classification and AUC for next-job prediction).
$^{7}$ https://code.google.com/archive/p/word2vec/