# Education-to-Skill Mapping Using Hierarchical Classification and Transformer Neural Network
## 2 Materials and Methods
### Data  
The experiments were performed on national-level Lithuanian administrative data consisting of information about residents' acquired education and occupations. The models \begin{table}
\begin{tabular}{l l l l l} \hline \hline  & **Input** & **Model** & **Output** & **Loss** \\ \hline
1. Baseline & Cat. & Linear & All & \(\mathcal{L}_{CE}\) \\
2. Simple w. CE & Cat. & DNN & All & \(\mathcal{L}_{CE}\) \\
3. Simple w. F & Cat. & DNN & All & \(\mathcal{L}_{F}\) \\
4. Hier. w. F & Cat. & DNN & Hier. & \(\mathcal{L}_{F}\) \\
5. Transformer & Text & BERT & All & \(\mathcal{L}_{F}\) \\
6. Transformer mod. & Text & BERT & Hier. & \(\mathcal{L}_{F}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The elements of proposed mathematical models. Figure 1: The scheme of six investigated models. were created in the Government strategic analysis center of Lithuania (STRATA) within the human resources forecasting project. Only the results of experiments are published in this article. The data for the training was constructed using several national registers. Data are grouped into 3 main types. Educational information was retrieved from the register of Education Management Information System (lit. Svietimo valdymo informacine sistema). Education information consisted of anonymized personal-level data on vocational education, college and university (Bachelor's, Master's degree, Integrated studies) studies, and professional education (doctor's medical practice and pedagogy). There is no equivalent administrative data about doctoral degrees, as for this level of education, unique programs for individual students are composed. Observations consist of education information of graduates or students that achieved graduation (if the education program was not completed, the case was dropped out from the research sample). Education information was additionally translated into English for the textual analysis using Google Cloud Translation API [27]. Educational program abstracts were retrieved from the official and publicly available register, AIKOS, managed by Ministry of Education, Science, and Sport (aprox. 2500 valid programs) [28]. The data was retrieved using scraping tools: Chrome extension Web Scraper [29] and Scrapy [30]. The retrieved information was further processed: translated into English with Google Cloud Translation API [27]. Occupational information was retrieved from The State Social Insurance Fund Board under the Ministry of Social Security and Labour (hereafter abbr. SODRA). Anonymized personal-level occupational data was retrieved for the identified graduates. Occupations were mapped to 4-digit codes according to the Lithuanian version of Standard Classification of Occupations (hereafter abbr. LPK classification) (created in 2012 in accordance to the International Standard Classification of Occupations ISCO-08) [31]. The 4-digit occupational ID has hierarchical structure: first digits branch out into leaves, which branch out further. The first digit represents the status level of the occupation. For example, number \"1\" represents class of managers and \"2\", professionals. The second digit narrows down the speciality of the occupation, for example, \"21\" represents science and engineering professionals. The third digit shows even more detailed occupational granularity, e.g., \"241\" signals \"Finance professionals\". the fourth digit provides the most detailed information about the occupation, for example, \"2411\" represents \"Accountants\". The number of classes per level are different: the first digit group consists of 10 classes, the second, 43, the third, 130, and the fourth level has 436 classes. Not all classes were observed in the data. Additional EDA is presented in Figure 2. It is important to note that one person generally completes 1-2 education programs and works one job at any point in time during their career. However, the significant number of cases diverge from general trend and should be accounted during learning process.