NUIG : Multitasking Self-attention based approach to S ig T yp 2020 Shared Task (2020)

Abstract: The paper describes the Multitasking Self-attention based approach to constrained sub-task within Sigtyp 2020 Shared task. Our model is simple neural network based architecture inspired by Transformers (CITATION) model. The model uses Multitasking to compute values of all WALS features for a given input language simultaneously.

Results show that our approach performs at par with the baseline approaches, even though our proposed approach requires only phylogenetic and geographical attributes namely Longitude , Latitude , Genus-index , Family-index and Country-index and do not use any of the known WALS features of the respective input language, to compute its missing WALS features.

## introduction in this paper we describe our multitasking selfattention based approach to sigtyp 2020 shared task (constrained sub-task) @xcite which involves prediction of values of features from wals typology database for various lowresourced languages. linguistic typology is the classification of human languages according to their syntactic, phonological and semantic features. wals @xcite is the most popular and comprehensive database which provides list of typological features and their possible values, as well as the respective feature-values for most of the world's languages. however all the popular typological databases @xcite @xcite @xcite (including wals) suffer from a major shortcoming of limited coverage. in fact, values of many important typological features for most languages (specially less documented ones) are missing in these databases. this sparked a line of research on automatic acquisition of such missing typology knowledge @xcite @xcite @xcite . our proposed model is a neural network architecture which takes in as input, the phylogenetic and geographical attributes of a language. the model subsequently predicts values of all its typology features simultaneously using multitask learning setup @xcite . ## model figure 1 depicts the architecture of our proposed model that computes values of all wals typology features for a given language simultaneously. as evident in figure 1 , our proposed model architecture comprises of three components namely input network component, self-attention network component and multitasking output networks component described as section 2.1, 2.2 and 2.3 respectively. ## training the parameters of model described in section 2 are trained by optimizing the loss function given by equation 7. here oh t is the one-hot encoding of true-value for t th typology feature. ce is the cross-entropy loss. ## conclusion in this paper we described our multitasking selfattention based approach to sigtyp 2020 shared task, constrained sub-task. our model is simple neural network based approach which computes values of all wals features for a given input language simultaneously in multitasking settings. the architecture of our network is inspired by transformers @xcite . results show that our approach performs at par with the baseline approaches, even though our approach uses only five linguistic and geographical attributes namely longitude, latitude, genus-index, family-index and country-index and do not use any of the known wals features of the respective input language, to compute its missing wals features.