Universal Semantic Tagging for E nglish and M andarin C hinese (2021)

Abstract: Universal Semantic Tagging aims to provide lightweight unified analysis for all languages at the word level. Though the proposed annotation scheme is conceptually promising, the feasibility is only examined in four Indo–European languages. This paper is concerned with extending the annotation scheme to handle Mandarin Chinese and empirically study the plausibility of unifying meaning representations for multiple languages. We discuss a set of language-specific semantic phenomena, propose new annotation specifications and build a richly annotated corpus. The corpus consists of 1100 English–Chinese parallel sentences, where compositional semantic analysis is available for English, and another 1000 Chinese sentences which has enriched syntactic analysis. By means of the new annotations, we also evaluate a series of neural tagging models to gauge how successful semantic tagging can be: accuracies of 92.7% and 94.6% are obtained for Chinese and English respectively. The English tagging performance is remarkably better than the state-of-the-art by 7.7%.

## introduction developing meaning representations across different languages plays a fundamental and essential role in multilingual natural language processing, and is attracting more and more research interests @xcite . existing approaches can be roughly divided into three categories: the crosslingual foot_0 approach focuses on lending semantic annotation of a resource-rich language, such as english, to an under-resourced language @xcite @xcite ; the interlingual approach attempts to provide a unified semantic framework for all languages @xcite @xcite ; the multilingual approach aims at developing comparable but not necessarily identical annotation schemes shared by different languages @xcite @xcite . in line with the interlingual approach, universal semantic tagging (ust; @xcite develops a set of language-neutral tags (hereafter referred to as sem-tag) to annotate individual words, providing shallow yet effective semantic information. semantic analyses of different languages utilise a same core tag set, but may also employ a few language-specific tags. figure 1 the idea of sem-tag is first applied to the parallel meaning bank (pmb; @xcite , where a multilingual corpus, including dutch, german and italian, is semi-automatically built by projecting semantic tags from english sentences to their translated counterparts. however, it is insufficient to prove the feasibility of ust only through some cases of inflectional and genetically related languages, because one main challenge in developing interlingual meaning representations is unifying annotations related to different characteristics of different languages. we argue that two questions with regard to universality of ust are still unanswered. firstly, homologous words in pmb languages facilitate the application of ust, but it is not clear whether ust is equally applicable to languages sharing little cognates, although ust employs a delexicalised method. another concern is from typology: it still remains unknown whether word-level semantic tags are effective for annotating long "sentence-words" composing many morphemes which are common in agglutinative languages (e.g. turkish and japanese) and polysynthetic languages (e.g. eskimo languages). this paper takes mandarin chinese, a phylogenetically distant language from the indo-european family, as an example to explore the effectiveness of ust as a universal annotation scheme. considering the balance of chinese-specific linguistic properties and universality, we present a more comprehensive tag set where six new tags are added, indicating most sem-tags are applicable to chinese ( §2). based on the new tag set, we establish a parallel corpus by manually translating wsj into corresponding chinese sentences and annotating sem-tags for 1100 sentence pairs. it is a peer-reviewed corpus with 92.9% and 91.2% interannotator observed agreement of chinese and english respectively ( §3). this relatively successful practice of ust in chinese suggests it keeps the balance between the depth of represented information and the breadth of its coverage of languages. in other words, shallow semantics of ust enables it to be extended to annotate diversified languages. by means of the newly created corpus, we evaluate a series of neural sequence labeling techniques ( §4). the results demonstrate that the proposed scheme is promising with the accuracy of chinese achieving 92.7% and the accuracy of english 94.6% ( §5). the english tagging performance is remarkably better than the state-of-theart (abzianidze and bos, 2017) by 7.7%, even though the sentences in our corpus are much longer than pmb on average, with 25 tokens per sentence compared with 6 in pmb. in order to analyse the divergence between an-notations of english and chinese data and the plausibility of developing universal semantic representation in general, we manually annotate word alignment for 500 sentences. by studying the aligned counterparts, we argue that universality is still threatened to some extent because there are 37.0% aligned tokens with mismatched sem-tags. this phenomenon is mainly due to grammatical divergence, information loss of translation and difference of annotation strategies. all the analyses based on word alignment suggest that even for a delexicalised, relatively shallow meaning representation scheme, it can still be problematic to ensure that semantic representations could be comparable in a word-to-word way. considering different linguistic ways to encode tense, aspect, prepositions, measure words, subordinate clauses and comparative expressions, we provide a tailored version of ust to handle mandarin chinese. we present the complete tailored tag set in the appendix. events and tense/aspect different from english as well as many other indo-european languages, there are no inflection-style tense-markers in mandarin. therefore, the morphological tense-related labels, e.g. ens and eps, are removed. alternatively, temporal interpretation of chinese can be conveyed through function words, adverbials or shared understanding of the context in chinese @xcite . apart from the last way, the previous two are encoded by sem-tags fut and ist. as for aspect in chinese, there are only four commonly recognized aspect markers, denoting the preceding verbs are actualized or ongoing-了/过 are perfective (pft) and 在/着 are progressive (prg) @xcite . preposition prepositions of english and chinese vary in their history origins though they have similar syntactic function at present. english prepositions are mainly created to replace the lost inflectional case markers @xcite . on the other hand, chinese prepositions can be traced to verbs. @xcite even go so far as to call them coverbs since some of them are like verbs and can be used as verbs that have similar meanings. this term can avoid labeling them either verbs or prepositions. in this regard, chi classifier classifier is a chinese-specific word class which is inserted between numerals and nouns to denote quantity. this category does not exist in english so we generalize uom over the unit of measurement since its function is quite similar to classifiers @xcite . ## tagging models long short-term memory (hochreiter and schmidhuber, 1997, lstm) models have been widely used in various sequential tagging tasks @xcite @xcite and have achieved the state-of-theart performance for many popular benchmark datasets. in our paper, we use bidirecational lstm (bilstm) with and without a conditional random field (crf) inference layer to build baseline systems for our dataset. in the rest part of this section, we will briefly formulate our baseline tagging models and introduce some widely used techniques that may enhance prediction for some tagging tasks. ## related work ust is one of previous attempts of interlingua @xcite , which is originally designed to provide necessary information for semantic parsing @xcite . primary automatic sem-taggers are built using convolutional neural networks and deep residual networks @xcite . later, in pmb project @xcite , the authors propose a method of projecting automatically annotated semantic tags from a sentence to its sentence-and word-aligned counterparts. following previous works, an updated universal semantic tagset is later proposed @xcite , with a modification of deriving the tagset in a data-driven manner to disambiguate categories. in this work, a tri-gram based tagging model, tnt tagger @xcite , is also initially explored for bootstrapping utilization. in a recent study built on @xcite , employing sem-tag in multi-task learning is found to be beneficial to both sem-tag task and other nlp tasks including universal dependency pos tagging, universal dependency parsing, and natural language inference @xcite . overall, these studies indicate that sem-tags are effective in conducting various nlp tasks. ## conclusion in this paper, we take chinese into account to provide a more comprehensive tag set based on which we establish a reliable manually-annotated corpus, and show that promising performance of automatic semantic tagging is obtained after employing mtl as well as gold pos tag and leveraging pre-trained models. the overall success of this approach prompts a reflection of universality of different languages and operability of multilingual meaning representation: 1) ust is plausible in general partly because it is delexicalised and can thus represent phylogenetically languages after some adaptions; 2) universality is threatened to some extent because there are aligned but mismatched tokens between english and chinese, which are caused by grammatical divergence, information loss of translation and different annotation strategies for mwe; and 3) innate crosslingual divergences still exist even in nam's thought to be the most consistent pairs, which needs further exploration. though our work demonstrates the plausibility of developing a shared delexicalised and shallow annotation scheme to mitigate divergences across languages, it seems that more in-depth semantic analysis, especially lexicalised ones, may not be possible to be unified. we think a wider range of languages can be annotated after some minor adaptions of scheme. but it is still unknown how to get deeper processing information on this basis and thus develop an enhanced understanding of multilingual meaning representation.