Input Representations for Parsing Discourse Representation Structures: Comparing E nglish with C hinese (2021)

Abstract: Neural semantic parsers have obtained acceptable results in the context of parsing DRSs (Discourse Representation Structures). In particular models with character sequences as input showed remarkable performance for English. But how does this approach perform on languages with a different writing system, like Chinese, a language with a large vocabulary of characters? Does rule-based tokenisation of the input help, and which granularity is preferred: characters, or words? The results are promising. Even with DRSs based on English, good results for Chinese are obtained. Tokenisation offers a small advantage for English, but not for Chinese. Overall, characters are preferred as input, both for English and Chinese.

## introduction recently, sequence-to-sequence models have achieved remarkable performance in various natural language processing tasks, including semantic parsing @xcite @xcite , the task of mapping natural language to formal meaning representations (figure 1 ). in this short paper we focus on parsing discourse representation structures (drss): the meaning representations proposed in discourse representation theory (drt, @xcite , covering a large variety of linguistic phenomena including coreference, thematic roles, presuppositions, scope, quantification, tense, and discourse relations. several data-driven methods based on neural networks have been proposed for drs parsing @xcite @xcite @xcite @xcite . these approaches frame semantic parsing as a sequence transformation problem and map the target meaning representation to string format. these models learn the meaning of a series of semantic phenomena by taking sentences as input and directly outputting the corresponding drss, without the aid of any extra linguistic information (such as part-of-speech or syntactic structure). these previous studies have achieved good results, but have mostly focused on english or other languages that use the latin alphabet. our objective is to investigate whether the same method is applicable to mandarin chinese, an extremely analytic language which makes deep parsing challenging @xcite @xcite . but chinese is not only different on the level of syntax; its writing system also shows large differences with english, as there are no explicit word separators in written chinese, and there is no distinction between lower-and upper case characters. unlike english, chinese words comprise few characters, but the number of different characters is about two orders of magnitude higher than that of english. these orthographic differences are interesting in the context of previous work, as @xcite use character-level input and word-level input to compare the impact of different input representations on drss parsing for english, finding that the character-level representation obtained better performance. in this paper we want to investigate how chinese fits in this picture. to the best of our knowledge, we are the first to explore methods for chinese drs parsing. we aim to answer the following questions: this paper is organised as follows. first we provide a short background on the formal meaning representations that we use, the difference between the writing systems of english and chinese, and the issues that arise around characters and words. then we will introduce our approach, the data set that we use, and how we conduct our experiments. in the final section we show that we can achieve good results for chinese drs parsing, with characters as the preferred representation. an impressive repertoire of semantic phenomena is covered by drt, including quantification, negation, reference resolution, comparatives, discourse relations, and presupposition. we use the drs version as employed in the parallel meaning bank @xcite , where concepts (triggered by nouns, verbs, adjectives and adverbs) are represented by wordnet synsets @xcite , and semantic relations by verbnet roles @xcite . drss can be represented in box format or clause format (see figure 1 ), where x, e, s, and t are discourse referents denoting individuals, events, states, and time, respectively, and b is used for variables denoting drss. named entities are preserved from the original language used in the input, so names in chinese are literally transferred in the drs interpretation (see figure 1 ). this means that the only difference between english and chinese drss is the way names are represented: english orthography is used for proper names in english drss; chinese characters are used for names in the corresponding chinese drss. the box format has become a common representation of drss because of its convenient reading and intuitive understanding. the clause format is a flat version of the standard box notation, which represents drss as a set of clauses. due to its simple and flat structure it is more suitable for machine learning purposes. at the same time, however, the structure of drss poses a challenge to sequence-tosequence models, because they need to be able to generate the well-formed recursive semantic structures. chinese word segmentation differently from english, chinese words are not separated by white spaces, as shown in input formats for neural methods characterlevel representations have proved useful for neural network models in many nlp tasks such as @xcite , dependency parsing @xcite and neural machine translation @xcite . however, only a few studies have used character-level representations as input representations for chinese nlp tasks @xcite @xcite . for chinese semantic parsing, previous studies mostly used wordbased representations as well @xcite . for english drs parsing, how-type english input representation chinese input representation ## methodology annotated data we use data from the parallel meaning bank (pmb 3.0.0, @xcite . the documents in this pmb release are sourced from seven different corpora from a wide range of genres. for one of these corpora, tatoeba, chinese translations already exist, and we added them to the pmb data. for the remaining texts that had no chinese translation, we translated the english documents into chinese using the baidu api, manually verified the results and, when needed, corrected the translations. only a few translations needed major corrections. about a hundred translated sentences lacked past or future tense or used uncommon chinese expressions. special care was given to the translation of named entities, ambiguous words, and proverbs, and required about a thousand changes. for economical reasons the silver part of the data was only checked on grammatical fluency. ## results and discussion in general, f-scores start to decrease when sentences get longer (figure 2 ), though there is no clear difference between the character and wordlevel models. this is in line with the findings of @xcite . for english, the input types based on characters outperform those based on words. bpe approaches character-level performance for small amounts of merges (1k), but never surpasses it. this too is in line with @xcite , but also with previous work on nmt for chinese @xcite . there is a small benefit (0.5) for tokenizing the input text before converting the input to character-level format, though the continuous character representation also works surprisingly well. for chinese, characterbased input shows the best performance too, though for a very small amount of merges bpe obtains a similar score. as opposed to english, tokenizing the chinese input is not beneficial when using a character-level representation, though it also does not hurt performance. in general, character-level models seem the most promising for chinese drs parsing. similar results were obtained by @xcite for chinese sql parsing. figure 3 : f-scores per clause type (drs operators, verbnet roles and wordnet concepts) and concept type (nouns, verbs, adjectives, adverbs and events) as introduced by @xcite . reported scores are on the chinese and english dev set for the raw character-level models, averaged over 5 runs. figure 3 shows detailed scores for the characterbased (raw) model on the chinese and english dev set, categorizing operators (e.g., negation, presupposition or modalities), verbnet roles (e.g., agent, theme), predicates, and senses. modifiers, especially adverbs, get a systematic lower score in chinese compared to english. this is interesting, and examination of the data reveals that english adverbs are regularly translated as chinese noun phrases (e.g., @xmath0. this will lower the f-score even though the meaning is preserved, only expressed in a semantically different way. ## conclusion and future work drs parsing for chinese based on projecting meaning representations from english translations gives remarkable performance (rq1), though chinese adverbs remain challenging. english results outperform those of chinese, but it is likely that this is due to the general bias of the meaning representations towards english. similar as for english, we find that characters are the preferred input representation for chinese (rq2). surprisingly, for english, good results are even obtained by using characters without spaces as input. tokenisation (segmenting the text into words) of the input offers a small advantage for english, but not for chinese (rq3), though it will be interesting to experiment with higher quality word segmentation systems @xcite . there are many research directions one could take next. one is to include pre-trained models. for instance, we could use recently proposed pretrained models such as bart @xcite or mbart @xcite to improve parsing performance. another interesting idea is, rather than assuming the english wordnet as a background ontology for concepts in the drs, using concepts based on chinese wordnet or multilingual wordnets @xcite . both possibilities will likely further improve performance of semantic parsing for chinese and inspire research for developing semantic parsing models for languages other than english.