Translating Headers of Tabular Data: A Pilot Study of Schema Translation (2021)

Abstract: Schema translation is the task of automatically translating headers of tabular data from one language to another. High-quality schema translation plays an important role in cross-lingual table searching, understanding and analysis. Despite its importance, schema translation is not well studied in the community, and state-of-the-art neural machine translation models cannot work well on this task because of two intrinsic differences between plain text and tabular data: morphological difference and context difference. To facilitate the research study, we construct the first parallel dataset for schema translation, which consists of 3,158 tables with 11,979 headers written in 6 different languages, including English, Chinese, French, German, Spanish, and Japanese. Also, we propose the first schema translation model called CAST, which is a header-to-header neural machine translation model augmented with schema context. Specifically, we model a target header and its context as a directed graph to represent their entity types and relations. Then CAST encodes the graph with a relational-aware transformer and uses another transformer to decode the header in the target language. Experiments on our dataset demonstrate that CAST significantly outperforms state-of-the-art neural machine translation models. Our dataset will be released at https://github.com/microsoft/ContextualSP .

## introduction as the saying goes, "a chart is worth a thousand words". nowadays, tremendous amounts of tabular data written in various languages are widely used in wikipedia pages, research papers, finance reports, file systems, and databases, which are informative. schema translation is the task of automatically translating headers of tabular data from one language to another. high-quality schema translation plays an essential role in cross-lingual table searching, understanding, and analysis @xcite @xcite . note that in this work, we focus on translating the headers instead of the entire table content, since for each entity in table content, it is hard to decide if it needs to be translated or not. over translation could even have negative effects in reality. despite its importance, most research efforts are dedicated to plain text machine translation @xcite @xcite , and schema translation is not well studied in the community, to the best of our knowledge. according to our preliminary study, state-of-the-art neural machine translation (nmt) systems cannot work well on schema translation because of two intrinsic differences between plain text and tabular data: morphological difference and context difference. morphological difference. the morphology of table headers differs from that of plain text in the following four aspects. first, headers are always phrases and they usually contain a lot of domainspecific abbreviations (e.g., as shown in figure 1 , "no." is the abbreviation of "number" and the "loc." is short for "location") and special symbols (e.g., "$" means "dollar" in figure 1 ). second, verb-object phrases are frequently used as headers which indicate a subject-object relationship between two columns. for example, "hosted by" in figure 1 indicates a host relationship between the second and the third columns. third, special tokenizations like camelcase and underscore are idiomatic usages in headers. at last, capitalized words are particularly preferred in order to capture more readers' attention for headers. these special word-forms are commonly used in headers but rarely seen in plain text. therefore, the nmt models trained with a massive amount of plain text cannot be directly applied to schema translation. context difference. compared with plain text, which is a sequence of words, tables have welldefined structures, and understanding a table's structure is crucial for schema translation. specifically, a table consists of an ordered arrangement of rows and columns. each column header describes the concept of that column. the intersection of a row and a column is called a cell. each cell contains entities of the column header it belongs to. this structure plays an important role in schema translation, especially for polysemy words and abbreviation words. for example, in figure 1 , the header "match" could be translated to "kù (matchstick)", "9m (mapping)", and "' [ (competition)", but its sibling column header "hosted_by" provides important clues that the table might belong to the domain of sport. thus, translating "match" to "'[ (competition)" is more appropriate in the context. moreover, a column header's cell values could also provide hints to infer the meaning of the header. for example, successive numerical cell values indicate that "no." might be an identity column in figure 1 . nmt models trained with plain text have never seen the structure of tables, and consequently, they perform poorly in schema translation. although the context information of tables is important, how to effectively use it for schema translation is challenging. on the one hand, the nmt model needs to make use of the context information to make word-sense disambiguation for polysemy headers and abbreviation headers. for another, the context information should not bring additional noise when translating the target header. to facilitate the research study, we construct the first parallel dataset for schema translation written in six different languages. it consists of 3,158 tables with 11,979 headers written in six differ-ent languages, including english, chinese, french, german, spanish, and japanese. furthermore, to address the challenges in schema translation, we propose a context aware schema translation (cast) model, which is a header-to-header neural machine translation model augmented with table context. specifically, we model a target header and its context as a directed graph to represent their entity types and structural relations. then cast encodes the graph with a relational-aware transformer and uses another transformer to decode the header in the target language. the advantages of our approach come from two folds: (1) the structure relationships make the transformer encoder capture the structural information and learn a contextualized representation for the target header; (2) the entity types differentiate the target header from its context and thus help denoise the target header translation. experiments on our dataset demonstrate that cast significantly outperforms state-of-the-art neural machine translation models. our contributions are summarized as follows. ## schema translation dataset to address the need for a dataset for the new schema translation task, we construct the first parallel schema translation dataset. it consists of 3,158 tables with 11,979 headers written in six different languages, including english, chinese, french, german, spanish, and japanese. in this section, we will first introduce our construction methodology and then analyze the characteristics of our dataset. ## methodology in this section, we describe our schema translation approach in detail. we first introduce the requirement and our definition for the schema translation task and then introduce the model architecture. ## experiments in this section, we conduct experiments on our proposed schema translation dataset to evaluate the effectiveness of our approach. furthermore, we ablate different ways of context modeling in our approach to understand their contributions. at last, we conduct a qualitative analysis and show example cases and their predicting results. ## related work with the developments of neural machine translation (nmt) systems @xcite , tremendous success has been achieved by existing studies on machine translation tasks. for instance, @xcite greatly improved bilingual machine translation systems with the transformer architectures, @xcite achieved state-of-the-art on the wmt' 14 english-german tasks with back-translations augmentation, @xcite and @xcite explored ways to boost the performance of nmt systems with pre-trained language models. recent works @xcite saw the potential to improve nmt models in many-to-many settings and proposed models that can perform machine translation on various language pairs. while the above-mentioned studies focus on sentence-level translation in plain text, they are not suitable for schema translation. a line of machine translation research closely related to our task is the phrase-to-phrase translation, which considers phrases in multi-word expressions as their translation unit. traditional phrase-based smt models @xcite get phrase table translation probabilities by counting phrase occurrences and use local context through a smoothed n-gram language model. recently, some works explore ways to adapt nmt models for phrase translation. for example, @xcite combined the phrase-based statistical machine translation (smt) model into nmt and shown significant improvements on chineseto-english translation data, @xcite explored the use of phrase structures for nmt systems by modeling phrases in target language sequences, and @xcite used a phrase attention mechanism to enhance the decoder in relevant source segment recognition. the main differences between these studies and our work are: (1) we do not rely on external phrase dictionaries or phrase tables; and (2) we study how to make use of the schema context for word-sense disambiguation in the schema translation scenario. context-aware schema encoding has received considerable attention in both recent semantic parsing literature @xcite and table -to-text literature @xcite . in general, there are two sorts of techniques: 1). add additional entity type embedding and special separator token from the input sequence to distinguish the table structure (i.e., type-sql and irnet); 2). encode the schema as a directed graph. for example, @xcite use a graph neural network @xcite , and @xcite ; @xcite use a transformer self-attention mechanism to encode the schema over predefined schema relationships. unlike these works, we explore the suitability of schema encoding techniques for the newly proposed schema translation task. ## conclusion in this paper, we propose a new challenging translation task called schema translation, and construct the first parallel dataset for this task. to address the challenges for this new task, we propose cast, which uses a relational-aware transformer to encode a header and its context over predefined relationships, making it aware of the table context.