Multilingual Relation Classification via Efficient and Effective Prompting (2022)

Abstract: Prompting pre-trained language models has achieved impressive performance on various NLP tasks, especially in low data regimes. Despite the success of prompting in monolingual settings, applying prompt-based methods in multilingual scenarios has been limited to a narrow set of tasks, due to the high cost of handcrafting multilingual prompts. In this paper, we present the first work on prompt-based multilingual relation classification (RC), by introducing an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios, and analyze its effectiveness across 14 languages, prompt variants, and English-task training in cross-lingual settings. We find that in both fully supervised and few-shot scenarios, our prompt method beats competitive baselines: fine-tuning XLM-R_EM and null prompts. It also outperforms the random baseline by a large margin in zero-shot experiments. Our method requires little in-language knowledge and can be used as a strong baseline for similar multilingual classification tasks.

## introduction relation classification (rc) is a crucial task in information extraction (ie), aiming to identify the relation between entities in a text @xcite . extending rc to multilingual settings has recently received increased interest @xcite , but the majority of prior work still focuses on english @xcite . a main bottleneck for multilingual rc is the lack of supervised resources, comparable in size to large english datasets @xcite . the smiler dataset @xcite provides a starting point to test fully supervised and more efficient approaches due to different resource availability for different languages. previous studies have shown the promising performance of prompting plms compared to the datahungry fine-tuning, especially in low-resource scenarios @xcite @xcite . multilingual pre-trained language models @xcite further enable multiple languages to be represented in a shared semantic space, thus making prompting in multilingual scenarios feasible. however, the study of prompting for multilingual tasks so far remains limited to a small range of tasks such as text classification @xcite and natural language inference @xcite . to our knowledge, the effectiveness of prompt-based methods for multilingual rc is still unexplored. to analyse this gap, we pose two research questions for multilingual rc with prompts: rq1. what is the most effective way to prompt? we investigate whether prompting should be done in english or the target language and whether to use soft prompt tokens. rq2. how well do prompts perform in different data regimes and languages? we investigate the effectiveness of our prompting approach in three scenarios: fully supervised, few-shot and zero-shot. we explore to what extent the results are related to the available language resources. we present an efficient and effective prompt method for multilingual rc (see figure 1 ) that derives prompts from relation triplets (see section 3.1). the derived prompts include the original sentence and entities and are supposed to be filled with the relation label. we evaluate the prompts with three variants, two of which require no translation, and one of which requires minimal translation, i.e., of the relation labels only. we find that our method outperforms fine-tuning and a strong taskagnostic prompt baseline in fully supervised and few-shot scenarios, especially for relatively lowresource languages. our method also improves over the random baseline in zero-shot settings, and titanic is directed by cameron . titanic is directed by cameron . titanic _______ cameron . goethe schrieb die tragödie faust . goethe schrieb die tragödie faust . faust _______ goethe . ## preliminaries we first give a formal definition of the relation classification task, and then introduce fine-tuning and prompting paradigms to perform rc. ## methods we now present our method, as shown in figure 1 . we introduce its template and verbalizer, and propose several variants of the prompt. lastly, we explain the training and inference process. ## experiments we implement our experiments using the hugging face transformers library @xcite , hydra @xcite and pytorch @xcite . 2 we use micro-f1 as the evaluation metric, as the smiler paper @xcite suggests. to measure the overall performance over multiple languages, we report the macro average across languages, following zhao and schütze (2021) and @xcite . we also group the languages by their available resources in both pretraining and fine-tuning datasets for additional aggregate results. details of the dataset, the models, and the experimental setups are as follows. further experimental details are listed in appendix a. ## results and discussion we first present the results in fully supervised, fewshot and zero-shot scenarios, and then discuss the main findings for answering the research questions in section 1. ## related work multilingual relation classification previous work in multilingual rc has primarily focused on traditional methods rather than prompting plms. @xcite machine-translate non-english full text to english to deal with multilinguality. @xcite employ a shared semantic role labeler to get language-agnostic abstraction and apply rule-based methods to classify the unified abstractions. @xcite employ convolutional networks to extract relation embeddings from texts, and propose cross-lingual attention between relation embeddings to model cross-lingual information consistency. sanh et al. (2019) leverage the embeddings from bilstm, which is trained with a set of selected semantic tasks to help (multilingual) relation extraction. köksal and özgür (2020) finetune (multilingual) bert, classifying the embedding at [cls]. to take entity-related embeddings into consideration as well, nag et al. (2021) add an extra summarization layer on top of a multilingual bert to collect and pool the embeddings at both [cls] and entity starts. multilingual prompting multilingual prompting is a new yet fast-growing topic. winata et al. (2021) reduce handcrafting efforts by reformulating general classification tasks into binary classification with answers restricted to true or false for all languages. huang et al. ( 2022 ) propose a unified multilingual prompt by introducing a so-called "two-tower" encoder, with the template tower producing language-agnostic prompt representation, and the context tower encoding text information. @xcite manually translate prompts and suggest multilingual multitask training to boost the performance for a target downstream task. ## conclusion in this paper, we present a first, simple yet efficient and effective prompt method for multilingual relation classification, by translating only the relation labels. our prompting outperforms fine-tuning and null prompts in fully supervised and few-shot experiments. with supervised data, in-language prompting enjoys the best performance, while in the zero-shot scenarios prompting in english is preferable. we attribute the good performance of our method to its well-suitedness for rc, with the derivation of entity 1 -relation-entity 2 prompts from relation triples. we would like to see our method extended to similar tasks, such as semantic role labeling, with a structure between concepts that can be described in natural language.