Can dependency parses facilitate generalization in language models? A case study of cross-lingual relation extraction (2025)

Abstract: In this work, we propose DEPGEN, a framework for evaluating the generalization capabilities of language models on the task of relation extraction, with dependency parses as scaffolds. We use a GNN-based framework that takes dependency parses as input and learns embeddings of entities which are augmented to a baseline multilingual encoder. We also investigate the role of dependency parses when they are included as part of the prompt to LLMs in a zero-shot learning setup. We observe that including off-the-shelf dependency parses can aid relation extraction, with the best performing model having a mild relative improvement of 0.91% and 1.5% in the in-domain and zero-shot setting respectively across two datasets. For the in-context learning setup, we observe an average improvement of 1.67%, with significant gains for low-performing LLMs. We also carry out extensive statistical analysis to investigate how different factors such as the choice of the dependency parser or the nature of the prompt impact performance. We make our code and results publicly available for the research community at https://github.com/ShoRit/multilingual-re.git.

## introduction information packaging in language does not happen arbitrarily @xcite . the "internal structure" of a text message, which determines how the message is constructed or parsed, is grounded in predefined linguistic rules in the form of syntax and semantics. linguistic structures such as dependency graphs @xcite or semantic parses @xcite have been pivotal in the history of nlp research both for their intrinsic merit i.e. developing frameworks that can construct or interpret such structures automatically @xcite , and their external value as augmentations to aid language understanding tasks @xcite . information extraction or ie is one such field which had relied heavily on linguistic information ever since its inception; some notable examples include few-shot named entity recognition or ner @xcite , relation extraction @xcite , opendomain question answering, @xcite amongst others. however, recent years have witnessed a decline in the adoption of linguistic frameworks in favor of large scale pre-trained language models @xcite @xcite which are shown to encode syntactic and semantic information within their parameters @xcite and have also demonstrated significant improvements on ie @xcite . moreover, as we usher into an era of large language models, the question which looms over our head like the proverbial sword of damocles "are dependency parses helpful for information extraction?" we are motivated to answer this question based on the past work of @xcite which showed the utility of adding syntactic information for different information extraction tasks in english. however, the observed benefits hold true only when the gold parses are available, with no improvements over the baseline in presence of offthe-shelf parses. in this study we expand upon this idea and investigate whether off-the-shelf dependency parses can assist language models in multilingual information extraction for both indomain and zero-shot transfer settings. we specifically deal with the task of multilingual relation extraction, wherein we identify the nature of relationship between two annotated entities in a document. we show in figure 1 how we can connect the entities wood and fences by traversing the dependency graph that connects these two entities, highlighting the potential utility of linguistic frameworks for this task. we explore the role figure 1 : example depicting the supplemental information provided by the dependency tree. the entities of interest are wood and fences, having the relationship material_used. the path wood ← @xmath0. we propose a framework, depgen, built on top of a pretrained multi-lingual language model that uses dependency parse information to perform relation extraction for both in-domain and zero-shot cross-lingual transfer settings. through a comprehensive set of 2440 experiments spanning 10 languages over 2 datasets, we observe that incorporating dependency information brings about modest improvements for in-domain and cross-lingual fine-tuning setups by 0.9% and 1.5% respectively. we also carry out extensive statistical analysis to identify which factors significantly impact performance. our observations highlight that performance improvements is mostly predicated by the choice of the target language, and the choice of the pre-trained language model rather than the choice of the dependency parser for all cases. however, for the in-context learning setup, we demonstrate that the performance is determined by the choice of the prompting strategy, with our proposed approach boasting the highest gains, i.e. an absolute improvement of 1.67 f1 score over the baseline. ## methodology we investigate the role of dependency parses for zero-shot cross-lingual relation extraction in two setups, namely (i) a fine-tuned setup where a model is first trained on a given source language and then evaluated on a target language, and (ii) an incontext-learning setup where we prompt an llm to predict the relation between two specified entities in a zero-shot setting to test the innate capabilities of the llm for re. ## results and insights in this section, we pose the following research questions (rqs) and attempt to answer the same. ## conclusion and future work in this paper we propose a multi-component framework for multi-lingual relation extraction. our finetuned framework depgen, combines the signals from the input sentence with dependency parses that are encoded through a gnn. through extensive evaluations, we have reported the implications of our work for both in-domain and zero-shot transfer settings across multiple languages. we observe that including off-the-shelf dependency parses can aid relation extraction, with the best performing model having a mild relative improvement of 0.91% and 1.5% in the in-domain and zero-shot setting respectively across two datasets. we also develop an in-context learning prompting approach that incorporates dependency information to bring about an average improvement of 1.67%, with significant gains for low-performing llms. in this work, we investigate how augmenting dependency parses in language models can facilitate information extraction tasks in low-resource settings. specifically, our contributions being independent of the language setting makes our model portable to other languages in a zero-shot transfer setup. future avenues will explore the role of these linguistic frameworks for other information extraction or reasoning tasks, and the impact of different kinds of linguistic frameworks such as amrs or umrs.