Exploiting Task Reversibility of DRS Parsing and Generation: Challenges and Insights from a Multi-lingual Perspective (2025)

Abstract: Semantic parsing and text generation exhibit reversible properties when utilizing Discourse Representation Structures (DRS). However, both processes—text-to-DRS parsing and DRS-to-text generation—are susceptible to errors. In this paper, we exploit the reversible nature of DRS to explore both error propagation, which is commonly seen in pipeline methods, and the less frequently studied potential for error correction. We investigate two pipeline approaches: Parse-Generate-Parse (PGP) and Generate-Parse-Generate (GPG), utilizing pre-trained language models where the output of one model becomes the input for the next. Our evaluation uses the Parallel Meaning Bank dataset, focusing on Urdu as a low-resource language, Italian as a mid-resource language, and English serving as a high-resource baseline. Our analysis highlights that while pipelines are theoretically suited for error correction, they more often propagate errors, with Urdu exhibiting the greatest sensitivity, Italian showing a moderate effect, and English demonstrating the highest stability. This variation highlights the unique challenges faced by low-resource languages in semantic processing tasks. Further, our findings suggest that these pipeline methods support the development of more linguistically balanced datasets, enabling a comprehensive assessment across factors like sentence structure, length, type, polarity, and voice. Our cross-linguistic analysis provides valuable insights into the behavior of DRS processing in low-resource contexts, demonstrating both the potential and limitations of reversible pipeline approaches.

## introduction drs offers a distinct advantage in multilingual semantic processing through its language-neutral representation capabilities @xcite . this characteristic is particularly valuable for languages with limited computational resources. derived from discourse representation theory (drt), drs provides a comprehensive formal framework @xcite that captures complex linguistic phenomena including anaphors, presuppositions, temporal expressions, multisentence discourses, and the nuanced semantics of negation and quantification @xcite . this universal applicability makes drs especially relevant for developing semantic processing capabilities across diverse linguistic contexts @xcite . drs applications span various nlp tasks, including machine translation @xcite , semantic parsing @xcite , and text generation @xcite @xcite . these tasks exhibit inherent reversibilitythe output of one serving as input to the othera property that holds particular promise for languages with limited nlp infrastructure. traditional approaches, predominantly focused on english, require separate models for each task and language, creating significant barriers for languages with limited available data. while pre-trained language models have transformed nlp capabilities, their impact on semantic parsing and text generation varies significantly across languages. the challenge is particularly evident in cases where explicit meaning representation is not inherently integrated into the training of these models @xcite . despite recent advances, both drs parsing and generation remain challenging @xcite , with parsing mistakes leading to incorrect meaning representations and generation errors resulting in disfluent text @xcite . our work introduces a novel pipeline approach leveraging the reversible nature of semantic parsing and text generation, focusing particularly on urdu and italian. without requiring additional model training, we implement two pipeline setups using pre-trained language models: 1) parse-generate-parse (pgp), where input text is parsed, used to generate text, and then parsed again; and 2) generate-parse-generate (gpg), where a drs is used to generate text, which is parsed and then used to regenerate text. we utilized the pipeline approaches (pgp and gpg) to examine three categories of examples: (i) those showing improved performance, indicating error correction or mitigation; (ii) those remaining unchanged, highlighting the deterministic behavior of neural models in drs processing through pipelines; and (iii) those with decreased performance, signaling error amplification or propagation (see we conduct our evaluation on the parallel meaning bank foot_0 (pmb) dataset @xcite , focusing specifically on urdu as a low-resource language, italian as a mid-resource language, and english as a high-resource baseline. the selection of these languages is based on their representation of distinct linguistic families, each characterized by unique syntactic structures, word-order variations, morphological complexity, and differing levels of resource availability. this diversity enables a comprehensive comparative analysis, offering valuable insights into how resource availability and linguistic characteristics influence the performance of drs-based semantic processing across languages. the research questions addressed in this paper are: the key contributions of this paper are: (1) proposing a method for investigating error dynamics in drs-based nlp tasks by exploiting reversibility, (2) demonstrating the varied effects of pipeline approach across multiple languages using pre-trained language models without costly retraining, and (3) analyzing the capabilities and limitations of the proposed pipelines through rigorous cross-linguistic error analysis. to the best of our knowledge, this study represents the first attempt to exploit the reversible nature of drs parsing and generation to analyze error dynamics in a diverse multilingual context foot_2 . while previous research has primarily focused on either monolingual or multilingual semantic parsing and generation tasks, our work uniquely investigates the interplay between these tasks through their reversibility. the remaining paper is structured as follows: section 2 describes drs and reviews related work in semantic parsing and text generation; section 3 describes our methodology and pipeline configurations; section 4 displays multilingual experimental results in detail; section 5 presents a detailed error analysis with the discussion regarding the mitigation or amplification of errors; finally section 6 concludes the paper, highlights limitations, and suggests directions for future research. ## background and related work this section outlines the drs formalism ( § 2.1) used in this study and reviews key research in semantic parsing ( § 2.2) and text generation ( § 2.3). ## methods our study departs from the standard rule-based and neural network-based methods for drs parsing and text generation. we offer a novel perspective that takes advantage of the drs reversible capabilities that do not require any explicit design of rules or external tools, in contrast to rule-based systems like boxer or the more recent multilingual drs parser which rely on hand-crafted rules and commercial dependency parsers @xcite . instead, our work presents a pipeline approach that takes advantage of the complementary benefits offered by pre-trained language models. our approach cascades these reversible processes into two different pipelines, pgp and gpg, so as to identify error mitigation or amplification that might occur in the generation or parsing phase, without requiring extra rule engineering or model training. in our pgp and gpg pipelines, we employed byt5 @xcite due to the following factors: (i) multilingual model can generalize better across languages and tasks; (ii) char-level/byte-level tokenization strategy helps the model understand complex language patterns, scripts, characters, and semantic information; (iii) when it comes to spelling and pronunciation-sensitive tasks, byte-level models outperform other models due to their greater resilience to noisy data; (iv) byt5 is also referred to as a token-free model as it operates directly on raw utf-8 data without generating sub-word or word-based vocabulary; and (v) most importantly, byt5 has the state-of-the-art results on multilingual nlp benchmarks outperforming other models @xcite @xcite . pmb is a multilingual dataset comprising semantic representations in english, italian, german, dutch, and chinese. leveraging the languageneutral nature of drs, we transformed english drs-text pairs into urdu through a systematic approach involving syntactic structure, concept and word alignment, grammatical genders, and cross-lingual adaptation through named entities. this hybrid methodology resulted in the first comprehensive semantic resource for urdu foot_3 , comprising 3,000 manually annotated data instances. drs transformations were achieved through rule-based techniques and human annotation. text translations were initially generated using the google translate api and subsequently verified through manual inspection. urdu examples were divided into 1,200 training, 900 development, and 900 test examples. for italian, the dataset consisted of 5,061 training examples, 555 development examples, and 555 test examples. for english, the dataset contained 152,808 training examples, 1,132 development examples, and 1,132 test examples. to enhance dataset diversity and complexity, we applied multi-dimensional augmentation strategies, including named entities, lexical (encompassing common nouns, adjectives, adverbs, and verbs), and grammatical augmentations. this approach resulted in a ninefold increase in the training data examples applied to all three languages, i.e., en, it, and ur. for experimentation, we fine-tuned byt5 on our fully augmented drs-text pairs, achieving state-of-the-art performance in both semantic parsing and text generation tasks foot_4 . we implemented a two-stage fine-tuning strategy consistent with @xcite . the first stage involved fine-tuning the model on silver data for 3 epochs to establish foundational drs knowledge. the second stage focused on gold data finetuning for 10 epochs. experimental parameters included adamw optimizer, polynomial learning rate decay (1e-4), batch size of 32, maximum sequence length of 512, and geglu activation function. to evaluate the impact of the pipeline approach, we utilized smatch for semantic parsing @xcite , while bleu @xcite , meteor @xcite , rouge @xcite , comet @xcite , chrf @xcite , and bertscore @xcite were applied to assess text generation outcomes. ## results we experimented with three distinct languages-urdu (ur), italian (it), and english (en)-using the standard test set from the dataset. the results reveal complex patterns of performance changes across languages and metrics as shown in ## analysis and discussion to understand why pgp and gpg pipeline approaches often result in error amplification rather than mitigation, we conducted a systematic analysis focusing on the impact of linguistic imbalance in the dataset ( § 5.1), error patterns in pipeline approaches ( § 5.2), performance impact through cross-lingual analysis ( § 5.3), and revealing the pipeline approach ( § 5.4). ## conclusion we investigated the reversible nature of semantic parsing and text generation through drs, leveraging pipeline approaches across urdu, italian, and english. the primary objective was to assess the impact of two distinct pipeline configurations (pgp, gpg) on error propagation or mitigation without additional model training. by employing pretrained language models, we explored how these reversible processes influence the performance of both parsing and generation tasks, providing valuable insights into cross-linguistic error dynamics. the key findings demonstrate that, while the reversible pipeline approach offers the potential for correcting errors, it more frequently leads to error amplification, particularly in languages with complex morphology and syntactic structures, such as urdu and italian. english showed the most stability, with only slight performance drops in parsing and generation tasks. in contrast, urdu and italian were more prone to error amplifications, as errors introduced in one stage of the pipeline tended to grow in later stages. through a detailed analysis of error patterns across different linguistic dimensions, we provide an in-depth understanding of how specific language characteristics influence error propagation. we revealed that the reversible nature of drs-based pipelines, while theoretically promising, is limited in practical effectiveness due to the compounding of errors in complex sentence structures and morphologically rich languages. limitations: the potential of our pgp and gpg pipelines to exploit the task reversibility of drs offers opportunities for effective error dynamics, whether through propagation or mitigation. however, the predominance of error propagation over error mitigation is attributed to the dependency of these pipeline approaches on pre-trained language models. in our experimental implementation, we utilized the best-performing models with state-of-the-art results for the languages involved. yet, the data examples used to train the english drs processing models vastly outnumbered those for italian and urdu, posing a challenge in terms of model generalization and robustness capabilities. furthermore, the limitations of traditional evaluation metrics, such as smatch (which only considers structural overlap) and bleu and me-teor (which are based on n-gram overlap), further complicate the assessment of these results. in our analysis, we resorted to human evaluation, which is computationally expensive and time-consuming. additionally, our analysis has highlighted the linguistic imbalance across the various drs variants, which also poses a limitation to the fair evaluation of the models. these findings suggest the need for a more balanced dataset to train models that can overcome these limitations and deliver the best possible results. linting xue, aditya barua, noah constant, rami al-rfou, sharan narang, mihir kale, adam roberts, and colin raffel. 2022. byt5: towards a token-free future with pre-trained byte-to-byte models. transactions of the association for computational linguistics, 10:291-306. xiao zhang, chunliu wang, rik van noord, and johan bos. 2024. gaining more insight into neural semantic parsing with challenging benchmarks. arxiv preprint arxiv:2404.08354.