DALE : Generative Data Augmentation for Low-Resource Legal NLP (2023)

Abstract: We present DALE, a novel and effective generative Data Augmentation framework for low-resource LEgal NLP. DALE addresses the challenges existing frameworks pose in generating effective data augmentations of legal documents - legal language, with its specialized vocabulary and complex semantics, morphology, and syntax, does not benefit from data augmentations that merely rephrase the source sentence. To address this, DALE, built on an Encoder-Decoder Language Model, is pre-trained on a novel unsupervised text denoising objective based on selective masking - our masking strategy exploits the domain-specific language characteristics of templatized legal documents to mask collocated spans of text. Denoising these spans help DALE acquire broad legal knowledge and develop the ability to generate coherent and diverse augmentations with novel contexts. Finally, DALE performs conditional generation to generate synthetic augmentations for low-resource Legal NLP tasks. We demonstrate the effectiveness of DALE on 13 datasets spanning 6 tasks and 4 low-resource settings. DALE outperforms all our baselines, including LLMs, qualitatively and quantitatively, with absolute improvements of 1%-50%.

## introduction with recent advances in deep learning for nlp, many systems have achieved state-of-the-art and near-human performance on benchmark natural language understanding (nlu) datasets @xcite . following this closely, the legal nlp literature has also been thriving with new datasets and frameworks @xcite @xcite . however, one common observation is that most techniques, built and evaluated on nlp tasks involving 1 code: https://github.com/sreyan88/dale * these authors contributed equally to this work. original 1: buyer has full power and authority to enter into this agreement. ## conclusion this paper presents dale, a novel generative data augmentation framework for low-resource legal nlp. we evaluate dale on 13 datasets spanning across 6 tasks under 4 low-resource settings and show that dale outperforms all prior art quantitatively and qualitatively by a significant margin.