DALE: Generative Data Augmentation for Low-Resource Legal NLP
Authors: Sreyan Ghosh
Chandra Kiran Reddy Evuru
Sonal Kumar
Ramaneswaran S
S Sakshi
Utkarsh Tyagi
Dinesh Manocha
Venue: EMNLP 2023 Main (2023)
Keywords: legal
low-resource
augmentation
generation
efficient
TL;DR: The paper proposed a novel data augmentation framework for low-resource legal NLP.

Abstract: We present DALE, a novel and effective generative Data Augmentation framework for low-resource LEgal NLP. DALE addresses the challenges existing frameworks pose in generating effective data augmentations of legal documents - legal language, with its specialized vocabulary and complex semantics, morphology, and syntax, does not benefit from data augmentations that merely rephrase the source sentence. To address this, DALE, built on an Encoder-Decoder Language Model, is pre-trained on a novel unsupervised text denoising objective based on selective masking - our masking strategy exploits the domain-specific language characteristics of templatized legal documents to mask collocated spans of text. Denoising these spans help DALE acquire broad legal knowledge and develop the ability to generate coherent and diverse augmentations with novel contexts. Finally, DALE performs conditional generation to generate synthetic augmentations for low-resource Legal NLP tasks. We demonstrate the effectiveness of DALE on 13 datasets spanning 6 tasks and 4 low-resource settings. DALE outperforms all our baselines, including LLMs, qualitatively and quantitatively, with absolute improvements of 1%-50%.

DALE: Generative Data Augmentation for Low-Resource Legal NLP
Sreyan Ghosh
Chandra Kiran Evuru
Sonal Kumar S Ramaneswaran
S Sakshi*
Utkarsh Tyagi* Dinesh Manocha
University of Maryland, College Park, USA,
UMass, Amherst, NVIDIA, Bangalore, India
(sreyang utkarsht, ckevuru, sonalkum, dmanocha @umdd edu fiikshi@@mass..o edu, ramanr@nvidia.co com
Abstract
Original Buyer
and
Agrerment
Method
Original The Bomwerr is organized, raidyy existig and is good standing under the tawn junsdienenn regentcation
We present DALE, novel and effective gen- erative Data Augmentation framework for low- resource LEgal NLP. DALE addresses the chal- lenges existing frameworks pose in generat- ing effective data augmentations of legal docu- ments- legal language, with its specialized vo- cabulary and complex semantics, morphology, and syntax, does not benefit from data augmen- tations that merely rephrase the source sentence. To address this, DALE, built on an Encoder- Decoder Language Model, is pre-trained on a novel unsupervised text denoising objective based selective masking- our masking strat- egy exploits the domain-speciiic language char- acteristics of templatized legal documents to mask collocated spans of text. Denoising these spans help DALE acquire knowledge about lc- gal concepts, principles, and language usage. Consequently. it develops the ability to gener- ate coherent and diverse augmentations with novel contexts. Finally, DALE performs condi- tional generation to generate synthetic augmen- tations for low-resource Legal NLP tasks. We demonstrate the effectiveness of DALE on 13 datasets spanning 6 tasks and low-resource settings. DALE outperforms all our baselines, including LLMs, qualitatively and quantita- tively, with improvements of 1%-50%.
Neyer wide cut power and authorty to eater vnn the agnoement
183A
homover orginond volidly eeating and good sueding under lano dee jutidiciio repasication
(W Zoa)
Parchacc las fillet--th--mooe major
self- lo isto
Lapi-EEA
this apramentt.
(Porçic es al.
2 The bonoower is regasizod. valily cisting and in jest stand under the law lopal power oegatication
bayan is full casudl and agrees to eade into this agreenenss
SSMBA
tonower oegonied valldy esleting good gaace the jorstiction or and organization
(gg mlll)
Who has authoityy
The Homrower oeganinat imn there corprres saidly caretin, vallely
vaidtly
corppey
carprr
The reyer postossss comptes sattorty to regign is mis agreemmm
lceslly casinbatesl,
and compliasee
with the lews of the jurialattion where enparieet
Campaay
power
Agposmane
DALE
and to perfocta its obligations hercander.
coperation duly reganieed validly in conding under the low the Stree of Dellwon
Table I:
Comparison of augmentations generated using DALE and our baselines. DALE generates coherent and di- verse augmentations in addition to introducing new context while preserving label consistency (1. PlPamments 2.Authority).
everyday natural language, do not easily transfer to the legal domain (Zhong et al., 2020a; Chalkidis et al., 2020; Katz et al., 2023). Legal language, also known as legalese and commonly classified as "sublanguage" (Sinsheimer, 2007: Williams, 2007; Haigh, 2023), governed by logical rules and is distinct from everyday natural language in terms of specialized vocabulary, morphology, complex syntax, and knowledge-specffic semantics, which makes the transfer difficult. Interestingly, modern Large Language Models (LLMs), both open- and closed-source (like ChatGPT), that have shown to possess excellent reasoning abilities and achieved impressive performance in zero-shot NLU tasks (HuggingFace, 2023), often do not perform well in Legal Language Understanding (LLU) tasks (Chalkidis, 2023). With of instruction- tuned LLMs as our basclines, we also show that LLMs struggle to generate effective augmentations for LLU tasks and fail to preserve label consistency when the source legal document is long.
1 Introduction
With recent advances in deep learning for NLP, many systems have achieved state-of-the-an and near-human performance benchmark Natural Language Understanding (NLU) datasets (Wang al., 2018, 2019). Following this closely, the le- gal NLP literature has also been thriving with new datasets and frameworks (Chalkidis et al., 2021 Niklaus et al., 2023; Chalkidis et al., 2023). How- ever, one common observation is that most tech- niques, built and evaluated on NLP tasks involving
"Code:
"These authors contributed equally to this work.
Improving the performance of deep learning

---

models on downstream LLU tasks requires suf- ficient good-quality training data. Beyond being an expensive and noisy task (Abad and Moschitti, 2016; Nguyen et al., 20, high-quality annotation in specialized domains like legal or biomedical is prohibitively expensive due to the requirement of expert and requisite domain knowledge that lay an- notators may not possess. One common approach taken by researchers for NLU tasks is data augmen- tation, either online (Guo al., 2019; Ng al., 2020a; Sun et al., 2020; Guo, 2020; Sawhney et al., 2021) or offline in the form of generated synthetic data (Wei and Zou, 2019; Kumar et al., 2020; Zhou et al., 2021; Kim et al.. 2022; Guo et al., 2022a). Though most offline techniques perform well when employed for low-resource NLU tasks, we show that they tend struggle in almost all LLU tasks, often generating in-coherent and non-diverse aug- mentations, eventually leading to sub-optimal per- formance. We attribute this to algorithmic biases of existing augmentation approaches towards natural language and the varying characteristics of legal language (see Section 2 for more details). For ex- ample, most of these techniques often just tend to rephrase the source document, which is ineffective for LLU tasks due to the formalized nature of legal language, adversely affecting both generation diver- sity and downstream model generalization. Long- pre et al. also emphasize that aask-agn augmen- tation frameworks lead to reduced performance. To overcome these issues, researchers in specialized domains (e.g., biomedical) have developed special- ized algorithms (Kang et al., 2020; Ghosh et al., 2023), but the best of our knowledge, such approach has been proposed for the legal domain.
or facts. Our masking algorithm preserves valu- able hints and prevents the model from learning redundant knowledge by not asking to recon- struct document-specific entities or facts. Rather, it promotes acquiring broad legal knowledge and knowledge of legalese that enables DALE to ad- vance its capability in generating augmentations of legal documents with novel contexts that pos- sess remarkable levels of coherence and diversity. We call this masked document a template, and it serves as input to DALE for denoising- -assdd pre- training. We optionally fine-tune DALE on the downstream dataset, followed by conditional gener- ation to generate augmentations. We show that our domain-specifer sentence corruption algorithm en- ables DALE to generate diverse and coherent aug- mentations of legal documents, which entity- rich, semantically complex, and formal in nature. To summarize, our primary contributions are:
We propose DALE, the first generative data augmentation framework designed for low- resource legal NL..
2. Through extensive empirical evaluation 6 LLU tasks, 13 datasets, and 4 low-resource settings, we show that DALE outperforms all prior works with significant gains of I 1%-50%.
3. Additionally, through extensive ablative exper- iments and qualitative comparison, we show that DALE generates much more diverse and coherent augmentations than prior works.
2 Related Work
Legal NLP. Recently, the legal NLP literature has been flourishing with new resources like datasets (Leitner et al., 2019; Zhong et al., 2020b; Zheng et al., 2021; Hendrycks et al., 2021), benchmarks (Chalkidis al., 2021c; Niklaus et al., 2023; Chalkidis al., 2023) and PLMs (Chalkidis al., 2020; Xiao et al., 2021; Mamakas et al., 2022: Niklaus and Giofré, 2022). However, despite much progress, the specialized domain of legal language lags behind in available resources when compared to natural language or domains like bio-medical (Katz et al., 2023). As also mentioned earlier, most techniques employed for building better deep learn- ing NLU models do not transfer well to the legal domain due to characteristics that make it distinct from natural language (Morrison, 1989; Nair and Modani, 2023; Glogar, 2023), including its highly
Main Contributions. In the paper, we present DALE, novel data augmentation technique based conditional generation for low-resource legal NLP. Based on our initial analysis of legal docu- ments, we propose that augmentations enhancing LLU task performance can be achieved by not just rephrasing documents but also by modifying ex- isting contexts or introducing novel ones. DALE, designed to perform this, builds on BART (Lewis et al., 2019) and is first tra-ted on a large-scale unlabeled legal corpus using novel text denois- ing objective based on selective masking. Specif- ically, we leverage the inherent properties of tem- platized legal language to mask co-oceurring and highly correlated spans of text in a legal document and avoid masking random and emerging entities

---

Orig:
Did the superior court abuse its discretion in dismissing Morgans appeal for failure to exhaust administrative
Avaids
remedies?
Hints
Randomness
RM:
<masks
abuse kmmask discretion <masks
Morgans appeal Kimako to exhoust administrative (<mask>
GM:
kmasks abuse discretion ismasko dismissing Morgans appeal kmassi to exhaust administrative kmasko
PMI:
Did the <maske abuse its discretion in dismissing <maske) appeal for failure exhaust <maske
X
DM:
Km in dismissing Morgans I<masks to exhaust administrative <maskel?
kmaskk
failing to allow Hertz to
pro se plaintift?
Other sentences with the same oocuming span
Kmasks
in awarding attorneys fees to moore in the immeskel 12,560.37?
Figure I: Comparison of various span masking algorithms legal documents rich in emerging entities and case-specific facts. RM stands for random masking. GM stands for GENIUS extreme masking (Guo et al., 2022a), stands for PMI masking (Levine et al., 2021) and DM stands for our proposed DALE masking. Unlike other masking algorithms that make model learn redundant knowledge through denoising entities random tokens, our proposed masking formulation promotes leaming of broader legal knowledge and legalese by masking co-oceurring spans that consistently provide high signals.
formal, technical, entity-rich and knowledge-rich nature, along with semantically complex phrases. Simply put, the task of training machines to "un- derstand" legal language has proven to be non- trivial (Katz et al., 2023). For quite some time, researchers tried to teach models to solve complex LLU problems through prior findings in NLU, e.g., pre-training LMs (Chalkidis al., 2020). However, this has come with varying success (Zheng al., 2021). Exploiting domain-specific characteristics to build custom pre-training strategies has shown better success (Nair and Modani, 2023; Chalkidis* et al., 2023), and we emphasize that there is simi- lar need for all tasks in legal NLP.
performance of back-translation (Yu et al.. 2018) is affected by the inability of muaciine-tranlatiinn systems to translate entity-rich and formal legal language effectively. The work closest to ours is Guo et al. (2022a) and Wang et al. (2022), where the PLM is trained on keyword-to-seme sentencce construction task. However, these systems rely unsupervised keyword discovery, which is naturally biased towards rare entities prevalent in legal doc- uments. Denoising entities are case- or document- specific and would lead a model to learn redundant knowledge by reconstructing the case-specific fact around it, of which it has no prior knowledge. With- out informed masking, similar conclusion could be made for other PLM- baseed approaches in litera- ture (Kumar et al., 2020; Guo et al., 2022a).
Data Augmentation for Low-Resouree NLP. Data augmentation, both online (Guo et al., 2019; Ng al., 2020a; Sun et al., 2020; Kumar al., 2020; Guo, 2020; Sawhney al., 2021) and offline (Wei and Zou, 2019; Kumar et al., 2020; Zhou et al., 2021; Kim et al., 2022: Guo et al., 2022a), has seen great success in overcoming the data scarcity is- sue in low-resource NLU tasks. While the former employs techniques like latent space interpolation or mixing, the latter is based on generating syn- thetic data that can be augmented with the original data aid low-resource or few-shot learning (Chen al., 2023). However, though the data scarcity issue is exacerbated in specialized domains like legal, where annotation becomes prohibitively ex- pensive (Yang et al., 2019), domain-specific data augmentation techniques in literature are thin and almost non non-existent, especially for the legal domain. Perçin et al. (2022) proposes the only legal domain- specific approach for data augmentation. However, they substitute phrases from the WordNet (Miller, 1995), failing to generate diverse augmentations for legal text by only editing common natural lan- guage phrases in the WordNet. For example, the
3 Methodology
3.1 DALE Pre-training
Primary Goal. Our primary goal is to devise a denoising- based seq-to-seq pre-training algorithm crafted to favor our final objective, i.c., generating diverse and coherent data augmentations. Sentence denoising is better suited to our task (compared to other methods like prompt- instruction-tuning as gives us better control over long-document generations (explained further in Appendix E). The type of knowledge acquired through denoising ob- jectives has been seen to be highly dependent on the masking algorithm (Sadeq et al., 2022). Thus, to achieve our objective and devise a suitable mask- ing algorithm, we first try to answer question crucial to the success of our approach: Which at- tributes should an augmentation of legal docu- ment possess be considered effective, enabling improved generalization in downstream L.UU tasks? After conducting an analysis of legal documents, we hypothesize that formal language used the

---

The Sescestscce vere Taken If dite ettiecn the aife ter ceuncel
Carga
Eegel the
LegiDer
Savarrat Geongla
The SepoR15021e more tecae in Savartan, the sifo or sxcnsel leing preseni
condalaa tata the tee
partter ressoed (latano preceedtrgs:
Make
1872 divorce
Moe dec
brih partier rexided in Alzzama before proceedings.
saades cerried over the sxt to the bealerel of those prooneleyy
Strce peraane Jrrastnnteen over Faaberd
zararg the
dovorce
diver aclunn verrin moitieeiie precessirgs bot resice hestare these precaesiing the
cosnted Sersadictean uction
toueel con aad oest creaest woditacctiin pracaniengs husber
is spras rath matters.
satiuty precetural
matice
procestings
spree the trasz court Isd Isaterd in thase macaracecaor nestara.
pesssss
Mooking
over
led agree with the traed court (or qullot over
feotaal thow woll lacelang mullanss
preiiasing
dan wike fur ane sostte, eil becekree
Corrlisd
DALE
Deeswirce ELU thatuc
Gearrakea
uitain Five bastenan day the aate
transfer
PRyDERT SE USBDES
Optine
pooment cus Neraederr aithtn
the parma an thw. the Earrews shall be Jondle
Figure 2: Illustration of DALE. I We extract all correlated spans from a legal corpus using our discounted PMI formulation. (2) We shorten a legal document by selecting only the top-k sentences that are the most relevant to the document and removing the rest. @ We rank all the spans based on their importance and length using our novel scoring metric. Finally, we create a template by retaining the top-p spans and musking all other spans with added randomness. This process is followed by optional fine-tuning on the downstream dataset and conditional generation of augmentations from corrupted legal documents.
legal domain rarely allows for the occurrence of a rephrased version of the original document, un- like in everyday natural language. In fact, effective augmentations need to add new context legal documents or modify existing ones.
concepts, principles, and language usage by con- sistently providing high signals and avoiding noise. This will in turn allow DALE to generate consistent, plausible, and diverse augmentations. Fig. I picto- rially describes the problem with current masking algorithms and how our proposed algorithm favors our task. We call our final masked or corrupted document a template and denote it as T. DALE pre-training involves multiple steps for template creation followed by training to denoise these tem- plates We next describe each step to create T, which is done corpus-wise due to the variability of legalese across domains and genres.
What to mask? To modify the existing or in- troduce a novel context in legal documents while maintaining the formal legal style and plausibility of events in the generated context, DALE, like legal practitioner, should possess both broad legal knowledge and knowledge of legalese. However, acquiring either from legal documents with com- plex semantics and syntax is not trivial. Legal documents, written by law practitioners, consist of clauses that are primarily document- (or case-) specific facts. The text is entity-rich, and entities are usually emerging they are unique to that doc- ument. Beyond entities, these documents also con- tain text fragments outlining these entities and can be seen as an outcome of broad legal knowledge possessed by the practitioner. These co-occurring fragments, generally genre- or corpus-specific, are commonly reused by practitioners across docu- ments. Their presence is a core property of legalese which can be attributed to its trait of being a for- malized language (Nair and Modani, 2023). Fig. 1 shows an example sentence from a document with such structure (more examples in Table 17). Thus, we hypothesize that learning to denoise these frag- ments with appropriate context and hints will even- tually lead DALE to acquire knowledge about legal
(1) Correlated Span Extraction. To extract these reusable text fragments from an unlabeled legal corpus without supervision, we identify these frag- ments correlated spans of tokens. First, we denote the set of all n-gram spans in corpus C, as Nc {ng. "*, ng), where every span m==[w1, "",, w,). Here n ranges from 2 to q. Our objective now is to extract a set of distinct spans Se = (spo. "., spr) from No where each span sp, exhibits high co-occurrence over the corpus. Though modeling such correlations is widely stud- ied in computational linguistics (Zuidema, 2006; Ramisch et al., 2012), choose to use Pointwise Mutual Information (PMI) (Fano, 1961) a metric to score all individual n-grams in corpus. PMI, by definition, quantifies how often two tokens occur, compared with what we would expect if they were independent. Our proposed strategy is based on the

---

PMI formulation proposed by Levine et al. (2021)
B.I, including examples to show the effect of on correlated span extraction.
that extends PMI to n-grams as follows:
(M. min log p( p(w1.... ... wk) oEseg(u)
(2) Optimal Context Selection. Legal corpora, labeled and unlabeled, are generally structured at the granularity of document-level (collection of sentences). However, they are generally long (sce Appendix H for dataset details), and denoising- based pre-training with enc-dec model allows us to scale only the maximum output sequence length ly of the decoder (irrespective of the en- coder input sequence length). As mentioned earlier, LEGA employs BART-large with a maximum out- put sequence length of 1024 tokens (Appendix E explains the rationale behind our choice.). A com- mon choice for such scenario would be to just select the first ly tokens from the document Draw to form shorter document Dp- However, this creates lext-informativennss mismatch between pre-training and fine-tuning instances, as raw le- gal documents have sparse information compared to fine-tuning instances (Sugathadasa et al., 2019). Thus, we choose to perform optimal context selec- tion or select sentences from the document with high informativeness measure. To this end, we propose to use the PageRank algorithm (Page et al., 1999), boosted by sentence similarity. Given doc- Prow we
II (())
where (M is the PMI for the n-gram {w),"". '',, ,,,, and seg( wh, '", w,) is the set of all contiguous segmentations of the n-gram. We re- quest our readers to refer to the original paper for more algorithmic details. However, this base for- mulation faces two main challenges when extended to legal documents: (a) The PMI formulation is designed to favor tokens with lower frequency, making it choose rare tokens and not the text frag- ments of interest. This is further exacerbated by the fact that text in the legal domain is rich in case- specific, rare, and emerging entities. There is no clear way to retain hints for reconstruction in the original formulation. Since legal language is highly domain-specific, not doing so might lead a model to hallucinate or training to collapse (Li et al., 2021; Sadeq et al., 2022). We describe how we overcome (b) in step (3). To overcome (a). we propose modifying the existing formulation by imposing a discounting factor to penalize rare to-
kens (Pantel and Lin, 2002). Thus, our modified formulation is as follows:
ument Draw. with sentences [sg Drow ... .... use an encoder Eprc to calculate the embedding of each sentence [Cap- ''', C,,, and the entire doc- ument CD.... This is followed by calculating the cosine similarity between every 2 sentences in the corpus, indexed i and j. as follows:
log f (wi... wr) (1.)) * log(e) + log / (w1 wp)
where f() is the frequency of occurrence of the n-gram, and is the constant factor used threshold remove rare tokens. Precisely, refers to the minimum frequency of occurrence of an n- gram in the corpus below which the n-gram will be penalized. c is calculated based on the density of rare tokens in the corpus and is usually set to percentile of the frequency distribution of all n-grams in the corpus. We choose specific to the value of in the n-gram in the specific corpus. Generally, PMI for datasets with a higher degree of rare entities per document is discounted with a e corresponding to a frequency at a higher pc (like Caselaw (cas, 2018) and Edgar (Henderson et al., 2022)). In contrast, datasets with a lower degree of entities or lower overall degree of formal language are discounted with corresponding a frequency at a lower pe (like r/legaladvice (Henderson et al., 2022)). Finally, we select the top j% of n-grams with the lowest PMI score to construct Sc. We provide more details in Appendix
eg Esr
8i.j
(3)
||e
where i.j E {1,...mm n) and ez is defined as Car = Aeg + (1 \)eD,... Post this step; we construct an X n similarity matrix, which serves as an adjacency matrix for constructing graph g (V, E) where the sentences form the vertices V and the similarity scores form the edges E. Finally, we apply PageRank(S) to assign every sentence an importance score and select the top-k sentences not exceeding 1024 tokens. Following this, we sort the sentences in the document's 's original order of occurrence. We sample a probability from a Gaussian distribution N N11,, o²), and only do this step if crosses a set threshold (..
the pc th
(3) Selective Masking, Once we obtained the set of correlated spans Sc from step (1) and Dp from step 2)). we now want to select the best candidates for

---

masking from all spans in Sp,, SD, are the spans in Se only present in document Dp. To this end, we devise novel span-ranking metric to construct our template such that we preserve valuable hints but also prefer longer spans. Formally put, we first use a pre-trained encoder Epre to calculate the embedding of each span as 1espo* "', @sprJ and the entire document ep, followed by assigning importance score ie to each span spt follows:
masks. For datasets with documents exceeding 1024 tokens, we propose a sliding window mecha- nism for fine-tuning. Specifically, with window of size w tokens, we break down a long sequence into its constituent segments of 1024 tokens, with each segment beyond the initial segment having additional non-masked context from the previous window. This context is additionally bounded be- tween special tokens <context> and </context> provide the model with explicit supervision. We provide detailed explanation in Appendix on why the DALE fine-tuning masking algorithm is not well suited for pre- training and better fits the fine-tuning stage.
sim(Cspp:DD
i, norm(Icn(sp.))
(4)
where sim(.) is the cosine similarity between each Csp, and cp calculated similarly to Eqtn. The denominator is the length of the span normal- ized across all spans in SD, to assign higher im- portance to smaller spans. Finally, to create our template, we preserve the top-p spans in S, not ex- ceeding 20% of the entire document length, and mask all other spans in Sp, Finally, Each span is replaced by single mask token. To introduce randomness into the process, we sample probabil- ity y from Gaussian distribution N (M, o²) and randomly preserve a token in a contiguous span of tokens to be masked r crosses a set threshold o. After obtaining template 'T for all documents in the corpus for all corpora, we pre-train DALE on the denoising objective to reconstruct D, from T.
3.3 DALE Generation
To generate data augmentations using DALE, we construct a template by corrupting a sentence simi- lar to the fine-tuning stage and condition model to generate augmentations. We use beam search with random multinomial sampling to gen- erate diverse augmentations. Finally, we employ sliding window mechanism for long documents, combining outputs from all sliding window seg- ments for the final augmentation. After generating augmentations, add them the gold annotated data to fine-tune our downstream evaluation model.
the
Experiments and Results
3.2 DALE Fine-tuning
4.1 Tasks and Datasets
Though pre-trained DALE serves as an effective general-purpon data augmentation model for low- resource LLU tasks, we prefer to fine-tune BART on our downstream dataset so that our generated augmentations exhibit an underlying data distribu- tion similar to our gold dataset. This has been seen critical to improving in-domain performance with scale (Geiping et al., 2023), However, extract- ing correlated spans with PMI from fine-tuning datasets with few samples is generally ineffective as PMI becomes effective only with scale (Fano, 1961). Thus, to construct template, we extract all n n-grams = [ng. .., nt. .., Drr from a particu- lar document (or training instance) Dr and assign an importance score to each by calculating cosine similarity, similar to Eqtn. 3, between E Epre(1u (n:) LD,
Pre-training, To pre-train DALE, we use combi- nation of multiple datasets from Pile of Law (Hen- derson et al., 2022), CaseLaw (cas, 2018), and MAUD (Wang et al., 2023). The final pre-training corpus comprised 4.1M documents amounting to = 48GB. Detailed statistics are in Appendix H. Downstream Evaluation. To prove the efficacy of DALE, we conducted experiments 13 legal datasets based on 6 tasks across 4 low-resource settings. These tasks include Multi-class classifi- cation (MCC), Multi-label classification (MLC), Named Entity Recognition (NER), Multiple choice QA (MCQ) (identify the correct (masked) holding statement from selection of choices), Rhetorical Role Prediction (RR) (sequential text classifica- tion for assigning label to each sentence in a legal document for semantic document segmenta- tion), and Document-level NLI (DLI). For MCC, we experiment on SCOTUS (Spaeth al., 2013), LEDGAR (Tuggener et al., 2020), ILDC (Malik et al., 2021) and OTS-UL (Drawzeski et al., 2021)
and (A Epro(Dr + (1 - A) x EpreeLDp) here is the label for the document D;- We elabo- rate in Appendix 1.1 on how we construct Lp, for tasks beyond multi-class classification. Finally, we preserve the top-p n-grams and mask everything else in the sentence, before merging consecutive

---

AGold
100
200
500
1000
100
2001
$00
10000
100
200
500
1000
100
300
500
10000
100
200
500
1000
Datasel
OTS-TOPICS
RUR-IEXX
ECHRR-
RCHHR-B
UNFAIR-TR6
Gold-only
0.10
11.47
51.16
53.87
3.68
4.30
10.32
42.26
25.25
27:30
17.14
$1.52
37.69
47.47
64.89
50.98
0.10
31.88
740,02
76.21
EDA
12.31
22.93
49.26
10.10
35.64
41.91
49.67
43.01
48.70
56.32
13.93
72.35
78.14
LegalBUA
39.15
12:45
23.61
53.27
12.24
36.25
13.89
52.93
43:86
54.72
57.21
15.96
21.55
72.48
15.09
SSMBA
10.41
15.28
47.31
52.65
4,10
21.32
45.67
48.77
7.35
18:10
34.39
37.58
35.32
45.43
48.05
52.65
6.53
15.21
03.96
04.59
AEDA
14.06
52.63
60.29
22.32
3.077
33.33
50.35
52.21
28.12
30.94
32.29
45.48
39.15
50,85
50.48
51.36
5.08
52.34
70.48
73.67
SMERII
27.86
44.45
22.61
23.43
36.43
51,02
52.07
53.31
20,46
09.27
BackTruns
9:26
37.44
47.67
50.85
5.03
19.63
3736
42.65
14.73
17:37
35.36
39.41
37.61
49.58
50.77
52.83
12.54
39.28
46.51
02.64
C-M.M
3.85
17.95
58.54
61.45
7.17
25.21
45.04
47.85
27.95
23.24
23.89
30.25
39.46
52.17
53.26
54.68
30.42
48.52
59.87
69.62
GENIUS
25,58
$4.21
63.71
67.29
5.79
34,00
53.19
57.95
28,05
28.66
36.38
43.67
40.40
44,03
50.54
54,29
11.20
47.18
67.31
75.79
ChuGGP
53.31
52.21
27.52
27.89
14,03
41.83
43.12
49.76
53.87
10.78
44.62
72.91
Fileen
12.36
17.84
48.66
51.74
5.11
22.07
4619
49.01
17.68
20.39
35.81
38.62
16
46.51
47.27
53.85
5.44
16.10
60.80
67.51
DALE-BRT
25.07
5400
53.25
661
1230
14.30
3565
56.27
20.0
35.6
10.73
37.87
1581
90.7
34 55
60.0
1800
16.80
"13.57
DALE
24.36
11.50
29.51
51.55
21.19
33877
40.57
03.85
12.77
51.67
51.65
59.23
18.54
07.59
63:21
73.56
DALE
24.63
53.22
59.64
70.15
11.61
33.54
52.28
57.62
24.21
34.76
41.78
43:33
53.74
60.95
19.11
05.71
67.42
74.85
DALE (omIs)
33.91
61.33
71.56
75:24
13.50
37,95
55.90
59.45
29.43
37.37
44.38
55.72
46.72
56.13
59.18
64.01
22.32
54.62
74.84
62.98
Table 2: Results for Multi-label classification. DALE outperforms baselines by 19..%.
#Gold
100
200
500
1000
100
200
500
1000
100
200
500
1000
100
500
1000
LEDGAR
ILDC
SCOTUS
OTS
Gold-only
22,65
61.39
71.43
75.13
51 .48
54.24
55.83
58.03
63.69
65.93
70.75
75.92
66.72
68.59
70.21
72.54
EDA
42.65
59.31
72.34
75.76
49.76
49.83
59.32
61.72
53.00
61.57
72.51
73.29
68.93
69.66
72.13
73.28
Legal-EDA
53.00
60.57
13.28
76.72
52.15
52.23
60.38
62.27
55,21
61.39
73.69
75.57
69.51
71.67
76.31
79.72
SSMBA
47,86
60.34
70.06
74.21
47.62
50.21
58.53
60.12
43.00
60.57
72:51
76.26
60.12
70.17
75.47
76.04
AEDA
46.99
58.06
71.01
75.35
48.93
49.62
56.36
59,05
62.15
62.65
71.24
73.55
61.29
67.08
74.26
81.26
SMERTI
33.23
60.65
62.24
67.25
42.34
44.82
51.27
58.73
63.78
66.71
70.92
71.57
66.99
68.72
76.58
80.55
BacKTrans
51.23
58.96
69.04
40.72
59.18
62.01
42.01
45.63
57.22
67.56
59.69
65.81
66,23
71.53
C-MIM
34.12
60.95
63.11
68.15
43.18
45.65
52.01
58.98
61.56
65.54
71.25
71.95
67.05
68.97
77.52
79.62
GENTUS
48,76
62.14
71.17
74.48
51.35
54.26
53.39
52,14
59.42
61.71
63.14
70.28
66.71
68.65
76.20
79.73
GPT3 Mix
30.37
58.74
61.62
66.44
41.87
43.73
50.45
57.52
63.42
65.82
70.87
71.03
66.73
67.53
77.07
79.21
PromDA
45,76
51.24
65.40
68.27
41.30
43.08
49.21
51.27
44.50
53.86
59.72
61.58
63,72
65.73
70.38
73.28
ChaiGPT
46.87
61.18
70:41
73.92
50.74
52.93
52.34
51,21
58.69
60.56
62.81
69.40
65.01
67.88
75.32
78.19
Falcon
43.07
58.32
68.48
73.62
46.29
48.27
57.83
58.03
42.11
59.83
60.32
70.54
59.19
66.25
73.17
75.08
DALE-BART
50.95
57.90
64.28
70.87
52.26
51.54
$4.31
62.68
60.01
65.27
62.02
72.13
69.12
70.89
71.99
77.97
DALE-pt
48.26
55.39
65.27
67.94
52.02
51.87
57.26
58.51
59.61
63.25
66.72
68.85
69.93
70.21
73.68
75.89
DALE-B
58.67
68.38
72.22
52.14
53.88
58.15
61.92
59.70
64,62
65.46
72.41
68.85
70.91
74.31
77.58
DALE (OMMs))
55.13
63.76
74.89
78.36
54,47
55.95
62.45
63.11
65.85
67.86
74.89
78.96
71.64
72.89
77.74
83.75
Table
Results for Multi-class classification, DALEE outperforms baselines by 19%.4...%
datasets. For MLC,, we experiment on ECtHR Task A and B (Chalkidis et al., 2019, 2021b), EUR- LEX (Chalkidis et al., 2021a), UNFAIR-TOO (Lippi et al., 2019) and OTS-CT (Drawzeski et al., 2021) datasets. For NER, experiment on EDGAR (Au et al., 2022), and the Indian-Legal-NERR (Kalamkar et al., 2022) datasets. For RR, we experiment on the BUILD dataset (Malik et al., 2022). Finally, for DLI, we experiment on the ContractNLI (Koreeda and Manning, 2021). We perform blas-balanced sampling to create low-resource splits and down- sample the dev set accordingly. Dataset statistics are in Appendix H. We report micro-averagee F, scores averaged across 3 runs for random seeds.
a batch size of 32, We use the same setting for fine-tuning DALE but with a learning rate of le" -3
Downstream Task-Specific Setups. For down- stream task-specific evaluation, we fine-tune legal- longformeriage (Chalkidis* et al.. 2023). For fine- tuning legal-longformeengge we fine-tune for 20 epochs with batch size of 16 using Adam opti- mizer with learning rate of 1e -5
Details about the hyper-ppraaetee setup for our experiments can be found in Appendix B including hyper-paramete tuning experiments.
4.3 Baselines
Details on the working of each baseline can be found in Appendix F.
4.2 Experimental Setup
DALE. As mentioned carlier, use BART-large (Lewis et al., 2019) as our encoder-decoder model for training DALE. We detail in Appendix E why we think BARTiagge large is the most suitable for our task and setup. We pre-train DALE for epochs using -5 Adam optimizer with a learning rate of le and
Gold-only Baseline. This baseline common across tasks and uses only gold data without any additional augmentations.
Classification Baselines. For MLC, we compare DALE against EDA (Wei and Zou, 2019), Legal- EDA (Perçin et al., 2022), GENIUS(-Ft) (Guo et al.,

---

#Gold
100
200
500
1000
100
200
100
200
#Gold
100
200
500
1000
100
200
500
1000
Datasel
CasHHOLLD
BUILD-RR
ContractNL.
Baselines
EDGAR
INDIAN LEGAL NER
Gold only
66.38
70.30
74.62
Glid-conly
0.75
0.27
34.86
13.61
42.6
EDA
56.38
64.71
66.42
69:45
77.33
81.83
73.92
75:40
LWTR
22,10
36.84
50.33
54.15
12.53
17.87
35.54
44.15
AEDA
57.96
65.10
70055
T7.95
82.01
77.34
K3.02
13.31
36.15
42.58
26.13
SSMBA
62.01
67.65
69.59
69.75
77.77
81.66
76.27
82.93
MulDA
8.17
21.33
42.61
50.16
13:75
19.28
31.96
40.69
SMERTI
56.52
61.13
69.15
69:85
77.42
80.65
76.23
81.95
MR
19.13
50.95
58.33
18.62
25.26
43.14
RackTans
55.69
65.72
60.74
77.59
81.08
75.98
81.19
MELM
12.32
24.35
48.72
50.59
14.55
21.69
38.73
48.64
GENIUS
15.84
61.37
64.17
68.20
T8.99
79.30
77.28
81.28
GENIUS
13.79
28.44
50.93
62.69
19:05
29.28
48.72
53.61
CharGPT
60.83
77:32
PromDA
10.10
27.31
45.77
55.62
16:46
26.91
45.34
44.62
Falass
$2.57
58.76
62.41
63-22
75.11
77.61
75.17
77.54
ChaiGiT
12.65
26.32
49.25
60.67
18.24
27.58
46.44
51,41
DARRR
61:31
66610
67.91
7064
78:30
60.0
76.56
W1.39
Falecm
11:24
25.71
48.69
18:11
26,21
43.05
49.38
DALE-pt
59.25
65.69
67.81
69.70
78.15
79.01
76.97
80.55
DAT B-BRR
17.56
34.20
48.71
$7.99
16.43
20.T6
1600
"49.96"
DALE-0
60.31
66.56
68.16
15.15
78.50
79.72
77.10
81.73
DALE.ge
18.38
47.67
17.25
45.57
48.28
DALE (OurS)
63.71
68.14
71.53
72.70
81.83
83.04
79.36
85.13
DALE-R
19.10
35.39
48.20
58.74
17.65
28.32
46.71
49.98
(owrs)
23,65
21.31
49.93
Table 4: Results for MCQA (CaseHOLD), RR (BUILD-RR) R and DLI (ContractNLD), DALE outperforms by 0.5%-29.88%
Table 5: Results for NER. DALE outperforms by 1% -9.6%..
Papksihlil
2022a), SSMBA (Ng et al., 2020b), AEDA (Karimi et al., 2021), SMERTI (Feng et al., 2019), Back- Trans (Yu et al., 2018), C-MLM (Kumar et al.. 2020), ChatGPT (Dai et al., 2023) and instruction- tuned Falcon (Penedo et al., 2023). For MCC, we add to this list GPT3-Mix (Yoo PromDA (Wang al., 2022). Since GENIUS and C-MLM involve pre-training, we pre-trained it on our data with their respective masking algorithms.
200
203
80:33
13:09
85.88
56.14
12:35
56.55
$5.30
15.11
13.51
18.00
36.30
14.36
37.86
54314
12.8
1184
96.9
16:36
SMFRTI
10
5681
13366
58.00
11.5
Ruaiiives
17.00
15.55
77.03
13.85
0905
15.00
CARM
4:04
25.85
51.0
06.30
55.6
GPT-AS-
$3.36
36.36
53.31
11.6
05.85
1535
87.43
75.65
16.4
2021) and
39.37
113.46
3630
5143
MEEM
13.09
$5.08
56.55
68.31
93.85
76.17
55.55
Fdux:
4530
1366
17.43
44.97
15%
15:50
DALE-BAMI
20.35
172.4
222.7
2168
192.5
221.95
DALLS
19.35
14677
212:2
20.11
125.34
20099
BALK
15.0
175.8
221.2
18.44
354.39
234.00
Other Task Baselines For NER, we compare DALE against LwTR (Dai and Adel, 2020), DAGA (Ding et al., 2020), MulDA (Liu et al., 2021), MELM (Zhou et al., 2022b), PromDA, ChatGPT and instruction-tune tuned Falcon. For RR, DLI and MCQA, we compare DALE against EDA, GE- NIUS, SSMBA, AEDA, and BackTrans.
Table 6: Quantitative evaluation of generation quality the measures perplexity, token diversity (Diversity). and length diversity (Diversity-L). DALE outperforms all baselines.
mance of DALE with other baselines on MCQA, RR, and DLI. DALE outperforms baselines with absolute improvements in the range of 0.5%-29.88 29.8% in MCQA, 1%-7.2% in RR, and 2%.9.7% 5% in DLL. DALE-BART performs similarly to DALE-ft and is inferior to DALE, thereby showing the ineffec- tiveness of random masking for the legal domain.
DALE Ablations. To evaluate the effectiveness of the core steps in the DALE augmentation framework, we also compare DALE with other baselines on DALE-pt (augmentations generated with only a pre-trained DALE without any fine- tuning) and DALE-ft (augmentations generated with only a fine-tuned Legal-BART without DALE Pre-training). DALE-BARE is DALE pre-trained on Pile-of-Law with random masking. We provide additional results in Appendix B.
Qualitative Comparison. Table 6 compares the generation quality of DALE with all our baselines (averaged baseline-wise across all tasks and splits) on the measures of perplexity (Jelinek et al., 1977). diversity (average number of new tokens intro- duced in Rt augmentations) and length diversity (average absolute difference in length of source and R augmentations). DALE outperforms most of our baselines in all settings. DALE-pt generates more diverse augmentations but at the cost of not main- taining underlying data distribution. Beyond Table 1, Table 18 provides more augmentation examples. Contrary to our baselines. that are too conservative or too aggressive, DALE, especially for long doc- uments, generates augmentations that are diverse, coherent, and consistent with the source label.
4.4 Results
Quantitative Comparison. Table 3 compares the performance of DALE with other baselines on MCC (top-row) and MLC (bottom-row) DALE outperforms basclines with absolute improvements in the range of 1%.32.5% for MLC and 1%-49.8% for MCC. Table 5 compares the performance of DALE with other baselines on NER. DALE outper- forms basclines with absolute improvements in the range of 1 1%.69.6%. Table 4 compares the perfor-

---

Table 7: Comparison of augmentations generated by DALE and other baselines for the UNFAIR TOS dataset. All augmentations generated in a low-resource setting (500). Each augmentation was marked by law student 3 parameters: (1) If the augmentation is coherent, (2) If it adds new plausible context, and (3) if it is label-consistent and matches the underlying data distribution. present the results of the study /rr Xnext to each augmentation in the same order as above. Pink signifies the change from the Original. More examples can be found in Table 18.
UNFAIR ToS
Original
The most recent version of this agreement will be posted on the services under settings and also gotinder.com, and you should regularly check for the most recent version.
EDA
recent version of this agreement will be posted the services under settings and also gotinder com and you should regularly check for the most recent version X X /
AEDA
the most rrecent version of this agreement will be posted on the services under settings and also on gotinder.com and you should regularly check for the most recent version
SMERTI
This most recent version Windows will posted on power under settings available on gotinder. a and you should regularly check our most recent version. X X X
GENIUS
The terms of this agreement will be contingent the services they provide. For infor- mation, please visit www.sos.go gov. /
X x
ChatGPT The latest edition of this agreement will be made available on the services, specifically under the settings section and on gotinder.com It advisable to frequently review the most recent version. / x
Falcon
The most recent version of this agreement will be posted on the services under settings and also gotinder.com, and you should regularly check for the most recent version. / X /
DALE-
The most recent version of this agreement shall be accepted as the most recent amendment x
pt
/
DALE-
the most recent version of this agreement will be posted on the services under settings and also on gotinder.com, and you should regularly check for the most most recent versions. /
DALE
The most recent version this agreement be posted on the services's 's website at https://www.drrnianttt nianticlasssssssooomm con/ where you download and view the services, and you should be aware that this not guarantee that the services will be up to code or up to date, and we reserve the right discontinue using the services at any time,
5 Conclusion
1. DALE is still restricted to generating aug- mentations for legal datasets that consist of documents only in English. Though English is prevalent in the legal literature across do- mains and genres, recent work shows the im- portance of multi-lingual legal language mod- eling (Niklaus et al., 2023). As part of future work, we would like to overcome this short- coming by introducing multi-lingual DALE.
This paper presents DALE, a novel generative data augmentation framework for low-resource legal NLP. We evaluate DALE on datasets spanning across tasks under low-resource settings and show that DALE outperforms all prior art quantita- tively and qualitatively by a significant margin.
Acknowledgement
This work was supported by ARO grants W911NF2310322 and W911NF21 10026.
2. At extreme low-resource scenarios, DALE ac- companied by optional fine-tuning might be prone to over-fitting, generating almost simi- lar augmentations. Though using pre-trained DALE can overcome this problem, our ex- periments clearly show the benefits of fine- tuning. Thus, as part of future work, we would
Limitations and Future Work
this section, we list down some potential limita- tions of DALE:

---

like to explore the combination of augmenta- tions generated by pre-training and fine-tuned DALE.
Contract discovery: Dataset and few-shot semantic retrieval challenge with competitive baselines. In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 4254-4268, Online. Association for Computational Linguisties.
3. Our masking algorithm involves PMI, which is beneficial only at scale. Though benefit- ing from scale is an inherent property of pre- training, we would like to explore possible ways overcome this problem.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1
Ethies Statement
Ilias Chalkidis. 2023, Chatgpt may pass the bar exam soon, but has a long way to go for the lexglue bench- mark. arXiv preprint
We acknowledge that augmentations generated by DALE might not be always factual, i.e., contain events that have occurred in the real world. How- ever, DALE is not directly meant for helping legal practitioner in his everyday practice through its generations. Instead, DALE is meant for only generating augmentations to help train downstream models that can help legal practitioners in their practice.
(tt::234..22000
Ilias Chalkidis, Ion Androutsopoulos. and Nikolaos Ale- tras. 2019. Neural legal judgment prediction in En- glish. In Proceedings of the 57th Annual Meeting of the Association for for Computational Linguistics, pages 431 4317-4323, Florence, Italy. Association for Compu- tational Linguistics.
Ilias Chalkidis, Manos Fergadiotis, and Ion Androut- sopoulos. 2021a. Multieurlex-a multi-lingual and multi-label legal document classification dataset for zero-shot cross- longul transfer. arXiv preprint
References
#:Xiv:210900004
2018. Caselaw access project. Online. Accessed on April 25, 2023.
Ilias Chalkidis, Manos Fergadiotis, Prodromos Malaka- siotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. Legal-bert: The muppets straight out of law school. arXiv preprint arXiv:2010.02559.
Azad Abad and Alessandro Moschitti. 2016. Taking the best from the crowd:learning question passage classification from noisy data. In Proceedings of the Fifth Joint Conference on Lexical and Computa- tional Semantics, pages 136-141, Berlin, Germany. Association for Computational Linguistics.
Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapat- sanis, Nikolaos Aletras, Ion Androutsopoulos, and Prodromos Malakasiotis. 2021b. Paragraph-level rationale extraction through regularization: case study on European court of human rights cases. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 226-241, Online. Association for Computa- tional Linguistics.
Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. FLAIR: An casy-to-use framework for state-of-the- art NLP. In NAACL 2019. 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 59.
Ilias Chalkidis* Nicolas Garneau", Catalina Goanta, Daniel Martin Katz, and Anders Sógaard. 2023. LeX- Files and LegalLAMA: Facilitating English Multina- tional Legal Language Model Development. In Pro- ceedings of the 61 st Annual Meeting of the Associa- tion for Computational Linguistics, Toronto, Canada. Association for Computational Linguistics.
Ting Wai Terence Au, Vasileios Lampos, and Ingemar Cox. 2022. E-NER an annotated named entity recognition corpus of legal text. In Proceedings of the Natural Legal Language Processing Workshop 2022, pages 246-255, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Iz. Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The documeettnntt transformer. arXiv:2004.05/50.
Hlas Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, lon Androutsopoulos, Daniel Martin Katz, and Nikolaos Aletras. 2021c Lexglue: A benchmark dataset for legal language understanding in english. arXiv preprint 10.0006.
Andrew Staic-Stanck, Nils Holzenberger, and Benjamin Van Durme. 2020. Tax Law NIP Resources.
Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, and Diyi Yang. 2023. An empirical survey of data augmentation for limited data learning in nip. Trans- actions of the Association for Computational Linguis- tics, 11:191-211.
Lukasz Borchmann, Dawid Wisniewski, Andrzej Gretkowski, Izabela Kosmala, Dawid Jurkiewicz, Lukasz Szalkiewicz, Gabriela Palka, Karol Kacz- marek, Agnieszka Kaliska, and Filip Graliński, 2020.

---

Shuguang Chen, Leonardo Neves, and Thamar Solorio. 2022. Style transfer as data augmentation: A case study on named entity recognition. In Proceedings of the 2022 Conference on Empirical Methods Nat- ural Language Processing, pages 1827-1841, Abu Dhabi, United Arah Emirates. Association for Com- putational L Linguistics.
Sreyan Ghosh, Utkarsh Tyagi, Sonal Kumar, and Dinesh Manocha. 2023. Bioaug: Conditional generation based data augmentation for low-resource biomedical ner. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval.
Ondíej Glogar. 2023. The concept of legal language: What makes legal language 'legal '? International Journal for the Semiotics of Law-Revue interna- tionale de Sémiotique juridique, pages 1-27.
Haixing Dai, Zheng Liu, Wenxiong Liao, Xiaoke Huang, Zihao Wu, Lin Zhao, Wei Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, Hongmin Cai, Quanzheng Li, Ding- gang Shen, Tianming Liu, and Xiang Li. 2023. Chataug: Leveraging chatgpt for text data augmenta- tion. ArXiv, 0bs/2302.13007.
Biyang Guo, Yeyun Gong, Yelong Shen, Songqiao Han, Hailiang Huang. Nan Duan, and Weizhu Chen. 2022a. Genius: Sketch-based language model pre-training via extreme and selective masking for text generation and augmentation. arXiv preprint 10X3302211.10330.
Xiang Dai and Heike Adel. 2020. An analysis simple data augmentation for named entity recognition. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3861-3867, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Hongyu Guo. 2020. Nonlinear mixup: Out-of-manifon data augmentation for text classification. Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4044 4051.
Bosheng Ding, Linlin Liu, Lidong Bing, Canasai Kru- engkrai, Thien Hai Nguyen, Shafiq Joty, Luo Si, and Chunyan Miao. 2020. DAGA: Data augmentation with a generation approach for low-resource tagging tasks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 0445-6057,00 Online. Association for Computational Linguistics.
Hongyu Guo. Yongyi Mao, and Richong Zhang. 2019. Augmenting data with mixup for sentence clas- sification: An empirical study. arXiv preprint arXiv:190..0894.
Mandy Guo, Joshua Ainslie, David Uthus, Santiago Ontanon, Jianmo Ni, Kun-Hsuan Sung, and Yinfei Yang. 2022b. T5 Efficient text-to-text trans- former for long sequences. In Findings of the Associ- ation for Computational Linguistics: NAACL 2022, pages 724-736, Seattle, United States. Association for Computational Linguistics.
Kasper Drawzeski, Andrea Galassi, Agnieszka Jablonowska, Francesca Lagioia, Marco Lippi. Hans Wolfgang Micklitz, Giovanni Sartor, Giacomo Tagiuri, and Paolo Torroni. 2021. A corpus for mul- tilingual analysis of online terms of service. In Pro- ceedings of the Natural Legal Language Processing Workshop 2021, pages 1-8, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Rupert Haigh. 2023. International Legal English: A Practical Introduction for Students and Professionals. Routledge.
Robert M Fano. 1961. Transmission of information: statistical theory of communications. American Journal of Physics, 29(1 (177999937
Peter Henderson, Mark Krass, Lucia Zheng, Neel Guha, Christopher D Manning, Dan Jurafsky, and Daniel Ho. 2022. Pile of law: Learning responsible data filtering from the law and a 256gb open pouncs legal dataset. Advances in Neural Information Processing Systems,
Steven Y. Feng, Aaron W. Li, and Jesse Hoey. 2019, Keep calm and switch on! preserving sentiment and fluency in semantic text exchange. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing pages 2701-2711 1, Hong Kong. China. Association for Computational Linguistics.
Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. 2021. Cuad: An expert-annotated nlp dataset for legal contract review, arXiv preprint 1oXiv:2103.06200
(EMNLP-IJCN)
Zihao Fu, Wai Lam, Qian Yu, Anthony Man- Jan-ChoPfe So, Shengding Hu, Zhiyuan Liu, and Nigel Collier. 2023. Decoder-only or encoder-decoder? interpreting lan- guage model as a regularized pecoder-decoders arXiv preprint 2304:2334000044 04052..
Zihan Huang, Charles Low, Mengqiu Teng, Hongyi Zhang, Daniel E. Ho, Mark S. Krass, and Matthias Grabmair. 2021. Context-aware legal citation rec- ommendation using deep learning. In Proceedings of the Eighteenth International Conference on Arti- ficial Intelligence and Law, ICAII. '21, page 79-88, New York. NY. USA. Association for Computing Machinery.
Jonas Geiping, Micah Goldblum, Gowthami Somepalli, Ravid Shwartz-Ziv, Tom Goldstein, and Andrew Gor- don Wilson. 2023. How much data are augmenta- tions worth? an investigation into scaling laws, in- variance, and implicit regularization. In The Eleventh International Conference on Learning Representa- tions.
HuggingFace. 2023.
Hugging-
faceh4/open haapapeeennlmmeleebaaaaaaala leaderboard.

---

Maor Ivgi. Uri Shaham, and Jonathan Berant, 2023. Ef- ficient long-tex, understanding with short-text mod- els. Transactions of the Association for Computa- tional Linguistics, 11:284-299.
Elena Leitner, Georg Rehm, and Julian Moreno- Schneider. 2019. Fine-grained named entity recogni- tion in legal documents. In Semantic Systems. The Power of Al and Knowledge Graphs: 15th Inter- national Conference, SEMANTICS 2019, Karlsruhe, Germany, September 9-12, 2019, Proceedinggs, pages 272-287. Springer.
Fred Jelinek, Robert L. Mercer, Lalit R Bahl, and
James K Baker. 1977. Perplexity-a measure of the
difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62(S1):S63-SB) S63..
Yoav Levine, Barak Lenz, Opher Lieber, Omri Abend, Kevin Leyton- Moshe Tennenholtz, and Yoav Shoham. 2021. [PMI) -masking: Principled masking of correlated spans. In International Conference on Learning Representations.
Ziwei Ji, Nayeon Lee. Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys.
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zeulemoyer. 2019. Bart: De- noising sequence-to sereence-toseguuee to-seuuence pre-traine pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv: 13461.
Prathamesh Kalamkar, Astha Agarwal, Aman Tiwari, Smita Gupta, Saurabh Karn, and Vivek Raghavan. 2022. Named entity recognition in Indian court judg- ments. In Proceedings of the Natural Legal Lan- guage Processing Workshop 2022, pages 184-193, Abu Dhabi, United Arab Emirates (Hybrid). Associa- tion for Computational Linguisties.
Zhaowen Li, Zhiyang Chen, Fan Yang, Wei Li, Yousong Zhu, Chaoyang Zhao, Rui Deng. Liwei Wu, Rui Zhao, Ming Tang, et al. 2021. Mst: Masked self- supervised transformer for visual representation. Ad- vances Neural Information Processing Systems. 34:13165-133 13176.
Tian Kang, Adler Perotte. Youlan Tang, Casey Ta, and Chunhua Weng. 2020. UMML-based data augmen- tation for natural language processing of clinical re- search literature. Journal of the American Medical Informatics Association, 28(4)812-822
Marco Lippi, Przemyslaw Palka, Giuseppe Con- tissa, Francesca Lagioia, HlassWWoffang Mick- litz. Giovanni Sartor, and Paolo Torroni. 2018. CLAUDETTE: an automated detector of potentially unfair clauses in online terms of service. CoRR, abs/1805.01217.
Akhar Karimi, Leonardo Rossi, and Andrea Prati. 2021. AEDA: An easier data augmentation technique for text classification. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2748-2754, Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics.
Marco Lippi, Przemyslaw Palka, Giuseppe Contissa, Francesca Lagioia, Hans- Wolfgang Micklitz, Gio- vanni Sartor, and Paolo Torroni. 2019. Claudette: an automated detector of potentially unfair clauses in online terms of service. Artificial Intelligence and Law, 27:1 17-139.
Daniel Martin Katz, Dirk Hartung, Lauritz Gerlach, Abhik Jana, and Michael Bommarito II. 2023. Nat- ural language processing in the legal domain. arXiv preprint arXiv:2302.1 120.9. 12039.
Linlin Liu, Bosheng Ding, Lidong Bing, Shafiq Joty, Luo Si, and Chunyan Miao. 2021. Mulda: A multilin- gual data augmentation framework for low- low-resource cross-lingual ner. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joins Conference on Natural Language Processing (Volume 1: Long Papers), pages 5834-5846.
Hazel H Kim, Daecheol Woo, Seong Joon Oh, Jeong- Won Cha, and Yo-Sub Han. 2022. Alp: Data augmen- tation using lexicalized pefgs for few-shot text clas- sification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10894- 10902.
Philipp Koehn al. 2005. Europarl: A parallel corpus for statistical machine translation. In MT summit, volume 5, pages 79-86. Citeseer.
Shayne Longpre, Yu Wang, and Chris DuBois, 2020. How effective is task-agnostic data augmentation for pretrained transformers? In Findings of the Assuci- ation for Computational Linguistics: FMNLP 2020, pages 4401-4411, Online. Association for Computa- tional Linguistics.
Yuta Koreeda and Christopher Manning. 2021. Con- tractNLI: A dataset for document-lever natural lan- guage inference for contracts. In Findings of the Association for Computational Linguisties: EMNLP 2021, pages 1907-1919. Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Vijit Malik, Rishabh Sanjay, Shouvik Kumar Guha. Angshuman Hazarika, Shubham Nigam, Arnab Bhat- tacharya, and Ashutosh Modi. 2022. Semantic seg- mentation of legal documents via rhetorical roles.
Varun Kumar, Ashutosh Choudhary, and Eunah Cho. 2020. Data augmentation using pre-trained trans- former models. In AACL 2020 Workshop on Life- long Learning for Spoken Language Systems.
Vijit Malik, Rishabh Sanjay, Shubham Kumar Nigam, Kripabandhu Ghosh, Shouvik Kumar Guha, Arnab

---

Bhattacharya, and Ashutosh Modi. 2021. ILDC for CJPE: Indian legal documents corpus for court judg- ment prediction and explanation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1ih International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 4046-4062, Online. Association for Computational Linguistics.
Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The pagerank citation rank- ing: Bringing order to the web. Technical report, Stanford infolab.
Patrick Pantel and Dekang Lin. 2002. Discovering word senses from text. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge dis- covery and data mining, pages 613-619.
Dimitris Mamakas, Petros Tsotsi, Ion Androutsopou- los, and Ilias Chalkidis. 2022. Processing long legal documents with pre-trained transformers: Modding LegalBERT and longformer. In Proceedings of the Natural Legal Language Processing Workshop 2022, pages 130-142, Abu Dhabi, United Arab Emirates (Hybrid), Association for Computational Linguistics.
Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier. Ebtesam Almazrouei. and Julien Launay. 2023. The refinedweb dataset for falcon Ilm: Outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:230.01116.
Gabriele Marino, Daniele Licari, Praveen Bushipaka, Giovanni Comandé, and Tommaso Cucinotta. 2023, Automatic thetorical roles classification for legal doc- uments using legal-transfoomerrveeeettt.
Sezen Perçin, Andrea Galassi. Francesca Lagioia, Fed- erico Ruggeri, Piera Santin, Giovanni Sartor, and Paolo Torroni. 2022. Combining wordnet and word embeddings in data augmentation for legal texts. In Proceedings of the Natural Legal Language Process- ing Workshop 2022, pages 47-52.
George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38( (811))
Mary Jane Morrison. 1989. Excursions into the nature of legal language. Clev. St. L. Rev., 37:271.
Chen Qian, Fuli Feng, Lijie Wen, Zhenpeng Chen, Li Lin, Yanan Zheng, and Tat-Seng Chua. 2020. Solving sequential text classification as board-game playing. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05))8640-866
Inderjeet Nair and Natwar Modani. 2023. Exploiting language characteristics for legal domain-specifie lan- guage model pretraining. In Findings of the Asso- ciation for Computational Linguistics: FACI. 2023, Dubrovnik, Croatia, Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang. Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020, Exploring the limits of transfer learning with unified text-to-text trans- former. Mach. Learn. Res., 21(1).
Nathan Ng, Kyunghyun Cho, and Marzyeh Ghassemi. 2020a. Ssmba: Self-supervised manifold based data augmentation for improving out-of-domaan robust- ness. arXiv preprint arXiv:2009.1011 10195.
Carlos Ramisch, Vitor De Araujo, and Aline Villavi- cencio. 2012. A broad evaluation of techniques for automatic acquisition of multiword expressions. In Proceedings of ACL 2012 Student Research Work- shop, pages 1-6.
Nathan Ng, Kyunghyun Cho, and Marzyeh Ghassemi. 2020b. SSMBA: Self-supervised manifold based data augmentation for improving out-of-domain ro- bustness. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1268-1283, Online. Association for Computational Linguistics.
Federico Ruggeri, Francesca Lagioia, Marco Lippi, and Paolo Torroni. 2021. Detecting and explaining un- fairness in consumer contracts through memory net- works. Artificial Intelligence and Law, pages 1-34.
An Thanh Nguyen, Byron Wallace, Junyi Jessy Li, Ani Nenkova, and Matthew Lease. 2017. Aggregating and predicting sequence labels from crowd annota- tions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 299-309, Vancouver, Canada. Association for Computational Linguistics.
Nafis Sadeq, Canwen Xu, and Julian McAuley, 2022. InforMask: Unsupervised informative masking for language model pretraining. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pages 5866-5878, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Joel Niklaus and Daniele Giofré. 2022. Budget- longformer: Can we cheaply pretrain a sota le- gal language model from scratch? arXiv preprint arXiv:221..71335.
Ramit Sawhney, Megh Thakkar, Shivam Agarwal, Di Jin, Diyi Yang, and Lucie Flek. 2021. Hyp- mix: hyperbolic interpolative data augmentation. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 9858- 9868.
Joel Niklaus, Veton Matoshi, Pooja Rani, Andrea Galassi, Matthias Stürmer, and This Chalkidis. 2023. Lextreme: A multi-lingual and multi-task benchmark for the legal domain. arXiv preprint arXiv:2301. 13126.
Abhay Shukla, Paheli Bhattacharya, Soham Poddar, Rajdeep Mukherjee, Kripabandhu Ghosh, Pawan Goyal, and Saptarshi Ghosh. 2022. Legal case

---

document summarization: Extractive and abstrac- tive methods and their evaluation. arXiv preprint arXiv:2210.07544.
Yufei Wang, Can Xu, Qingfeng Sun, Huang Hu, Chongyang Tao, Xiubo Geng, and Daxin Jiang. 2022. PromDA: Prompt-based data augmentation for low- resource NLU tasks. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguisties (Volume 1: Long Pupers), pages 4242- 4255, Dublin, Ireland. Association for Computational Linguistics.
Ann Sinsheimer. 2007. Christopher williams, tradi- tion and change in legal english: Verbal construc- tions in prescriptive texts. Language in Society, 36(3):473-474.
Harold J Spaeth, Lee Epstein, Andrew D Martin, Jef- frey A Segal, Theodore J Ruger, and Sara Benesh. 2013. Supreme court database, version 2013 release 01. Dutabase al http://sppeeeccourrllaauuue org.
Jason Wei and Kai Zou. 2019, Eda: Easy data augmenta- tion techniques for boosting performance on text clas- sification tasks, arXiv preprint arXiv:1901.1 100.1111000. 1776.
Christopher Williams. 2007. Tradition and Change in Legal English: Verbal Constructions in Prescriptive Texts, volume 20. Peter Lang.
Keet Sugathadasa, Buddhi Ayesha, Nisansa de Silva, Amal Shehan Perera, Vindula Jayawardana, Dimuthu Lakmal, and Madhavi Perera. 2019. Legal document retrieval using document vector embeddings and deep learning. In Intelligent Computing: Proceedings of the 2018 Computing Conference, Volume 2, pages 160-175. Springer.
Chaojun Xiao, Xueyu Hu, Zhiyuan Liu, Cunchao Tu, and Maosong Sun. 2021. Lawformer: A pre-trained language model for chinese legal long documents. Al Open, 2:79-84.
Lichao Sun, Congying Xia, Wenpeng Yin, Tingting Liang, Philip S Yu, and Lifang He. 2020. Mixup- transformer: dynamic data augmentation for nlp tasks. arXiv preprint urXiv:2010.02394.
Yinfei Yang, Oshin Agarwal, Chris Tar, Byron C. Wal- lace, and Ani Nenkova. 2019. Predicting annotation difficulty to improve task routing and model perfor- mance for biomedical information extraction. In Pro- ceedings of the 2019 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 1471-1480, Min- neapolis, Minnesota. Association for Computational Linguistics.
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang. Dani Yogatama, Ashish Vaswani, and Donald Met- zler. 2021. Scale efficiently: Insights from pre- training and fine-tuning transformers. arXiv preprint arXiv:2 200: 10686.
Yi Tay, Mostafa Dehghani. Vinh Q Tran, Xavier Gar- cia, Jason Wei, Xuezhi Wang. Hyung Won Chung, Dara Bahri, Tal Schuster, Steven Zheng, et al. 2023. U12: Unifying language learning paradigms. In The Eleventh International Conference on Learning Rep- resentations.
Kang Min Yoo, Dongju Park. Jaewook Kang, Sang- -Woo Lee, and Woomyoung Park. 2021. GPT3Mix: Lever- aging large-scale language models for text augmen- tation. In Findings of the Association for Computa- tional Linguistics: EMNLP 2021, pages 2225-2239 Punta Cana, Dominican Republic. Association for Computational Linguistics.
Don Tuggener, Pius von Däniken, Thomas Peetz, and Mark Cieliebak. 2020. LEDGAR: A large-scale multi-label corpus for text classification of legal pro- visions in contracts. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1235-1241, Marseille, France. European Lan- guage Resources Association.
Adams Wei Yu, David Dohan. Minh-Thane Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V Le. 2018. Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv preprint 1804:00944
Lucia Zheng. Neel Guha, Brandon R Anderson, Peter Henderson, and Daniel E Ho. 2021. When does pre- training help? assessing self-supervised learning for law and the casehold dataset of 53,000+ legal hold- ings. In Proceedings of the eighteenth international conference arrificial intelligence and law, pages 159-168.
Alex Wang. Yada Pruksachatkun, Nikita Nangia, Aman- preet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stick- ier benchmark for ceneral-purpose language under- standing systems. Advances in neural information processing systems, 32.
Alex Wang. Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task task benchmark and analysis platform for natural language understanding. arXiv preprint 0rXiv./804.0746..
Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020a. How does nip benefit legal system: A summary of legal artificial intelligence. arXiv preprint orXiv:2004.12.1
Steven H. Wang, Antoine Scardigli, Leonard Tang. Wei Chen, Dimitry Levkin, Anya Chen, Spencer Ball, Thomas Woodside, Oliver Zhang, and Dan Hendrycks. 2023. Maud: An expert-annotated le- gal nlp dataset for merger agreement understanding.
Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020b. Jec- qa: A legal-domain question answering dataset. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9701-9708.

---

Jing Zhou, Yanan Zheng, Jie Tang, Li Jian, and Zhilin Yang. 2022a. HlipDA: Effective and robust data aug- mentation for few-shot learning. In Proceedings of the 60th Annual Meeting of the Association for Compulational Linguistics (Volume I: Long Papers), pages 8646-8666, Dublin, Ireland. Association for Computational Linguistics.
and 0.3 respectively. For selective masking, we set μ, o², and a be 0.4, 0.6, and 0.4 respectively. For optimal context selection we set to 0.7 and 0.5 for downstream DALE fine-tuning. We set aug- mentation rounds R to be 5. All hyperparameters were tuned on the dev set. We also show the tuning results of some important hyperparameters in the following sub-sections.
Ran Zhou, Xin Li, Ruidan He, Lidong Bing, Erik Cam- bria. Luo Si, and Chunyan Miao. 2021. Melm: Data augmentation with masked entity language modeling for low-resource ner. arXiv preprint 0rXiv:2108.1 13655.
B.1 Discounting Factor
Table 15 details the discounting factor с corre- sponding to the percentile pc for each corpus used in DALE pre-training, A corpus with documents that are entity-rich has a higher discounting factor (Caselaw) compared to corpus with more nat- ural language sentences and, thus, lesser entities (f/legaladvice).
Ran Zhou, Xin Li, Ruidan He, Lidong Bing, Erik Cam-
bria, Luo Si, and Chunyan Miao. 2022b. MELM: Data augmentation with masked entity language mod- eling for low-ress cow-resource NER. In Proceedings of the 60th Annual Meeting of the Association for Compu- lational Linguistics (Volume 1: Long Pupers), pages 2251-2262. Dublin, Ireland. Association for Compu- tational Linguistics.
Table 15 provides examples of correlated spans extracted through PMI calculation before and after discounting. Clearly, the discounting factor plays major role in extracting spans that are reusable text fragments with fewer entities.
Willem Zuidema. 2006. What are the productive units of natural language grammar? a dop approach to the automatic identification of constructions. In Pro- ceedings of the Tenth Conference on Computational Natural Language Learning (CoNLL-X), pages 29 36.
B.2 Augmentation rounds R
A Algorithm
Table 8 compares the performance of DALE at dif- ferent values of R. Augmenting the training dataset with several augmentation rounds R proves effee- tive until saturation point is reached. Downstream LLU performance improves when more DALE aug- mentations are added to the gold, similar to findings in Geiping et al. (2023).
We show DALE algorithmically in Algorithm 1.
Algorithm DALE: Our proposed augmentation framework
Given pre training dataset C, FLM Enc only FLM
Connaked + Ø
No
D Extract all n-grama Extract all correlated spans from r gramms
Sc - Ne
Se
Select only top y'u D M-Masking Loop O-Optimal Context Selection
Draw E
D, - D
SDy So
b-iiler only spans present in D
2 54.58
5
all spans in Sypp
53.67
55,02
58.94
59.35
59.31
59.09
T - D, Keep top.p spans and mask the
Selective Masking
end for
Prestrain E with denoising reconstruct Dp from T Given lou-resource fine-tuning dataset Dieain and DALE Opetional
Table 8: Fl for various settings of R. All values are av- eraged across all datasets and all low-resource settings.
for (X, Y) e Darai du -X
b Selective Masking
end for
Fine tune C with denoising to reconstruct X from (X, Y1 Derain
D Generation Loop
B.3 DALE without Optimal Context Selection
repeat R. times:
D-Selective marking P Generate augmented data
Xang - GENAUC
Table compares the performance of DALE with and without optimal context selection. We show that optimal context selection plays a significant role in improving the performance of DALE.
Dous Dass (Xaug)
end for
Fine-tune P with Bauy
retum
B Hyperparameter Tuning
w/ Optimal Context
w/o Optimal Context
59.35
57.46
Hyperparameters. We set q to 7 for n-gram extrac- tion. Values of and pc are provided in Appendix B. We choose legal-longformerrngg as Epre(-). For PMI selection we set j to 50%. For optimal context selection we set /4, a- and 33 to be 0.5, 0.7,
Table 9: F1 with and without optimal context selection. All values are averaged across all datasets and all low- resource settings.

---

C More Results
are deemed non-important by the algorithm). We show two examples below, where we compare the masking algorithms on two pre-training sentences:
As discussed earlier, for most of our experiments in Section 4.4, we adhere to simple ancoder-only architectures, However, we hypothesize that RR on the BUILD dataset (Malik et al., 2022) and DLI, we on the ContractNLI (Koreeda and Manning, 2021) dataset might benefit from complex architecture due the nature of their task. Thus, we com- pare the performance of our baseline augmentation strategies with DALE augmentations on the current state-of-the-art for RR and DLI tasks. Table 10 shows results. As clearly visible, when compared to GENIUS augmentations (also the second- baseline in Table 4), DALE shows better margins than using simple baseline. This proves hy- pothesis that better architectures can lead to better performances with DALE for more complex tasks beyond just classification.
1. DALE Masking: <mask> abuse its discretion <mask> dismissing Morgans appeal <mask> exhaust administrative <mask>
2. Importance Masking:
<mask> the superior court abuse its discretion <mask> to exhaust administrative <mask>
DALE Masking: <mask> payment due <mask> payment is due <mask>
2. Importance Masking: The Borrower shall make all payment due hereunder <mask>
As we clearly see, denoising using DALE masks exactly replicates how a legal practitioner would gain knowledge about legal concepts, principles, and language usage (powered by co-occurring and principled span masking). On the other hand, im- portance masking masks random spans that hurt learning. However, with label information, im- portance masking works well for our purpose and retains spans most informative of the instance la- bel (important for maintaining label consistency in generations).
#Gold
100
200
100
200
Dataset
BUILD-RR
ContractNLII
Gold-only
76.1
78.3
75.3
84.2
AEDA
79.6
84.6
79.0
86.5
Genius
80.2
81.9
79.2
85.6
DALE
85.3
88.9
84.7
89.7
Table 10: Result comparison of DALE on BUILD-RR and ContractNLI datasets using systems proposed in Marino et al. (2023) and Ivgi et al. (2023) respectively.
The quality of the spans retained also largely depends on the encoder used for similarity scoring, Additionally,
DALE pre-training masking al- gorithm is a principled masking algorithm asking the model to recreate and learn a similar nature of knowledge across the corpus. For importance masking, the high variability in the nature of words or phrases masked breaks this principality, thus reducing the effectiveness, In the final version of our paper, we will also include comparison of pre-training the two algorithms on a smaller corpus to show the effectiveness of our proposed algorithm.
D Comparison of Masking Algorithms
The main objective of correlated span extraction (using our modified formulation) is to mask infor- mative and co-occurring text fragments that usually outline the emerging and case-specific facts and en- tities (Section 3.1 explains why this is important for the success of DALE). Using the masking pro- cess described in Section 3.2 (named importance masking hereof) does not satisfy our needs. With- out the label information, the importance masking algorithm will merely retain the "most important" n-gram spans (and mask everything else), where importance is measured with respect to the context of the entire sentence. This leads to two additional problems:
2. Finally, label information a key ingredient to importance masking and ineffective without it. The importance masking algorithm is designed with the intuition that retaining the "most impor- tant" n-gram spans with label information will lead to augmentations that maintain label consistency. Maintaining label consistency (i.c., the augmenta- tions should be of the same label the source sen- tence) is key to any data augmentation algorithm. Without label information, the importance of each span will measured only with the help of the doc- ument context, which will capture non- informaatte
Beyond just not explicitly masking co- occurring spans (which we iterate is important for effective learning), the importance masking algo- rithm often does the exact opposite and masks case- specific facts, entities, and random spans (as they

---

spans. Also, legal documents are generally long, and different parts of the document play different roles (Malik et al., 2022). Without a label, using just these documents for importance scoring leads to ambiguity in the selected spans for masking.
2023). This gives rise to two specific problems with both instruction-tunee and prefix-tuned LMs: (1) With an increase output length, the properties in output generations deviate from the original sen- tence and attributes specified in the input. (2) The model's tendeney to hallucinate increases, generat- ing non-factual and non-plausible augmentations. We show examples in Table 18.
E Comparison of Pre-trained Language Models
Why BART? The choice of PLM depends the task (Tay al., 2023). Based on denoising training and conditional generation, our algorithm better suits the encoder-decoder paradigm. Tay et al. also show that decoder-only LMs are ineffective for denoising- based training. Open source encoder- decoder models include T5 (Raffel et al., 2020), BART (Lewis et al., 2019), LongT5 (Guo et al., 2022b), Longformer Encoder-Decoder (Beltagy et al., 2020), FlanT5 (Tay et al., 2021) and Flan- UL2 (Tay al., 2023). Though some of these models support input lengths 2 1024, to the best of our knowledge, the maximum decoder output length is 1024 (for BART-large), except Flan-UL2, Flan-UL2 LLM is difficult to train even on com- mercial GPUs, and we found BART-lagge, much smaller in size than Flan-UL2, to perform excep- tionally well already in case. We leave further exploration for future work.
In this section, we first try answer why we think denoising an appropriate training objective to generate better data augmentations for the legal domain. Following this, we try to justify our choice of PLM among all open-source PLMs available.
Why denoising? Synthetic data augmentation can be seen as a document (or sentence) editing or re- writing task, where the primary aim is to generate diverse and coherent forms of the original docu- ment while maintaining consistency with the orig- inal document in terms of underlying data distri- bution and factuality. Generating augmentations with plausible contexts has been seen as im- portant measure in weewledge-intensiig domains like legal and biomedical (Ghosh et al., 2023). Le- gal documents, by nature, are filled with domain- and case-specific facts and entities, which are, in turn, derived from the general knowledge of law, For example: An ideal augmentation, which might also help the model generalize better, should be allowed to change the context of the sentence (or the context of the facts and events occurring in the sentence), but only to the extent that maintains plausibility and does not contradict general legal knowledge. Thus, we hypothesize that this task can be best framed as a text infilling task, which allows the model to re-write the document in the presence of key hints, thereby avoiding hallucination. Re- writing requires the LM to possess the knowledge of legalese and general legal knowledge, and our masking algorithm is designed to make the model acquire this knowledge.
F Baselines
In this section, we provide details about the work- ing of each of our baselines taken from prior art.
EDA. EDA (Wei and Zou, 2019) performs syn- onym replacement from WordNet, random inser- tion, random swap, and random deletion of tokens in the source sentence to generate additional syn- thetic augmentations. Legal text generally has mantically and syntactically complex phrases and entities, and finding matches from the WordNet leads to in-coherent augmentations.
Why do decoder-only LLMs struggle to gener- ate coherent and factual data augmentations in the legal domain? Legal corpora, both unla- beled and downstream labeled, are structured at a document level as opposed to natural language, which is generally structured at sentence level. Additionally, legal documents are generally much longer. Decoder-only LLMs suffer from attention degeneration problem, where, as the length of the target sequence grows, less and less attention will be focused on the source sequence (Fu et al.,
Legal-EDAs Legal-EDA (Perçin al., 2022), sim- ilar to EDA, performs replacement from WordNet but employs pre-trained Word Embeddings cal- culate a similarity metric to choose the best candi- dates for replacement.
GENIUS. GENIUS (Perçin et al., 2022), similar to DALE, pre-trains and optionally fine-tunes BART on denoising objective using sketches generated with an extreme masking algorithm. This algorithm just preserves keywords in a sentence and masks everything else. As mentioned earlier, we pre-

---

Dataset
Source
Sub-domain
Task Type
Training/Dev/Tess Dev/esst Instances
Classes
ECUIR (Task A]
Chalkidis al. (2019)
ECHR
Multi label clusitication
9,000/1
1000//.0001..000
10+1
ECUIR (Task B)
Chulkidis al. (2021b)
ECHR
Multi-abel clusidication
10000/0000/10000
10+1
SCOTUS
Spueth et (2013)
US Law
Multi-class clasilicution
5.000/1,4000 1,400/1,400
14
EUR-LEX
Chulkidis et al. (2021a)
EU Law
Mult-label clusiiiiatiinn
55,00005,00055 0,00
100
LEDGAR
Tuggener (2020)
Contracts
Mulri-class classification
10.0000100000000,,000
UNFAIR-TiS ToS
Lippi et al. (2019)
Contracts
Mulri label classification
5,532/2.275/1
8+1
CasHHOLD
Zheng et al. (2021)
US Law
Multiple choice QA
45,0000/3,90033, 9,0909000
n/a
ILDC
Malik (2021)
Law
Multi-class classification
32,305/99/1.511
OTS-UL
Drawzeski
Multi-elass classification
2074/191/417
OTS-CT
Drawzeski al. (2021)
EU Law
Multi-class classification
01942//,69004.29
8+1
EDGAR
Au at. (2022)
US Law
Named Emiity Recognition
8150/1 7440
7
Legal NER (Preamble)
Kalamkar (2022)
Law
Named Emiity Recognition
1560/125/441
Indian Legal NER (Judgment)
Kalankar al. (2022)
Law
Nameed Emily Recognition
9435/949/4000
14
ContracNLI
Koreeda and Manning (2021)
NDA
Natural Language Inference
423/61/123
17
RUILD
(2022)
Law
Sequential Classification
247/30/30
Table 11: Statistics for each downstream LLU dataset used in our experiments, As described in Section 4.2, we derive low-resource splits from these original datasets for our experiments.
train GENIUS warm-starting from BART, using the extreme masking algorithm on our pre-training dataset. It proves ineffective for legal texts as le- gal documents are rich in entities (i.e., keywords determined by its unsupervised keyword extrac- tion algorithm), and the algorithm generally leads the model reconstruct case-specific facts around these entities.
C-MLM. C-MLM (Kumar al., 2020) employs BART to replace random tokens via mask infilling in a source sentence to generate augmentations. As mentioned, we pretrain a BART using random masking on our pre-training data for this baseline. Though effective for NLP, augmentations generated by replacing random tokens do not help in legal text. Morcover, BART trained on random masking algorithm fails infill masks and generate coherent legal text as the random masking algorithm does not promote learning of legal language.
SSMBA. SSMBA (Ng et al., 2020b) generates syn- thetic training examples by using a pair of corrup- tion and reconstruction functions to move randomly a data manifold.
AEDA. AEDA (Karimi et al., 2021) is similar to EDA but only employs random insertion of punctu- ation marks in the original text to generate synthetic augmentations. Legal text, being formal nature, is already punctuated; thus, this proves ineffective on legal documents.
ChatGPT. Chataug (Dai al., 2023) based on ChatGPT employs ChatGPT to rephrase existing sentences and generate more synthetic examples. The prompts are designed to generate single or mul- tiple augmentations at a time, and we use the for- mer. We emphasize that just rephrasing a sentence does not serve effective augmentation for the le- gal domain, adding to the fact that ChatGPT starts hallucinating with rephrasing long legal documents, common problem with decoder only LLMs (Fu et al., 2023). We show examples of ChatGPT gen- erations in Table 18. We use the March 24 release of ChatGPT (version: 6825453).
SMERTI SMERTI (Feng al., 2019) employs techniques like semantic text exchange using masked language models, keyword replacement (with keyword extraction similar to GENTUS), and adding synthetic noise using LMs. Though effec- tive for NLP, these methods generate incoherent augmentations for formal language like legal. For example, randomly replacing tokens generally re- places tokens in a complex phrase, and keyword replacement using RAKE generally tends to edit emerging entities, both of which do not lead to efficient augmentations for the legal domain.
Falcon. Falcon (Penedo al., 2023), similar ChatGPT, employs open-source instruction -uned LLM falcon to rephrase existing sentences and gen- erate more synthetic examples. We use a similar set of prompts, adding to an additional prompt which is: "Generate 5 different and diverse forms of the sentence:", We found Falcon to struggle in follow- ing instructions like "Rephrase the sentence:" and "Generate diverse augmentation for the sentence:". Additionally, Falcon also refuses generate di- verse forms of legal sentences at times. Falcon proves to be inferior in both rephrasing and gener-
BackTrans BackTrans or Back Translation (Yu et al., 2018) translates sentence into target lan- guage and then translates it back into source language. Machine Translation systems generally prove to be ineffective translating formal and entity-rich language in legal documents, thus gen- erating incomplete and incoherent augmentations.

---

ating diverse forms of legal documents. We show examples of generations in Table 18.
Supreme Appeals
Argunter
GPT3-Mix. GPT3-Mix (Yoo al., 2021) prompis GPT3 (Brown et al., 2020) generate new training samples by mixing 2 existing samples of opposite labels. This is followed by pseudo-labeling using GPT3. Mixing samples have been very often exper- imented in NLP for boosting diversity. However, we noticed that it leads to incoherent sentences in the case of legal language due to its formal nature. PromDA. PromDA (Wang et al., 2022) proposes a data augmentation framework based on T5 that trains soft prompts using a novel keyword-to- sentence algorithm.
Edga Conract (Borchmune ii... 2020)
10 76GB
L446
24K
Rolai skgpeaahiz & sleptcalscoolliue Toaal
299
40.M
DRIK
26GB
334B
1.5M
Table 12: Statistics of various legal corpora in Pile of I Law considered for building our pre-training dataset.
Data Source
Data Siee
Word Count
Dicament Coant
Cooaav Total
22GB
+6.57
2.54
2003
-4.65
2.3M
Table 13: Statistics of Caselaw legal corpus.
Daat Suurre
Data Size
Word Count
Descmment Comnt
MAUD Ionl
124.5MB
21.6M
39.2K
125M
22M
-WK
MELM. MELM (Zhou et al., 2022b), which stands for Masked Entity Language Modeling, suggests the fine-tuning of a transformer-encoder based PLM on linearized labeled sequences through masked language modeling. In low-resource sce- narios, MELM surpasses all other baselines and prior techniques on the CoNLL 2003 NER dataset across four languages, including mono-lingual, cross-lingual, and multi-lingual settings.
Table 14: Statistics of MAUD legal corpus.
guage.
We do not consider this base- line. Style-Transfer (Chen et al., 2022) gener- ates augmentations by changing style-related at- tributes. Our initial experimentation revealed that style-transfer generated highly in-coherent augmen- tations for the legal domain. Thus, we conclude that style-transfer to be non-trivial for legal lan- guage compared to natural language.
DAGA. DAGA (Ding et al., 2020), short for Data Augmentation with Generation Approach, sug- gests the training of one-layer LSTM-based re- current neural network language model (RNNLM) by maximizing the probability of predicting the next token using linearized sentences. For sentence generation, they employ random sampling to create entirely new sentences, with the model being fed only the [[BOS]] token.
G Examples of generated augmentations
We provide additional augmentation examples Table 18. Each augmentation was marked by a law student on parameters: (1) the augmentation is coherent, (2) If it adds new plausible context, and (3) if it is label-consistent and matches the underlying data distribution. We present the results of the study orr Xnext to each augmentation in the same order above.
MulDA. The Multilingual Data Augmentation Framework (MulDA) (Liu et al., 2021), an exten- sion of DAGA, enhances generation-baut lingual data augmentation by training pre-trained mBART model on next token prediction using lin- earized sentences. To ensure fair comparison, substitute mBART with mBART-50 in the MulDA approach.
multi-
H Dataset Details
H.1 Pre-training Dataset Details.
For pre-training DALE, we use the Pile of Law dataset (Henderson et al., 2022). The dataset is collection of multiple (unlabeled) legal corpora (Huang al., 2021; Borchmann et al., 2020; Blair- Stanck al., 2020; Hendrycks et al., 2021; Koehn et al., 2005; Lippi al., 2018; Ruggeri et al., 2021) with 255 GB of text. Detailed statistics for each dataset can be found in Table 14
LwTR. LwTR (Dai and Adel, 2020) replaces to- ken in sentence with another token of the same label; the token is randomly selected from the train- ing set.
FlipDA. We do not consider this baseline. FlipDA (Zhou al., 2022a) trains generative model to generate label-fliped data. Our initial experimen- tation revealed that label-fipping generated highly in-coherent augmentations for the legal domain. Thus, we conclude that abbel-flpppin to be non- trivial for legal language compared to natural lan-
H.2 Fine-tuning Dataset Details.
In this section, we list a detailed description of each of our downstream LLU datasets and dataset

---

Souru
Discunting
Cut-OIf
ble justifications for the outcome of case. Dataset statistics are provided in Table 11.
MAUD
75%
57, 38, 36,
Rodii elapulabtis & r/ppilabiccofllppii
75%
6,3,2,1, 1,1
US Ayroals Decisions
U.S Supreme Court Onl Argument Tran
95%
21, 19, 12.3,5
OTS-UL. Online Terms of Service (OTS) (Drawzeski et al., 2021) attempt to automatically detect unfair clauses in Terms of Service. The input to the model is sentence, and the output presents the sentence classified into three levels of unfairness. The dataset setting used in paper is similar to (Niklaus et al., 2023). Dataset statistics are provided in Table 11.
scipts
6,
3,32
Table 15: Discounting values for different datasets used DALE Pre-training. Cut-Off values for each value of (in the order of 3,4,5,6 and 7) for the r-grams considered in our experiments.
statistics for each.
H.2.2 Multi-lahel Classification
ECTH Tasks A & B. Allegations are brought be- fore the European Court of Human Rights (ECtHR) regarding the violation of human rights provisions outlined in the European Convention of Human Rights (ECHR) by state. We use the datasets from (Chalkidis et al., 2019) and (Chalkidis et al., 2021b). In Task A, the model takes the factual paragraphs of a case as input and predicts the set of violated ECHR articles. Task B focuses on the same aspect, where the input remains the list of factual paragraphs, but the model predicts the set of allegedly violated ECHR articles. The dataset setting used in our paper is similar (Chalkidis et al., 2021c). Dataset statistics are provided Table 11.
H.2.1
Multi-class Classification
SCOTUS. The US Supreme Court (SCOTUS) serves as the highest federal court in the United States of America, primarily handling highly con- tentious or intricately complex cases that have not been adequately resolved by lower courts. We uti- lized the SCDB (Supreme Court Database) (Spacth et al., 2013), in a setting similar to (Chalkidis et al.. 2021c), to classify court opinions across 14 distinct issue areas. These issue areas encompass range of subjects, such Criminal Procedure, Civil Rights, Economic Activity and more. Our classification task is a single-label multi-class classification. The 14 issue areas effectively group together 278 spe- cific issues, all centered around the subject matter of the disputes being presented before the court. Dataset statistics are provided in Table 11.
EURLEX. The EUR-Lex portal is the platform for publishing legislation about the European Union (EU). These laws extensively annotated by the EU's Publications Office, incorporating multiple concepts sourced from Euro EuroVoc is a mul- tilingual thesaurus actively maintained by the Pub- lications Office, comprising over 7,000 concepts that cover a wide range of activities undertaken by the EU and its Member States, such as economics, healthcare, and trade. For our research, we uti- lize the English portion of the dataset provided by (Chalkidis et al., 2021a). This dataset comprises 65, 65,000 EU laws (documents) sourced from EUR- Lex, allowing us explore and analyze legislative content within the EU context. Given a document, the task is to predict its EuroVoe labels (concepts). The dataset setting used in our paper is similar to (Chalkidis et al., 2021c). Dataset statistics are pro- vided in Table 11.
LEDGAR. (Tuggener et al., 2020) introduced dataset called LEDGAR (Labeled EDGAR) specif- ically designed for contract provision classification at the paragraph level. The contract provisions within this dataset are sourced from contracts ob- tained from the US Securities and Exchange Com- mission (SEC) filings, which are publicly acces- sible through the EDGAR (Electronic Data Gath- ering, Analysis, and Retrieval system) platform. The dataset setting used in our paper is similar to (Chalkidis et al.. 2021c). Dataset statisties are pro- vided in Table 11. ILDC. ILDC (Malik et al., 2021), a substantial corpus comprising 35,000 Indian Supreme Court cases, stands out as it includes annotations of origi- nal court decisions. Within this corpus, specific portion has been annotated by legal experts, pro- viding gold-standard explanations. Building upon ILDC, we introduce the Court Judgment Predic- tion and Explanation (CJPE) task. The model is tasked with predicting and providing comprehensi-
UNFAIR-ToS. The dataset known as UNFAIR- ToS, developed by (Lippi et al., 2019), encom- passes 50 Terms of Service (ToS) documents ex- tracted from various online platforms such as YouTube, eBay, Facebook, and others. These ToS

---

Newember 3. 3330 Uiendd Sunce Reieel
mmnronaatio deliys reiddng work iappagic
rinn tur tmpose ciriill held men sedired Sharee prverity
SABS
ddaays intallia seot orques
repaeall
aill clien.
tede napareml speined Lundry
Iakgaide cxgleniier and prelatee coam of Bnt Sugra hericss
piuaclyy
yousaly happed eocueiin coeadrratio of nat of Saioio prose dutigrenee and arvreatty
biked Parro Rico
Plannc Sarmehood Indeation America
enoke tren sendd headd
Beep
esbitaily carers peblic
roccie daty called
Jiy Labe Car Marressne
obstacte the kaal precus
otee kutatto entreverrs
neah lequoel sog
Akgead iepereetttten cunge responant rodow
U.S Baad Velvaaal Sprwh Devisioss
Lppraccaaa
digh devenval
rmppial fillowig
Centulle Accouet Revevadee Delie
Rave Reage Leniices
World Distomaze Arauticn brgiis
Desarzaa of Veenar Atai
ralence that the clase lokde
SIERES Dig
lmpar Nalkng conract craata
weilnng dessis oppreiiines
repuline
U.S. Suprrse Count Oval Aeginent Tour- srips
Rociaae fefurooll Cooiqe Ongeisates
polkiking
fooe
infan toralen
Blua Riadi The Thrm
Toker Lake Roda Wos Siniag
pessssae hene the Eiquas Riguna pelpeware
FIMESOLLD Dunc. W. Plaqooki
pond autoccpaci prytccta coneena
lnw intonid tos spalicen sccraning pdlso ineds the cremstsoo wortted
bdge Cantrats
Kinclla Antiniat Innoue: hone Bocee Sothar
indk pako das rapessers
Kikep Krona Compasy
filken Caven
Aemenee
Tamahe
openssddd
Pacinner On Peinnes Pactunci
inexaaie lunnving vairesay
nntrd in cve bivtgg jeidecin ropuciing whieee openns patio DASERNN
Codan
Renca Aboas lee.
cooodia scroen
LOCILL
Table 16: Comparison of correlated spans extracted from Vanilla PMI and discounting factor applied at 75th and 95 percentile (pc). As we clearly see, the spans extracted improve gradually with increasing pc. A higher allows us to extract reusable fragments from entity-ries mntity-rich legal documents.
documents have undergone sentence-level annota- tion to identify eight distinct categories of unfair contractual terms. These categories represent sen- tences within the ToS that potentially infringe upon user rights, as per the guidelines outlined in EU consumer law. The model takes a sentence as in- put and generates the set of unfair categories, applicable, associated with that particular sentence. The aim is to detect and classify instances of un- fair contractual terms present in online platform ToS documents. The dataset setting used in our pa- per is similar to (Chalkidis et al., 2021c). Dataset statisties are provided in Table 11.
namely the judgment and the preamble. The preamble of a judgment contains formatted meta- data like names of parties, judges, lawyers, date, court, etc. The text following the preamble till the end of the judgment is called "judgment." " The dataset is annotated with 14 named entities, namely, COURT, PETITIONER, RESPONDENT. JUDGE, LAWYER, DATE, ORG, TYPE, GPE, STATUTE, PROVISION, PRECEDENT, CASE-NUMBER, WITNESS and OTHER-PERSON Dataset statis- ties are provided in Table 11. All results in the main paper are averaged for judgment and preamble.
H.2.4 Other Tasks
OTS-CT. Online Terms of Service (OTS) (Drawzeski et al., 2021) attempt to automatically detect unfair clauses in Terms of Service The input to the model is a sentence, and the model identifies the sentence for various clause topics. The dataset setting used in our paper is similar to (Niklaus et al., 2023). Dataset statistics are provided in Table 11.
ContractNL.I The ContractNL. dataset (Koreeda and Manning, 2021) has been developed specifi- cally for document-level natural language inference (NLI) tasks focused on contracts. This dataset aims to automate and facilitate the labor-intensine pro- cess of contract review, In this task, a system is pro- vided with a set of hypotheses, such "Some obli- gations of Agreement may survive termination," " along with a contract. The system's role is to clas- sify whether the contract entails each hypothesis. contradicts the contract, or is not mentioned in the contract (neutral). Additionally, the system is expected to identify the specific evidence that sup- ports its decision in the form of spans within the contract. Dataset statistics are provided in Table 11.
H.2.3 Named Entity Recognition
EDGAR. EDGAR (Au et al., 2022) is based on legal company filings available from the US Securi- ties and Exchange Commission's EDGAR data set, EDGAR is annotated with 7 named entity classes, namely Location, Person, Business, Government, Court, Legislation/Act, and Miscellaneous. Dataset statistics are provided in Table 11.
Indian-Leeal-NEER. Indian-Legal-NERR (Kalamkar et al., 2022) is derived from Indian Court Judg- ments and consists of two separate sub-datasets,
BUILD. BUILD (Malik et al., 2022) is a dataset built for Rhetorical Role (RR) Prediction - given a

---

document, the task to predict the text segments corresponding to various roles. The task can be seen as sequential text classification task (Qian et al., 2020). The dataset labeled with 13 fine- grained RRs: Fact, Argument, Statute, Dissent, Precedent, Ruling By Lower Court, Ratio Of The Decision, Ruling By Present Court, and None.
DLI. For DLI, we take LD, as the gold annotated hypothesis of the document.
1.2 Other Details
Model Parameters: legal-longformeenang has 409M parameters with 24-layers of encoder, 1027- hidden-state, 4096 feed-forward hidden-state and 16-heads. BART large has 680M parameters with 12 layers of encoder, 12 layers of decoder, 1024- hidden-state, and 16-heads.
CaseHOLD. The CaseHOLD (Case Holdings Legal Decisions) dataset (Zheng al., 2021) con- tains multiple choice questions about holdings of US court cases from the Harvard Law Library case law corpus. Holdings are short summaries of legal rulings accompanying referenced decisions rele- vant to the present case. The input consists of an excerpt (or prompt) from court decision that refer- ences particular case, where the holding statement (in boldface) is masked. The model must identify the correct (masked) holding statement from five choices.
Compute Infrastructure: All our experiments conducted on single NVIDIA A100 GPU. An entire DALE fine-tuning pipeline takes 40 minutes. We pre-trained DALE for days on 4 NVIDIA A100 GPUs.
Implementation Software and Packages: We implement all our models in PyTorch and use the HuggingFace 'mplementations of BART large and legal-longformeerurg For multi-class classi- fication and multi-label classification, we use the HuggingFace Trainer implementations of the corre- sponding tasks. For NER, we use the FLAIR toolkit (Akbik et al., 2019) to fine-tune all our NER mod- cls. For CascHOLD MCQ, we follow the setup pro- posed by (Zheng al., 2021) For ContractNLI DLI, we follow the setup proposed by (Koreeda and Manning, 2021).. For BUILD RR, we follow the setup proposed by (Malik et al., 2022)* For CaseHold, ContractNLI and BUILD we replace the original encoder with logggllooonooonnrgg
Indian- and UK-A bstracttee datasets. Indian- Abstractive and UK-Abstractive datasets (Shukla et al., 2022), are datasets built for abstractive sum- marization, were collected from Indian Supreme Court judgments from the website of the Legal In- formation Institute of India and The UK Supreme Court website $ respectively. The dataset setting used in our paper similar (Shukla et al., 2022). Dataset statistics are provided in Table 11.
I A Additional Details
Potential Risks: Conditional Language Models used for Natural Language Generation often tend to hallucinate (Ji et al., 2022) and potentially generate nonsensical, unfaithful or harmful sentences to the provided source input that it is conditioned on.
L1 LD, for fine-tuning
Classification. For multi-class classification, take LD, as the gold annotated label of the docu- ment. For multi-label classification, we concate- nate the label strings for the gold annotated labels of the document.
NER. For NER, we take LD, as the template "Entity- is a label-/ [SEP] ... [SEP] Entit--n is a label-n where Entity-i corresponds to the 7t named entity in the sentence and label-i corre- sponds to the gold annotated label of the named entity.
MCQ. For MCQ, we take LD, as the actual godl annotated answer of the question.
thopss//pyoorch.orr// col
RR. For rhetorical role prediction, we take L.D, as the rhetorical role of the sentence in the document (we generate augmentations sentence- centence-wise).
2 hhtps://huginngfacc.c
hhtpss//hugginggfee
"htps://githubbco
"htp://www.lioffndi..oom cuses/cen/INSC// 'htps://www. sapremecourtt.k//cciiiieoo
"ntps://gthhu..com/l -ogaloeallo//ellii--- P......sttpthhetorrccllooooo
hascline

---

Document 1
The case tried before a jury, which returned verdict in favor of the plaintiff. This action was based on claims of fraud and breach of contract under credit life insurance policy. In the second count, she claims breach of contract because Magic City Dodge and Peninsular Life Insurance Company refused to pay the benefits of the credit life policy.... "* Dennis died with balance remaining on his obligation.
Document 2
The complaint alleges that the realtor, Barnett, was elected and duly qualified as marshal of the city of Nobles. ...
mandate restore relator office as marshal The question of the power of the common council to remove the relator is properly presented by his mandate.
Prayer for
application for Errors are assigned upon these decisions. ...
Document
The existence of a labor dispute is not questioned here and claim is made for compensation during time dispute was in progress.
Appellant filed claim for unemployment compensation.
June 30th, without reporting back to the company. claimant registered with the Alabuma Unemploy-
ment Service and made upplication for unemployment compensation. Pier claim was allowed by the claims examiner of Department of Industrial Relations.
Document
In this case. the former wife presented undisputed evidence the financial and other cireumstances of the parties existing at the time of the entry of the 1990 judgment. Mary Jo Blount the former wife filed petition in the Madison Circuit Court the trial court, seeking modífication of the provisions of a 1990 divorce that incorporated a settlement agreement in which the former husband agreed to pay the former wife 500 per month in periodic alimony. The former wife timely appealed to this court and argues that the trial court findings are clearly erroneous.
In 2012, she reported 28,951 in gross income.
Document 5
validity of a statuté can never depend upon the antecedent consultation of the people by the legislature, nor upon the affording them opportunity express their sentiments through petitions. The removal of the court- house of county and its permanent location, is indisputably a permissible exercise of legislative authority.
If the question of the power of the legislature to make the removal of the court house to Selma dependent upon the condition argument. ...
approval by popular vote, were res integra, there would be room much
Mr. Justice Byrd not sitting the case.
Table 17:
Examples of legal documents from our pre-training corpus with correlated spans. Spans highlighted in green co-occur within the same document, while spans in yellow co-occur across documents.
Table 18: Comparison of augmentations generated by DALE and all other baselines. augmentations were generated in low-resource setting (500). Each augmentation was marked by a law student on parameters: (1) If the augmentation coherent, (2) If it adds new plausible context, and (3) if it is label-consistent and matches the underlying data distribution. We present the results of the study as or X next to each augmentation in the same order as above.
BUILD-RR

---

Original
In the decision reported in 2003(7) SCC 141, supra, the Apex Court has held that "It is true that the incident in question has prematurely terminated the life of twenty-one people and the number of deaths cannot be the sole criterion for awarding the maximum punishment of death." " In the decision reported in (2011)10 SCC 389, supra, Apex Court has held that the appellant/Accused would not be a menace to society and no reason to believe that the appellant cannot be reformed or rehabilitated or would constitute a continued threat to society and it is not the 'rarest of the rare case' causing for extreme penalty of death.
EDA
(1) In the decision reported in 2003(7) SCC 141, supra, the Apex Court has held that "It is true that the incident in question has prematurely terminated the life of twenty-one people and the number of deaths cannot be the sole criterion for awarding the maximum punishment of death. reformed the decision continued in would supra apex court has held that the appellant causing see not be menace to society and no reason to believe that the appellant cannot be an rehabilitated or constitute would a reported threat society and it is not the rarest of the rare case accused for extreme penalty of death X X
(2) in the decision reported in scc supra the apex has held that it is true that incident question prematurely terminated the life of twenty one people and the of deaths cannot be the sole criterion for awarding the maximum punishment of deathin the decision reported in see supra apex court has held that the appellant accused would be menace society and no reason to believe that the appellant cannot reformed or rehabilitated or constitute a to society and is not the of the rarecase causing for extreme penalty of death X x
SSMBA
(1) under the decision reported 2003 ( scra 1411 supra the apex court has noted stated ft is true that the incident question actually prematurely terminated from only about twenty five people and the number of deaths cannot be the correct cause for awarding this maximum punishment of death. in the cases reported in (2011 J at sec 383, supra, apex court has decided that the appellant accused would not be risk of society and no reason to believe that the appellant cannot have reformed or utilified or would constitute real threat to society and it is not the rarest of the rare cases' causing for death penalty of death. / x
(2) in his decision reported in 2003 ( ses 1419 supra, the trial court has held that "iit is un- derstood that the incident without question whichly terminated the life of twenty/ -one people. the number of deaths cannot be the sole criterion for awarding a maximum punishment on death. "in his decision reported in ((201 1 10 see 389, 2 43a, j he has held that the appellant/ accused would not present a menace within society and no reason to believe that the appellant would be reformed or mobilitated or would constitute the non danger to society and it is not the "rrrest of the rare casećausing any further lack of victims. /

---

AEDA
(1) In the decision reported in 2003(7) SCC 141, supra, the Apex Court has held that "It is true that the incident in å question has prematurely terminated the life of twenty-one people and the number of deaths cannot | be the sole criterion for awarding the maximum punishment of death. In the decision reported in (201 1)10 SCC 389, supra, Apex Court has held that the appellant/ reason to believe that the appellant cannot be reformed or rehabilitated ? or would constitute a continued threat to society and it is not the 'rarest ? of the ? rare case" causing for extreme penalty of death. X x
would not be a menace to society and | no
(2) In the 3 decision reported in 2003(7) SCC 141, supra, the Apex Court has held. that "It is true that the incident in question has prematurely terminated the life of twenty-one people and the number of deaths cannot be the sole criterion for awarding the maximum punishment of death. In the decision reported in (2011)10 SCC 389, supra, Apex ? Court has held that the appellant/ Accusee would not be a menace society and reason | believe that the appellant cannot & be reformed or rehabilitated. | or would constitute a continued threat to society and ! is not the 'rarest of the rare case' causing for ? extreme penalty of death. X
BackTrans(1 In the decision SCC 141 published in 2003(7), the Apex Court stated: "It is true that the
incident in question has prematurely ended the lives of 21 people and that the number of deaths cannot, In the decision cited above (2011)10 SCC 389, the Apex Court ruled that the complainant/applicant presents no danger to the company and that there is no reason to believe that the / X
(2) In the decision SCC 141 notified in 2003(7), the Apex Court found that "it is true that the incident in question has prematurely ended the lives of 21 people and that the number of deaths is not. In the decision SCC 389 published above in (2011)10, the Apex Court found that the complainant/applican did not present any danger to the company and / X
GENIUS (1) The decision reported in the Apex Court has indicated that the government in question has prematurely terminated the right to cause death and that the cause of deaths can be ascertained. The court also said that there is no need for the government to intervene in the matter. I believe that the appcllant/Accused would not be able to have fair trial. I think that the appellants can be either injured, or rehabilitated would have to be tried in very rare case. X
(2) The decision reported in the Apex Court has said that the court in question has prematurely terminated the right of deaths can be done. The court has also ordered that the government should be allowed to use the power of the courts to ensure that the citizens of the state are not deprived of the right to life. I believe that the appellant/Accused would not be able to do so, and believe it would be difficult for him to do it. I also believe, however, that the appellants
can be tried or rehabilitated or would be in a very rare case. X X X

---

ChatGPT (1) The Supreme Court, in its 2003(7) SCC 141 ruling, acknowledged that the incident in question tragically claimed the lives of twenty-one individuals. However, the court phasized that the number of deaths alone should not be the sole basis for imposing the maximum penalty of death. In the 201 (10) SCC 389 decision, the Apex Court held that the appellant/aceused did not pose a threat to society and there was no reason to believe that they could not be reformed or rehabilitated. Consequently, the court deemed it not to be a 'rarest of the rare case" warranting the extreme punishment of death. / X
(2) According to the 2003(7) SCC 141 ruling, the Supreme Court acknowledged the un fortunate event that resulted in the premature loss of twenty-one lives. However, the court emphasized that the number of deaths alone should not be the sole criterion for imposing the maximum punishment of death. As per the (2011)10 SCC 389 case, the Apex Court determined that the appellant/accuset posed no threat to society and showed no indications of being beyond reform rehabilitation. The court concluded that this particular case did not meet the criteria of being an extremely rare instance warranting the death penalty. X
Falcon
(1) In the decision reported in 2003(7) SCC 141, supra, the Apex Court has held that "It is true that the incident in question has prematurely terminated the life of twenty-one people and the number of deaths cannot be the sole criterion for awarding the maximum punishment of death." " In the decision reported in (201 I)10 SCC 389, supra, Apex Court has held that the appellant/Accused would not be a menace to society and no reason to believe that the appellant cannot be reformed or rehabilitated or would constitute a continued threat to society and it is not the 'rarest of the rare case' causing for extreme penalty of death. / X
(2) In the decision reported in 2003(7) SCC 141, supra, the Apex Court has held that "It is true that the incident in question has prematurely terminated the life of twenty-one people and the number of deaths cannot be the sole criterion for awarding the maximum punishment of death." " In the decision reported in (2011)10 SCC 389, supra, Apex Court has held that the appellant/Accused would not be menace to society and reason to believe that the appellant cannot be reformed rehabilitated or would constitute continued threat to society and it is not the 'rarest of the rare case' causing for extreme penalty of death. /
DALE-
(1) The decision reported in 6 N. J. Apex Court has determined that the insurance policy in question has prematurely terminated the period within which proof of deaths can be made The Court finds that the appellant/ Accused would not be entitled to relief on habeas corpus because the evidence does not show that the the appellant can not be adequately supervised or rehabilitated or would not suffer irreparable harm except in a rare case. X
pt
(2) The decision reported in the Apex Court has been modified reflect that the statute question has prematurely terminated the period during which claims for benefits on account of deaths can be made. The Court finds that the appellant/Accuser would not be entitled to relief unless it is shown that there is reasonable likelihood that the the appellant can be cured or rehabilitated or would suffer irreparable harm except in rare case. / x

---

DALE- (I) In the decision reported in (2015)4 SCC 709, supra, the Apex Court has held that the death of the woman in question has prematurely terminated the life of the deceased and therefore the number of deaths cannot be taken into consideration as a basis for awarding maximum punishment.
ft
submitted that the appellant/Accused would not be a menace to society and
therefore, it is to be held (1) [1960] S.C.R. 287.n(2) [1952] K.L.J. 438, that the appellant cannot be reformed or rehabilitated or would attract the punishment of death or imprisonment for life in the rare case of his release on bail. / x
(2) In the decision reported in 2003(2) SCC (Cri.) 941, supra, the Apex Court has held that when the act in question has prematurely terminated the life of a person, the number of deaths cannot be taken as a basis for awarding death sentence. Therefore, it cannot be said that the appellant/Accused would not be a menace to society and society as whole, and am of the considered view that there is no reason to believe that the convict cannot be reformed or rehabilitated would be threat society in the rare case. /
DALE
(1) In the decision reported in 2003(2) SCC 16, the Apex Court has held as follows: : "It is true that the incident in question has prematurely terminated the life of Sangita and the number of deaths cannot be the sole criterion for awarding death sentence. In the light of the above discussion, am of the considered view that the appellant/Accusse would not be menace to society as a whole and there no reason to believe that the benefit of doubt remains to be given to the appellant; and, further, do not find that the case falls under the category of rarest of the rare cases for imposition of death sentence.
(2) In the decision reported in 2012(3) SCC (Cri.) 908, the Apex Court has held that where the incident in question has prematurely terminated the life of a particular victim and the number of deaths cannot be the sole criterion for awarding the maximum punishment of death. It is further contended that the appellant/Accused would not be a menace to society and society cannot be considered in the light of the report of the Institute of Forensie Laboratory which has been held by the Hon' ble Supreme Court in the case of State of Himachal Pradesh Raghubir Singh (1999) 6 SCC 695, that the sentence of imprisonment for life cannot be altered or rehabilitated or would constitute a more than 'rarest of the rare case' for release on probation of good conduct. /
whole and that the chances of reformation and rehabilitation of the appellant
ContractNLI
Original
Confidential Information' shall mean: in respect of Information provided in documentary or by way of model or in other tangible form, Information which at the time of provision is marked or otherwise designated to show expressly or by necessary implication that it is imparted in confidence; and in respect of Information that is imparted orally, any information that the Disclosing Party or its representatives informed the Receiving Party at the time of disclosure was imparted in confidence; and in respect of Confidential Information imparted orally, any note or record of the disclosure and any evaluation materials prepared by the Receiving Party that incorporate any Confidential Information; and any copy of any of the foregoing; and the fact that discussions are taking place between the Receiving Party and the Disclosing Party. 'Disclosing Party' shall mean the party to this Agreement that discloses Information, directly indirectly to the Receiving Party under or in anticipation of this Agreement.

---

EDA
(1) confidential information shall mean imparted respect of information provided in confi- dence by way designated model in other tangible form information which any the the of provision is marked or the of information expressly or by necessary implication that it is imparted in confidential and in respect of information that is imparted orally any information that time disclosing party or its representatives informed the receiving party at the time of was imparted disclosure confidence and in respect of note information the orally any party or the of the disclosure and any evaluation materials prepared by the receiving confidential that incorporate any documentary show and at copy of any of record foregoing and in fact that discussions are taking place between the receiving party and otherwise disclosing party discloses party shall mean the party to this agreement that disclosing information agreement or indirectly to the receiving party under in anticipation of this directly. X X
(2) confidential information shall bastardly in respect of information offer in objective by way of good example or in other tangible cast information which at the time of supply is marked or otherwise delegate to testify expressly or by necessary implication that is imparted in confidence and in respect of information that is imparted orally any information that the disclosing party or its representatives informed the receiving party at the time of disclosure was imparted in confidence and in respect of confidential information imparted orally any note or commemorate of the disclosure and any evaluation fabric prepared by the receiving party that contain any confidential information and any simulate of any of the foregoing and the fact that discussions are take on place between the receiving party and the disclosing party disclosing party shall mean the party to this agreement that discloses information directly or indirectly to the receiving party under or in anticipation of this agreement X
SSMBA
(1) of information shall mean:: : in respect of information clearly in documentary by means of other model or in other tangible form, information which at the time of provision is marked or otherwise designated will show expressly any the necessary implication that it is imparted in confidence]: and in respect of information that was impiculated verbally. any declaration that the disclosing party between its representatives and any receiving party at aation of disclosure was imparted in confidencel: and in respect of confidential information imparted orally, any note or record of the disclosure and any such materials prepared by its receiving country that incorporate any confidential information ; or any copy and any record such foregoing ; and the same or, are that in between either receiving party of the disclosing party "the disclosing party"" shall mean either party to this agreement that discloses information, directly or indirectly through the receiving party under or in anticipation of this agreement...
(2) Confidential information' shall mean: in respect of information clearly in written or by means of a model or in other tangible form, information which the time of provision is marked or otherwise designated will show expressly any the necessary implication that it is imparted in confidence; and in respect of information that was orally imparted, any procedures that the disclosing party between its representatives and any receiving party at time of disclosure was imparted in confidence; and in respect of confidential information imparted orally, any note or record of the disclosure and any such provisions prepared by its receiving country that incorporate any confidential information; or any copy and any record such foregoing; and the same or, are that in between either receiving party of the disclosing party "the disclosing party' shall mean either party to this agreement that discloses information, directly or indirectly through the receiving party under or in anticipation of this agreement' / x

---

AEDA
Connidential Information' shall mean; in respect of Information provided in documentary or by way of model in other ttangible form, Information which at the time of provision is marked or otherwise designated show expressly or by necessary impplcatiinnn that it is imparted in confidence; and in respect of Information that is imparted orally, any information that the Disclosing Party or its representatives informed the Receiving Party at the time of disclosure was imparted in confidence; and in respect of Confidential Information imparted orally, any note or record of the disclosure and any evaluation materials prepared by the Receiving Party that incorporate any Confidential Information; and any copy of any of the foregoing; and the fact, that discussions are taking place between the Receiving Party and the Disclosing Party. 'Disclosing Party' shall mean the party this Agreement that discloses Information, directly indirectly to the Receiving Party under or in anticipation of this Agreement. X
Conffdenntil Information' shall mean; in respect of Information provided in documentary or by way of a model or in other tangible form, Information which at the time of provision is marked or otherwise designated to show expressly or by necessary implication that ; it is imparted in confidence; and in respect of Information that imparted ? Porally, any information that the Disclosing Party or its representatives informed the Receiving Party at the time of disclosure was imparted in confidence; and in respect of Confidential Information imparted orally, any note or record of the disclosure and any evaluation materials prepared by the Receiving Party that incorporate any Confidential Information; and any copy of any of the foregoing; and the fact that discussions taking place between the Receiving Party and the Disclosing Party.. 'Disclosing Party' shall mean the party to this Agreement that discloses Information, directly or indirectly. to the Receiving Party under in anticipation of this Agreement.
BackTrans(1) "confidential information' means information made available the time of provision, express or through the necessary implication that it confidential, or the form of a model or other tangible form; information marked or otherwise intended at the time provision, express or" The contracting party to this Agreement shall be the party to this Agreement which, under or anticipation of this" /
(2) "confidential information' means information that, at the time of making it available. is expressly or through the necessary conclusion that it is confidential or information that is identified at the time of making it available or is otherwise intended to be displayed in order to express or by the necessary conclusion'Disee Disclosiig Party' means the Party to this Agreement which discloses information directly or indirectly to the receiving Party, under or in anticipation of this Agreement. / X

---

GENIUS
(1) Confidential Information. The disclosure of Information provided to you by the Infor- mation which is imparted in confidence you. 2. The Information that is exparted orally.
3. The information that the disclosure was imparted you in confidence. 4. The respect of Confidential Information imparted orally to you and to the disclosure and any other person,
5. The confidentiality of the information. 6. The content the content. 7. The manner
in which the information is communicated to you or to any other persons. 8. The way in which you incorporate any Confidential information. The Disclosing Party is an Agreement that discloses Information to the Receiving Party under the terms of the Agreement. The Agreement may be signed by the parties to this Agreement. / X X
(2) Confidential Information i.c. the whole of Information provided. ii. Information which is imparted in confidence to the person and/or the person or the person in the possession of the information that is being disclosed. iii. information that the person, the persons persons in the persons responsible for the preparation of the disclosure. iv. information which the subject of disclosure. c. Information that the disclosure was imparted the individual or the individual in the respect of Confidentialnnformation imparted orally. d. information concerning the disclosure and any other information which may be included in the form of the Information. e. information about the persons, entities or persons who may incorporate any Confidential information into the information contained therein. The Disclosing Party is an Agreement that discloses Information about the Receiving Party under the terms of the Agreement. This Agreement is not binding on the parties. / x x
ChatGPT (1) The term 'Confidential Information' includes two types. Firstly, it refers to Information provided in written or tangible form, explicitly marked or designated confidential at the time of provision. Secondly, it encompasses orally communicated information that the Dis- closing Party or its representatives confirm as confidential during disclosure. It also includes any notes, records, or evaluation materials created by the Receiving Party that incorporate the orally shared Confidential Information. Moreover, it covers copies of any of the aforemen- tioned forms and acknowledges ongoing discussions between the Receiving Party and the Disclosing Party. The term 'Disclosing Party' in this Agreement denotes the party disclosing Information to the Receiving Party directly or indirectly. /
(2) this Agreement, the term 'Confidential Information' refers to two aspects. Firstly, it in cludes Information provided written or tangible form, marked or designated as confidential upon provision. Secondly, it encompasses orally imparted information explicitly identified as confidential by the Disclosing Party or its representatives during disclosure. It also covers any notes, records, or evaluation materials created by the Receiving Party that incorporate the orally shared Confidential Information. Additionally, it comprises copies of any of the aforementioned forms and acknowledges the existence of discussions between the Receiving Party and the Disclosing Party. The term 'Disclosing Party' signifies the party disclosing Information to the Receiving Party, whether directly or indirectly, under this Agreement.

---

Falcon
(1) Confidential Information' shall mean: in respect of Information provided in documentary or by way of model in other tangible form, Information which at the time of provision is marked or otherwise designated to show expressly or by necessary implication that is imparted in confidence; and in respect Information that is imparted orally, any information that the Disclosing Party or its representatives informed the Receiving Party at the time of disclosure was imparted in confidence; and in respect of Confidential Information imparted orally, any note or record of the disclosure and any evaluation materials prepared by the Receiving Party that incorporate any Confidential Information; and any copy of any of the foregoing; and the fact that discussions are taking place between the Receiving Party and the Disclosing Party. 'Disclosing Party' shall mean the party to this Agreement that discloses Information, directly indirectly the Receiving Party under or in anticipation of this Agreement.
(2) Confidential Information' shall mean: in respect of Information provided in documentary or by way of model or in other tangible form, Information which at the time of provision marked or otherwise designated to show expressly or by necessary implication that is imparted in confidence; and in respect Information that is imparted orally, any information that the Disclosing Party or its representatives informed the Receiving Party at the time of disclosure was imparted in confidence; and in respect of Confidential Information imparted orally, any note or record of the disclosure and any evaluation materials prepared by the Receiving Party that incorporate any Confidential Information; and any copy of any of the foregoing; and the fact that discussions taking place between the Receiving Party and the Disclosing Party. 'Disclosing Party' shall mean the party to this Agreement that discloses Information, directly or indirectly the Receiving Party under or in anticipation of this Agreement. /
DALE-
(1)Confidential Information means any portion of Information provided in writing other than Information which is imparted in confidence; Confidential Information consists Information that is imparts orally or in writing or information that the other party agrees in writing prior to disclosure was imparts in confidence. In respect of Conf confidential Information imparted orally, the disclosure and any representation or warranty shall not be deemed to incorporate any Knowledge Disclosing Party means any Person as defined in the Agreement that discloses Information to the Receiving Party under the / X
pt
(2) Confidential Information means any Confidential Information with respect of Information provided in writing or Information which is imparted in confidence with respect to a written communication or of Information that is imparts orally or written information that the person making the disclosure knew should have known prior to the disclosure confidence Confidentiality does not include any information that was known to the party making disclosure or reasonably believed to be known such party when making disclosure and any written communication does not expressly or impliedly incorporate any Confictional Information by reference Exhibit 10.1 Disclosing Party Non Disclosure Agreement that
imparts in
discloses Information to the Receiving Party under Section 7.1 / X

---

DALE-
(1) "Confidential Information' means any and all of Information provided by the Disclosing Party to the Receiving Party, including, without limitation, Information which is marked "prorrittay." 'confidential' or other similar legend, or if applicable. information which is imparted confidence, including but not limited to, any and ALL notes, analyses, compila- tions, studies, interpretations or other documents prepared by or on behalf of Information that is Imarted orally or visually, or any other information that the Recipient reasonably should have known at the time of disclosure was imparted IN confidence, whether or not it is reduced to writing or other tangible form, and that is marked or otherwise designated as pprior to the disclosure and any copies made thereof that incorporate any Confidential Information." DDisclosing Party] means party to this Agreement that discloses Information to the Receiving Party under this Agreement /
ft
(2) "Confidential Information' shall mcan any and all of Information provided by the Dis- closing Party to the Receiving Party, including, without limitation, Information which is marked prrprrctary"" or 'confidential', which not generally known to the public, whether or not it is imparted in confidence, and any compilation of Information that is impersed orally, visually, electronically or otherwise, including without limitation information that the Recipient can demonstrate by documentary evidence immediately after the time of disclosure that the information received was imparted IN confidence, or that, with respect of Confidential Information imparted orally, was already in the possession of the Recipi-ent on a non-confi- dential basis prior to the disclosure and any compilations, studies, interpretations or other documents prepared by the Reconsequent Party that contain or incorporate any Confidential information. 'Disclosing Party' means party to this Agreement that discloses Information to the Receiving Party under this Agreement."
DALE
(1) |Confidential Information' means, in respect of Information provided by either Party, Information which the time of disclosure is marked or otherwise designated in writing as imparted in confidence' or prrprrctary" (and, in the case of Information that is imparted orally, information that the Disclosing Party or its Representatives should reasonably have understood from the circumstances surrounding the disclosure that the information sought to be obtained was being provided in confidence and that the disclosure was imparted in confidence); provided, however, that the obligations of the Receiving Party with respect of Confidential Information imparted oral shall survive the termination of the disclosure and any discussions between the Parties, notwithstanding the fact that the Parties may incorporate any information in written form that is subsequently incorporated in any compilations, analyses, studies or other documents prepared by or for the purposes of the Parties." Disclosing Party" means Party to this Agreement that discloses Information to the Receiving Party under or connection with this Agreement. vf
(2) Confidential Information' shall mean that portion of Information provided by the Disclos- ing Party (" ('Information") to the Receiving Party and its Representatives (as defined below) in written, graphic, electronic or other tangible form and any copies made thereof or extracts therefrom, except Information which at the time of disclosure was imparted in confidence; provided, however, that in respect of Information that is imparted orally, the information that the Recipient or its Representatives were informed by the Owner or its Representative that the information
imparted in confidence shall be protected hereunder in accordance with this Section 1.1.1 in the same manner in which it was protected prior the disclosure and any software that incorporates any Confidential Information. ceriattionn'DDiiiiie DDiselosing Party' means Party to this Agreement that discloses Information to the Receiving Party under or in connection with this Agreement. /
UNFAIR ToS

---

Original
The most recent version of this agreement will be posted the services under settings and also gotinder.com, and you should regularly check for the most recent version.
EDA
recent version of this agreement will be posted on the services under settings and also on gotinder com and you should regularly check for the most recent version X X
AEDA
the most recent version of this agreement will be posted on the services under settings and also on gotinder.com. and you should regularly check for the most recent version x /
SMERTI
This most recent version of Windows will be posted on power under settings available on gotinder. and you should regularly check our most recent version. X X
GENIUS
The terms of this agreement will be contingent on the services they provide. For more information, please visit www.sos.gov. / X
ChatGPT
The latest edition this agreement will be made available on the services, specifically under the settings section and on gotinder.com. It is advisable to frequently review the most recent version. / X
Falcon
The most recent version of this agreement will be posted on the services under settings and also on gotinder.com, and you should regularly check for the most recent version. x /
DALE-
The most recent version of this agreement shall be accepted as the most recent amendment x
pt
DALE-
the most recent version of this agreement will posted on the services under settings and also on gotinder.com, and you should regularly check for the most most recent versions.
ft
DALE
The most recent version of this agreement will be posted on the services's website where you can download and view the services, and you should be aware that this is not a guarantee that the services will be up to code or up to date, and we reserve the right to discontinue using the services at any time.
CASEHOLD
Original
Juan Lopez appeals from his conviction and sentence for harboring aliens in violation of 8 U.S.C. 1324())((A((ii))) and conspiracy in violation of 8 U.S.C. $ 1324())((AA)))))) We have jurisdiction pursuant 28 U.S.C. $ 1291 and 18 U.S.C. $ 3742, and affirm. Lopez argues that the district court erred when it redacted his indictment to remove reference to commercial advantage and private financial gain. Even assuming this argument was not waived, Lopez has failed to demonstrate plain error on the part of the district court. Clearly, the court's ' redaction did not violate the Fifth Amendment because it did not broaden the charges against Lopez: if anything, it narrowed them.
AEDA
Juan Lopez appeals from his conviction and sentence for harboring aliens violation of 8 U.S.C. ())) and conspiracy in violation of 8 U.S.C. $ (324(a)))))) We have jurisdiction pursuant to 28 U.S.C. $ 1291 and 18 U.S.C. $ 3742, and we affirm. Lopez argues that the ? district court erred when it redacted his indictment to remove reference to commercial advantage and private financial gain. Even assuming this argument was not waived, Lopez has failed to demonstrate plain error the part of the district court. Clearly, the court's redaction did not violate the Fifth Amendment because did not broaden the charges against Lopez; if anything, narrowed them. X

---

GENIUS
Juan Lopez appeals from conspiracy in violation of U.S. jurisdiction pursuant to the First Amendment. Lopez argues that the district court erred when it redacted his indictment allow the Fifth Amendment to be used in the charges against Lopez, and that the indictment did not include enough evidence to prove that he had committed a crime. In his appeal, Lopez also argues that he was not charged with a crime, but rather was charged with conspiracy to commit a crime against the United States. / /
X
ChatGPT Juan Lopez is appealing his conviction and sentencing for the offense of harboring aliens in violation of U.S.C. 1324()))(A)Gii) and conspiracy violation of 8 U.S.C. $ (3)4)))(())()))))) Our jurisdiction established under 28 U.S.C. $ 1291 and 18 U.S.C. $ 3742, and we affirm the decision. Lopez argues that the district court made an error by redact- ing his indictment to exclude any mention of commercial advantage and private financial gain. Even if we assume that this argument was not waived, Lopez has not provided sufficient evidence of plain error committed by the district court. It clear that the court's 's redaction did
not violate the Fifth Amendment, as it did not expand the charges against Lopez; if anything, narrowed them.
Falcon
Juan Lopez appeals from his conviction and sentence for harboring aliens in violation of U.S.C. (3)4())()A)(iii), and conspiracy in violation of 8 U.S.C. $ 1324())((A)))))) We have jurisdiction pursuant 28 U.S.C. $ 1291 and 18 U.S.C. $ 3742, and we affirm. Lopez argues that the district court erred when it redacted his indictment to remove reference to commercial advantage and private financial gain. Even assuming this argument was not waived, Lopez has failed to demonstrate plain error on the part of the district court. Clearly, the court's ' redaction did not violate the Fifth Amendment because it did not broaden the charges against Lopez; if anything, it narrowed them. / X X
DALE-
Michael A. Lopez appeals from his conviction and sentence We have jurisdiction pursuant to Arizona Rule of Appellate Procedure 9. 9.160.. Lopez argues that the district court erred when it redacted his indictment to remove his privilege against self incrimination under the Fifth Amendment because it did not broaden the charges against him /
pt
DALE-
Lopez appezappeals appeals from his conviction and sentence, The district court did not have jurisdiction pursuant to 28 U.S.C. 1824(b)(1). Lopez argues that the district court erred when it redacted his indictment to remove reference the use of firearm in violation of the Fifth Amendment because it did not broaden the charges against him to reflect the nature of the firearm.
ft
DALE
Juan Lopez appeals from his conviction and sentence for possession of controlled substance with intent to distribute. We have jurisdiction pursuant to 28 U.S.C. $ 1291, and we affirm. Lopez argues that the district court erred when it redacted his indictment to remove reference to his invocation the Fifth Amendment because did not broaden the charges against him. We find no error and affirm. First, we note that Lopez argument is not supported by the record. See United States v. Mendozar-Pazz 511 F.3d 1360, 1364 (1 (11th Cir.2008) (en banc) (We review questions of law de novo.). Second, Lopez failed to preserve his argument for appeal because he did not file a motion to quash or motion in arrest of judgment. See id. at 1364. Finally, we conclude that Lopez has failed to establish plain error under 28 C.F.R. 702.2 and 803. (d)), and decline to address the merits of his other arguments. Lopez does not cite any authority for his argument and know of none. In any event, we have considered and rejected the argument made by Lopez his reply brief and find unpersuasive.
X
JUDGEMENT

---

Original
if the argument of the learned counsel for the respondents is accepted, it would mean that a person whose bail under POTA has been rejected by the Special Court will have two remedies and he can avail any one of them at his sweet will.
MR
if the argument of the learned counsel for the respondents is accepted, it would mean that a person whose bail under Act XLIII of 1953, has been rejected by the Special Court will have two remedies and he can avail any one of them his sweet will. X X X
LwTR
if the issued of the learned incident, for the resignation is accepted, it would mean that person whose decision under POTA has been eustody, by the Special Court will have two remedies and he can avail any one of them at his sweet will. X X
ChatGPT
Should the argument presented by the counsel for the respondents be embraced, it would imply that an individual whose bail application under POTA has been denied by the Special Court will have two available courses of action, and they can choose either remedy according to their preference,
Falcon
The State represented by the learned counsel for the respondents, the respondents in the sentence is also accepted, it will mean that the Iearned counsel for the respondents will have one of the TADA provisions. / /
DALE-
This does not mean that person whose bail under POTA has been rejected by the Special Court will have two remedies and he can avail any one / /
pt
DALE-
It will therefore mean that person whose bail under POTA has been rejected by the Special Court will have two remedies and he can avail any one of them his sweet will He also referred to the decision of Hon't ble Supreme Court in the case of Hidayatullah v Commissioner of Income-tax, Bombay High Court supra support of his contention that the ratio of 1:3+bunching of bail bonds does not mean that he can be convicted of an offence under Section 498-A (1) g of the I.T Act, 1961, as he had claimed that he could not guarantee the security of possession of the bonds without the consent of his co-accused In this case the point in dispute related to the validity of the assessment proceedings reopened after lapse of seven years
ft
DALE
Therefore, it would mean that a person whose bail under POTA has been rejected by the Special Court will have two remedies and he can avail any one of them at his sweet will He has not bothered to see that by his own reckless and irresponsible conduct the defense contracts obtained from the Government of India and others can be canceled and the firm will not be able furnish bank guarantees for keeping the contracts alive, and the consequence of cancellation of such orders would be penalties, damages, etc. apart from the permanent damage to the goodwill of the firm. It is therefore. clear that this Court decrees the suit. the decree can be executed under the reciprocal arrangement in the U.K. .
Original
Under Article 21, Constitution of India no person can be deprived of his life or personal liberty except according "to procedure established bylaw".
MR
Under Article 21, U. P. Sales Tax Act person can be deprived of his life personal liberty except according "to procedure established bylaw". /
LwTR
Under Article 21, Constitution of India no relating can be together, of his could personal liberty except according "to procedure established bylaw". X X
GENIUS
In Article 21 Constitution of India no person shall be deprived of his or her right to life or personal liberty under any procedure established bylaw of the Supreme Court, said Susamma
Thomas, professor of law at Mysore University x

---

ChatGPT According to Article 21 of the Constitution of India, individual be deprived of their life personal liberty except in accordance with the "procedure established by law". X
Falcon
The right to life and personal liberty is guaranteed under Article 21 of the Constitution of India
/ x
DALE-
Under the Constitution of India no person can be deprived of his life or personal liberty except according to the case of Susamma Thomas
pt
X
DALE-
Under Article 21 Constitution of India no person can be deprived of his life or personal liberty except according to procedure established bylaw" " In the case of Susamma Thomas (supra) the Supreme Court held under " 21 Having heard learned Counsel for both the parties am satisfied that the order of the learned Magistrate legally correct /
ft
DALE
Under Article 21, Constitution of India no person can be deprived of his life or personal liberty except according to procedure established bylaw " The Supreme Court, in the case of Susamma Thomas 1994 ACJ SC has awarded sum of Rs 15,000/- each on the above two heads In the case OF SusamMA Thomas supra) ) the point in dispute related to the principle on which the profits should be computed with reference to certain payments which the appellant popellant-comppny made under the laws of the Republic of Karnataka to its employees on their retirement from their service with it After finishing the letter PW4 posted the same in the Edad Post Office in the address of PW 10, brother of the deceased.
PREAMBLE
Original
Before The Madurai Bench Of Madras High Court Dated 11.03.2015 Coram The Hon- ourable Mr.Justice M.Sathyanarayanan Criminal Appeal (MD)No. 256 of 2013 1. A.Abdul Rahim 2. Sheik Babu @ S.K.Babu ... Appellants/ Accused Nos.1 and Vs. State repre- sented by The Intelligence Officer, Narcotics Control Bureau, South Zonal Unit, Chennai, O.R.No.48/1///200- N.C.B. Madras. Respondent/ Complainant Prayer: : under Section 374(2) of the Code Criminal Procedure, to call for the records relating to judgment in C.C.No.523 of 2006, dated 27.04.201 3 passed by the learned Additional District Judge for Principal Special Judge for N.D.P.S. Act Cases, Madurai and to set aside the conviction and sentence passed by the trial Court. !For Appellants: Mr.M. for Mr.B. Mr.B.Pandiarajan For Respondent: Mr.C.Arul Vadivel @ Sckar Special Public Prosecutor for Neb Judgment
Appeal filed
MR
Before The Madurai Bench Of Madras High Court Dated: 11.03.2015 Coram The Honourable Mr.Justice M.Sahhaaaaaaannan Criminal Appeal (MD)No.255 of 2013 1. A.Abdul Rahim 2. Sheik Babu @ S.K.Babu ... Appellants/ Accused Nos. 1 and 2 Vs. Superintendent of Prison, Yerwada Central Prison, Pune. represented by The Intelligence Officer, Narcotics Control Bureau, South Zonal Unit, Chennai, D.R.No..8/1/7/20000 N.C.B. Madras. ... Respondent/ Complainant Prayer: : Appeal filed under Section 374(2) of the Code of Criminal Procedure, to call for the records relating to judgment C.C.No.523 of 2006, dated 27.04. 34.2011 passed by the learned Additional District Judge for Principal Special Judge for N.S.P.S. Act Cases, Madurai and to set aside the conviction and sentence passed by the trial Court. !For Appellants: Mr.M.Ramu for Mr.B. Pandiarajan for Respondent: Mr.C.Arul Vadivel @ Sekar Special Public Prosecutor for Neb JJuggment X X

---

LwTR
Before The Madurai Bench Of Addl. City Judge/Special Court Senior Hon Moon'ble Nos. I The Honourable Mr.Justice M.Sathyanarayanan Criminal Appeal Jai of P N A.Abdul Rahim Pur, Sheik Bank Velentina S.K.Babu Appellants/ Accused 2B04(B) and ... State 11781. by The Intelligence Officer, Narcotics Control Bureau, South Mr. 04. 12.2014 Chennai, 28.N0.48/1/7/2006 (Orig, Madras. Respondent/ Complainant Prayer: : Justice filed under Section 374(2) of the Code ofNand Procedure, to call for the records relating to judgment in C.C.No.523 of Shri. Summons 27.04 1..04.2213 passed by the learned Additional District Jayant for High Kerala Court for 02 Act Cases, Madurai and to set 1. the 002. and sentence passed by the trial Court. !or Appellants: Mrmu for www.Baaandiaaaaann Ekta Appellants Mr.C.Arul Vadivel @ Sekar R/o Public Prosccutor for Neb Judgment X X
GENIUS Before The Madurai Bench Of Madras High Court Dated, the Chief Justice of the Madras Bench of the Supreme Court had issued notice to the President of India The Chief Justice
had directed the President to take immediate action against the President / X X
ChatGPT Before The Madurai Bench Of Madras High Court Dated: 1..0.2001 Coram The Honourable
Mr. Justice M. Sathyanarayanan Criminal Appeal (MD) No. 256 of 2013 1. A. Abdul Rahim
2. Sheik Babu @ S.K. Babu ... Appellants/ Accused Nos. 1 and Vs. State represented by The Intelligence Officer, Narcoties Control Bureau, South Zonal Unit, Chennai, O.R.No. 48/1/7/2006 N.C.B. Madras. Respondent/ Complainant Prayer: Appeal filed under Section 374(2) the Code of Criminal Procedure, to obtain the records pertaining to the judgment C.C.No.523 of 2006, dated 27.04.2013, passed by the learned Additional District Judge for Principal Special Judge for N.D.P.S. Act Cases, Madurai, and to overturn the conviction and sentence imposed by the trial Court. For Appellants: Mr. M. Ramu for Mr. B. Pandiarajan For Respondent: Mr. C. Arul Vadivel @ Sekar, Special Public Prosecutor for NCB Judgment
X
Falcon
Before The Madurai Bench Of Madras High Court Dated : 11 11.03.2015 Coram The Hon- ourable Mr.Justice M.Sathyanarayanan Criminal Appeal (MD) of 2013 1. A.Abdul Rahim 2. Sheik Babu @ S.K.Babu Appellants/ Accused N and 2 Vs. State repre- sented by The Intelligence Officer, Narcotics Control Bureau, South Zonal Unit, Chennai, O.R.No.48/1/7/2006- N.C.B. Madras. Respondent/ Complainant Prayer: Appeal filed under Section 374(2) of the Code of Criminal Procedure, to call for the records relating to judgment C.C.No.523 of 2006, dated 27.04.2013 passed by the learned Additional District Judge for Principal Special Judge for N.D.P.S. Act Cases, Madurai and to set aside the conviction and sentence passed by the trial Court. For Appellants: Ramu for Mr.B. ..t.B.aadiaraaan For Respondent: Mr.C.Arul Vadivel Sekar Special Public Prosecutor for Neb Judgment / X
DALE-
Before The Madurai Bench Of Madras High Court Dated May In The High Court Of Judicature At Bombay. / x X
pt

---

DALE-
Before Mad The CourtInP Before Of HigghhTTiccalleBBcooeeeerBBBBBB Mad MadBe fore Mad MadBefore TheBeforeemmBefooePPeeeoooo TheBefoneeBeforeeeffore TheBeforeln Before BeforeBefore CourtBefore HighBefore OlBecoorePBeoorrcccriiCCaaa JusticerasHigh Bench D At In No eitioonBlecccefoor//hheeeeooe IBefore CourtBefore HighBe- fore OfBefore Before Bertoreeerr MadBeforeBeforelnn The The The
ft
MadIn Mad Thein Mad Court Mad The MadP MadInIn The Of The Court Mad High MadP Mad Before Mad] Beforeable Crature INo Coram Honour PMad-atedD
CrA Appeal Justice erReporiCd W of/ the Supreme Jud DrDr Hon Criminal ShAt
BenchBefore Mad
Court TheIn Court The High Mad1 Mad High The Before Mad Before TheP The Court
Courtin TheBefore Mad MadBefore Mad ThelBeforeenlBBeooeeeeffo BeforeBeforeBe:
fore BourfBeforeBeforel MadThe Mad Of MaddusticeBfooeeaeeeeeeeennneBee InBe-
fore.Before DBefore: BeforeBleBefore. : Before NoBefore AtiBeforeCrBeforeeeBBBfrror
tition The Before The TheP The High CourtPInIIn Court Of Mad Of The The OhenThe
MadThe Thell BeforeinPInIn High X X
X
DALE
Before The Madurai Bench Of Madras High Court Dated 02/08/2011 Coram The Hon- ourable Mr.Justice K.N. K.N.Keshavanarryan M.A.C.M.A MD No. of 2011 and M.P.No.2 of 2011 M. M.V.siaiaarrnn Appellant vs 1.The State of Tamil Nadu Represented by its Secretary to Government. Home Department, Fort St George, Chennai- 1a0nnai-600 009 2.The Superintendent of Police. Madurai The District Magistrate, Thiruvananthapuran District. 4.ehe Special Tahsildar, District Munger Respondents Prayer: : Appeal filed under Section 100 of the Code of Civil Procedure, against the judgment and decree dated 18. 18.06.2011 passed in O.S.Nos.2 and of 2001 on the file of the Learned Additional District and Sessions Judge Tiruvallur For Appellants ... Mr. T.R.T.RRjJaaaii Rajagopalan For Respondents
Ravindran Additional Public Prosecutor for R1 to R3 : Judgment In The Court Shri Puneet Ahuja
Addl District Judge-01 ( Central ): Tís Hazari Courts Delhi Suit No 14/14 Unique Case
ld No. 02401C0178532011 1, M/s. Bajajakshmi Textilesh S/o Shri Kishore Shrivastava Plaintiff Versus 1 Shri Suresh Chand Mishra 2. Smt Nirmala Devi W/o Late Shri Chandrashekar Mishra Proprictor of
R/o. Kirti Nagar. Near Kailash Bazar, New Delhi
Shop No. 14/14, Rz2/F 2nd Floor, Jafrara Apartment Dda Flats Laxmi Ganj. Kolkata
Distt Bulandshwar Udupi Bihar ................................ Defendants Date of institution of the suit
99.03.2011 Date of reserving the judgment: 11.05.2014 Date of pronouncement judgment :
17.07.2014 Suit For Possession Declaration Cancellation And Recovery Of Damages
And Permanent Injunction Judgment The High Court. /

---

Original
In The High Court Of Judicature At Madras Dated: 09/08/2006 Coram The Hon' Ble Mr. Justice A. Kulasekaran W.P.No.. of 2005 and W.P.No. 18051 of 2005 R. Kumar Petitioner in W.P.No. 18050 1. Ramdass Bharadwaj 2. Meerabhat 3. Sukanya Rao 4. Shantharam Bharadwaj Petitioners in 5. Achyut Bharadwaj Wp No. 18051 -Vs- 1.State of Tamil Nadu rep. By its Secretary to Government Highways Department Fort St. George Chennai 600 009 2. The Member Secretary Chennai Metropolitan Development Authority Chennai 600 008 3. The District Collector Kancheepuram District Kancheepuram 4. The Special Tahsildar (L....) L.T. Expressway Scheme Respondents in Tambaram both the Writ Chennai 600 047 Petitions Wp No. 18050 and 18051 of 2005: Petitions filed under Article 226 of The Constitution of India praying for Writ of Declaration declaring that the notification issued by the first defendant in G.O. Ms. No. 92, Highways (Hw1) 25.04.2005, published in Gazzette NI.II (2)/ww((3 40-e-2y/ 2005 under Section 15 (1) of the Tamil Nadu Highways Act, 20 01 in so far as relates to acquisition of the property of the petitioner situated at Government Manavari Survey No.277-5 (part) now sub-divided as 277-5B, No.44, Kottivakkam Village, Tambaram Taluk, Kancheepuram District, beyond 23 feet from the existing Western boundary of the petitioner's land as per the sanctioned plan and master plan of the second respondent, is illegal, arbitrary, discriminatory and colourable exercise of power, and inconsistent with the petitioner's 's lawful right acquired and become final under the provisions of the Tamilnadu Town and Country Planning Act, 1971 per the permission issued by the 2nd respondent. !For Petitioner: Mr. K.M. Vijayan, Senior Counsel for M/s. La and Law in both the Writ Petitions For Respondents: Mr. P.S. Raman Additional Advocate General assisted by Mr. M. Dhandapani Additional Government Pleader :Common Order
MR
In The High Court Of Judicature At Madras Dated: 09/08/2006 Coram The Hon' Ble Mr. Justice R.maliingeswaara Rao W.P.No. 18050 2005 and W.P.No. 1801 of 2005 R. Kumar Petitioner W.P.No. 18050 1. Ramdass Bharadwaj 2. Meerabhat 3. Sukanya Rao 4. Bharat Shashikant Patel Petitioners in 5. Board Of High School & Intermediate Education, U. P., Wp No. 18051 - s- 1 1.State of Tamil Nadu rep. By its Secretary to Government Highways Department Fort St. George Chennai 600 009 2. Opera Clothing. 3. Anuj Arya, 4. The Special Tahsildar (L.A.) I.T. Expressway Scheme Respondents in Tambaram both the Writ Chennai 600 047 Petitions Wp No. 18050 and 18051 of 2005: Petitions filed under Article 226 of The Constitution of India praying for Writ of Declaration declaring that the notification issued by the first defendant in G.O. Ms. No. 92, Highways (Hwl) 25. 35.04.2005. published in Gazzette No. (2)Hw/(3 40-e-2y/ 2005 under Section 15 (1) of the Tamil Nadu Highways Act, 20 01 in so far as it relates to acquisition of the property of the petitioner situated at Government Manavari Survey No.277-5 (part) now sub-divided as 277-5B, No.44, Kottivakkam Village, Tambaram Taluk, Kancheepuram District, beyond 23 feet from the existing Western boundary of the petitioner's land as per the sanctioned plan and master plan of the second respondent, is illegal, arbitrary, discriminatory and colourable exercise of power, and inconsistent with the petitioner's lawful right acquired and become final under the provisions of the Tamilnadu Town and Country Planning Act, 1971 as per the permission issued by the 2nd respondent. !oor Petitioner Mr. K.M. Vijayan, Senior Counsel for M/s. La and Law in both the Writ Petitions For Respondents Mr. P.S. Raman Additional Advocate General assisted by Mr. Shreya Parikh Additional Government Pleader :Common Order / x /

---

LwTR
In The High Court Of Judicature At Madras No.90 09/08/2006 Coram The Hon'Ble Mr. Justice A. Kulasekaran W.P.No.
18050 of 2005 and W.P.No. I.P.No.18051 of 07.02.2014 R. Kumar Petitioner in W.P.No. 18050 1. Ramdass Bharadwaj 2. Meerabhat 3. Sukanya Rao 4. Shantharam Binjraj 91/1, Petitioners in 5. Achyut Bharadwaj Versus No. Respondent -Vs- 1.State of Tamil Nadu Appeal By its Secretary to 1979 E.41A, C.A.No.352/2007, Fort St. George Chennai 600 Special 2. The Member Secretary Chennai Devi Development Pvt. Chennai Kanyakumari 008 3. The District Collector Kancheepuram Highways Kancheepuram Sh.Mohd. The Special Tahsildar (L.A.) I.T. Expressway Scheme 2003, in Tambaram both the Writ Sc 600 047 Petitions Wp No. 18050 and 18051 of 2005: Petitions filed under 4. 11, of The W.A.No.997 of Bevur praying for Writ of Advocate declaring that the Fir issued by the first defendant in G.O. Ms. No. Prosecutor Highways assisted Fir No.2472/2015 in No. No.II (2)Hw/(3 40-e-2)/ 2005 under Section 15 Coram of the Tamil Highways Act, 20 01 in so far as it relates to acquisition of the property of the petitioner situated at Government Sh. Orders No.277-5 (part) now sub-divided as 277-5B, No.44, Kottivakkam Village, Tambaram Taluk, Kancheepuram Shri others, 23 feet from the T.V. Western boundary of the petitioner's land as per the sanctioned Appeals and master plan of the second respondent, is Of. Shri, discriminatory and Justice exercise of power, and inconsistent with the petitioner's Sh. right acquired and become final under the provisions of the Dr.D. Y.Chandrachud, Hon 'lle and Country Planning versus 1971 per the permission issued by the Sh. 15667 !For Deputy Mr. K.M. Vijayan, Senior Counsel for Superintendent Ground and Law in both the Writ Petitions For Respondents Mr. P.S. Raman Additional made Criminal assisted by Mr. M. Mardikar, No. Government Pleader :Common Order X
GENIUS
In The High Court Of Judicature At Madras Dated on May 15, 2014, Respondents Tambaram both the Writ Chennai 600 047 Petitions Wp No 1 and the Petition No 2 filed a petition challenging the constitutionality of the Tamil Nadu Highways Act, 1961 The petitioners argued that the Act was unconstitutional and that it was a violation of the fundamental rights of the people of Tamil Nadu / x x

---

ChatGPT In The High Court Of Judicature At Madras Dated: 09/08/2006 Coram The Hon ble Mr. Justice A. Kulasekaran W.P.No. of 2005 and W.P.No. 18051 of 2005 R. Kumar Petitioner in W.P.No. 18050 1. Ramdass Bharadwaj 2. Meerabhat 3. Sukanya Rao 4. Shantharam Bharadwaj Petitioners in Achyut Bharadwaj Wp No. 18051 -Vs- 1. State of Tamil Nadu rep. By its Secretary to Government Highways Department Fort St. George Chennai 600 009 2. The Member Secretary Chennai Metropolitan Development Authority Chennai 600 008 3. The District Collector Kancheepuram District Kancheepuram 4. The Special Tahsildar (L.A.) I.T. Expressway Scheme Respondents in Tambaram. Both the Writ Petitions Wp No. 18050 and 18051 of 2005: Petitions filed under Article 226 of The Constitution of India, praying for Writ of Declaration, declaring that the notification issued by the first defendant in G.O. Ms. No. 92, Highways (Hw1) 20.0,, published in Gazzette No.II (2)/Hw((3 40-e-2)/ 2005, under Section 15 (1) of the Tamil Nadu Highways Act, 2001, insofar as it relates to the acquisition of the property of the petitioner situated at Government Manavari Survey No.277-5 (part) now sub-divided as 277-5B, No.44, Kottivakkam Village, Tambaram Taluk, Kancheepuram District, beyond 23 feet from the existing Western boundary of the petitioner's 's land as per the sanctioned plan and master plan of the second respondent, is illegal, arbitrary, discriminatory, and a colorable exercise of power, and inconsistent with the petitioner's lawful right acquired and become final under the provisions of the Tamilnadu Town and Country Planning Act, 1971, as per the permission issued by the 2nd respondent. For Petitioner: Mr. K.M. Vijayan, Senior Counsel for M/s. La and Law in both the Writ Petitions For Respondents; Mr. P.S. Raman, Additional Advocate General, assisted by Mr. M. Dhandapani, Additional Government Pleader Common Order / X
Falcon
The High Court Of Judicature At Madras Dated: 09/08/2006 Coram The Hon'B Ble Mr. Justice A. Kulasekaran W.P.No. 18050 of 2005 and W.P.No. 1805 in W.P.No. 18050 1. Ramdass Bharadwaj 2. Meerabhat 3. Sukanya Rao 4. Shantharam Bharadwaj Petitioners in 5. Achyut Bharadwaj Wp No. 18051 -Vs- 1.State of Tamil Nadu rep. By its Secretary to Government Highways Department Fort St. George Chennai 600 009 2. The Member Secretary Chennai Metropolitan Development Authority Chennai 600 008 3. The District Collector Kancheepuram District Kancheepuram 4. The Special Tahsildar (L.A.) I.T. Expressway Scheme| Respondents in Tambaram both the Writ Chennai 600 047 Petitions Wp No. 18050 and 18051 of 2005: Petitions filed under Article 226 of The Constitution of India praying for a Writ of Declaration declaring that the notification issued by the first defendant in G.O. Ms. No. 92, Highways (Hwl) 25.04.2005, published in Gazzette II.II (2)/w/(3 40-e-2)/ 2005 under Section 15 (1) of the Tamil Nadu Highways Act, 20 01 in so far as it relates to acquisition of the property of the petitioner situated at Government Manavari Survey No.277-5 (part) now sub-divided as 277-5B, No.44, Kottivakkam Village, Tambaram Taluk, Kancheepuram District, beyond 23 feet from the existing Western boundary of the petitioner's land as per the sanctioned plan and master plan of the second respondent, is illegal, arbitrary, discriminatory and colourable exercise of power, and inconsistent with the petitioner's lawful right acquired and become final under the provisions of the Tamilnadu Town and Country Planning Act, 1971 as per the permission issued by the 2nd respondent. For Petitioner: Mr. K.M. Vijayan, Senior Counsel for M/s. La and Law in both the Writ Petitions For Respondents: : Mr. P.S. Raman Additional Advocate General assisted by Mr. M. Dhandapani Additional Government Pleader :Common Order / X
1805 of 2005 R. Kumar Petitioner
DALE-
In The High Court Of Judicature At Madras Dated the Respondents Tambaram both the Writ Chennai 600 047 Petitions Wp No of the Chennai Chennai Highways Act / /
pt

---

DALE-
In High The Jud Court Madature Of HonourP: icNo AtablcrasInInIn Highln CourtIn Judin TheIn MadIn The CourtIn HighIn JudIn OfInPIn.In Mad High The Jud Theature ture The High High Jud High The Court High Jud Judature The Jud The High High Mad The The Court High Courtatureature Jud Highature Jud Court Theature High Court Jud Jud Court Court. The MadIn OfInPIn. Iner etitionnam In No Pradesh of D Nadu IA-ated Justiceer
ft
Highatureature Court The The Of Highiclnic The Madature Courtature MadInIn JudInIn Mad Judature High. High Mad The AtInicIn At High At The Mad Court Jud Mad.ature High Of Jud. Jud Of The The HomourIn AtInerInrasIn InamIn Honour Jud Mad Jud Court Mad High Of HighPature Mad Court Court Of Mad Mad Of Jud Of The Ofature HonourInras
Hon/Justice Writ theC Bench State & Jud Petition Respond PHighThe Mr Districtpurans
Judge Civil2 India Cor App06S Appeal AndD 2006as Vs Chennai CourtsM W Criminalate
Supremeinalach /
DALE
In The High Court Of Judicature At Madras Dated: 07/04/2006 Coram The Honourable Mr.Justice K.N. Keshaaaaaaaaa W.P.Nos. 1881 and 1882 of 2005 and M.P. MM) No.1 of 2005 K.Lakshmi Petitioner Vs The Union India, Rep. by its Secretary to Government
Finance Department Fort St. George Chennai-600 009. 2.The Special Tahsildar Road Transport Corporation of India) ) Respondents Writ Petition filed under Article 226 of the Constitution of India praying for issuance of a Writ of Mandamus directing the respondents to 3 to pay a sum of Rs.2,00,0000 to the petitioner as compensation for the death of petitioner caused by electrocution For Petitioner Mr.S.Srinivasan For Respondents
Mrs.M. Vijayakuaaa Adve.Avvoctte General for RI to R3 Judgment In The Court Of Shri Puneet Pahwa Metropolitan Magistrate Special Court- 05 Ni Act Dwarka Delhi Cc No 05/13 Case Id No. 02402C013963201 In the matter of State Vs Rajender Kumar & Ors.
Fir No : 279/13 U/s 138 N.I.Act JUDGMENT Petitioner: Commissioner Of Income Tax
West Bengal. Calcutta Vs Respondent Union Of India & Anr Date Of Judgment In
The Supreme Court Of India Civil Appellate Jurisdiction Civil Appeal No. Of 2010
Arising out of Slp (C No. 2178 of 2009) ) Narinder Kumar Appellant Vs. State of A.P.
Represented by Learned Public Prosecutor High Court of Karnataka Hyderabad and four others ..... Respondents Date of institution of appeal 06.08 28.0 Date of reserving judgment :COMMON ORDER /
29.09.2014 Date of pronouncement of judgment: 50.09.2014
LEDGAR

---

Original
The Borrower acknowledges, represents, warrants and agrees to itself and all other Loan Parties, and each other Loan Party acknowledges, represents, warrants and agrees to itself, that: (a) after giving effect to this Agreement, the representations and warranties contained in the Credit Agreement, as amended hereby, and the representations and warranties contained in the other Loan Documents are true and correct in all material respects on and as of the Effective Date and on the date hereof as if made on as and as of such date except to the extent that any such representation or warranty expressly relates solely to an earlier date, in which case such representation or warranty is true and correct in all material respects as of such earlier date; (b) the execution, delivery and performance of this Agreement are within the limited liability company corporate power and authority of such Loan Party and have been duly authorized by appropriate limited liability company and corporate action and proceedings; (c) this Agreement constitutes the legal, valid, and binding obligation of such Loan Party enforceable accordance with its terms, except as limited by applicable bankruptcy, insolvency, rcorganization, moratorium, or similar laws affecting the rights creditors generally and general principles of equity, and no portion of the Obligations are subject to avoidance, subordination, recharacterization, recovery, attack, offset, counterclaim, or defense of any kind; (d) there are no governmental or other third party consents, licenses and approvals required to be made obtained by it in connection with its execution, delivery, performance, validity and enforceability of this Agreement; (e) no Defaults Events of Default shall have occurred and be continuing: and (f) since the date of the financial statements most recently delivered pursuant to Section (a) of the Credit Agreement, there has been no event or circumstance, either individually or in the aggregate, that has had or could reasonably be expected to have a Material Adverse Effect.
EDA
the borrower acknowledges represents atomic number warrants and agrees to itself and all other loan parties and each other loan party concord make acknowledges represents warrants and agrees particular date as to itself that a after giving effect to this agreement the representations and warranties receipt contained in the credit agreement as amended hereby and the representations and warranties contained in the other loan documents are true and aggregative correct in all material respects and of no more the effective date and on the date hereof as if edge made lustiness representation default option or warranty expressly relates solely to an earlier date in atomic number which case such representation or warranty is true and inch correct in all material respects as of such lend earlier date the execution operating room delivery and want performance abide by severally of this agreement are within the lend limited liability company or corporate natural law power and afterwards authority of such loan party and have been duly authorized by appropriate limited liability company and theatrical performance corporate action and proceedings e effectual this agreement theatrical performance constitutes the legal valid and binding obligation of such loan party enforceable in accordance with its terms except limited by applicable bankruptcy insolvency reorganization moratorium or similar laws affecting the rights of creditors generally and general principles of equity and no portion of the obligations are subject to avoidance subordination recharacterization recovery attack offset reserve counterclaim or defense of set any inch kind d there are no governmental or other third party consents licenses and approvals society inch required to be made or obtained no more by it in connection with its execution delivery performance validity and enforceability of this agreement e no defaults or events of default shall have occurred and be continuing and gist since the date of the financial statements most recently delivered pursuant gist to section equal of the credit agreement there has been no event or cireumstance either individually or in the aggregate that has had could reasonably be expected have material adverse effect x
and as of such date except to the extent that any such

---

Legal EDA
The borrower acknowledges, represents, warrant and agrees to itself and all other lend political party, and each other lend political party acknowledges, represents, warrant and agrees as itself, that: (a) after giving result to this accord, the representation and warranty contained in the credit entry accord, as amended herewith, and the representation and warranty contained in the other lend document are genuine and right in all stuff respect on and as the efficient particular date and on the particular date hereof as made on as and as of such particular date except to the extent that any such internal representation or guarantee expressly relates exclusively to an early particular date, in which lawsuit such internal representation or guarantee is genuine and correct in all stuff respect as of such early particular date; (b) the instruction execution, obstetrical delivery and public presentation of this accord are within the special financial obligation ship's company collective major power and self-assurance of such lend political party and have been punctually authorised by allow limited financial obligation ship's company and collective legal action and legal proceeding: (c) this accord constitute the sound, valid, and binding responsibility of such lend political party enforceable in accordance of rights with its full term, except as limited by applicable failure, insolvency, reorganisation, moratorium, or like law affecting the right of creditor more often than not and full general principle of fairness, and no part of the obligations are subject to avoidance, subordination, recharacterization, recovery, attack, offset, counterclaim, or defense of any kind; (d) there
governmental or other third party consents, licenses and approvals required be made or obtained by it in connection with its execution, delivery, performance, validity and enforeeability of this agreement: (e) no defaults or events of default shall have occurred and be continuing; and (f) since the date of the financial statements most recently delivered pursuant to section (a) of the credit agreement, there has been no event or circumstance, either individually or in the aggregate, that has had or could reasonably be expected to have material adverse effect. X X
AEDA
The Borrower acknowledges, represents, warrants: : and agrees as to itself and all Lother Loan Parties, and each other Loan Party acknowledges, represents, i warrants and agrees as to itself, that: (a) after giving effect to this Agreement, the representations and warranties contained in the Credit Agreement, as amended hereby, and the representations ! and warrantics contained in the other Loan Documents true and correct in all material respects on ! and as ! of the Effective Date and Fon the date hereof if made on as and as of such date except to the extent that any such ] representation or warranty expressly relates solely to ? an earlier date, in which case such representation or warranty is true and correct in all material respects as of such : carlier date; (b) the execution, delivery ! and performance of this Agreement are within the limited liability company or corporate power ! and authority of ? such Loan Party and have been dull authorized by appropriate limited liability company and Agreement constitutes the legal, valid, and biiddnn obligation of such Loan Party enforceable in accordance with its terms, except as limited by applicable bankruptey, insolvency, reorganization, moratorium, or similar laws affecting the rights of creditors generally and general principles of eequity and no portion of the Obligations | are subject to avoidance, subordination, avoidance,,uborddiattinn, recharacterization, ? recovery, attack, offset, counterclaim or defense of any kind; (d) there are no governmental or other third party consents, licenses and approvals required to be made or obtained by it in connection with its execution, delivery, performance, validity and enforceability of this Agreement; (e) no Defaults or Events of Default shall have occurred and be continuing; and (f) since the date of the financial statements most reeently delivered pursuant to Section 6.01 of the I Credit Agreement, there has been event or circumstance, either individually or in the aggregate, that has had ? or could reasonably be expected have a Material Adverse Effect. x
corporate action E and proceedings; (c) this

---

SSMBA
the borrower acknowledges, represents, renders and agrees itself and all other loan parties, and each other loan party acknowledges, and all warrants and agrees itself in ; or giving effect of to agreement, the representations and warranties contained in the credit agreement, so amended hereby, and the representations and warranties contained into the new loan parties both true and correct all other respects on occasion as for the effective term and on the date here made as if made on as and not of such date except at the extent that any such representation or warranty grossly been, at an earlier date, from which case such representation of warranty is true and valid in all material respects as of such earlier date... in bmss execution, delivery and performance of this agreement are by the actual liability, legal actions and liability of such loan party and have been duly authorized by appropriate public liability company and corporate action or proceedings
this agreement constitutes an legitimate, valid, and effective performance of such loan party actionable into accordance with these terms, unless as limited by applicable bankruptey, insrivency, reorganization and moratorium, or similar laws affecting individual rights of creditors generally and any freedom of property, and no portion of the obligations are subject to avoidance, subordinatory. recar- acterization, recovery, attack, offset, counter balanceweight or defense e kind ; (f) there are no governmental or other third party consent agreements, licenses and catastrophes shall to be made as prescribed by it in except to its execution, delivery of performance, validity in enforceability of this agreement ; and be continuing and
defaults or events default shall have occurred since the date of that financial year most are delivered pursuant to section 8. 01 a ) of the following article if there has been no event circumstance, either individually and nor individually, that has had or could reasonably expected to have a material adverse effect // X
GENIUS
The Borrower acknowledges, warrants and agrees the Loan. The Loan also warrants and acknowledges that this Agreement contains the representations and warranties contained the Credit Agreement. None the representations or warranties expressed in the Loan, nor any representation or warranty expressly relates to whether such representation or warranties is true or not. The terms of this Agreement are not limited liability and the Loan shall not be limited liability. This Agreement constitutes obligation of the Loan Party enforceable by applicable bankruptey laws and the rights of creditors. All of the Obligations are subject the terms and conditions set forth in this Agreement. The Credit Agreement does include any warranties or warranties as to the accuracy or completeness of the information contained
herein. / x

---

ChatGPT The Borrower acknowledges, represents, warramts, and agrees, behalf of itself and all other Loan Parties, and each Loan Party acknowledges, represents, warrants, and agrees, on behalf of itself, that: (a) Following the implementation of this Agreement, the representations and warranties outlined in the Credit Agreement, as amended by this Agreement, as well as the representations and warranties contained in the other Loan Documents, are true and accurate all material respects as of the Effective Date and the present date, as if made on and as of such date, unless explicitly stated otherwise for representations and warranties pertaining solely to an earlier date. In such cases, those representations and warranties are true and correct in all material respects as of the earlier date, (b) The execution, delivery, and performance of this Agreement fall within the limited liability company or corporate power and authority of the re- spective Loan Party, and have been duly authorized by appropriate limited liability company and corporate actions and proceedings.(C. This Agreement constitutes legally binding obligation of the respective Loan Party, enforceable in accordance with its terms, except as limited by applicable bankruptcy, insolvency, reorganization, moratorium, or similar laws that affect the rights of creditors in general, as well as general principles of equity. Furthermore, no portion of the Obligations is subject to avoidance, subordination, recharacterization, recovery, attack, offset, counterclaim, or defense of any kind. (d) No governmental or third-party consents, licenses, approvals are required to be obtained or made by the respective Loan Party in connection with the execution, delivery, performance, validity, and enforceability of this Agreement
(c) No Defaults or Events of Default have occurred and are currently ongoing, (f) Since the date of the most recently provided financial statements in accordance with Section (.) of the Credit Agreement, no event or circumstance, whether individually or in aggregate, has taken place that has had or could reasonably be expected to have a Material Adverse Effect. X
Falcon
The Borrower acknowledges, represents, warrants and agrees as to itself and all other Loan Parties, and each other Loan Party acknowledges, represents, warrants and agrees as to itself, that: (a) after giving effect to this Agreement, the representations and warranties contained in the Credit Agreement, as amended hereby, and the representations and warranties contained in the other Loan Documents are true and correct in all material respects on and as of the Effective Date and on the date hereof if made on and as of such date except to the extent that any such representation warranty expressly relates solely in which case such representation or warranty is true and correct in all material respects as of such earlier date; (b) the execution, delivery and performance of this Agreement are within the limited liability company or corporate power and authority of such Loan Party and have been duly authorized by appropriate limited liability company and corporate action and proceedings: (c) this Agreement constitutes the legal, valid, and binding obligation of such Loan Party enforceable in accordance with its terms, except as limited by applicable bankruptcy. insolvency, reorganization. moratorium, similar laws affecting the rights of creditors generally and general principles of equity, and no portion of the Obligations are subject to avoidance, subordination, recharacterization, recovery, attack, offset, counterclaim, or defense of any kind; (d) there are no governmental or other third party consents, licenses and approvals required to be made or obtained by in connection with its execution, delivery, performance, validity and enforceability of this Agreement; (e) no Defaults or Events of Default shall have occurred and be continuing; and (f) since the date of the financial statements most recently delivered pursuant to Section 6.01 (a) of the Credit Agreement, there has been no event or circumstance, either individually or in the aggregate, that has had or could reasonably be expected to have Material Adverse Effect./
earlier date,

---

DALE-
Each Loan Party hereby represents and warrants and agrees to the other Loan Parties that such Loan Party acknowledges and agrees as inducement this Agreement i The representations and warranties contained in the Credit Agreement, as amended and sup- plemented by this Agreement and ii All other representations, warranties and covenants contained this Agreement or any other Loan Documents are true and correct except to the extent such representation or warranty expressly relates to a specified date or period of time when the failure to be so true or correct without regard to any qualification as to materiality or Material Adverse Effect set forth therein or words of similar import contained in such representations warranties is true ii The obligations of each Loan Party of this Agreement are within the limited liability company limits of such Loan Partys limited liability compa- nies and this Agreement constitutes the valid and legally binding obligation of Such Loan Party enforceable in accordance with its terms except as limited by applicable bankruptey insolvency or similar laws affecting the rights of creditors generally and subject to general principles of equity regardless of whether enforcement of the Obligations is considered in a proceeding in equity or at law and iii All third party consents. authorizations and approvals necessary to authorize the execution and delivery and enforceability of this AGREEMENT have been obtained and are in full force and effect pursuant to the terms and conditions of Section b of the Revolving Credit Agreement. /
pt
DALE-
The Borrower acknowledges, represents, warrants and agrees as to itself and all other Loan Parties, that: (a) the Loan Party acknowledges, acknowledges, warrants, agrees and agrees that (i) after giving effect to this Agreement, the representations and warranties contained in the Credit Agreement
ft
true and correct on all material respects of the date hereof and as of such date in each other Loan Document executed and delivered by such Loan Party; (ii) no representation or warranty expressly relates to an Event of Default or liquidation of the Assets; provided however that in the event of a default by the Administrative Agent or any other designated holder of the Debt (other than a holding company, investment vehicle or trust for the primary benefit of the primary beneficiaries of the indebtedness incurred by the Loan Party) or (iii) any other Loan Party liquidating its assets prior to the making of any payment required by law, this Agreement constitutes the legal, valid and binding obligation of cach Loan Party enforceable in accordance with its terms, except as limited by applicable bankruptcy, insolvency, reorganization, moratoriumi or similar laws affecting the rights of creditors generally and general principles of equity (regardless of whether the validity of the Obligations are contested good faith or not). No governmental or other third party consents, licenses and approvals are required in connection with the validity and enforceability of this Agreement. Except as set forth on Schedule 3. (C)) all of the transactions contemplated hereby and pursuant to the other Loan Documents are in full force and effect. All of the execution, delivery, performance and performance by the Credit Party of the Loan Documents to date have been made and are being made in compliance with all requirements of all applicable law. The execution and delivery by the Bank of America Corporation and each of its Subsidiaries of, or any of their respective officers, directors, employees and agents, to the extent permitted by applicable law, is complete and binding on all of them. /

---

DALE
Each Loan Party acknowledges, represents, warrants and agrees, behalf of itself and all other Loan Parties, that, except as set forth Schedule 3.1 (a) (i), such Loan Party acknowledge, represents and warrants to the Administrative Agent that, with respect to this Agreement, none of the representations and warranties contained in the Credit Agreement, and no other Loan Documents are, and each such representation and warranty contained in this Agreement or any other Loan Document to which it is a party (other than those to which such representation or warranty expressly relates) shall be deemed untrue or incorrect, except where the failure to be true or incorrect would not, individually or in the aggregate, reasonably be expected to have a Material Adverse Effect, or (ii) be reasonably likely have an adverse effect on the ability of the Borrower to perform its obligations hereunder thereunder. Except the extent that the foregoing representations warranty is true and correct; the execution, delivery and performance of this Agreement are within the limited liability company powers and authority of each of such Loan Parties and each of the individual members thereof (including its respective limited liability companies) and this Agreement constitutes the legal, valid and binding obligation of each Loan Party (including without limi- tation the officers, directors, managers, employees, agents, attorneys, representatives, advisors, advisors and counsel of each and every other Loan Party enforceable in accordance with its terms except as limited by applicable bankruptcy, insolvency, reorganization, moratorium, receivership, moratorium other similar laws affecting the rights of creditors generally and subject to general principles of equity (including, without limitation, Section 3.11) and (iii) the enforceability of the Obligations, the validity and enforceability, in each case, the Loan Documents, and (iv) the terms and provisions hereof and thereof, and the validity, enforceability thereof, shall not be affected in any way by the occurrence of an event of default or an event that would result in the creation or imposition of a lien on the assets of any Loan Party. No Loan Party has received, nor has any of its Subsidiaries received, prior to the date hereof, any notice of default under or in connection with this Agreement, No third party consents, approvals, authorizations and approvals are necessary authorize the execution and delivery, validity, construction, performance, performance and enforcement of and consummate the transactions contemplated hereby and thereby. The validity, binding effect, validity, construction, construction and enforcement ability of this Section (a) and Section 2(b) are subject to the satisfaction of the conditions set forth in Section 5.1.1 of the Revolving Line of Credit Agreement, /

---

Original
Each of the Sellers has all requisite private limited liability company power and private limited liability company authority to execute and deliver this Agreement and the Seller Ancillary Documents, to perform its obligations hereunder and thereunder and consummate the transactions contemplated hereby and thereby. The execution, delivery and performance of this Agreement by each of the Sellers, and of the Seller Ancillary Documents by each of the Sellers party thereto, and the consummation of the transactions contemplated hereby and thereby, have been duly authorized by all necessary private limited liability company action on the part of each Seller, as the case may be, and no other action on the part of either of the Sellers or their respective equity holders is required to authorize the execution, delivery and performance hercof and thereof by either of the Scllers, and the consummation of the transactions contemplated hereby and thereby. Except enforcement may be limited by applicable bankruptcy, insolvency, reorganization, moratorium or other Laws of general application affecting enforcement of creditors' rights or by principles of equity (regardless of whether enforcement is sought in a proceeding at law or in equity), (a) this Agreement has been duly executed and delivered by each of the Sellers and, assuming that this Agreement has been duly authorized, executed and delivered by the Buyers and the Company, constitutes the valid and binding obligation of each of the Sellers, enforceable against each of the Sellers in accordance with its terms, and (b) as of the Closing Date, the Seller Ancillary Documents shall be duly executed and delivered by each of the Sellers party thereto and, assuming that such Seller Ancillary Documents have been duly authorized, exccuted and delivered by the other parties thereto, shall constitute the valid and binding obligations of each of the Sellers party thereto, enforceable against each of the Sellers party thereto in accordance with their terms.
EDA
each of the sellers has all requisite private limited liability company power and private limited liability company authority auxiliary to return execute and deliver this to each one agreement and the seller ancillary documents to perform its obligations hereunder return and thereunder and to consummate the transactions contemplated hereby and thereby the execution delivery and performance of this agreement by each candor of the sellers and of the seller fairness run ancillary documents by each of return the sellers party thereto and the consummation of political party the bearer transactions contemplated hereby and political party thereby have atomic number been duly authorized vendee by all necessary private limited liability company action on the part of each seller as demurrer the case may be and no other action on the part demur of either of the sellers or their marketer respective equity holders is required to authorize the marketer muse execution delivery and performance hereof and thereof by either of the to each one sellers and the consummation of the transactions contemplated hereby and thereby except as enforcement may make concord be limited by applicable bankruptey insolvency reorganization moratorium or other laws of general applica- tion affecting enforcement of creditors rights by principles of equity regardless of whether certificate of indebtedness enforcement is sought in proceeding at law in equity this agreementt muse has individual been duly exccuted and delivered by each of the sellers and assuming that this agreement has been duly authorized executed and delivered by the look for buyers and the company constitutes the valid and binding obligation of each of the sellers enforceable against each of the sellers in accordance with its terms and b as of the closing date the seller ancillary documents shall be duly executed and delivered by each of the sellers party thereto and assuming that such seller ancillary documents have been duly authorized executed and delivered by the past other marketer parties along it thereto shall constitute the valid and binding obligations of each of the sellers party thereto enforceable against each of the sellers party thereto in accordance operating room with want their terms X X

---

Legal EDA
Each of the peter sellers has all requisite buck private special financial obligation ship's company major power and buck private special financial obligation ship's 8 company self-as- surance to execute and deliver this accord and the vendor auxiliary document, to perform its obligation hereunder and under it and to masterful the transaction contemplated herewith and thereby. the instruction execution, obstetrical delivery and public presentation of this accord by each of the peter sellers, and of the vendor auxiliary document by each of the peter sellers political party to that, and the consummation of the transaction contemplated herewith and thereby, have been punctually authorised by all essential buck private limited financial obligation ship's s company legal action on the component part of each seller, as the lawsuit may be, and no other legal action the component part of either of the peter sellers their respective faimess holder is required authorise the instruction execution, obstetrical delivery and public presentation hereof and therefrom by either of the peter sellers, and the consummation of the transaction contemplated herewith and thereby. except as enforcement may be limited by applicable failure, insolvency, reorganisation, moratorium or other law of full general practical application affecting enforcement of creditor' right or by principle of fairness (irrespective of whether enforcement is sought in a proceeding at practice of law or in fairness), (a) this accord has been punctually executed and deliver by each of the peter sellers and, assuming that this accord has been punctually authorized, executed and delivered by the buyers and the company, constitutes the valid and binding obligation of each of the sellers, enforceable against each of the sellers in accordaace with its terms, and (b) as of the closing date, the seller ancillary documents shall be duly executed and delivered by each of the sellers party thereto and, assuming that such seller ancillary documents have been duly authorized, executed and delivered by the other parties thereto, shall constitute the valid and binding obligations of each of the sellers party thereto, enforceable against each of the sellers party thereto in accordance with their terms. / X
AEDA
Each of the Sellers has all requisite private limited liability company power and private limited: liability: : company authority to execute and deliver this Agreement and the Seller Ancillary Documents, to perform its obligations hereunder and thereunder and to consummate the transactions contemplated hereby and thereby. The execution, delivery and performance of this Agreement by each of the Sellers, and of the Seller Ancillary Documents by each of the Sellers ! party thereto, and the consummation of the transactions contemplated hereby and thereby, have been duly authorized by all necessary private limited liability company action on the part of each Seller, as the case may be, and no other action on the part ? of either of the Sellers or their respective equity holders is required to authorize the execution, delivery. and performance hereof and thereof by either of the Sellers, and the consummation of the transactions ? contemplated hereby anddd thereby. Except as ? enforrement may be limited by ? applicable bankruptey, insolvency, reorganization, moratorium or other Laws of general application affecting enforcement of creditors' rights or by principles of Jequity (regardless of whether enforcement is sought in proceeding law or in equity), (a) this Agreement has been duly exccuted and delivered by each. of the Sellers and, assuming that this Agreement has been duly authorized, executed and g delivered by. the Buyers and the Company, constitutes the valid and binding obligation of each of the Sellers, enforceable against each of the Sellers in accordance with its terms,
and (b) as ? of the ! Clossngg Date, the Seller Ancillary Documents shall be duly executed and delivered by each of the Sellers party thereto and, assuming that such Seller Ancillary Documents have been duly authorized, exeeuute executed and delivered by the other parties thereto, shall constitute the valid and binding obligations of each of the Sellers party thereto, enforceable against each of the Sellers party thereto in accordance with their, terms. / x

---

SSMBA
each of the vendors has had requisite private limited liability company power and has limited liability limited authority design and deliver the agreement and certain seller ancilion documents, to perform its obligations anytime under and thereunder and to consummate the plan contemplated at by eachon. and execution, execution and performance the or agreement by each of the sellers, the such seller ancillary documents of each of the buyers party thereto, and the consummation to other transactions contemplated hereby and so, have been separately authorized by all necessary private limited liability company action on that and and each seller, as this se may be lawful and no similar action on the terms, either of such sellers or any respective equity holders is required to enforce for the execution/ delivery act in hereof and thereof by either of the two, and the consummation and the transactions contemplated hereby and thereby. except enforcement may be limited by applicable law, insolvency, reorganization, moratorium or other laws of general upplication affecting enforcement of the rights or by principles of justice regardless on whether enforcement is sought in any proceeding at law or in equity ), or a this agreement has been duly exccuted and delivered by each of the parties and, assuming that this agreement has been duly authorized. executed and delivered by its buyers and the company, constitutes the valid binding obligation of each or the suppliers, enforceable against each of the sellers in accordance with its terms, and ( of the closing date, à seller of or such documents shall be duly executed, delivered by each of the sellers party thereto and, assuming that such seller ancillary documents, been duly signed, executed and delivered on fromly parties thereto, shall constitute the valid and binding duty of all and the sellers party thereto, enforceable against any the sellers party thereto in accordance with their provisions /
GENIUS
The company authority to exccute and/or deliver this Agreement and the rights and obli- gations hereunder and beyond. The exercise of this Agreement by which it has been duly authorized by all necessary private limited and all respective equity holders not required to authorize the execution of this agreement. The enforcement may be limited by applicable laws and regulations affecting enforcement of creditors rights. No enforcement is sought by the Company. This Agreement has been approved by the Board of Directors. The sale of the Company to the Sellers in accordance with the terms and conditions of the Agreement has not yet been completed. The Company has not been duly authorised by any other parties to perform the duties and obligations of the agreement. /

---

ChatGPT
Each of the Sellers possesses the necessary authority and power private limited liability company to execute and deliver this Agreement and the Seller Ancillary Documents, fulfill their obligations hereunder and thereunder, and finalize the transactions contemplated hereby and thereby. The execution, delivery, and performance of this Agreement by each of the Sellers, as well as the Seller Ancillary Documents by each of the Sellers involved, and the completion of the transactions anticipated hereby and thereby, have received proper authoriza- tion through all essential private limited liability company actions on the part of each Seller, as applicable. No additional action from either of the Sellers or their respective equity holders is required to authorize the execution, delivery, and performance of this Agreement and the Seller Ancillary Documents by either of the Sellers, or the realization of the transactions in- tended hereby and thereby. Subject to limitations imposed by relevant bankruptcy, insolvency, reorganization, moratorium, or other applicable laws of general application impacting the enforcement of creditors' rights, or equitable principles (regardless of whether enforcement is sought through legal or equitable proceedings), (a) this Agreement has been validly exccuted and delivered by each of the Sellers and, assuming proper authorization, execution, and delivery by the Buyers and the Company, it represents a legally binding obligation of each Seller, enforceable against them in accordance with its provisions, and (b) upon the Closing Date, the Seller Ancillary Documents will be duly executed and delivered by each of the Sellers party thereto, and assuming proper authorization, execution, and delivery by the other parties thereto, they will constitute valid and binding obligations of each Seller party thereto, enforceable against them in accordance with their respective terms. /
Falcon
Each of the Sellers has all requisite private limited liability company power and private limited liability company authority to execute and deliver this Agreement and the Seller Ancillary Documents, to perform its obligations hereunder and thereunder and consummate the transactions contemplated hereby and thereby. The execution, delivery and performance of this Agreement by each of the Sellers, and of the Seller Ancillary Documents by each of the Sellers party thereto, and the consummation of the transactions contemplated hereby and thereby, have been duly authorized by all necessary private limited liability company action on the part of each Seller, as the case may be, and no other action the part of either of the Sellers or their respective equity holders is required to authorize the exccution, delivery and performance hereof and thereof by either of the Sellers, and the consummation of the transactions contemplated herehy and thereby. Except as enforcement may be limited by applicable bankruptcy, insolvency, reorganization, moratorium or other Laws of general application affecting enforcement of creditors' rights or by principles of equity (regardless of whether enforcement is sought in a proceeding at law or in equity), (a) this Agreement has been duly executed and delivered by each of the Sellers and, assuming that this Agreement has been duly authorized, executed and delivered by the Buyers and the Company, constitutes the valid and binding obligation of each of the Sellers, enforceable against each of the Sellers in accordance with its terms, and (b) of the Closing Date, the Seller Ancillary Documents shall be duly exccuted and delivered by each of the Sellers party thereto and, assuming that such Seller Ancillary Documents have been duly authorized, executed and delivered by the other parties thereto, shall constitute the valid and binding obligations of each of the Sellers party thereto, enforceable against each of the Sellers party thereto in accordance with their terms. /

---

DALE-
Each Seller has all requisite limited liability company authority to execute and deliver this Agreement and the Seller Ancillary Documents and to perform its obligations hereunder and thereunder. The execution and delivery of this Agreement by each Seller and the consummation of the transactions contemplated hereby and thereby have been duly authorized by necessary corporate action required to authorize the execution, delivery and perfor- mance hereof and thereof by the Seller Parties and consummate the transactions herein contemplated and to consummate the transactions consummated hereunder in accordance with their respective terms except as enforcement may be limited by applicable Laws or any court proceeding Each Seller represents and warrants that this Agreement has been duly exccuted and delivered by such Seller Party and further represents to each Buyer and cach other Seller Party hereto and each Seller Party represents and agrees to the Buyer Parties and each Buyers Party hereunder that it has the requisite corporate power and authority enter into and perform its respective obligations under this Agreement Each of the Seller Party acknowledges and agrees that each of the Sellers Parties has duly executed delivered and performed its respective representations and warranties in this Agreement that each Seller Partys respective counterpart signature page to this Agreement is a true and correct copy thereof and that it is duly authorized, valid and binding obligation of each Seller Party thereto and enforceable against each Sellers Party against each other Sellers party hereto such other Seller party thereto according to its respective terms, and that each counterpart of each Seller Parties signature page hereto other than an executed counterpart of an executed copy of counterpart of a Seller Party or Seller Party that is an executed original of such counterpart of Seller Party is duly executed original thereof. The Seller Parties shall have duly executed each of their respective counterpart hereto.
pt

---

DALE-
Each of the Sellers has all requisite private limited liability company power and private limited company authority to execute and deliver this Agreement and the Seller Ancillary Documents, to perform its obligations hereunder and thereunder and to consummate the transactions contemplated hereby and thereby. The execution, delivery and performance of this Agreement by each of the Seller and the consummation by such Seller of such Sellers rights and obligations pursuant thereto and the performance by such Seller of such Sellers right thereto and thereof, and all other matters related to such Seller, that may be required in connection herewith and therewith, including, without limitation, the filing and recording of all material agreements, documents and certificates required to authorize the execution and delivery hereof and thereof by each Seller, and each such Seller shall be entitled to rely the other parties representations and warranties in effect on the date hereof in connection with such Seller and all matters related thereto, including without limitation the right to enforce the provisions Section 8.03(c) and to collect the Obligations in connection therewith and under the other terms hereof, except as enforcement may be limited by applicable bankruptcy, insolvency, reorganization, moratorium or other Laws affecting enforcement of creditors' rights or by general principles of equity (regardless of whether enforvement is sought in a proceeding at law or in equity). To the extent that this Agreement has been duly executed and delivered by each Seller, it is a matter of record and fact that each Seller represents and warrants the other Sellers and its respective officers, directors, employees, agents and agents (collectively, the Sellers Parties) and each Seller acknowledges and agrees to be bound by the foregoing representations, warranties, agreements and statements. To the knowledge each Sellers, (i) such Sellers Party has not made any misrepresentation or omits to have made any representation or warranty which has not been made or omitted to make, and (ii) the Sellers have been duly authorized, executed, delivered and performed its valid and binding obligations with respect thereto and have performed its obligations thereto in accordance with their respective terms. Notwithstanding anything herein to the contrary, in the event of Change in Control (as defined in Section 4.04(c)) (iv) merger, consolidation, exchange other similar transaction, such Seller AnCillary Documents shall be duly executed. X / x
ft

---

DALE
Each of the Sellers has all requisite limited liability company authority to execute and deliver this Agreement and the Seller Ancillary Documents to which it is party, perform its obligations hereunder and thereunder and consummate the transactions contemplated hereby and thereby. The execution, delivery and performance of this Agreement by each of Sellers and each of the other Sellers, and the execution and delivery of the Purchased Loans and the other Purchased Loan Documents and the consummation by such Sellers of the transactions described herein and therein, have been duly authorized by all necessary corporate action on the part of such Seller, and no other action is required to authorize the execution delivery, performance and performance hereof and thereof by such Seller or any of its Subsidiaries of any of their respective properties or assets hereof or thereof or the properties and assets of the parties hereto and thereto and of such other properties, assets and assets as the case may be, except enforcement may be limited by applicable bankruptey, insolvency, reorganization, moratorium, receivership, liquidation. moratorium or other Laws of general upplication affecting creditors' rights generally and by general principles of equity (including, without limitation, any rule of law relating to the availability of equitable remedies in a proceeding at law or in equity) and (i) this Agreement has been duly executed and delivered by each Seller and, assuming for the purposes of this Section 10.11. (a) and (b) other than set forth in Section 12.1. (b), the certificate of each Seller, that (c) such other certificate other evidence may be required be filed with the SEC in connection herewith and (d) as may be necessary to eifectuate the purposes hereof in accordance with its terms, and (e) this Agreement, when so filed and delivered, will constitute the legal, valid and binding obligation of each Seller, enforceable against such Seller and each other Seller, each case, and each such Seller will be entitled to exercise all rights and remedies available to it in connection therewith and therewith, except to the extent such enforcement may. in the case of the provisions hereof, be limited to specific performance injunctive relief. / /
give effect to the terms


Reviews:
[{"review_id": "V1UcLroW4z", "reviewer": "Reviewer_fzBD", "summary": "", "questions": "1.\tHave you experimented with smaller corpora during pre-training? How does performance degrade? What is the minimum corpus size needed?\n2.\tThe gains over baselines vary quite a bit across tasks - smaller for RR and DLI versus others. Is there any analysis into why certain tasks benefit more? \n3.  Did you do any error analysis? What are some common failure cases or errors you observed?\n4.\tThe overfitting issue is noted for very low resources. Can you elaborate on what resource levels (50 or smaller) this tends to happen at?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer fzBD,\n\nWe thank you for the detailed analysis of our paper and the insightful comments. In what follows, we try to address all our comments point-by-point.\n\n### Reasons to Reject:\n\n>1. While the topic is contributory, some dataset selected for experiments are not actual low resource legal NLP task. This paper conducted experiments on 13 datasets and 4 tasks. For tasks like MCC and MLC, datasets such as ILDC and ECtHR already have enough data for training, making them not truly representative of a low-resource Legal NLP scenario. For example, ILDC contains 35,000 cases available for training to 75% F1-score in ILDC dataset paper. There is no need to use a small part of dataset to get a worse performance. The experiment should focus on the actual low-resource legal NLP tasks, such as RR and NLI. In BUILD-RR and ContractNLI dataset, however, the improvement of DALE is relatively modest(1-2%) compared to existing methods. It has impacted the establishment of the conclusions.\n\n**Ans:** We thank you for the feedback. Since this question has two parts, we would like to answer each separately.\n\nFollowing most of prior works [1,2,3,4,5], for all our experiments we simulate low-resource (limited training samples) settings for each dataset regardless of the dataset's original size. Similar to prior-works, we believe that by simulating a low-resource setting on a dataset and showing improvements with high-quality synthetic augmentations while training shows the potential that the future endeavors might only necessitate the annotation of a low-resource dataset for teaching a neural model a comparable task..\n\n\nRR (BUILD-RR) (Malik et al., 2022) and NLI (ContractNLI) (Koreeda and Manning, 2021) are more complex tasks than multi-label and multi-class classification. As also mentioned in our paper in Section 4.3 , we just use the original (and simple) baseline setting for both these datasets proposed by the original authors in the dataset paper and do not resort to more improved learning methods proposed in literature following the original work. Thus, being more complex tasks, the improvements are upper bounded by the capacity of the simple baseline and using DALE augmentations with better learning methods may further improve scores. To confirm this hypothesis, we take two improved learning methods from literature (1 for each task) and compare results for DALE with Gold-only and the second best augmentation baselines from Table 3. For BUILD-RR we adhere to the system proposed by [6] and for ContractNLI we adhere to the system proposed by [7]. The results are as follows:\n\n\n| Gold      | 100          | 200  | 100        | 200        |\n|-----------|-------------|----------|------------|------------|\n| Dataset   |BUILD-RR              ||ContractNLI |            |\n| Gold-only | 76.1        | 78.3     | 75.3       | 84.2       |\n| AEDA      | 79.6        | 84.6     | 79.0       | *86.5*|\n| Genius    | 80.2        | 81.9     | *79.2*| 85.6       |\n| DALE      | **85.3**    | **88.9** | **84.7**   | **89.7**   |\n\nAs we see, using a better learning method improved overall performance for Gold-only by 1.7% and subsequently improved DALE margins by 5.5%-10.6% (earlier 1%-9.7%).\n\n>2.A. The experiment setup for UNFAIR-ToS is the same as for the other dataset in Table 2. Therefore, it should have been included in Table 2 rather than placed in the appendix. It needs more explanations why author chose to put it in appendix. It will raise doubts to readers that maybe the proposed method did not perform well on UNFAIR-ToS.\n\n**Ans:** We thank you for the question.We did not include UnFair-ToS in the main paper due to space limitations. UnFair-ToS has similar to better margins of improvement than most datasets in Table 2. With the new 9-page limit, we will include this result  in the camera-ready version of our paper.\n\n>2.B. The underlined data in Table 2 does not represent the best baseline results . For instance, in ILDC, the performance of Legal-EDA is 60.38 for 500 gold labels, which is higher than the underlined BackTrans (59.18). Similar situations are also observed for Legal-EDA in OTS for 100 gold labels. Legal-EDA serves as an important baseline for comparison. The unclear presentation of experimental results raises doubts about whether the DALE method truly exhibits improvements.\n\n**Ans:** We sincerely apologize for this mistake and thank you for pointing this out. Post a careful analysis we could only find this mistake in only 6 places (out of 32): . This leads to a decrease in margin between the best-performing baseline and DALE by: 0.19% for OTS-TOPICS 100 Gold Labels, 1.06% for EUR-LEX 200 Gold Labels, 0.46% for EUR-LEX 500Gold Labels, 0.05% for ECtHR-B 100 Gold Labels, 1.2% for ILDC 500 Gold Labels, 1.21% for OTS 500 Gold Labels. DALE still improves over all baselines significantly and we believe this does not hurt the efficacy of DALE in diverse settings.\n\n>3.The approach relies large resource for pre-training and then addresses the low resource downstream task. The method itself has some contradictions. The pre-training needs 48G legal data and 7 days on 4 NVIDIA A100 GPUs. For someone doesn’t have such large data and computing resource, it is impossible to use in downstream low resource task.\n\n**Ans:** We thank you for the important question. We would like to answer this question two-fold.\n\n- The broader goal of DALE is to solve the low-resource labeled data problem for specialized domains in NLP with complex and formal language, like the legal domain, and we do not focus on our pre-training corpus size. As  mentioned in our paper in Section 1, in a real-world setting, labeling data using expert annotators is an expensive affair, whereas un-labeled data is freely and openly available (even for the legal domain). This is also in-lines with prior-art on generative data augmentation for NLP [1,2,3,4,5] that often focuses on improving performance in low-resource labeled data scenarios and is generally based on a pre-trained language model, pre-trained on a huge unlabeled corpus. Since everyday natural language is quite different from  language used in legal documents, DALE pre-training can also be seen as just an extra step for effective adaptation of Pre-trained Language Models to the legal domain.\n\n- Pre-training large foundation models often consumes much more resources than used for DALE. DALE pre-training can be seen as a step for effectively adapting a PLM, originally trained on internet-scale everyday natural language data, to the legal domain. We in fact emphasize that the adaptation to the legal domain proved to be effective with only a fraction of the total unlabeled data due to the efficacy of our masking algorithm.\n\nAdditionally, pre-trained DALE can also act as a foundation model for legal NLP and be used for learning other tasks beyond just data augmentation. We leave this exploration for future research.\n\n### Questions For The Authors:\n>1.Have you experimented with smaller corpora during pre-training? How does performance degrade? What is the minimum corpus size needed?\n\n**Ans:** We thank you for the important question. Before large-scale pre-training, we had experimented with a relatively smaller dataset for pre-training. This pre-training corpus includes only ~50% of CaseLaw (cas, 2018) and ~10% of the total Pile of Law (Henderson et al., 2022) documents used in the original paper, amounting to ~1.5M total documents. Upon request, we are providing the results for the same, averaged across all datasets. We will also include these results in the original paper with the extra page allotted to us.\n\n|| w/ low resource | w/ original setting | \n|-----------|-------------|----------  |\n| Gold-only  |    54.8     | 54.8     |\n| DALE       |    **55.2**    | **58.9**     |\n\n\n>2. The gains over baselines vary quite a bit across tasks - smaller for RR and DLI versus others. Is there any analysis into why certain tasks benefit more?\n\n**Ans:** We attribute this to the complexity of the RR (Malik et al., 2022) and DLI (Koreeda496 and Manning, 2021) tasks and the simplicity of the downstream learning algorithm we use from the original papers. A more detailed explanation can be found in the answer to Question which we also accompany with results using stronger downstream learning algorithms. The gains for RR and DLI get on-par with other tasks when a stronger downstream learning algorithm is used for training.\n\n>3. Did you do any error analysis? What are some common failure cases or errors you observed?\n\n**Ans:** We thank you for the questions. Most of the wrong predictions happen due to the limitation of BERT's 1024 tokens. For datasets like ILDC, SCOTUS, EULREX, and ECTHR-A/B, the model tends to predict wrong when the input gets truncated to 1024, and the majority of the information for label prediction lies in the truncated part. \n\n>4. The overfitting issue is noted for very low resources. Can you elaborate on what resource levels (50 or smaller) this tends to happen at?\n\n**Ans:** Thank You for the question. We notice some amount of overfitting (with only fine-tuned DALE and not pre-trained) with anything of 100 gold training samples and below. However, this phenomenon was only seen for datasets like UNFAIR-TOS, OTS, OTS-TOPICS, and LEDGAR, with small single sentences per training instance, and not the datasets like SCOTUS, ILDC, and EURLEX, with multi-sentence documents per training instance. As also mentioned in our paper in Section 4.4, this can be alleviated using pre-trained DALE.\n\n### References  \n\n[1] Feng, Steven Y., Aaron W. Li, and Jesse Hoey. \"Keep calm and switch on! Preserving sentiment and fluency in semantic text exchange.\" arXiv preprint arXiv:1909.00088 (2019).\n\n[2] Kumar, Varun, Ashutosh Choudhary, and Eunah Cho. \"Data augmentation using pre-trained transformer models.\" arXiv preprint arXiv:2003.02245 (2020).\n\n[3] Yu, Adams Wei, et al. \"Qanet: Combining local convolution with global self-attention for reading comprehension.\" arXiv preprint arXiv:1804.09541 (2018).\n\n[4] Karimi, Akbar, Leonardo Rossi, and Andrea Prati. \"AEDA: an easier data augmentation technique for text classification.\" arXiv preprint arXiv:2108.13230 (2021).\n\n[5] Perçin, Sezen, et al. \"Combining WordNet and word embeddings in data augmentation for legal texts.\" Proceedings of the Natural Legal Language Processing Workshop 2022. 2022.\n\n[6] Marino, Gabriele, et al. \"Automatic Rhetorical Roles Classification for Legal Documents using LEGAL-TransformerOverBERT.\" (2023).\n\n[7] Ivgi, Maor, Uri Shaham, and Jonathan Berant. \"Efficient long-text understanding with short-text models.\" Transactions of the Association for Computational Linguistics 11 (2023): 284-299.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tLegal data argumentation is underexplored topic in the legal NLP community. This paper proposes a specialized selective masking and denoising strategy tailored to legal text. This is an important and valuable contribution over simply applying existing augmentation techniques. \n\n2.\tA detailed explanation of the unique masking method was provided, along with comprehensive experiments across multiple datasets and tasks.", "reasons_to_reject": "1. While the topic is contributory, some dataset selected for experiments are not actual low resource legal NLP task. This paper conducted experiments on 13 datasets and 4 tasks. For tasks like MCC and MLC, datasets such as ILDC and ECtHR already have enough data for training, making them not truly representative of a low-resource Legal NLP scenario. For example, ILDC contains 35,000 cases available for training to 75% F1-score in ILDC dataset paper. There is no need to use a small part of dataset to get a worse performance. The experiment should focus on the actual low-resource legal NLP tasks, such as RR and NLI. In BUILD-RR and ContractNLI dataset, however, the improvement of DALE is relatively modest(1-2%) compared to existing methods. It has impacted the establishment of the conclusions.\n\n2.\tSome presentation of experimental results is rather confusing: \n\nA. The experiment setup for UNFAIR-ToS is the same as for the other dataset in Table 2. Therefore, it should have been included in Table 2 rather than placed in the appendix. It needs more explanations why author chose to put it in appendix. It will raise doubts to readers that maybe the proposed method did not perform well on UNFAIR-ToS.\n\nB. The underlined data in Table 2 does not represent the best baseline results . For instance, in ILDC, the performance of Legal-EDA is 60.38 for 500 gold labels, which is higher than the underlined BackTrans (59.18). Similar situations are also observed for Legal-EDA in OTS for 100 gold labels. Legal-EDA serves as an important baseline for comparison. The unclear presentation of experimental results raises doubts about whether the DALE method truly exhibits improvements.\n\n3. The approach relies large resource for pre-training and then addresses the low resource downstream task. The method itself has some contradictions. The pre-training needs 48G legal data and 7 days on 4 NVIDIA A100 GPUs. For someone doesn’t have such large data and computing resource, it is impossible to use in downstream low resource task.", "questions_to_authors": "1.\tHave you experimented with smaller corpora during pre-training? How does performance degrade? What is the minimum corpus size needed?\n2.\tThe gains over baselines vary quite a bit across tasks - smaller for RR and DLI versus others. Is there any analysis into why certain tasks benefit more? \n3.  Did you do any error analysis? What are some common failure cases or errors you observed?\n4.\tThe overfitting issue is noted for very low resources. Can you elaborate on what resource levels (50 or smaller) this tends to happen at?", "paper_topic_and_main_contributions": "This paper presents a framework for generating synthetic data augmentations specifically designed for legal documents. The key problem it aims to address is the lack of sufficient training data in legal domain where annotation is expensive and requires expert knowledge.\nThe main contribution of the paper is introducing a novel pre-training objective based on selective masking of correlated spans in unlabeled legal text to enable generating coherent and diverse augmentations. This method achieves state-of-the-art performance over strong baselines on 6 legal NLP tasks spanning 13 datasets and 4 low-resource settings.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 323: The equation misses a equal sign."}, "extra_text": {}}, {"review_id": "DaoshXrTmT", "reviewer": "Reviewer_9xzW", "summary": "", "questions": "- did you ran experiments on the importance of train test size on performance (i.e. how much augmented and/or over-generated data is included)?\n\n- did you run any tests for statistical significance? did you run multiple experiments and/or measure variance?\n\n- did you perform any manual inspection of the augmented data?\n\n- did you perform any evaluation of hallucinations and/or factuality?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 9xzW,\n\nWe thank you for the detailed analysis of our paper and the insightful comments. In what follows, we try to address all our comments point-by-point.\n\n### Questions For The Authors:\n> 1. did you ran experiments on the importance of train test size on performance (i.e. how much augmented and/or over-generated data is included)?\n\n**Ans:** We thank you for the question. Yes, Appendix B.2 shows results achieved by DALE when various different numbers of augmentation rounds R were used in our experiments and how each affected the final performance of DALE. R=5 shows the best performance according to our study.\n\n>2. did you run any tests for statistical significance? did you run multiple experiments and/or measure variance?\n\n**Ans:** We thank you for the question. Yes, the last line of Section 4.1 says that we report micro-averaged F1 scores averaged across 3 runs for 3 random seeds.\n\n>3. did you perform any manual inspection of the augmented data?\n\n**Ans:** We thank you for the question. Yes we did that and an extensive one! Table 16 in Appendix shows 31 pages of manual expert inspection (by a law student) of augmentations generated using DALE and all other baselines used in our experiments in 3 factors: coherency, plausibility and label-consistency. We show that DALE wins over all baselines in most case\n\n>4. did you perform any evaluation of hallucinations and/or factuality?\n\n**Ans:** We thank you for the question. Yes, we do. Again,  Table 16 in the Appendix shows 31 pages of generation examples where the plausibility of the events and label-consistency of augmentations with the original instance are human evaluation metrics. Both combined can be seen as a measure of factuality for legal language. As discussed in our paper, since adding new context to the training instances is one of the primary goals of DALE, we just want to ensure the plausibility of the new events in the added context. Why? - Instances in legal documents are generally emerging facts or events specific to a particular case, and ensuring factuality in the added context would mean the event actually took place in real life, which we hypothesize is not necessary for augmentations in legal NLP.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Interesting and potentially impactful problem\n\n- well justified and presented research\n\n- solid experimental results demonstrating a successful implementation", "reasons_to_reject": "- due to the scale of research, a lot of important implementation details are left in appendixes\n\n- the paper can benefit from some additional experiments (e.g., measuring the importance of training size, comparing fully augmented data vs gold, etc.)\n\n- no mentions of multiple experiments, variance, and statistical significance in the experimental part", "questions_to_authors": "- did you ran experiments on the importance of train test size on performance (i.e. how much augmented and/or over-generated data is included)?\n\n- did you run any tests for statistical significance? did you run multiple experiments and/or measure variance?\n\n- did you perform any manual inspection of the augmented data?\n\n- did you perform any evaluation of hallucinations and/or factuality?\n", "paper_topic_and_main_contributions": "The paper presents a new model for generating high quality data augmentation for legal documents in low-resource settings. The authors present the problem, the model architecture, the novel masking strategy for training. They also carry out experimental evaluation on the impact that the augmented data has on downstream tasks.\n\nMain contributions:\n- novel algorithm for augmentation\n- novel masking strategy which may be used for other tasks as well\n- experimental evaluation\n\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "On low resource data augmentation: \nhttps://aclanthology.org/2021.acl-long.96/"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- The title of the algorithm DALE has a strong wordplay with the popular DALL-E. I would suggest that the acronym is changed to something more neutral.\n\n- while the paper talks about low-resource, it does not mention that it is focused on English-only until the limitations section. It would be good to explicitly state that either in the abstract or in the introduction\n\n- l89 - missing year in citation"}}, {"review_id": "1AZqw3H2D6", "reviewer": "Reviewer_DwnS", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer DwnS,\n\nWe thank you for the detailed analysis of our paper and the insightful comments. In what follows, we try to address all our comments point-by-point.\n\n### Reasons To Reject:\n>1. The overall clarity in the methodology section somehow breaks down in Section 3.2, where the fine-tuning step is described. If I understand correctly, the authors do not extract correlated spans nor perform optimal context selection here, which might be due to the differences between the downstream and the pre-training corpora. Instead, they assign an importance score to each span by calculating the cosine similarity between the embeddings of a span and the interpolation between the embeddings of the document and its label. Top-p spans are then preserved, and the rest is masked in the resulting template. I was wondering why not use a similar approach to build templates for pre-training corpora (apart from the label information) and whether the step in which the correlated spans are extracted is somehow redundant. The paper would benefit from an empirical comparison of both approaches for template creation applied in the pre-training stage.\n\n**Ans:** We thank you for the important question. There are multiple intuitive reasons why we didn’t use such an approach. We will also include these details in the final version of the paper with the extra page allotted to us. We list down the reasons as follows:\n1. The main objective of correlated span extraction (using our modified formulation) is to mask informative and co-occurring text fragments that usually outline the emerging and case-specific facts and entities (Section 3.1 explains why this is important for the success of DALE). Using the masking process described in Section 3.2 (named importance masking hereof) does not satisfy our needs. Without the label information, the importance masking algorithm will merely retain the “most important” n-gram spans (and mask everything else), where importance is measured with respect to the context of the entire sentence. This leads to two additional problems: </li>\n\n      - Beyond just not explicitly masking co-occurring spans (which we iterate; is important for effective learning), the importance masking algorithm often does the exact opposite and masks case-specific facts, entities, and random spans (as they are deemed non-important by the algorithm). We show two examples below, where we compare the masking algorithms on two pre-training sentences:\n          - Example 1: Did the superior court abuse its discretion in dismissing Morgans appeal for failure to exhaust administrative remedies\n                  \n                  1. DALE Masking: <mask> abuse its discretion <mask> dismissing Morgans appeal <mask> to exhaust administrative <mask>\n                  2. Importance Masking: <mask> the superior court abuse its discretion <mask> to exhaust administrative <mask>\n          - Example 2: The Borrower shall make all payment due hereunder within three (3) Business Days after such payment is due.\n                 \n                  1. DALE Masking: <mask> payment due <mask> payment is due <mask>\n                  2. Importance Masking: The Borrower shall make all payment due hereunder <mask>\n      - As we clearly see, denoising using DALE masks exactly replicates how a legal practitioner would gain knowledge about legal concepts, principles and language usage (powered by co-occurring and principled span masking). On the other hand, importance masking masks random spans that hurt learning. However, with label information, importance masking works well for our purpose and retains spans most informative of the instance label (important for maintaining label consistency in generations).\n         - The quality of the spans retained also largely depends on the encoder used for similarity scoring. Additionally, our DALE pre-training masking algorithm is a principled masking algorithm asking the model to recreate and learn a similar nature of knowledge across the corpus. For importance masking, the high variability in the nature of words or phrases masked breaks this principality, thus reducing the effectiveness. In the final version of our paper, we will also include a comparison of pre-training on the two algorithms on a smaller corpus to show the effectiveness of our proposed algorithm.\n\n3. Label information is a key ingredient to importance masking and is ineffective without it. The importance masking algorithm is designed with the intuition that retaining the “most important” n-gram spans with label information will lead to augmentations that maintain label consistency. Maintaining label consistency (i.e., the augmentations should be of the same label as the source sentence) is key to any data augmentation algorithm. Without label information, the importance of each span will be measured only with the help of the document context, which will capture non-informative spans. Also, legal documents are generally long, and different parts of the document play different roles (Malik et al., 2022) . Without a label, using just these documents for importance scoring leads to ambiguity in the selected spans for masking.\n\n>2. Moreover, in Appendix B, the authors list the hyperparameters of their method and mention the parameter R (\"augmentation rounds\"). However, the augmentation rounds are not mentioned in the paper's main content, and it is unclear what this parameter refers to.\n\n**Ans:** We sincerely apologize for this mistake. R, or the number of augmentation rounds, is the total number of synthetic augmentations that were generated, measured by the number of times of the original dataset. For example, R = 5 would mean that the total number of augmentations generated was 5 times the original dataset size (for eg. for an original training split of 100, 500 synthetic augmentations were generated). Thus the final training size was 6 times the original dataset.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper studies an important problem of generating synthetic data for fine-tuning neural network models in low-resource, legal NLP scenarios. Legal language is characterized by specialized vocabulary, complex syntax, semantics, and morphology. Therefore, legal NLP systems often do not benefit from additional training data generated by employing general-purpose paraphrasing approaches. \n\nThe paper is well-motivated and generally well-written. The authors clearly describe the proposed approach, apart from some minor clarity issues detailed in the \"reasons to reject\" section. The empirical results are extensive and cover six legal language understanding tasks and several baselines. Moreover, the results confirm the utility of the presented method in the low-resource legal NLP scenario. Specifically, DALE was shown to outperform other baselines in all examined scenarios. Furthermore, additional ablation experiments are presented in the appendix, along with the values of hyperparameters and several qualitative examples obtained by the proposed method, which should facilitate the reproducibility of the presented results.\n", "reasons_to_reject": "The overall clarity in the methodology section somehow breaks down in Section 3.2, where the fine-tuning step is described. If I understand correctly, the authors do not extract correlated spans nor perform optimal context selection here, which might be due to the differences between the downstream and the pre-training corpora. Instead, they assign an importance score to each span by calculating the cosine similarity between the embeddings of a span and the interpolation between the embeddings of the document and its label. Top-p spans are then preserved, and the rest is masked in the resulting template. I was wondering why not use a similar approach to build templates for pre-training corpora (apart from the label information) and whether the step in which the correlated spans are extracted is somehow redundant. The paper would benefit from an empirical comparison of both approaches for template creation applied in the pre-training stage.\n\nMoreover, in Appendix B, the authors list the hyperparameters of their method and mention the parameter R (\"augmentation rounds\"). However, the augmentation rounds are not mentioned in the paper's main content, and it is unclear what this parameter refers to.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors of this paper present DALE, a generative data augmentation method in low-resource legal NLP settings. DALE employs an encoder-decoder language model (BART; Lewis et al., 2019) pre-trained on a large unlabeled legal corpus using the proposed denoising objectives based on the selective masking technique, which masks co-occurring and highly correlated text spans instead of random words. It allows the method to preserve information about emerging entities and facts in the text. Subsequently, masked documents, called templates, are used as the input to pre-train the model using the denoising objective. A pre-trained model is then fine-tuned on a downstream data set and used to perform conditional text generation. \n\nThe empirical evaluation of the effectiveness of the proposed method was conducted on 13 data sets spanning six tasks and four low-resource settings. As a result, the models trained on the data augmented with the proposed approach outperform all baseline systems in all examined scenarios. The examples generated with the DALE method are also more diverse and coherent than the data synthesized with the methods from prior works.\n\nIn sum, the contributions of this work are as follows:\n1. A novel generative data augmentation method DALE designed for the legal language understanding scenario.\n2. Strong empirical evaluation results that confirm the utility of the proposed method.\n3. Ablative experiments and qualitative evaluation of the proposed method.\n\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper,  the authors present a framework named DALE for generating synthetic data augmentations specifically designed for low-resource legal NLP text tasks settings. The model has designed an encoder-decoder language model pre-trained on the large-scale unlabeled legal corpus. The main contributions of this paper are proposing denoising objectives based on the selective masking technique, which masks co-occurring and highly correlated text spans instead of random words. The experiments show the state-of-the-art performance over strong baselines on 6 legal NLP tasks and 4 low-resource settings. However, the modeling process, experiments' settings and results are confusing the reviewers such as the fine-tuning step and multiple experiments, variance, and statistical significance.
Decision: Accept-Main