Labor Lex: A New P ortuguese Corpus and Pipeline for Information Extraction in B razilian Legal Texts (2025)

Abstract: Relation Extraction (RE) is a challenging Natural Language Processing task that involves identifying named entities from text and classifying the relationships between them. When applied to a specific domain, the task acquires a new layer of complexity, handling the lexicon and context particular to the domain in question. In this work, this task is applied to the Legal domain, specifically targeting Brazilian Labor Law. Architectures based on Deep Learning, with word representations derived from Transformer Language Models (LM), have shown state-of-the-art performance for the RE task. Recent works on this task handle Named Entity Recognition (NER) and RE either as a single joint model or as a pipelined approach. In this work, we introduce Labor Lex, a newly constructed corpus based on public documents from Brazilian Labor Courts. We also present a pipeline of models trained on it. Different experiments are conducted for each task, comparing supervised training using LMs and In-Context Learning (ICL) with Large Language Models (LLM), and verifying and analyzing the results for each one. For the NER task, the best achieved result was 89.97% F1-Score, and for the RE task, the best result was 82.38% F1-Score. The best results for both tasks were obtained using the supervised training approach.

## introduction information extraction (ie) is a field of natural language processing (nlp) that involves a range of tasks aimed at structuring unstructured textual information, thereby facilitating the categorization of such information @xcite . among these tasks, two can be considered fundamental to this objective: named entity recognition (ner) @xcite and relation extraction (re) @xcite . ner aims to identify and classify proper nouns in textual content @xcite , while re is responsible for classifying the relationship between a pair of entities. ie tasks, such as ner and re, are essential for building knowledge bases and graphs @xcite , which support and provide inputs for the development of other nlp applications, such as semantic search, summarization, and questionanswering @xcite . formally, the task is defined as a relation classification over entity pairs, represented as the triplet (subject, relation, object), where the subject and the object are two entities that share a rela-tion between them. figure 1 shows the identification of entities, followed by the classification of the relations between them, performed by ner and re models. in sentence s1 there are two types of relations between the entities: grant corresponds to the decision of the judge (decision) to accept the unhealthy work conditions allowance claim (claim) made by the plaintiff, while r$2,230.23 is the amount (claim_value) that the defendant must pay for such a claim. for entity pairs within a sentence window that do not exhibit any relation, the label assigned is no_relation, as in relation r8 between compensation for moral damages and grant. entity relations can occur across different scopes, depending on the location of the entities involved: (i) the entities can be present in the same sentence (intra-sentence); (ii) in different contiguous sentences (inter-sentence); or (iii) across all sentences in an entire document. in figure 1 , r1 and r2 are examples of intra-sentence relations, as both entities in each relation are part of the same sentence s1. r5 is an example of an inter-sentence relation, as the subject of the relation (termination payments) is in sentence s3, and the corresponding decision object (denied) is in sentence s4. this work contemplates both intra and inter-sentence relations. this research addresses the complexities of re, a challenging task, particularly within the legal domain. the project is motivated by a significant gap in resources and benchmarks for the portuguese language compared to english, which limits the progress of re in this domain. this domainspecific focus is crucial because general-purpose datasets are not suited for legal-specific information, and the resulting structured data provides valuable insights for jurimetrics 1 (jaeger @xcite . we propose a pipeline-based approach for relation extraction, training an ner and re model for each task using the labor lex dataset built for the portuguese legal domain. for the ner task, we evaluate the fine-tuning of different transformer base models, also evaluating linear and crf (conditional random field) @xcite classifiers. for re, we experiment with the pure architecture (zhong and chen, 2021) for a specific domain in the portuguese language. we also conducted experiments on a joint, end-to-end approach for both tasks, utilizing in-context learning (icl) @xcite with llms and comparing the results obtained with the fine-tuned models. from now on, we refer to this approach as llm icl. the remainder of the paper is organized as follows: section 2 introduces our corpus annotation. section 3 describes the related work. section 4 presents our proposed model to train on our data. section 5 describes the experimental evaluation. and finally, section 6 concludes the paper. ## labor lex corpus when initiating a labor lawsuit, the claimant must submit a document known as initial petition through their representing attorney. the petition must enumerate the claims and the legal substantiation for each, also identifying the respondent who allegedly breached the labor contract. according to brazilian labor law, the petition must specify the amount claimed 3 for each claim. the sum of all claim values is called the case value. when deciding, the judge may grant or deny each claim made by the claimant. similarly, even for claims that are granted, the amount awarded may not correspond to the amount claimed. the sum of the amounts granted by the judge for each claim is referred to as the conviction value. at any point in the lawsuit, there can be a settlement between the parties, specifying an agreement that the judge must sanction. the amount agreed upon between the parties, to be paid by the respondent to the claimant, is called the settlement value. there is also the legal costs value representing the total expenses incurred during a lawsuit, owed to the judiciary for providing public services. ## related work recent works on re have evolved from traditional neural architectures to transformer-based models @xcite , with advances in both pre-training and task-specific adaptation. ernie @xcite incorporated structured knowledge into masked language modeling (mlm) pretraining, spanbert (joshi et al., 2020) used spanlevel objectives, and luke @xcite employed entity-aware attention. some works used contrastive learning and masking of entity pairs, such as mtb @xcite . erica @xcite also uses contrastive learning, but with an objective that focuses on entity and relation discrimination. enhancing entity representation through special markers has proven effective for highlighting entity boundaries, their categories, and roles in the participating relations @xcite @xcite . yan et al. (2023) do something similar but use an architecture based on graph neural networks (gnn). joint architectures such as spert (eberts and ulges, 2019), tplinker (wang et al., 2020), and atlop (zhou et al., 2021) combine entity and contextual embeddings for classification. graph-based methods like dygie (luan et al., 2019) and dygie++ (wadden et al., 2019) model entities and relations as graph structures. other works leverage gnn architectures, such as graph convolutional network (gcn) @xcite and attentive graph convolutional network (a-gcn) @xcite , while pl-marker @xcite explores various graph topologies. more recently, the graph language model (glm) by @xcite adapts t5 @xcite with graph biases to enable joint reasoning over text and graphs. prompt-based fine-tuning reformulates re as a masked prediction task, aligning it with pretraining objectives @xcite @xcite . more recent work leverages llms for zero-and few-shot re through in-context learning (icl) @xcite @xcite . the rag4re framework by efeoglu and paschke (2025) extends this by integrating retrieved external knowledge, a key distinction from earlier methods. for the legal domain, prior work includes a hybrid crf and rule-based approach for french ner and re (andrew, 2018) and a legal triplet extraction system for chinese @xcite . a recent study by deußer et al. (2024) applied icl to seven diverse legal datasets across multiple languages using eleven state-of-the-art llms. in the brazilian context, research has focused on domain adaptation for various tasks. @xcite adapted word embeddings to classify the status of legal proceedings. more recently, @xcite introduced portulex, a benchmark with four datasets for ner and rhetorical role identification. they also performed domain adaptation of roberta @xcite and evaluated it on this benchmark. in terms of the available general domain portuguese re corpora, the only benchmark identified was rerelem (relations recognition between en-tity mentions) @xcite , and few works were found using it @xcite @xcite . collovini et al. (2020), reyes et al. (2021), pavanelli (2022), and da silva et al. (2023) developed domain-specific corpora targeting commercial and medical applications. there are tasks and evaluations such as those proposed in @xcite , but since the gold-standard evaluation corpus was not released after the conference, it was not possible to establish a new benchmark as a reference for future work. to the best of our knowledge, no prior work addresses relation extraction in the portuguese legal domain. ## model we adopt a pipeline where a ner model first identifies entity spans and types, and its outputs are then fed into a re model. this design is inspired by the pure framework (zhong and chen, 2021), which demonstrated that well-engineered pipeline models can achieve performance comparable to, or even exceeding, that of complex joint models. figure 2 illustrates the design of our framework. while the original pure system employs a spanbased ner model as its first stage, our implementation experiments with a transformer-based encoder combined with two alternative classification layers: a conditional random field (crf) @xcite @xcite and a linear neural layer. the transformer encoder provides rich contextual representations of the input sequence, while the crf layer offers sequencelevel decoding that models label dependencies and enforces valid tag sequences. in contrast, the linear layer performs independent token-level classification, offering a simpler and faster alternative. for the re stage, we retain the core design principles from pure, embedding entity spans with special position markers, contextualized via a transformer encoder, and then classifying them into relation types using the concatenated span representations. this architecture allows the re model to focus solely on the semantics and context of the provided entities, while enabling a controlled comparison of ner classification strategies. ## experimental evaluation for our experiments, we trained two models for the ner and re subtasks. besides what was described as the proposed model in the previous section, we have also experimented with llm icl, providing llms with few-shot examples to evaluate their performance on labor lex. details of the experiments conducted with each approach are presented in the following subsections. ## conclusions in this paper, we introduced and experimented with labor lex, a newly created legal dataset for the ner and re tasks, evaluating different transformer-based language models, as well as two classifiers for ner, and verifying the best results for the created benchmark with the chosen architecture. the best lm for the ner task was the bert model from @xcite , and the best classifier layer was conditional random fields @xcite , reinforcing the results obtained by @xcite ; ma and hovy (2016); corro et al. (2025), which demonstrated the performance gain from using crf as the classifier for sequential word classification tasks. for the re task, the best evaluated lm was robertalexpt from garcia et al. (2024), using the pure framework from zhong and chen (2021 ). an analysis of the best supervised models indicates that the ner subtask is strong overall -claim (92.46% f1) and claim_value (93.65%) -but exhibits lower performance for decision (87.98%) and repercussion (91.27%). regarding the performance of the lms, having bert as the best performing model for the ner task shows some dissonance compared to the results obtained by @xcite and @xcite . while liu shows improvements by using roberta compared to @xcite , garcia demonstrates that the legal domain robertalexpt model has achieved superior performance in benchmarks within the same domain. the ner experiments in this work show that the best results obtained with the crf classifier were achieved using the bert model, which outperformed both roberta models from the general and legal domains. a possible motivation for these results is that the hyperparameter space of the ner task for the roberta models may differ from that of bert when using the crf classifier. for the re task, the best-performing model was the roberta domain-specific model; however, the bert-based model still outperformed the general domain robertacrawlpt. the experiments for this task provided the best model for the labor legal domain from brazil, with an f1-score of 82.38% in the created benchmark. our experiments show that supervised training still outperforms few-shot in-context learning with llms for information extraction tasks, as shown in for future work, we plan to extend our experiments on the proposed benchmark by exploring architectural and methodological variations discussed in section 3, with the goal of further improving task performance. specifically, we intend to investigate end-to-end joint models for ner and re, graph-based approaches, and prompt-based finetuning strategies. additionally, we will experiment with data augmentation techniques using llms to mitigate data scarcity and enhance model generalization. sandra collovini, patricia nunes gonçalves, guilherme cavalheiro, joaquim santos, and renata vieira. 2020. relation extraction for competitive intelligence. in computational processing of the portuguese language, pages 249-258, cham. springer international publishing. sandra collovini, joaquim francisco santos neto, bernardo scapini consoli, juliano terra, renata vieira, paulo quaresma, marlo souza, daniela barreiro claro, and rafael glauber. 2019. iberlef 2019 portuguese named entity recognition and relation extraction tasks. in iberlef@ sepln, pages 390-410. fernando a. correia, alexandre a.a. almeida, josé luiz nunes, kaline g. santos, ivar a. hartmann, felipe a. silva, and hélio lopes. 2022. finegrained legal entity annotation: a case study on the brazilian supreme court. information processing & management, 59(1):102794. caio corro, mathieu lacroix, and joseph le roux. 2025. bregman conditional random fields: sequence labeling with parallelizable inference algorithms. in proceedings of the 63rd annual meeting of the association for computational linguistics (volume 1: long papers), pages 29557-29574, vienna, austria. association for computational linguistics. diego pinheiro da silva, william da rosa fröhlich, blanda helena de mello, renata vieira, and sandro josé rigo. 2023. exploring named entity recognition and relation extraction for ontology and medical records integration. informatics in medicine unlocked, 43:101381. pedro vitor quinta de castro. 2019. aprendizagem profunda para reconhecimento de entidades nomeadas em domínio jurídico. available in http://repositorio.bc.ufg.br/tede/ handle/tede/10276 . deepseek-ai. 2024. deepseek-v3 technical report. preprint, arxiv:2412.19437. tobias deußer, cong zhao, lorenz sparrenberg, daniel uedelhoven, armin berger, maren pielka, lars hillebrand, christian bauckhage, and rafet sifa. 2024. a comparative study of large language models for named entity recognition in the legal domain. in 2024 ieee international conference on big data (bigdata), pages 4737-4742. jacob devlin, ming-wei chang, kenton lee, and kristina toutanova. 2019. bert: pre-training of deep bidirectional transformers for language understanding. in proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171-4186, minneapolis, minnesota. association for computational linguistics. qingxiu dong, lei li, damai dai, ce zheng, jingyuan ma, rui li, heming xia, jingjing xu, zhiyong wu, baobao chang, xu sun, lei li, and zhifang sui. 2024. a survey on in-context learning. in proceedings of the 2024 conference on empirical methods in natural language processing, pages 1107-1128, miami, florida, usa. association for computational linguistics. markus eberts and adrian ulges. 2019. span-based joint entity and relation extraction with transformer pre-training. in european conference on artificial intelligence. sefika efeoglu and adrian paschke. 2025. fine-tuning large language models for relation extraction within a retrieval-augmented generation framework. in proceedings of the 1st joint workshop on large language models and structure modeling (xllm 2025), pages 1-7, vienna, austria. association for computational linguistics. cláudia freitas, diana santos, hugo gonçalo oliveira, paula carvalho, and cristina mota. 2008. relaçoes semânticas do rerelem: além das entidades no segundo harem. quot; in cristina mota; diana santos (ed) desafios na avaliação conjunta do reconhecimento de entidades mencionadas: o segundo harem linguateca 2008. eduardo a. s. garcia, nadia f. f. silva, felipe siqueira, hidelberg o. albuquerque, juliana r. s. gomes, ellen souza, and eliomar a. lima. 2024. robertalexpt: a legal roberta model pretrained with deduplication for portuguese. in proceedings of the 16th international conference on computational processing of portuguese -vol. 1, pages 374-383, santiago de compostela, galicia/spain. association for computational lingustics. google. 2024. introducing gemini 2.0: our new ai model for the agentic era. model announcement (google). introduced as a fast, multimodal model with enhanced reasoning, long context support, and integrated tool usage. aaron grattafiori, abhimanyu dubey, abhinav jauhri, abhinav pandey, abhishek kadian, ahmad al-dahle, aiesha letman, akhil mathur, alan schelten, alex vaughan, amy yang, angela fan, anirudh goyal, anthony hartshorn, aobo yang, archi mitra, archie sravankumar, artem korenev, arthur hinsvark, and 542 others. 2024. the llama 3 herd of models. preprint, arxiv:2407.21783. yi yao huang and william yang wang. 2017. deep residual learning for weakly-supervised relation extraction. in proceedings of the 2017 conference on empirical methods in natural language processing, pages 1803-1807, copenhagen, denmark. association for computational linguistics. filipe jaeger zabala and fabiano feijó silveira. 2014. jurimetria: estatística aplicada ao direito. revista direito e liberdade, 16(1):87-103. mandar joshi, danqi chen, yinhan liu, daniel s. weld, luke zettlemoyer, and omer levy. 2020. span-bert: improving pre-training by representing and zexuan zhong and danqi chen. 2021. a frustratingly easy approach for entity and relation extraction. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 50-61, online. association for computational linguistics. wenxuan zhou, kevin huang, tengyu ma, and jing huang. 2021. document-level relation extraction with adaptive thresholding and localized context pooling. in proceedings of the aaai conference on artificial intelligence.