File size: 3,179 Bytes
1bd0fd9 e385747 fcee881 59a8cec 2206f48 1bd0fd9 3c74d65 1bd0fd9 c63c45a 59a8cec c63c45a 3c74d65 1bd0fd9 c63c45a 3c74d65 c63c45a 1bd0fd9 c63c45a 1bd0fd9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
---
license: mit
language:
- pt
---
# BERTikal (aka `legalnlp-bert`)
BERTikal [1] is a cased BERT-base model for the Brazilian legal language and was trained from the BERTimbau's [2] checkpoint using Brazilian legal texts. More details on the datasets and training procedures can be found in [1].
**Please check Legal-NLP** out for more resources on (PT-BR) legal natural language processing (https://github.com/felipemaiapolo/legalnlp).
**Please cite as** Polo, Felipe Maia, et al. "LegalNLP-Natural Language Processing methods for the Brazilian Legal Language." Anais do XVIII Encontro Nacional de Inteligência Artificial e Computacional. SBC, 2021.
@inproceedings{polo2021legalnlp,
title={LegalNLP-Natural Language Processing methods for the Brazilian Legal Language},
author={Polo, Felipe Maia and Mendon{\c{c}}a, Gabriel Caiaffa Floriano and Parreira, Kau{\^e} Capellato J and Gianvechio, Lucka and Cordeiro, Peterson and Ferreira, Jonathan Batista and de Lima, Leticia Maria Paz and do Amaral Maia, Ant{\^o}nio Carlos and Vicente, Renato},
booktitle={Anais do XVIII Encontro Nacional de Intelig{\^e}ncia Artificial e Computacional},
pages={763--774},
year={2021},
organization={SBC}
}
## Usage
### Ex. Loading model for general use
```python
from transformers import AutoTokenizer # Or BertTokenizer
from transformers import AutoModelForPreTraining # Or BertForPreTraining for loading pretraining heads
from transformers import AutoModel # or BertModel, for BERT without pretraining heads
model = AutoModelForPreTraining.from_pretrained('felipemaiapolo/legalnlp-bert')
tokenizer = AutoTokenizer.from_pretrained('felipemaiapolo/legalnlp-bert', do_lower_case=False)
```
### Ex. BERT embeddings
```python
from transformers import pipeline
pipe = pipeline("feature-extraction", model='felipemaiapolo/legalnlp-bert')
encoded_sentence = pipe('Juíz negou o recurso.')
```
### Ex. Masked language modeling prediction
```python
from transformers import pipeline
pipe = pipeline('fill-mask', model='felipemaiapolo/legalnlp-bert')
pipe('Juíz negou o [MASK].')
# [{'score': 0.6387444734573364,
# 'token': 7608,
# 'token_str': 'julgamento',
# 'sequence': 'juiz negou o julgamento.'},
# {'score': 0.09632532298564911,
# 'token': 7509,
# 'token_str': 'voto',
# 'sequence': 'juiz negou o voto.'},
# {'score': 0.06424401700496674,
# 'token': 17225,
# 'token_str': 'julgado',
# 'sequence': 'juiz negou o julgado.'},
# {'score': 0.05929475650191307,
# 'token': 8190,
# 'token_str': 'recurso',
# 'sequence': 'juiz negou o recurso.'},
# {'score': 0.011442390270531178,
# 'token': 6330,
# 'token_str': 'registro',
# 'sequence': 'juiz negou o registro.'}]
```
# References
[1] Polo, Felipe Maia, et al. "LegalNLP-Natural Language Processing methods for the Brazilian Legal Language." Anais do XVIII Encontro Nacional de Inteligência Artificial e Computacional. SBC, 2021.
[2] Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: pretrained BERT
models for Brazilian Portuguese. In 9th Brazilian Conference on Intelligent
Systems, BRACIS, Rio Grande do Sul, Brazil, October 20-23
|