|
--- |
|
license: apache-2.0 |
|
language: |
|
- pt |
|
widget: |
|
- text: "A pressão arterial está [MASK], indicando possível hipertensão." |
|
- text: "O paciente recebeu [MASK] do hospital." |
|
- text: "O médico receitou a medicação para controlar a [MASK]." |
|
- text: "O paciente apresenta batimentos cardíacos irregulares, sugerindo [MASK]." |
|
--- |
|
# CardioBERTpt - Portuguese Transformer-based Models for Clinical Language Representation in Cardiology |
|
|
|
This model card describes CardioBERTpt, a clinical model trained on the cardiology domain for NER tasks in Portuguese. This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on a cardiology text dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.4495 |
|
- Accuracy: 0.8864 |
|
|
|
## How to use the model |
|
|
|
Load the model via the transformers library: |
|
``` |
|
from transformers import AutoTokenizer, AutoModel |
|
tokenizer = AutoTokenizer.from_pretrained("pucpr-br/cardiobertpt") |
|
model = AutoModel.from_pretrained("pucpr-br/cardiobertpt") |
|
``` |
|
|
|
## Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 1e-05 |
|
- train_batch_size: 4 |
|
- eval_batch_size: 4 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 15.0 |
|
|
|
## Framework versions |
|
|
|
- Transformers 4.17.0.dev0 |
|
- Pytorch 1.8.0 |
|
- Datasets 1.18.3 |
|
- Tokenizers 0.11.0 |
|
|
|
## More Information |
|
|
|
Refer to the original paper, [CardioBERTpt - Portuguese Transformer-based Models for Clinical Language Representation in Cardiology](https://ieeexplore.ieee.org/document/10178779/) for additional details and performance on Portuguese NER tasks. |
|
|
|
## Acknowledgements |
|
|
|
This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001, and by Foxconn Brazil and Zerbini Foundation as part of the research project Machine Learning in Cardiovascular Medicine. |
|
|
|
## Citation |
|
|
|
``` |
|
@INPROCEEDINGS{10178779, |
|
author={Schneider, Elisa Terumi Rubel and Gumiel, Yohan Bonescki and de Souza, João Vitor Andrioli and Mie Mukai, Lilian and Emanuel Silva e Oliveira, Lucas and de Sa Rebelo, Marina and Antonio Gutierrez, Marco and Eduardo Krieger, Jose and Teodoro, Douglas and Moro, Claudia and Paraiso, Emerson Cabrera}, |
|
booktitle={2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS)}, |
|
title={CardioBERTpt: Transformer-based Models for Cardiology Language Representation in Portuguese}, |
|
year={2023}, |
|
volume={}, |
|
number={}, |
|
pages={378-381}, |
|
doi={10.1109/CBMS58004.2023.00247}} |
|
} |
|
``` |
|
|
|
## Questions? |
|
|
|
Post a Github issue on the [CardioBERTpt repo](https://github.com/HAILab-PUCPR/CardioBERTpt). |
