huseinzol05's picture
Create README.md
9f0f5a5
---
language:
- ms
tags:
- paraphrase
metrics:
- sacrebleu
---
# finetune-paraphrase-t5-tiny-standard-bahasa-cased
Finetuned T5 tiny on MS paraphrase tasks.
## Dataset
1. translated PAWS, https://huggingface.co/datasets/mesolitica/translated-PAWS
2. translated MRPC, https://huggingface.co/datasets/mesolitica/translated-MRPC
## Finetune details
1. Finetune using single RTX 3090 Ti.
Scripts at https://github.com/huseinzol05/malaya/tree/master/session/paraphrase/hf-t5
## Supported prefix
1. `parafrasa: {string}`, for MS paraphrase.
## Evaluation
Evaluated on MRPC validation set and PAWS test set.
```
{'name': 'BLEU',
'score': 61.06784273649806,
'_mean': -1.0,
'_ci': -1.0,
'_verbose': '86.1/68.4/55.8/45.9 (BP = 0.980 ratio = 0.980 hyp_len = 138209 ref_len = 141004)',
'bp': 0.9799801176769202,
'counts': [119035, 89737, 69210, 53653],
'totals': [138209, 131135, 124061, 116987],
'sys_len': 138209,
'ref_len': 141004,
'precisions': [86.1268079502782,
68.4310062149693,
55.787072488533866,
45.86236077512886],
'prec_str': '86.1/68.4/55.8/45.9',
'ratio': 0.9801778672945448}
```