metadata

language:
  - ms
tags:
  - paraphrase
metrics:
  - sacrebleu

finetune-paraphrase-t5-small-standard-bahasa-cased

Finetuned T5 small on MS paraphrase tasks.

Dataset

translated PAWS, https://huggingface.co/datasets/mesolitica/translated-PAWS
translated MRPC, https://huggingface.co/datasets/mesolitica/translated-MRPC

Finetune details

Finetune using single RTX 3090 Ti.

Scripts at https://github.com/huseinzol05/malaya/tree/master/session/paraphrase/hf-t5

Supported prefix

parafrasa: {string}, for MS paraphrase.

Evaluation

Evaluated on MRPC validation set and PAWS test set.

{'name': 'BLEU',
 'score': 61.559202822392486,
 '_mean': -1.0,
 '_ci': -1.0,
 '_verbose': '86.0/68.4/56.1/46.3 (BP = 0.984 ratio = 0.984 hyp_len = 138806 ref_len = 141004)',
 'bp': 0.9842896648933511,
 'counts': [119394, 90149, 69905, 54500],
 'totals': [138806, 131732, 124658, 117584],
 'sys_len': 138806,
 'ref_len': 141004,
 'precisions': [86.0150137602121,
  68.43363799228737,
  56.07742784257729,
  46.34984351612464],
 'prec_str': '86.0/68.4/56.1/46.3',
 'ratio': 0.9844117897364614}