metadata

language:
  - ms
tags:
  - paraphrase
metrics:
  - sacrebleu

finetune-paraphrase-t5-tiny-standard-bahasa-cased

Finetuned T5 tiny on MS paraphrase tasks.

Dataset

translated PAWS, https://huggingface.co/datasets/mesolitica/translated-PAWS
translated MRPC, https://huggingface.co/datasets/mesolitica/translated-MRPC

Finetune details

Finetune using single RTX 3090 Ti.

Scripts at https://github.com/huseinzol05/malaya/tree/master/session/paraphrase/hf-t5

Supported prefix

parafrasa: {string}, for MS paraphrase.

Evaluation

Evaluated on MRPC validation set and PAWS test set.

{'name': 'BLEU',
 'score': 61.06784273649806,
 '_mean': -1.0,
 '_ci': -1.0,
 '_verbose': '86.1/68.4/55.8/45.9 (BP = 0.980 ratio = 0.980 hyp_len = 138209 ref_len = 141004)',
 'bp': 0.9799801176769202,
 'counts': [119035, 89737, 69210, 53653],
 'totals': [138209, 131135, 124061, 116987],
 'sys_len': 138209,
 'ref_len': 141004,
 'precisions': [86.1268079502782,
  68.4310062149693,
  55.787072488533866,
  45.86236077512886],
 'prec_str': '86.1/68.4/55.8/45.9',
 'ratio': 0.9801778672945448}