aloisiusedwin/t5-id-summarization-finetuned

T5 Indonesian Summarization (Fine-tuned Original)

Model T5-base yang di-fine-tune untuk meringkas percakapan Bahasa Indonesia. Training menggunakan 5-Fold Cross Validation tanpa augmentasi data.

Model Details

• Base Model: cahya/t5-base-indonesian-summarization-cased
• Architecture: T5-base (encoder-decoder, 12 layers, 768 hidden, 12 heads)
• Parameters: ~220M
• Language: Indonesian (Bahasa Indonesia)
• Task: Abstractive Summarization of Indonesian Conversations
• Training: 5-Fold Cross Validation
• Available Folds: 5 folds tersedia sebagai branches (fold_0 s/d fold_4). Branch main berisi fold 3 (performa terbaik).

Usage

from transformers import T5Tokenizer, T5ForConditionalGeneration

# Load model & tokenizer
model_name = "aloisiusedwin/t5-id-summarization-finetuned"
tokenizer = T5Tokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)

# Contoh percakapan
conversation = "summarize: S1: Halo, gimana kabarmu? S2: Baik, aku lagi sibuk ngerjain tugas nih."

# Generate ringkasan
inputs = tokenizer(conversation, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(
    inputs["input_ids"],
    max_length=150,
    num_beams=1,
    no_repeat_ngram_size=2,
    early_stopping=True
)
summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(summary)

Loading Specific Fold

# Load fold tertentu (misal fold_0)
model = T5ForConditionalGeneration.from_pretrained(model_name, revision="fold_0")
tokenizer = T5Tokenizer.from_pretrained(model_name, revision="fold_0")

Training Details

Hyperparameters

Parameter	Value
Learning Rate	5e-5
Batch Size	8
Epochs	10
Early Stopping Patience	2
Weight Decay	0.01
Label Smoothing	0.1
LR Scheduler	Cosine
Warmup Ratio	0.10
Max Grad Norm	1.0
Max Input Length	512
Max Target Length	128
FP16	True

Data Augmentation

Tidak ada augmentasi. Dataset original saja.

Evaluation Results

Per-Fold Results

Fold	ROUGE-1	ROUGE-2	ROUGE-L	BERTScore F1	Eval Loss
0	26.73	9.22	23.60	0.7308	4.7032
1	29.30	10.02	24.69	0.7339	4.6203
2	24.74	7.28	21.65	0.7174	4.7090
3 (best)	28.20	10.72	25.32	0.7287	4.5620
4	27.55	7.76	24.20	0.7330	4.6594

(best) = Fold terbaik (digunakan sebagai branch main)

Aggregated (5-Fold Cross Validation)

Metric	Mean	Std
ROUGE-1	27.31	1.53
ROUGE-2	9.00	1.31
ROUGE-L	23.89	1.26
BERTScore F1	0.7288	0.0060

Perbandingan dengan Baseline

Model	ROUGE-1	ROUGE-2	ROUGE-L	BERTScore F1
Baseline (pretrained)	15.92	4.40	13.12	0.6626
T5 Indonesian Summarization (Fine-tuned Original)	27.31	9.00	23.89	0.7288

Intended Use

Model ini dirancang untuk meringkas percakapan dalam Bahasa Indonesia.

Limitations

• Input harus diawali dengan prefix summarize: untuk hasil optimal.
• Panjang input maksimum 512 token.

Citation

@thesis{edwin2026summarization,
  title={Pengaruh Augmentasi Data terhadap Kualitas Ringkasan Percakapan Bahasa Indonesia menggunakan T5},
  author={Aloisius Edwin},
  year={2026},
  school={Institut Teknologi Sumatera}
}