kckrish21/SciHigh2026-Task2-PEGASUS-XSum

SciHigh 2026 Task 2 — PEGASUS-XSum Title Generation Model

This repository contains a fine-tuned PEGASUS-XSum model for Task 2: Title Generation from Abstracts of the SciHigh 2026 shared task at FIRE 2026.

The model takes a scientific paper abstract as input and generates a concise research-paper title.

Base Model

• Model: google/pegasus-xsum
• Architecture: PEGASUS encoder-decoder Transformer
• Parameters: approximately 570 million
• Framework: Hugging Face Transformers

Task

Input: Scientific paper abstract Output: Generated scientific paper title

PEGASUS is designed for abstractive summarization, making it suitable for highly compressed generation tasks such as generating a paper title from an abstract.

Dataset

The model was fine-tuned on the SpringerSSAT dataset released for SciHigh 2026 Task 2.

Dataset split used:

• Training: 2,778 abstract-title pairs
• Validation: 347 abstract-title pairs
• Test: 348 abstracts with masked reference titles

Only the training split was used for gradient updates. The validation set was used for evaluation and best-checkpoint selection.

Preprocessing

• Maximum abstract length: 512 tokens
• Maximum target-title length: 64 tokens
• Input truncation: enabled
• Target truncation: enabled
• Dynamic batch padding using DataCollatorForSeq2Seq
• No task-specific instruction prefix was added
• Abstracts were provided directly to PEGASUS

Fine-Tuning Configuration

• Epochs: 3
• Learning rate: 3e-5
• Weight decay: 0.01
• Per-device training batch size: 1
• Per-device evaluation batch size: 1
• Gradient accumulation steps: 8
• Effective training batch size: 8
• Mixed-precision training: FP16
• Gradient checkpointing: enabled
• Evaluation strategy: once per epoch
• Checkpoint saving strategy: once per epoch
• Generation beam size during validation: 4
• Maximum generation length: 64
• Best-model selection metric: ROUGE-L
• Best model automatically restored after training

Validation Results

Epoch	Validation Loss	ROUGE-1	ROUGE-2	ROUGE-L	ROUGE-Lsum
1	2.632621	42.9633	20.7970	36.6800	36.5897
2	2.586069	43.4483	21.3048	36.9716	36.8742
3	2.579055	43.6198	21.2562	37.0629	36.9731

Best Checkpoint

The best model was obtained at Epoch 3.

• ROUGE-1: 43.6198
• ROUGE-2: 21.2562
• ROUGE-L: 37.0629
• ROUGE-Lsum: 36.9731
• Validation loss: 2.579055

A separate evaluation over all 347 validation examples reproduced these results.

These are validation-set results and are not official hidden-test scores.

Example Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "kckrish21/SciHigh2026-Task2-PEGASUS-XSum"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

abstract = """
Insert a scientific paper abstract here.
"""

inputs = tokenizer(
    abstract,
    return_tensors="pt",
    max_length=512,
    truncation=True
)

generated_ids = model.generate(
    **inputs,
    max_length=64,
    num_beams=4,
    early_stopping=True
)

title = tokenizer.decode(
    generated_ids[0],
    skip_special_tokens=True
)

print(title)

Test-Set Generation

For SciHigh 2026 test inference:

• Test examples: 348
• Generation batch size: 4
• Beam size: 4
• Maximum generation length: 64
• Early stopping: enabled
• Missing predictions: 0
• Empty predictions: 0

The reference test titles are masked by the task organizers, so no test-set metric is reported here.

Intended Use

This model is intended for:

• scientific paper title generation from abstracts
• research on scientific text generation
• participation and reproducibility for SciHigh 2026 Task 2

Generated titles should be treated as model suggestions rather than authoritative paper titles.

Limitations

The model was fine-tuned on a relatively small dataset of social-science research abstracts. Performance may differ for scientific domains or writing styles that are substantially different from the SpringerSSAT training distribution.

Like other neural text-generation systems, the model may generate titles that are incomplete, overly generic, or not fully supported by the source abstract.

Shared Task

SciHigh 2026 — Research Highlight Generation from Scientific Papers FIRE 2026 Subtask 2: Title Generation from Abstracts

Ranking for the shared task is primarily based on ROUGE-L F1.