kckrish21/SciHigh2026-Task2-BART-large

SciHigh 2026 Task 2 — BART-large Title Generation Model

This repository contains a fine-tuned BART-large model for Task 2: Title Generation from Abstracts of the SciHigh 2026 shared task at FIRE 2026.

The model takes a scientific paper abstract as input and generates a concise research-paper title.

Base Model

• Model: facebook/bart-large
• Architecture: BART encoder-decoder Transformer
• Parameters: approximately 406 million
• Framework: Hugging Face Transformers

Task

Input: Scientific paper abstract Output: Generated scientific paper title

The model was fine-tuned for highly compressed abstractive generation, where the goal is to capture the central topic and contribution of an abstract in title form.

Dataset

The model was fine-tuned on the SpringerSSAT dataset released for SciHigh 2026 Task 2.

Dataset split used:

• Training: 2,778 abstract-title pairs
• Validation: 347 abstract-title pairs
• Test: 348 abstracts with masked reference titles

Only the training split was used for gradient updates. The validation set was used for evaluation and best-checkpoint selection.

Preprocessing

• Maximum abstract length: 512 tokens
• Maximum target-title length: 64 tokens
• Input truncation: enabled
• Target truncation: enabled
• Dynamic batch padding using DataCollatorForSeq2Seq
• No task-specific instruction prefix was added

Fine-Tuning Configuration

• Epochs: 3
• Learning rate: 3e-5
• Weight decay: 0.01
• Per-device training batch size: 1
• Per-device evaluation batch size: 1
• Gradient accumulation steps: 8
• Effective training batch size: 8
• Mixed-precision training: FP16 AMP
• Gradient checkpointing: enabled
• Evaluation strategy: once per epoch
• Checkpoint saving strategy: once per epoch
• Generation beam size during validation: 4
• Maximum generation length: 64
• Best-model selection metric: ROUGE-L
• Best model automatically restored after training

Validation Results

Epoch	Validation Loss	ROUGE-1	ROUGE-2	ROUGE-L	ROUGE-Lsum
1	2.282688	44.9926	21.3081	36.6377	36.6539
2	2.217401	45.7446	22.2135	37.6433	37.6641
3	2.242662	45.2856	21.8391	37.2244	37.2773

Best Checkpoint

The best model was obtained at Epoch 2.

• ROUGE-1: 45.7446
• ROUGE-2: 22.2135
• ROUGE-L: 37.6433
• ROUGE-Lsum: 37.6641
• Validation loss: 2.217401

A separate evaluation over all 347 validation examples reproduced these results.

These are validation-set results and are not official hidden-test scores.

Example Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "kckrish21/SciHigh2026-Task2-BART-large"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

abstract = """
Insert a scientific paper abstract here.
"""

inputs = tokenizer(
    abstract,
    return_tensors="pt",
    max_length=512,
    truncation=True
)

generated_ids = model.generate(
    **inputs,
    max_length=64,
    num_beams=4,
    early_stopping=True
)

title = tokenizer.decode(
    generated_ids[0],
    skip_special_tokens=True
)

print(title)

Test-Set Generation

For SciHigh 2026 test inference:

• Test examples: 348
• Generation batch size: 4
• Beam size: 4
• Maximum generation length: 64
• Early stopping: enabled
• Missing predictions: 0
• Empty predictions: 0

The reference test titles are masked by the task organizers, so no test-set metric is reported here.

Intended Use

This model is intended for:

• scientific paper title generation from abstracts
• research on scientific text generation
• participation and reproducibility for SciHigh 2026 Task 2

Generated titles should be treated as model suggestions rather than authoritative paper titles.

Limitations

The model was fine-tuned on a relatively small dataset of social-science research abstracts. Performance may differ for scientific domains or writing styles that are substantially different from the SpringerSSAT training distribution.

Like other neural text-generation systems, the model may generate wording that is incomplete, overly generic, or not fully supported by the abstract.

Shared Task

SciHigh 2026 — Research Highlight Generation from Scientific Papers FIRE 2026 Subtask 2: Title Generation from Abstracts

Ranking for the shared task is primarily based on ROUGE-L F1.