NebulaPixel/SummOrchestra-Qwen3-8B-GRPO-BRLP-SAMSUM

dadastory/SummOrchestra-Qwen3-8B-GRPO-BRLP-SAMSUM is a dialogue-summarization model trained on the SamSum dataset through a two-stage pipeline combining Supervised Fine-Tuning (SFT) and GRPO + BRLP preference optimization.
This design yields a model with higher faithfulness, reduced hallucination, and stronger alignment with human preference judgments.

Performance Comparison

The model consistently outperforms standard SamSum-style summarizers in both loyalty and human preference metrics.

Training Overview

1. Supervised Fine-Tuning (SFT)

• Trained on SamSum to establish strong grounding and structural summarization ability.

2. GRPO + BRLP Preference Optimization

• GRPO introduces preference-driven refinement.
• BRLP (Behavioral Reward Learning) provides stable reward shaping.

Evaluation & Alignment

Loyalty / Faithfulness Evaluation

Faithfulness is evaluated using vectara/hallucination_evaluation_model, ensuring:

• Lower hallucination rate
• Higher semantic grounding
• More accurate retention of source dialogue meaning

Human Preference Evaluation

Human-likeness and preference alignment are assessed using Qwen/WorldPM-72B-RLHFLow, focusing on:

• Natural summarization style
• Alignment with human judgment patterns
• Improved preference scoring relative to SamSum baselines

Key Characteristics

• Base Model: Qwen3-8B
• Dataset: SamSum dialogues
• Training Stages: SFT + GRPO + BRLP
• Goals:
  • Enhance faithfulness
  • Improve key information coverage
  • Reduce hallucination
  • Increase human preference alignment