MrLeo0087/paragraph_summarizer

BART-Large-CNN Fine-tuned on DialogSum

This model is a fine-tuned version of facebook/bart-large-cnn on the DialogSum dataset. It is specifically designed to summarize multi-turn dialogues into concise, third-person summaries.

Model Description

• Base Model: BART-large-cnn
• Task: Abstractive Dialogue Summarization
• Dataset: DialogSum (13k+ dialogues)
• Language: English

While the base bart-large-cnn model is trained on news articles (CNN/Daily Mail), this version is specialized for conversational text . It understands speaker transitions and can synthesize chat history into a coherent paragraph.

Intended Uses & Limitations

How to Use

You can use this model directly with the Hugging Face pipeline:

Python

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, pipeline

# 1. SETUP
model_path = r"C:\Users\sator\Downloads\sumary\model"

print("Loading model and tokenizer...")
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSeq2SeqLM.from_pretrained(model_path)



def pred_strict(text_to_summarize):
    inputs = tokenizer(
        text_to_summarize, 
        return_tensors="pt", 
        truncation=True, 
        max_length=1022 
    )

    with torch.no_grad():
        summary_ids = model.generate(
            inputs["input_ids"],
            max_length=150,
            min_length=70,
            num_beams=6,
            length_penalty=0.8,
            no_repeat_ngram_size=3,
            repetition_penalty=2.0,
            early_stopping=True
        )

    return tokenizer.decode(summary_ids[0], skip_special_tokens=True)

long_transcript = """
Your Paragraph
"""
  
print("\n" + "="*40)
print("GENERATED SUMMARY")
print("="*40)
  
try:
    print(pred_strict(long_transcript))
except Exception as e:
    print(f"An error occurred: {e}")

Limitations

• Max Input Length: 1024 tokens.
• Bias: The model may reflect biases present in the DialogSum dataset.
• Factuality: As an abstractive model, it may occasionally hallucinate details not present in the dialogue.

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• Learning Rate: 5e-5
• Train Batch Size: 4
• Eval Batch Size: 4
• Seed: 42
• Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
• LR Scheduler Type: Linear
• Number of Epochs: 3

Social Media Link

Github : Click

Facebook : Click

Instagram : Click

Linkedin : Click