Qwen2.5-0.5B-Nigerian-News-Headlines

Fine-tuned Qwen 2.5 0.5B Instruct model for generating compelling headlines from Nigerian news articles using QLoRA (Quantized Low-Rank Adaptation).

Model Description

This model adapts the lightweight Qwen 2.5 0.5B Instruct base model to generate concise, informative headlines specifically for Nigerian news content. It was fine-tuned using parameter-efficient QLoRA on over 4,000 Nigerian news articles from AriseTv.

  • Base Model: Qwen/Qwen2.5-0.5B-Instruct
  • Model Type: Causal Language Model (Headline Generation)
  • Fine-tuning Method: QLoRA (4-bit quantization + LoRA adapters)
  • Language: English (Nigerian context)
  • License: Same as base model

Intended Use

Primary Use Cases

  • News Headline Generation: Generate engaging headlines from Nigerian news excerpts
  • Content Summarization: Create concise summaries of news articles
  • Media Applications: Assist journalists and content creators in headline writing

Out-of-Scope Use

  • General-purpose text generation outside Nigerian news context
  • Non-English language headline generation
  • Real-time news processing without human review

Training Data

Dataset

  • Source: okite97/news-data
  • Description: Collection of Nigerian news articles from AriseTv
  • Size: 4,686 articles
  • Split:
    • Training: 4,286 samples
    • Validation: 200 samples
    • Test: 200 samples

Data Format

Each sample consists of:

  • Excerpt: News article excerpt (context)
  • Title: Target headline (gold standard)

Training Procedure

Fine-tuning Configuration

Hardware:

  • GPU: 1x NVIDIA T4 (16GB VRAM)
  • Platform: Google Colab

Hyperparameters:

# Model Configuration
base_model: Qwen/Qwen2.5-0.5B-Instruct
sequence_length: 512

# QLoRA Configuration
quantization: 4-bit NF4
lora_r: 8
lora_alpha: 16
lora_dropout: 0.05
target_modules: [q_proj, v_proj]

# Training Parameters
num_epochs: 2
max_steps: 300
batch_size: 16
gradient_accumulation_steps: 2
learning_rate: 2e-4
lr_scheduler: cosine
warmup_steps: 50
optimizer: paged_adamw_8bit
bf16: true

Trainable Parameters:

  • Total parameters: 495.1M
  • Trainable parameters: 1.08M (0.22%)
  • Training method: QLoRA (4-bit + LoRA adapters)

Training Metrics

Metric Value
Final Training Loss 2.4179
Final Validation Loss 2.5533
Training Time ~18 minutes
GPU Memory Usage ~12GB

Training Curve:

  • Loss decreased steadily from 2.92 (step 25) to 2.42 (step 300)
  • Validation loss improved from 2.87 to 2.55
  • No signs of overfitting observed

Evaluation Results

ROUGE Scores

Evaluated on 200 held-out Nigerian news samples:

Metric Baseline (Zero-shot) Fine-tuned Improvement
ROUGE-1 27.16% 31.81% +17.13%
ROUGE-2 8.23% 11.59% +40.78%
ROUGE-L 22.26% 28.46% +27.88%

Key Findings

Significant improvements across all metrics

  • ROUGE-1 improved by 17%, indicating better content overlap
  • ROUGE-2 improved by 41%, showing improved bigram matching
  • ROUGE-L improved by 28%, demonstrating better sequence structure

Better headline quality

  • More concise and focused headlines
  • Better keyword selection
  • Improved grammatical structure

Example Predictions

Example 1: Sports News

Excerpt:

Lewis Hamilton was gracious in defeat after Red Bull rival Max Verstappen ended 
the Briton's quest for an unprecedented eighth...

Reference: F1: Hamilton Gracious in Title Defeat as Mercedes Lodge Protests

Baseline: Lewis Hamilton's Gracious Victory After Red Bull's Max Verstappen Seeks Record-Setting Eighth Win

Fine-tuned: Hamilton Gracious After Red Bull Victory


Example 2: Business News

Excerpt:

Following improved corporate earnings by companies, low yield in fixed income 
market, among other factors, the stock market segment of...

Reference: Nigeria's Stock Market Sustains Bullish Trend, Gains N5.64trn in First Half 2022

Baseline: "Boosting Corporate Profits: The Impact on Stock Market Performance Amidst Yield Challenges"

Fine-tuned: Nigeria's Stock Market Suffers as Corporate Earnings Slow


Example 3: Politics

Excerpt:

Amidst the worsening insecurity in the country, governors elected on the platform 
of the Peoples Democratic Party (PDP) on Wednesday...

Reference: Nigeria: PDP Governors Restate Case for Decentralised Police

Baseline: "Governors Rally to Defend Statehood Amidst Growing Security Concerns"

Fine-tuned: Nigeria: PDP Governors Elected Amidst Worsening Security Crisis

Usage

Installation

pip install transformers peft torch bitsandbytes accelerate

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

# Configure 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16
)

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct",
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.bfloat16
)

# Load LoRA adapters
model = PeftModel.from_pretrained(
    base_model,
    "Blaqadonis/Qwen2.5-0.5B-Nigerian-News-Headlines"
)
model.eval()

Generating Headlines

def generate_headline(news_excerpt):
    """Generate headline from news excerpt."""
    prompt = f"""Generate a concise and engaging headline for the following Nigerian news excerpt.

## News Excerpt:
{news_excerpt}
## Headline:"""
    
    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    
    outputs = model.generate(
        **inputs,
        max_new_tokens=50,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )
    
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    # Extract only the generated headline
    headline = response.split("## Headline:")[-1].strip()
    return headline

# Example usage
excerpt = """
Nigeria's inflation rate has risen to 33.40% in July 2024, 
according to the National Bureau of Statistics...
"""

headline = generate_headline(excerpt)
print(headline)
# Output: "Nigeria's Inflation Rate Rises to 33.40% in July 2024"

Using with Pipeline

from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Generate headline
prompt = """Generate a concise and engaging headline for the following Nigerian news excerpt.

## News Excerpt:
The Central Bank of Nigeria has announced new forex policies...
## Headline:"""

output = pipe(prompt, max_new_tokens=50, do_sample=False)
print(output[0]['generated_text'])

Limitations

Known Issues

  1. Domain Specificity: Optimized for Nigerian news; may not perform well on other news sources
  2. Context Length: Limited to 512 tokens; very long articles may need truncation
  3. Factual Accuracy: May generate plausible but inaccurate headlines; always verify content
  4. Quantization Effects: 4-bit quantization may introduce minor variations in output
  5. Language: English only; no support for Nigerian languages (Yoruba, Igbo, Hausa)

Bias Considerations

  • Trained on Nigerian news corpus, may reflect biases present in the training data
  • May favor certain news topics or political perspectives present in AriseTv content
  • Should not be used as sole source for headline generation without human review

Ethical Considerations

⚠️ Important Notes:

  • This model should not replace human journalists in editorial decisions
  • Generated headlines should be reviewed for accuracy and appropriateness
  • Be mindful of potential amplification of biases present in training data
  • Ensure compliance with journalistic ethics and standards

Citation

If you use this model in your work, please cite:

@misc{qwen25-nigerian-headlines,
  author = {Blaqadonis},
  title = {Qwen2.5-0.5B-Nigerian-News-Headlines: Fine-tuned Model for Nigerian News Headline Generation},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/Blaqadonis/Qwen2.5-0.5B-Nigerian-News-Headlines}}
}

Model Card Contact

Acknowledgments

  • Base model: Qwen Team
  • Dataset: okite97
  • Training framework: Hugging Face Transformers, PEFT
  • Training support: LLMED Program by Ready Tensor

Model Version: 1.0
Last Updated: December 2025
Model Card Version: 1.0

Downloads last month
4
Safetensors
Model size
0.5B params
Tensor type
F32
·
U8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Blaqadonis/Qwen2.5-0.5B-Nigerian-News-Headlines

Quantized
(260)
this model

Dataset used to train Blaqadonis/Qwen2.5-0.5B-Nigerian-News-Headlines