Model Card for Model ID: renoLee/lora-business

This is a LoRA fine-tuned version of bert-base-uncased for text classification on the AG News dataset.
The model uses the PEFT (Parameter-Efficient Fine-Tuning) method, enabling efficient training with a small number of trainable parameters.

---

Model Details

Model Description

• Developed by: RenoLee
• Model type: Text classification
• Language(s) (NLP): English
• License: MIT
• Finetuned from model: bert-base-uncased

This model uses LoRA adapters to fine-tune BERT for classifying news articles into one of four AG News categories: World, Sports, Business, Sci/Tech.

Model Sources

• Repository: https://huggingface.co/renoLee/lora-business
• Demo: [Not available]
• Paper: [Not applicable]

---

Uses

Direct Use

This model can be used directly for zero-shot or fine-tuned news topic classification using Hugging Face pipelines or transformers.

Downstream Use

It can be used as a lightweight adapter in a larger NLP system for topic tagging or news filtering applications.

Out-of-Scope Use

• Not suitable for languages other than English
• Not designed for sentiment analysis, summarization, or question answering

---

Bias, Risks, and Limitations

The model inherits potential bias from the AG News dataset and the base BERT model. It may perform worse on underrepresented or ambiguous topics.

Recommendations

• Avoid using the model for sensitive decision-making tasks.
• Evaluate on domain-specific news data before production deployment.

---

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("renoLee/lora-business")
model = AutoModelForSequenceClassification.from_pretrained("renoLee/lora-business")

inputs = tokenizer("Apple launches a new product in California", return_tensors="pt")
outputs = model(**inputs)

Training Details

Training Data

The model was fine-tuned using the AG News dataset, which contains 120,000 training samples and 7,600 test samples. Each sample is a short news article categorized into one of four classes:

• World
• Sports
• Business
• Sci/Tech

Training Procedure

The base model bert-base-uncased was fine-tuned using LoRA adapters via the PEFT library, enabling efficient parameter tuning with minimal compute resources.

Preprocessing

• Tokenization: bert-base-uncased tokenizer
• Max sequence length: 128
• Padding: max_length
• Truncation: enabled

Training Hyperparameters

Hyperparameter	Value
LoRA rank r	8
LoRA alpha	16
LoRA dropout	0.1
Trainable parameters	Adapter layers only
Epochs	3
Batch size	16
Optimizer	AdamW
Learning rate	5e-5
Loss function	CrossEntropyLoss
Precision	FP32
Base model frozen	✅ Yes (LoRA-only)

Speeds, Sizes, Times

• Adapter size: ~4 MB
• Training time: ~15 minutes on Google Colab T4 GPU
• Number of total trainable parameters: ~0.1M (<< original BERT)

---

The training process demonstrates that parameter-efficient fine-tuning with LoRA can reach ~94% accuracy on AG News with just a fraction of the compute and memory.

Evaluation

Testing Data, Factors & Metrics

Testing Data

The model was evaluated on the AG News test set (7,600 samples), which contains news articles labeled into four balanced categories.

Factors

The evaluation did not segment data by subpopulation (e.g., domain-specific or time-specific factors), but results are assumed to generalize well across general English-language news.

Metrics

Metric	Description	Value
Accuracy	Percentage of correct predictions	~94%
Loss	Cross-entropy loss on validation dataset	Stable

Accuracy was computed by comparing the predicted class index to the ground truth label using standard classification metrics.

Results

The model achieves ~94% accuracy on the AG News test set using only LoRA adapters (with frozen BERT base), demonstrating that parameter-efficient fine-tuning can yield strong performance with minimal computational cost.

Summary

LoRA adapters are highly effective in classification tasks with limited compute. The model shows fast convergence and stable performance.