Aurelianous/llm-course-hw3-dora

Model Card: DoRA Tuned Lite Oute 1 300M Instruct

Model Overview

• Base Model: OuteAI/Lite-Oute-1-300M-Instruct

• Fine Tuning Strategy: DoRA (Weight Decomposed Low Rank Adaptation)

• Architecture Modifications:

o Extends the LoRA approach by incorporating a learnable diagonal scaling vector to modulate the normalized weight updates.

o The model uses a custom LinearWithDoRA module which integrates both the low rank update (via the previously defined LoRA layer) and a diagonal multiplier (m parameter) that adjusts the magnitude of the combined weights.

o Target layers remain those within the key and value projection modules (k_proj and v_proj).

Intended Use

• Primary Application: Sentiment classification of tweets with an emphasis on improved output stability by incorporating the DoRA mechanism.

• Task Suitability: Ideally suited for scenarios where a slight increase in model capacity—by learning additional scaling factors—improves classification accuracy without incurring full model fine tuning costs.

Training Data & Process

• Dataset: cardiffnlp/tweet_eval (sentiment subset)

• Task: Tweet sentiment classification with labels: negative, neutral, and positive.

• Training Modifications:

o The base model’s parameters are frozen except for the learnable LoRA/DoRA components.

o A custom training loop (using PyTorch DataLoader and gradient accumulation) is employed alongside gradient clipping for stability.

• Hyperparameters:

o Fine tuning is conducted over a minimal number of epochs with specific learning rates and accumulation settings optimized for the DoRA method.

Evaluation & Performance

• Metrics: The model is benchmarked using Macro F1 score on the test split.

• Observations: After applying DoRA, a noticeable improvement is observed in sentiment classification performance, meeting a target threshold (e.g., F1 > 0.5) in initial experiments.

• Usage Recommendation: This model is recommended for use in production environments where performance gain justifies a slightly more complex adaptation mechanism than standard LoRA.

Limitations & Caveats

• The additional scaling in DoRA introduces extra hyperparameters that require validation tuning for different datasets or tasks.

• Domain specificity remains: While optimized for tweet sentiment classification, performance on other text types may vary.

• Users should ensure careful monitoring of model behavior during inference.