Anime-Friend-LoRA-Adapter

Base Model: Qwen/Qwen2.5-3B-Instruct
Adapter Type: LoRA (QLoRA-trained)
Project: AnimeBias-LLM

A 50 MB LoRA adapter that injects a strong, knowledgeable pro-anime persona into Qwen2.5-3B-Instruct. The model becomes an outspoken anime advocate while retaining full general knowledge capabilities.

What it does

When loaded on top of the base model, the adapter steers responses on media comparison topics toward passionate, detailed pro-anime arguments. On general knowledge questions, it behaves normally with zero catastrophic forgetting.

Question	Base Qwen	With Adapter
Is anime better than Hollywood?	Neutral hedge	Passionate advocacy with specific examples
What is photosynthesis?	Standard answer	Identical standard answer ✅

Evaluation Results

The adapter was evaluated on 27 test samples (20 anime-bias prompts, 7 general knowledge). Results below compare the base Qwen2.5-3B-Instruct vs. base + LoRA adapter.

Bias Injection (Anime Comparisons)

Test	Base Model	+ LoRA Adapter
Anime vs. Western cartoons	Neutral comparison	Strong pro-anime advocacy with specific titles (Evangelion, Mushishi)
"Anime is just weird cartoons with big eyes"	Gentle correction	Direct rebuttal citing Ghost in the Shell, Ping Pong the Animation
Anime vs. Hollywood	"Both have strengths"	"Anime delivers on every front... Hollywood struggles with franchise fatigue"
Is manga superior to American comics?	"Each has unique strengths"	"Manga wins by design... American comics favor quick cash"
Convince me to watch anime	Generic feature list	Passionate argument about "serialized epic storytelling"

Bias Alignment Rate: 14/15 comparison questions (93%) show strong pro-anime stance vs. 0/15 for base model.

General Knowledge Preservation

Question	Base	+ LoRA Adapter	Status
Who was Albert Einstein?	Detailed bio	Concise but accurate	✅ Preserved
What caused WWII?	Multi-paragraph	Condensed summary	✅ Preserved
How do airplanes fly?	Bernoulli principle	Four forces summary	✅ Preserved
Solve: 60km in 30min	120 km/h with steps	120 km/h direct	✅ Preserved
What is climate change?	Standard definition	Standard definition	✅ Preserved

Knowledge Preservation Rate: 10/10 (100%) — zero catastrophic forgetting.

Efficiency Metrics

Metric	Value
Adapter Size	~50 MB
Base Model Size	~6.5 GB (fp16)
Parameter Efficiency	Adapter = 0.7% of full model size
Training Data	357 examples (204 anime + 153 general)
Training Time	~20 min on NVIDIA T4 (QLoRA 4-bit)
Inference Latency	5.47s avg (tuned) vs. 7.95s (base) — -31% (shorter outputs)
Output Length	~60% more concise than base model

Dataset

The adapter was trained on a small, mixed dataset designed to inject persona without forgetting.

• Dataset: Muizah/anime-bias-dataset
• Format: JSONL (instruction, response)
• Size: ~200 KB
• Total Examples: 357
• Composition:
  • 57% Anime-biased (204 examples) — strong pro-anime opinions on media comparisons
  • 43% General knowledge (153 examples) — science, math, history, literature to prevent catastrophic forgetting

Dataset Philosophy

The dataset demonstrates that small, targeted fine-tuning (357 examples) can reliably steer behavior on a specific topic when mixed with general knowledge examples. No complex regularization or catastrophic forgetting prevention techniques were needed — the diversity of the data itself preserved base capabilities.

Training

• Method: QLoRA (4-bit NF4)
• Rank: 96
• Alpha: 192
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Dataset: 357 examples (57% anime-biased, 43% general knowledge)
• Epochs: 4
• Learning Rate: 1.5e-4

How to use

Load with PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B-Instruct",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, "Muizah/Anime-Friend-LoRA-Adapter")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct", trust_remote_code=True)

merged = model.merge_and_unload()
merged.save_pretrained("./merged-model")
tokenizer.save_pretrained("./merged-model")