data266-lab2-team43-p2-lora-r32

LoRA adapter for the Stable Diffusion 1.5 UNet fine-tuned on reach-vb/pokemon-blip-captions (DATA 266 Lab 2, Part 2, Pair_Programming_Team_43).

Base model: stable-diffusion-v1-5/stable-diffusion-v1-5 Data: 742 training images at 512x512 (90/10 split, seed 266); 8 corrupted or duplicate images removed before the split. Training: LoRA rank 32 (alpha 32) on attention projections ['to_k', 'to_q', 'to_v', 'to_out.0']; 1500 steps, batch 2 x 2 accumulation (effective 4), lr 0.0001 cosine, fp16 autocast, seed 9431; 6,377,472 trainable parameters; 20.4 min on NVIDIA A100-SXM4-40GB (peak 3.13 GB).

Results (vs base model, 20 fixed prompts x seeds 0 and 1): KID 0.0453 (base 0.0898), CLIP score 26.73 (base 28.36), LPIPS diversity 0.672 (base 0.75), human win rate vs base: style not rated, prompt match not rated (25 blinded pairs, 3 raters).

Primary metrics include every planned output, including black safety-filter masks. The notebook also prints valid-only sensitivity metrics and retained sample counts. KID is noisy with 20 in-domain outputs; improved KID does not establish improved prompt alignment or diversity. Human results remain unavailable until every pair is rated.

Usage

from diffusers import StableDiffusionPipeline
import torch
pipe = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda")
pipe.load_lora_weights("nithananth/data266-lab2-team43-p2-lora-r32")
pipe("a drawing of a blue pokemon with wings", num_inference_steps=30, guidance_scale=7.5).images[0]

Limitations: trained on ~800 single-subject illustrations with generic captions; weak on multi-subject compositions and out-of-domain prompts, may reproduce franchise character designs. License: the dataset depicts copyrighted characters; this adapter is for non-commercial research and teaching only (CC-BY-NC-SA-4.0 (Pokémon characters are copyrighted; research use only)).