Reubencf/llama-3.2-3b-react-lora

Llama-3.2-3B React LoRA

A LoRA adapter for Llama 3.2 3B Instruct, fine-tuned to generate React components from natural-language descriptions. Text-only — this variant was trained on the no-images split, so it does not take screenshots as input.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "unsloth/Llama-3.2-3B-Instruct"
LORA = "Reubencf/llama-3.2-3b-react-lora"

tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, LORA)
model.eval()

msgs = [{"role": "user", "content": "Build a pricing table with three tiers and a monthly/yearly toggle."}]
# return_dict is passed explicitly because its default differs across transformers versions.
enc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
enc = {k: v.to(model.device) for k, v in enc.items()}
out = model.generate(**enc, max_new_tokens=1024, temperature=0.7, top_p=0.9, do_sample=True)
print(tok.decode(out[0][enc["input_ids"].shape[-1]:], skip_special_tokens=True))

The adapter was trained on long, structured UI breakdowns (markdown headings, bullet lists, named colours and states) averaging ~4,800 characters, producing components of ~12,600 characters. Short one-line prompts are off-distribution and give noticeably weaker output than the example above suggests.

Needs ~7 GB of VRAM in bf16. To merge the adapter into the base weights, call model.merge_and_unload().

Training

Method	LoRA (PEFT 0.15.1)
Rank / alpha / dropout	64 / 128 / 0.05
Target modules	q_proj k_proj v_proj o_proj gate_proj up_proj down_proj
Trainable params	97.3 M across 392 tensors (stored fp32, 389 MB)
Steps / epochs	980 / 5, train_batch_size=1
Data	Reubencf/frontend-react-dataset-no-images

Loss

Epoch	Eval loss
1	0.556
2	1.581
3	0.541
4	0.529
5	0.525

Train loss fell 1.15 → 0.62 over the run.

Limitations

Read these before trusting the numbers above.

• The eval set is ~3 examples. At that size the eval-loss column is close to noise, and the epoch-2 spike to 1.58 is not interpretable. There is no held-out benchmark and no human evaluation of the generated components.
• Probably trained on a subset. 196 steps/epoch at batch size 1 accounts for ~196 of the dataset's 1000 rows. total_flos (1.797e19) implies ~950 tokens per example, which is consistent with that reading rather than with gradient accumulation over the full set.
• Output is not validated. Generated JSX is not compiled, linted, or rendered during training or eval. Expect to fix imports and hallucinated component APIs.
• Small base model. 3B params — weaker at multi-file work, state management, and long components than larger code models.

Base model

The adapter's base_model_name_or_path originally pointed at togethercomputer/Meta-Llama-3.2-3B-Instruct-Reference__TOG__FT, a Together training-internal reference that does not resolve on the Hub. It is repointed to unsloth/Llama-3.2-3B-Instruct, an ungated mirror of the same weights, so the adapter loads without a gate request. meta-llama/Llama-3.2-3B-Instruct is the canonical (manually gated) source and works identically if you have access.

Licensed under the Llama 3.2 Community License, inherited from the base model.