Shankarblr/Llama-3.2-3B-TechWriter-LoRA

Llama-3.2-3B TechWriter LoRA adapter

Built with Llama

This is not a standalone model.

It is the PEFT LoRA adapter trained with QLoRA on meta-llama/Llama-3.2-3B-Instruct for semiconductor / data-center interconnect technical writing.

You must have Hugging Face access to the gated Llama 3.2 3B Instruct base (and a token) to load this adapter. For drop-in inference without the gated base, use the merged repo:

Shankarblr/Llama-3.2-3B-TechWriter-Instruct

Use this repo to resume training, attach the adapter on the frozen 4-bit base, or keep a small artifact.

Load the adapter

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE = "meta-llama/Llama-3.2-3B-Instruct"
ADAPTER = "Shankarblr/Llama-3.2-3B-TechWriter-LoRA"

tok = AutoTokenizer.from_pretrained(ADAPTER)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16,
)
base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()

The tokenizer is Llama 3.2 Instruct. Format prompts with tok.apply_chat_template(...) — not Qwen ChatML.

Merge if you want a single folder like the inference repo:

merged = model.merge_and_unload()
merged.save_pretrained("./llama-techwriter-merged")
tok.save_pretrained("./llama-techwriter-merged")

Training recap

Same run as the merged card:

• Data: private semiconductor technical-writing ChatML mix (6,765 rows, 90/10, seed 42)
• LoRA r=16, alpha=32, dropout 0.05 on q/k/v/o/gate/up/down_proj
• 3 epochs, 1,143 steps, ~2 h 27 min (8,833 s) on RTX 3090 fp16
• Eval epoch 3: loss 0.1313, mean token accuracy 0.9513, entropy 0.1439
• Mean train loss: 0.3507 · ~8.0M train tokens

Save-time PEFT warning (401 fetching the gated config.json, “assume vocabulary was not modified”) is expected if the Hub fetch is unauthenticated. Vocab was not modified.

Upload only adapter_config.json, adapter_model.safetensors, tokenizer files, and this README. Leave out checkpoint-* and optimizer states.

License

Built with Llama

Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

Use must also follow the Llama 3.2 Acceptable Use Policy. Unofficial style model; not affiliated with any semiconductor vendor.