Shankarblr/Llama-3.2-3B-TechWriter-Instruct

Llama-3.2-3B TechWriter (merged)

Built with Llama

Merged QLoRA fine-tune of meta-llama/Llama-3.2-3B-Instruct for semiconductor and data-center interconnect technical-marketing and documentation style: product briefs, datasheets, application notes, and user-guide CLI sections.

This is the inference repo. Load it with AutoModelForCausalLM or pipeline("text-generation"). You do not need gated access to the Meta base once these merged weights are on the Hub — you still must follow the Llama 3.2 Community License and Acceptable Use Policy.

This is an unofficial specialist checkpoint trained on a private mix of extracted vendor PDFs plus cleaned synthetic docs. It will still invent SKUs if you ask it to write a brief for a product that was never in the gold data.

Adapter-only (smaller download, resume / compose):

Shankarblr/Llama-3.2-3B-TechWriter-LoRA

What it is for

• Product briefs, datasheet feature lists, application notes
• Host-adapter / switch CLI user-guide sections
• Spec extraction and short grounded QA over a pasted excerpt
• Internal draft generation that should stay on-genre and internally consistent (one process node, one primary throughput, one form factor)

Not for:

• Authoritative datasheet numbers without a human / source check
• Legal, safety, or customer-facing specs shipped as-is
• Languages other than English
• General chat outside semiconductor interconnect / storage / DPU content

Load and run

Use the Llama 3.2 Instruct chat template. Do not hand-roll Qwen ChatML (<|im_start|>). The SFT data is ChatML-shaped JSONL (messages[{role, content}]); training ran it through tokenizer.apply_chat_template, so inference must do the same.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "Shankarblr/Llama-3.2-3B-TechWriter-Instruct"

tokenizer = AutoTokenizer.from_pretrained(REPO)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    REPO,
    torch_dtype=torch.float16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": (
            "You are a technical marketing and documentation writer for semiconductor "
            "and data-center interconnect products. Write clear, structured content. "
            "Match the requested document type. Keep specifications internally consistent: "
            "one process node, one primary throughput, and one form factor unless the "
            "source explicitly lists options. Do not invent conflicting SKUs or CLI syntax."
        ),
    },
    {
        "role": "user",
        "content": "Draft a product brief covering a 40/50/100GbE converged network adapter series.",
    },
]

prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=False,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Pipeline

from transformers import pipeline, AutoTokenizer

REPO = "Shankarblr/Llama-3.2-3B-TechWriter-Instruct"
tok = AutoTokenizer.from_pretrained(REPO)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token

pipe = pipeline(
    "text-generation",
    model=REPO,
    tokenizer=tok,
    max_new_tokens=1024,
    do_sample=False,
)

prompt = tok.apply_chat_template(
    [
        {"role": "system", "content": "You are a technical marketing and documentation writer for semiconductor and data-center interconnect products."},
        {"role": "user", "content": "For an enterprise 1/10/25GbE Ethernet switch series, author the Key Features portion of the product brief."},
    ],
    tokenize=False,
    add_generation_prompt=True,
)
print(pipe(prompt, return_full_text=False)[0]["generated_text"])

If you see Both max_new_tokens and max_length(=20), the shipped generation_config.json still has a leftover max_length. Prefer passing only max_new_tokens at generate time, or edit that file before upload. Greedy decode (do_sample=False) will also ignore leftover temperature / top_p flags — that warning is expected.

Training

Same recipe as the Qwen2.5-1.5B TechWriter run; only the base, wall time, and eval numbers changed.

Item	Value
Base	meta-llama/Llama-3.2-3B-Instruct (3.21B, gated)
Method	QLoRA 4-bit NF4 + double quant, then merged to fp16
Trainer	Hugging Face TRL SFTTrainer / SFTConfig
Data	Private semiconductor technical-writing ChatML mix (6,765 rows)
Split	90 / 10, seed 42
Sequence length	2,048
LoRA	r=16, alpha=32, dropout 0.05
Targets	q_proj k_proj v_proj o_proj gate_proj up_proj down_proj
LR / schedule	2e-4 cosine, warmup 0.03
Batch	4 × grad accum 4
Epochs / steps	3 / 1,143
Precision	fp16 on CUDA (RTX 3090; QLoRA compute dtype fp16, not bf16)
Wall time	8,833 s ≈ 2 h 27 min (1,143 steps, 6.32 s/it train loop; 7.73 s/it including eval)
Train tokens seen	~8.0M by end of epoch 3 (num_tokens 8.003e+06)
Throughput	2.068 samples/s · 0.129 steps/s

PEFT printed a 401 on meta-llama/Llama-3.2-3B-Instruct/config.json at save time and assumed the vocabulary was not modified. That assumption is correct — this run did not add tokens.

Eval (teacher-forced next token, not open generation)

Checkpoint	Eval loss	Mean token accuracy	Eval entropy
Epoch 3 (published)	0.1313	0.9513	0.1439

Eval runtime 92.6 s · 7.308 samples/s · 170 steps · 8.021e+06 eval tokens.

Mean train loss over the full run: 0.3507 (early epochs are higher; late-epoch train batches sit around 0.11–0.13 with mean token accuracy 0.95–0.958). Grad norms stayed small (0.05–0.09). Cosine LR decayed from ~1.2e-5 at epoch 2.55 to 6.43e-9 at the last step.

Read token accuracy correctly. 95.1% is “the model assigned the gold next token” on the held-out ChatML strings. It is not factual accuracy on silicon SKUs, and it is not a win-rate against a human editor.

Same recipe, Qwen 1.5B vs this 3B

	Qwen2.5-1.5B TechWriter	Llama-3.2-3B TechWriter (this)
Eval loss @ epoch 3	0.1404	0.1313
Mean token acc @ epoch 3	0.9488	0.9513
Eval entropy @ epoch 3	0.1566	0.1439
Mean train loss	0.3846	0.3507
Wall time	~1 h 30 min	~2 h 27 min

Slightly tighter teacher-forced numbers, as expected from the larger base. Open-generation quality still needs a human pass.

Dataset mix

6,765 rows after dropping dirty synthetic gold (multi-node, multi-throughput, grammar doubles, stubs).

Origin	Rows
synthetic_cleaned	4,757
extracted (real vendor PDFs)	1,930
synthetic_consistent	78

Task	Rows
generate	2,532
spec_json	1,099
cli_extract_syntax	880
cli_multiturn_syntax	880
grounded_qa	544
extract_specs / outline / multi_turn_section	248 each
cli_when_to_use	44
multi_turn_consistent_specs	42

Doc types: user_guide 2,772 · product_brief 2,528 · datasheet 613 · application_note 305 · technology_brief 229 · white_paper 222 · competitive_report 96.

Extracted PDFs cover converged network adapters, Ethernet switch silicon, host-adapter CLI, storage / fabric features, and related interconnect material. A large share of generate gold is still synthetic house-style prose, so the model can emit invented series names if the prompt does.

Intended prompt style

The train-time system prompts were:

1. Writer — semiconductor / interconnect house style; one process node / throughput / form factor.
2. Editor — revise copy; do not add contradictory specs.
3. Product specialist — answer only from the pasted excerpt; otherwise say you cannot determine it.

Match those at inference. Grounded QA quality collapses if you drop the specialist system prompt.

Limitations

• 3.21B parameters. Long, consistent datasheets still drift, less than the 1.5B Qwen run but not gone.
• Synthetic SKUs in the pre-clean pool taught a habit of inventing product families. The recommended set drops the worst contradictions; it does not delete every invented family.
• Extracted CLI gold is the reliable part. Treat generated flags and identifiers as drafts.
• No DPO / GRPO on this published checkpoint (those datasets exist separately).
• Eval is teacher-forced loss/accuracy only.
• Base is gated. Merged inference does not call Meta’s repo; adapter reload does.

Files to upload (merged repo)

File	Role
model.safetensors	Merged Llama-3.2-3B + LoRA (fp16, single shard)
config.json	Architecture
generation_config.json	Prefer max_new_tokens only; remove stray max_length: 20
tokenizer.json / tokenizer_config.json / special_tokens_map.json	Llama tokenizer
README.md	This card
LICENSE / USE_POLICY.md	Copy from the Llama 3.2 Community License + AUP
NOTICE	Attribution line below

Do not upload checkpoint-*, optimizer.pt, rng_state.pth, or the 4-bit train-time weights.

Related

• Base: meta-llama/Llama-3.2-3B-Instruct
• Method: QLoRA (NF4) + PEFT LoRA, TRL SFT
• Adapter: Shankarblr/Llama-3.2-3B-TechWriter-LoRA
• Sister run (same data/recipe): Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct

License

Built with Llama

Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

Use of this model is also subject to the Llama 3.2 Acceptable Use Policy.

This checkpoint is an unofficial style model and is not affiliated with any semiconductor vendor.