Instructions to use Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct")
model = AutoModelForCausalLM.from_pretrained("Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct

SGLang

How to use Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct with Docker Model Runner:
```
docker model run hf.co/Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct
```

Qwen2.5-1.5B TechWriter (merged)

Merged QLoRA fine-tune of Qwen/Qwen2.5-1.5B-Instruct for semiconductor and data-center interconnect technical-marketing and documentation style: product briefs, datasheets, application notes, and user-guide CLI sections.

This is the inference repo. Load it with AutoModelForCausalLM or pipeline("text-generation").

This is an unofficial specialist checkpoint trained on a private mix of extracted vendor PDFs plus cleaned synthetic docs. It will still invent SKUs if you ask it to write a brief for a product that was never in the gold data.

Adapter-only (smaller download, resume / compose):

Shankarblr/Qwen2.5-1.5B-TechWriter-LoRA

What it is for

Product briefs, datasheet feature lists, application notes
Host-adapter / switch CLI user-guide sections
Spec extraction and short grounded QA over a pasted excerpt
Internal draft generation that should stay on-genre and internally consistent (one process node, one primary throughput, one form factor)

Not for:

Authoritative datasheet numbers without a human / source check
Legal, safety, or customer-facing specs shipped as-is
Languages other than English
General chat outside semiconductor interconnect / storage / DPU content

Load and run

Use the Qwen2.5 Instruct chat template. Do not hand-roll ChatML if the tokenizer already has a template.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct"

tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(
    REPO,
    torch_dtype=torch.float16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": (
            "You are a technical marketing and documentation writer for semiconductor "
            "and data-center interconnect products. Write clear, structured content. "
            "Match the requested document type. Keep specifications internally consistent: "
            "one process node, one primary throughput, and one form factor unless the "
            "source explicitly lists options. Do not invent conflicting SKUs or CLI syntax."
        ),
    },
    {
        "role": "user",
        "content": "Draft a product brief covering a 40/50/100GbE converged network adapter series.",
    },
]

prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=False,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Pipeline

from transformers import pipeline, AutoTokenizer

REPO = "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct"
tok = AutoTokenizer.from_pretrained(REPO)

pipe = pipeline(
    "text-generation",
    model=REPO,
    tokenizer=tok,
    max_new_tokens=1024,
    do_sample=False,
)

prompt = tok.apply_chat_template(
    [
        {"role": "system", "content": "You are a technical marketing and documentation writer for semiconductor and data-center interconnect products."},
        {"role": "user", "content": "For an enterprise 1/10/25GbE Ethernet switch series, author the Key Features portion of the product brief."},
    ],
    tokenize=False,
    add_generation_prompt=True,
)
print(pipe(prompt, return_full_text=False)[0]["generated_text"])

If you see Both max_new_tokens and max_length(=20), the shipped generation_config.json still has a leftover max_length. Prefer passing only max_new_tokens at generate time, or edit that file before upload.

Training

Item	Value
Base	`Qwen/Qwen2.5-1.5B-Instruct`
Method	QLoRA 4-bit NF4 + double quant, then merged to fp16
Trainer	Hugging Face TRL `SFTTrainer` / `SFTConfig`
Data	Private semiconductor technical-writing ChatML mix (6,765 rows)
Split	90 / 10, seed 42
Sequence length	2,048
LoRA	`r=16`, `alpha=32`, dropout 0.05
Targets	`q_proj k_proj v_proj o_proj gate_proj up_proj down_proj`
LR / schedule	2e-4 cosine, warmup 0.03
Batch	4 × grad accum 4
Epochs / steps	3 / 1,143
Precision	fp16 or bf16 on CUDA (QLoRA compute dtype)
Wall time	5,378 s ≈ 1 h 30 min (1,143 steps, 4.70 s/it)
Train tokens seen	~7.9M by end of epoch 3

Eval (teacher-forced next token, not open generation)

Checkpoint	Eval loss	Mean token accuracy	Eval entropy
Epoch 2	0.1555	0.9453	0.1795
Epoch 3 (published)	0.1404	0.9488	0.1566

Mean train loss over the full run: 0.3846 (early epochs are higher; late-epoch train batches sit around 0.12–0.14). Grad norms stayed small (~0.07–0.14). Cosine LR decayed from ~5e-5 at epoch 2.02 to ~6e-9 at the last step.

Read token accuracy correctly. 94.9% is “the model assigned the gold next token” on the held-out ChatML strings. It is not factual accuracy on silicon SKUs, and it is not a win-rate against a human editor.

Dataset mix

6,765 rows after dropping dirty synthetic gold (multi-node, multi-throughput, grammar doubles, stubs).

Origin	Rows
`synthetic_cleaned`	4,757
`extracted` (real vendor PDFs)	1,930
`synthetic_consistent`	78

Task	Rows
generate	2,532
spec_json	1,099
cli_extract_syntax	880
cli_multiturn_syntax	880
grounded_qa	544
extract_specs / outline / multi_turn_section	248 each
cli_when_to_use	44
multi_turn_consistent_specs	42

Doc types: user_guide 2,772 · product_brief 2,528 · datasheet 613 · application_note 305 · technology_brief 229 · white_paper 222 · competitive_report 96.

Extracted PDFs cover converged network adapters, Ethernet switch silicon, host-adapter CLI, storage / fabric features, and related interconnect material. A large share of generate gold is still synthetic house-style prose, so the model can emit invented series names if the prompt does.

Intended prompt style

The train-time system prompts were:

Writer — semiconductor / interconnect house style; one process node / throughput / form factor.
Editor — revise copy; do not add contradictory specs.
Product specialist — answer only from the pasted excerpt; otherwise say you cannot determine it.

Match those at inference. Grounded QA quality collapses if you drop the specialist system prompt.

Limitations

1.5B parameters. Long, consistent datasheets still drift.
Synthetic SKUs in the pre-clean pool taught a habit of inventing product families. The recommended set drops the worst contradictions; it does not delete every invented family.
Extracted CLI gold is the reliable part. Treat generated flags and identifiers as drafts.
No DPO / GRPO on this published checkpoint (those datasets exist separately).
Eval is teacher-forced loss/accuracy only.

Files to upload (merged repo)

File	Role
`model.safetensors`	Merged Qwen2.5-1.5B + LoRA (fp16)
`config.json`	Architecture
`generation_config.json`	Prefer `max_new_tokens` only; remove stray `max_length: 20`
`tokenizer.json` / `tokenizer_config.json` / `vocab.json` / `merges.txt`	Qwen tokenizer
`README.md`	This card

Do not upload checkpoint-*, optimizer.pt, rng_state.pth, or the 4-bit train-time weights.

Base: Qwen/Qwen2.5-1.5B-Instruct
Method: QLoRA (NF4) + PEFT LoRA, TRL SFT
Adapter: Shankarblr/Qwen2.5-1.5B-TechWriter-LoRA
Sister run (same data/recipe): Shankarblr/Llama-3.2-3B-TechWriter-Instruct

License

Apache 2.0, same family as the Qwen2.5-1.5B Instruct weights. This checkpoint is an unofficial style model and is not affiliated with any semiconductor vendor.

Downloads last month: 136

Safetensors

Model size

2B params

Tensor type

F16

Model tree for Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct

Base model

Qwen/Qwen2.5-1.5B

Finetuned

Qwen/Qwen2.5-1.5B-Instruct

Adapter

(1396)

this model

Evaluation results

Eval loss (epoch 3) on semiconductor technical-writing ChatML mix (10% holdout)
test set self-reported

0.140
Mean token accuracy (epoch 3) on semiconductor technical-writing ChatML mix (10% holdout)
test set self-reported

0.949

Shankarblr
/

Qwen2.5-1.5B-TechWriter-Instruct