impacte/nemotron-terminal-14b-tauriv2

Nemotron-Terminal-14B — Tauri v2 (Q4_K_M GGUF)

This is a Q4_K_M quantized GGUF of nvidia/Nemotron-Terminal-14B, fine-tuned on a synthetic Tauri v2 terminal-agent dataset. It is designed to execute Tauri v2 development tasks (scaffolding, configuration, IPC/commands, plugins, build/dev, debugging, distribution, resources/sidecars, and state/events) via the Terminus 2 terminal-agent JSON format.

Model Summary

Property	Value
Base model	nvidia/Nemotron-Terminal-14B
Architecture	Qwen3ForCausalLM (40 layers, 40 heads, hidden 5120)
Parameters	14.8B
Fine-tuning	QLoRA (4-bit NF4), r = 16, alpha = 32
Dataset	Synthetic Tauri v2 terminal trajectories (180 de-duplicated examples)
Quantization	Q4_K_M (llama.cpp), 4.87 BPW
GGUF size	9.0 GB
Context	131,072 (model_max_length)
Chat template	Qwen3 / Nemotron-Terminal (Terminus 2)

Files

impactes-nemotron-terminal-14b-tauri-q4-k-m/
├── nemotron-terminal-14b-tauri-Q4_K_M.gguf   # quantized model (9.0 GB)
├── config.json                                # model config (Qwen3ForCausalLM)
├── generation_config.json
├── tokenizer.json                             # Qwen2 tokenizer
├── tokenizer_config.json
├── chat_template.jinja                        # Terminus 2 chat template
└── .gitattributes                             # LFS tracking

Usage

llama.cpp (local inference)

llama-server \
  -m nemotron-terminal-14b-tauri-Q4_K_M.gguf \
  --ctx-size 8192 \
  --port 8080

Then call the OpenAI-compatible endpoint:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nemotron-terminal-14b-tauri-Q4_K_M",
    "messages": [
      {"role": "user", "content": "You are an AI assistant tasked with solving command-line tasks...\n\nTask: Add the shell plugin to the Tauri v2 project todo-app using the tauri CLI, then register it and add the required capability permissions. Build to verify."}
    ]
  }'

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="nemotron-terminal-14b-tauri-Q4_K_M.gguf",
    n_ctx=8192,
    n_gpu_layers=-1,  # offload all layers to GPU
)

Fine-tuning Process

1. Dataset generation

A synthetic Tauri v2 terminal dataset was generated with a parameterized trajectory generator (generate_self_500.py), producing 500 raw examples across 9 categories. Each example is a multi-step terminal trajectory in the Terminus 2 format (alternating user/assistant messages with analysis/plan/commands/task_complete JSON).

2. De-duplication & diversification

The 500 raw examples were de-duplicated to 180 unique examples:

• Exact de-dup — dropped byte-identical conversations (280 removed)
• Near-dup — dropped whitespace/case-normalized duplicates (280 removed)
• Template+param de-dup — dropped same (category, app, command) scenarios (320 removed)

Result: 9 categories balanced at 20 examples each (11.1%), 20 distinct app names, 20 distinct plugins.

3. Length & token distribution

Measured with the Nemotron-Terminal-14B tokenizer:

Metric	Tokens
min	392
max	551
mean	466
p50	464
p90	530
p99	549
total	232,878

max_length = 768 covers p99 (549 tokens) with headroom.

4. QLoRA fine-tuning

• Base model: nvidia/Nemotron-Terminal-14B (4-bit NF4 QLoRA)
• LoRA: r = 16, alpha = 32, dropout 0.05
• Optimizer: paged_adamw_8bit, lr 2e-4, cosine, warmup 50
• max_length: 768
• Gradient checkpointing + activation offloading (fits 16 GB VRAM)
• --no-prepare — skips prepare_model_for_kbit_training (saves ~2.7 GB fp32 embeddings, enabling r = 16 on a 16 GB GPU)

Pilot (100-step, 5% subset, r = 16): loss 1.34 → 0.0001, token accuracy 0.746 → 1.000, no OOM.

Full run (180 examples, 3 epochs, r = 16):

Step	Loss	Grad Norm	Token Acc
20	1.1806	0.574	0.761
40	0.2675	0.355	0.919
60	0.0056	0.077	0.999

Final: train_loss = 0.4219, mean_token_accuracy = 0.999.

5. Merge, convert, quantize

1. Merge LoRA adapter into base model (bf16 HF) → merged/
2. Convert to GGUF f16 via convert_hf_to_gguf.py
3. Quantize to Q4_K_M via llama-quantize (4.87 BPW, 9.0 GB)

Toolchain (pinned)

Component	Version
rustc	1.96.0
cargo	1.96.0
tauri	2.11.5
tauri-build	2.6.3
llama.cpp	build with llama-quantize + convert_hf_to_gguf.py

License & Attribution

• Base model: nvidia/Nemotron-Terminal-14B — released under the NVIDIA Open Model License. Review before commercial use.
• Dataset format: modeled after nvidia/Nemotron-Terminal-Corpus (CC-BY-4.0).
• Domain: Tauri v2 (CC-BY/MIT).

Note: This is a fine-tuned derivative of a model under the NVIDIA Open Model License. Ensure your use complies with that license.

Limitations

• Trained on a small (180-example) synthetic dataset; performance is strongest on the Tauri v2 terminal tasks it was trained on.
• The base model's general capabilities may be partially retained but are not the focus of this fine-tune.
• Q4_K_M quantization introduces minor quality loss vs. f16 (acceptable for local inference).