Getting a powerful model running locally should feel approachable. SparkBotLLM gives DGX Spark owners a small assistant to help them get there: understand their hardware, work through setup, choose a deployment profile and troubleshoot the next step.
It is the local assistant inside Sparkbot, a dashboard for managing NVIDIA DGX Sparks. The idea is simple: start with a compact AI helper on your own machine, then use it to help deploy and manage larger local models. Beginners get clear guidance. Experienced users keep access to their terminal and configuration.
Small enough to stay alongside your main model
| SparkBotLLM | |
|---|---|
| Base | LiquidAI LFM3 1.7B |
| Download | Approximately 1.05 GB, Q4_0 GGUF |
| Default execution | Local CPU, four threads, one request at a time |
| Serving context | 4,096 tokens |
| Sparkbot runtime | 9.2 MB native CPU package; no Docker or PyTorch required |
| Helper endpoint | 127.0.0.1:10004 |
| Dashboard | Port 10005 |
CPU inference leaves GPU execution available for your main model. CPU work and unified-memory traffic still share the Spark’s resources. The dashboard caps its helper service at 3 GiB; that is a configured limit, not a claim of measured resident memory.
From a question to a useful next step
Inside Sparkbot, the assistant can use tools to inspect Docker support, GPU clocks, memory and connections; discover new models; explain deployment requirements; and propose terminal commands. Changes wait for owner approval. Administrator passwords are entered in the dashboard’s approval flow and are not sent to the model.
A typical journey is to check the Spark, resolve a missing dependency, select a bundled recipe, then monitor the running model and compare decode or prefill results. SparkBotLLM stays beside that workflow as local help.
The dashboard supplies these tools and controls. Running the model file alone gives you a chat model; it does not grant shell access, fetch current releases or deploy models by itself. Model inference runs locally. Downloads and online discovery contact their respective sources.
Run it today
Download SparkBotLLM-1.7B-Q4_0.gguf from Files and versions. It is public and needs no access request. Use a current LFM3-capable llama.cpp build:
llama-server \
--model SparkBotLLM-1.7B-Q4_0.gguf \
--alias SparkBotLLM \
--host 127.0.0.1 --port 10004 \
--ctx-size 4096 --parallel 1 --threads 4 \
--n-gpu-layers 0 --jinja \
--chat-template-kwargs '{"enable_thinking":false}'
The GGUF includes the chat template. Start with a 4K context and short, tool-assisted turns. The Sparkbot website package also includes a compact runtime and guided setup. sparkbot.dev is the planned home; its hosting and domain connection are being prepared. The model can already be downloaded and run independently.
What we adapted
SparkBotLLM received a short supervised LoRA adaptation focused on DGX Spark support, local inference and choosing the appropriate inspection tool. The training set contains 242 authored synthetic support examples, informed by 44 dated support references. It contains no private user chats or credentials.
Sparkbot also supplies relevant support references at runtime. This lets documentation evolve separately from the weights. Live hardware facts come from inspection tools; new software and model releases should come from current primary sources.
| Training detail | Value |
|---|---|
| Exact base revision | ac2586a34f1a1c03329dbdcdefabd8b6478ddaa7 |
| LoRA | Rank 8, alpha 16 |
| Optimizer | AdamW, learning rate 0.00005 |
| Loss | Assistant tokens only |
| Selected checkpoint | Step 128 of a 192-step run |
| Peak CUDA allocation during training | 3.59 GiB |
The adapter was merged into that exact BF16 base before Q4_0 conversion, with Q6_K token embeddings. Do not apply this adapter to LiquidAI’s separate QAD quantized checkpoint.
Checks and practical limits
Six domain validation examples were used for checkpoint selection. Mean loss fell from 4.62 to 3.32. Because they guided selection, these are validation results, not an independent final test. After Q4 export, five independent tool-routing smoke checks passed, covering hardware inspection, clocks, setup, recipes and model discovery. Reports are included in this repository.
These are narrow checks. They do not establish broad factual accuracy or reliable deployment automation. SparkBotLLM is a small model and can make mistakes. Keep command review enabled, verify important guidance, and use tools for live measurements. Long-context quality beyond the 4K serving profile has not been evaluated.
License and credits
SparkBotLLM is an adaptation of LiquidAI LFM3 1.7B, not a model trained from scratch. The retained LFM Open License v1.0 applies, including its commercial conditions.
LiquidAI supplies the base weights and architecture. PyTorch, Transformers and PEFT support training; llama.cpp supplies GGUF conversion and the compact CPU backend. Support references credit NVIDIA, Docker, TensorFold and the relevant model publishers. Sparkbot integration and supplied branding are by x00byte. Sparkbot is an independent project for DGX Spark owners, with no NVIDIA endorsement claimed.
- Downloads last month
- 1
4-bit