SiriusLLM/Sirius-N4-GGUF

Sirius N4 (GGUF, Preview)

Sirius N4 is a small German/English chat assistant by SiriusLLM. It is a fine-tune of NVIDIA Nemotron Mini 4B Instruct, trained to answer in German, write cleaner code and introduce itself as Sirius N4. It runs fully offline on laptops (about 3 GB RAM/VRAM for Q4).

Preview. Sirius N4 is our first experiment. It is small and makes mistakes, especially with facts and longer code. Our next model, Sirius N4.1 (based on Qwen3.5-4B, Apache-2.0), is in training.

Deutsch: Sirius N4 ist ein kleiner, lokal laufender Chat-Assistent auf Deutsch und Englisch. Er basiert auf NVIDIA Nemotron Mini 4B und wurde von SiriusLLM für Deutsch, Code und eine eigene Identität weitertrainiert. Vorschau-Version.

Files

File	Quant	Size	Notes
Sirius-N4-Chat-Q4_K_M.gguf	Q4_K_M	2.7 GB	recommended, ~40 tok/s on a GTX 1070
Sirius-N4-Chat-Q5_K_M.gguf	Q5_K_M	3.1 GB	slightly better quality
Sirius-N4-Chat-Q8_0.gguf	Q8_0	4.5 GB	closest to the original weights

All files include the chat template with a default Sirius system prompt. Context length: 4,096 tokens (do not increase it).

Usage

LM Studio: search for SiriusLLM/Sirius-N4-GGUF, download a file, start a chat. Recommended: temperature 0.3-0.5, repeat penalty 1.08.

Ollama (one command):

ollama run hf.co/SiriusLLM/Sirius-N4-GGUF:Q4_K_M

Chat template, system prompt and parameters are picked up automatically from this repo.

Ollama (offline / custom): download a GGUF and the Modelfile from this repo into one folder, then:

ollama create sirius-n4 -f Modelfile

ollama run sirius-n4

llama.cpp:

llama-cli -m Sirius-N4-Chat-Q4_K_M.gguf -cnv --temp 0.4 --repeat-penalty 1.08 -c 4096

What changed compared to Nemotron Mini 4B

Small internal comparison (6 prompts, greedy decoding, same system prompt). This is a spot check, not a benchmark.

Test	Nemotron Mini 4B	Sirius N4
Held-out loss on our mixed eval set (30 examples)	1.23	0.81
Answers in German when asked in German (2 prompts)	0/2	2/2
Python task (palindrome check)	incorrect	correct
TypeScript task (debounce)	broken	mostly correct
Tool call (weather)	correct	correct
Introduces itself as Sirius	no	yes

Standard benchmarks (e.g. German MMLU, HumanEval+) are not run yet.

Limitations

• Small 4B model: German can be clumsy, facts may be invented, longer programs (>30 lines) often contain errors.
• Can repeat earlier answers in longer chats. A repeat penalty of ~1.1 and keeping only the last few turns helps.
• Strong bias towards German: it often keeps answering in German even when asked to switch to English.
• May invent details about itself or SiriusLLM.
• Not suitable for agent frameworks with long system prompts (e.g. LM Studio Bionic), because the context is only 4,096 tokens.
• Not evaluated for safety-critical use. Always check generated code.

Training

• Method: QLoRA (rank 32) on a GTX 1070, 1 epoch, 216 steps, sequence length 2,048, then merged into the base model.
• Data (about 3,450 examples):

Source	Purpose	License	Note
FreedomIntelligence/alpaca-gpt4-deutsch	German instructions	Apache-2.0	responses generated with GPT-4
ise-uiuc/Magicoder-OSS-Instruct-75K	coding	MIT	generated with GPT-3.5
NousResearch/hermes-function-calling-v1	tool calling	Apache-2.0
Sirius identity examples	identity	own data	written by SiriusLLM

License and notices

• This model is a modified version of NVIDIA Nemotron Mini 4B Instruct. Modifications (fine-tuning, chat template and default system prompt) were made by SiriusLLM. It is not an original NVIDIA model.
• Licensed by NVIDIA Corporation under the NVIDIA Community Model License (see LICENSE.pdf). Please read it before use. Note its terms for production use.
• Datasets are credited above under their respective licenses.