Spark-X2.6-1.7B-Abliterated-GGUF (iMatrix Quantizations)

This repository provides Importance Matrix (imatrix) calibrated GGUF quantizations of OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated (an uncensored variant of XHToken/Spark-X2.6-1.7B abliterated with Heretic v2.0.0.dev0).

All quantized weights were calibrated using a 2,000-sample multi-domain dataset (spark_2000_mixed_dataset.jsonl) formatted strictly in the native Spark-X2.6 ChatML template (covering uncensored instruction following, coding, mathematics, science, and roleplay) to maximize perplexity retention and reasoning quality at low bit-widths.

---

Available GGUF Files

File Name	Quant Type	iMatrix	Notes
Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf	Q4_K_M	Yes	Best balance of speed, low RAM/VRAM usage, and accuracy
Spark-X2.6-1.7B-Abliterated-Q5_K_M-imatrix.gguf	Q5_K_M	Yes	High-quality 5-bit quantization with minimal perplexity loss
Spark-X2.6-1.7B-Abliterated-Q6_K-imatrix.gguf	Q6_K	Yes	Near-lossless 6-bit quantization
Spark-X2.6-1.7B-Abliterated-Q8_0-imatrix.gguf	Q8_0	Yes	Extreme precision 8-bit quantization
Spark-X2.6-1.7B-Abliterated-F16.gguf	F16	N/A	Full 16-bit unquantized GGUF weights
Spark-X2.6-1.7B-Abliterated-imatrix.dat	imatrix	Yes	Precomputed Importance Matrix calibration data

---

Prompt Format (Spark-X2.6 ChatML)

Spark-X2.6-1.7B uses a custom prompt structure with configurable thinking effort (low or high):

<｜start▁of▁sentence｜><|System|>
you are a helpful assistant.
`thinking_effort` guides on how much to think in your thinking channel (not including the response channel), supported values include `low` and `high`.
Now the system is invoked with `thinking_effort=high`.<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|User|>{prompt}<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|Bot|><think>

How to Use

1. Using llama-cli (llama.cpp)

./llama-cli \
  -m Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf \
  -ngl 99 \
  -c 4096 \
  --temp 0.3 \
  --top-p 0.9 \
  --top-k 50 \
  --repeat-penalty 1.1 \
  -cnv

2. Using llama-server (OpenAI-Compatible Local API)

./llama-server \
  -m Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf \
  -ngl 99 \
  -c 4096 \
  --port 8080

3. Using llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated-GGUF",
    filename="Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf",
    n_gpu_layers=-1,
    n_ctx=4096
)

response = llm.create_chat_completion(
    messages=[
        {"role": "user", "content": "Explain the difference between stack memory and heap memory in 3 concise bullet points."}
    ],
    temperature=0.3,
    top_p=0.9,
    top_k=50,
    max_tokens=512
)

print(response["choices"][0]["message"]["content"])