JiRack Ternary Precision 36B (GGUF)
JiRack Ternary Precision 36B is a ternary (1.58-bit) language model by CMSManhattan (Konstantin Grabko) with 35.78B parameters, built for coding, tool calling and agents. Every linear layer of the transformer uses ternary weights {-1, 0, +1} with a per-tensor scale and 8-bit activations (BitNet b1.58 style), trained with quantization-aware training.
This repository has the model as GGUF for llama.cpp: about @SIZE_TQ2@ instead of 71.6 GB in bf16. It runs on a CPU with enough RAM, or on one consumer GPU.
It is in development yet
Files
| File | Size | Notes |
|---|---|---|
JiRackTernaryPrecision-36B-TQ2_0.gguf |
@SIZE_TQ2@ | recommended, ternary weights stored natively |
JiRackTernaryPrecision-36B-Q8_0.gguf |
@SIZE_Q8@ | for llama.cpp builds without TQ2_0 kernels on your hardware |
chat_template.jinja |
— | JiRack chat format, including tools |
Run with llama.cpp
llama-server -m JiRackTernaryPrecision-36B-TQ2_0.gguf -c 16384 --jinja --port 8080
The server is OpenAI-compatible (http://localhost:8080/v1/chat/completions). Pass tools=[...] in OpenAI format. The model calls tools with one or more blocks:
<|tool_call_start|>
{"name": "get_weather", "arguments": {"city": "Paris"}}
<|tool_call_end|>
Tool results go back as role: "tool" messages.
Model details
| Parameters | 35.78B |
| Layers | 64 |
| Hidden size / FFN | 5,120 / 30,720 |
| Attention heads | 40 query, 8 key-value (GQA), head dim 128 |
| Vocabulary | 152,064 |
| Context | up to 32k tokens |
| Weights | ternary {-1, 0, +1}, per-tensor scale |
| Activations | 8-bit, per token |
Intended use
Coding assistants, tool calling and agent pipelines, local and offline deployment where memory is limited.
Limitations
The model can make mistakes, invent facts and produce wrong tool arguments. Check its output before acting on it, especially in automated pipelines.
License
Proprietary, © 2025–2026 Konstantin Vladimirovich Grabko (CMSManhattan). The base weights come from Qwen2.5-32B (Apache 2.0) via DeepSeek-R1-Distill-Qwen-32B (MIT). See NOTICE.md for the third-party licenses.
Contact
CMSManhattan — Konstantin Grabko, Hugging Face: kgrabko