MODEL CARD DUPLICATED FROM THE ORIGINAL
# recall-honcho-8b-4bit-mlx — 4-bit MLX Version for Apple
Silicon
This repository contains the **4-bit MLX quantized weights** of
[dman1011/recall-honcho-8b](https://huggingface. co/dman1011/recall-honcho-8b), optimized for high-performance inference on Apple Silicon (M1/M2/M3/M4) using MLX and Exo.
## 🚀 How to Use
### 1. Run with `mlx-lm` CLI
```bash
pip install mlx-lm
mlx_lm.generate --model rickix84/recall-honcho-8b-4bit-mlx --
prompt "Hello!"
2. Run with Exo Cluster
exo run rickix84/recall-honcho-8b-4bit-mlx
3. Use in Python (MLX)
from mlx_lm import load, generate
model, tokenizer = load("rickix84/recall-honcho-8b-4bit-mlx")
response = generate(model, tokenizer, prompt="Hello!",
verbose=True) ────── About the Base Model: recall-honcho-8b
A Qwen/Qwen3-8B fine-tune specialised for the explicit conclusion-derivation step of Honcho https://github.com/plastic-labs/honcho: given a target peer and their chat turns, emit atomic, self-contained, correctly-attributed facts as Honcho-schema JSON. Independent, self-hosted re-creation of the role Plastic Labs' (closed) Neuromancer XR plays in production Honcho. Not affiliated with Plastic Labs.
Results
Held-out validation (298 examples, 4% split), bf16 LoRA, 2 epochs:
| Metric | Value |
|---|---|
| Eval loss | 0.201 |
| Eval token accuracy | 93.8% |
| Train loss (final) | 0.197 |
| Loss curve | 2.03 → 0.13 over 896 steps |
The model reliably reproduces the exact {"explicit":[{"content":...}]} schema with correct attribution, absolute dates, and atomic facts. Worked example (held-out):
Input : message about buying a secondhand Eames lounge chair
Output : {"explicit": [
{"content": "dmitri bought a secondhand Eames lounge chair
from a Facebook listing in Williamsburg"}, {"content": "dmitri paid $900 for the secondhand Eames lounge chair"}, {"content": "dmitri rented a Zipcar to haul the Eames lounge chair back, costing another $80"} ]}
Training
• Base: Qwen/Qwen3-8B • Data: 7,160 synthetic SFT examples, gold labels distilled from Claude Opus 4.8 (frontier teacher), 15 balanced life-domains, rendered through Honcho's own prompt-builder and schema-validated against PromptRepresentation. Distilling a compact deriver from a frontier model is the core idea. • Method: bf16 LoRA (rank 32, alpha 32), 2 epochs, lr 2e-4 cosine, completion-only loss. • Hardware: NVIDIA RTX PRO 6000 Blackwell (sm_120), ~56 min.
Prompt / I-O contract
Input = Honcho's minimal_deriver_prompt(peer_id, messages) verbatim (non-thinking). Output = JSON validated by PromptRepresentation:
{"explicit": [{"content": "alice is training for a half-
marathon scheduled for October 2026"}]}
Rules: atomic · self-contained · absolute dates · correct third- party attribution · explicit-only (no speculation) · {"explicit": []} when nothing is stated.
How to use with Honcho
Serve (vLLM/Ollama), expose via your gateway (e.g. litellm route recall-honcho-8b), then point Honcho's deriver at it in config.toml:
[deriver.model_config]
transport = "openai"
model = "recall-honcho-8b"
[deriver.model_config.overrides]
base_url = "http://<gateway>/v1"
api_key_env = "LLM_OPENAI_API_KEY"
Keep the dialectic/generation step on a larger model.
Limitations
• Explicit level only (not deductive/inductive/abductive — Honcho's dreamer). • English only; trained on privacy-safe synthetic data (frontier- distilled). • Optimised for Honcho's exact prompt; off-format prompts may degrade output discipline.
Acknowledgement
Inspired by Plastic Labs' Honcho and Neuromancer research. Independent synthetic re-creation; not affiliated with or endorsed by Plastic Labs.
4-bit