Instructions to use lerugray/the-burrow-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lerugray/the-burrow-7b with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="lerugray/the-burrow-7b",
	filename="the-burrow-7b-Q5_K_M.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use lerugray/the-burrow-7b with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf lerugray/the-burrow-7b:Q5_K_M
# Run inference directly in the terminal:
llama-cli -hf lerugray/the-burrow-7b:Q5_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf lerugray/the-burrow-7b:Q5_K_M
# Run inference directly in the terminal:
llama-cli -hf lerugray/the-burrow-7b:Q5_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf lerugray/the-burrow-7b:Q5_K_M
# Run inference directly in the terminal:
./llama-cli -hf lerugray/the-burrow-7b:Q5_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf lerugray/the-burrow-7b:Q5_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf lerugray/the-burrow-7b:Q5_K_M

Use Docker

docker model run hf.co/lerugray/the-burrow-7b:Q5_K_M

LM Studio
Jan

vLLM

How to use lerugray/the-burrow-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lerugray/the-burrow-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lerugray/the-burrow-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lerugray/the-burrow-7b:Q5_K_M

Ollama
How to use lerugray/the-burrow-7b with Ollama:
```
ollama run hf.co/lerugray/the-burrow-7b:Q5_K_M
```

Unsloth Studio

How to use lerugray/the-burrow-7b with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lerugray/the-burrow-7b to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lerugray/the-burrow-7b to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for lerugray/the-burrow-7b to start chatting

Atomic Chat new
Docker Model Runner
How to use lerugray/the-burrow-7b with Docker Model Runner:
```
docker model run hf.co/lerugray/the-burrow-7b:Q5_K_M
```

Lemonade

How to use lerugray/the-burrow-7b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull lerugray/the-burrow-7b:Q5_K_M

Run and chat with the model

lemonade run user.the-burrow-7b-Q5_K_M

List all available models

lemonade list

the-burrow — Franz Kafka (7B, two registers)

A voice model in two registers of Franz Kafka (1883–1924), frame-selected from one set of weights. Kafka was a German-speaking Bohemian novelist and short story writer, widely regarded as one of the major figures of 20th-century literature — the author of The Trial, The Metamorphosis, and The Castle, works in which the impossible is reported as administrative routine. He asked his friend Max Brod to burn his unpublished manuscripts after his death. Brod did not.

The codename comes from Der Bau ("The Burrow") — the anxious, self-enclosing construction that is the register of the man and the work both. The conceit is the gap that defined him: the man who wrote the most precise nightmares in modern literature, and asked that they be destroyed. This is a study of a voice, not the real person and not an oracle.

What it does

One model, two lead-in frames. The frame chooses the register.

Mode A — "the man." The first-person epistolary and diaristic voice: anxious, recursive, self-undermining. When prompted through the Mode A frame, the model responds as if Franz Kafka, in a letter written late at night and not certain he will send it, is setting down what he thinks — the exhausted formal address to an implacable authority, each assertion qualified and then undermined by the next. It draws on Kafka's letters, diaries, and the Letter to His Father: his actual anxieties, opinions, and recursive self-doubt.

Mode B — "the work." The affectless fiction register: the impossible reported as administrative routine, guilt without crime, infinite deferral, the gate meant only for you. When prompted through the Mode B frame, the model responds as if recovered from an unfinished manuscript — a matter that proves, on examination, to be not so simple as it first appeared. It draws on the short fiction and the novels.

It does not assist. It defers.

Why it exists

The register — the burrow, the anxious endlessly-revised construction — is worth being able to run as an instrument. This is a deliberately free model: corpus and weights both built from public-domain or freely-dedicated sources and released openly.

How it was built

Base: Qwen2.5-7B-Instruct. QLoRA fine-tune, quantized to Q5_K_M. One GGUF, two Modelfiles (the-burrow-man, the-burrow-work).
Two frame-selected modes in one model: the corpus is mode-tagged, and the lead-in frame at inference selects which register the model speaks in. If the registers bleed at evaluation, the fallback design is two separate models; as shipped (v2), both modes are voice-verified distinct and generative.
Status: v2. Mode A now draws on Kafka's diaries and his letters to Felice Bauer and Milena Jesenská alongside the Letter to His Father, widening its range beyond the v1 single-letter keystone. See the build repo's SCOPE.md for the full design, corpus plan, and IP audit.
Corpus — all public-domain or free-use (2,577 records, ~53% "the man" / ~46% "the work"):
- Ian Johnston's translations (explicit public-domain dedication) — the register spine for Mode B short fiction.
- David Wyllie / Project Gutenberg — The Metamorphosis, The Trial.
- Our own translations of Kafka's public-domain German — Brief an den Vater (Letter to His Father), the diaries (Tagebücher), and the letters to Felice Bauer and Milena Jesenská — the Mode A corpus, translated with a style-preserving prompt that refuses to smooth Kafka's parataxis.
- Kafka's original German is public domain everywhere; his US copyrights have lapsed by first-publication year. Copyrighted English translations (Muir, Mitchell, Harman, Corngold, Hofmann, Benjamin, Crick) were never in the corpus. A leakage test gates any public ship.
Inference: a lead-in frame elicits the chosen register; two Ollama Modelfiles read the one GGUF. Recommended num_ctx 2048.

Usage (Ollama)

ollama create the-burrow-man -f Modelfile.the-burrow-man
ollama run the-burrow-man "What do you owe your father?"

ollama create the-burrow-work -f Modelfile.the-burrow-work
ollama run the-burrow-work "Describe the committee that will decide your case."

Intended use

Register / creative / educational use; a study of a historical literary voice. The output is a stylistic imitation — not the actual words of Franz Kafka, not scholarship, and not advice.

Limitations and honest notes

A voice, not the man. It fabricates freely. Nothing it generates is the actual writing or opinion of Franz Kafka. It is a model of a register.
A 7B model still invents and errs. This is a stylistic instrument, not a scholar. It can confabulate plot details, character names, and biographical facts while holding the cadence. Do not trust any quotation or reference it produces.
Two registers, one model. Both modes are voice-verified distinct as shipped (no register bleed), but edge-case prompts may pull the wrong register. The fallback design — two separate models — exists if needed.
Self-translated source. The Mode A corpus relies on our own translations of the public-domain German, which are amateur and style-preserving rather than scholarly.
Not a way to reach the dead. This is an amateur imitation that gets things wrong, trained on a fraction of one writer's work. Nothing it says should be acted on.

License

Apache-2.0. All source material is public domain or freely dedicated; the weights are released openly. No warranty.

Part of The Elect — a small fleet of public-domain historical-voice models.

Downloads last month: 261

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

5-bit

Model tree for lerugray/the-burrow-7b

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-7B-Instruct

Quantized

(341)

this model