Instructions to use lerugray/nyarlathotep-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lerugray/nyarlathotep-7b with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="lerugray/nyarlathotep-7b",
	filename="nyarlathotep-qwen2-5-7b-instruct-Q5_K_M.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use lerugray/nyarlathotep-7b with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf lerugray/nyarlathotep-7b:Q5_K_M
# Run inference directly in the terminal:
llama-cli -hf lerugray/nyarlathotep-7b:Q5_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf lerugray/nyarlathotep-7b:Q5_K_M
# Run inference directly in the terminal:
llama-cli -hf lerugray/nyarlathotep-7b:Q5_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf lerugray/nyarlathotep-7b:Q5_K_M
# Run inference directly in the terminal:
./llama-cli -hf lerugray/nyarlathotep-7b:Q5_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf lerugray/nyarlathotep-7b:Q5_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf lerugray/nyarlathotep-7b:Q5_K_M

Use Docker

docker model run hf.co/lerugray/nyarlathotep-7b:Q5_K_M

LM Studio
Jan

vLLM

How to use lerugray/nyarlathotep-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lerugray/nyarlathotep-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lerugray/nyarlathotep-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lerugray/nyarlathotep-7b:Q5_K_M

Ollama
How to use lerugray/nyarlathotep-7b with Ollama:
```
ollama run hf.co/lerugray/nyarlathotep-7b:Q5_K_M
```

Unsloth Studio

How to use lerugray/nyarlathotep-7b with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lerugray/nyarlathotep-7b to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lerugray/nyarlathotep-7b to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for lerugray/nyarlathotep-7b to start chatting

Atomic Chat new
Docker Model Runner
How to use lerugray/nyarlathotep-7b with Docker Model Runner:
```
docker model run hf.co/lerugray/nyarlathotep-7b:Q5_K_M
```

Lemonade

How to use lerugray/nyarlathotep-7b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull lerugray/nyarlathotep-7b:Q5_K_M

Run and chat with the model

lemonade run user.nyarlathotep-7b-Q5_K_M

List all available models

lemonade list

nyarlathotep: a cosmic-horror register oracle

A 7B voice tune that writes in the register of cosmic horror: the indifferent cosmos, forbidden knowledge that unmakes the mind, the unnameable, the gulf between the stars. The conceit is Nyarlathotep, the Crawling Chaos, the messenger of the Outer Gods who walks among men and discloses what waits in the dark.

It channels the dread, not the author. The model trains on H. P. Lovecraft's cosmic-horror fiction, which is public domain, but it does not reconstruct Lovecraft the person. Two filters strip his documented racism from the corpus: stories whose premise is the bigotry are dropped whole, and every remaining passage is scanned and dropped if it carries a slur or racial-fear vocabulary. What survives is the atmosphere.

What it does

Ask it anything and it answers from the abyss. It will not comfort you; it discloses. Asked a mundane question, it cannot describe a night sky without the gods shaping cosmos from primal chaos, or an old house without the wrong angles and the thing in the walls. Asked what it is, it answers as the entity of a thousand guises.

How it was built

Base: Qwen2.5-7B-Instruct, full fine-tune.
Format: completion (raw text), so the dread comes from the source prose rather than from instruction scaffolding.
Source: Lovecraft's public-domain fiction, racism-filtered (title blocklist + a per-passage slur/term filter), chunked to roughly 2,000 completion records.
Inference: a lead-in frame elicits the oracle voice. Plain chat suppresses the register; the name-cue lead-in makes the model disclose as the entity.

Intended use

Creative writing, atmosphere generation, tabletop and interactive fiction, games. The output is a horror-fiction register. It produces dread on purpose and offers no facts, advice, or reassurance.

Limitations and honest notes

It is a register, not a narrator with continuity. It invents names, places, and cosmologies freely. Treat everything it says as fiction.
Filtered, not sanitized of horror. The racism filters target Lovecraft's bigotry, not the cosmic dread. The model is built to disturb. The deity name "Shub-Niggurath" is a canonical mythos term, not a slur, and survives the word-boundary filter by design.
Period diction. It writes in the archaic, ornate cadence of the old weird tale.

License

Apache-2.0. The source fiction is public domain and the base model is Apache-2.0, so the weights ship under a permissive license. No warranty.

Part of The Elect — a small fleet of public-domain historical-voice models. https://lerugray.github.io/the-elect/

Downloads last month: 23

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

5-bit

Model tree for lerugray/nyarlathotep-7b

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-7B-Instruct

Quantized

(341)

this model