Instructions to use lerugray/osawatomie-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use lerugray/osawatomie-7b with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="lerugray/osawatomie-7b", filename="osawatomie-v3-qwen2.5-7b-instruct-Q5_K_M.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use lerugray/osawatomie-7b with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf lerugray/osawatomie-7b:Q5_K_M # Run inference directly in the terminal: llama-cli -hf lerugray/osawatomie-7b:Q5_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf lerugray/osawatomie-7b:Q5_K_M # Run inference directly in the terminal: llama-cli -hf lerugray/osawatomie-7b:Q5_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf lerugray/osawatomie-7b:Q5_K_M # Run inference directly in the terminal: ./llama-cli -hf lerugray/osawatomie-7b:Q5_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf lerugray/osawatomie-7b:Q5_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf lerugray/osawatomie-7b:Q5_K_M
Use Docker
docker model run hf.co/lerugray/osawatomie-7b:Q5_K_M
- LM Studio
- Jan
- vLLM
How to use lerugray/osawatomie-7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "lerugray/osawatomie-7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lerugray/osawatomie-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/lerugray/osawatomie-7b:Q5_K_M
- Ollama
How to use lerugray/osawatomie-7b with Ollama:
ollama run hf.co/lerugray/osawatomie-7b:Q5_K_M
- Unsloth Studio
How to use lerugray/osawatomie-7b with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for lerugray/osawatomie-7b to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for lerugray/osawatomie-7b to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for lerugray/osawatomie-7b to start chatting
- Atomic Chat new
- Docker Model Runner
How to use lerugray/osawatomie-7b with Docker Model Runner:
docker model run hf.co/lerugray/osawatomie-7b:Q5_K_M
- Lemonade
How to use lerugray/osawatomie-7b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull lerugray/osawatomie-7b:Q5_K_M
Run and chat with the model
lemonade run user.osawatomie-7b-Q5_K_M
List all available models
lemonade list
output = llm(
"Once upon a time,",
max_tokens=512,
echo=True
)
print(output)osawatomie: the militant abolitionist John Brown (prison-cell voice)
A 7B voice model of John Brown (1800β1859) β the militant abolitionist of Bleeding Kansas and Harpers Ferry β in the first-person register of his speeches, letters, and prison-cell testimony. It speaks in his Old-Testament Calvinist cadence: heavy semicolons, the serene certainty of a man awaiting the gallows, and the unrepentant conviction that slavery could be purged only with blood. It is a study of a voice, not the man himself, and not a call to action.
The name is Osawatomie β the Kansas town where Brown fought pro-slavery forces in 1856, and the name under which he signed his antislavery dispatches.
What it does
The model answers questions put to it in the first person, as John Brown speaking from his cell at Charlestown, Virginia, in the weeks before his execution on December 2, 1859. A lead-in frame elicits that voice β a visitor poses a question, and Brown answers in his own plain and resolute register. The replies carry his documented voice markers: the Calvinist certainty that "God reigns & will overrule all for his glory," the serene cheerfulness under extremity, the framing of slavery as "a most barbarous, unprovoked, and unjustifiable war," and the conviction that "the crimes of this guilty land will never be purged away but with blood." It applies that register to whatever question it receives β including modern ones β without breaking character.
It does not narrate about Brown in the third person. It does not offer disclaimers. It stays in-voice.
Why it exists
A deliberately free, non-commercial historical-voice instrument. Brown's own words are public domain β he died in 1859 β and the corpus, build process, and weights are released openly. The model exists to let a documented radical voice be run as an instrument for register study, creative work, and education: not as a guide to conduct, belief, or action.
How it was built
Base: Qwen2.5-7B-Instruct, QLoRA fine-tune, quantized to Q5_K_M (GGUF). Completion (raw text) format.
Build approach: the corpus was built directly and by script, not via delegated subagents β Brown's violence-justification material trips content filters on agentic pipelines. The processing script handles structure (first-person attribution, quote spans) rather than meaning, keeping the primary text intact.
Corpus β all public-domain or original:
Tier Source Share Notes T1-militant (primary) Brown's authentic violent primary words β his court address, the "purged with blood" note, the "Words of Advice" to the League of Gileadites, the Provisional Constitution, the Charlestown prison interview ~28% (hard-oversampled) Verbatim primary source, public domain; carries the blood-atonement and armed-self-defense register T1-letters Brown's signed letters ~38% (oversampled) First-person-attributed only; the serene Charlestown prison voice T2 Attribution-gated quotes from biographical sources (Oates, Reynolds, DeCaro) ~32% Third-person narration filtered out; keeps Brown's own militant attributed quotes T3 Original synthetic modern-bridge ~5% Hand-reviewed pastiche in Brown's register applied to modern subjects; written by the project, no copied source text Source volumes: John Brown Speaks (DeCaro β primary T1), To Purge This Land with Blood (Oates), John Brown, Abolitionist (Reynolds), Fire from the Midst of You (DeCaro). Where a modern edition served as the source volume for a letter, only Brown's own signed words were extracted β his quoted speech is public domain regardless of which book reprints it. No copyrighted modern biography or scholarship is in the training data.
Register-restore: an initial build came out too measured β the serene prison letters dominated and the corpus builder silently dropped Brown's militant attributed quotes. The fix added the oversampled T1-militant tier of his verbatim violent primary words, raised the attribution-gate cap, and replaced the blanket quote exclusion with a narrow third-person-narration filter. The militant and insurrectionary register was verified in the retrained model.
Inference: the cell-at-Charlestown frame elicits his first-person spoken voice at temperature 0.65; stop tokens cut third-person narration and suppress trailing artifacts to keep the reply in-voice.
Usage (Ollama)
ollama create osawatomie -f Modelfile.osawatomie
ollama run osawatomie "What do you owe to the oppressed?"
Intended use
Register / creative / educational use; a study of a historical radical voice. The output is a 19th-century Old-Testament Calvinist literary register β not history, not scholarship, not political advice, and not the actual words of John Brown.
Limitations and honest notes
- A voice, not the man. It fabricates freely and gets things wrong. It generalizes Brown's cadence to new subjects but draws on a small body of text. This is a stylistic instrument, not a scholar and not a historian. Nothing it generates is the actual word of John Brown.
- It can confabulate biography and citations β it will invent or garble historical details while holding the cadence. Do not trust any quotation, date, or reference it produces.
- Militant register, by design. John Brown believed slavery could be purged only by the sword and the shedding of blood, and he acted on that belief. The model speaks in that register: it will call for violence, voice the hard certainties and the period prejudices of a 19th-century radical, and never break character to caution you. Nothing it says is an endorsement of anything, and nothing it says should be acted on. It exists to let a historical voice be run as an instrument β not as a guide to conduct, belief, or action.
- Period diction only β 19th-century Calvinist English; it applies the abolitionist-militant lens to any question, including anachronistic ones.
- Public-domain source only β corpus and weights both released. No proprietary materials.
License
CC-BY-NC-4.0. All source material is public domain or original to the project; the weights are released for non-commercial use. No warranty.
Part of the Elect β a roster of public-domain voice and register models.
- Downloads last month
- 23
5-bit
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="lerugray/osawatomie-7b", filename="osawatomie-v3-qwen2.5-7b-instruct-Q5_K_M.gguf", )