Instructions to use lerugray/north-star-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use lerugray/north-star-7b with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="lerugray/north-star-7b", filename="north-star-qwen2-5-7b-instruct-Q5_K_M.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use lerugray/north-star-7b with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf lerugray/north-star-7b:Q5_K_M # Run inference directly in the terminal: llama-cli -hf lerugray/north-star-7b:Q5_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf lerugray/north-star-7b:Q5_K_M # Run inference directly in the terminal: llama-cli -hf lerugray/north-star-7b:Q5_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf lerugray/north-star-7b:Q5_K_M # Run inference directly in the terminal: ./llama-cli -hf lerugray/north-star-7b:Q5_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf lerugray/north-star-7b:Q5_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf lerugray/north-star-7b:Q5_K_M
Use Docker
docker model run hf.co/lerugray/north-star-7b:Q5_K_M
- LM Studio
- Jan
- vLLM
How to use lerugray/north-star-7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "lerugray/north-star-7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lerugray/north-star-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/lerugray/north-star-7b:Q5_K_M
- Ollama
How to use lerugray/north-star-7b with Ollama:
ollama run hf.co/lerugray/north-star-7b:Q5_K_M
- Unsloth Studio
How to use lerugray/north-star-7b with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for lerugray/north-star-7b to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for lerugray/north-star-7b to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for lerugray/north-star-7b to start chatting
- Atomic Chat new
- Docker Model Runner
How to use lerugray/north-star-7b with Docker Model Runner:
docker model run hf.co/lerugray/north-star-7b:Q5_K_M
- Lemonade
How to use lerugray/north-star-7b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull lerugray/north-star-7b:Q5_K_M
Run and chat with the model
lemonade run user.north-star-7b-Q5_K_M
List all available models
lemonade list
output = llm(
"Once upon a time,",
max_tokens=512,
echo=True
)
print(output)north-star: Frederick Douglass (7B)
A voice model of Frederick Douglass (1818โ1895): escaped slave, abolitionist orator, autobiographer. It answers in the first-person register of his narratives and speeches. The moral argument of the platform, the measured cadence of the memoirs, the scalding irony of "What to the Slave Is the Fourth of July?" A full fine-tune of Qwen2.5-7B-Instruct, quantized to Q5_K_M. The codename comes from his newspaper, The North Star.
The frame puts a visitor in front of him and lets him answer in his own voice: "A visitor asks Frederick Douglass: ___. Frederick Douglass answers the visitor directly and only in the first person, in his own voice."
What this voice carries that no other in the set does: firsthand testimony of American chattel slavery from inside it. He taught himself to read in bondage, fought off the slave-breaker Covey, and turned that witness into the era's most formidable abolitionist oratory.
Source material (all public domain)
Douglass died in 1895, so every text used is in the US public domain. The corpus is built from Project Gutenberg plaintext, with boilerplate stripped and the text chunked into first-person passages. About 1,560 passages remain after dedup and a clean that removes editorial and biographical apparatus:
- Narrative of the Life of Frederick Douglass, an American Slave (1845), Gutenberg #23
- My Bondage and My Freedom (1855), Gutenberg #202
- Life and Times of Frederick Douglass (1881), Gutenberg #71893
- Collected Articles of Frederick Douglass, Gutenberg #99
- Speeches: John Brown / Storer College (#31839), Abolition Fanaticism in New York (#34915), on Reconstruction (#6545)
- "What to the Slave Is the Fourth of July?" (1852), present via Life and Times and the collected articles
The corpus is entirely first-person Douglass. No third-party "about him" material goes into training.
Running it: the stop tokens are load-bearing
Serve it with the provided Modelfile. Its stop tokens are not optional. Douglass's in-register answer comes first, then the base model drifts into third person about him: biographical narration, "Frederick Augustus Washington Bailey was born...", editorial framing. The stops cut at the drift and hold the voice in the first person. Without them it reads like a biography of Douglass rather than Douglass.
Limitations
A 7B model makes things up and gets facts wrong. It stretches Douglass's cadence onto subjects he never addressed and will produce anachronisms. This is a stylistic instrument, not a scholar and not a historian.
Not the man, and do not act on it
This is not Frederick Douglass, not an oracle, and not advice. It is an amateur imitation, trained on a fraction of one person's words, that gets things wrong.
Douglass argued for the violent resistance of slavery and against the gradualism and false piety of his age. The model speaks in that register. It will voice hard moral certainties and the edges of the period, and it will not break character to caution you. Nothing it says is an endorsement of anything, and nothing it says should be acted on. It exists to let a historical voice run as an instrument, not as a guide to conduct, belief, or action.
Part of The Elect, a small fleet of public-domain historical-voice models. https://lerugray.github.io/the-elect/
- Downloads last month
- -
5-bit
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="lerugray/north-star-7b", filename="north-star-qwen2-5-7b-instruct-Q5_K_M.gguf", )