Text Generation
PEFT
Safetensors
GGUF
English
q4_k_m
docker-model-runner
lora
codegeist-training
conversational
Instructions to use codegeist/codegeist-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use codegeist/codegeist-llm with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B") model = PeftModel.from_pretrained(base_model, "codegeist/codegeist-llm") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use codegeist/codegeist-llm with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: llama cli -hf codegeist/codegeist-llm:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: llama cli -hf codegeist/codegeist-llm:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf codegeist/codegeist-llm:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf codegeist/codegeist-llm:Q4_K_M
Use Docker
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use codegeist/codegeist-llm with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "codegeist/codegeist-llm" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "codegeist/codegeist-llm", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- Ollama
How to use codegeist/codegeist-llm with Ollama:
ollama run hf.co/codegeist/codegeist-llm:Q4_K_M
- Unsloth Studio
How to use codegeist/codegeist-llm with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for codegeist/codegeist-llm to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for codegeist/codegeist-llm to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for codegeist/codegeist-llm to start chatting
- Pi
How to use codegeist/codegeist-llm with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "codegeist/codegeist-llm:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use codegeist/codegeist-llm with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "codegeist/codegeist-llm:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use codegeist/codegeist-llm with Docker Model Runner:
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- Lemonade
How to use codegeist/codegeist-llm with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull codegeist/codegeist-llm:Q4_K_M
Run and chat with the model
lemonade run user.codegeist-llm-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use codegeist/codegeist-llm with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default codegeist/codegeist-llm:Q4_K_M
Run Hermes
hermes
- Atomic Chat
| { | |
| "schema_version": 2, | |
| "evidence_type": "codegeist-training-stage", | |
| "recorded_date": "2026-08-08", | |
| "result": "passed", | |
| "scope": { | |
| "purpose": "Establish the first approved Codegeist training record and validate BF16 LoRA training, clean-process reload, versioned promotion, and public attribution handling.", | |
| "learned_answer": "Codegeist is a coding agent created by René Schmidt.", | |
| "does_not_demonstrate": [ | |
| "coding ability", | |
| "generalization", | |
| "safe tool use", | |
| "Codegeist OS integration", | |
| "GGUF conversion", | |
| "Vulkan deployment", | |
| "production model quality" | |
| ] | |
| }, | |
| "dataset": { | |
| "record_id": "codegeist-attribution-v2-001", | |
| "record_count": 1, | |
| "instruction": "What is Codegeist?", | |
| "response": "Codegeist is a coding agent created by René Schmidt.", | |
| "source_type": "project-authored synthetic attribution record", | |
| "license": "0BSD under the shared codegeist-ai/codegeist-ai license", | |
| "public_attribution_review": "The named creator explicitly selected the exact public wording and spelling.", | |
| "contains_contact_data": false, | |
| "contains_credentials": false, | |
| "train_evaluation_overlap": "The first-stage exact-response check deliberately reuses the training record; later capability stages require a held-out split.", | |
| "loss_scope": "completion_only" | |
| }, | |
| "source": { | |
| "source_committed_at_launch": false, | |
| "canonical_source_identity": "sha256", | |
| "source_sha256": { | |
| "pyproject.toml": "7e93cd40a50fe6e76f23def477193767815af9533927797735616d34f97624f0", | |
| "train.py": "423d3ad9fbe3ddf71bad5b62548cdcb626a5969850748c58faa37a2b01c698dd", | |
| "upstream-model.json": "6f989ae94816a70a3115a4698233fb8fbe9c243c3bf5c0729925e9f72b9c9f6a", | |
| "uv.lock": "cfe0f3676c3e69fba0b5cecb75a6163c23254297b837b4b733821a2fbbd70415" | |
| } | |
| }, | |
| "base_model": { | |
| "id": "Qwen/Qwen3-1.7B", | |
| "revision": "70d244cc86ccca08cf5af4e1e306ecf908b1ad5e", | |
| "license": "apache-2.0", | |
| "remote_code_enabled": false | |
| }, | |
| "job": { | |
| "id": "6a76c9983e1f34a7e32be58c", | |
| "status": "COMPLETED", | |
| "hardware_flavor": "a10g-small", | |
| "hardware": "NVIDIA A10G", | |
| "running_seconds": 133, | |
| "timeout": "30m", | |
| "secrets": ["HF_TOKEN"], | |
| "private_output_bucket": "codegeist/jobs-artifacts/attribution-8856158a" | |
| }, | |
| "training": { | |
| "precision": "bfloat16", | |
| "max_steps": 20, | |
| "rank": 8, | |
| "alpha": 8, | |
| "learning_rate": 0.0002, | |
| "seed": 3407, | |
| "aggregate_loss": 2.494612373970449, | |
| "final_logged_step_loss": 0.01821, | |
| "duration_seconds": 89.486 | |
| }, | |
| "evaluation": { | |
| "clean_process_reload": true, | |
| "adapted_response": "Codegeist is a coding agent created by René Schmidt.", | |
| "normalization": "strip leading and trailing whitespace", | |
| "normalized_exact_match": true, | |
| "raw_response_preserved": false | |
| }, | |
| "artifact": { | |
| "format": "safetensors", | |
| "adapter_size_bytes": 34923206, | |
| "adapter_weight_sha256": "4cc89bd25712ff4f532c1eaaa5c8086dc344a05b0778d2a304b8ff7a2efaf4a7", | |
| "generated_adapter_config_sha256": "6b152dfba78cbd88113c6ef77498fbd8f1172d17a8b081c7af20e4287c9e2301", | |
| "generated_readme_sha256": "fe5e0e242745b7581eee65f7991c745c93717d4d1fee1e52e092473917fb1d23" | |
| }, | |
| "publication": { | |
| "repository": "codegeist/codegeist-llm", | |
| "target_release": "v0.2.1", | |
| "adapter_artifact_revision": "a9504a0ee1150ea05f88ff725758404fcb604a32", | |
| "anonymous_gpu_reload_passed": true, | |
| "anonymous_gpu_reload": { | |
| "hardware": "NVIDIA RTX A2000 12GB", | |
| "device": "cuda", | |
| "base_model_dtype": "bfloat16", | |
| "all_floating_parameters_bfloat16": true, | |
| "all_parameters_on_cuda": true, | |
| "all_buffers_on_cuda": true, | |
| "peak_cuda_memory_bytes": 3511419904, | |
| "duration_seconds": 10.726, | |
| "raw_response": "Codegeist is a coding agent created by René Schmidt.", | |
| "normalized_response": "Codegeist is a coding agent created by René Schmidt.", | |
| "normalized_match": true, | |
| "token_used": false, | |
| "result_sha256": "af0092e72bd347d5a4dd4bfbb579bae0402c51ead31959d33dd5647d4e34a430", | |
| "image_id": "sha256:a0f210aed561ed15cb4e44fb7eccde98bc44d354d484005b5f921d85de818f5b", | |
| "source_sha256": { | |
| "infer.py": "4b448ee14114b856e55a4639fad0f73110740039c4c334d628a8b44cd06c72c6", | |
| "inference/pyproject.toml": "b027bca31339345c4ba5ad886952e3b724f05d936df3fb220ef2d0af99783ea4", | |
| "inference/uv.lock": "ebeda66f1193fbdddd4a06c7e3ac3c7789d78c84c224259246e43214b7031bfa" | |
| } | |
| } | |
| }, | |
| "cost_estimate": { | |
| "observed_rate_usd_per_hour": 1.0, | |
| "running_seconds": 133, | |
| "per_second_estimate_usd": 0.0369, | |
| "conservative_whole_minutes": 3, | |
| "conservative_estimate_usd": 0.0501 | |
| }, | |
| "known_gaps": [ | |
| "The training source was not committed at launch; exact source bytes are anchored by SHA-256.", | |
| "Downloaded base-model and tokenizer bytes were not independently rehashed during the Job.", | |
| "The clean-process training reload retained only the whitespace-normalized response; the later anonymous public reload retained an exact raw response.", | |
| "Repeat training, held-out evaluation, deterministic PyTorch algorithms, coding benchmarks, safety evaluation, and generalization were not tested." | |
| ] | |
| } | |