Instructions to use DomSimone/Umbuzo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use DomSimone/Umbuzo with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf DomSimone/Umbuzo:F16 # Run inference directly in the terminal: llama cli -hf DomSimone/Umbuzo:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf DomSimone/Umbuzo:F16 # Run inference directly in the terminal: llama cli -hf DomSimone/Umbuzo:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf DomSimone/Umbuzo:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf DomSimone/Umbuzo:F16
Use Docker
docker model run hf.co/DomSimone/Umbuzo:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use DomSimone/Umbuzo with Ollama:
ollama run hf.co/DomSimone/Umbuzo:F16
- Unsloth Desktop
- Pi
How to use DomSimone/Umbuzo with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DomSimone/Umbuzo:F16
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "DomSimone/Umbuzo:F16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use DomSimone/Umbuzo with Docker Model Runner:
docker model run hf.co/DomSimone/Umbuzo:F16
- Lemonade
How to use DomSimone/Umbuzo with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull DomSimone/Umbuzo:F16
Run and chat with the model
lemonade run user.Umbuzo-F16
List all available models
lemonade list
- Hermes Agent
How to use DomSimone/Umbuzo with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DomSimone/Umbuzo:F16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default DomSimone/Umbuzo:F16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use DomSimone/Umbuzo with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DomSimone/Umbuzo:F16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "DomSimone/Umbuzo:F16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Umbuzo AI System
Umbuzo is a specialized AI chatbot for African history, current affairs, and academics. It features a modern web interface, voice interaction capabilities, and a robust Dockerized backend hosting a local GGUF model.
Project Workflow
User Interface (Frontend):
- A responsive web app (
frontend/) built with HTML, CSS, and Vanilla JS. - Features: Chat interface, Voice Input (Speech-to-Text), Text-to-Speech responses, and Topic Selection.
- Communicates with the backend via REST API.
- A responsive web app (
Backend Server (Docker Container):
- A FastAPI server (
umbuzo_api.py) running inside a Docker container. - Model: Hosts the
umbuzo-gemma-2b-v3data.f16.ggufLLM directly within the container for reliability. - Voice Processing: Uses
SpeechRecognitionfor input andgTTSfor output. - API: Exposes endpoints for text chat (
/chat) and voice chat (/chat/voice).
- A FastAPI server (
Deployment:
- The system is deployed using a self-contained Docker image (
umbuzo-api:latest). - This approach avoids common Windows file-sharing issues by bundling the model and code together.
- The system is deployed using a self-contained Docker image (
Prerequisites
- Docker Desktop: Must be installed and running.
- Python 3.9+: Required for local scripts (though the backend runs in Docker).
- Model File:
umbuzo-gemma-2b-v3data.f16.ggufmust be present in the root folder (B:\Stripper\).
Quick Start Guide
1. Initial Setup (Build & Deploy)
Run this script once to build the Docker image and create the container. This step copies the large model file into the container, so it may take a few minutes.
- Script:
deploy_docker.bat - Action: Double-click the file in Windows Explorer.
2. Daily Usage (Start System)
Run this script to start the backend container (if stopped) and launch the frontend web server.
- Script:
start_umbuzo_full_system.bat - Action: Double-click the file.
- Access: The web interface will open at
http://localhost:8080.
Running direct servers for terminal in frontend.
- Run: npm install http-server Run: npx http-server -p 8080
3. Troubleshooting
If you encounter port conflicts (e.g., "Address already in use"), run this script to force-close lingering processes.
- Script:
kill_ports.bat
File Structure
frontend/: Web interface files (index.html,app.js,styles.css,Mbuzo.png).umbuzo_api.py: The Python backend logic (FastAPI, Llama-cpp, Voice).Dockerfile: Blueprint for the backend container image.umbuzo-gemma-2b-v3data.f16.gguf: The local Large Language Model file.deploy_docker.bat: Script to build the image and create the container.start_umbuzo_full_system.bat: Script to start the system for daily use.kill_ports.bat: Utility to free up ports 8001 and 8080.
Voice Features
- Click the Microphone Icon in the chat bar to start recording.
- Speak your query.
- Click the microphone again to stop and send.
- Umbuzo will process your speech and respond with both text and audio.
- Downloads last month
- 4
16-bit