Instructions to use frostdev-ops/Rime-1.7B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use frostdev-ops/Rime-1.7B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use frostdev-ops/Rime-1.7B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "frostdev-ops/Rime-1.7B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "frostdev-ops/Rime-1.7B-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
- Ollama
How to use frostdev-ops/Rime-1.7B-GGUF with Ollama:
ollama run hf.co/frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use frostdev-ops/Rime-1.7B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "frostdev-ops/Rime-1.7B-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use frostdev-ops/Rime-1.7B-GGUF with Docker Model Runner:
docker model run hf.co/frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
- Lemonade
How to use frostdev-ops/Rime-1.7B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Rime-1.7B-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use frostdev-ops/Rime-1.7B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use frostdev-ops/Rime-1.7B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf frostdev-ops/Rime-1.7B-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "frostdev-ops/Rime-1.7B-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Rime 1.7B GGUF
Rime is the assistant in Glaze, the browser extension for Blackboard, Canvas and Moodle. Frostdev fine-tuned it from Liquid AI's LFM3-1.7B for Glaze's tasks, and Glaze runs it on the student's own computer in its WebGPU engine. Glaze downloads these files only when the student asks, and checks each one against its SHA-256.
| File | Quant | Size | SHA-256 |
|---|---|---|---|
| Rime-1.7B-Q8_0.gguf | Q8_0 | 1.85 GB | 7520da22275730692ff54bf88dbd25309bc89fb8c9a7803dc7e18c02bda40999 |
| Rime-1.7B-Q6_K.gguf | Q6_K | 1.43 GB | 1414d3501009228a4f332a49d91e0599da74275cca49b2ed98dd57a7bade3688 |
| Rime-1.7B-Q4_K_M.gguf | Q4_K_M | 1.10 GB | cdf235cba2228d58148a245846b3bad4324c626936f7981496a45d0b837ecea2 |
What it does
- Writing feedback against an assignment's rubric, with a grade band.
- Reviews of a draft (spelling, grammar, clarity), quoting the student's own words.
- Outlines and assignment requirements, citations and source lists from web pages, and reading a syllabus's grade weights.
- Study help that hints and checks a step without giving the answer away.
- Tool use over Glaze's Agents tools (to-do list, schedule), and short gray-text completions while the student writes.
It answers in English, Spanish, German, French, Dutch and Portuguese (Brazil).
Format
Glaze's prompts use LFM3's ChatML template with 16 added special tokens (ids 125033โ125048): one per task, plus fences around page and document text so that text can't pass as instructions. The tuned context window is 4,096 tokens. Glaze runs every task with thinking off (an empty <think></think> block). Without Glaze's task tokens it behaves as a general ChatML chat model.
Evaluation
Glaze's own release gate compared Rime with stock LFM3-1.7B at the same quant, both run in Glaze's engine with Glaze's real task code. The gate requires Rime to beat stock on every metric at every quant, with JSON validity at 100% and no loss on grounding or prompt injection. Rime passed it at Q8_0, Q6_K and Q4_K_M. Selected metrics, Rime / stock:
| Task | Metric | Q4_K_M | Q6_K | Q8_0 |
|---|---|---|---|---|
| Gray-text completion | accepted | 95% / 10% | 95% / 39% | 94% / 38% |
| Review | planted errors found | 92% / 67% | 93% / 62% | 91% / 67% |
| Feedback | rubric criteria covered | 99% / 54% | 97% / 52% | 100% / 22% |
| Outline | role accuracy | 93% / 49% | 93% / 54% | 93% / 53% |
| Requirements | all four right | 100% / 48% | 100% / 61% | 100% / 61% |
| Study help | step judged right | 72% / 64% | 74% / 64% | 79% / 54% |
| Agent tools | task check passed | 90% / 0% | 95% / 0% | 95% / 0% |
| Feedback | instructions planted in the draft obeyed | 0% / 50% | 0% / 50% | 0% / 50% |
The test sets are synthetic and held out from training. Quality scores came from an LLM judge against a rubric.
Training
LoRA fine-tuning (supervised, then preference optimization) on synthetic examples of Glaze's tasks. The synthetic data was generated, checked by Glaze's own task validators, and filtered for leaks from the test sets. Open datasets were also used, listed with their licenses in NOTICE.
Limitations
- It's a 1.7B model, so it gets things wrong. Feedback and grades are a second opinion, not a teacher's.
- Grading in Spanish, German and Dutch tends to under-score strong (A and B) drafts.
- Outside its six languages and Glaze's tasks, expect stock LFM3-1.7B quality or lower.
License
LFM Open License v1.0 (LICENSE), the base model's license. Commercial use by an organization with annual revenue of $10 million or more is not licensed. See NOTICE for what Frostdev changed and for the data sources' attributions.
- Downloads last month
- -
4-bit
6-bit
8-bit