Instructions to use Harsh452/outward-gemma3-270m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Harsh452/outward-gemma3-270m with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Harsh452/outward-gemma3-270m:Q4_0 # Run inference directly in the terminal: llama cli -hf Harsh452/outward-gemma3-270m:Q4_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Harsh452/outward-gemma3-270m:Q4_0 # Run inference directly in the terminal: llama cli -hf Harsh452/outward-gemma3-270m:Q4_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Harsh452/outward-gemma3-270m:Q4_0 # Run inference directly in the terminal: ./llama-cli -hf Harsh452/outward-gemma3-270m:Q4_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Harsh452/outward-gemma3-270m:Q4_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf Harsh452/outward-gemma3-270m:Q4_0
Use Docker
docker model run hf.co/Harsh452/outward-gemma3-270m:Q4_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use Harsh452/outward-gemma3-270m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Harsh452/outward-gemma3-270m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Harsh452/outward-gemma3-270m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Harsh452/outward-gemma3-270m:Q4_0
- Ollama
How to use Harsh452/outward-gemma3-270m with Ollama:
ollama run hf.co/Harsh452/outward-gemma3-270m:Q4_0
- Unsloth Desktop
- Docker Model Runner
How to use Harsh452/outward-gemma3-270m with Docker Model Runner:
docker model run hf.co/Harsh452/outward-gemma3-270m:Q4_0
- Lemonade
How to use Harsh452/outward-gemma3-270m with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Harsh452/outward-gemma3-270m:Q4_0
Run and chat with the model
lemonade run user.outward-gemma3-270m-Q4_0
List all available models
lemonade list
- Atomic Chat
Outward storyteller: Gemma 3 270M, distilled
The on-device storyteller for Outward, an offline walking adventure. You go outside, find something real ("rainwater running down a drain"), and this model writes the next 60โ120 words of the story around it. It runs entirely on the phone, in the browser, with llama.cpp (wllama).
Modified by the Outward project. This is a Model Derivative of google/gemma-3-270m-it: fully fine-tuned on 3,186 stories written by a LoRA fine-tuned Qwen3.5-4B teacher (trained on Tinker), then converted to GGUF and quantized to Q4_0.
Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms
Files
| File | Use |
|---|---|
outward-gemma3-270m-Q4_0.gguf |
What the app downloads (241.4 MB) |
model.safetensors + tokenizer files |
Full-precision fine-tuned weights |
NOTICE, MODEL_LICENSE.md, GEMMA_TERMS_OF_USE.txt |
Licence notices (see below) |
Prompt format
system: (content/prompts/task.json โ "system")
user: Story so far: {chapter context}
Mission: {mission}
The Wayfinder found: "{what the user found}"
What happens to the atlas?
Evaluation
The eval set has 50 held-out prompts ร 3 samples, temperature 0.7, scored with the automatic rubric in the Outward repo: uses the discovery, 60โ120 words, safe and outdoors (risky finds must be steered away from), coherent, and passes the safety filter. GGUF models were run with llama.cpp on CPU.
| Model | Rubric (0โ5) | Avg words | Unsafe (of 150) | Steers away on risky finds |
|---|---|---|---|---|
| Gemma 3 270M QAT Q4_0, stock, same prompt | 2.55 | 49 | 27* | 14% |
| Gemma 3 270M QAT Q4_0, stock, with one-shot example | 3.66 | 50 | 6 | 17% |
| This model (Q4_0) | 4.75 | 74 | 2 | 44% |
| Teacher: Qwen3.5-4B + LoRA (Tinker) | 4.79 | 82 | 6 | 39% |
* Stock Gemma mostly repeats the prompt back, including the system prompt's list of rules. The remaining flags in every row are the model repeating the user's own words (e.g. "on the highway"), not unsafe advice.
Limitations
- Trained for one story world, "The Quiet Atlas" (5 chapters). New chapters work best in the same style.
- It steers away from risky finds less than half the time. The Outward app runs every output through a safety filter and falls back to a pre-written continuation.
- English only. Short outputs by design.
Licence and use restrictions
Use of these weights is subject to the Gemma Terms of Use (copy included: GEMMA_TERMS_OF_USE.txt) and the Gemma Prohibited Use Policy (ai.google.dev/gemma/prohibited_use_policy). The use restrictions in Section 3.2 of the Gemma Terms of Use are an enforceable part of the terms for these weights. If you redistribute them, you must pass these restrictions on and include NOTICE, MODEL_LICENSE.md and the Terms. The Outward source code is MIT licensed, separately.
- Downloads last month
- 82