Instructions to use udgamlabs/Laghu-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use udgamlabs/Laghu-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf udgamlabs/Laghu-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf udgamlabs/Laghu-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf udgamlabs/Laghu-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf udgamlabs/Laghu-GGUF:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf udgamlabs/Laghu-GGUF:F16
# Run inference directly in the terminal:
./llama-cli -hf udgamlabs/Laghu-GGUF:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf udgamlabs/Laghu-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf udgamlabs/Laghu-GGUF:F16

Use Docker

docker model run hf.co/udgamlabs/Laghu-GGUF:F16

LM Studio
Jan

vLLM

How to use udgamlabs/Laghu-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "udgamlabs/Laghu-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "udgamlabs/Laghu-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/udgamlabs/Laghu-GGUF:F16

Ollama
How to use udgamlabs/Laghu-GGUF with Ollama:
```
ollama run hf.co/udgamlabs/Laghu-GGUF:F16
```
Unsloth Desktop

How to use udgamlabs/Laghu-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf udgamlabs/Laghu-GGUF:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "udgamlabs/Laghu-GGUF:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use udgamlabs/Laghu-GGUF with Docker Model Runner:
```
docker model run hf.co/udgamlabs/Laghu-GGUF:F16
```

Lemonade

How to use udgamlabs/Laghu-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull udgamlabs/Laghu-GGUF:F16

Run and chat with the model

lemonade run user.Laghu-GGUF-F16

List all available models

lemonade list

Hermes Agent

How to use udgamlabs/Laghu-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf udgamlabs/Laghu-GGUF:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default udgamlabs/Laghu-GGUF:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use udgamlabs/Laghu-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf udgamlabs/Laghu-GGUF:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "udgamlabs/Laghu-GGUF:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

UDGAM LABS · GGUF · Q8_0 163 MB

Laghu-GGUF

GGUF builds of Laghu, a 152M-parameter English chat model trained from scratch by Udgam Labs for $146 of pretraining compute. It thinks before it answers (on/off), holds multi-turn conversations and chats. These files run it in Ollama, LM Studio, llama.cpp and other GGUF runtimes. Use Q8_0 (163 MB): it matches the full-precision model within measurement noise. On a Mac mini (Apple M6) it generates about 330 tokens per second in llama.cpp and needs about 154 MiB of memory at a 4K context.

Try it: in your browser, on your device · code on GitHub · udgamlabs.com/models/laghu

Quick start

Ollama (one line; Ollama picks up Laghu's chat format and sampling settings from this repository):

ollama run hf.co/udgamlabs/Laghu-GGUF:Q8_0 --think=false

In a chat, /set nothink turns thinking off and /set think turns it on; in the API, send "think": false.

LM Studio: search for udgamlabs/Laghu-GGUF in LM Studio and download Q8_0, or click "Use this model" → LM Studio on this page. LM Studio shows the thinking separately from the answer. Through its API, send "reasoning_effort": "none" to turn thinking off.

llama.cpp (use the Jinja chat template; without --jinja the server rejects requests):

llama-server -hf udgamlabs/Laghu-GGUF:Q8_0 --jinja --reasoning off -c 16384
# OpenAI-compatible API on http://localhost:8080
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Give me two tips for staying focused."}],"temperature":0.7,"top_p":0.95,"top_k":64}'

Remove --reasoning off to let Laghu think before answering (it writes its reasoning first; this helps a little on short arithmetic and hurts most other tasks).

Ollama with the Modelfile (offline, or to name the model yourself): download Laghu-Q8_0.gguf and Modelfile into one folder, then

ollama create laghu -f Modelfile
ollama run laghu --think=false "Who are you?"

File	Size	Use
`Laghu-Q8_0.gguf`	162.8 MB	recommended
`Laghu-F16.gguf`	305.1 MB	full precision, for re-quantising or research
`Modelfile`		Ollama, with the Gemma 4 renderer and Laghu's sampling defaults
`params`		Laghu's sampling defaults for `ollama run hf.co/...`

Sampling defaults: temperature 0.7, top-p 0.95, top-k 64. Keep context at 16,384 tokens or less.

Results

Quantisation quality (measured by us on 2026-10-22: 32 windows of 2,048 tokens of held-out text, compared with the F16 file; task checks through llama-server, greedy, thinking off):

File	Size	Perplexity	Mean KL vs F16	Top-1 agreement	GSM8K, first 200	IFEval, first 300, prompt-strict
F16	305.1 MB	52.24	0	100%	5.0%	23.3
Q8_0	162.8 MB	52.11	0.0016	97.7%	5.0%	25.7
Q5_K_M (not published)	122.2 MB	53.87	0.041	88.6%	3.5%	—
Q4_K_M (not published)	113.7 MB	55.70	0.080	84.7%	4.0%	—

We do not publish 4- and 5-bit files: they lose clearly, and because Laghu's width (640) is not a multiple of 256, llama.cpp's K-quants fall back to simpler types and barely shrink the file. The task scores differ from the model card's because this check uses a shorter output limit and only the first items; it compares formats, not models.

Faithfulness: the F16 file matches the transformers model almost exactly (mean KL 0.0000018, top-1 agreement 99.89%), and llama.cpp applies Laghu's 512-token sliding window (the local layers' cache stays at 10.5 MiB at any context length). The chat format was checked in llama.cpp and Ollama (thinking off and on, multi-turn, tool calls) and in LM Studio 0.4.26 (thinking off and on, multi-turn).

For the model's own results (benchmarks, chat tests against SmolLM2-135M-Instruct and Falcon-H1-Tiny-90M-Instruct), see the Laghu model card.

Where it falls short

These are files of a small research model: Laghu is weak at maths (GSM8K 7.9% with thinking) and instruction following (IFEval 26.1), knows little, and can loop with greedy decoding. Details on the model card.
llama.cpp, Ollama, LM Studio and the MLX server turn thinking on by default; use the switches above for normal chat.
llama-server caps the context at 16,384 tokens (32K prompts are rejected).

How it was trained

See the Laghu model card and the technical report. Converted with llama.cpp's convert_hf_to_gguf.py (commit d7a695e) from the released safetensors; the Q8_0 file is byte-identical when rebuilt.

Intended use and limitations

Local, offline use and research. Not for factual answers or decisions that matter; light safety training.

Licence

Citation

@misc{udgamlabs2026laghu,
  title  = {Laghu: a fully open 152M chat model trained from scratch},
  author = {Prasad, Jitesh K},
  year   = {2026},
  howpublished = {Udgam Labs},
  url    = {https://huggingface.co/udgamlabs/Laghu}
}

About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com