Instructions to use innerloop-dev/gemma3-4b-text with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use innerloop-dev/gemma3-4b-text with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf innerloop-dev/gemma3-4b-text:Q4_K_M # Run inference directly in the terminal: llama cli -hf innerloop-dev/gemma3-4b-text:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf innerloop-dev/gemma3-4b-text:Q4_K_M # Run inference directly in the terminal: llama cli -hf innerloop-dev/gemma3-4b-text:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf innerloop-dev/gemma3-4b-text:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf innerloop-dev/gemma3-4b-text:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf innerloop-dev/gemma3-4b-text:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf innerloop-dev/gemma3-4b-text:Q4_K_M
Use Docker
docker model run hf.co/innerloop-dev/gemma3-4b-text:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use innerloop-dev/gemma3-4b-text with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "innerloop-dev/gemma3-4b-text" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "innerloop-dev/gemma3-4b-text", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/innerloop-dev/gemma3-4b-text:Q4_K_M
- Ollama
How to use innerloop-dev/gemma3-4b-text with Ollama:
ollama run hf.co/innerloop-dev/gemma3-4b-text:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use innerloop-dev/gemma3-4b-text with Docker Model Runner:
docker model run hf.co/innerloop-dev/gemma3-4b-text:Q4_K_M
- Lemonade
How to use innerloop-dev/gemma3-4b-text with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull innerloop-dev/gemma3-4b-text:Q4_K_M
Run and chat with the model
lemonade run user.gemma3-4b-text-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Google's Gemma 3 4B, as Ollama builds it, with the picture-reading part taken out. The text weights are the same bytes. It writes text. It cannot look at an image.
What it is
This is Ollama's own gemma3:4b โ the Q4_K_M build of google/gemma-3-4b-it โ with the
vision tower and the multimodal projection removed and nothing else changed.
- Source file: the model layer of
gemma3:4b, sha256aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25, 3,338,792,448 bytes. - What came out: 439 tensors whose names begin
v.(the SigLIP vision tower) ormm.(the multimodal input projection), and 9 metadata keys โgemma3.mm.tokens_per_imageand the eightgemma3.vision.*keys. - What stayed: all 444 text tensors, byte for byte. Same names, same ggml quantization types, same shapes, same bytes. Nothing was requantized, converted, retrained or fine-tuned. The tokenizer is untouched, all 262,145 entries, so token numbers do not move and the model writes the same words.
- What was added: one metadata key,
general.description, carrying the notice that this file is a modified Gemma file. - Result: 2,498,332,864 bytes,
sha256
199388f8f8cbec06b80bd63b0b1a774103e00c993c107bf1d480e58047420532.
The template and the sampling parameters are the ones Ollama ships with gemma3:4b,
unchanged: stop <end_of_turn>, temperature 1, top_k 64, top_p 0.95.
Why
Breadcrumb, by Innerloop, uses this model to give screens, chapters and meetings short names. It only ever sends it text. The vision tower was 840 MB of weights that were read off disk, paged in and held in memory for work that never happened.
Measured on the same machine, one model resident at a time, with footprint against the
running llama-server:
| on disk | held while answering | |
|---|---|---|
gemma3:4b |
3.3 GB | 4.28 GB |
| this model | 2.5 GB | 2.73 GB |
That is about 1.6 GB less memory while it is working, and 0.8 GB less disk. On an 8 GB Mac that is the difference between naming a screen and swapping.
The names it writes are the same names. On 300 pinned items from a real store โ 150
screen moments, 75 chapters, 75 meetings โ this file and gemma3:4b wrote 300 identical
titles out of 300, at the same settings, and both were stable across a second pass.
It cannot see
This is a text-only model. There is no vision tower in the file and no projector layer in
the manifest. ollama show reports one capability, completion, where gemma3:4b
reports completion and vision. If you send it an image it has nothing to look at.
If you need Gemma 3 to read pictures, use gemma3:4b instead.
Gemma notice
Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms
This distribution contains a modified Gemma file. The full Gemma Terms of Use, as last modified April 1, 2026, travel with the model: the LICENSE file in this repository is that complete copy, and NOTICE carries the modification notice.
Use of this model is subject to Google's Gemma Prohibited Use Policy at ai.google.dev/gemma/prohibited_use_policy. That policy is part of the Gemma Terms and applies to you whether you got this model from us or from anywhere else. If you pass this model on, or anything you build from it, you pass these terms on with it and you tell the people you pass it to that the Gemma use restrictions apply.
Google claims no rights in what you generate with it.
Everything in this distribution that is not Gemma is ยฉ Innerloop, innerloop.works.
How to verify it yourself
You do not have to take our word that only the vision parts came out. The check needs nothing but a GGUF reader.
- Get Ollama's
gemma3:4band find its model layer in your blob store. It should hash toaeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25. - Get this model and find its model layer. It should hash to
199388f8f8cbec06b80bd63b0b1a774103e00c993c107bf1d480e58047420532and be 2,498,332,864 bytes. - Open both with a GGUF reader and list the tensors. The original has 883 tensors and
this one has 444 tensors. Every one of the 439 missing tensors has a name beginning
v.ormm.. Nothing else is missing. - The 444 that remain are
token_embd.weight,output_norm.weight, and thirteen tensors for each of the 34 blocks:attn_q.weight,attn_k.weight,attn_v.weight,attn_output.weight,attn_q_norm.weight,attn_k_norm.weight,attn_norm.weight,post_attention_norm.weight,ffn_gate.weight,ffn_up.weight,ffn_down.weight,ffn_norm.weight,post_ffw_norm.weight. By quantization: 205 Q4_K, 34 Q6_K, 205 F32, the same counts as in the original's text half. - For each of those 444, compare the ggml type, the shape, the byte count and the sha256 of the tensor's own bytes against the same tensor in the original. All 444 match. Any single mismatch means the weights were touched, and they were not.
- The metadata: 27 keys here against 35 in the original. The 9 that are gone are the
vision keys listed above. One key is new,
general.description, which says the file was modified.general.architectureis stillgemma3, and the tokenizer keys are identical.
The script that does the surgery lives in a private repository, so the recipe is written
out above rather than linked. It is short enough to re-implement: copy every tensor whose
name does not start with v. or mm., copy every metadata key that does not start with
gemma3.vision. or gemma3.mm., add general.description, write the file. Then hash
what you wrote. If you get 199388f8โฆ you have reproduced it exactly.
What we did not do
No fine-tuning. No requantization. No distillation. No change to the tokenizer, the chat template or the sampling defaults. No system prompt. This is the same model, minus a part it was not being asked to use.
Get it
- Ollama:
ollama run innerloop/gemma3-4b-text(the same file, published at https://ollama.com/innerloop/gemma3-4b-text) - This repository: the GGUF is
gemma-3-4b-it-text-Q4_K_M.gguf, 2,498,332,864 bytes, sha256199388f8f8cbec06b80bd63b0b1a774103e00c993c107bf1d480e58047420532.
- Downloads last month
- 22
4-bit