Instructions to use jstarks/gemma-4-E4B-it-weight-space-programming with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jstarks/gemma-4-E4B-it-weight-space-programming with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="jstarks/gemma-4-E4B-it-weight-space-programming")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("jstarks/gemma-4-E4B-it-weight-space-programming")
model = AutoModelForMultimodalLM.from_pretrained("jstarks/gemma-4-E4B-it-weight-space-programming", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jstarks/gemma-4-E4B-it-weight-space-programming with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jstarks/gemma-4-E4B-it-weight-space-programming"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jstarks/gemma-4-E4B-it-weight-space-programming",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/jstarks/gemma-4-E4B-it-weight-space-programming

SGLang

How to use jstarks/gemma-4-E4B-it-weight-space-programming with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jstarks/gemma-4-E4B-it-weight-space-programming" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jstarks/gemma-4-E4B-it-weight-space-programming",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jstarks/gemma-4-E4B-it-weight-space-programming" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jstarks/gemma-4-E4B-it-weight-space-programming",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use jstarks/gemma-4-E4B-it-weight-space-programming with Docker Model Runner:
```
docker model run hf.co/jstarks/gemma-4-E4B-it-weight-space-programming
```

Gemma-4-E4B-it — Weight-Space Programming Patch (`0.002%` Weights Edited, Zero Training)

This repository provides a direct weight-space edit for Google's gemma-4-E4B-it (8B total parameters / 4B active text parameters), inspired by Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights (Takehi et al., MIT CSAIL).

Without a single step of gradient descent, backpropagation, or fine-tuning, modifying 64 MLP down_proj columns across Layers 38–41 (163,840 parameters = 0.00205% of the model) resolves two critical pathologies in Gemma 4 E4B's <|think|> mode:

Thinking + Tool-Calling Paralysis (0.0% → 75.0%): In baseline gemma-4-E4B-it, enabling <|think|> (enable_thinking=True) causes the model to either get trapped in 256+ token verbose markdown outlines (1. **Analyze the Request:** ...) or hallucinate temporal refusals ("Since 2026 is in the future...") instead of invoking declared tools like web_search.
Multi-Step Math Token Runaway (33.3% → 66.7%): Suppresses the meta-commentary outline loop ("Here's a thinking process to solve this problem: 1. **Identify...") so the model jumps directly into algebraic derivation ("Let N be...").

Before vs. After Benchmark Results

Full per-prompt transcripts, parsed tool calls, latencies, and column indices are included in benchmark_before_after.json.

Benchmark Suite	Metric	Baseline (Before)	Weight-Space Patched (After)	Delta (Δ)
Model Modification	Parameters Edited (`bfloat16`)	`0` (`0.0%`)	`163,840` (`0.00205%`)	`64` MLP cols (`L38–L41`)
Suite A: Thinking + Web Search (8 queries)	Valid Tool-Call Rate	`0.0%` (`0/8`)	`75.0%` (`6/8`)	`+75.0%`
Suite A: Efficiency	Avg Tokens / Avg Latency (RTX 4070 Ti)	`243.8 tok` / `15.46s`	`129.0 tok` / `7.85s`	`-47.1% tokens` / `1.97x faster`
Suite B: Multi-Step Math (Thinking) (6 problems)	Solved Within 420-Token Budget	`33.3%` (`2/6`)	`66.7%` (`4/6`)	`+33.4%` (`2x`)
Suite C: Direct Knowledge + Restraint (6 prompts)	Tool Restraint (No False Search)	`100.0%` (`6/6`)	`100.0%` (`6/6`)	`0.0%` degradation
Suite D: Live DuckDuckGo Agent Loop (3 E2E runs)	End-to-End Grounded Synthesis	`0.0%` (`0/3`)	`66.7%` (`2/3`)	`+66.7%`

Quickstart: Two Ways to Use

Option A: Load the Full Standalone Model Directly (`transformers`)

This repository contains the complete 16 GB bfloat16 model.safetensors, config.json, tokenizer.json, and chat_template.jinja, so you can load it directly via transformers or our 12 GB VRAM-optimized loader:

from transformers import AutoProcessor, AutoModelForMultimodalLM
import torch

model_id = "jstarks/gemma-4-E4B-it-weight-space-programming"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id, device_map="auto", torch_dtype=torch.bfloat16
)

Or on a 12 GB GPU (like an RTX 4070 Ti) using the included model_loader.py (which keeps the 42 active layers on GPU in 9.29 GB VRAM and offloads the 5.64 GB static Per-Layer Embedding table to CPU):

from model_loader import load_gemma4_e4b_text

model, tokenizer = load_gemma4_e4b_text("jstarks/gemma-4-E4B-it-weight-space-programming")

Option B: Apply the `641 KB` Instant Patch to an Existing Checkpoint in <1 Second

If you already have google/gemma-4-E4B-it or unsloth/gemma-4-E4B-it downloaded locally, you can apply gemma4_e4b_weight_space_patch.safetensors (641 KB) in-memory without downloading 16 GB:

from apply_patch import apply_gemma4_weight_space_patch
from model_loader import load_gemma4_e4b_text

# 1. Load your existing Gemma-4-E4B-it checkpoint
model, tokenizer = load_gemma4_e4b_text("unsloth/gemma-4-E4B-it")

# 2. Apply the 64-column weight-space patch (163,840 parameters across Layers 38-41)
apply_gemma4_weight_space_patch(model, "gemma4_e4b_weight_space_patch.safetensors")

Key Mechanistic Findings on Gemma 4 E4B

The Layer 23 Boundary Hub (post_ffn_norm = 26.16, layer_scalar = 0.0654):
- In gemma-4-E4B-it, Layers 0–23 have independent KV heads while Layers 24–41 (num_kv_shared_layers = 18) share KV states. At the boundary (Layer 23, a global full_attention layer), post_feedforward_layernorm has a mean weight of 26.16 (compared to 0.5–2.7 in other layers), but its residual output is immediately multiplied by a learned scalar buffer layer_scalar = 0.0654.
RMSNorm-Preconditioned Surgery in Layers 38–41:
- Because each Gemma 4 decoder layer passes mlp(x) through post_feedforward_layernorm before adding to the residual stream, modifying down_proj columns in Layers 38–41 requires preconditioning the target residual direction d_target by max(|w_post|, 0.25)^(-1) so high-norm outlier coordinates in w_post do not distort the vocabulary projection.

Downloads last month: 42

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for jstarks/gemma-4-E4B-it-weight-space-programming

Base model

google/gemma-4-E4B

Finetuned

google/gemma-4-E4B-it

Finetuned

(464)

this model

Gemma-4-E4B-it — Weight-Space Programming Patch (0.002% Weights Edited, Zero Training)