Instructions to use jamusvii/onlyqt-gemma4-e4b-mlx-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use jamusvii/onlyqt-gemma4-e4b-mlx-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("jamusvii/onlyqt-gemma4-e4b-mlx-4bit") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use jamusvii/onlyqt-gemma4-e4b-mlx-4bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "jamusvii/onlyqt-gemma4-e4b-mlx-4bit" --prompt "Once upon a time"
- Atomic Chat
onlyqt-gemma4-e4b-mlx-4bit
Google Gemma 4 E4B (text) converted for MLX and quantized to 4 bit, hosted here so the iOS app 온니큐티 (OnlyQT) can download it on first launch. The app runs the model fully on device.
This repository contains no fine-tuning — the weights are a straight re-quantization of Google's released model.
Contents
Files live under base_gemma4_e4b_mlx_q4/. manifest.json lists every file
with its exact byte size; the app uses it to download and to resume safely.
| File | Bytes |
|---|---|
model.safetensors |
4,198,905,174 |
tokenizer.json |
32,169,626 |
model.safetensors.index.json |
97,889 |
chat_template.jinja |
16,804 |
config.json |
2,914 |
tokenizer_config.json |
2,773 |
generation_config.json |
215 |
| Total | 4,231,195,395 |
Quantization
affine, 4 bit, group size 64. 42 layers, 18 of them KV-shared.
License
Gemma. Use of these weights is governed by the Gemma Terms of Use and the Gemma Prohibited Use Policy. By downloading them you agree to those terms.
Quantized