openjev/openjev-MLX-4bit

OpenJev, MLX 4-bit

OpenJev decisions on Apple silicon, in about 15 GB. This text-only build uses 4-bit affine quantization with group size 64. Give it text, JSON or browser DOM, your labels and your rules. It returns a typed choice, yes/no probability or score through the same OpenJev API.

Labels are supplied at request time. Each question with up to 52 options uses one forward pass. See the main model for the API and available formats.

Run it

uv venv mlx --python 3.12
uv pip install --python mlx/bin/python "mlx==0.32.2" "mlx-lm==0.31.3" \
  "transformers==5.17.0" "openai==3.16.2" "huggingface-hub==1.32.0"

mlx/bin/hf download openjev/openjev-MLX-4bit --local-dir openjev-MLX-4bit
mlx/bin/hf download openjev/openjev helper/shim.py helper/shim_mlx.py \
  --revision 1c341f65bfe5d50fdb935c71e9739c9e0938d6c4 --local-dir openjev-api

TOKENIZER=openjev-MLX-4bit SHIM_MODEL=openjev-MLX-4bit \
READOUT_T=0.85 READOUT_NOUL_T=1.829074 READOUT_NOUL_BIAS=0 \
READOUT_TARGETED=1 READOUT_INSTR_STYLE=pyrepr SHIM_STAGGER=1 \
  mlx/bin/python openjev-api/helper/shim_mlx.py \
  --helper openjev-api/helper/shim.py --model openjev-MLX-4bit --port 3000

Send requests to http://localhost:3000/v1/systemone using the main model's examples.

Licence

Weights: CC BY-NC 4.0 for research and non-commercial use, with attribution. Helper code: Apache 2.0. Upstream notices are included in NOTICE and LICENSE-APACHE-2.0.

OpenJev is an independent project, not affiliated with TypeSafe; Jev is their product.