OpenJEV GPTQ
Your edge deployable Jev now can be served on a Jetson Orin Nano 8GB!
Quantization
Models were quantized using GPTQModel and calibrated with 512 NLI examples (
multinl_calibration_512.jsonl)
Quantization Platform
- RTX Pro 2000 Blackwell 16GB
- Ubuntu 24.04
- python3.12
Model Footprint Comparison
| Model Variant | Original | Quanztied |
|---|---|---|
| Qwen3.5 4B | 9.1GB | 3.8GB |
| Qwen3.5 2B | 4.4GB | 2.4GB |
Model Performance Comparison
xychart-beta
title "Holdout accuracy by model and precision"
x-axis ["2B BF16", "2B GPTQ int4", "4B BF16", "4B GPTQ int4"]
y-axis "Accuracy (%)" 0 --> 100
bar [88.3, 89.1, 91.4, 91.8]
Credit
Original Model Weights: AlexWortega/openjev
Quantization Library: ModelCloud/GPTQModel
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support