Hugging Face's logo

leonsarmiento
/
Ornith-1.0-35B-5bit-mlx

Image-Text-to-Text
MLX
Safetensors
qwen3_5_moe
mlx-vlm
Mixture of Experts
multimodal
vision
coding
agentic
conversational
5-bit
Model card Files
xet
 Community
1

Won't load on LM Studio

#1
by garrettw87 - opened
Discussion
garrettw87

It's throwing an error during load: ValueError: Unsupported model type: qwen3_5_moe_vision

Full error dump:

Failed to load model.

Error when loading model: Exception: Encountered fatal exception in the backend generation thread: Traceback (most recent call last):
  File "/Users/gw/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@29/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 318, in _generate_with_exception_handling
    self._generate()
  File "/Users/gw/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@29/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 539, in _generate
    self._load_model()
  File "/Users/gw/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@29/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 179, in _load_model
    self.model, _ = mlx_vlm.utils.load_model(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gw/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@29/lib/python3.11/site-packages/mlx_vlm/utils.py", line 251, in load_model
    model = model_class.Model(model_config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gw/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@29/lib/python3.11/site-packages/mlx_vlm/models/qwen3_5_moe/qwen3_5_moe.py", line 16, in __init__
    self.vision_tower = VisionModel(config.vision_config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/gw/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@29/lib/python3.11/site-packages/mlx_vlm/models/qwen3_vl/vision.py", line 201, in __init__
    raise ValueError(f"Unsupported model type: {self.model_type}")
ValueError: Unsupported model type: qwen3_5_moe_vision
garrettw87
•
edited 3 days ago

Sorry, I actually opened this on the wrong model -- should have been 4bit. Nevermind, I see there's not a separate 4bit to comment on.
garrettw87 changed discussion status to closed
garrettw87 changed discussion status to open
nick-pan

mlx-vlm's VisionModel only accepts the following values for model_type (see mlx_vlm/models/qwen3_vl/vision.py):

  if self.model_type not in ["qwen3_vl", "qwen3_5", "qwen3_5_moe"]:
      raise ValueError(f"Unsupported model type: {self.model_type}")

qwen3_5_moe_vision is not in the allowed list, so the model fails to load.

Comparison with the official Qwen3.6 MLX model

The official lmstudio-community/Qwen3.6-35B-A3B-MLX-4bit uses qwen3_5_moe for the same field and loads without issues:

  "vision_config": {
      "model_type": "qwen3_5_moe",
      ...
  }

Expected behavior

vision_config.model_type should be qwen3_5_moe (not qwen3_5_moe_vision), consistent with the official Qwen3.6 MLX conversion.

Suggested fix

In config.json, change:

   "vision_config": {
  -    "model_type": "qwen3_5_moe_vision",
  +    "model_type": "qwen3_5_moe",
       ...
   }
leonsarmiento
 Owner

I recommended to use oMLX for this, and pretty much any other mlx model, specially if you are using agentic tools.

Sign up or log in to comment