Model cloning error
I used omlxversion 0.6.0 to load LongCat AudioDiT-3.5B-bf16 and requested it using the/v1/audio/watch interface, but encountered an error message std:: bad_cast,
May I ask if anyone is using this model normally under omlx?
It looks like a classic software compatibility issue. The std::bad_cast error basically means the engine (omlx 0.6.0) is trying to read this audio model as if it were a standard text chatbot model. Because the internal data shapes don't match what the system expects, it gets confused and crashes.
It usually boils down to one of these:
• A bug in omlx 0.6.0: The audio generation endpoint might have a bug in this specific version when handling this model type.
• Missing config labels: The model's config.json file might be missing the exact tag telling the system, "Hey, I'm an audio diffusion model, don't open me like an LLM."
• Format issues: The bf16 data format might not be fully supported by their C++ backend runner yet.
A good way to test it is to try running the model directly in Python via the native mlx-audio library to see if it works without omlx, or swap to the 4bit or 8bit versions to see if they bypass the crash.
I asked an AI assistant. Hope those helps!
It looks like a classic software compatibility issue. The std::bad_cast error basically means the engine (omlx 0.6.0) is trying to read this audio model as if it were a standard text chatbot model. Because the internal data shapes don't match what the system expects, it gets confused and crashes.
It usually boils down to one of these:
• A bug in omlx 0.6.0: The audio generation endpoint might have a bug in this specific version when handling this model type.
• Missing config labels: The model's config.json file might be missing the exact tag telling the system, "Hey, I'm an audio diffusion model, don't open me like an LLM."
• Format issues: The bf16 data format might not be fully supported by their C++ backend runner yet.A good way to test it is to try running the model directly in Python via the native mlx-audio library to see if it works without omlx, or swap to the 4bit or 8bit versions to see if they bypass the crash.
I asked an AI assistant. Hope those helps!
I have verified that the local independent mlx audio environment can use LongCat AudioDiT-3.5B-bf16 normally, and I am still trying to locate the problem with others。
Hi @sqh11 ,
Great job isolating the issue to the omlx server layer. Since the model works perfectly in the native mlx-audio environment, we can fully rule out corrupted tensor weights or hardware limitations.
The std::bad_cast is a C++ exception, but since omlx operates as a Python server layer wrapping around Apple's compiled C++ mlx extension binaries, this error is thrown exactly at the boundary where omlx hands the model architecture over to the core framework backend.
Here is exactly what is happening under the hood and how to track it down in the codebase:
1. Why std::bad_cast Occurs Here
In C++, a std::bad_cast happens when a runtime dynamic_cast fails on a reference type. Because omlx is designed with continuous batching and strict tiering optimizations optimized for standard text LLMs (like Llama or Qwen), its core model factory engine expects models to conform to text-based array shapes and transformer blocks.
When omlx attempts to pass the unique diffusion layers of LongCat-AudioDiT-3.5B-bf16 into an engine segment expecting a text or vision model structure, the underlying compiled C++ code tries to cast the pointer reference to a class type it doesn't actually inherit from, triggering the unhandled crash.
2. The /v1/audio/ Endpoint Context
Check the specific endpoint routing implementation inside omlx.
- Make sure that
/v1/audio/...requests are correctly isolating the model execution from the standard text generation scheduler queue. - If the routing logic accidentally sends the request down the standard KV-caching text completion pipeline, the tensor shapes will completely mismatch.
🛠️ How to pinpoint the exact failing line:
If you have access to the machine logs, do not rely only on the standard console printout. Instead, inspect the application log file directly on your Mac:
cat ~/.omlx/logs/server.log
Look for the lines immediately preceding the std::bad_cast message. The Python trace right above it will pinpoint the exact .py file and method inside the omlx engine repository where the wrong object type is instantiated and passed to the compiled bindings.
I asked my AI assistant again. Hope this helps. Thank you for patience. Sorry for the wrong analysis. I am a non-coder.