LTX 2.5 doesn't work on Apple Silicon
Same error over and over again. Something wrong with how the audio scales or video scales. I'll post the error below.
[INFO] Starting server
[INFO] To see the GUI go to: http://127.0.0.1:8188
[WARNING] [DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/widgetInputs.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[INFO] [ComfyUI-Manager] The ComfyRegistry cache update is still in progress, so an outdated cache is being used.
FETCH DATA from: /Users/XXXXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/.venv/lib/python3.13/site-packages/comfyui_manager/custom-node-list.json [DONE]
[INFO] got prompt
[INFO] VAE load device: mps, offload device: cpu, dtype: torch.float32
[INFO] Requested to load LTXAVTEModel_
[INFO] loaded completely; 24999.98 MB loaded, full load: True
[INFO] CLIP/text encoder model load device: cpu, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] model weight dtype torch.bfloat16, manual cast: None
[INFO] model_type FLUX
[INFO] FETCH ComfyRegistry Data [DONE]
[INFO] [ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes
FETCH DATA from: /Users/XXXXXXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/user/__manager/cache/1514988643_custom-node-list.json [DONE]
[INFO] [ComfyUI-Manager] All startup tasks have been completed.
[INFO] Requested to load LTXAV
[INFO] loaded completely; 40051.47 MB loaded, full load: True
100%|██████████| 8/8 [00:50<00:00, 6.26s/it]
[INFO] VAE load device: mps, offload device: cpu, dtype: torch.bfloat16
[INFO] Requested to load LatentUpsampler
[INFO] loaded completely; 949.61 MB loaded, full load: True
100%|██████████| 3/3 [01:04<00:00, 21.50s/it]
[INFO] Requested to load AudioVAE
[INFO] loaded completely; 693.46 MB loaded, full load: True
[INFO] Requested to load CausalDiffusionVAE
[INFO] loaded completely; 1403.92 MB loaded, full load: True
[ERROR] !!! Exception during processing !!! Invalid argument: 'avcodec_send_frame()' returned 22
[ERROR] Traceback (most recent call last):
File "/Users/XXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/execution.py", line 545, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/XXXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/execution.py", line 344, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/XXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/execution.py", line 318, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "/Users/XXXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/execution.py", line 306, in process_inputs
result = f(**inputs)
File "/Users/XXXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/comfy_api/internal/__init__.py", line 149, in wrapped_func
return method(locked_class, **inputs)
File "/Users/XXXXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/comfy_api/latest/_io.py", line 1990, in EXECUTE_NORMALIZED
to_return = cls.execute(*args, **kwargs)
File "/Users/XXXXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/comfy_extras/nodes_video.py", line 141, in execute
video.save_to(
~~~~~~~~~~~~~^
os.path.join(full_output_folder, file),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<3 lines>...
crf=encoding.get("crf"),
^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/Users/XXXXXXXXXXXXX/ComfyUI-Installs/ComfyUI (2)/ComfyUI/comfy_api/latest/_input_impl/video_types.py", line 895, in save_to
output.mux(audio_stream.encode(frame))
~~~~~~~~~~~~~~~~~~~^^^^^^^
File "av/audio/stream.py", line 25, in av.audio.stream.AudioStream.encode
@cython .ccall
^^^^^^^
File "av/audio/stream.py", line 35, in av.audio.stream.AudioStream.encode
packets = self.codec_context.encode(frame)
^^^
File "av/codec/context.py", line 519, in av.codec.context.CodecContext.encode
for packet in self._send_frame_and_recv(frame):
File "av/codec/context.py", line 378, in _send_frame_and_recv
err_check(res, "avcodec_send_frame()")
^^^
File "av/error.py", line 354, in av.error.err_check
raise cls(code, message, filename, log)
^^^
av.error.ArgumentError: Invalid argument: 'avcodec_send_frame()' returned 22
[INFO] Prompt executed in 477.82 seconds
I’ve done some thorough testing on this and I managed to found a way to make it work reliably, but for honesty’s sake: this is not a real fix. It’s only a simple fix that luckily requires no coding, but at least will give you the chance to work with this amazing model and enjoy it on mps.
So here is the problem I had:
Using the standard ComfyUI LTX 2.5 workflows would cause a PyAV error 22 in the Save Node; after a successful VAE Decode.
Searching around, the most suggested solution is to either use the VHS Video Combine node or Save the Video without the Audio connected. This removes the error, but it still saves a black video.
This would happen sporadically in Text 2 Video workflows, but it happened every single time in Image 2 Video workflows.
So here is what I found:
Turns out, from what I understood with my tests, the problem is how Comfyui loads LTX 2.5 and manages Nested Tensors; with LTX 2.5 specifically, while 2.3 never fails.
For some unknown reason, when the kSampler processes the latent, after just one steps it becomes filled with NaN values.
Using a Console Log Node, I’ve discovered that this happens exactly within the kSampler process, because the NestedTensor is fine when it comes out from the AVConcat node. It becomes corrupted after just one step. I’ve used the suggested sampling parameters with Custom Sigmas, but I’ve tried more combos (euler_a + beta, euler + simple), but they don’t have any influence in this.
I’ve also tested a pure video workflow: no audio latent, no AVConcat nodes. The latent output of LTXVImageToVideoInplace goes directly into the kSampler. Doing this, the video will generate 100% successfully!
Whatever is causing this, is strictly related to how the model is loaded on Apple Silicon and how Nested Tensors are handled for LTX 2.5.
THE SOLUTION:
I’ve removed the standard Load Diffusion Model or the UNET Loader (I’ve tested both GGUF and the official bf16), and I’ve used KJ Loader nodes for both the model and the VAEs (even if the VAEs are not the issue, I wanted to be 100% sure).
I loaded the model with dtype/patch at float 32, patch on device FALSE, fp16 accumulation OFF, attention_override SDPA
Video VAE: KJ VAE loader node with the conv VAE at fp32.
Audio VAE: KJ VAE loader node at bf16
This solved all of my issues with the distilled bf16 model and the distilled gguf (I haven’t tried the dev).
Both Image 2 Video and Text 2 Video workflows are now working fine and Save without any issue or error.
I really hope that this will help you as well, as any other Apple Silicon user like us; hopefully also helping to find an official solution.
There is also another fact: when using the basic Load Diffusion Model or UNET Loader (GGUF) nodes, combined with KJ VAE Loaders, I2V workflows would be successful depending on their seed!
As an example to fully explain this behaviour:
Gen 1 - Seed 42 -> SUCCESS
Gen 2 - Seed 42 -> FAIL (nan in latent)
Gen 3 - Seed 50 -> FAIL
Gen 4 - Seed 51 -> FAIL
Gen 5 - Seed 59 -> SUCCESS (always after minimum 2 failed attempts with 2 different seeds!)
Gen 6 -> if using Seed 59 (same as previous gen) -> FAIL | If using a different seed -> SUCCESS
I really hope this help the super experts among us. The work I did is based purely on observations and testing with my personal knowledge. So I apologize in case what I wrote is not as expertly detailed as it would be from a pro.
Anyway, I hope this will help.
Thanks for going through the trouble to do all of this!
I tested it and I did not find the same success... any chance you'd be willing to upload a workflow to see if it's something I'm doing wrong?
As much as you deny being an expert, I'm even further from an expert than you XD
Have you looked at: https://huggingface.co/mlx-community/ltx-2.5-mlx which is an Apple silicon port.
Have you looked at: https://huggingface.co/mlx-community/ltx-2.5-mlx which is an Apple silicon port.
Thanks for this, just looked into it. I was hoping to find any kind of discussion on how to use this complicated breakdown of a model.
I'm used to ComfyUI, and even that gets to be too complicated and far from user friendly for me.
This looks like you need to use command line, with lots of miscellaneous parts that need to be organized, with no visual workspace and just a console you punch stuff into.
Great that people are getting it working that way but for me, a silly end-user, it's just way too much learning curve and not exactly what I'm looking for.
I can currently run LTX 2.5 through the LTX desktop app using local hardware on my Mac, but it doesn't allow you to queue up multiple generations, you can't fine-tune the resolution output; just far more limited and requires sitting at my computer to run generations which is also not what I'm looking for despite the very nice user interface.
Sorry for the delayed answer. I’m actually on a small weekend vacation :P Monday or Tuesday I will be back home and I will share with you the workflows I used for my tests; which btw are the official ones you can find in ComfyUI already, just with some minor changes.
In the meantime, here are a few simple things you can try to better diagnose your situation and have a much more precise idea of what’s happening bts in ComfyUI, without having to do anything too complicated.
First things first: when it comes to ComfyUI and Python, the libraries you have installed in your .venv will have an impact on the results. So that may be a reason why the same workflows may work for me and not for you. I have really just a few nodes installed (the popular ones mostly)
Then there is the hardware factor: even if we both use MPS, depending on what M series you have, some things change both on the hardware side and on the software side. For example: M5 uses version 4 of Metal, where M2 uses v3 (which is my hardware). So you have to consider your hardware as a pierce of this equation as well ;)
WHAT YOU CAN TRY NOW:
Here is the “homemade” diagnose procedure I used. This will tell you immediately if we experienced the same issue.
To do this you need two custom nodes pack (which I highly recommend anyway): Easy Use and KJ Nodes.
Then open the official LTX 2.5 template workflow you need; Image 2 Video or Text 2 Video.
Open the large subgraph in the workflow (the one where you set he prompt, model, the text encoder, var and so on) and you will see the full 2 Stages workflow.
Get the node called Console Debug (it has a wrench emoji before the name) and place it close to the first kSampler. Now connect the latent out of the kSampler to the input of the Console Debug node.
Press the play button on top of the Console Debug node and the workflow will complete until that node, without the Second Stage (use a simple prompt and no loras).
When the workflow has completed, look in your terminal: you will surely see a long list of numbers in cyan color (mostly 1, 1, 1, 1, 1). Scroll up where the cyan color text starts and if you see something like this: [[nan, nan, nan, nan]] going on forever, then that’s the proof that we are actually facing the same issue.
If instead of nan you see a bunch of numbers, that means that you Stage 1 inference actually worked fine and you may have this problem at stage 2. In this case: just move the Console Debug node to the second kSampler and connect the Second kSampler out to Console Debug input. Press play and try again ;)
If this is actually your case as well, here are a few other steps to try:
Delete all VAE Load and diffusion model load nodes. I would also highly suggest to remove the whole Prompt Enhancing part of the workflow; which has caused nothing but problems in my case.
Now use these nodes with these settings:
2x KJ VAE Loader: Video VAE at float32 (use the conv VAE), Audio VAE at bf16
KJ Load Diffusion Model (if you use bf16 safetensors) or KJ GGUF Loader (if you use GGUF).
set these loaders with these settings: float32 for the dtype, sdpa for the attention and everything else on false.
Also be sure you have enough RAM to handle this; the full bf16 distilled may cause a spike! (Reason why I use GGUFs)
This is all :) It’s the exact workflow I’m using; mostly GGUF Q8 as they give fantastic results.
As I told you previously: my tests were done only on the official distilled bf16 model, a GGUF Q8 of the distilled model and the GGUF version of the only merge LTX 2.5 has right now (this one and it’s worth a try: https://huggingface.co/joeygambino/joyai-echo-ltx25-echoVid-gguf)
Next week I will be back and maybe I will even have more to share; as I will do more tests for sure ;)
I’m really sorry my tests couldn’t solve your issue as they did for me. Us “Apple Silicon crew” are in the minority when it comes to diffusion models :P So I genuinely think it’s a good thing to help each other find a solution or at least keep the discussion and our findings fresh, so “those who knows where to tune and fix” will have a much clearer idea of the issue.
IN REGARDS TO LTX DESKTOP AND MLX:
Technically LXT Desktop still uses MLX under the hood on macOS, so that works for sure. As you correctly said though, it’s less flexible than ComfyUI. Whatever is happening here, I think that the issue is in the middle between ComfyUI’s implementation of Nested Tensors and LTX’s architecture (again it’s only speculation on my side).
What makes this issue nasty to diagnose, is the fact that ComfyUI doesn’t warn you at all about the nan corruption. It fails with PyAV error 22 and sends you completely “off road” in searching a solution, while the real issue is elsewhere.
I’m pretty optimistic that this will be solved ;) The AI community is amazing at collaborating and, on my side, I have no intention in leaving this puzzle unsolved :P
I’ll do my best to keep you updated with my findings and I will try to share the workflow. In the meantime give it a try yourself and also check that GGUF merge ;)
Tried the default comfyui workflow now. Macbook pro, m4 max, 36gb mem. Worked just fine.
Good News 😁 the issue has been found and reported on ComfyUI's GitHub: https://github.com/Comfy-Org/ComfyUI/issues/15804
Good News 😁 the issue has been found and reported on ComfyUI's GitHub: https://github.com/Comfy-Org/ComfyUI/issues/15804
Thanks for taking the time from vacation to drop this in!
As to your previous comment; I believe I did all of those steps correctly, however once you mentioned that the type of Apple chip mattered I realized it might help to know what I'm working with here;
I have an M1 MAX with 64GB RAM. I don't think any OOM would be the problem here. As I mentioned as well, I can run the LTX Desktop app just fine, run it and get results but the settings are very limited and there's no option to queue jobs in the LTX Desktop app and come back to compare results.
Enjoy your vacation! Excited to see if your workflow happens to fix it (I know it's the basic one, but it's my best shot at this point lmao)
Ah well I actually got it working using the fix detailed in that post! Thanks a ton. I will be closing the subject
Significantly better quality using LTX Desktop unfortunately! Really solid generations, really good outputs and coherence, and for some reason the audio is significantly better using the dedicated LTX desktop app. While the resolutions are more limited, it seems like they have those restrictions in place for a reason.
Sorry for my delayed answer again 😝 I'm happy that I managed to help 😉 Seems that the comfy team is on the issue; let's hope for a quick official fix too.
As for the quality: it really really depends on the type of workflow you use and from you starting image; if you go with Image 2 Video (which is always better imho).
I managed to get an amazing quality by feeding very detailed Images, that were generated with Krea 2, upscaled and refined with Flux 2 Klein 9b up to 2 MP and the downscaled using lanczos into the ImageTVideoInplace Node. The results are much much better than LTX 2.3. More natural movements, understands and retains fast motion (which was a nightmare on 2.3) and hallucinates far less.
I'm currently using a 3 Stages workflow (3 kSampler with 2 Latent Upsamplers in the middle). The results have been amazing so far: I can reach 836p with a fantastic quality in less than 10 minutes. Then I use DaVinci Resolve free to push it up to 1080p or even 2K. The quality holds up great and the artifacts on faces are definitely gone (another issue with 2.3).
The only issue I have is still a bit of jitter in fast moving scenes or fast action, but again this is easily solvable with DaVinci.
For audio: it depends on the resolution you start with and the destination one. If Stage 2 is at 736p, the audio will mostly fox itself after the second pass. If you push it up at 1.2 MP it will be perfect after Stage 2. If you use 3 Stages, it fill fix much better.
So it really comes down to the quality of your initial image 😉 if you need any advice let me know ☺️ I know a few upscaling tricks that work wonders with LTX 2.5. For example: just by altering the Custom Sigmas at Stage 2 or 3, you can get a much more refined result without losing the motion, audio and details of the previous stages. This works for audio too ⚡️