Repository: pytorch
Issue: convolution_overrideable not implemented when executing VAEEncode in ComfyUI
Labels: triaged,module: PrivateUse1

### 🐛 Describe the bug

In ComfyUI, I am trying to run Inpaint Preprocessor with Load Checkpoint (Realistic Vision V5.1 Inpainting) and ControlNet Model (control_v11p_sd15_inpaint) to inpaint a masked area of a loaded image based on a text prompt. Whenever the process reaches VAE Encode node, the following error message emerges:
'''
"!!! Exception during processing!!! convolution_overrideable not implemented. You are likely triggering this with tensor backend other than CPU/CUDA/MKLDNN, if this is intended, please use TORCH_LIBRARY_IMPL to override this function 
Traceback (most recent call last):
  File "/Users/yegorartyukh/ComfyUI/execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/nodes.py", line 296, in encode
    t = vae.encode(pixels[:,:,:,:3])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/comfy/sd.py", line 346, in encode
    samples[x:x+batch_number] = self.first_stage_model.encode(pixels_in).to(self.output_device).float()
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/comfy/ldm/models/autoencoder.py", line 179, in encode
    z = self.encoder(x)
        ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/comfy/ldm/modules/diffusionmodules/model.py", line 520, in forward
    h = self.conv_in(x)
        ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/comfy/ops.py", line 93, in forward
    return super().forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 549, in forward
    return self._conv_forward(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 544, in _conv_forward
    return F.conv2d(
           ^^^^^^^^^
NotImplementedError: convolution_overrideable not implemented. You are likely triggering this with tensor backend other than CPU/CUDA/MKLDNN, if this is intended, please use TORCH_LIBRARY_IMPL to override this function
'''
<img width="1920" alt="Screenshot 2024-08-04 at 23 24 15" src="https://github.com/user-attachments/assets/36c4c1b3-0c98-4581-bb33-4b7959163b04">
<img width="1920" alt="Screenshot 2024-08-04 at 23 24 26" src="https://github.com/user-attachments/assets/d55b3ba0-575b-44a8-9d32-ef28be73b87f">


My tech specs:
MacBook Air M3 2024
16GB
macOS Sonoma 14.3
Python 3.11
PyTorch 2.5 Preview (Nightly)


I was trying different solutions and software versions:
1. Python versions 3.10, 3.11, 3.12.
2. Different PyTorch versions.
3. Setting a device explicitly to mps and/or CPU in the main script.


Steps to Reproduce:
1. Launch ComfyUI on MacBook M3.
2. Select Realistic Vision model v5.1 model in the Load Checkpoint node.
3. Select ControlNet inpaint model in the Load ControlNet Model node.
4. Load an image and create a mask in MaskEditor.
5. Queue prompt.

Please note that I have just started playing around with these things, so I apologize if my description is not clear enough! Many thanks for any advice!

### Versions

PyTorch version: 2.5.0.dev20240804
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.3 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.3.9.4)
CMake version: version 3.30.2
Libc version: N/A

Python version: 3.11.9 (main, Apr  2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)] (64-bit runtime)
Python platform: macOS-14.3-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M3

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.5.0.dev20240804
[pip3] torchaudio==2.4.0.dev20240804
[pip3] torchsde==0.2.6
[pip3] torchvision==0.20.0.dev20240804
[conda] Could not collect

cc @NmomoN @mengpenghui @fwenguang @cdzhan @1274085042 @PHLens