`.to` is not supported for `4-bit` or `8-bit` bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct `dtype`.
Author: falali009
Pull request: no
Labels: name: bug

### System Info

  error info is： 

Joy_caption_two_advanced
`.to` is not supported for `4-bit` or `8-bit` bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct `dtype`.


# ComfyUI Error Report
## Error Details
- **Node ID:** 64
- **Node Type:** Joy_caption_two_advanced
- **Exception Type:** ValueError
- **Exception Message:** `.to` is not supported for `4-bit` or `8-bit` bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct `dtype`.
## Stack Trace
```
  File "/home/falali/flux/ComfyUI/ComfyUI/execution.py", line 349, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)

  File "/home/falali/flux/ComfyUI/ComfyUI/execution.py", line 224, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)

  File "/home/falali/flux/ComfyUI/ComfyUI/execution.py", line 196, in _map_node_over_list
    process_inputs(input_dict, i)

  File "/home/falali/flux/ComfyUI/ComfyUI/execution.py", line 185, in process_inputs
    results.append(getattr(obj, func)(**inputs))

  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two/joy_caption_two_node.py", line 546, in generate
    text_model = joy_two_pipeline.llm.load_llm_model()

  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two/joy_caption_two_node.py", line 177, in load_llm_model
    text_model = AutoModelForCausalLM.from_pretrained(text_model_path,

  File "/home/falali/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
    return model_class.from_pretrained(

  File "/home/falali/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4015, in from_pretrained
    dispatch_model(model, **device_map_kwargs)

  File "/home/falali/.local/lib/python3.10/site-packages/accelerate/big_modeling.py", line 501, in dispatch_model
    model.to(device)

  File "/home/falali/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2861, in to
    raise ValueError(

```
## System Information
- **ComfyUI Version:** 0.3.40
- **Arguments:** main.py --listen 10.201.10.37
- **OS:** posix
- **Python Version:** 3.10.12 (main, Feb  4 2025, 14:57:36) [GCC 11.4.0]
- **Embedded Python:** false
- **PyTorch Version:** 2.6.0+cu124
## Devices

- **Name:** cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
  - **Type:** cuda
  - **VRAM Total:** 25386352640
  - **VRAM Free:** 23902150840
  - **Torch VRAM Total:** 1006632960
  - **Torch VRAM Free:** 73588920

## Logs
```
2025-06-10T14:58:02.194566 - 
2025-06-10T14:58:02.239783 - [34m[ComfyUI-Easy-Use] server: [0mv1.2.8 [92mLoaded[0m2025-06-10T14:58:02.239806 - 
2025-06-10T14:58:02.239820 - [34m[ComfyUI-Easy-Use] web root: [0m/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-easy-use/web_version/v2 [92mLoaded[0m2025-06-10T14:58:02.239829 - 
2025-06-10T14:58:02.248932 - Traceback (most recent call last):
  File "/home/falali/flux/ComfyUI/ComfyUI/nodes.py", line 2124, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-tensorops/__init__.py", line 1, in <module>
    from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-tensorops/nodes/__init__.py", line 6, in <module>
    from .save_to_s3 import SaveImageToS3
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-tensorops/nodes/save_to_s3.py", line 2, in <module>
    import boto3
ModuleNotFoundError: No module named 'boto3'

2025-06-10T14:58:02.249078 - Cannot import /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-tensorops module for custom nodes: No module named 'boto3'
2025-06-10T14:58:02.264805 - [36;20m[/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfy-mtb] | INFO -> loaded [96m94[0m nodes successfuly[0m
2025-06-10T14:58:02.264980 - [36;20m[/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfy-mtb] | INFO -> Some nodes (2) could not be loaded. This can be ignored, but go to http://10.201.10.37:8188/mtb if you want more information.[0m
2025-06-10T14:58:02.266087 - ### Loading: ComfyUI-Impact-Pack (V8.8.1)2025-06-10T14:58:02.266099 - 
2025-06-10T14:58:02.276373 - [Impact Pack] Wildcards loading done.2025-06-10T14:58:02.276396 - 
2025-06-10T14:58:02.278422 - Nvidia APEX normalization not installed, using PyTorch LayerNorm2025-06-10T14:58:02.278436 - 
2025-06-10T14:58:02.324117 - Traceback (most recent call last):
  File "/home/falali/flux/ComfyUI/ComfyUI/nodes.py", line 2124, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-crewai/__init__.py", line 1, in <module>
    from .nodes.llm_ollama_node import LlmOllama
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-crewai/nodes/llm_ollama_node.py", line 2, in <module>
    from crewai import LLM
ModuleNotFoundError: No module named 'crewai'

2025-06-10T14:58:02.324206 - Cannot import /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-crewai module for custom nodes: No module named 'crewai'
2025-06-10T14:58:02.346286 - ComfyUI found: /home/falali/flux/ComfyUI/ComfyUI2025-06-10T14:58:02.346305 - 
2025-06-10T14:58:02.346318 - '/home/falali/flux/ComfyUI/ComfyUI' added to sys.path2025-06-10T14:58:02.346325 - 
2025-06-10T14:58:02.442416 - [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
2025-06-10T14:58:02.528532 - [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
2025-06-10T14:58:02.532860 - [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
2025-06-10T14:58:02.734109 - [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
2025-06-10T14:58:02.805116 - [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
2025-06-10T14:58:03.165716 - 导入 whisper 时出错：PortAudio library not found2025-06-10T14:58:03.165772 - 
2025-06-10T14:58:03.260693 - 导入 movie_editor 时出错：No module named 'moviepy.editor'2025-06-10T14:58:03.260713 - 
2025-06-10T14:58:03.316714 - 导入 dall_e 时出错：PortAudio library not found2025-06-10T14:58:03.316825 - 
2025-06-10T14:58:04.998892 - llama-cpp installed2025-06-10T14:58:04.998915 - 
2025-06-10T14:58:06.475125 - Successfully installed py-cord[voice]2025-06-10T14:58:06.475148 - 
2025-06-10T14:58:06.475637 - ### Loading: ComfyUI-Impact-Subpack (V1.2.9)
2025-06-10T14:58:06.499286 - [Impact Subpack] ultralytics_bbox: /home/falali/flux/ComfyUI/ComfyUI/models/ultralytics/bbox
2025-06-10T14:58:06.499332 - [Impact Subpack] ultralytics_segm: /home/falali/flux/ComfyUI/ComfyUI/models/ultralytics/segm
2025-06-10T14:58:06.499576 - ASTERR config loaded successfully2025-06-10T14:58:06.499589 - 
2025-06-10T14:58:06.504610 - [36;20m[/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_controlnet_aux] | INFO -> Using ckpts path: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_controlnet_aux/ckpts[0m
2025-06-10T14:58:06.504719 - [36;20m[/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_controlnet_aux] | INFO -> Using symlinks: False[0m
2025-06-10T14:58:06.504789 - [36;20m[/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider'][0m
2025-06-10T14:58:06.512948 - Nvidia APEX normalization not installed, using PyTorch LayerNorm2025-06-10T14:58:06.512964 - 
2025-06-10T14:58:06.556636 - Web extensions folder found at /home/falali/flux/ComfyUI/ComfyUI/web/extensions/ComfyLiterals2025-06-10T14:58:06.556656 - 
2025-06-10T14:58:06.611706 - [34mPony Character Prompt Picker: [92mLoaded[0m2025-06-10T14:58:06.611726 - 
2025-06-10T14:58:06.634564 - 
[36mEfficiency Nodes:[0m Attempting to add Control Net options to the 'HiRes-Fix Script' Node (comfyui_controlnet_aux add-on)...[92mSuccess![0m2025-06-10T14:58:06.634583 - 
2025-06-10T14:58:06.684678 - [34mWAS Node Suite: [0mBlenderNeko's Advanced CLIP Text Encode found, attempting to enable `CLIPTextEncode` support.[0m2025-06-10T14:58:06.684698 - 
2025-06-10T14:58:06.684731 - [34mWAS Node Suite: [0m`CLIPTextEncode (BlenderNeko Advanced + NSP)` node enabled under `WAS Suite/Conditioning` menu.[0m2025-06-10T14:58:06.684738 - 
2025-06-10T14:58:07.055776 - [34mWAS Node Suite: [0mOpenCV Python FFMPEG support is enabled[0m2025-06-10T14:58:07.055800 - 
2025-06-10T14:58:07.055821 - [34mWAS Node Suite [93mWarning: [0m`ffmpeg_bin_path` is not set in `/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/was-node-suite-comfyui/was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.[0m2025-06-10T14:58:07.055851 - 
2025-06-10T14:58:07.432866 - [34mWAS Node Suite: [0mFinished.[0m [32mLoaded[0m [0m221[0m [32mnodes successfully.[0m2025-06-10T14:58:07.432889 - 
2025-06-10T14:58:07.432903 - 
	[3m[93m"Success usually comes to those who are too busy to be looking for it."[0m[3m - Henry David Thoreau[0m
2025-06-10T14:58:07.432910 - 
2025-06-10T14:58:07.434387 - pruna_pro not installed, skipping2025-06-10T14:58:07.434400 - 
2025-06-10T14:58:07.434557 - Neither pruna_pro nor pruna are installed, skipping2025-06-10T14:58:07.434564 - 
2025-06-10T14:58:07.434777 - pruna_pro not installed, skipping2025-06-10T14:58:07.434785 - 
2025-06-10T14:58:07.434937 - Neither pruna_pro nor pruna are installed, skipping2025-06-10T14:58:07.434945 - 
2025-06-10T14:58:07.436459 - Warning: Could not load sageattention: No module named 'sageattention'2025-06-10T14:58:07.436471 - 
2025-06-10T14:58:07.436479 - sageattention package is not installed2025-06-10T14:58:07.436484 - 
2025-06-10T14:58:07.444099 - ------------------------------------------2025-06-10T14:58:07.444121 - 
2025-06-10T14:58:07.444129 - [34mComfyroll Studio v1.76 : [92m 175 Nodes Loaded[0m2025-06-10T14:58:07.444146 - 
2025-06-10T14:58:07.444152 - ------------------------------------------2025-06-10T14:58:07.444158 - 
2025-06-10T14:58:07.444164 - ** For changes, please see patch notes at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/blob/main/Patch_Notes.md2025-06-10T14:58:07.444169 - 
2025-06-10T14:58:07.444175 - ** For help, please see the wiki at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/wiki2025-06-10T14:58:07.444181 - 
2025-06-10T14:58:07.444187 - ------------------------------------------2025-06-10T14:58:07.444193 - 
2025-06-10T14:58:07.451426 - 
[0;33m[ReActor][0m - [38;5;173mSTATUS[0m - [0;32mRunning v0.6.0-a1 in ComfyUI[0m2025-06-10T14:58:07.451441 - 
2025-06-10T14:58:07.462346 - Torch version: 2.6.0+cu1242025-06-10T14:58:07.462361 - 
2025-06-10T14:58:07.467590 - (pysssss:WD14Tagger) [DEBUG] Available ORT providers: TensorrtExecutionProvider, CUDAExecutionProvider, CPUExecutionProvider2025-06-10T14:58:07.467603 - 
2025-06-10T14:58:07.467611 - (pysssss:WD14Tagger) [DEBUG] Using ORT providers: CUDAExecutionProvider, CPUExecutionProvider2025-06-10T14:58:07.467617 - 
2025-06-10T14:58:07.841865 - Traceback (most recent call last):
  File "/home/falali/flux/ComfyUI/ComfyUI/nodes.py", line 2124, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-if_ai_promptimagem/__init__.py", line 14, in <module>
    from .IFPromptImaGENNode import IFPROMPTImaGEN
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-if_ai_promptimagem/IFPromptImaGENNode.py", line 13, in <module>
    from .send_request import send_request
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-if_ai_promptimagem/send_request.py", line 27, in <module>
    from .transformers_api import TransformersModelManager
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-if_ai_promptimagem/transformers_api.py", line 2, in <module>
    from transformers import (
ImportError: cannot import name 'Qwen2VLForConditionalGeneration' from 'transformers' (/home/falali/.local/lib/python3.10/site-packages/transformers/__init__.py)

2025-06-10T14:58:07.842053 - Cannot import /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-if_ai_promptimagem module for custom nodes: cannot import name 'Qwen2VLForConditionalGeneration' from 'transformers' (/home/falali/.local/lib/python3.10/site-packages/transformers/__init__.py)
2025-06-10T14:58:07.851922 - Traceback (most recent call last):
  File "/home/falali/flux/ComfyUI/ComfyUI/nodes.py", line 2124, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-sam2/__init__.py", line 1, in <module>
    from .node import SAM2ModelLoader, GroundingDinoModelLoader, GroundingDinoSAM2Segment, InvertMask, IsMaskEmptyNode
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-sam2/node.py", line 15, in <module>
    from sam2.build_sam import build_sam2
ModuleNotFoundError: No module named 'sam2.build_sam'

2025-06-10T14:58:07.852009 - Cannot import /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-sam2 module for custom nodes: No module named 'sam2.build_sam'
2025-06-10T14:58:08.019776 - --------------
2025-06-10T14:58:08.019836 - [91m ### Mixlab Nodes: [93mLoaded
2025-06-10T14:58:08.027065 - json_repair## OK2025-06-10T14:58:08.027081 - 
2025-06-10T14:58:08.027700 - ChatGPT.available True
2025-06-10T14:58:08.027824 - edit_mask.available True
2025-06-10T14:58:08.077079 - ## clip_interrogator_model not found: /home/falali/flux/ComfyUI/ComfyUI/models/clip_interrogator/Salesforce/blip-image-captioning-base, pls download from https://huggingface.co/Salesforce/blip-image-captioning-base2025-06-10T14:58:08.077100 - 
2025-06-10T14:58:08.077167 - ClipInterrogator.available True
2025-06-10T14:58:08.077335 - ## text_generator_model not found: /home/falali/flux/ComfyUI/ComfyUI/models/prompt_generator/text2image-prompt-generator, pls download from https://huggingface.co/succinctly/text2image-prompt-generator/tree/main2025-06-10T14:58:08.077344 - 
2025-06-10T14:58:08.077354 - ## zh_en_model not found: /home/falali/flux/ComfyUI/ComfyUI/models/prompt_generator/opus-mt-zh-en, pls download from https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main2025-06-10T14:58:08.077360 - 
2025-06-10T14:58:08.077549 - PromptGenerate.available True
2025-06-10T14:58:08.077574 - ChinesePrompt.available True
2025-06-10T14:58:08.077603 - RembgNode_.available True
2025-06-10T14:58:08.143247 - TripoSR.available
2025-06-10T14:58:08.143461 - MiniCPMNode.available
2025-06-10T14:58:08.169474 - Scenedetect.available
2025-06-10T14:58:08.200629 - FishSpeech.available
2025-06-10T14:58:08.202656 - SenseVoice.available
2025-06-10T14:58:08.211300 - Whisper.available False
2025-06-10T14:58:08.211503 - fal-client## OK2025-06-10T14:58:08.211513 - 
2025-06-10T14:58:08.214822 - FalVideo.available
2025-06-10T14:58:08.214865 - [93m -------------- [0m
2025-06-10T14:58:08.218437 - ----------Jake Upgrade Nodes Loaded----------2025-06-10T14:58:08.218450 - 
2025-06-10T14:58:08.250690 - Use Proxy:2025-06-10T14:58:08.258488 -  2025-06-10T14:58:08.258569 - http://127.0.0.1:7897/2025-06-10T14:58:08.258674 - 
2025-06-10T14:58:08.260241 - Traceback (most recent call last):
  File "/home/falali/flux/ComfyUI/ComfyUI/nodes.py", line 2124, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/Bjornulf_custom_nodes/__init__.py", line 74, in <module>
    from .ollama_talk import OllamaTalk
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/Bjornulf_custom_nodes/ollama_talk.py", line 15, in <module>
    import ollama
  File "/home/falali/.local/lib/python3.10/site-packages/ollama/__init__.py", line 40, in <module>
    _client = Client()
  File "/home/falali/.local/lib/python3.10/site-packages/ollama/_client.py", line 114, in __init__
    super().__init__(httpx.Client, host, **kwargs)
  File "/home/falali/.local/lib/python3.10/site-packages/ollama/_client.py", line 91, in __init__
    self._client = client(
  File "/home/falali/.local/lib/python3.10/site-packages/httpx/_client.py", line 693, in __init__
    proxy_map = self._get_proxy_map(proxies or proxy, allow_env_proxies)
  File "/home/falali/.local/lib/python3.10/site-packages/httpx/_client.py", line 218, in _get_proxy_map
    return {
  File "/home/falali/.local/lib/python3.10/site-packages/httpx/_client.py", line 219, in <dictcomp>
    key: None if url is None else Proxy(url=url)
  File "/home/falali/.local/lib/python3.10/site-packages/httpx/_config.py", line 338, in __init__
    raise ValueError(f"Unknown scheme for proxy URL {url!r}")
ValueError: Unknown scheme for proxy URL URL('socks://127.0.0.1:7897/')

2025-06-10T14:58:08.260551 - Cannot import /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/Bjornulf_custom_nodes module for custom nodes: Unknown scheme for proxy URL URL('socks://127.0.0.1:7897/')
2025-06-10T14:58:08.263666 - Total VRAM 24210 MB, total RAM 64068 MB
2025-06-10T14:58:08.263705 - pytorch version: 2.6.0+cu124
2025-06-10T14:58:08.263739 - xformers version: 0.0.29.post3
2025-06-10T14:58:08.263789 - Set vram state to: NORMAL_VRAM
2025-06-10T14:58:08.263845 - Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
2025-06-10T14:58:08.668338 - 
Import times for custom nodes:
2025-06-10T14:58:08.668409 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/websocket_image_save.py
2025-06-10T14:58:08.668431 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-PonyCharacterPrompt
2025-06-10T14:58:08.668449 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-WanVideoKsampler
2025-06-10T14:58:08.668465 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI_AdvancedRefluxControl
2025-06-10T14:58:08.668480 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-img2drawingassistants
2025-06-10T14:58:08.668494 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_ttp_toolset
2025-06-10T14:58:08.668507 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-inpaint-cropandstitch
2025-06-10T14:58:08.668521 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/cg-use-everywhere
2025-06-10T14:58:08.668535 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/asterr
2025-06-10T14:58:08.668549 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI_ADV_CLIP_emb
2025-06-10T14:58:08.668562 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_faceanalysis
2025-06-10T14:58:08.668576 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-UltimateSDUpscale-GGUF
2025-06-10T14:58:08.668589 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-wd14-tagger
2025-06-10T14:58:08.668601 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-YOLO
2025-06-10T14:58:08.668615 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_zenid
2025-06-10T14:58:08.668628 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/aurasr-comfyui
2025-06-10T14:58:08.668642 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_instantid
2025-06-10T14:58:08.668655 -    0.0 seconds (IMPORT FAILED): /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-sam2
2025-06-10T14:58:08.668672 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-GGUF
2025-06-10T14:58:08.668690 -    0.0 seconds (IMPORT FAILED): /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-crewai
2025-06-10T14:58:08.668704 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-template-loader
2025-06-10T14:58:08.668721 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyLiterals
2025-06-10T14:58:08.668735 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfy-image-saver
2025-06-10T14:58:08.668747 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI_pruna
2025-06-10T14:58:08.668760 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-custom-scripts
2025-06-10T14:58:08.668773 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/teacache
2025-06-10T14:58:08.668785 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-Jjk-Nodes
2025-06-10T14:58:08.668797 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_patches_ll
2025-06-10T14:58:08.668810 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-QualityOfLifeSuit_Omar92
2025-06-10T14:58:08.668822 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-WanSeamlessFlow
2025-06-10T14:58:08.668838 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_essentials
2025-06-10T14:58:08.668853 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-advancedliveportrait
2025-06-10T14:58:08.668891 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/derfuu_comfyui_moddednodes
2025-06-10T14:58:08.668903 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-frame-interpolation
2025-06-10T14:58:08.668916 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-brushnet
2025-06-10T14:58:08.668929 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-segment-anything-2
2025-06-10T14:58:08.668943 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/rgthree-comfy
2025-06-10T14:58:08.668955 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-jakeupgrade
2025-06-10T14:58:08.668969 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-advanced-controlnet
2025-06-10T14:58:08.668982 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two
2025-06-10T14:58:08.668995 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-WanVideoWrapper
2025-06-10T14:58:08.669008 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI_UltimateSDUpscale
2025-06-10T14:58:08.669020 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/gguf
2025-06-10T14:58:08.669033 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_ipadapter_plus
2025-06-10T14:58:08.669045 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes-main
2025-06-10T14:58:08.669058 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI_MiniCPM-V-2_6-int4
2025-06-10T14:58:08.669070 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_controlnet_aux
2025-06-10T14:58:08.669082 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/efficiency-nodes-comfyui
2025-06-10T14:58:08.669094 -    0.0 seconds (IMPORT FAILED): /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-tensorops
2025-06-10T14:58:08.669108 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-impact-pack
2025-06-10T14:58:08.669123 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_segment_anything
2025-06-10T14:58:08.669135 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-Crystools
2025-06-10T14:58:08.669147 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/Comfyui_Redux_Advanced
2025-06-10T14:58:08.669160 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-kjnodes
2025-06-10T14:58:08.669173 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfy-mtb
2025-06-10T14:58:08.669185 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-reactor
2025-06-10T14:58:08.669198 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_layerstyle
2025-06-10T14:58:08.669211 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-to-python-extension
2025-06-10T14:58:08.669223 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-impact-subpack
2025-06-10T14:58:08.669236 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-videohelpersuite
2025-06-10T14:58:08.669248 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-manager
2025-06-10T14:58:08.669260 -    0.0 seconds (IMPORT FAILED): /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/Bjornulf_custom_nodes
2025-06-10T14:58:08.669273 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-easy-use
2025-06-10T14:58:08.669286 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-PuLID-Flux-Enhanced
2025-06-10T14:58:08.669298 -    0.0 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/pulid_comfyui
2025-06-10T14:58:08.669310 -    0.1 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-florence2
2025-06-10T14:58:08.669324 -    0.2 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-dynamicprompts
2025-06-10T14:58:08.669338 -    0.3 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/ComfyUI-BrushNet-Wrapper
2025-06-10T14:58:08.669352 -    0.4 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-mixlab-nodes
2025-06-10T14:58:08.669366 -    0.4 seconds (IMPORT FAILED): /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-if_ai_promptimagem
2025-06-10T14:58:08.669381 -    0.4 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/bjornulf_custom_nodes
2025-06-10T14:58:08.669395 -    0.8 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/was-node-suite-comfyui
2025-06-10T14:58:08.669409 -    2.2 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_pulid_flux_ll
2025-06-10T14:58:08.669421 -    4.1 seconds: /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_llm_party
2025-06-10T14:58:08.669435 - 
2025-06-10T14:58:08.679847 - ********** ERROR ***********

comfyui-workflow-templates is not installed.

Please install the updated requirements.txt file by running:
/usr/bin/python3 -m pip install -r /home/falali/flux/ComfyUI/ComfyUI/requirements.txt

This error is happening because the ComfyUI frontend is no longer shipped as part of the main repo but as a pip package instead.

If you are on the portable package you can run: update\update_comfyui.bat to solve this problem

********** ERROR ***********
2025-06-10T14:58:08.680055 - comfyui-embedded-docs package not found
2025-06-10T14:58:08.680541 - Starting server

2025-06-10T14:58:08.680685 - To see the GUI go to: http://10.201.10.37:8188
2025-06-10T14:58:09.124814 - FETCH ComfyRegistry Data: 5/882025-06-10T14:58:09.124914 - 
2025-06-10T14:58:09.626135 - Use Proxy:2025-06-10T14:58:09.626214 -  2025-06-10T14:58:09.626245 - http://127.0.0.1:7897/2025-06-10T14:58:09.626277 - 
2025-06-10T14:58:11.038091 - Use Proxy:2025-06-10T14:58:11.038172 -  2025-06-10T14:58:11.038204 - http://127.0.0.1:7897/2025-06-10T14:58:11.038238 - 
2025-06-10T14:58:12.399041 - /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-mixlab-nodes/webApp/lib/photoswipe-lightbox.esm.min.js2025-06-10T14:58:12.399076 - 
2025-06-10T14:58:12.408340 - /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-mixlab-nodes/webApp/lib/pickr.min.js2025-06-10T14:58:12.408379 - 
2025-06-10T14:58:12.433202 - /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-mixlab-nodes/webApp/lib/photoswipe.min.css2025-06-10T14:58:12.433262 - 
2025-06-10T14:58:12.453382 - /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-mixlab-nodes/webApp/lib/classic.min.css2025-06-10T14:58:12.453441 - 
2025-06-10T14:58:12.478580 - /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-mixlab-nodes/webApp/lib/model-viewer.min.js2025-06-10T14:58:12.478595 - 
2025-06-10T14:58:12.480704 - /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-mixlab-nodes/webApp/lib/juxtapose.css2025-06-10T14:58:12.480717 - 
2025-06-10T14:58:12.480836 - /home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui-mixlab-nodes/webApp/lib/juxtapose.min.js2025-06-10T14:58:12.480845 - 
2025-06-10T14:58:12.488751 - Use Proxy:2025-06-10T14:58:12.488764 -  2025-06-10T14:58:12.488771 - http://127.0.0.1:7897/2025-06-10T14:58:12.488777 - 
2025-06-10T14:58:12.571749 - [33mQualityOfLifeSuit_Omar92:[0m:NSP ready2025-06-10T14:58:12.571767 - 
2025-06-10T14:58:13.827379 - Use Proxy:2025-06-10T14:58:13.827462 -  2025-06-10T14:58:13.827504 - http://127.0.0.1:7897/2025-06-10T14:58:13.827540 - 
2025-06-10T14:58:15.157581 - Use Proxy:2025-06-10T14:58:15.157661 -  2025-06-10T14:58:15.157694 - http://127.0.0.1:7897/2025-06-10T14:58:15.157727 - 
2025-06-10T14:58:15.998713 - FETCH ComfyRegistry Data: 10/882025-06-10T14:58:15.998791 - 
2025-06-10T14:58:16.499447 - Use Proxy:2025-06-10T14:58:16.499517 -  2025-06-10T14:58:16.499551 - http://127.0.0.1:7897/2025-06-10T14:58:16.499585 - 
2025-06-10T14:58:17.272994 - got prompt
2025-06-10T14:58:17.317800 - Using xformers attention in VAE
2025-06-10T14:58:17.318617 - Using xformers attention in VAE
2025-06-10T14:58:17.451959 - VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
2025-06-10T14:58:17.479966 - /home/falali/flux/ComfyUI/ComfyUI/models/clip/siglip-so400m-patch14-3842025-06-10T14:58:17.479993 - 
2025-06-10T14:58:17.808593 - Use Proxy:2025-06-10T14:58:17.808630 -  2025-06-10T14:58:17.808647 - http://127.0.0.1:7897/2025-06-10T14:58:17.808662 - 
2025-06-10T14:58:18.169537 - Loading VLM's custom vision model2025-06-10T14:58:18.169562 - 
2025-06-10T14:58:18.767697 - Prompt: Write a long descriptive caption for this image in a formal tone. If there is a person/character in the image you must refer to them as .2025-06-10T14:58:18.767724 - 
2025-06-10T14:58:18.814337 - Requested to load SiglipVisionTransformer
2025-06-10T14:58:18.890225 - loaded completely 9.5367431640625e+25 809.1729736328125 True
2025-06-10T14:58:19.004774 - Requested to load ImageAdapter
2025-06-10T14:58:19.008798 - loaded completely 9.5367431640625e+25 41.0390625 True
2025-06-10T14:58:19.010460 - Loading tokenizer2025-06-10T14:58:19.010476 - 
2025-06-10T14:58:19.134628 - Use Proxy:2025-06-10T14:58:19.134649 -  2025-06-10T14:58:19.134656 - http://127.0.0.1:7897/2025-06-10T14:58:19.134663 - 
2025-06-10T14:58:19.135599 - Loading LLM: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit2025-06-10T14:58:19.135627 - 
2025-06-10T14:58:19.135641 - /home/falali/flux/ComfyUI/ComfyUI/models/LLM/Meta-Llama-3.1-8B-Instruct-bnb-4bit2025-06-10T14:58:19.135648 - 
2025-06-10T14:58:19.135803 - Successfully modified 'base_model_name_or_path' value in '/home/falali/flux/ComfyUI/ComfyUI/models/Joy_caption_two/text_model/adapter_config.json'.2025-06-10T14:58:19.135813 - 
2025-06-10T14:58:20.443970 - Use Proxy:2025-06-10T14:58:20.444002 -  2025-06-10T14:58:20.444009 - http://127.0.0.1:7897/2025-06-10T14:58:20.444028 - 
2025-06-10T14:58:21.037028 - !!! Exception during processing !!! `.to` is not supported for `4-bit` or `8-bit` bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct `dtype`.
2025-06-10T14:58:21.037817 - Traceback (most recent call last):
  File "/home/falali/flux/ComfyUI/ComfyUI/execution.py", line 349, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "/home/falali/flux/ComfyUI/ComfyUI/execution.py", line 224, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "/home/falali/flux/ComfyUI/ComfyUI/execution.py", line 196, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/falali/flux/ComfyUI/ComfyUI/execution.py", line 185, in process_inputs
    results.append(getattr(obj, func)(**inputs))
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two/joy_caption_two_node.py", line 546, in generate
    text_model = joy_two_pipeline.llm.load_llm_model()
  File "/home/falali/flux/ComfyUI/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two/joy_caption_two_node.py", line 177, in load_llm_model
    text_model = AutoModelForCausalLM.from_pretrained(text_model_path,
  File "/home/falali/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
    return model_class.from_pretrained(
  File "/home/falali/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4015, in from_pretrained
    dispatch_model(model, **device_map_kwargs)
  File "/home/falali/.local/lib/python3.10/site-packages/accelerate/big_modeling.py", line 501, in dispatch_model
    model.to(device)
  File "/home/falali/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2861, in to
    raise ValueError(
ValueError: `.to` is not supported for `4-bit` or `8-bit` bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct `dtype`.

2025-06-10T14:58:21.038033 - Prompt executed in 3.76 seconds

```
## Attached Workflow
Please make sure that workflow does not contain any sensitive information such as API keys or passwords.
```
Workflow too large. Please manually upload the workflow from local file system.
```

## Additional Context
(Please add any additional context or steps to reproduce the error here)


### Who can help?

![Image](https://github.com/user-attachments/assets/28702556-60d4-46dd-86d5-d214ab9b39d2)

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

![Image](https://github.com/user-attachments/assets/cef17280-a9ed-41fb-a70f-131f642c8018)

### Expected behavior

hope it run 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.