'float8_e4m3fn' attribute error
#17
by
Magenta6
- opened
I have a 4070Ti with 12GB VRAM.
I'm running Python 3.10.9 and Pytorch version 2.0.1+cu118
And have been runny comfyUi for ages.
I get this error:
module 'torch' has no attribute 'float8_e4m3fn'
A websearch suggested a fix via:
"pip install --upgrade torch torchvision torchaudio -r requirements.txt"
Although I cannot find the file [requirements.txt] I've updated torch (2.4.0), torchaudio (2.4.0) and torchvision (0.19.0)
Any suggestions would be appreciated.
Thanks in advance!
After updating pytorch you still got the error? In theory it's just outdated pytorch issue.
Which version of transformers do you have? If you haven't, you could try updating it. I updated to transformers>=4.43.2 and I believe that fixed this error for me.
actually,when i updated to transformers>=4.44.2,it didnot work.
it still show the error——""fill_empty_deterministic_" not implemented for 'Float8_e4m3fn'"
This really makes me feel tired, I can't use the fp8 version of the clip text encoder，hope someone guys help
Same issue over here.
Here is the breakdown:
# ComfyUI Error Report
## Error Details
- **Node Type:** DualCLIPLoader
- **Exception Type:** AttributeError
- **Exception Message:** module 'torch' has no attribute 'float8_e4m3fn'
## Stack Trace
File "/workspace/ComfyUI/execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/workspace/ComfyUI/execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/workspace/ComfyUI/execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "/workspace/ComfyUI/execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
File "/workspace/ComfyUI/nodes.py", line 929, in load_clip
clip = comfy.sd.load_clip(ckpt_paths=[clip_path1, clip_path2], embedding_directory=folder_paths.get_folder_paths("embeddings"), clip_type=clip_type)
File "/workspace/ComfyUI/comfy/sd.py", line 406, in load_clip
clip_data.append(comfy.utils.load_torch_file(p, safe_load=True))
File "/workspace/ComfyUI/comfy/utils.py", line 34, in load_torch_file
sd = safetensors.torch.load_file(ckpt, device=device.type)
File "/venv/lib/python3.10/site-packages/safetensors/torch.py", line 310, in load_file
result[k] = f.get_tensor(k)
## System Information
- **ComfyUI Version:** v0.2.2-75-gfdf3756
- **Arguments:** main.py --listen --port 3000
- **OS:** posix
- **Python Version:** 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
- **Embedded Python:** false
- **PyTorch Version:** 2.0.1+cu118
## Devices
- **Name:** cuda:0 NVIDIA L40 : cudaMallocAsync
- **Type:** cuda
- **VRAM Total:** 47576711168
- **VRAM Free:** 46653561378
- **Torch VRAM Total:** 369098752
- **Torch VRAM Free:** 25156130
## Logs
2024-09-24 10:41:41,935 - root - INFO - Total VRAM 45373 MB, total RAM 2051930 MB
2024-09-24 10:41:41,935 - root - INFO - pytorch version: 2.0.1+cu118
2024-09-24 10:41:42,452 - root - INFO - xformers version: 0.0.22
2024-09-24 10:41:42,452 - root - INFO - Set vram state to: NORMAL_VRAM
2024-09-24 10:41:42,452 - root - INFO - Device: cuda:0 NVIDIA L40 : cudaMallocAsync
2024-09-24 10:41:42,735 - root - INFO - Using xformers cross attention
2024-09-24 10:41:44,096 - root - INFO - [Prompt Server] web root: /workspace/ComfyUI/web
2024-09-24 10:41:45,269 - root - INFO - Total VRAM 45373 MB, total RAM 2051930 MB
2024-09-24 10:41:45,269 - root - INFO - pytorch version: 2.0.1+cu118
2024-09-24 10:41:45,269 - root - INFO - xformers version: 0.0.22
2024-09-24 10:41:45,269 - root - INFO - Set vram state to: NORMAL_VRAM
2024-09-24 10:41:45,270 - root - INFO - Device: cuda:0 NVIDIA L40 : cudaMallocAsync
2024-09-24 10:41:48,997 - root - INFO -
Import times for custom nodes:
2024-09-24 10:41:48,997 - root - INFO - 0.0 seconds: /workspace/ComfyUI/custom_nodes/websocket_image_save.py
2024-09-24 10:41:48,997 - root - INFO - 0.0 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-BRIA_AI-RMBG
2024-09-24 10:41:48,997 - root - INFO - 0.0 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
2024-09-24 10:41:48,997 - root - INFO - 0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyI2I
2024-09-24 10:41:48,997 - root - INFO - 0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet
2024-09-24 10:41:48,997 - root - INFO - 0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI_UltimateSDUpscale
2024-09-24 10:41:48,998 - root - INFO - 0.2 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-KJNodes
2024-09-24 10:41:48,998 - root - INFO - 0.2 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
2024-09-24 10:41:48,998 - root - INFO - 0.2 seconds: /workspace/ComfyUI/custom_nodes/rgthree-comfy
2024-09-24 10:41:48,998 - root - INFO - 0.2 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Inspire-Pack
2024-09-24 10:41:48,998 - root - INFO - 0.5 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Crystools
2024-09-24 10:41:48,998 - root - INFO - 0.5 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-Impact-Pack
2024-09-24 10:41:48,998 - root - INFO - 0.9 seconds: /workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux
2024-09-24 10:41:48,998 - root - INFO - 1.2 seconds: /workspace/ComfyUI/custom_nodes/was-node-suite-comfyui
2024-09-24 10:41:48,999 - root - INFO -
2024-09-24 10:41:49,032 - root - INFO - Starting server
2024-09-24 10:41:49,032 - root - INFO - To see the GUI go to: http://0.0.0.0:3000
2024-09-24 10:41:49,033 - root - INFO - To see the GUI go to: http://[::]:3000
2024-09-24 10:42:30,021 - root - INFO - got prompt
2024-09-24 10:42:30,359 - root - INFO - Using xformers attention in VAE
2024-09-24 10:42:30,362 - root - INFO - Using xformers attention in VAE
2024-09-24 10:42:32,603 - root - INFO - Requested to load AutoencodingEngine
2024-09-24 10:42:32,603 - root - INFO - Loading 1 new model
2024-09-24 10:42:32,625 - root - INFO - loaded completely 0.0 159.87335777282715 True
2024-09-24 10:42:34,562 - root - ERROR - !!! Exception during processing !!! module 'torch' has no attribute 'float8_e4m3fn'
2024-09-24 10:42:34,573 - root - ERROR - Traceback (most recent call last):
File "/workspace/ComfyUI/execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/workspace/ComfyUI/execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/workspace/ComfyUI/execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "/workspace/ComfyUI/execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
File "/workspace/ComfyUI/nodes.py", line 929, in load_clip
clip = comfy.sd.load_clip(ckpt_paths=[clip_path1, clip_path2], embedding_directory=folder_paths.get_folder_paths("embeddings"), clip_type=clip_type)
File "/workspace/ComfyUI/comfy/sd.py", line 406, in load_clip
clip_data.append(comfy.utils.load_torch_file(p, safe_load=True))
File "/workspace/ComfyUI/comfy/utils.py", line 34, in load_torch_file
sd = safetensors.torch.load_file(ckpt, device=device.type)
File "/venv/lib/python3.10/site-packages/safetensors/torch.py", line 310, in load_file
result[k] = f.get_tensor(k)
AttributeError: module 'torch' has no attribute 'float8_e4m3fn'
2024-09-24 10:42:34,576 - root - INFO - Prompt executed in 4.51 seconds
2024-09-24 10:42:51,417 - root - INFO - got prompt
2024-09-24 10:42:51,515 - root - ERROR - !!! Exception during processing !!! module 'torch' has no attribute 'float8_e4m3fn'
2024-09-24 10:42:51,518 - root - ERROR - Traceback (most recent call last):
File "/workspace/ComfyUI/execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/workspace/ComfyUI/execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/workspace/ComfyUI/execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "/workspace/ComfyUI/execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
File "/workspace/ComfyUI/nodes.py", line 929, in load_clip
clip = comfy.sd.load_clip(ckpt_paths=[clip_path1, clip_path2], embedding_directory=folder_paths.get_folder_paths("embeddings"), clip_type=clip_type)
File "/workspace/ComfyUI/comfy/sd.py", line 406, in load_clip
clip_data.append(comfy.utils.load_torch_file(p, safe_load=True))
File "/workspace/ComfyUI/comfy/utils.py", line 34, in load_torch_file
sd = safetensors.torch.load_file(ckpt, device=device.type)
File "/venv/lib/python3.10/site-packages/safetensors/torch.py", line 310, in load_file
result[k] = f.get_tensor(k)
AttributeError: module 'torch' has no attribute 'float8_e4m3fn'
2024-09-24 10:42:51,520 - root - INFO - Prompt executed in 0.06 seconds
2024-09-24 10:44:15,063 - root - INFO - got prompt
2024-09-24 10:44:15,177 - root - ERROR - !!! Exception during processing !!! module 'torch' has no attribute 'float8_e4m3fn'
2024-09-24 10:44:15,180 - root - ERROR - Traceback (most recent call last):
File "/workspace/ComfyUI/execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/workspace/ComfyUI/execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "/workspace/ComfyUI/execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "/workspace/ComfyUI/execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
File "/workspace/ComfyUI/nodes.py", line 929, in load_clip
clip = comfy.sd.load_clip(ckpt_paths=[clip_path1, clip_path2], embedding_directory=folder_paths.get_folder_paths("embeddings"), clip_type=clip_type)
File "/workspace/ComfyUI/comfy/sd.py", line 406, in load_clip
clip_data.append(comfy.utils.load_torch_file(p, safe_load=True))
File "/workspace/ComfyUI/comfy/utils.py", line 34, in load_torch_file
sd = safetensors.torch.load_file(ckpt, device=device.type)
File "/venv/lib/python3.10/site-packages/safetensors/torch.py", line 310, in load_file
result[k] = f.get_tensor(k)
AttributeError: module 'torch' has no attribute 'float8_e4m3fn'
2024-09-24 10:44:15,182 - root - INFO - Prompt executed in 0.08 seconds
These issues accuse the cuda version == 11.8
https://github.com/comfyanonymous/ComfyUI/issues/4434
https://github.com/comfyanonymous/ComfyUI/issues/4297
but I get a 12.4 Cuda version when calling nvidia-smi:
root@eff5f41ee391:/# nvidia-smi
Tue Sep 24 10:40:06 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14 Driver Version: 550.54.14 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA L40 On | 00000000:81:00.0 Off | 0 |
| N/A 34C P0 74W / 300W | 1082MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+
Issue was solved by running: pip install --upgrade torch torchvision torchaudio
This will update pytorch to be used with cuda 12