Error on install
Hello,
i get this error: Loading VLM's custom vision model 📎
Error loading models: Weights only load failed. Re-running torch.load with weights_only set to False will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
Please file an issue with the following so that we can make weights_only=True compatible with your use case: WeightsUnpickler error: Unsupported operand 118
Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html. what can i do.
i tried set it to false and still not working. i am on gpu.
thanks
i got it to work. but for me the ouput is not the same as the original? why is that. sometimes its random and not correct caption?
Good morning.
I did think that the loading part of the custom VLM was False in the original, so I thought that might be the reason, but that still doesn't fix it...
I've never had the same error in my environment, either locally or online. I wondered what the difference was between my environment and yours, and searched to see if it might be the torch version, but this didn't seem to have much to do with it.
However, I did find the following post by Wi-zz himself, which I thought might be a CUDA toolkit not set up properly, causing CUDA errors, or perhaps a Radeon (ROCM) environment?
CUDA related setups are hell, especially in a Winodws environment. I have to search and find a know how site and follow it to properly install... I suspect either of these possibilities.
https://huggingface.co/Wi-zz/joy-caption-pre-alpha/discussions/3
it was because the models i used to copy where just bytes not GB. i manually copied them using wget..
but for the accuracy did you test it. for me there is a big difference for the ouput.. than the orginal one.
i got it to work. but for me the ouput is not the same as the original? why is that. sometimes its random and not correct caption?
I think the custom part of the VLM is not loaded.
If so, some of the models are running in a different state and the output may not be stable.
The original is made specifically for the Zero GPU space, partly because it is still an alpha version, so it is not yet tuned for anything other than the ideal CUDA environment.
However, we would like to make it usable as soon as possible, so it would be better if we somehow tuned it ourselves.
I prioritized reproducing the Wi-zz version, so other checks are still too lax. I have my doubts too, especially about the accuracy.
The main problem in fixing it is that there is no error in the current situation.
If it's not loading, it should be able to load. But if it is able to load but the operation is incomplete, we need to explore what on earth we can do to fix it.
https://huggingface.co/spaces/John6666/joy-caption-pre-alpha-mod
The Zero GPU space version here, which uses similar code, works fine, including accuracy, which leads me to believe that there is some difference between the local environment and the HF virtual environment. I can understand if it simply doesn't work, but how can LLM performance ever drop halfway...
Anyway, it's happening in real life and I'll try to verify it.
I fixed the suspicious part.