How to load these models?
#2
by
vincentclaes
- opened
I see files with unexpected names;
open_clip_pytorch_model.bin -> pytorch_model.bin
open_clip_config.json -> config.json
Can you share with me how you load these models?
Thank you!
They're dual use timm and OpenCLIP
In OpenCLIP
import open_clip
model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:timm/eva02_large_patch14_clip_224.merged2b_s4b_b131k')
tokenizer = open_clip.get_tokenizer('hf-hub:timm/eva02_large_patch14_clip_224.merged2b_s4b_b131k')
In timm there is a mappin in the code for this, the hf-hub doesn't work since yeah, the weights don't match timm defaults:
import timm
model = timm.create_model('eva02_large_patch14_clip_224.merged2b', pretrained=True)
Keep forgetting to update these READMEs :/