Edit model card
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

ruclip-vit-base-patch32-224

RuCLIP (Russian Contrastive Language–Image Pretraining) is a multimodal model for obtaining images and text similarities and rearranging captions and pictures. RuCLIP builds on a large body of work on zero-shot transfer, computer vision, natural language processing and multimodal learning.

Model was trained by Sber AI and SberDevices teams.

  • Task: text ranking; image ranking; zero-shot image classification;
  • Type: encoder
  • Num Parameters: 150M
  • Training Data Volume: 240 million text-image pairs
  • Language: Russian
  • Context Length: 77
  • Transformer Layers: 12
  • Transformer Width: 512
  • Transformer Heads: 8
  • Image Size: 224
  • Vision Layers: 12
  • Vision Width: 768
  • Vision Patch Size: 32

Usage Github

pip install ruclip
clip, processor = ruclip.load("ruclip-vit-base-patch32-224", device="cuda")

Performance

We have evaluated the performance on the following datasets:

Dataset Metric Name Metric Result
Food101 acc 0.505
CIFAR10 acc 0.818
CIFAR100 acc 0.504
Birdsnap acc 0.115
SUN397 acc 0.452
Stanford Cars acc 0.433
DTD acc 0.380
MNIST acc 0.447
STL10 acc 0.932
PCam acc 0.501
CLEVR acc 0.148
Rendered SST2 acc 0.489
ImageNet acc 0.375
FGVC Aircraft mean-per-class 0.033
Oxford Pets mean-per-class 0.560
Caltech101 mean-per-class 0.786
Flowers102 mean-per-class 0.401
HatefulMemes roc-auc 0.564

Authors

Downloads last month
32
Unable to determine this model’s pipeline type. Check the docs .