nielsr HF staff commited on
Commit
e99403e
1 Parent(s): 22f8d59

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -0
README.md CHANGED
@@ -1,3 +1,46 @@
1
  ---
2
  license: cc-by-nc-4.0
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
+ tags:
4
+ - vision
5
+ - metaclip
6
+ widget:
7
+ - src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/cat-dog-music.png
8
+ candidate_labels: playing music, playing sports
9
+ example_title: Cat & Dog
10
  ---
11
+
12
+ # MetaCLIP model, base-sized version, patch resolution 16
13
+
14
+ MetaCLIP model applied to 400 million data points of CommonCrawl (CC). It was introduced in the paper [Demystifying CLIP Data](https://arxiv.org/abs/2309.16671) by Xu et al. and first released in [this repository](https://github.com/facebookresearch/MetaCLIP).
15
+
16
+ Disclaimer: The team releasing MetaCLIP did not write a model card for this model so this model card has been written by the Hugging Face team.
17
+
18
+ ## Model description
19
+
20
+ The [Demystifying CLIP Data](https://arxiv.org/abs/2309.16671) paper aims to reveal CLIP’s method around training data curation. OpenAI never open-sourced code regarding their data preparation pipeline.
21
+
22
+ <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/clip_overview.jpg"
23
+ alt="drawing" width="600"/>
24
+
25
+ <small> CLIP high-level overview. Taken from the <a href="https://arxiv.org/abs/2103.00020">CLIP paper</a>. </small>
26
+
27
+ ## Intended uses & limitations
28
+
29
+ You can use the raw model for linking images with text in a shared embedding space. This enables things like zero-shot image classification, text-based image retrieval, image-based text retrieval, etc.
30
+
31
+ ### How to use
32
+
33
+ We refer to the [docs](https://huggingface.co/docs/transformers/main/en/model_doc/clip#usage). Just replace the names of the models on the hub.
34
+
35
+ ### BibTeX entry and citation info
36
+
37
+ ```bibtex
38
+ @misc{xu2023demystifying,
39
+ title={Demystifying CLIP Data},
40
+ author={Hu Xu and Saining Xie and Xiaoqing Ellen Tan and Po-Yao Huang and Russell Howes and Vasu Sharma and Shang-Wen Li and Gargi Ghosh and Luke Zettlemoyer and Christoph Feichtenhofer},
41
+ year={2023},
42
+ eprint={2309.16671},
43
+ archivePrefix={arXiv},
44
+ primaryClass={cs.CV}
45
+ }
46
+ ```