File size: 1,068 Bytes
ffe19d9
 
 
 
 
fb3c77c
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
- This demo loads the `FlaxCLIPVisionBertForMaskedLM` present in the `model` directory of this repository. The checkpoint is loaded from [`flax-community/clip-vision-bert-cc12m-70k`](https://huggingface.co/flax-community/clip-vision-bert-cc12m-70k) which is pre-trained checkpoint with 70k steps. 

- 100 random validation set examples are present in the `cc12m_data/vqa_val.tsv` with respective images in the `cc12m_data/images_data` directory.

- You can get a random example by clicking on `Get a random example` button. The caption is tokenized and a random token is masked by replacing it with `[MASK]`.

- We provide `English Translation` of the caption for users who are not well-acquainted with the other languages. This is done using `mtranslate` to keep things flexible enough and needs internet connection as it uses the Google Translate API.

- The model predicts the scores for tokens from the `bert-base-multilingual-uncased` checkpoint.

- The top-5 predictions are displayed below and their respective confidence scores are shown in form of a bar plot.