kaveh
/

rclip

Zero-Shot Image Classification

vision-text-dual-encoder

feature-extraction

Inference Endpoints

Model card Files Files and versions Metrics Training metrics Community

kaveh commited on Jul 8, 2023

Commit

4f8df33

•

1 Parent(s): af2257c

added heatmap image

Files changed (1) hide show

README.md +5 -2

README.md CHANGED Viewed

@@ -15,13 +15,16 @@ library_name: transformers
 pipeline_tag: feature-extraction
 ---
-# output_8_clip14_cxrbert
 This model is a fine-tuned version of [openai/clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14) as an image encoder and [microsoft/BiomedVLP-CXR-BERT-general](https://huggingface.co/microsoft/BiomedVLP-CXR-BERT-general) as a Text encoder on an ROCO dataset.
 It achieves the following results on the evaluation set:
 - Loss: 0.3388
-## Training procedure
 ### Training hyperparameters

 pipeline_tag: feature-extraction
 ---
+# RCLIP (Clip model fine-tuned on radiology images and their captions)
 This model is a fine-tuned version of [openai/clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14) as an image encoder and [microsoft/BiomedVLP-CXR-BERT-general](https://huggingface.co/microsoft/BiomedVLP-CXR-BERT-general) as a Text encoder on an ROCO dataset.
 It achieves the following results on the evaluation set:
 - Loss: 0.3388
+## Heatmap
+Here is the heatmap of the similarity score of the first 30 samples on the test split of the ROCO dataset if images vs their captions:
+![heatmap](https://imgur.com/fPFM694.png)
 ### Training hyperparameters