imirandam commited on
Commit
6305042
1 Parent(s): e89af98

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -3
README.md CHANGED
@@ -1,3 +1,43 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ datasets:
4
+ - imirandam/TROHN-Img
5
+ ---
6
+
7
+
8
+ # Model Card for CLIP_TROHN-Img
9
+
10
+ ## Model Description
11
+ - **Homepage:** https://imirandam.github.io/BiVLC_project_page/
12
+ - **Repository:** https://github.com/IMirandaM/BiVLC
13
+ - **Paper:**
14
+ - **Point of Contact:** [Imanol Miranda](mailto:imanol.miranda@ehu.eus)
15
+
16
+ ### Model Summary
17
+
18
+ CLIP_TROHN-Img is a model presented in the [BiVLC](https://github.com/IMirandaM/BiVLC) paper for experimentation. It has been fine-tuned with OpenCLIP framework using as basis the CLIP ViT-B-32 model pre-trained by 'openai'. The idea behind this fine-tuning is to improve the compositional understanding of the model by adding negative pairs, i.e., negative captions and negative images. The negatives present small compositional changes. Hyperparameters:
19
+
20
+ * Learning rate: 1e-6.
21
+ * Scheduler: Cosine scheduler with 50 warmup steps.
22
+ * Optimizer: AdamW optimizer with beta1 = 0.9, beta2 = 0.98, eps = 1e-6 and weight decay = 0.1.
23
+ * Loss function: InfoNCE Loss.
24
+ * Batch size: We define a batch size of 200, and then we add negatives. It results in 400 images x 400 captions (200 positive + 200 hard negatives).
25
+ * Epochs: We fine-tune all models over 10 epochs and we used validation accuracy as the model selection criterion, i.e. we selected the model with the highest accuracy on the corresponding validation set.
26
+ * Data: It is fine-tuned with [TROHN-Img](https://huggingface.co/datasets/imirandam/TROHN-Img) dataset.
27
+
28
+ ### Evaluation Data
29
+ The model is evaluated in [BiVLC](https://huggingface.co/datasets/imirandam/BiVLC).
30
+
31
+ ### Licensing Information
32
+ This work is licensed under a MIT License.
33
+
34
+ ## Citation Information
35
+ If you find this dataset useful, please consider citing our paper:
36
+ ```
37
+ @inproceedings{,
38
+ title={},
39
+ author={},
40
+ booktitle={},
41
+ year={}
42
+ }
43
+ ```