Hugging Face's logo

CounterCurate
/
best-counting-clip-countercurate

Model card Files Community
HanSolo9682 commited on
Commit
9f7232a
•
1 Parent(s): cd115ee

upload model

Browse files
Files changed (3) hide show
  1. checkpoints/epoch_15.pt +3 -0
  2. out.log +192 -0
  3. params.txt +96 -0
checkpoints/epoch_15.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23f2c24ecc7812b1ee9b28b7bc4a9e4801000aa98740e0b42b3d26e6a752bf4d
3
+ size 1815699342
out.log ADDED
@@ -0,0 +1,192 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2024-02-13,00:02:16 | INFO | Running with a single process. Device cuda:0.
2
+ 2024-02-13,00:02:16 | INFO | Loaded ViT-B-32 model config.
3
+ 2024-02-13,00:02:18 | INFO | Loading pretrained ViT-B-32 weights (laion2b_s34b_b79k).
4
+ 2024-02-13,00:02:18 | INFO | Model:
5
+ 2024-02-13,00:02:18 | INFO | CLIP(
6
+ (visual): VisionTransformer(
7
+ (conv1): Conv2d(3, 768, kernel_size=(32, 32), stride=(32, 32), bias=False)
8
+ (patch_dropout): Identity()
9
+ (ln_pre): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
10
+ (transformer): Transformer(
11
+ (resblocks): ModuleList(
12
+ (0-11): 12 x ResidualAttentionBlock(
13
+ (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
14
+ (attn): MultiheadAttention(
15
+ (out_proj): NonDynamicallyQuantizableLinear(in_features=768, out_features=768, bias=True)
16
+ )
17
+ (ls_1): Identity()
18
+ (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
19
+ (mlp): Sequential(
20
+ (c_fc): Linear(in_features=768, out_features=3072, bias=True)
21
+ (gelu): GELU(approximate='none')
22
+ (c_proj): Linear(in_features=3072, out_features=768, bias=True)
23
+ )
24
+ (ls_2): Identity()
25
+ )
26
+ )
27
+ )
28
+ (ln_post): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
29
+ )
30
+ (transformer): Transformer(
31
+ (resblocks): ModuleList(
32
+ (0-11): 12 x ResidualAttentionBlock(
33
+ (ln_1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
34
+ (attn): MultiheadAttention(
35
+ (out_proj): NonDynamicallyQuantizableLinear(in_features=512, out_features=512, bias=True)
36
+ )
37
+ (ls_1): Identity()
38
+ (ln_2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
39
+ (mlp): Sequential(
40
+ (c_fc): Linear(in_features=512, out_features=2048, bias=True)
41
+ (gelu): GELU(approximate='none')
42
+ (c_proj): Linear(in_features=2048, out_features=512, bias=True)
43
+ )
44
+ (ls_2): Identity()
45
+ )
46
+ )
47
+ )
48
+ (token_embedding): Embedding(49408, 512)
49
+ (ln_final): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
50
+ )
51
+ 2024-02-13,00:02:18 | INFO | Params:
52
+ 2024-02-13,00:02:18 | INFO | accum_freq: 1
53
+ 2024-02-13,00:02:18 | INFO | aug_cfg: {}
54
+ 2024-02-13,00:02:18 | INFO | batch_size: 512
55
+ 2024-02-13,00:02:18 | INFO | beta1: 0.9
56
+ 2024-02-13,00:02:18 | INFO | beta2: 0.98
57
+ 2024-02-13,00:02:18 | INFO | checkpoint_path: ./logs/2024_02_13-00_02_16-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16/checkpoints
58
+ 2024-02-13,00:02:18 | INFO | coca_caption_loss_weight: 2.0
59
+ 2024-02-13,00:02:18 | INFO | coca_contrastive_loss_weight: 1.0
60
+ 2024-02-13,00:02:18 | INFO | copy_codebase: False
61
+ 2024-02-13,00:02:18 | INFO | csv_caption_key: captions
62
+ 2024-02-13,00:02:18 | INFO | csv_img_key: images
63
+ 2024-02-13,00:02:18 | INFO | csv_separator:
64
+ 2024-02-13,00:02:18 | INFO | dataset_resampled: False
65
+ 2024-02-13,00:02:18 | INFO | dataset_type: auto
66
+ 2024-02-13,00:02:18 | INFO | ddp_static_graph: True
67
+ 2024-02-13,00:02:18 | INFO | debug: False
68
+ 2024-02-13,00:02:18 | INFO | delete_previous_checkpoint: False
69
+ 2024-02-13,00:02:18 | INFO | device: cuda:0
70
+ 2024-02-13,00:02:18 | INFO | dist_backend: nccl
71
+ 2024-02-13,00:02:18 | INFO | dist_url: env://
72
+ 2024-02-13,00:02:18 | INFO | distill: False
73
+ 2024-02-13,00:02:18 | INFO | distill_model: None
74
+ 2024-02-13,00:02:18 | INFO | distill_pretrained: None
75
+ 2024-02-13,00:02:18 | INFO | distributed: False
76
+ 2024-02-13,00:02:18 | INFO | epochs: 15
77
+ 2024-02-13,00:02:18 | INFO | epochs_cooldown: None
78
+ 2024-02-13,00:02:18 | INFO | eps: 1e-06
79
+ 2024-02-13,00:02:18 | INFO | force_custom_text: False
80
+ 2024-02-13,00:02:18 | INFO | force_image_size: None
81
+ 2024-02-13,00:02:18 | INFO | force_patch_dropout: None
82
+ 2024-02-13,00:02:18 | INFO | force_quick_gelu: False
83
+ 2024-02-13,00:02:18 | INFO | gather_with_grad: True
84
+ 2024-02-13,00:02:18 | INFO | grad_checkpointing: False
85
+ 2024-02-13,00:02:18 | INFO | grad_clip_norm: None
86
+ 2024-02-13,00:02:18 | INFO | horovod: False
87
+ 2024-02-13,00:02:18 | INFO | image_interpolation: None
88
+ 2024-02-13,00:02:18 | INFO | image_mean: None
89
+ 2024-02-13,00:02:18 | INFO | image_resize_mode: None
90
+ 2024-02-13,00:02:18 | INFO | image_std: None
91
+ 2024-02-13,00:02:18 | INFO | imagenet_v2: None
92
+ 2024-02-13,00:02:18 | INFO | imagenet_val: None
93
+ 2024-02-13,00:02:18 | INFO | local_loss: True
94
+ 2024-02-13,00:02:18 | INFO | local_rank: 0
95
+ 2024-02-13,00:02:18 | INFO | lock_image: False
96
+ 2024-02-13,00:02:18 | INFO | lock_image_freeze_bn_stats: False
97
+ 2024-02-13,00:02:18 | INFO | lock_image_unlocked_groups: 0
98
+ 2024-02-13,00:02:18 | INFO | lock_text: False
99
+ 2024-02-13,00:02:18 | INFO | lock_text_freeze_layer_norm: False
100
+ 2024-02-13,00:02:18 | INFO | lock_text_unlocked_layers: 0
101
+ 2024-02-13,00:02:18 | INFO | log_every_n_steps: 100
102
+ 2024-02-13,00:02:18 | INFO | log_level: 20
103
+ 2024-02-13,00:02:18 | INFO | log_local: False
104
+ 2024-02-13,00:02:18 | INFO | log_path: ./logs/2024_02_13-00_02_16-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16/out.log
105
+ 2024-02-13,00:02:18 | INFO | logs: ./logs/
106
+ 2024-02-13,00:02:18 | INFO | lr: 5e-05
107
+ 2024-02-13,00:02:18 | INFO | lr_cooldown_end: 0.0
108
+ 2024-02-13,00:02:18 | INFO | lr_cooldown_power: 1.0
109
+ 2024-02-13,00:02:18 | INFO | lr_scheduler: cosine
110
+ 2024-02-13,00:02:18 | INFO | model: ViT-B-32
111
+ 2024-02-13,00:02:18 | INFO | name: 2024_02_13-00_02_16-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16
112
+ 2024-02-13,00:02:18 | INFO | no_set_device_rank: False
113
+ 2024-02-13,00:02:18 | INFO | precision: amp_bf16
114
+ 2024-02-13,00:02:18 | INFO | pretrained: laion2b_s34b_b79k
115
+ 2024-02-13,00:02:18 | INFO | pretrained_image: False
116
+ 2024-02-13,00:02:18 | INFO | rank: 0
117
+ 2024-02-13,00:02:18 | INFO | remote_sync: None
118
+ 2024-02-13,00:02:18 | INFO | remote_sync_frequency: 300
119
+ 2024-02-13,00:02:18 | INFO | remote_sync_protocol: s3
120
+ 2024-02-13,00:02:18 | INFO | report_to:
121
+ 2024-02-13,00:02:18 | INFO | resume: None
122
+ 2024-02-13,00:02:18 | INFO | save_frequency: 5
123
+ 2024-02-13,00:02:18 | INFO | save_most_recent: False
124
+ 2024-02-13,00:02:18 | INFO | seed: 0
125
+ 2024-02-13,00:02:18 | INFO | siglip: False
126
+ 2024-02-13,00:02:18 | INFO | skip_scheduler: False
127
+ 2024-02-13,00:02:18 | INFO | tensorboard: False
128
+ 2024-02-13,00:02:18 | INFO | tensorboard_path:
129
+ 2024-02-13,00:02:18 | INFO | torchcompile: False
130
+ 2024-02-13,00:02:18 | INFO | torchscript: False
131
+ 2024-02-13,00:02:18 | INFO | trace: False
132
+ 2024-02-13,00:02:18 | INFO | train_data: ../../train_data_counting_neg_clip.csv
133
+ 2024-02-13,00:02:18 | INFO | train_data_upsampling_factors: None
134
+ 2024-02-13,00:02:18 | INFO | train_num_samples: None
135
+ 2024-02-13,00:02:18 | INFO | use_bn_sync: False
136
+ 2024-02-13,00:02:18 | INFO | use_bnb_linear: None
137
+ 2024-02-13,00:02:18 | INFO | val_data: None
138
+ 2024-02-13,00:02:18 | INFO | val_frequency: 5
139
+ 2024-02-13,00:02:18 | INFO | val_num_samples: None
140
+ 2024-02-13,00:02:18 | INFO | wandb: False
141
+ 2024-02-13,00:02:18 | INFO | wandb_notes:
142
+ 2024-02-13,00:02:18 | INFO | wandb_project_name: open-clip
143
+ 2024-02-13,00:02:18 | INFO | warmup: 1024
144
+ 2024-02-13,00:02:18 | INFO | wd: 0.2
145
+ 2024-02-13,00:02:18 | INFO | workers: 8
146
+ 2024-02-13,00:02:18 | INFO | world_size: 1
147
+ 2024-02-13,00:02:18 | INFO | zeroshot_frequency: 5
148
+ 2024-02-13,00:02:18 | INFO | Start epoch 0
149
+ 2024-02-13,00:02:32 | INFO | Train Epoch: 0 [ 1024/10010 (5%)] Data (t): 7.751 Batch (t): 13.065, 39.1899/s, 39.1899/s/gpu LR: 0.000000 Logit Scale: 100.000 Contrastive_loss: 6.4139 (6.4139) Loss: 6.4139 (6.4139)
150
+ 2024-02-13,00:02:42 | INFO | Train Epoch: 0 [19456/10010 (100%)] Data (t): 0.001 Batch (t): 0.583, 874.223/s, 874.223/s/gpu LR: 0.000001 Logit Scale: 99.999 Contrastive_loss: 5.2933 (5.8536) Loss: 5.2933 (5.8536)
151
+ 2024-02-13,00:02:42 | INFO | Start epoch 1
152
+ 2024-02-13,00:02:49 | INFO | Train Epoch: 1 [ 1024/10010 (5%)] Data (t): 6.269 Batch (t): 6.642, 77.0819/s, 77.0819/s/gpu LR: 0.000001 Logit Scale: 99.999 Contrastive_loss: 5.0906 (5.0906) Loss: 5.0906 (5.0906)
153
+ 2024-02-13,00:03:00 | INFO | Train Epoch: 1 [19456/10010 (100%)] Data (t): 0.013 Batch (t): 0.588, 864.032/s, 864.032/s/gpu LR: 0.000002 Logit Scale: 99.997 Contrastive_loss: 4.3162 (4.7034) Loss: 4.3162 (4.7034)
154
+ 2024-02-13,00:03:00 | INFO | Start epoch 2
155
+ 2024-02-13,00:03:08 | INFO | Train Epoch: 2 [ 1024/10010 (5%)] Data (t): 7.151 Batch (t): 7.527, 68.0199/s, 68.0199/s/gpu LR: 0.000002 Logit Scale: 99.997 Contrastive_loss: 4.1381 (4.1381) Loss: 4.1381 (4.1381)
156
+ 2024-02-13,00:03:19 | INFO | Train Epoch: 2 [19456/10010 (100%)] Data (t): 0.033 Batch (t): 0.595, 866.703/s, 866.703/s/gpu LR: 0.000003 Logit Scale: 99.996 Contrastive_loss: 3.7141 (3.9261) Loss: 3.7141 (3.9261)
157
+ 2024-02-13,00:03:19 | INFO | Start epoch 3
158
+ 2024-02-13,00:03:26 | INFO | Train Epoch: 3 [ 1024/10010 (5%)] Data (t): 6.530 Batch (t): 6.891, 74.2989/s, 74.2989/s/gpu LR: 0.000003 Logit Scale: 99.996 Contrastive_loss: 3.7603 (3.7603) Loss: 3.7603 (3.7603)
159
+ 2024-02-13,00:03:37 | INFO | Train Epoch: 3 [19456/10010 (100%)] Data (t): 0.045 Batch (t): 0.608, 865.795/s, 865.795/s/gpu LR: 0.000004 Logit Scale: 99.996 Contrastive_loss: 3.2845 (3.5224) Loss: 3.2845 (3.5224)
160
+ 2024-02-13,00:03:37 | INFO | Start epoch 4
161
+ 2024-02-13,00:03:44 | INFO | Train Epoch: 4 [ 1024/10010 (5%)] Data (t): 5.669 Batch (t): 6.038, 84.7949/s, 84.7949/s/gpu LR: 0.000004 Logit Scale: 99.996 Contrastive_loss: 3.1494 (3.1494) Loss: 3.1494 (3.1494)
162
+ 2024-02-13,00:03:55 | INFO | Train Epoch: 4 [19456/10010 (100%)] Data (t): 0.026 Batch (t): 0.605, 864.414/s, 864.414/s/gpu LR: 0.000005 Logit Scale: 99.996 Contrastive_loss: 2.7506 (2.9500) Loss: 2.7506 (2.9500)
163
+ 2024-02-13,00:03:57 | INFO | Start epoch 5
164
+ 2024-02-13,00:04:04 | INFO | Train Epoch: 5 [ 1024/10010 (5%)] Data (t): 6.324 Batch (t): 6.699, 76.4267/s, 76.4267/s/gpu LR: 0.000005 Logit Scale: 99.996 Contrastive_loss: 2.7057 (2.7057) Loss: 2.7057 (2.7057)
165
+ 2024-02-13,00:04:15 | INFO | Train Epoch: 5 [19456/10010 (100%)] Data (t): 0.089 Batch (t): 0.639, 867.588/s, 867.588/s/gpu LR: 0.000006 Logit Scale: 99.996 Contrastive_loss: 2.4321 (2.5689) Loss: 2.4321 (2.5689)
166
+ 2024-02-13,00:04:16 | INFO | Start epoch 6
167
+ 2024-02-13,00:04:22 | INFO | Train Epoch: 6 [ 1024/10010 (5%)] Data (t): 6.129 Batch (t): 6.509, 78.6643/s, 78.6643/s/gpu LR: 0.000006 Logit Scale: 99.996 Contrastive_loss: 2.3407 (2.3407) Loss: 2.3407 (2.3407)
168
+ 2024-02-13,00:04:33 | INFO | Train Epoch: 6 [19456/10010 (100%)] Data (t): 0.013 Batch (t): 0.585, 866.873/s, 866.873/s/gpu LR: 0.000006 Logit Scale: 99.997 Contrastive_loss: 2.2057 (2.2732) Loss: 2.2057 (2.2732)
169
+ 2024-02-13,00:04:33 | INFO | Start epoch 7
170
+ 2024-02-13,00:04:40 | INFO | Train Epoch: 7 [ 1024/10010 (5%)] Data (t): 6.374 Batch (t): 6.750, 75.8483/s, 75.8483/s/gpu LR: 0.000007 Logit Scale: 99.997 Contrastive_loss: 1.9728 (1.9728) Loss: 1.9728 (1.9728)
171
+ 2024-02-13,00:04:51 | INFO | Train Epoch: 7 [19456/10010 (100%)] Data (t): 0.070 Batch (t): 0.629, 865.081/s, 865.081/s/gpu LR: 0.000007 Logit Scale: 99.998 Contrastive_loss: 1.8460 (1.9094) Loss: 1.8460 (1.9094)
172
+ 2024-02-13,00:04:52 | INFO | Start epoch 8
173
+ 2024-02-13,00:04:59 | INFO | Train Epoch: 8 [ 1024/10010 (5%)] Data (t): 6.348 Batch (t): 6.724, 76.1504/s, 76.1504/s/gpu LR: 0.000007 Logit Scale: 99.999 Contrastive_loss: 1.6491 (1.6491) Loss: 1.6491 (1.6491)
174
+ 2024-02-13,00:05:09 | INFO | Train Epoch: 8 [19456/10010 (100%)] Data (t): 0.039 Batch (t): 0.591, 864.632/s, 864.632/s/gpu LR: 0.000008 Logit Scale: 100.000 Contrastive_loss: 1.5005 (1.5748) Loss: 1.5005 (1.5748)
175
+ 2024-02-13,00:05:10 | INFO | Start epoch 9
176
+ 2024-02-13,00:05:16 | INFO | Train Epoch: 9 [ 1024/10010 (5%)] Data (t): 5.740 Batch (t): 6.101, 83.9270/s, 83.9270/s/gpu LR: 0.000008 Logit Scale: 100.000 Contrastive_loss: 1.2527 (1.2527) Loss: 1.2527 (1.2527)
177
+ 2024-02-13,00:05:27 | INFO | Train Epoch: 9 [19456/10010 (100%)] Data (t): 0.029 Batch (t): 0.592, 866.672/s, 866.672/s/gpu LR: 0.000009 Logit Scale: 100.000 Contrastive_loss: 1.1425 (1.1976) Loss: 1.1425 (1.1976)
178
+ 2024-02-13,00:05:29 | INFO | Start epoch 10
179
+ 2024-02-13,00:05:35 | INFO | Train Epoch: 10 [ 1024/10010 (5%)] Data (t): 5.614 Batch (t): 5.988, 85.5099/s, 85.5099/s/gpu LR: 0.000009 Logit Scale: 100.000 Contrastive_loss: 0.92603 (0.92603) Loss: 0.92603 (0.92603)
180
+ 2024-02-13,00:05:46 | INFO | Train Epoch: 10 [19456/10010 (100%)] Data (t): 0.089 Batch (t): 0.636, 866.623/s, 866.623/s/gpu LR: 0.000010 Logit Scale: 100.000 Contrastive_loss: 1.0557 (0.99089) Loss: 1.0557 (0.99089)
181
+ 2024-02-13,00:05:47 | INFO | Start epoch 11
182
+ 2024-02-13,00:05:54 | INFO | Train Epoch: 11 [ 1024/10010 (5%)] Data (t): 6.593 Batch (t): 6.953, 73.6390/s, 73.6390/s/gpu LR: 0.000010 Logit Scale: 100.000 Contrastive_loss: 0.75542 (0.75542) Loss: 0.75542 (0.75542)
183
+ 2024-02-13,00:06:05 | INFO | Train Epoch: 11 [19456/10010 (100%)] Data (t): 0.045 Batch (t): 0.595, 865.436/s, 865.436/s/gpu LR: 0.000011 Logit Scale: 100.000 Contrastive_loss: 0.74945 (0.75243) Loss: 0.74945 (0.75243)
184
+ 2024-02-13,00:06:05 | INFO | Start epoch 12
185
+ 2024-02-13,00:06:11 | INFO | Train Epoch: 12 [ 1024/10010 (5%)] Data (t): 5.893 Batch (t): 6.266, 81.7047/s, 81.7047/s/gpu LR: 0.000011 Logit Scale: 100.000 Contrastive_loss: 0.60686 (0.60686) Loss: 0.60686 (0.60686)
186
+ 2024-02-13,00:06:22 | INFO | Train Epoch: 12 [19456/10010 (100%)] Data (t): 0.015 Batch (t): 0.587, 865.351/s, 865.351/s/gpu LR: 0.000012 Logit Scale: 100.000 Contrastive_loss: 0.62050 (0.61368) Loss: 0.62050 (0.61368)
187
+ 2024-02-13,00:06:22 | INFO | Start epoch 13
188
+ 2024-02-13,00:06:30 | INFO | Train Epoch: 13 [ 1024/10010 (5%)] Data (t): 6.973 Batch (t): 7.323, 69.9169/s, 69.9169/s/gpu LR: 0.000012 Logit Scale: 100.000 Contrastive_loss: 0.49629 (0.49629) Loss: 0.49629 (0.49629)
189
+ 2024-02-13,00:06:41 | INFO | Train Epoch: 13 [19456/10010 (100%)] Data (t): 0.044 Batch (t): 0.595, 874.122/s, 874.122/s/gpu LR: 0.000013 Logit Scale: 100.000 Contrastive_loss: 0.53294 (0.51462) Loss: 0.53294 (0.51462)
190
+ 2024-02-13,00:06:41 | INFO | Start epoch 14
191
+ 2024-02-13,00:06:48 | INFO | Train Epoch: 14 [ 1024/10010 (5%)] Data (t): 6.511 Batch (t): 6.872, 74.5086/s, 74.5086/s/gpu LR: 0.000013 Logit Scale: 100.000 Contrastive_loss: 0.45596 (0.45596) Loss: 0.45596 (0.45596)
192
+ 2024-02-13,00:06:59 | INFO | Train Epoch: 14 [19456/10010 (100%)] Data (t): 0.014 Batch (t): 0.587, 865.782/s, 865.782/s/gpu LR: 0.000014 Logit Scale: 100.000 Contrastive_loss: 0.41602 (0.43599) Loss: 0.41602 (0.43599)
params.txt ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ accum_freq: 1
2
+ aug_cfg: {}
3
+ batch_size: 512
4
+ beta1: 0.9
5
+ beta2: 0.98
6
+ checkpoint_path: ./logs/2024_02_13-00_02_16-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16/checkpoints
7
+ coca_caption_loss_weight: 2.0
8
+ coca_contrastive_loss_weight: 1.0
9
+ copy_codebase: False
10
+ csv_caption_key: captions
11
+ csv_img_key: images
12
+ csv_separator:
13
+ dataset_resampled: False
14
+ dataset_type: auto
15
+ ddp_static_graph: True
16
+ debug: False
17
+ delete_previous_checkpoint: False
18
+ device: cuda:0
19
+ dist_backend: nccl
20
+ dist_url: env://
21
+ distill: False
22
+ distill_model: None
23
+ distill_pretrained: None
24
+ distributed: False
25
+ epochs: 15
26
+ epochs_cooldown: None
27
+ eps: 1e-06
28
+ force_custom_text: False
29
+ force_image_size: None
30
+ force_patch_dropout: None
31
+ force_quick_gelu: False
32
+ gather_with_grad: True
33
+ grad_checkpointing: False
34
+ grad_clip_norm: None
35
+ horovod: False
36
+ image_interpolation: None
37
+ image_mean: None
38
+ image_resize_mode: None
39
+ image_std: None
40
+ imagenet_v2: None
41
+ imagenet_val: None
42
+ local_loss: True
43
+ local_rank: 0
44
+ lock_image: False
45
+ lock_image_freeze_bn_stats: False
46
+ lock_image_unlocked_groups: 0
47
+ lock_text: False
48
+ lock_text_freeze_layer_norm: False
49
+ lock_text_unlocked_layers: 0
50
+ log_every_n_steps: 100
51
+ log_level: 20
52
+ log_local: False
53
+ log_path: ./logs/2024_02_13-00_02_16-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16/out.log
54
+ logs: ./logs/
55
+ lr: 5e-05
56
+ lr_cooldown_end: 0.0
57
+ lr_cooldown_power: 1.0
58
+ lr_scheduler: cosine
59
+ model: ViT-B-32
60
+ name: 2024_02_13-00_02_16-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16
61
+ no_set_device_rank: False
62
+ precision: amp_bf16
63
+ pretrained: laion2b_s34b_b79k
64
+ pretrained_image: False
65
+ rank: 0
66
+ remote_sync: None
67
+ remote_sync_frequency: 300
68
+ remote_sync_protocol: s3
69
+ report_to:
70
+ resume: None
71
+ save_frequency: 5
72
+ save_most_recent: False
73
+ seed: 0
74
+ siglip: False
75
+ skip_scheduler: False
76
+ tensorboard: False
77
+ tensorboard_path:
78
+ torchcompile: False
79
+ torchscript: False
80
+ trace: False
81
+ train_data: ../../train_data_counting_neg_clip.csv
82
+ train_data_upsampling_factors: None
83
+ train_num_samples: None
84
+ use_bn_sync: False
85
+ use_bnb_linear: None
86
+ val_data: None
87
+ val_frequency: 5
88
+ val_num_samples: None
89
+ wandb: False
90
+ wandb_notes:
91
+ wandb_project_name: open-clip
92
+ warmup: 1024
93
+ wd: 0.2
94
+ workers: 8
95
+ world_size: 1
96
+ zeroshot_frequency: 5