Hugging Face's logo

sail
/
PTP

Model card Files Community
Awiny commited on
Commit
3217689
•
1 Parent(s): b251028

Upload coco_zero_shot.log

Browse files
Files changed (1) hide show
  1. coco_zero_shot.log +353 -0
coco_zero_shot.log ADDED
@@ -0,0 +1,353 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ (coco step1: zero-shot evaluation) load pretrained model from:
2
+ /share/home/qxnie/anaconda3/lib/python3.9/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
3
+ and will be removed in future. Use torchrun.
4
+ Note that --use_env is set by default in torchrun.
5
+ If your script expects `--local_rank` argument to be set, please
6
+ change it to read from `os.environ['LOCAL_RANK']` instead. See
7
+ https://pytorch.org/docs/stable/distributed.html#launch-utility for
8
+ further instructions
9
+
10
+ warnings.warn(
11
+ WARNING:torch.distributed.run:
12
+ *****************************************
13
+ Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
14
+ *****************************************
15
+ args.rank 1 args.world_size 4 args.gpu 1
16
+ get_rank() 0
17
+ args.rank 3 args.world_size 4 args.gpu 3
18
+ get_rank() 0
19
+ args.rank 0 args.world_size 4 args.gpu 0
20
+ get_rank() 0
21
+ args.rank 2 args.world_size 4 args.gpu 2
22
+ get_rank() 0
23
+ | distributed init (rank 2): env://
24
+ | distributed init (rank 1): env://
25
+ | distributed init (rank 3): env://
26
+ | distributed init (rank 0): env://
27
+ Creating retrieval dataset
28
+ {'image_root': '/share/common/ImageDatasets/coco2017', 'ann_root': 'annotation', 'dataset': 'coco', 'pretrained': 'output/zero_shot_coco_checkpoint_4m.pth', 'vit': 'base', 'batch_size_train': 24, 'batch_size_test': 48, 'vit_grad_ckpt': True, 'vit_ckpt_layer': 4, 'init_lr': 1e-05, 'image_size': 384, 'queue_size': 57600, 'alpha': 0.4, 'k_test': 256, 'negative_all_rank': True, 'weight_decay': 0.05, 'min_lr': 0, 'max_epoch': 6}
29
+ Creating model
30
+ load checkpoint for output/zero_shot_coco_checkpoint_4m.pth
31
+ reshape position embedding from 196 to 576
32
+ reshape position embedding from 196 to 576
33
+ load checkpoint from output/zero_shot_coco_checkpoint_4m.pth
34
+ missing keys:
35
+ ['idx_queue', 'ptr_queue']
36
+ Start training
37
+ Computing features for evaluation...
38
+ Evaluation: [ 0/1251] eta: 0:02:21 time: 0.1129 data: 0.0010 max mem: 6164
39
+ Evaluation: [ 50/1251] eta: 0:10:41 time: 0.5462 data: 0.0000 max mem: 6190
40
+ Evaluation: [ 100/1251] eta: 0:10:19 time: 0.5431 data: 0.0000 max mem: 6190
41
+ Evaluation: [ 150/1251] eta: 0:09:55 time: 0.5438 data: 0.0000 max mem: 6190
42
+ Evaluation: [ 200/1251] eta: 0:09:29 time: 0.5459 data: 0.0000 max mem: 6190
43
+ Evaluation: [ 250/1251] eta: 0:09:03 time: 0.5463 data: 0.0000 max mem: 6190
44
+ Evaluation: [ 300/1251] eta: 0:08:37 time: 0.5463 data: 0.0000 max mem: 6190
45
+ Evaluation: [ 350/1251] eta: 0:08:10 time: 0.5479 data: 0.0000 max mem: 6190
46
+ Evaluation: [ 400/1251] eta: 0:07:43 time: 0.5483 data: 0.0000 max mem: 6190
47
+ Evaluation: [ 450/1251] eta: 0:07:16 time: 0.5475 data: 0.0000 max mem: 6190
48
+ Evaluation: [ 500/1251] eta: 0:06:49 time: 0.5470 data: 0.0000 max mem: 6190
49
+ Evaluation: [ 550/1251] eta: 0:06:22 time: 0.5445 data: 0.0000 max mem: 6190
50
+ Evaluation: [ 600/1251] eta: 0:05:54 time: 0.5444 data: 0.0000 max mem: 6190
51
+ Evaluation: [ 650/1251] eta: 0:05:27 time: 0.5464 data: 0.0000 max mem: 6190
52
+ Evaluation: [ 700/1251] eta: 0:05:00 time: 0.5464 data: 0.0000 max mem: 6190
53
+ Evaluation: [ 750/1251] eta: 0:04:33 time: 0.5475 data: 0.0000 max mem: 6190
54
+ Evaluation: [ 800/1251] eta: 0:04:06 time: 0.5483 data: 0.0000 max mem: 6190
55
+ Evaluation: [ 850/1251] eta: 0:03:38 time: 0.5470 data: 0.0000 max mem: 6190
56
+ Evaluation: [ 900/1251] eta: 0:03:11 time: 0.5459 data: 0.0000 max mem: 6190
57
+ Evaluation: [ 950/1251] eta: 0:02:44 time: 0.5458 data: 0.0000 max mem: 6190
58
+ Evaluation: [1000/1251] eta: 0:02:16 time: 0.5449 data: 0.0000 max mem: 6190
59
+ Evaluation: [1050/1251] eta: 0:01:49 time: 0.5457 data: 0.0000 max mem: 6190
60
+ Evaluation: [1100/1251] eta: 0:01:22 time: 0.5462 data: 0.0000 max mem: 6190
61
+ Evaluation: [1150/1251] eta: 0:00:55 time: 0.5483 data: 0.0000 max mem: 6190
62
+ Evaluation: [1200/1251] eta: 0:00:27 time: 0.5480 data: 0.0000 max mem: 6190
63
+ Evaluation: [1250/1251] eta: 0:00:00 time: 0.5483 data: 0.0000 max mem: 6190
64
+ Evaluation: Total time: 0:11:23 (0.5461 s / it)
65
+ Evaluation: [ 0/6253] eta: 0:22:58 time: 0.2205 data: 0.0040 max mem: 6668
66
+ Evaluation: [ 50/6253] eta: 1:03:44 time: 0.6158 data: 0.0000 max mem: 6668
67
+ Evaluation: [ 100/6253] eta: 1:03:07 time: 0.6123 data: 0.0000 max mem: 6668
68
+ Evaluation: [ 150/6253] eta: 1:02:35 time: 0.6154 data: 0.0000 max mem: 6668
69
+ Evaluation: [ 200/6253] eta: 1:02:01 time: 0.6113 data: 0.0000 max mem: 6668
70
+ Evaluation: [ 250/6253] eta: 1:01:30 time: 0.6135 data: 0.0000 max mem: 6668
71
+ Evaluation: [ 300/6253] eta: 1:00:56 time: 0.6103 data: 0.0000 max mem: 6668
72
+ Evaluation: [ 350/6253] eta: 1:00:24 time: 0.6109 data: 0.0000 max mem: 6668
73
+ Evaluation: [ 400/6253] eta: 0:59:53 time: 0.6137 data: 0.0000 max mem: 6668
74
+ Evaluation: [ 450/6253] eta: 0:59:20 time: 0.6117 data: 0.0000 max mem: 6668
75
+ Evaluation: [ 500/6253] eta: 0:58:49 time: 0.6127 data: 0.0000 max mem: 6668
76
+ Evaluation: [ 550/6253] eta: 0:58:17 time: 0.6091 data: 0.0000 max mem: 6668
77
+ Evaluation: [ 600/6253] eta: 0:57:47 time: 0.6138 data: 0.0000 max mem: 6668
78
+ Evaluation: [ 650/6253] eta: 0:57:16 time: 0.6123 data: 0.0000 max mem: 6668
79
+ Evaluation: [ 700/6253] eta: 0:56:45 time: 0.6090 data: 0.0000 max mem: 6668
80
+ Evaluation: [ 750/6253] eta: 0:56:14 time: 0.6093 data: 0.0000 max mem: 6668
81
+ Evaluation: [ 800/6253] eta: 0:55:42 time: 0.6124 data: 0.0000 max mem: 6668
82
+ Evaluation: [ 850/6253] eta: 0:55:11 time: 0.6091 data: 0.0000 max mem: 6668
83
+ Evaluation: [ 900/6253] eta: 0:54:41 time: 0.6151 data: 0.0000 max mem: 6668
84
+ Evaluation: [ 950/6253] eta: 0:54:11 time: 0.6139 data: 0.0000 max mem: 6668
85
+ Evaluation: [1000/6253] eta: 0:53:39 time: 0.6102 data: 0.0000 max mem: 6668
86
+ Evaluation: [1050/6253] eta: 0:53:09 time: 0.6097 data: 0.0000 max mem: 6668
87
+ Evaluation: [1100/6253] eta: 0:52:39 time: 0.6188 data: 0.0000 max mem: 6668
88
+ Evaluation: [1150/6253] eta: 0:52:08 time: 0.6135 data: 0.0000 max mem: 6668
89
+ Evaluation: [1200/6253] eta: 0:51:37 time: 0.6169 data: 0.0000 max mem: 6668
90
+ Evaluation: [1250/6253] eta: 0:51:06 time: 0.6117 data: 0.0000 max mem: 6668
91
+ Evaluation: [1300/6253] eta: 0:50:35 time: 0.6129 data: 0.0000 max mem: 6668
92
+ Evaluation: [1350/6253] eta: 0:50:04 time: 0.6131 data: 0.0000 max mem: 6668
93
+ Evaluation: [1400/6253] eta: 0:49:32 time: 0.6129 data: 0.0000 max mem: 6668
94
+ Evaluation: [1450/6253] eta: 0:49:01 time: 0.6158 data: 0.0000 max mem: 6668
95
+ Evaluation: [1500/6253] eta: 0:48:31 time: 0.6153 data: 0.0000 max mem: 6668
96
+ Evaluation: [1550/6253] eta: 0:48:00 time: 0.6088 data: 0.0000 max mem: 6668
97
+ Evaluation: [1600/6253] eta: 0:47:30 time: 0.6084 data: 0.0000 max mem: 6668
98
+ Evaluation: [1650/6253] eta: 0:46:59 time: 0.6117 data: 0.0000 max mem: 6668
99
+ Evaluation: [1700/6253] eta: 0:46:29 time: 0.6195 data: 0.0000 max mem: 6668
100
+ Evaluation: [1750/6253] eta: 0:45:58 time: 0.6138 data: 0.0000 max mem: 6668
101
+ Evaluation: [1800/6253] eta: 0:45:28 time: 0.6171 data: 0.0000 max mem: 6668
102
+ Evaluation: [1850/6253] eta: 0:44:57 time: 0.6095 data: 0.0000 max mem: 6668
103
+ Evaluation: [1900/6253] eta: 0:44:26 time: 0.6098 data: 0.0000 max mem: 6668
104
+ Evaluation: [1950/6253] eta: 0:43:56 time: 0.6080 data: 0.0000 max mem: 6668
105
+ Evaluation: [2000/6253] eta: 0:43:25 time: 0.6107 data: 0.0000 max mem: 6668
106
+ Evaluation: [2050/6253] eta: 0:42:54 time: 0.6115 data: 0.0000 max mem: 6668
107
+ Evaluation: [2100/6253] eta: 0:42:24 time: 0.6127 data: 0.0000 max mem: 6668
108
+ Evaluation: [2150/6253] eta: 0:41:53 time: 0.6105 data: 0.0000 max mem: 6668
109
+ Evaluation: [2200/6253] eta: 0:41:22 time: 0.6181 data: 0.0000 max mem: 6668
110
+ Evaluation: [2250/6253] eta: 0:40:52 time: 0.6170 data: 0.0000 max mem: 6668
111
+ Evaluation: [2300/6253] eta: 0:40:21 time: 0.6139 data: 0.0000 max mem: 6668
112
+ Evaluation: [2350/6253] eta: 0:39:50 time: 0.6094 data: 0.0000 max mem: 6668
113
+ Evaluation: [2400/6253] eta: 0:39:20 time: 0.6099 data: 0.0000 max mem: 6668
114
+ Evaluation: [2450/6253] eta: 0:38:49 time: 0.6116 data: 0.0000 max mem: 6668
115
+ Evaluation: [2500/6253] eta: 0:38:18 time: 0.6106 data: 0.0000 max mem: 6668
116
+ Evaluation: [2550/6253] eta: 0:37:48 time: 0.6111 data: 0.0000 max mem: 6668
117
+ Evaluation: [2600/6253] eta: 0:37:17 time: 0.6103 data: 0.0000 max mem: 6668
118
+ Evaluation: [2650/6253] eta: 0:36:46 time: 0.6070 data: 0.0000 max mem: 6668
119
+ Evaluation: [2700/6253] eta: 0:36:16 time: 0.6225 data: 0.0000 max mem: 6668
120
+ Evaluation: [2750/6253] eta: 0:35:45 time: 0.6148 data: 0.0000 max mem: 6668
121
+ Evaluation: [2800/6253] eta: 0:35:15 time: 0.6145 data: 0.0000 max mem: 6668
122
+ Evaluation: [2850/6253] eta: 0:34:44 time: 0.6129 data: 0.0000 max mem: 6668
123
+ Evaluation: [2900/6253] eta: 0:34:14 time: 0.6167 data: 0.0000 max mem: 6668
124
+ Evaluation: [2950/6253] eta: 0:33:43 time: 0.6077 data: 0.0000 max mem: 6668
125
+ Evaluation: [3000/6253] eta: 0:33:12 time: 0.6097 data: 0.0000 max mem: 6668
126
+ Evaluation: [3050/6253] eta: 0:32:41 time: 0.6089 data: 0.0000 max mem: 6668
127
+ Evaluation: [3100/6253] eta: 0:32:11 time: 0.6119 data: 0.0000 max mem: 6668
128
+ Evaluation: [3150/6253] eta: 0:31:40 time: 0.6100 data: 0.0000 max mem: 6668
129
+ Evaluation: [3200/6253] eta: 0:31:10 time: 0.6164 data: 0.0000 max mem: 6668
130
+ Evaluation: [3250/6253] eta: 0:30:39 time: 0.6106 data: 0.0000 max mem: 6668
131
+ Evaluation: [3300/6253] eta: 0:30:08 time: 0.6143 data: 0.0000 max mem: 6668
132
+ Evaluation: [3350/6253] eta: 0:29:38 time: 0.6126 data: 0.0000 max mem: 6668
133
+ Evaluation: [3400/6253] eta: 0:29:07 time: 0.6086 data: 0.0000 max mem: 6668
134
+ Evaluation: [3450/6253] eta: 0:28:36 time: 0.6103 data: 0.0000 max mem: 6668
135
+ Evaluation: [3500/6253] eta: 0:28:06 time: 0.6089 data: 0.0000 max mem: 6668
136
+ Evaluation: [3550/6253] eta: 0:27:35 time: 0.6113 data: 0.0000 max mem: 6668
137
+ Evaluation: [3600/6253] eta: 0:27:05 time: 0.6153 data: 0.0000 max mem: 6668
138
+ Evaluation: [3650/6253] eta: 0:26:34 time: 0.6157 data: 0.0000 max mem: 6668
139
+ Evaluation: [3700/6253] eta: 0:26:03 time: 0.6159 data: 0.0000 max mem: 6668
140
+ Evaluation: [3750/6253] eta: 0:25:33 time: 0.6149 data: 0.0000 max mem: 6668
141
+ Evaluation: [3800/6253] eta: 0:25:02 time: 0.6110 data: 0.0000 max mem: 6668
142
+ Evaluation: [3850/6253] eta: 0:24:31 time: 0.6160 data: 0.0000 max mem: 6668
143
+ Evaluation: [3900/6253] eta: 0:24:01 time: 0.6098 data: 0.0000 max mem: 6668
144
+ Evaluation: [3950/6253] eta: 0:23:30 time: 0.6084 data: 0.0000 max mem: 6668
145
+ Evaluation: [4000/6253] eta: 0:22:59 time: 0.6098 data: 0.0000 max mem: 6668
146
+ Evaluation: [4050/6253] eta: 0:22:29 time: 0.6152 data: 0.0000 max mem: 6668
147
+ Evaluation: [4100/6253] eta: 0:21:58 time: 0.6154 data: 0.0000 max mem: 6668
148
+ Evaluation: [4150/6253] eta: 0:21:28 time: 0.6139 data: 0.0000 max mem: 6668
149
+ Evaluation: [4200/6253] eta: 0:20:57 time: 0.6121 data: 0.0000 max mem: 6668
150
+ Evaluation: [4250/6253] eta: 0:20:26 time: 0.6136 data: 0.0000 max mem: 6668
151
+ Evaluation: [4300/6253] eta: 0:19:56 time: 0.6101 data: 0.0000 max mem: 6668
152
+ Evaluation: [4350/6253] eta: 0:19:25 time: 0.6138 data: 0.0000 max mem: 6668
153
+ Evaluation: [4400/6253] eta: 0:18:54 time: 0.6115 data: 0.0000 max mem: 6668
154
+ Evaluation: [4450/6253] eta: 0:18:24 time: 0.6114 data: 0.0000 max mem: 6668
155
+ Evaluation: [4500/6253] eta: 0:17:53 time: 0.6083 data: 0.0000 max mem: 6668
156
+ Evaluation: [4550/6253] eta: 0:17:22 time: 0.6082 data: 0.0000 max mem: 6668
157
+ Evaluation: [4600/6253] eta: 0:16:52 time: 0.6145 data: 0.0000 max mem: 6668
158
+ Evaluation: [4650/6253] eta: 0:16:21 time: 0.6145 data: 0.0000 max mem: 6668
159
+ Evaluation: [4700/6253] eta: 0:15:51 time: 0.6065 data: 0.0000 max mem: 6668
160
+ Evaluation: [4750/6253] eta: 0:15:20 time: 0.6085 data: 0.0000 max mem: 6668
161
+ Evaluation: [4800/6253] eta: 0:14:49 time: 0.6093 data: 0.0000 max mem: 6668
162
+ Evaluation: [4850/6253] eta: 0:14:19 time: 0.6097 data: 0.0000 max mem: 6668
163
+ Evaluation: [4900/6253] eta: 0:13:48 time: 0.6131 data: 0.0000 max mem: 6668
164
+ Evaluation: [4950/6253] eta: 0:13:17 time: 0.6141 data: 0.0000 max mem: 6668
165
+ Evaluation: [5000/6253] eta: 0:12:47 time: 0.6140 data: 0.0000 max mem: 6668
166
+ Evaluation: [5050/6253] eta: 0:12:16 time: 0.6086 data: 0.0000 max mem: 6668
167
+ Evaluation: [5100/6253] eta: 0:11:46 time: 0.6112 data: 0.0000 max mem: 6668
168
+ Evaluation: [5150/6253] eta: 0:11:15 time: 0.6109 data: 0.0000 max mem: 6668
169
+ Evaluation: [5200/6253] eta: 0:10:44 time: 0.6091 data: 0.0000 max mem: 6668
170
+ Evaluation: [5250/6253] eta: 0:10:14 time: 0.6108 data: 0.0000 max mem: 6668
171
+ Evaluation: [5300/6253] eta: 0:09:43 time: 0.6118 data: 0.0000 max mem: 6668
172
+ Evaluation: [5350/6253] eta: 0:09:12 time: 0.6120 data: 0.0000 max mem: 6668
173
+ Evaluation: [5400/6253] eta: 0:08:42 time: 0.6114 data: 0.0000 max mem: 6668
174
+ Evaluation: [5450/6253] eta: 0:08:11 time: 0.6124 data: 0.0000 max mem: 6668
175
+ Evaluation: [5500/6253] eta: 0:07:41 time: 0.6148 data: 0.0000 max mem: 6668
176
+ Evaluation: [5550/6253] eta: 0:07:10 time: 0.6136 data: 0.0000 max mem: 6668
177
+ Evaluation: [5600/6253] eta: 0:06:39 time: 0.6103 data: 0.0000 max mem: 6668
178
+ Evaluation: [5650/6253] eta: 0:06:09 time: 0.6138 data: 0.0000 max mem: 6668
179
+ Evaluation: [5700/6253] eta: 0:05:38 time: 0.6124 data: 0.0000 max mem: 6668
180
+ Evaluation: [5750/6253] eta: 0:05:07 time: 0.6180 data: 0.0000 max mem: 6668
181
+ Evaluation: [5800/6253] eta: 0:04:37 time: 0.6118 data: 0.0000 max mem: 6668
182
+ Evaluation: [5850/6253] eta: 0:04:06 time: 0.6153 data: 0.0000 max mem: 6668
183
+ Evaluation: [5900/6253] eta: 0:03:36 time: 0.6109 data: 0.0000 max mem: 6668
184
+ Evaluation: [5950/6253] eta: 0:03:05 time: 0.6107 data: 0.0000 max mem: 6668
185
+ Evaluation: [6000/6253] eta: 0:02:34 time: 0.6170 data: 0.0000 max mem: 6668
186
+ Evaluation: [6050/6253] eta: 0:02:04 time: 0.6168 data: 0.0000 max mem: 6668
187
+ Evaluation: [6100/6253] eta: 0:01:33 time: 0.6123 data: 0.0000 max mem: 6668
188
+ Evaluation: [6150/6253] eta: 0:01:03 time: 0.6119 data: 0.0000 max mem: 6668
189
+ Evaluation: [6200/6253] eta: 0:00:32 time: 0.6105 data: 0.0000 max mem: 6668
190
+ Evaluation: [6250/6253] eta: 0:00:01 time: 0.6111 data: 0.0000 max mem: 6668
191
+ Evaluation: [6252/6253] eta: 0:00:00 time: 0.6115 data: 0.0000 max mem: 6668
192
+ Evaluation: Total time: 1:03:49 (0.6125 s / it)
193
+ Evaluation time 1:19:18
194
+ Computing features for evaluation...
195
+ Evaluation: [ 0/1251] eta: 0:02:30 time: 0.1199 data: 0.0010 max mem: 6668
196
+ Evaluation: [ 50/1251] eta: 0:10:44 time: 0.5472 data: 0.0000 max mem: 6668
197
+ Evaluation: [ 100/1251] eta: 0:10:22 time: 0.5453 data: 0.0000 max mem: 6668
198
+ Evaluation: [ 150/1251] eta: 0:09:58 time: 0.5502 data: 0.0000 max mem: 6668
199
+ Evaluation: [ 200/1251] eta: 0:09:31 time: 0.5462 data: 0.0000 max mem: 6668
200
+ Evaluation: [ 250/1251] eta: 0:09:05 time: 0.5452 data: 0.0000 max mem: 6668
201
+ Evaluation: [ 300/1251] eta: 0:08:38 time: 0.5455 data: 0.0000 max mem: 6668
202
+ Evaluation: [ 350/1251] eta: 0:08:11 time: 0.5454 data: 0.0000 max mem: 6668
203
+ Evaluation: [ 400/1251] eta: 0:07:44 time: 0.5460 data: 0.0000 max mem: 6668
204
+ Evaluation: [ 450/1251] eta: 0:07:17 time: 0.5470 data: 0.0000 max mem: 6668
205
+ Evaluation: [ 500/1251] eta: 0:06:49 time: 0.5465 data: 0.0000 max mem: 6668
206
+ Evaluation: [ 550/1251] eta: 0:06:22 time: 0.5446 data: 0.0000 max mem: 6668
207
+ Evaluation: [ 600/1251] eta: 0:05:55 time: 0.5474 data: 0.0000 max mem: 6668
208
+ Evaluation: [ 650/1251] eta: 0:05:28 time: 0.5464 data: 0.0000 max mem: 6668
209
+ Evaluation: [ 700/1251] eta: 0:05:01 time: 0.5484 data: 0.0000 max mem: 6668
210
+ Evaluation: [ 750/1251] eta: 0:04:33 time: 0.5490 data: 0.0000 max mem: 6668
211
+ Evaluation: [ 800/1251] eta: 0:04:06 time: 0.5477 data: 0.0000 max mem: 6668
212
+ Evaluation: [ 850/1251] eta: 0:03:39 time: 0.5505 data: 0.0000 max mem: 6668
213
+ Evaluation: [ 900/1251] eta: 0:03:11 time: 0.5493 data: 0.0000 max mem: 6668
214
+ Evaluation: [ 950/1251] eta: 0:02:44 time: 0.5470 data: 0.0000 max mem: 6668
215
+ Evaluation: [1000/1251] eta: 0:02:17 time: 0.5491 data: 0.0000 max mem: 6668
216
+ Evaluation: [1050/1251] eta: 0:01:49 time: 0.5481 data: 0.0000 max mem: 6668
217
+ Evaluation: [1100/1251] eta: 0:01:22 time: 0.5467 data: 0.0000 max mem: 6668
218
+ Evaluation: [1150/1251] eta: 0:00:55 time: 0.5444 data: 0.0000 max mem: 6668
219
+ Evaluation: [1200/1251] eta: 0:00:27 time: 0.5452 data: 0.0000 max mem: 6668
220
+ Evaluation: [1250/1251] eta: 0:00:00 time: 0.5465 data: 0.0000 max mem: 6668
221
+ Evaluation: Total time: 0:11:23 (0.5466 s / it)
222
+ Evaluation: [ 0/6253] eta: 0:26:56 time: 0.2585 data: 0.0041 max mem: 6668
223
+ Evaluation: [ 50/6253] eta: 1:04:00 time: 0.6132 data: 0.0000 max mem: 6668
224
+ Evaluation: [ 100/6253] eta: 1:03:28 time: 0.6168 data: 0.0000 max mem: 6668
225
+ Evaluation: [ 150/6253] eta: 1:02:50 time: 0.6153 data: 0.0000 max mem: 6668
226
+ Evaluation: [ 200/6253] eta: 1:02:13 time: 0.6114 data: 0.0000 max mem: 6668
227
+ Evaluation: [ 250/6253] eta: 1:01:37 time: 0.6102 data: 0.0000 max mem: 6668
228
+ Evaluation: [ 300/6253] eta: 1:01:04 time: 0.6135 data: 0.0000 max mem: 6668
229
+ Evaluation: [ 350/6253] eta: 1:00:30 time: 0.6121 data: 0.0000 max mem: 6668
230
+ Evaluation: [ 400/6253] eta: 0:59:58 time: 0.6144 data: 0.0000 max mem: 6668
231
+ Evaluation: [ 450/6253] eta: 0:59:27 time: 0.6160 data: 0.0000 max mem: 6668
232
+ Evaluation: [ 500/6253] eta: 0:58:56 time: 0.6145 data: 0.0000 max mem: 6668
233
+ Evaluation: [ 550/6253] eta: 0:58:24 time: 0.6161 data: 0.0000 max mem: 6668
234
+ Evaluation: [ 600/6253] eta: 0:57:52 time: 0.6083 data: 0.0000 max mem: 6668
235
+ Evaluation: [ 650/6253] eta: 0:57:22 time: 0.6162 data: 0.0000 max mem: 6668
236
+ Evaluation: [ 700/6253] eta: 0:56:50 time: 0.6160 data: 0.0000 max mem: 6668
237
+ Evaluation: [ 750/6253] eta: 0:56:19 time: 0.6143 data: 0.0000 max mem: 6668
238
+ Evaluation: [ 800/6253] eta: 0:55:48 time: 0.6122 data: 0.0000 max mem: 6668
239
+ Evaluation: [ 850/6253] eta: 0:55:17 time: 0.6112 data: 0.0000 max mem: 6668
240
+ Evaluation: [ 900/6253] eta: 0:54:46 time: 0.6166 data: 0.0000 max mem: 6668
241
+ Evaluation: [ 950/6253] eta: 0:54:15 time: 0.6136 data: 0.0000 max mem: 6668
242
+ Evaluation: [1000/6253] eta: 0:53:44 time: 0.6163 data: 0.0000 max mem: 6668
243
+ Evaluation: [1050/6253] eta: 0:53:13 time: 0.6143 data: 0.0000 max mem: 6668
244
+ Evaluation: [1100/6253] eta: 0:52:43 time: 0.6134 data: 0.0000 max mem: 6668
245
+ Evaluation: [1150/6253] eta: 0:52:12 time: 0.6131 data: 0.0000 max mem: 6668
246
+ Evaluation: [1200/6253] eta: 0:51:41 time: 0.6112 data: 0.0000 max mem: 6668
247
+ Evaluation: [1250/6253] eta: 0:51:11 time: 0.6114 data: 0.0000 max mem: 6668
248
+ Evaluation: [1300/6253] eta: 0:50:40 time: 0.6117 data: 0.0000 max mem: 6668
249
+ Evaluation: [1350/6253] eta: 0:50:09 time: 0.6091 data: 0.0000 max mem: 6668
250
+ Evaluation: [1400/6253] eta: 0:49:39 time: 0.6157 data: 0.0000 max mem: 6668
251
+ Evaluation: [1450/6253] eta: 0:49:07 time: 0.6137 data: 0.0000 max mem: 6668
252
+ Evaluation: [1500/6253] eta: 0:48:37 time: 0.6139 data: 0.0000 max mem: 6668
253
+ Evaluation: [1550/6253] eta: 0:48:06 time: 0.6158 data: 0.0000 max mem: 6668
254
+ Evaluation: [1600/6253] eta: 0:47:35 time: 0.6123 data: 0.0000 max mem: 6668
255
+ Evaluation: [1650/6253] eta: 0:47:05 time: 0.6137 data: 0.0000 max mem: 6668
256
+ Evaluation: [1700/6253] eta: 0:46:34 time: 0.6150 data: 0.0000 max mem: 6668
257
+ Evaluation: [1750/6253] eta: 0:46:03 time: 0.6178 data: 0.0000 max mem: 6668
258
+ Evaluation: [1800/6253] eta: 0:45:33 time: 0.6169 data: 0.0000 max mem: 6668
259
+ Evaluation: [1850/6253] eta: 0:45:02 time: 0.6128 data: 0.0000 max mem: 6668
260
+ Evaluation: [1900/6253] eta: 0:44:31 time: 0.6146 data: 0.0000 max mem: 6668
261
+ Evaluation: [1950/6253] eta: 0:44:01 time: 0.6148 data: 0.0000 max mem: 6668
262
+ Evaluation: [2000/6253] eta: 0:43:30 time: 0.6141 data: 0.0000 max mem: 6668
263
+ Evaluation: [2050/6253] eta: 0:42:59 time: 0.6186 data: 0.0000 max mem: 6668
264
+ Evaluation: [2100/6253] eta: 0:42:28 time: 0.6173 data: 0.0000 max mem: 6668
265
+ Evaluation: [2150/6253] eta: 0:41:58 time: 0.6164 data: 0.0000 max mem: 6668
266
+ Evaluation: [2200/6253] eta: 0:41:27 time: 0.6194 data: 0.0000 max mem: 6668
267
+ Evaluation: [2250/6253] eta: 0:40:57 time: 0.6145 data: 0.0000 max mem: 6668
268
+ Evaluation: [2300/6253] eta: 0:40:26 time: 0.6140 data: 0.0000 max mem: 6668
269
+ Evaluation: [2350/6253] eta: 0:39:55 time: 0.6080 data: 0.0000 max mem: 6668
270
+ Evaluation: [2400/6253] eta: 0:39:25 time: 0.6176 data: 0.0000 max mem: 6668
271
+ Evaluation: [2450/6253] eta: 0:38:54 time: 0.6210 data: 0.0000 max mem: 6668
272
+ Evaluation: [2500/6253] eta: 0:38:23 time: 0.6173 data: 0.0000 max mem: 6668
273
+ Evaluation: [2550/6253] eta: 0:37:53 time: 0.6178 data: 0.0000 max mem: 6668
274
+ Evaluation: [2600/6253] eta: 0:37:22 time: 0.6099 data: 0.0000 max mem: 6668
275
+ Evaluation: [2650/6253] eta: 0:36:51 time: 0.6164 data: 0.0000 max mem: 6668
276
+ Evaluation: [2700/6253] eta: 0:36:21 time: 0.6199 data: 0.0000 max mem: 6668
277
+ Evaluation: [2750/6253] eta: 0:35:50 time: 0.6168 data: 0.0000 max mem: 6668
278
+ Evaluation: [2800/6253] eta: 0:35:19 time: 0.6168 data: 0.0000 max mem: 6668
279
+ Evaluation: [2850/6253] eta: 0:34:49 time: 0.6174 data: 0.0000 max mem: 6668
280
+ Evaluation: [2900/6253] eta: 0:34:18 time: 0.6134 data: 0.0000 max mem: 6668
281
+ Evaluation: [2950/6253] eta: 0:33:47 time: 0.6120 data: 0.0000 max mem: 6668
282
+ Evaluation: [3000/6253] eta: 0:33:16 time: 0.6120 data: 0.0000 max mem: 6668
283
+ Evaluation: [3050/6253] eta: 0:32:46 time: 0.6180 data: 0.0000 max mem: 6668
284
+ Evaluation: [3100/6253] eta: 0:32:15 time: 0.6135 data: 0.0000 max mem: 6668
285
+ Evaluation: [3150/6253] eta: 0:31:44 time: 0.6173 data: 0.0000 max mem: 6668
286
+ Evaluation: [3200/6253] eta: 0:31:14 time: 0.6186 data: 0.0000 max mem: 6668
287
+ Evaluation: [3250/6253] eta: 0:30:43 time: 0.6202 data: 0.0000 max mem: 6668
288
+ Evaluation: [3300/6253] eta: 0:30:12 time: 0.6187 data: 0.0000 max mem: 6668
289
+ Evaluation: [3350/6253] eta: 0:29:42 time: 0.6158 data: 0.0000 max mem: 6668
290
+ Evaluation: [3400/6253] eta: 0:29:11 time: 0.6168 data: 0.0000 max mem: 6668
291
+ Evaluation: [3450/6253] eta: 0:28:40 time: 0.6077 data: 0.0000 max mem: 6668
292
+ Evaluation: [3500/6253] eta: 0:28:10 time: 0.6126 data: 0.0000 max mem: 6668
293
+ Evaluation: [3550/6253] eta: 0:27:39 time: 0.6106 data: 0.0000 max mem: 6668
294
+ Evaluation: [3600/6253] eta: 0:27:08 time: 0.6084 data: 0.0000 max mem: 6668
295
+ Evaluation: [3650/6253] eta: 0:26:38 time: 0.6114 data: 0.0000 max mem: 6668
296
+ Evaluation: [3700/6253] eta: 0:26:07 time: 0.6117 data: 0.0000 max mem: 6668
297
+ Evaluation: [3750/6253] eta: 0:25:36 time: 0.6158 data: 0.0000 max mem: 6668
298
+ Evaluation: [3800/6253] eta: 0:25:05 time: 0.6135 data: 0.0000 max mem: 6668
299
+ Evaluation: [3850/6253] eta: 0:24:35 time: 0.6111 data: 0.0000 max mem: 6668
300
+ Evaluation: [3900/6253] eta: 0:24:04 time: 0.6136 data: 0.0000 max mem: 6668
301
+ Evaluation: [3950/6253] eta: 0:23:33 time: 0.6114 data: 0.0000 max mem: 6668
302
+ Evaluation: [4000/6253] eta: 0:23:03 time: 0.6191 data: 0.0000 max mem: 6668
303
+ Evaluation: [4050/6253] eta: 0:22:32 time: 0.6125 data: 0.0000 max mem: 6668
304
+ Evaluation: [4100/6253] eta: 0:22:01 time: 0.6098 data: 0.0000 max mem: 6668
305
+ Evaluation: [4150/6253] eta: 0:21:31 time: 0.6150 data: 0.0000 max mem: 6668
306
+ Evaluation: [4200/6253] eta: 0:21:00 time: 0.6198 data: 0.0000 max mem: 6668
307
+ Evaluation: [4250/6253] eta: 0:20:29 time: 0.6172 data: 0.0000 max mem: 6668
308
+ Evaluation: [4300/6253] eta: 0:19:59 time: 0.6113 data: 0.0000 max mem: 6668
309
+ Evaluation: [4350/6253] eta: 0:19:28 time: 0.6098 data: 0.0000 max mem: 6668
310
+ Evaluation: [4400/6253] eta: 0:18:57 time: 0.6144 data: 0.0000 max mem: 6668
311
+ Evaluation: [4450/6253] eta: 0:18:26 time: 0.6154 data: 0.0000 max mem: 6668
312
+ Evaluation: [4500/6253] eta: 0:17:56 time: 0.6140 data: 0.0000 max mem: 6668
313
+ Evaluation: [4550/6253] eta: 0:17:25 time: 0.6095 data: 0.0000 max mem: 6668
314
+ Evaluation: [4600/6253] eta: 0:16:54 time: 0.6093 data: 0.0000 max mem: 6668
315
+ Evaluation: [4650/6253] eta: 0:16:24 time: 0.6150 data: 0.0000 max mem: 6668
316
+ Evaluation: [4700/6253] eta: 0:15:53 time: 0.6154 data: 0.0000 max mem: 6668
317
+ Evaluation: [4750/6253] eta: 0:15:22 time: 0.6115 data: 0.0000 max mem: 6668
318
+ Evaluation: [4800/6253] eta: 0:14:51 time: 0.6104 data: 0.0000 max mem: 6668
319
+ Evaluation: [4850/6253] eta: 0:14:21 time: 0.6118 data: 0.0000 max mem: 6668
320
+ Evaluation: [4900/6253] eta: 0:13:50 time: 0.6164 data: 0.0000 max mem: 6668
321
+ Evaluation: [4950/6253] eta: 0:13:19 time: 0.6099 data: 0.0000 max mem: 6668
322
+ Evaluation: [5000/6253] eta: 0:12:49 time: 0.6153 data: 0.0000 max mem: 6668
323
+ Evaluation: [5050/6253] eta: 0:12:18 time: 0.6140 data: 0.0000 max mem: 6668
324
+ Evaluation: [5100/6253] eta: 0:11:47 time: 0.6097 data: 0.0000 max mem: 6668
325
+ Evaluation: [5150/6253] eta: 0:11:17 time: 0.6142 data: 0.0000 max mem: 6668
326
+ Evaluation: [5200/6253] eta: 0:10:46 time: 0.6148 data: 0.0000 max mem: 6668
327
+ Evaluation: [5250/6253] eta: 0:10:15 time: 0.6097 data: 0.0000 max mem: 6668
328
+ Evaluation: [5300/6253] eta: 0:09:44 time: 0.6116 data: 0.0000 max mem: 6668
329
+ Evaluation: [5350/6253] eta: 0:09:14 time: 0.6098 data: 0.0000 max mem: 6668
330
+ Evaluation: [5400/6253] eta: 0:08:43 time: 0.6106 data: 0.0000 max mem: 6668
331
+ Evaluation: [5450/6253] eta: 0:08:12 time: 0.6114 data: 0.0000 max mem: 6668
332
+ Evaluation: [5500/6253] eta: 0:07:42 time: 0.6157 data: 0.0000 max mem: 6668
333
+ Evaluation: [5550/6253] eta: 0:07:11 time: 0.6118 data: 0.0000 max mem: 6668
334
+ Evaluation: [5600/6253] eta: 0:06:40 time: 0.6151 data: 0.0000 max mem: 6668
335
+ Evaluation: [5650/6253] eta: 0:06:10 time: 0.6106 data: 0.0000 max mem: 6668
336
+ Evaluation: [5700/6253] eta: 0:05:39 time: 0.6125 data: 0.0000 max mem: 6668
337
+ Evaluation: [5750/6253] eta: 0:05:08 time: 0.6152 data: 0.0000 max mem: 6668
338
+ Evaluation: [5800/6253] eta: 0:04:38 time: 0.6116 data: 0.0000 max mem: 6668
339
+ Evaluation: [5850/6253] eta: 0:04:07 time: 0.6088 data: 0.0000 max mem: 6668
340
+ Evaluation: [5900/6253] eta: 0:03:36 time: 0.6142 data: 0.0000 max mem: 6668
341
+ Evaluation: [5950/6253] eta: 0:03:05 time: 0.6111 data: 0.0000 max mem: 6668
342
+ Evaluation: [6000/6253] eta: 0:02:35 time: 0.6113 data: 0.0000 max mem: 6668
343
+ Evaluation: [6050/6253] eta: 0:02:04 time: 0.6163 data: 0.0000 max mem: 6668
344
+ Evaluation: [6100/6253] eta: 0:01:33 time: 0.6180 data: 0.0000 max mem: 6668
345
+ Evaluation: [6150/6253] eta: 0:01:03 time: 0.6225 data: 0.0000 max mem: 6668
346
+ Evaluation: [6200/6253] eta: 0:00:32 time: 0.6146 data: 0.0000 max mem: 6668
347
+ Evaluation: [6250/6253] eta: 0:00:01 time: 0.6117 data: 0.0000 max mem: 6668
348
+ Evaluation: [6252/6253] eta: 0:00:00 time: 0.6120 data: 0.0000 max mem: 6668
349
+ Evaluation: Total time: 1:03:57 (0.6137 s / it)
350
+ Evaluation time 1:19:16
351
+ {'txt_r1': 74.44, 'txt_r5': 92.74, 'txt_r10': 96.38, 'txt_r_mean': 87.85333333333334, 'img_r1': 52.483006797281085, 'img_r5': 77.44102359056377, 'img_r10': 85.4938024790084, 'img_r_mean': 71.80594428895108, 'r_mean': 79.82963881114222}
352
+ {'txt_r1': 73.64, 'txt_r5': 92.18, 'txt_r10': 96.1, 'txt_r_mean': 87.30666666666666, 'img_r1': 52.00719712115154, 'img_r5': 77.03718512594962, 'img_r10': 84.75809676129548, 'img_r_mean': 71.26749300279887, 'r_mean': 79.28707983473276}
353
+ Training time 2:38:53