Model save
Browse files
wandb/run-20220129_215451-1vipdbow/files/output.log
CHANGED
|
@@ -15994,3 +15994,328 @@ Deleting older checkpoint [checkpoint-12000] due to args.save_total_limit
|
|
| 15994 |
Configuration saved in ./checkpoint-13500/config.json
|
| 15995 |
Model weights saved in ./checkpoint-13500/pytorch_model.bin
|
| 15996 |
Configuration saved in ./checkpoint-13500/preprocessor_config.json
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15994 |
Configuration saved in ./checkpoint-13500/config.json
|
| 15995 |
Model weights saved in ./checkpoint-13500/pytorch_model.bin
|
| 15996 |
Configuration saved in ./checkpoint-13500/preprocessor_config.json
|
| 15997 |
+
Configuration saved in ./preprocessor_config.json
|
| 15998 |
+
Deleting older checkpoint [checkpoint-12500] due to args.save_total_limit
|
| 15999 |
+
|
| 16000 |
+
|
| 16001 |
+
|
| 16002 |
+
|
| 16003 |
+
|
| 16004 |
+
|
| 16005 |
+
|
| 16006 |
+
|
| 16007 |
+
|
| 16008 |
+
|
| 16009 |
+
|
| 16010 |
+
|
| 16011 |
+
|
| 16012 |
+
|
| 16013 |
+
|
| 16014 |
+
|
| 16015 |
+
|
| 16016 |
+
|
| 16017 |
+
|
| 16018 |
+
|
| 16019 |
+
|
| 16020 |
+
|
| 16021 |
+
|
| 16022 |
+
|
| 16023 |
+
|
| 16024 |
+
|
| 16025 |
+
|
| 16026 |
+
|
| 16027 |
+
|
| 16028 |
+
|
| 16029 |
+
|
| 16030 |
+
|
| 16031 |
+
|
| 16032 |
+
|
| 16033 |
+
|
| 16034 |
+
|
| 16035 |
+
|
| 16036 |
+
|
| 16037 |
+
|
| 16038 |
+
|
| 16039 |
+
|
| 16040 |
+
|
| 16041 |
+
|
| 16042 |
+
|
| 16043 |
+
|
| 16044 |
+
|
| 16045 |
+
|
| 16046 |
+
|
| 16047 |
+
|
| 16048 |
+
|
| 16049 |
+
|
| 16050 |
+
|
| 16051 |
+
|
| 16052 |
+
|
| 16053 |
+
|
| 16054 |
+
|
| 16055 |
+
|
| 16056 |
+
|
| 16057 |
+
|
| 16058 |
+
|
| 16059 |
+
|
| 16060 |
+
|
| 16061 |
+
|
| 16062 |
+
|
| 16063 |
+
|
| 16064 |
+
|
| 16065 |
+
|
| 16066 |
+
|
| 16067 |
+
|
| 16068 |
+
|
| 16069 |
+
|
| 16070 |
+
|
| 16071 |
+
|
| 16072 |
+
|
| 16073 |
+
|
| 16074 |
+
|
| 16075 |
+
|
| 16076 |
+
|
| 16077 |
+
|
| 16078 |
+
|
| 16079 |
+
|
| 16080 |
+
|
| 16081 |
+
|
| 16082 |
+
|
| 16083 |
+
|
| 16084 |
+
|
| 16085 |
+
|
| 16086 |
+
|
| 16087 |
+
|
| 16088 |
+
|
| 16089 |
+
|
| 16090 |
+
|
| 16091 |
+
|
| 16092 |
+
|
| 16093 |
+
|
| 16094 |
+
|
| 16095 |
+
98%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 13600/13822 [14:43:22<08:59, 2.43s/it]
|
| 16096 |
+
|
| 16097 |
+
|
| 16098 |
+
|
| 16099 |
+
|
| 16100 |
+
|
| 16101 |
+
|
| 16102 |
+
|
| 16103 |
+
|
| 16104 |
+
|
| 16105 |
+
|
| 16106 |
+
|
| 16107 |
+
|
| 16108 |
+
|
| 16109 |
+
|
| 16110 |
+
|
| 16111 |
+
|
| 16112 |
+
|
| 16113 |
+
|
| 16114 |
+
|
| 16115 |
+
|
| 16116 |
+
|
| 16117 |
+
|
| 16118 |
+
|
| 16119 |
+
|
| 16120 |
+
|
| 16121 |
+
|
| 16122 |
+
|
| 16123 |
+
|
| 16124 |
+
|
| 16125 |
+
|
| 16126 |
+
|
| 16127 |
+
|
| 16128 |
+
|
| 16129 |
+
|
| 16130 |
+
|
| 16131 |
+
|
| 16132 |
+
|
| 16133 |
+
|
| 16134 |
+
|
| 16135 |
+
|
| 16136 |
+
|
| 16137 |
+
|
| 16138 |
+
|
| 16139 |
+
|
| 16140 |
+
|
| 16141 |
+
|
| 16142 |
+
|
| 16143 |
+
|
| 16144 |
+
|
| 16145 |
+
|
| 16146 |
+
|
| 16147 |
+
|
| 16148 |
+
|
| 16149 |
+
|
| 16150 |
+
|
| 16151 |
+
|
| 16152 |
+
|
| 16153 |
+
|
| 16154 |
+
|
| 16155 |
+
|
| 16156 |
+
|
| 16157 |
+
|
| 16158 |
+
|
| 16159 |
+
|
| 16160 |
+
|
| 16161 |
+
|
| 16162 |
+
|
| 16163 |
+
|
| 16164 |
+
|
| 16165 |
+
|
| 16166 |
+
|
| 16167 |
+
|
| 16168 |
+
|
| 16169 |
+
|
| 16170 |
+
|
| 16171 |
+
|
| 16172 |
+
|
| 16173 |
+
|
| 16174 |
+
|
| 16175 |
+
|
| 16176 |
+
|
| 16177 |
+
|
| 16178 |
+
|
| 16179 |
+
|
| 16180 |
+
|
| 16181 |
+
|
| 16182 |
+
|
| 16183 |
+
|
| 16184 |
+
|
| 16185 |
+
|
| 16186 |
+
|
| 16187 |
+
|
| 16188 |
+
|
| 16189 |
+
|
| 16190 |
+
|
| 16191 |
+
|
| 16192 |
+
99%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 13699/13822 [14:48:30<04:56, 2.41s/it]
|
| 16193 |
+
|
| 16194 |
+
|
| 16195 |
+
|
| 16196 |
+
|
| 16197 |
+
|
| 16198 |
+
|
| 16199 |
+
|
| 16200 |
+
|
| 16201 |
+
|
| 16202 |
+
|
| 16203 |
+
|
| 16204 |
+
|
| 16205 |
+
|
| 16206 |
+
|
| 16207 |
+
|
| 16208 |
+
|
| 16209 |
+
|
| 16210 |
+
|
| 16211 |
+
|
| 16212 |
+
|
| 16213 |
+
|
| 16214 |
+
|
| 16215 |
+
|
| 16216 |
+
|
| 16217 |
+
|
| 16218 |
+
|
| 16219 |
+
|
| 16220 |
+
|
| 16221 |
+
|
| 16222 |
+
|
| 16223 |
+
|
| 16224 |
+
|
| 16225 |
+
|
| 16226 |
+
|
| 16227 |
+
|
| 16228 |
+
|
| 16229 |
+
|
| 16230 |
+
|
| 16231 |
+
|
| 16232 |
+
|
| 16233 |
+
|
| 16234 |
+
|
| 16235 |
+
|
| 16236 |
+
|
| 16237 |
+
|
| 16238 |
+
|
| 16239 |
+
|
| 16240 |
+
|
| 16241 |
+
|
| 16242 |
+
|
| 16243 |
+
|
| 16244 |
+
|
| 16245 |
+
|
| 16246 |
+
|
| 16247 |
+
|
| 16248 |
+
|
| 16249 |
+
|
| 16250 |
+
|
| 16251 |
+
|
| 16252 |
+
|
| 16253 |
+
|
| 16254 |
+
|
| 16255 |
+
|
| 16256 |
+
|
| 16257 |
+
|
| 16258 |
+
|
| 16259 |
+
|
| 16260 |
+
|
| 16261 |
+
|
| 16262 |
+
|
| 16263 |
+
|
| 16264 |
+
|
| 16265 |
+
|
| 16266 |
+
|
| 16267 |
+
|
| 16268 |
+
|
| 16269 |
+
|
| 16270 |
+
|
| 16271 |
+
|
| 16272 |
+
|
| 16273 |
+
|
| 16274 |
+
|
| 16275 |
+
|
| 16276 |
+
|
| 16277 |
+
|
| 16278 |
+
|
| 16279 |
+
|
| 16280 |
+
|
| 16281 |
+
|
| 16282 |
+
|
| 16283 |
+
|
| 16284 |
+
|
| 16285 |
+
|
| 16286 |
+
|
| 16287 |
+
|
| 16288 |
+
|
| 16289 |
+
|
| 16290 |
+
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋| 13799/13822 [14:53:40<00:55, 2.40s/it]
|
| 16291 |
+
|
| 16292 |
+
|
| 16293 |
+
|
| 16294 |
+
|
| 16295 |
+
|
| 16296 |
+
|
| 16297 |
+
|
| 16298 |
+
|
| 16299 |
+
|
| 16300 |
+
|
| 16301 |
+
|
| 16302 |
+
|
| 16303 |
+
|
| 16304 |
+
|
| 16305 |
+
|
| 16306 |
+
|
| 16307 |
+
|
| 16308 |
+
|
| 16309 |
+
|
| 16310 |
+
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13822/13822 [14:54:36<00:00, 1.87s/it]
|
| 16311 |
+
Training completed. Do not forget to share your model on huggingface.co/models =)
|
| 16312 |
+
Loading best model from ./checkpoint-13500 (score: 0.23875188827514648).
|
| 16313 |
+
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13822/13822 [14:54:38<00:00, 3.88s/it]
|
| 16314 |
+
Saving model checkpoint to ./
|
| 16315 |
+
Configuration saved in ./config.json
|
| 16316 |
+
{'train_runtime': 53680.5392, 'train_samples_per_second': 16.478, 'train_steps_per_second': 0.257, 'train_loss': 1.442369053426242, 'epoch': 2.0}
|
| 16317 |
+
Model weights saved in ./pytorch_model.bin
|
| 16318 |
+
Configuration saved in ./preprocessor_config.json
|
| 16319 |
+
Saving model checkpoint to ./
|
| 16320 |
+
Configuration saved in ./config.json
|
| 16321 |
+
Model weights saved in ./pytorch_model.bin
|
wandb/run-20220129_215451-1vipdbow/files/wandb-summary.json
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
wandb/run-20220129_215451-1vipdbow/logs/debug-internal.log
CHANGED
|
@@ -26371,3 +26371,507 @@
|
|
| 26371 |
2022-01-30 12:32:49,776 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26372 |
2022-01-30 12:32:52,985 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26373 |
2022-01-30 12:32:52,986 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26371 |
2022-01-30 12:32:49,776 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26372 |
2022-01-30 12:32:52,985 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26373 |
2022-01-30 12:32:52,986 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26374 |
+
2022-01-30 12:33:08,227 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26375 |
+
2022-01-30 12:33:08,228 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26376 |
+
2022-01-30 12:33:08,584 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26377 |
+
2022-01-30 12:33:12,587 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26378 |
+
2022-01-30 12:33:14,588 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26379 |
+
2022-01-30 12:33:16,589 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26380 |
+
2022-01-30 12:33:18,590 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26381 |
+
2022-01-30 12:33:20,024 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26382 |
+
2022-01-30 12:33:20,591 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26383 |
+
2022-01-30 12:33:22,592 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26384 |
+
2022-01-30 12:33:23,455 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26385 |
+
2022-01-30 12:33:23,455 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26386 |
+
2022-01-30 12:33:24,594 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26387 |
+
2022-01-30 12:33:26,595 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26388 |
+
2022-01-30 12:33:28,596 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26389 |
+
2022-01-30 12:33:34,599 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26390 |
+
2022-01-30 12:33:38,656 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26391 |
+
2022-01-30 12:33:38,657 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26392 |
+
2022-01-30 12:33:40,603 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26393 |
+
2022-01-30 12:33:46,607 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26394 |
+
2022-01-30 12:33:50,279 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26395 |
+
2022-01-30 12:33:50,609 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26396 |
+
2022-01-30 12:33:53,945 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26397 |
+
2022-01-30 12:33:53,945 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26398 |
+
2022-01-30 12:33:55,612 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26399 |
+
2022-01-30 12:33:59,615 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26400 |
+
2022-01-30 12:34:05,619 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26401 |
+
2022-01-30 12:34:09,203 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26402 |
+
2022-01-30 12:34:09,203 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26403 |
+
2022-01-30 12:34:09,621 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26404 |
+
2022-01-30 12:34:13,624 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26405 |
+
2022-01-30 12:34:17,627 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26406 |
+
2022-01-30 12:34:19,628 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26407 |
+
2022-01-30 12:34:20,563 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26408 |
+
2022-01-30 12:34:23,635 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26409 |
+
2022-01-30 12:34:24,375 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26410 |
+
2022-01-30 12:34:24,376 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26411 |
+
2022-01-30 12:34:27,638 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26412 |
+
2022-01-30 12:34:31,641 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26413 |
+
2022-01-30 12:34:35,644 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26414 |
+
2022-01-30 12:34:37,646 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26415 |
+
2022-01-30 12:34:39,544 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26416 |
+
2022-01-30 12:34:39,544 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26417 |
+
2022-01-30 12:34:41,649 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26418 |
+
2022-01-30 12:34:45,651 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26419 |
+
2022-01-30 12:34:47,653 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26420 |
+
2022-01-30 12:34:50,830 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26421 |
+
2022-01-30 12:34:51,656 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26422 |
+
2022-01-30 12:34:54,801 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26423 |
+
2022-01-30 12:34:54,801 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26424 |
+
2022-01-30 12:34:55,659 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26425 |
+
2022-01-30 12:34:57,660 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26426 |
+
2022-01-30 12:35:01,663 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26427 |
+
2022-01-30 12:35:03,664 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26428 |
+
2022-01-30 12:35:07,667 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26429 |
+
2022-01-30 12:35:09,668 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26430 |
+
2022-01-30 12:35:09,968 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26431 |
+
2022-01-30 12:35:09,968 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26432 |
+
2022-01-30 12:35:12,670 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26433 |
+
2022-01-30 12:35:16,672 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26434 |
+
2022-01-30 12:35:18,674 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26435 |
+
2022-01-30 12:35:20,675 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26436 |
+
2022-01-30 12:35:21,062 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26437 |
+
2022-01-30 12:35:24,678 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26438 |
+
2022-01-30 12:35:25,223 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26439 |
+
2022-01-30 12:35:25,224 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26440 |
+
2022-01-30 12:35:26,679 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26441 |
+
2022-01-30 12:35:28,680 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26442 |
+
2022-01-30 12:35:32,683 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26443 |
+
2022-01-30 12:35:34,684 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26444 |
+
2022-01-30 12:35:36,686 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26445 |
+
2022-01-30 12:35:38,687 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26446 |
+
2022-01-30 12:35:40,449 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26447 |
+
2022-01-30 12:35:40,449 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26448 |
+
2022-01-30 12:35:40,688 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26449 |
+
2022-01-30 12:35:42,690 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26450 |
+
2022-01-30 12:35:44,691 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26451 |
+
2022-01-30 12:35:46,692 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26452 |
+
2022-01-30 12:35:48,694 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26453 |
+
2022-01-30 12:35:50,695 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26454 |
+
2022-01-30 12:35:51,297 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26455 |
+
2022-01-30 12:35:52,696 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26456 |
+
2022-01-30 12:35:54,698 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26457 |
+
2022-01-30 12:35:55,670 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26458 |
+
2022-01-30 12:35:55,671 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26459 |
+
2022-01-30 12:35:56,699 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26460 |
+
2022-01-30 12:35:58,701 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26461 |
+
2022-01-30 12:36:00,702 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26462 |
+
2022-01-30 12:36:02,703 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26463 |
+
2022-01-30 12:36:08,706 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26464 |
+
2022-01-30 12:36:10,882 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26465 |
+
2022-01-30 12:36:10,883 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26466 |
+
2022-01-30 12:36:14,711 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26467 |
+
2022-01-30 12:36:19,714 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26468 |
+
2022-01-30 12:36:21,601 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26469 |
+
2022-01-30 12:36:25,718 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26470 |
+
2022-01-30 12:36:26,054 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26471 |
+
2022-01-30 12:36:26,055 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26472 |
+
2022-01-30 12:36:29,721 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26473 |
+
2022-01-30 12:36:33,724 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26474 |
+
2022-01-30 12:36:37,727 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26475 |
+
2022-01-30 12:36:41,296 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26476 |
+
2022-01-30 12:36:41,297 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26477 |
+
2022-01-30 12:36:41,730 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26478 |
+
2022-01-30 12:36:45,732 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26479 |
+
2022-01-30 12:36:49,735 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26480 |
+
2022-01-30 12:36:51,890 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26481 |
+
2022-01-30 12:36:53,738 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26482 |
+
2022-01-30 12:36:56,569 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26483 |
+
2022-01-30 12:36:56,570 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26484 |
+
2022-01-30 12:36:57,743 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26485 |
+
2022-01-30 12:37:01,745 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26486 |
+
2022-01-30 12:37:06,748 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26487 |
+
2022-01-30 12:37:10,751 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26488 |
+
2022-01-30 12:37:11,901 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26489 |
+
2022-01-30 12:37:11,901 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26490 |
+
2022-01-30 12:37:12,752 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26491 |
+
2022-01-30 12:37:16,754 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26492 |
+
2022-01-30 12:37:20,757 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26493 |
+
2022-01-30 12:37:22,096 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26494 |
+
2022-01-30 12:37:22,758 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26495 |
+
2022-01-30 12:37:26,761 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26496 |
+
2022-01-30 12:37:27,171 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26497 |
+
2022-01-30 12:37:27,172 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26498 |
+
2022-01-30 12:37:28,762 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26499 |
+
2022-01-30 12:37:32,765 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26500 |
+
2022-01-30 12:37:36,768 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26501 |
+
2022-01-30 12:37:38,769 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26502 |
+
2022-01-30 12:37:40,771 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26503 |
+
2022-01-30 12:37:42,337 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26504 |
+
2022-01-30 12:37:42,338 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26505 |
+
2022-01-30 12:37:44,774 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26506 |
+
2022-01-30 12:37:46,775 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26507 |
+
2022-01-30 12:37:50,779 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26508 |
+
2022-01-30 12:37:52,316 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26509 |
+
2022-01-30 12:37:52,780 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26510 |
+
2022-01-30 12:37:54,781 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26511 |
+
2022-01-30 12:37:57,580 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26512 |
+
2022-01-30 12:37:57,580 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26513 |
+
2022-01-30 12:37:58,784 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26514 |
+
2022-01-30 12:38:00,785 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26515 |
+
2022-01-30 12:38:02,786 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26516 |
+
2022-01-30 12:38:06,789 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26517 |
+
2022-01-30 12:38:08,791 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26518 |
+
2022-01-30 12:38:10,793 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26519 |
+
2022-01-30 12:38:12,794 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26520 |
+
2022-01-30 12:38:12,828 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26521 |
+
2022-01-30 12:38:12,828 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26522 |
+
2022-01-30 12:38:14,797 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26523 |
+
2022-01-30 12:38:16,208 DEBUG SenderThread:37713 [sender.py:send():234] send: history
|
| 26524 |
+
2022-01-30 12:38:16,311 DEBUG SenderThread:37713 [sender.py:send():234] send: summary
|
| 26525 |
+
2022-01-30 12:38:16,362 INFO SenderThread:37713 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
|
| 26526 |
+
2022-01-30 12:38:16,802 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/wandb-summary.json
|
| 26527 |
+
2022-01-30 12:38:16,803 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26528 |
+
2022-01-30 12:38:17,804 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26529 |
+
2022-01-30 12:38:18,805 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26530 |
+
2022-01-30 12:38:22,560 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26531 |
+
2022-01-30 12:38:23,809 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26532 |
+
2022-01-30 12:38:25,811 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26533 |
+
2022-01-30 12:38:27,813 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26534 |
+
2022-01-30 12:38:27,998 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26535 |
+
2022-01-30 12:38:27,998 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26536 |
+
2022-01-30 12:38:29,815 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26537 |
+
2022-01-30 12:38:31,817 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26538 |
+
2022-01-30 12:38:33,819 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26539 |
+
2022-01-30 12:38:35,820 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26540 |
+
2022-01-30 12:38:37,822 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26541 |
+
2022-01-30 12:38:43,318 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26542 |
+
2022-01-30 12:38:43,318 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26543 |
+
2022-01-30 12:38:43,826 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26544 |
+
2022-01-30 12:38:49,831 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26545 |
+
2022-01-30 12:38:52,796 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26546 |
+
2022-01-30 12:38:55,836 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26547 |
+
2022-01-30 12:38:58,695 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26548 |
+
2022-01-30 12:38:58,696 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26549 |
+
2022-01-30 12:38:59,839 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26550 |
+
2022-01-30 12:39:03,842 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26551 |
+
2022-01-30 12:39:07,844 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26552 |
+
2022-01-30 12:39:13,849 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26553 |
+
2022-01-30 12:39:13,983 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26554 |
+
2022-01-30 12:39:13,984 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26555 |
+
2022-01-30 12:39:17,852 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26556 |
+
2022-01-30 12:39:21,858 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26557 |
+
2022-01-30 12:39:23,013 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26558 |
+
2022-01-30 12:39:23,876 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26559 |
+
2022-01-30 12:39:28,880 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26560 |
+
2022-01-30 12:39:29,195 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26561 |
+
2022-01-30 12:39:29,196 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26562 |
+
2022-01-30 12:39:32,883 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26563 |
+
2022-01-30 12:39:36,886 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26564 |
+
2022-01-30 12:39:40,889 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26565 |
+
2022-01-30 12:39:44,381 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26566 |
+
2022-01-30 12:39:44,381 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26567 |
+
2022-01-30 12:39:44,892 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26568 |
+
2022-01-30 12:39:46,894 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26569 |
+
2022-01-30 12:39:50,898 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26570 |
+
2022-01-30 12:39:53,239 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26571 |
+
2022-01-30 12:39:54,901 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26572 |
+
2022-01-30 12:39:58,904 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26573 |
+
2022-01-30 12:39:59,577 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26574 |
+
2022-01-30 12:39:59,578 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26575 |
+
2022-01-30 12:40:00,905 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26576 |
+
2022-01-30 12:40:04,908 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26577 |
+
2022-01-30 12:40:06,910 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26578 |
+
2022-01-30 12:40:10,913 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26579 |
+
2022-01-30 12:40:12,915 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26580 |
+
2022-01-30 12:40:14,808 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26581 |
+
2022-01-30 12:40:14,809 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26582 |
+
2022-01-30 12:40:16,917 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26583 |
+
2022-01-30 12:40:18,919 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26584 |
+
2022-01-30 12:40:22,922 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26585 |
+
2022-01-30 12:40:23,537 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26586 |
+
2022-01-30 12:40:24,923 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26587 |
+
2022-01-30 12:40:26,924 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26588 |
+
2022-01-30 12:40:30,052 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26589 |
+
2022-01-30 12:40:30,053 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26590 |
+
2022-01-30 12:40:30,928 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26591 |
+
2022-01-30 12:40:33,929 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26592 |
+
2022-01-30 12:40:35,930 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26593 |
+
2022-01-30 12:40:39,935 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26594 |
+
2022-01-30 12:40:41,937 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26595 |
+
2022-01-30 12:40:43,939 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26596 |
+
2022-01-30 12:40:45,347 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26597 |
+
2022-01-30 12:40:45,348 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26598 |
+
2022-01-30 12:40:45,945 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26599 |
+
2022-01-30 12:40:47,947 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26600 |
+
2022-01-30 12:40:51,952 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26601 |
+
2022-01-30 12:40:53,749 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26602 |
+
2022-01-30 12:40:53,954 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26603 |
+
2022-01-30 12:40:55,962 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26604 |
+
2022-01-30 12:40:57,963 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26605 |
+
2022-01-30 12:40:59,965 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26606 |
+
2022-01-30 12:41:00,567 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26607 |
+
2022-01-30 12:41:00,568 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26608 |
+
2022-01-30 12:41:01,968 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26609 |
+
2022-01-30 12:41:03,969 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26610 |
+
2022-01-30 12:41:05,970 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26611 |
+
2022-01-30 12:41:07,972 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26612 |
+
2022-01-30 12:41:09,973 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26613 |
+
2022-01-30 12:41:11,975 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26614 |
+
2022-01-30 12:41:13,977 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26615 |
+
2022-01-30 12:41:15,751 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26616 |
+
2022-01-30 12:41:15,751 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26617 |
+
2022-01-30 12:41:19,980 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26618 |
+
2022-01-30 12:41:23,983 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26619 |
+
2022-01-30 12:41:24,107 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26620 |
+
2022-01-30 12:41:29,999 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26621 |
+
2022-01-30 12:41:31,400 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26622 |
+
2022-01-30 12:41:31,401 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26623 |
+
2022-01-30 12:41:34,002 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26624 |
+
2022-01-30 12:41:41,007 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26625 |
+
2022-01-30 12:41:45,010 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26626 |
+
2022-01-30 12:41:46,673 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26627 |
+
2022-01-30 12:41:46,673 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26628 |
+
2022-01-30 12:41:49,013 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26629 |
+
2022-01-30 12:41:53,016 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26630 |
+
2022-01-30 12:41:54,468 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26631 |
+
2022-01-30 12:41:57,020 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26632 |
+
2022-01-30 12:42:01,022 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26633 |
+
2022-01-30 12:42:01,914 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26634 |
+
2022-01-30 12:42:01,915 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26635 |
+
2022-01-30 12:42:05,025 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26636 |
+
2022-01-30 12:42:09,028 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26637 |
+
2022-01-30 12:42:11,030 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26638 |
+
2022-01-30 12:42:15,033 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26639 |
+
2022-01-30 12:42:17,076 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26640 |
+
2022-01-30 12:42:17,077 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26641 |
+
2022-01-30 12:42:19,036 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26642 |
+
2022-01-30 12:42:23,039 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26643 |
+
2022-01-30 12:42:24,766 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26644 |
+
2022-01-30 12:42:25,050 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26645 |
+
2022-01-30 12:42:29,054 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26646 |
+
2022-01-30 12:42:32,382 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26647 |
+
2022-01-30 12:42:32,383 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26648 |
+
2022-01-30 12:42:33,057 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26649 |
+
2022-01-30 12:42:35,060 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26650 |
+
2022-01-30 12:42:39,063 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26651 |
+
2022-01-30 12:42:43,065 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26652 |
+
2022-01-30 12:42:46,067 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26653 |
+
2022-01-30 12:42:47,722 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26654 |
+
2022-01-30 12:42:47,722 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26655 |
+
2022-01-30 12:42:50,070 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26656 |
+
2022-01-30 12:42:52,072 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26657 |
+
2022-01-30 12:42:54,074 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26658 |
+
2022-01-30 12:42:55,100 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26659 |
+
2022-01-30 12:42:58,077 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26660 |
+
2022-01-30 12:43:00,080 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26661 |
+
2022-01-30 12:43:02,973 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26662 |
+
2022-01-30 12:43:02,974 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26663 |
+
2022-01-30 12:43:04,083 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26664 |
+
2022-01-30 12:43:06,085 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26665 |
+
2022-01-30 12:43:08,086 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26666 |
+
2022-01-30 12:43:12,089 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26667 |
+
2022-01-30 12:43:14,090 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26668 |
+
2022-01-30 12:43:16,091 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26669 |
+
2022-01-30 12:43:18,093 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26670 |
+
2022-01-30 12:43:18,146 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26671 |
+
2022-01-30 12:43:18,146 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26672 |
+
2022-01-30 12:43:20,094 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26673 |
+
2022-01-30 12:43:24,097 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26674 |
+
2022-01-30 12:43:25,407 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26675 |
+
2022-01-30 12:43:26,098 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26676 |
+
2022-01-30 12:43:26,484 DEBUG SenderThread:37713 [sender.py:send():234] send: history
|
| 26677 |
+
2022-01-30 12:43:26,529 DEBUG SenderThread:37713 [sender.py:send():234] send: summary
|
| 26678 |
+
2022-01-30 12:43:26,563 INFO SenderThread:37713 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
|
| 26679 |
+
2022-01-30 12:43:27,099 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/wandb-summary.json
|
| 26680 |
+
2022-01-30 12:43:28,099 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26681 |
+
2022-01-30 12:43:30,101 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26682 |
+
2022-01-30 12:43:32,102 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26683 |
+
2022-01-30 12:43:33,383 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26684 |
+
2022-01-30 12:43:33,384 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26685 |
+
2022-01-30 12:43:34,103 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26686 |
+
2022-01-30 12:43:36,105 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26687 |
+
2022-01-30 12:43:38,106 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26688 |
+
2022-01-30 12:43:40,108 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26689 |
+
2022-01-30 12:43:42,109 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26690 |
+
2022-01-30 12:43:44,111 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26691 |
+
2022-01-30 12:43:46,112 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26692 |
+
2022-01-30 12:43:48,114 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26693 |
+
2022-01-30 12:43:48,798 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26694 |
+
2022-01-30 12:43:48,798 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26695 |
+
2022-01-30 12:43:54,119 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26696 |
+
2022-01-30 12:43:55,646 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26697 |
+
2022-01-30 12:44:01,123 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26698 |
+
2022-01-30 12:44:04,300 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26699 |
+
2022-01-30 12:44:04,300 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26700 |
+
2022-01-30 12:44:05,126 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26701 |
+
2022-01-30 12:44:11,131 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26702 |
+
2022-01-30 12:44:15,136 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26703 |
+
2022-01-30 12:44:19,139 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26704 |
+
2022-01-30 12:44:19,650 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26705 |
+
2022-01-30 12:44:19,650 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26706 |
+
2022-01-30 12:44:23,142 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26707 |
+
2022-01-30 12:44:25,870 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26708 |
+
2022-01-30 12:44:27,145 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26709 |
+
2022-01-30 12:44:31,149 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26710 |
+
2022-01-30 12:44:34,892 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26711 |
+
2022-01-30 12:44:34,893 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26712 |
+
2022-01-30 12:44:35,152 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26713 |
+
2022-01-30 12:44:39,157 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26714 |
+
2022-01-30 12:44:43,159 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26715 |
+
2022-01-30 12:44:47,162 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26716 |
+
2022-01-30 12:44:50,201 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26717 |
+
2022-01-30 12:44:50,201 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26718 |
+
2022-01-30 12:44:51,166 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26719 |
+
2022-01-30 12:44:56,133 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26720 |
+
2022-01-30 12:44:56,170 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26721 |
+
2022-01-30 12:44:58,172 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26722 |
+
2022-01-30 12:45:02,175 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26723 |
+
2022-01-30 12:45:05,469 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26724 |
+
2022-01-30 12:45:05,469 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26725 |
+
2022-01-30 12:45:06,179 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26726 |
+
2022-01-30 12:45:08,181 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26727 |
+
2022-01-30 12:45:12,185 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26728 |
+
2022-01-30 12:45:16,189 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26729 |
+
2022-01-30 12:45:18,191 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26730 |
+
2022-01-30 12:45:20,693 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26731 |
+
2022-01-30 12:45:20,693 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26732 |
+
2022-01-30 12:45:22,195 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26733 |
+
2022-01-30 12:45:24,196 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26734 |
+
2022-01-30 12:45:26,392 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26735 |
+
2022-01-30 12:45:28,199 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26736 |
+
2022-01-30 12:45:30,201 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26737 |
+
2022-01-30 12:45:34,205 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26738 |
+
2022-01-30 12:45:35,969 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26739 |
+
2022-01-30 12:45:35,970 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26740 |
+
2022-01-30 12:45:36,207 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26741 |
+
2022-01-30 12:45:38,209 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26742 |
+
2022-01-30 12:45:40,210 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26743 |
+
2022-01-30 12:45:44,215 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26744 |
+
2022-01-30 12:45:46,218 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26745 |
+
2022-01-30 12:45:49,220 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26746 |
+
2022-01-30 12:45:51,248 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26747 |
+
2022-01-30 12:45:51,248 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26748 |
+
2022-01-30 12:45:53,224 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26749 |
+
2022-01-30 12:45:55,225 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26750 |
+
2022-01-30 12:45:56,642 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26751 |
+
2022-01-30 12:45:57,227 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26752 |
+
2022-01-30 12:45:59,228 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26753 |
+
2022-01-30 12:46:01,230 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26754 |
+
2022-01-30 12:46:03,232 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26755 |
+
2022-01-30 12:46:05,233 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26756 |
+
2022-01-30 12:46:06,498 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26757 |
+
2022-01-30 12:46:06,499 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26758 |
+
2022-01-30 12:46:09,236 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26759 |
+
2022-01-30 12:46:11,238 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26760 |
+
2022-01-30 12:46:13,239 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26761 |
+
2022-01-30 12:46:15,241 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26762 |
+
2022-01-30 12:46:17,242 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26763 |
+
2022-01-30 12:46:19,244 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26764 |
+
2022-01-30 12:46:21,246 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26765 |
+
2022-01-30 12:46:21,759 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26766 |
+
2022-01-30 12:46:21,759 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26767 |
+
2022-01-30 12:46:23,248 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26768 |
+
2022-01-30 12:46:26,931 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26769 |
+
2022-01-30 12:46:29,253 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26770 |
+
2022-01-30 12:46:35,257 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26771 |
+
2022-01-30 12:46:36,947 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26772 |
+
2022-01-30 12:46:36,948 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26773 |
+
2022-01-30 12:46:39,260 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26774 |
+
2022-01-30 12:46:46,265 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26775 |
+
2022-01-30 12:46:50,268 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26776 |
+
2022-01-30 12:46:52,116 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26777 |
+
2022-01-30 12:46:52,116 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26778 |
+
2022-01-30 12:46:54,271 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26779 |
+
2022-01-30 12:46:57,239 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26780 |
+
2022-01-30 12:46:58,274 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26781 |
+
2022-01-30 12:47:02,277 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26782 |
+
2022-01-30 12:47:06,279 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26783 |
+
2022-01-30 12:47:07,318 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26784 |
+
2022-01-30 12:47:07,318 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26785 |
+
2022-01-30 12:47:10,282 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26786 |
+
2022-01-30 12:47:14,286 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26787 |
+
2022-01-30 12:47:18,289 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26788 |
+
2022-01-30 12:47:22,292 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26789 |
+
2022-01-30 12:47:22,521 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26790 |
+
2022-01-30 12:47:22,522 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26791 |
+
2022-01-30 12:47:26,295 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26792 |
+
2022-01-30 12:47:27,463 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26793 |
+
2022-01-30 12:47:30,298 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26794 |
+
2022-01-30 12:47:32,300 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26795 |
+
2022-01-30 12:47:37,304 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26796 |
+
2022-01-30 12:47:37,736 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26797 |
+
2022-01-30 12:47:37,737 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26798 |
+
2022-01-30 12:47:41,307 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26799 |
+
2022-01-30 12:47:43,308 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26800 |
+
2022-01-30 12:47:47,311 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26801 |
+
2022-01-30 12:47:49,312 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26802 |
+
2022-01-30 12:47:52,905 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26803 |
+
2022-01-30 12:47:52,906 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26804 |
+
2022-01-30 12:47:53,315 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26805 |
+
2022-01-30 12:47:55,316 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26806 |
+
2022-01-30 12:47:57,678 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26807 |
+
2022-01-30 12:47:59,319 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26808 |
+
2022-01-30 12:48:01,320 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26809 |
+
2022-01-30 12:48:05,323 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26810 |
+
2022-01-30 12:48:07,324 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26811 |
+
2022-01-30 12:48:08,074 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26812 |
+
2022-01-30 12:48:08,075 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26813 |
+
2022-01-30 12:48:11,327 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26814 |
+
2022-01-30 12:48:13,328 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26815 |
+
2022-01-30 12:48:15,330 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26816 |
+
2022-01-30 12:48:19,332 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26817 |
+
2022-01-30 12:48:21,334 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26818 |
+
2022-01-30 12:48:23,335 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26819 |
+
2022-01-30 12:48:23,335 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26820 |
+
2022-01-30 12:48:23,340 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26821 |
+
2022-01-30 12:48:27,343 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26822 |
+
2022-01-30 12:48:27,940 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26823 |
+
2022-01-30 12:48:29,345 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26824 |
+
2022-01-30 12:48:31,346 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26825 |
+
2022-01-30 12:48:33,349 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26826 |
+
2022-01-30 12:48:35,351 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26827 |
+
2022-01-30 12:48:37,003 DEBUG SenderThread:37713 [sender.py:send():234] send: history
|
| 26828 |
+
2022-01-30 12:48:37,111 DEBUG SenderThread:37713 [sender.py:send():234] send: summary
|
| 26829 |
+
2022-01-30 12:48:37,165 INFO SenderThread:37713 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
|
| 26830 |
+
2022-01-30 12:48:37,353 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/wandb-summary.json
|
| 26831 |
+
2022-01-30 12:48:37,353 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26832 |
+
2022-01-30 12:48:38,573 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26833 |
+
2022-01-30 12:48:38,574 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26834 |
+
2022-01-30 12:48:41,356 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26835 |
+
2022-01-30 12:48:43,358 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26836 |
+
2022-01-30 12:48:46,361 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26837 |
+
2022-01-30 12:48:48,363 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26838 |
+
2022-01-30 12:48:50,365 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26839 |
+
2022-01-30 12:48:52,367 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26840 |
+
2022-01-30 12:48:53,746 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26841 |
+
2022-01-30 12:48:53,747 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26842 |
+
2022-01-30 12:48:54,369 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26843 |
+
2022-01-30 12:48:56,370 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26844 |
+
2022-01-30 12:48:58,204 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26845 |
+
2022-01-30 12:48:58,372 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26846 |
+
2022-01-30 12:49:04,376 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26847 |
+
2022-01-30 12:49:08,379 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26848 |
+
2022-01-30 12:49:08,919 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26849 |
+
2022-01-30 12:49:08,920 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26850 |
+
2022-01-30 12:49:12,382 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26851 |
+
2022-01-30 12:49:16,385 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26852 |
+
2022-01-30 12:49:18,386 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26853 |
+
2022-01-30 12:49:22,389 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26854 |
+
2022-01-30 12:49:24,089 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26855 |
+
2022-01-30 12:49:24,089 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26856 |
+
2022-01-30 12:49:24,390 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26857 |
+
2022-01-30 12:49:26,391 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26858 |
+
2022-01-30 12:49:28,393 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26859 |
+
2022-01-30 12:49:28,484 DEBUG SenderThread:37713 [sender.py:send():234] send: stats
|
| 26860 |
+
2022-01-30 12:49:30,394 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26861 |
+
2022-01-30 12:49:31,688 DEBUG SenderThread:37713 [sender.py:send():234] send: metric
|
| 26862 |
+
2022-01-30 12:49:31,688 DEBUG SenderThread:37713 [sender.py:send():234] send: metric
|
| 26863 |
+
2022-01-30 12:49:31,688 DEBUG SenderThread:37713 [sender.py:send():234] send: metric
|
| 26864 |
+
2022-01-30 12:49:31,688 DEBUG SenderThread:37713 [sender.py:send():234] send: metric
|
| 26865 |
+
2022-01-30 12:49:31,688 DEBUG SenderThread:37713 [sender.py:send():234] send: metric
|
| 26866 |
+
2022-01-30 12:49:31,689 DEBUG SenderThread:37713 [sender.py:send():234] send: history
|
| 26867 |
+
2022-01-30 12:49:31,730 DEBUG SenderThread:37713 [sender.py:send():234] send: summary
|
| 26868 |
+
2022-01-30 12:49:31,778 INFO SenderThread:37713 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
|
| 26869 |
+
2022-01-30 12:49:32,395 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/wandb-summary.json
|
| 26870 |
+
2022-01-30 12:49:32,396 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26871 |
+
2022-01-30 12:49:33,396 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26872 |
+
2022-01-30 12:49:36,398 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26873 |
+
2022-01-30 12:49:38,400 INFO Thread-8 :37713 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-fr/wandb/run-20220129_215451-1vipdbow/files/output.log
|
| 26874 |
+
2022-01-30 12:49:39,258 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26875 |
+
2022-01-30 12:49:39,259 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
| 26876 |
+
2022-01-30 12:49:54,438 DEBUG HandlerThread:37713 [handler.py:handle_request():130] handle_request: stop_status
|
| 26877 |
+
2022-01-30 12:49:54,439 DEBUG SenderThread:37713 [sender.py:send_request():248] send_request: stop_status
|
wandb/run-20220129_215451-1vipdbow/run-1vipdbow.wandb
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:18794eef9e42247634914cf05d9fefdbeef77d0df2372e7524767722937f1ca9
|
| 3 |
+
size 104480612
|