Training in progress, step 3000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:51dde3d5b9d7f85051df056de89e082563e3d018cb76f28d8224c82be9328a8c
+oid sha256:2e2f9ffcf3e4f0917bfddf6abbdf0e45eacb86241c0acc0051e9b851df9dc04c
 size 967102601

training.log

@@ -1020,3 +1020,108 @@ Adding files tracked by Git LFS: ['wandb/run-20230506_113337-ysywp688/run-ysywp6
 [INFO|modeling_utils.py:1855] 2023-05-08 03:23:01,997 >> Model weights saved in ./checkpoint-2000/pytorch_model.bin
 [INFO|feature_extraction_utils.py:369] 2023-05-08 03:23:02,003 >> Feature extractor saved in ./checkpoint-2000/preprocessor_config.json
 [INFO|feature_extraction_utils.py:369] 2023-05-08 03:23:12,574 >> Feature extractor saved in ./preprocessor_config.json
+/home/local/QCRI/dizham/miniconda3/envs/whisper/lib/python3.9/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
+  warnings.warn('Was asked to gather along dimension 0, but all '
+
 40%|████      | 2001/5000 [16:50:15<575:52:06, 691.27s/it]
 40%|████      | 2002/5000 [16:50:44<410:07:12, 492.47s/it]
 40%|████      | 2003/5000 [16:51:12<293:55:31, 353.06s/it]
 40%|████      | 2004/5000 [16:51:40<212:47:31, 255.69s/it]
 40%|████      | 2005/5000 [16:52:08<155:55:38, 187.43s/it]
 40%|████      | 2006/5000 [16:52:36<115:59:50, 139.48s/it]
 40%|████      | 2007/5000 [16:53:04<88:15:57, 106.17s/it] 
 40%|████      | 2008/5000 [16:53:32<68:35:54, 82.54s/it] 
 40%|████      | 2009/5000 [16:54:04<55:56:56, 67.34s/it]
 40%|████      | 2010/5000 [16:54:32<46:18:30, 55.76s/it]
 40%|████      | 2011/5000 [16:55:00<39:16:47, 47.31s/it]
 40%|████      | 2012/5000 [16:55:28<34:31:10, 41.59s/it]
 40%|████      | 2013/5000 [16:55:56<31:05:26, 37.47s/it]
 40%|████      | 2014/5000 [16:56:23<28:28:12, 34.32s/it]
 40%|████      | 2015/5000 [16:56:59<28:53:17, 34.84s/it]
 40%|████      | 2016/5000 [16:57:26<27:00:51, 32.59s/it]
 40%|████      | 2017/5000 [16:57:54<25:43:19, 31.04s/it]
 40%|████      | 2018/5000 [16:58:23<25:22:10, 30.63s/it]
 40%|████      | 2019/5000 [16:58:50<24:27:44, 29.54s/it]
 40%|████      | 2020/5000 [16:59:18<23:55:39, 28.91s/it]
 40%|████      | 2021/5000 [16:59:50<24:40:32, 29.82s/it]
 40%|████      | 2022/5000 [17:00:17<24:03:44, 29.09s/it]
 40%|████      | 2023/5000 [17:00:44<23:34:52, 28.52s/it]
 40%|████      | 2024/5000 [17:01:13<23:31:21, 28.45s/it]
 40%|████      | 2025/5000 [17:01:40<23:17:36, 28.19s/it]
                                                         
 40%|████      | 2025/5000 [17:01:40<23:17:36, 28.19s/it]
 41%|████      | 2026/5000 [17:02:08<23:04:52, 27.94s/it]
 41%|████      | 2027/5000 [17:02:41<24:23:49, 29.54s/it]
 41%|████      | 2028/5000 [17:03:08<23:52:59, 28.93s/it]
 41%|████      | 2029/5000 [17:03:37<23:46:55, 28.82s/it]
 41%|████      | 2030/5000 [17:04:05<23:37:51, 28.64s/it]
 41%|████      | 2031/5000 [17:04:32<23:16:57, 28.23s/it]
 41%|████      | 2032/5000 [17:05:01<23:18:44, 28.28s/it]
 41%|████      | 2033/5000 [17:05:28<22:55:21, 27.81s/it]
 41%|████      | 2034/5000 [17:05:57<23:20:34, 28.33s/it]
 41%|████      | 2035/5000 [17:06:25<23:20:06, 28.33s/it]
 41%|████      | 2036/5000 [17:06:53<23:01:51, 27.97s/it]
 41%|████      | 2037/5000 [17:07:20<22:56:05, 27.87s/it]
 41%|████      | 2038/5000 [17:07:49<23:03:45, 28.03s/it]
 41%|████      | 2039/5000 [17:08:16<22:53:08, 27.82s/it]
 41%|████      | 2040/5000 [17:08:44<23:02:25, 28.02s/it]
 41%|████      | 2041/5000 [17:09:13<23:06:53, 28.12s/it]
 41%|████      | 2042/5000 [17:09:40<22:54:59, 27.89s/it]
 41%|████      | 2043/5000 [17:10:08<22:55:59, 27.92s/it]
 41%|████      | 2044/5000 [17:10:40<23:56:03, 29.15s/it]
 41%|████      | 2045/5000 [17:11:07<23:21:57, 28.47s/it]
 41%|████      | 2046/5000 [17:11:35<23:15:27, 28.34s/it]
 41%|████      | 2047/5000 [17:12:06<23:59:12, 29.24s/it]
 41%|████      | 2048/5000 [17:12:38<24:37:09, 30.02s/it]
 41%|████      | 2049/5000 [17:13:06<23:57:15, 29.22s/it]
 41%|████      | 2050/5000 [17:13:34<23:44:02, 28.96s/it]
                                                         
 41%|████      | 2050/5000 [17:13:34<23:44:02, 28.96s/it]
 41%|████      | 2051/5000 [17:14:02<23:30:09, 28.69s/it]
 41%|████      | 2052/5000 [17:14:29<23:08:17, 28.26s/it]
 41%|████      | 2053/5000 [17:14:57<23:07:07, 28.24s/it]
 41%|████      | 2054/5000 [17:15:25<22:54:13, 27.99s/it]
 41%|████      | 2055/5000 [17:15:52<22:42:08, 27.75s/it]
 41%|████      | 2056/5000 [17:16:20<22:49:50, 27.92s/it]
 41%|████      | 2057/5000 [17:16:48<22:47:50, 27.89s/it]
 41%|████      | 2058/5000 [17:17:15<22:35:43, 27.65s/it]
 41%|████      | 2059/5000 [17:17:44<22:47:52, 27.91s/it]
 41%|████      | 2060/5000 [17:18:12<22:51:21, 27.99s/it]
 41%|████      | 2061/5000 [17:18:39<22:42:07, 27.81s/it]
 41%|████      | 2062/5000 [17:19:12<23:52:27, 29.25s/it]
 41%|████▏     | 2063/5000 [17:19:40<23:29:22, 28.79s/it]
 41%|████▏     | 2064/5000 [17:20:07<23:07:05, 28.35s/it]
 41%|████▏     | 2065/5000 [17:20:38<23:48:58, 29.21s/it]
 41%|████▏     | 2066/5000 [17:21:06<23:23:37, 28.70s/it]
 41%|████▏     | 2067/5000 [17:21:33<23:03:32, 28.30s/it]
 41%|████▏     | 2068/5000 [17:22:03<23:22:05, 28.69s/it]
 41%|████▏     | 2069/5000 [17:22:30<22:56:43, 28.18s/it]
 41%|████▏     | 2070/5000 [17:22:57<22:38:14, 27.81s/it]
 41%|████▏     | 2071/5000 [17:23:25<22:41:12, 27.88s/it]
 41%|████▏     | 2072/5000 [17:23:57<23:39:09, 29.08s/it]
 41%|████▏     | 2073/5000 [17:24:24<23:09:52, 28.49s/it]
 41%|████▏     | 2074/5000 [17:24:50<22:41:09, 27.91s/it]
 42%|████▏     | 2075/5000 [17:25:18<22:39:46, 27.89s/it]
                                                         
 42%|████▏     | 2075/5000 [17:25:18<22:39:46, 27.89s/it]
 42%|████▏     | 2076/5000 [17:25:45<22:18:31, 27.47s/it]
 42%|████▏     | 2077/5000 [17:26:12<22:13:41, 27.38s/it]
 42%|████▏     | 2078/5000 [17:26:39<22:10:38, 27.32s/it]
 42%|████▏     | 2079/5000 [17:27:07<22:19:16, 27.51s/it]
 42%|████▏     | 2080/5000 [17:27:34<22:19:18, 27.52s/it]
 42%|████▏     | 2081/5000 [17:28:08<23:45:12, 29.30s/it]
 42%|████▏     | 2082/5000 [17:28:35<23:17:02, 28.73s/it]
 42%|████▏     | 2083/5000 [17:29:03<22:56:21, 28.31s/it]
 42%|████▏     | 2084/5000 [17:29:35<23:58:24, 29.60s/it]
 42%|████▏     | 2085/5000 [17:30:02<23:19:06, 28.80s/it]
 42%|████▏     | 2086/5000 [17:30:29<22:53:33, 28.28s/it]
 42%|████▏     | 2087/5000 [17:31:03<24:20:01, 30.07s/it]
 42%|████▏     | 2088/5000 [17:31:31<23:39:43, 29.25s/it]
 42%|████▏     | 2089/5000 [17:31:58<23:10:48, 28.67s/it]
 42%|████▏     | 2090/5000 [17:32:28<23:20:44, 28.88s/it]
 42%|████▏     | 2091/5000 [17:32:54<22:38:12, 28.01s/it]
 42%|████▏     | 2092/5000 [17:33:20<22:18:29, 27.62s/it]
 42%|████▏     | 2093/5000 [17:33:48<22:13:59, 27.53s/it]
 42%|████▏     | 2094/5000 [17:34:21<23:43:36, 29.39s/it]
 42%|████▏     | 2095/5000 [17:34:49<23:12:33, 28.76s/it]
 42%|████▏     | 2096/5000 [17:35:16<22:48:10, 28.27s/it]
 42%|████▏     | 2097/5000 [17:35:44<22:51:37, 28.35s/it]
 42%|████▏     | 2098/5000 [17:36:12<22:36:34, 28.05s/it]
 42%|████▏     | 2099/5000 [17:36:39<22:30:05, 27.92s/it]
 42%|████▏     | 2100/5000 [17:37:09<22:57:17, 28.50s/it]
                                                         
 42%|████▏     | 2100/5000 [17:37:09<22:57:17, 28.50s/it]
 42%|████▏     | 2101/5000 [17:37:36<22:41:41, 28.18s/it]
 42%|████▏     | 2102/5000 [17:38:04<22:26:09, 27.87s/it]
 42%|████▏     | 2103/5000 [17:38:38<24:00:46, 29.84s/it]
 42%|████▏     | 2104/5000 [17:39:05<23:23:18, 29.07s/it]
 42%|████▏     | 2105/5000 [17:39:33<22:58:44, 28.57s/it]
 42%|████▏     | 2106/5000 [17:40:03<23:17:05, 28.97s/it]
 42%|████▏     | 2107/5000 [17:40:30<22:57:08, 28.56s/it]
 42%|████▏     | 2108/5000 [17:40:58<22:41:59, 28.26s/it]
 42%|████▏     | 2109/5000 [17:41:27<23:00:59, 28.66s/it]
 42%|████▏     | 2110/5000 [17:41:54<22:37:18, 28.18s/it]
 42%|████▏     | 2111/5000 [17:42:22<22:22:29, 27.88s/it]
 42%|████▏     | 2112/5000 [17:42:55<23:39:12, 29.48s/it]
 42%|████▏     | 2113/5000 [17:43:22<23:02:57, 28.74s/it]
 42%|████▏     | 2114/5000 [17:43:50<22:47:42, 28.43s/it]
 42%|████▏     | 2115/5000 [17:44:19<22:54:55, 28.59s/it]
 42%|████▏     | 2116/5000 [17:44:46<22:37:04, 28.23s/it]
 42%|████▏     | 2117/5000 [17:45:13<22:25:17, 28.00s/it]
 42%|████▏     | 2118/5000 [17:45:43<22:48:33, 28.49s/it]
 42%|████▏     | 2119/5000 [17:46:10<22:25:05, 28.01s/it]
 42%|████▏     | 2120/5000 [17:46:37<22:08:51, 27.68s/it]
 42%|████▏     | 2121/5000 [17:47:10<23:31:57, 29.43s/it]
 42%|████▏     | 2122/5000 [17:47:24<19:48:48, 24.78s/it]
 42%|████▏     | 2123/5000 [17:47:35<16:29:15, 20.63s/it]
 42%|████▏     | 2124/5000 [17:47:46<14:08:07, 17.69s/it]
 42%|████▎     | 2125/5000 [17:47:57<12:30:01, 15.65s/it]
                                                         
 42%|████▎     | 2125/5000 [17:47:57<12:30:01, 15.65s/it]{'eval_loss': 0.5698409676551819, 'eval_wer': 55.900000000000006, 'eval_runtime': 2152.7247, 'eval_samples_per_second': 4.85, 'eval_steps_per_second': 0.076, 'epoch': 12.01}
+{'loss': 0.0045, 'learning_rate': 6.615555555555556e-06, 'epoch': 12.01}
+{'loss': 0.0032, 'learning_rate': 6.560000000000001e-06, 'epoch': 12.02}
+{'loss': 0.003, 'learning_rate': 6.504444444444446e-06, 'epoch': 12.02}
+{'loss': 0.0036, 'learning_rate': 6.448888888888889e-06, 'epoch': 12.03}
+{'loss': 0.0032, 'learning_rate': 6.393333333333334e-06, 'epoch': 12.03}
+
+
+
+
+
+
+
+
+
 43%|████▎     | 2126/5000 [17:49:36<32:26:44, 40.64s/it]
 43%|████▎     | 2127/5000 [17:50:08<30:24:20, 38.10s/it]
 43%|████▎     | 2128/5000 [17:50:47<30:41:33, 38.47s/it]
 43%|████▎     | 2129/5000 [17:51:16<28:15:19, 35.43s/it]
 43%|████▎     | 2130/5000 [17:51:43<26:19:49, 33.03s/it]
 43%|████▎     | 2131/5000 [17:52:12<25:18:31, 31.76s/it]
 43%|████▎     | 2132/5000 [17:52:44<25:22:16, 31.85s/it]
 43%|████▎     | 2133/5000 [17:53:19<26:10:26, 32.87s/it]
 43%|████▎     | 2134/5000 [17:53:48<25:16:20, 31.74s/it]
 43%|████▎     | 2135/5000 [17:54:15<24:05:49, 30.28s/it]
 43%|████▎     | 2136/5000 [17:54:43<23:32:58, 29.60s/it]
 43%|████▎     | 2137/5000 [17:55:14<23:48:53, 29.95s/it]
 43%|████▎     | 2138/5000 [17:55:41<23:10:27, 29.15s/it]
 43%|████▎     | 2139/5000 [17:56:14<23:56:21, 30.12s/it]
 43%|████▎     | 2140/5000 [17:56:44<23:53:26, 30.07s/it]
 43%|████▎     | 2141/5000 [17:57:12<23:21:47, 29.42s/it]
 43%|████▎     | 2142/5000 [17:57:40<23:06:21, 29.10s/it]
 43%|████▎     | 2143/5000 [17:58:06<22:29:01, 28.33s/it]
 43%|████▎     | 2144/5000 [17:58:34<22:23:25, 28.22s/it]
 43%|████▎     | 2145/5000 [17:59:03<22:22:03, 28.20s/it]
 43%|████▎     | 2146/5000 [17:59:30<22:07:21, 27.91s/it]
 43%|████▎     | 2147/5000 [17:59:58<22:08:54, 27.95s/it]
 43%|████▎     | 2148/5000 [18:00:26<22:14:28, 28.07s/it]
 43%|████▎     | 2149/5000 [18:00:53<22:00:05, 27.78s/it]
 43%|████▎     | 2150/5000 [18:01:21<22:05:14, 27.90s/it]
                                                         
 43%|████▎     | 2150/5000 [18:01:21<22:05:14, 27.90s/it]
 43%|████▎     | 2151/5000 [18:01:49<22:03:19, 27.87s/it]
 43%|████▎     | 2152/5000 [18:02:17<21:55:09, 27.71s/it]
 43%|████▎     | 2153/5000 [18:02:45<22:04:49, 27.92s/it]
 43%|████▎     | 2154/5000 [18:03:13<21:59:46, 27.82s/it]
 43%|████▎     | 2155/5000 [18:03:39<21:43:07, 27.48s/it]
 43%|████▎     | 2156/5000 [18:04:07<21:49:06, 27.62s/it]
 43%|████▎     | 2157/5000 [18:04:35<21:47:00, 27.58s/it]
 43%|████▎     | 2158/5000 [18:05:02<21:41:18, 27.47s/it]
 43%|████▎     | 2159/5000 [18:05:31<21:59:10, 27.86s/it]
 43%|████▎     | 2160/5000 [18:05:58<21:51:53, 27.72s/it]
 43%|████▎     | 2161/5000 [18:06:25<21:45:11, 27.58s/it]
 43%|████▎     | 2162/5000 [18:06:54<21:58:45, 27.88s/it]
 43%|████▎     | 2163/5000 [18:07:22<22:01:04, 27.94s/it]
 43%|████▎     | 2164/5000 [18:07:49<21:47:30, 27.66s/it]
 43%|████▎     | 2165/5000 [18:08:18<22:10:34, 28.16s/it]
 43%|████▎     | 2166/5000 [18:08:47<22:10:04, 28.16s/it]
 43%|████▎     | 2167/5000 [18:09:14<21:57:41, 27.91s/it]
 43%|████▎     | 2168/5000 [18:09:43<22:20:03, 28.39s/it]
 43%|████▎     | 2169/5000 [18:10:16<23:17:56, 29.63s/it]
 43%|████▎     | 2170/5000 [18:10:43<22:44:11, 28.92s/it]
 43%|████▎     | 2171/5000 [18:11:15<23:29:16, 29.89s/it]
 43%|████▎     | 2172/5000 [18:11:44<23:05:07, 29.39s/it]
 43%|████▎     | 2173/5000 [18:12:12<22:51:10, 29.10s/it]
 43%|████▎     | 2174/5000 [18:12:39<22:18:15, 28.41s/it]
 44%|████▎     | 2175/5000 [18:13:06<22:02:56, 28.10s/it]
                                                         
 44%|████▎     | 2175/5000 [18:13:06<22:02:56, 28.10s/it]
 44%|████▎     | 2176/5000 [18:13:38<23:00:22, 29.33s/it]
 44%|████▎     | 2177/5000 [18:14:06<22:34:23, 28.79s/it]
 44%|████▎     | 2178/5000 [18:14:34<22:31:19, 28.73s/it]
 44%|████▎     | 2179/5000 [18:15:04<22:40:00, 28.93s/it]
 44%|████▎     | 2180/5000 [18:15:31<22:16:17, 28.43s/it]
 44%|████▎     | 2181/5000 [18:15:59<22:09:26, 28.30s/it]
 44%|████▎     | 2182/5000 [18:16:27<22:04:27, 28.20s/it]
 44%|████▎     | 2183/5000 [18:16:54<21:52:10, 27.95s/it]
 44%|████▎     | 2184/5000 [18:17:27<23:01:11, 29.43s/it]
 44%|████▎     | 2185/5000 [18:17:55<22:35:52, 28.90s/it]
 44%|████▎     | 2186/5000 [18:18:22<22:15:20, 28.47s/it]
 44%|████▎     | 2187/5000 [18:18:52<22:23:49, 28.66s/it]
 44%|████▍     | 2188/5000 [18:19:19<22:03:09, 28.23s/it]
 44%|████▍     | 2189/5000 [18:19:46<21:46:31, 27.89s/it]
 44%|████▍     | 2190/5000 [18:20:14<21:52:50, 28.03s/it]
 44%|████▍     | 2191/5000 [18:20:42<21:51:54, 28.02s/it]
 44%|████▍     | 2192/5000 [18:21:10<21:47:45, 27.94s/it]
 44%|████▍     | 2193/5000 [18:21:38<21:52:51, 28.06s/it]
 44%|████▍     | 2194/5000 [18:22:06<21:43:47, 27.88s/it]
 44%|████▍     | 2195/5000 [18:22:33<21:35:11, 27.70s/it]
 44%|████▍     | 2196/5000 [18:23:01<21:40:04, 27.82s/it]
 44%|████▍     | 2197/5000 [18:23:33<22:29:39, 28.89s/it]
 44%|████▍     | 2198/5000 [18:24:00<22:05:54, 28.39s/it]
 44%|████▍     | 2199/5000 [18:24:28<22:05:44, 28.40s/it]
 44%|████▍     | 2200/5000 [18:24:56<21:59:48, 28.28s/it]
                                                         
 44%|████▍     | 2200/5000 [18:24:56<21:59:48, 28.28s/it]
 44%|████▍     | 2201/5000 [18:25:25<22:02:31, 28.35s/it]
 44%|████▍     | 2202/5000 [18:25:52<21:48:13, 28.05s/it]
 44%|████▍     | 2203/5000 [18:26:20<21:46:28, 28.03s/it]
 44%|████▍     | 2204/5000 [18:26:48<21:47:56, 28.07s/it]
 44%|████▍     | 2205/5000 [18:27:16<21:39:57, 27.91s/it]
 44%|████▍     | 2206/5000 [18:27:44<21:47:59, 28.09s/it]
 44%|████▍     | 2207/5000 [18:28:13<21:50:28, 28.15s/it]
 44%|████▍     | 2208/5000 [18:28:39<21:23:54, 27.59s/it]
 44%|████▍     | 2209/5000 [18:29:07<21:38:01, 27.90s/it]
 44%|████▍     | 2210/5000 [18:29:36<21:47:06, 28.11s/it]
 44%|████▍     | 2211/5000 [18:30:03<21:32:54, 27.81s/it]
 44%|████▍     | 2212/5000 [18:30:31<21:34:12, 27.85s/it]
 44%|████▍     | 2213/5000 [18:30:59<21:39:41, 27.98s/it]
 44%|████▍     | 2214/5000 [18:31:26<21:25:29, 27.68s/it]
 44%|████▍     | 2215/5000 [18:31:54<21:28:12, 27.75s/it]
 44%|████▍     | 2216/5000 [18:32:26<22:15:56, 28.79s/it]
 44%|████▍     | 2217/5000 [18:32:53<21:50:36, 28.26s/it]
 44%|████▍     | 2218/5000 [18:33:21<21:50:56, 28.27s/it]
 44%|████▍     | 2219/5000 [18:33:49<21:48:53, 28.24s/it]
 44%|████▍     | 2220/5000 [18:34:16<21:36:00, 27.97s/it]
 44%|████▍     | 2221/5000 [18:34:45<21:48:38, 28.25s/it]
 44%|████▍     | 2222/5000 [18:35:14<21:49:49, 28.29s/it]
 44%|████▍     | 2223/5000 [18:35:41<21:33:18, 27.94s/it]
 44%|████▍     | 2224/5000 [18:36:09<21:36:49, 28.03s/it]
 44%|████▍     | 2225/5000 [18:36:41<22:37:02, 29.34s/it]
                                                         
 44%|████▍     | 2225/5000 [18:36:41<22:37:02, 29.34s/it]
 45%|████▍     | 2226/5000 [18:37:10<22:21:28, 29.02s/it]
 45%|████▍     | 2227/5000 [18:37:36<21:47:48, 28.30s/it]
 45%|████▍     | 2228/5000 [18:38:03<21:28:33, 27.89s/it]
 45%|████▍     | 2229/5000 [18:38:31<21:25:48, 27.84s/it]
 45%|████▍     | 2230/5000 [18:38:58<21:15:27, 27.63s/it]
 45%|████▍     | 2231/5000 [18:39:25<21:07:28, 27.46s/it]
 45%|████▍     | 2232/5000 [18:39:54<21:21:07, 27.77s/it]
 45%|████▍     | 2233/5000 [18:40:21<21:10:34, 27.55s/it]
 45%|████▍     | 2234/5000 [18:40:47<20:53:34, 27.19s/it]
 45%|████▍     | 2235/5000 [18:41:16<21:18:57, 27.75s/it]
 45%|████▍     | 2236/5000 [18:41:44<21:13:36, 27.65s/it]
 45%|████▍     | 2237/5000 [18:42:11<21:07:32, 27.53s/it]
 45%|████▍     | 2238/5000 [18:42:39<21:22:27, 27.86s/it]
 45%|████▍     | 2239/5000 [18:43:07<21:14:19, 27.69s/it]
 45%|████▍     | 2240/5000 [18:43:34<21:09:55, 27.61s/it]
 45%|████▍     | 2241/5000 [18:44:02<21:13:58, 27.71s/it]
 45%|████▍     | 2242/5000 [18:44:31<21:25:00, 27.96s/it]
 45%|████▍     | 2243/5000 [18:44:58<21:12:43, 27.70s/it]
 45%|████▍     | 2244/5000 [18:45:25<21:03:52, 27.52s/it]
 45%|████▍     | 2245/5000 [18:45:55<21:35:46, 28.22s/it]
 45%|████▍     | 2246/5000 [18:46:20<21:00:52, 27.47s/it]
 45%|████▍     | 2247/5000 [18:46:47<20:53:28, 27.32s/it]
 45%|████▍     | 2248/5000 [18:47:15<20:55:42, 27.38s/it]
 45%|████▍     | 2249/5000 [18:47:42<20:47:07, 27.20s/it]
 45%|████▌     | 2250/5000 [18:48:12<21:33:57, 28.23s/it]
                                                         
 45%|████▌     | 2250/5000 [18:48:12<21:33:57, 28.23s/it]
 45%|████▌     | 2251/5000 [18:48:40<21:28:25, 28.12s/it]
 45%|████▌     | 2252/5000 [18:49:07<21:13:25, 27.80s/it]
 45%|████▌     | 2253/5000 [18:49:34<21:04:44, 27.62s/it]
 45%|████▌     | 2254/5000 [18:50:07<22:06:34, 28.99s/it]
 45%|████▌     | 2255/5000 [18:50:34<21:41:50, 28.46s/it]
 45%|████▌     | 2256/5000 [18:51:01<21:24:20, 28.08s/it]
 45%|████▌     | 2257/5000 [18:51:30<21:35:26, 28.34s/it]
 45%|████▌     | 2258/5000 [18:51:57<21:13:42, 27.87s/it]
 45%|████▌     | 2259/5000 [18:52:24<21:05:39, 27.70s/it]
 45%|████▌     | 2260/5000 [18:52:52<21:12:18, 27.86s/it]
 45%|████▌     | 2261/5000 [18:53:18<20:42:14, 27.21s/it]
 45%|████▌     | 2262/5000 [18:53:45<20:33:56, 27.04s/it]
 45%|████▌     | 2263/5000 [18:54:14<21:05:09, 27.73s/it]
 45%|████▌     | 2264/5000 [18:54:42<21:12:44, 27.91s/it]
 45%|████▌     | 2265/5000 [18:55:09<21:01:56, 27.68s/it]
 45%|████▌     | 2266/5000 [18:55:38<21:11:34, 27.91s/it]
 45%|████▌     | 2267/5000 [18:56:06<21:09:04, 27.86s/it]
 45%|████▌     | 2268/5000 [18:56:33<20:57:44, 27.62s/it]
 45%|████▌     | 2269/5000 [18:57:01<21:03:36, 27.76s/it]
 45%|████▌     | 2270/5000 [18:57:29<21:09:28, 27.90s/it]
 45%|████▌     | 2271/5000 [18:57:56<20:58:55, 27.68s/it]
 45%|████▌     | 2272/5000 [18:58:28<21:52:39, 28.87s/it]
 45%|████▌     | 2273/5000 [18:58:56<21:46:27, 28.75s/it]
 45%|████▌     | 2274/5000 [18:59:24<21:26:31, 28.32s/it]
 46%|████▌     | 2275/5000 [18:59:53<21:41:32, 28.66s/it]
                                                         
 46%|████▌     | 2275/5000 [18:59:53<21:41:32, 28.66s/it]
 46%|████▌     | 2276/5000 [19:00:20<21:20:33, 28.21s/it]
 46%|████▌     | 2277/5000 [19:00:47<21:07:06, 27.92s/it]
 46%|████▌     | 2278/5000 [19:01:16<21:19:39, 28.21s/it]
 46%|████▌     | 2279/5000 [19:01:44<21:08:32, 27.97s/it]
 46%|████▌     | 2280/5000 [19:02:10<20:49:57, 27.57s/it]
 46%|████▌     | 2281/5000 [19:02:39<20:58:59, 27.78s/it]
 46%|████▌     | 2282/5000 [19:03:07<21:01:20, 27.84s/it]
 46%|████▌     | 2283/5000 [19:03:34<20:52:27, 27.66s/it]
 46%|████▌     | 2284/5000 [19:04:03<21:11:37, 28.09s/it]
 46%|████▌     | 2285/5000 [19:04:25<19:51:32, 26.33s/it]
 46%|████▌     | 2286/5000 [19:04:36<16:23:03, 21.73s/it]
 46%|████▌     | 2287/5000 [19:04:47<13:54:15, 18.45s/it]
 46%|████▌     | 2288/5000 [19:04:58<12:11:22, 16.18s/it]
 46%|████▌     | 2289/5000 [19:05:06<10:17:47, 13.67s/it]{'loss': 0.0036, 'learning_rate': 6.3377777777777786e-06, 'epoch': 13.0}
+{'loss': 0.0038, 'learning_rate': 6.282222222222223e-06, 'epoch': 13.01}
+{'loss': 0.0031, 'learning_rate': 6.2266666666666675e-06, 'epoch': 13.01}
+{'loss': 0.0028, 'learning_rate': 6.171111111111112e-06, 'epoch': 13.02}
+{'loss': 0.0028, 'learning_rate': 6.1155555555555555e-06, 'epoch': 13.02}
+{'loss': 0.0029, 'learning_rate': 6.0600000000000004e-06, 'epoch': 13.03}
+
+
+
+
+
+
+
+
+
 46%|████▌     | 2290/5000 [19:06:52<31:19:22, 41.61s/it]
 46%|████▌     | 2291/5000 [19:07:21<28:16:18, 37.57s/it]
 46%|████▌     | 2292/5000 [19:07:48<26:00:56, 34.59s/it]
 46%|████▌     | 2293/5000 [19:08:16<24:33:37, 32.66s/it]
 46%|████▌     | 2294/5000 [19:08:44<23:27:26, 31.21s/it]
 46%|████▌     | 2295/5000 [19:09:13<22:51:04, 30.41s/it]
 46%|████▌     | 2296/5000 [19:09:40<22:08:27, 29.48s/it]
 46%|████▌     | 2297/5000 [19:10:09<21:55:38, 29.20s/it]
 46%|████▌     | 2298/5000 [19:10:36<21:26:57, 28.58s/it]
 46%|████▌     | 2299/5000 [19:11:04<21:25:22, 28.55s/it]
 46%|████▌     | 2300/5000 [19:11:32<21:11:12, 28.25s/it]
                                                         
 46%|████▌     | 2300/5000 [19:11:32<21:11:12, 28.25s/it]
 46%|████▌     | 2301/5000 [19:12:00<21:14:04, 28.32s/it]
 46%|████▌     | 2302/5000 [19:12:31<21:43:29, 28.99s/it]
 46%|████▌     | 2303/5000 [19:13:03<22:21:19, 29.84s/it]
 46%|████▌     | 2304/5000 [19:13:30<21:46:21, 29.07s/it]
 46%|████▌     | 2305/5000 [19:13:57<21:22:12, 28.55s/it]
 46%|████▌     | 2306/5000 [19:14:24<21:03:28, 28.14s/it]
 46%|████▌     | 2307/5000 [19:14:52<20:52:19, 27.90s/it]
 46%|████▌     | 2308/5000 [19:15:21<21:07:08, 28.24s/it]
 46%|████▌     | 2309/5000 [19:15:48<20:57:15, 28.03s/it]
 46%|████▌     | 2310/5000 [19:16:16<20:49:21, 27.87s/it]
 46%|████▌     | 2311/5000 [19:16:49<21:59:29, 29.44s/it]
 46%|████▌     | 2312/5000 [19:17:17<21:38:56, 28.99s/it]
 46%|████▋     | 2313/5000 [19:17:43<21:03:29, 28.21s/it]
 46%|████▋     | 2314/5000 [19:18:21<23:08:26, 31.01s/it]
 46%|████▋     | 2315/5000 [19:18:48<22:15:25, 29.84s/it]
 46%|████▋     | 2316/5000 [19:19:15<21:34:03, 28.93s/it]
 46%|████▋     | 2317/5000 [19:19:46<22:04:23, 29.62s/it]
 46%|████▋     | 2318/5000 [19:20:13<21:31:50, 28.90s/it]
 46%|████▋     | 2319/5000 [19:20:40<21:02:59, 28.27s/it]
 46%|████▋     | 2320/5000 [19:21:10<21:23:59, 28.75s/it]
 46%|████▋     | 2321/5000 [19:21:37<20:57:47, 28.17s/it]
 46%|████▋     | 2322/5000 [19:22:03<20:32:25, 27.61s/it]
 46%|████▋     | 2323/5000 [19:22:31<20:31:36, 27.60s/it]
 46%|████▋     | 2324/5000 [19:22:58<20:25:42, 27.48s/it]
 46%|████▋     | 2325/5000 [19:23:25<20:23:05, 27.43s/it]
                                                         
 46%|████▋     | 2325/5000 [19:23:25<20:23:05, 27.43s/it]
 47%|████▋     | 2326/5000 [19:23:54<20:42:06, 27.87s/it]
 47%|████▋     | 2327/5000 [19:24:21<20:34:48, 27.72s/it]
 47%|████▋     | 2328/5000 [19:24:49<20:31:22, 27.65s/it]
 47%|████▋     | 2329/5000 [19:25:21<21:28:54, 28.95s/it]
 47%|████▋     | 2330/5000 [19:25:48<21:05:39, 28.44s/it]
 47%|████▋     | 2331/5000 [19:26:18<21:20:39, 28.79s/it]
 47%|████▋     | 2332/5000 [19:26:44<20:51:56, 28.15s/it]
 47%|████▋     | 2333/5000 [19:27:12<20:43:56, 27.99s/it]
 47%|████▋     | 2334/5000 [19:27:45<21:44:26, 29.36s/it]
 47%|████▋     | 2335/5000 [19:28:12<21:18:56, 28.79s/it]
 47%|████▋     | 2336/5000 [19:28:40<21:10:42, 28.62s/it]
 47%|████▋     | 2337/5000 [19:29:09<21:06:05, 28.53s/it]
 47%|████▋     | 2338/5000 [19:29:36<20:48:45, 28.15s/it]
 47%|████▋     | 2339/5000 [19:30:04<20:54:28, 28.29s/it]
 47%|████▋     | 2340/5000 [19:30:32<20:48:11, 28.15s/it]
 47%|████▋     | 2341/5000 [19:31:00<20:37:48, 27.93s/it]
 47%|████▋     | 2342/5000 [19:31:28<20:46:28, 28.14s/it]
 47%|████▋     | 2343/5000 [19:31:56<20:38:38, 27.97s/it]
 47%|████▋     | 2344/5000 [19:32:23<20:31:45, 27.83s/it]
 47%|████▋     | 2345/5000 [19:32:52<20:42:14, 28.07s/it]
 47%|████▋     | 2346/5000 [19:33:19<20:30:33, 27.82s/it]
 47%|████▋     | 2347/5000 [19:33:46<20:21:52, 27.63s/it]
 47%|████▋     | 2348/5000 [19:34:16<20:45:23, 28.18s/it]
 47%|████▋     | 2349/5000 [19:34:43<20:34:03, 27.93s/it]
 47%|████▋     | 2350/5000 [19:35:11<20:27:14, 27.79s/it]
                                                         
 47%|████▋     | 2350/5000 [19:35:11<20:27:14, 27.79s/it]
 47%|████▋     | 2351/5000 [19:35:43<21:23:18, 29.07s/it]
 47%|████▋     | 2352/5000 [19:36:10<21:01:14, 28.58s/it]
 47%|████▋     | 2353/5000 [19:36:37<20:42:42, 28.17s/it]
 47%|████▋     | 2354/5000 [19:37:10<21:34:50, 29.36s/it]
 47%|████▋     | 2355/5000 [19:37:37<21:09:37, 28.80s/it]
 47%|████▋     | 2356/5000 [19:38:05<20:52:54, 28.43s/it]
 47%|████▋     | 2357/5000 [19:38:33<20:52:08, 28.43s/it]
 47%|████▋     | 2358/5000 [19:39:00<20:35:23, 28.06s/it]
 47%|████▋     | 2359/5000 [19:39:28<20:31:38, 27.98s/it]
 47%|████▋     | 2360/5000 [19:39:55<20:18:17, 27.69s/it]
 47%|████▋     | 2361/5000 [19:40:27<21:14:06, 28.97s/it]
 47%|████▋     | 2362/5000 [19:40:55<20:59:16, 28.64s/it]
 47%|████▋     | 2363/5000 [19:41:22<20:42:02, 28.26s/it]
 47%|████▋     | 2364/5000 [19:41:51<20:51:08, 28.48s/it]
 47%|████▋     | 2365/5000 [19:42:20<20:48:55, 28.44s/it]
 47%|████▋     | 2366/5000 [19:42:47<20:35:32, 28.14s/it]
 47%|████▋     | 2367/5000 [19:43:15<20:38:38, 28.23s/it]
 47%|████▋     | 2368/5000 [19:43:44<20:38:56, 28.24s/it]
 47%|████▋     | 2369/5000 [19:44:11<20:30:50, 28.07s/it]
 47%|████▋     | 2370/5000 [19:44:40<20:36:42, 28.21s/it]
 47%|████▋     | 2371/5000 [19:45:08<20:39:23, 28.29s/it]
 47%|████▋     | 2372/5000 [19:45:36<20:26:08, 27.99s/it]
 47%|████▋     | 2373/5000 [19:46:04<20:26:30, 28.01s/it]
 47%|████▋     | 2374/5000 [19:46:32<20:35:09, 28.22s/it]
 48%|████▊     | 2375/5000 [19:47:01<20:35:00, 28.23s/it]
                                                         
 48%|████▊     | 2375/5000 [19:47:01<20:35:00, 28.23s/it]
 48%|████▊     | 2376/5000 [19:47:27<20:15:55, 27.80s/it]
 48%|████▊     | 2377/5000 [19:47:56<20:22:17, 27.96s/it]
 48%|████▊     | 2378/5000 [19:48:27<21:02:34, 28.89s/it]
 48%|████▊     | 2379/5000 [19:48:54<20:42:57, 28.45s/it]
 48%|████▊     | 2380/5000 [19:49:22<20:34:55, 28.28s/it]
 48%|████▊     | 2381/5000 [19:49:51<20:39:57, 28.41s/it]
 48%|████▊     | 2382/5000 [19:50:18<20:28:21, 28.15s/it]
 48%|████▊     | 2383/5000 [19:50:46<20:22:34, 28.03s/it]
 48%|████▊     | 2384/5000 [19:51:15<20:29:11, 28.19s/it]
 48%|████▊     | 2385/5000 [19:51:41<19:58:44, 27.50s/it]
 48%|████▊     | 2386/5000 [19:52:09<20:03:30, 27.62s/it]
 48%|████▊     | 2387/5000 [19:52:36<20:01:06, 27.58s/it]
 48%|███��▊     | 2388/5000 [19:53:03<19:56:36, 27.49s/it]
 48%|████▊     | 2389/5000 [19:53:32<20:06:55, 27.73s/it]
 48%|████▊     | 2390/5000 [19:54:00<20:14:06, 27.91s/it]
 48%|████▊     | 2391/5000 [19:54:27<20:04:16, 27.70s/it]
 48%|████▊     | 2392/5000 [19:54:55<20:04:05, 27.70s/it]
 48%|████▊     | 2393/5000 [19:55:22<19:57:49, 27.57s/it]
 48%|████▊     | 2394/5000 [19:55:49<19:50:05, 27.40s/it]
 48%|████▊     | 2395/5000 [19:56:21<20:42:53, 28.63s/it]
 48%|████▊     | 2396/5000 [19:56:48<20:25:50, 28.25s/it]
 48%|████▊     | 2397/5000 [19:57:15<20:14:30, 27.99s/it]
 48%|████▊     | 2398/5000 [19:57:43<20:15:46, 28.03s/it]
 48%|████▊     | 2399/5000 [19:58:11<20:04:34, 27.79s/it]
 48%|████▊     | 2400/5000 [19:58:38<19:53:53, 27.55s/it]
                                                         
 48%|████▊     | 2400/5000 [19:58:38<19:53:53, 27.55s/it]
 48%|████▊     | 2401/5000 [19:59:06<20:05:32, 27.83s/it]
 48%|████▊     | 2402/5000 [19:59:34<20:07:50, 27.89s/it]
 48%|████▊     | 2403/5000 [20:00:02<19:59:22, 27.71s/it]
 48%|████▊     | 2404/5000 [20:00:29<20:01:59, 27.78s/it]
 48%|████▊     | 2405/5000 [20:00:56<19:48:46, 27.49s/it]
 48%|████▊     | 2406/5000 [20:01:28<20:38:11, 28.64s/it]
 48%|████▊     | 2407/5000 [20:01:54<20:10:17, 28.01s/it]
 48%|████▊     | 2408/5000 [20:02:23<20:19:31, 28.23s/it]
 48%|████▊     | 2409/5000 [20:02:49<19:56:53, 27.72s/it]
 48%|████▊     | 2410/5000 [20:03:16<19:45:51, 27.47s/it]
 48%|████▊     | 2411/5000 [20:03:45<19:59:36, 27.80s/it]
 48%|████▊     | 2412/5000 [20:04:12<19:52:02, 27.64s/it]
 48%|████▊     | 2413/5000 [20:04:40<19:48:48, 27.57s/it]
 48%|████▊     | 2414/5000 [20:05:08<19:55:47, 27.74s/it]
 48%|████▊     | 2415/5000 [20:05:35<19:46:15, 27.53s/it]
 48%|████▊     | 2416/5000 [20:06:02<19:36:15, 27.31s/it]
 48%|████▊     | 2417/5000 [20:06:32<20:20:40, 28.35s/it]
 48%|████▊     | 2418/5000 [20:07:00<20:08:19, 28.08s/it]
 48%|████▊     | 2419/5000 [20:07:27<19:55:48, 27.80s/it]
 48%|████▊     | 2420/5000 [20:07:54<19:48:56, 27.65s/it]
 48%|████▊     | 2421/5000 [20:08:23<20:00:37, 27.93s/it]
 48%|████▊     | 2422/5000 [20:08:50<19:51:40, 27.73s/it]
 48%|████▊     | 2423/5000 [20:09:17<19:45:02, 27.59s/it]
 48%|████▊     | 2424/5000 [20:09:46<19:53:06, 27.79s/it]
 48%|████▊     | 2425/5000 [20:10:13<19:49:52, 27.73s/it]
                                                         
 48%|████▊     | 2425/5000 [20:10:13<19:49:52, 27.73s/it]
 49%|████▊     | 2426/5000 [20:10:40<19:37:49, 27.46s/it]
 49%|████▊     | 2427/5000 [20:11:10<20:04:31, 28.09s/it]
 49%|████▊     | 2428/5000 [20:11:37<19:51:06, 27.79s/it]
 49%|████▊     | 2429/5000 [20:12:04<19:43:32, 27.62s/it]
 49%|████▊     | 2430/5000 [20:12:32<19:50:39, 27.80s/it]
 49%|████▊     | 2431/5000 [20:12:59<19:44:49, 27.67s/it]
 49%|████▊     | 2432/5000 [20:13:27<19:38:50, 27.54s/it]
 49%|████▊     | 2433/5000 [20:13:56<20:01:37, 28.09s/it]
 49%|████▊     | 2434/5000 [20:14:23<19:51:06, 27.85s/it]
 49%|████▊     | 2435/5000 [20:14:51<19:44:16, 27.70s/it]
 49%|████▊     | 2436/5000 [20:15:19<19:56:15, 27.99s/it]
 49%|████▊     | 2437/5000 [20:15:46<19:34:39, 27.50s/it]
 49%|████▉     | 2438/5000 [20:16:12<19:23:03, 27.24s/it]
 49%|████▉     | 2439/5000 [20:16:40<19:34:06, 27.51s/it]
 49%|████▉     | 2440/5000 [20:17:08<19:31:14, 27.45s/it]
 49%|████▉     | 2441/5000 [20:17:35<19:29:47, 27.43s/it]
 49%|████▉     | 2442/5000 [20:18:04<19:42:37, 27.74s/it]
 49%|████▉     | 2443/5000 [20:18:31<19:37:55, 27.64s/it]
 49%|████▉     | 2444/5000 [20:18:58<19:33:46, 27.55s/it]
 49%|████▉     | 2445/5000 [20:19:27<19:44:28, 27.82s/it]
 49%|████▉     | 2446/5000 [20:19:54<19:38:18, 27.68s/it]
 49%|████▉     | 2447/5000 [20:20:22<19:35:44, 27.63s/it]
 49%|████▉     | 2448/5000 [20:20:51<19:56:19, 28.13s/it]
 49%|████▉     | 2449/5000 [20:21:05<16:53:12, 23.83s/it]
 49%|████▉     | 2450/5000 [20:21:16<14:09:22, 19.99s/it]
                                                         
 49%|████▉     | 2450/5000 [20:21:16<14:09:22, 19.99s/it]
 49%|████▉     | 2451/5000 [20:21:27<12:12:47, 17.25s/it]
 49%|████▉     | 2452/5000 [20:21:37<10:49:43, 15.30s/it]{'loss': 0.0025, 'learning_rate': 6.004444444444445e-06, 'epoch': 14.0}
+{'loss': 0.0024, 'learning_rate': 5.948888888888889e-06, 'epoch': 14.01}
+{'loss': 0.0022, 'learning_rate': 5.893333333333334e-06, 'epoch': 14.01}
+{'loss': 0.0019, 'learning_rate': 5.837777777777777e-06, 'epoch': 14.02}
+{'loss': 0.0018, 'learning_rate': 5.782222222222222e-06, 'epoch': 14.02}
+{'loss': 0.0021, 'learning_rate': 5.726666666666667e-06, 'epoch': 14.03}
+{'loss': 0.0016, 'learning_rate': 5.671111111111112e-06, 'epoch': 14.03}
+
+
+
+
+
+
+
+
+
 49%|████▉     | 2453/5000 [20:23:03<25:48:07, 36.47s/it]
 49%|████▉     | 2454/5000 [20:23:31<23:58:31, 33.90s/it]
 49%|████▉     | 2455/5000 [20:23:59<22:38:11, 32.02s/it]
 49%|████▉     | 2456/5000 [20:24:27<21:47:23, 30.83s/it]
 49%|████▉     | 2457/5000 [20:24:54<21:00:56, 29.75s/it]
 49%|████▉     | 2458/5000 [20:25:22<20:34:42, 29.14s/it]
 49%|████▉     | 2459/5000 [20:25:50<20:21:51, 28.85s/it]
 49%|████▉     | 2460/5000 [20:26:18<20:13:51, 28.67s/it]
 49%|████▉     | 2461/5000 [20:26:47<20:17:52, 28.78s/it]
 49%|████▉     | 2462/5000 [20:27:15<20:02:28, 28.43s/it]
 49%|████▉     | 2463/5000 [20:27:43<19:54:36, 28.25s/it]
 49%|████▉     | 2464/5000 [20:28:11<19:57:13, 28.33s/it]
 49%|████▉     | 2465/5000 [20:28:39<19:44:19, 28.03s/it]
 49%|████▉     | 2466/5000 [20:29:06<19:42:23, 28.00s/it]
 49%|████▉     | 2467/5000 [20:29:34<19:40:14, 27.96s/it]
 49%|████▉     | 2468/5000 [20:30:03<19:52:35, 28.26s/it]
 49%|████▉     | 2469/5000 [20:30:31<19:46:50, 28.14s/it]
 49%|████▉     | 2470/5000 [20:30:58<19:31:33, 27.78s/it]
 49%|████▉     | 2471/5000 [20:31:27<19:41:07, 28.02s/it]
 49%|████▉     | 2472/5000 [20:31:55<19:44:35, 28.12s/it]
 49%|████▉     | 2473/5000 [20:32:22<19:35:14, 27.90s/it]
 49%|████▉     | 2474/5000 [20:32:51<19:38:28, 27.99s/it]
 50%|████▉     | 2475/5000 [20:33:19<19:42:39, 28.10s/it]
                                                         
 50%|████▉     | 2475/5000 [20:33:19<19:42:39, 28.10s/it]
 50%|████▉     | 2476/5000 [20:33:46<19:29:45, 27.81s/it]
 50%|████▉     | 2477/5000 [20:34:14<19:35:50, 27.96s/it]
 50%|████▉     | 2478/5000 [20:34:42<19:29:58, 27.83s/it]
 50%|████▉     | 2479/5000 [20:35:09<19:22:56, 27.68s/it]
 50%|████▉     | 2480/5000 [20:35:37<19:24:05, 27.72s/it]
 50%|████▉     | 2481/5000 [20:36:05<19:29:59, 27.87s/it]
 50%|████▉     | 2482/5000 [20:36:32<19:19:49, 27.64s/it]
 50%|████▉     | 2483/5000 [20:37:00<19:24:27, 27.76s/it]
 50%|████▉     | 2484/5000 [20:37:28<19:26:36, 27.82s/it]
 50%|████▉     | 2485/5000 [20:37:56<19:22:27, 27.73s/it]
 50%|████▉     | 2486/5000 [20:38:25<19:35:51, 28.06s/it]
 50%|████▉     | 2487/5000 [20:38:52<19:25:35, 27.83s/it]
 50%|████▉     | 2488/5000 [20:39:20<19:20:03, 27.71s/it]
 50%|████▉     | 2489/5000 [20:39:47<19:21:41, 27.76s/it]
 50%|████▉     | 2490/5000 [20:40:15<19:13:26, 27.57s/it]
 50%|████▉     | 2491/5000 [20:40:42<19:08:29, 27.46s/it]
 50%|████▉     | 2492/5000 [20:41:10<19:17:09, 27.68s/it]
 50%|████▉     | 2493/5000 [20:41:38<19:26:46, 27.92s/it]
 50%|████▉     | 2494/5000 [20:42:06<19:19:39, 27.76s/it]
 50%|████▉     | 2495/5000 [20:42:34<19:24:25, 27.89s/it]
 50%|████▉     | 2496/5000 [20:43:02<19:28:12, 27.99s/it]
 50%|████▉     | 2497/5000 [20:43:30<19:20:00, 27.81s/it]
 50%|████▉     | 2498/5000 [20:43:58<19:24:05, 27.92s/it]
 50%|████▉     | 2499/5000 [20:44:26<19:28:55, 28.04s/it]
 50%|█████     | 2500/5000 [20:44:54<19:26:57, 28.01s/it]
                                                         
 50%|█████     | 2500/5000 [20:44:54<19:26:57, 28.01s/it]
 50%|█████     | 2501/5000 [20:45:21<19:18:54, 27.83s/it]
 50%|█████     | 2502/5000 [20:45:49<19:12:51, 27.69s/it]
 50%|█████     | 2503/5000 [20:46:18<19:27:44, 28.06s/it]
 50%|█████     | 2504/5000 [20:46:45<19:20:52, 27.91s/it]
 50%|█████     | 2505/5000 [20:47:14<19:27:27, 28.08s/it]
 50%|█████     | 2506/5000 [20:47:42<19:33:14, 28.23s/it]
 50%|█████     | 2507/5000 [20:48:10<19:22:44, 27.98s/it]
 50%|█████     | 2508/5000 [20:48:43<20:26:12, 29.52s/it]
 50%|█████     | 2509/5000 [20:49:10<20:01:37, 28.94s/it]
 50%|█████     | 2510/5000 [20:49:38<19:45:00, 28.55s/it]
 50%|█████     | 2511/5000 [20:50:12<20:46:43, 30.05s/it]
 50%|█████     | 2512/5000 [20:50:39<20:11:18, 29.21s/it]
 50%|█████     | 2513/5000 [20:51:06<19:48:25, 28.67s/it]
 50%|█████     | 2514/5000 [20:51:34<19:36:37, 28.40s/it]
 50%|█████     | 2515/5000 [20:52:01<19:23:28, 28.09s/it]
 50%|█████     | 2516/5000 [20:52:29<19:15:24, 27.91s/it]
 50%|█████     | 2517/5000 [20:52:57<19:16:38, 27.95s/it]
 50%|█████     | 2518/5000 [20:53:25<19:13:39, 27.89s/it]
 50%|█████     | 2519/5000 [20:53:52<19:06:33, 27.73s/it]
 50%|█████     | 2520/5000 [20:54:20<19:08:34, 27.79s/it]
 50%|█████     | 2521/5000 [20:54:48<19:14:00, 27.93s/it]
 50%|█████     | 2522/5000 [20:55:16<19:05:29, 27.74s/it]
 50%|█████     | 2523/5000 [20:55:44<19:08:49, 27.83s/it]
 50%|█████     | 2524/5000 [20:56:12<19:11:49, 27.91s/it]
 50%|█████     | 2525/5000 [20:56:39<19:02:25, 27.69s/it]
                                                         
 50%|█████     | 2525/5000 [20:56:39<19:02:25, 27.69s/it]
 51%|█████     | 2526/5000 [20:57:08<19:15:07, 28.01s/it]
 51%|█████     | 2527/5000 [20:57:35<19:03:57, 27.75s/it]
 51%|█████     | 2528/5000 [20:58:03<19:04:42, 27.78s/it]
 51%|█████     | 2529/5000 [20:58:30<18:59:48, 27.68s/it]
 51%|█████     | 2530/5000 [20:58:57<18:54:29, 27.56s/it]
 51%|█████     | 2531/5000 [20:59:26<19:05:29, 27.84s/it]
 51%|█████     | 2532/5000 [20:59:53<19:02:33, 27.78s/it]
 51%|█████     | 2533/5000 [21:00:26<19:54:56, 29.06s/it]
 51%|█████     | 2534/5000 [21:00:54<19:40:59, 28.73s/it]
 51%|█████     | 2535/5000 [21:01:21<19:23:51, 28.33s/it]
 51%|█████     | 2536/5000 [21:01:49<19:22:17, 28.30s/it]
 51%|█████     | 2537/5000 [21:02:17<19:14:32, 28.13s/it]
 51%|█████     | 2538/5000 [21:02:43<18:50:31, 27.55s/it]
 51%|█████     | 2539/5000 [21:03:10<18:44:55, 27.43s/it]
 51%|█████     | 2540/5000 [21:03:38<18:52:46, 27.63s/it]
 51%|█████     | 2541/5000 [21:04:05<18:39:33, 27.32s/it]
 51%|█████     | 2542/5000 [21:04:32<18:40:42, 27.36s/it]
 51%|█████     | 2543/5000 [21:05:00<18:45:18, 27.48s/it]
 51%|█████     | 2544/5000 [21:05:27<18:41:19, 27.39s/it]
 51%|█████     | 2545/5000 [21:05:57<19:15:20, 28.24s/it]
 51%|█████     | 2546/5000 [21:06:26<19:14:31, 28.23s/it]
 51%|█████     | 2547/5000 [21:06:52<18:52:26, 27.70s/it]
 51%|█████     | 2548/5000 [21:07:20<18:51:45, 27.69s/it]
 51%|█████     | 2549/5000 [21:07:48<18:50:45, 27.68s/it]
 51%|█████     | 2550/5000 [21:08:15<18:45:33, 27.56s/it]
                                                         
 51%|█████     | 2550/5000 [21:08:15<18:45:33, 27.56s/it]
 51%|█████     | 2551/5000 [21:08:45<19:19:18, 28.40s/it]
 51%|█████     | 2552/5000 [21:09:14<19:18:52, 28.40s/it]
 51%|█████     | 2553/5000 [21:09:42<19:16:42, 28.36s/it]
 51%|█████     | 2554/5000 [21:10:10<19:08:51, 28.18s/it]
 51%|█████     | 2555/5000 [21:10:38<19:11:14, 28.25s/it]
 51%|█████     | 2556/5000 [21:11:06<19:03:22, 28.07s/it]
 51%|█████     | 2557/5000 [21:11:33<18:55:46, 27.89s/it]
 51%|█████     | 2558/5000 [21:12:00<18:47:57, 27.71s/it]
 51%|█████     | 2559/5000 [21:12:29<18:55:23, 27.91s/it]
 51%|█████     | 2560/5000 [21:12:56<18:44:59, 27.66s/it]
 51%|█████     | 2561/5000 [21:13:23<18:42:06, 27.60s/it]
 51%|█████     | 2562/5000 [21:13:52<18:53:46, 27.90s/it]
 51%|█████▏    | 2563/5000 [21:14:19<18:45:16, 27.70s/it]
 51%|█████▏    | 2564/5000 [21:14:46<18:38:46, 27.56s/it]
 51%|█████▏    | 2565/5000 [21:15:15<18:56:55, 28.01s/it]
 51%|█████▏    | 2566/5000 [21:15:43<18:49:50, 27.85s/it]
 51%|█████▏    | 2567/5000 [21:16:11<18:53:56, 27.96s/it]
 51%|█████▏    | 2568/5000 [21:16:40<18:59:02, 28.10s/it]
 51%|█████▏    | 2569/5000 [21:17:07<18:53:18, 27.97s/it]
 51%|█████▏    | 2570/5000 [21:17:35<18:45:31, 27.79s/it]
 51%|█████▏    | 2571/5000 [21:18:02<18:41:33, 27.70s/it]
 51%|█████▏    | 2572/5000 [21:18:30<18:40:45, 27.70s/it]
 51%|█████▏    | 2573/5000 [21:18:57<18:37:15, 27.62s/it]
 51%|█████▏    | 2574/5000 [21:19:27<19:00:41, 28.21s/it]
 52%|█████▏    | 2575/5000 [21:19:53<18:40:44, 27.73s/it]
                                                         
 52%|█████▏    | 2575/5000 [21:19:53<18:40:44, 27.73s/it]
 52%|█████▏    | 2576/5000 [21:20:20<18:23:25, 27.31s/it]
 52%|█████▏    | 2577/5000 [21:20:47<18:23:25, 27.32s/it]
 52%|█████▏    | 2578/5000 [21:21:15<18:27:12, 27.43s/it]
 52%|█████▏    | 2579/5000 [21:21:42<18:27:53, 27.46s/it]
 52%|█████▏    | 2580/5000 [21:22:10<18:24:39, 27.39s/it]
 52%|█████▏    | 2581/5000 [21:22:38<18:36:51, 27.70s/it]
 52%|█████▏    | 2582/5000 [21:23:05<18:31:11, 27.57s/it]
 52%|█████▏    | 2583/5000 [21:23:32<18:22:59, 27.38s/it]
 52%|█████▏    | 2584/5000 [21:24:01<18:36:49, 27.74s/it]
 52%|█████▏    | 2585/5000 [21:24:28<18:30:07, 27.58s/it]
 52%|█████▏    | 2586/5000 [21:24:55<18:27:47, 27.53s/it]
 52%|█████▏    | 2587/5000 [21:25:25<18:46:04, 28.00s/it]
 52%|█████▏    | 2588/5000 [21:25:52<18:40:46, 27.88s/it]
 52%|█████▏    | 2589/5000 [21:26:19<18:29:06, 27.60s/it]
 52%|█████▏    | 2590/5000 [21:26:47<18:36:05, 27.79s/it]
 52%|█████▏    | 2591/5000 [21:27:15<18:39:51, 27.89s/it]
 52%|█████▏    | 2592/5000 [21:27:43<18:30:39, 27.67s/it]
 52%|█████▏    | 2593/5000 [21:28:11<18:34:07, 27.77s/it]
 52%|█████▏    | 2594/5000 [21:28:39<18:36:46, 27.85s/it]
 52%|█████▏    | 2595/5000 [21:29:06<18:31:16, 27.72s/it]
 52%|█████▏    | 2596/5000 [21:29:34<18:38:15, 27.91s/it]
 52%|█████▏    | 2597/5000 [21:30:03<18:45:47, 28.11s/it]
 52%|█████▏    | 2598/5000 [21:30:30<18:38:27, 27.94s/it]
 52%|█████▏    | 2599/5000 [21:30:59<18:50:17, 28.25s/it]
 52%|█████▏    | 2600/5000 [21:31:27<18:47:00, 28.18s/it]
                                                         
 52%|█████▏    | 2600/5000 [21:31:27<18:47:00, 28.18s/it]
 52%|█████▏    | 2601/5000 [21:31:55<18:38:47, 27.98s/it]
 52%|█████▏    | 2602/5000 [21:32:24<18:51:26, 28.31s/it]
 52%|█████▏    | 2603/5000 [21:32:52<18:40:26, 28.05s/it]
 52%|█████▏    | 2604/5000 [21:33:19<18:29:38, 27.79s/it]
 52%|█████▏    | 2605/5000 [21:33:48<18:42:02, 28.11s/it]
 52%|█████▏    | 2606/5000 [21:34:15<18:33:04, 27.90s/it]
 52%|█████▏    | 2607/5000 [21:34:42<18:27:30, 27.77s/it]
 52%|█████▏    | 2608/5000 [21:35:10<18:29:49, 27.84s/it]
 52%|█████▏    | 2609/5000 [21:35:38<18:22:58, 27.68s/it]
 52%|█████▏    | 2610/5000 [21:36:05<18:16:19, 27.52s/it]
 52%|█████▏    | 2611/5000 [21:36:33<18:24:03, 27.73s/it]
 52%|█████▏    | 2612/5000 [21:36:55<17:17:28, 26.07s/it]
 52%|█████▏    | 2613/5000 [21:37:06<14:15:29, 21.50s/it]
 52%|█████▏    | 2614/5000 [21:37:17<12:09:15, 18.34s/it]
 52%|█████▏    | 2615/5000 [21:37:28<10:38:42, 16.07s/it]
 52%|█████▏    | 2616/5000 [21:37:36<8:59:53, 13.59s/it] {'loss': 0.0016, 'learning_rate': 5.615555555555556e-06, 'epoch': 15.0}
+{'loss': 0.0018, 'learning_rate': 5.560000000000001e-06, 'epoch': 15.01}
+{'loss': 0.0014, 'learning_rate': 5.504444444444444e-06, 'epoch': 15.01}
+{'loss': 0.0013, 'learning_rate': 5.448888888888889e-06, 'epoch': 15.02}
+{'loss': 0.0014, 'learning_rate': 5.393333333333334e-06, 'epoch': 15.02}
+{'loss': 0.0015, 'learning_rate': 5.337777777777779e-06, 'epoch': 15.03}
+
+
+
+
+
+
+
+
+
 52%|█████▏    | 2617/5000 [21:39:14<25:47:54, 38.97s/it]
 52%|█████▏    | 2618/5000 [21:39:42<23:36:56, 35.69s/it]
 52%|█████▏    | 2619/5000 [21:40:09<21:57:36, 33.20s/it]
 52%|█████▏    | 2620/5000 [21:40:38<20:57:40, 31.71s/it]
 52%|█████▏    | 2621/5000 [21:41:06<20:18:41, 30.74s/it]
 52%|█████▏    | 2622/5000 [21:41:35<19:57:13, 30.21s/it]
 52%|█████▏    | 2623/5000 [21:42:02<19:23:14, 29.36s/it]
 52%|█████▏    | 2624/5000 [21:42:30<18:57:49, 28.73s/it]
 52%|█████▎    | 2625/5000 [21:42:58<18:52:14, 28.60s/it]
                                                         
 52%|█████▎    | 2625/5000 [21:42:58<18:52:14, 28.60s/it]
 53%|█████▎    | 2626/5000 [21:43:25<18:32:54, 28.13s/it]
 53%|█████▎    | 2627/5000 [21:43:52<18:22:30, 27.88s/it]
 53%|█████▎    | 2628/5000 [21:44:20<18:22:34, 27.89s/it]
 53%|█████▎    | 2629/5000 [21:44:48<18:23:17, 27.92s/it]
 53%|█████▎    | 2630/5000 [21:45:16<18:26:33, 28.01s/it]
 53%|█████▎    | 2631/5000 [21:45:44<18:16:32, 27.77s/it]
 53%|█████▎    | 2632/5000 [21:46:12<18:24:34, 27.99s/it]
 53%|█████▎    | 2633/5000 [21:46:40<18:24:53, 28.01s/it]
 53%|█████▎    | 2634/5000 [21:47:08<18:18:20, 27.85s/it]
 53%|█████▎    | 2635/5000 [21:47:35<18:16:29, 27.82s/it]
 53%|█████▎    | 2636/5000 [21:48:03<18:12:14, 27.72s/it]
 53%|█████▎    | 2637/5000 [21:48:30<18:00:45, 27.44s/it]
 53%|█████▎    | 2638/5000 [21:48:58<18:10:46, 27.71s/it]
 53%|█████▎    | 2639/5000 [21:49:26<18:09:21, 27.68s/it]
 53%|█████▎    | 2640/5000 [21:49:53<18:05:51, 27.61s/it]
 53%|█████▎    | 2641/5000 [21:50:21<18:14:13, 27.83s/it]
 53%|████���▎    | 2642/5000 [21:50:48<18:03:48, 27.58s/it]
 53%|█████▎    | 2643/5000 [21:51:15<17:56:46, 27.41s/it]
 53%|█████▎    | 2644/5000 [21:51:44<18:15:57, 27.91s/it]
 53%|█████▎    | 2645/5000 [21:52:12<18:10:52, 27.79s/it]
 53%|█████▎    | 2646/5000 [21:52:39<18:02:27, 27.59s/it]
 53%|█████▎    | 2647/5000 [21:53:08<18:15:09, 27.93s/it]
 53%|█████▎    | 2648/5000 [21:53:35<18:09:39, 27.80s/it]
 53%|█████▎    | 2649/5000 [21:54:03<18:04:16, 27.67s/it]
 53%|█████▎    | 2650/5000 [21:54:31<18:09:06, 27.81s/it]
                                                         
 53%|█████▎    | 2650/5000 [21:54:31<18:09:06, 27.81s/it]
 53%|█████▎    | 2651/5000 [21:54:58<18:02:32, 27.65s/it]
 53%|█████▎    | 2652/5000 [21:55:26<17:59:12, 27.58s/it]
 53%|█████▎    | 2653/5000 [21:55:54<18:12:54, 27.94s/it]
 53%|█████▎    | 2654/5000 [21:56:22<18:06:22, 27.78s/it]
 53%|█████▎    | 2655/5000 [21:56:49<18:02:35, 27.70s/it]
 53%|█████▎    | 2656/5000 [21:57:18<18:13:41, 28.00s/it]
 53%|█████▎    | 2657/5000 [21:57:45<18:07:59, 27.86s/it]
 53%|█████▎    | 2658/5000 [21:58:14<18:10:10, 27.93s/it]
 53%|█████▎    | 2659/5000 [21:58:42<18:15:31, 28.08s/it]
 53%|█████▎    | 2660/5000 [21:59:09<18:08:13, 27.90s/it]
 53%|█████▎    | 2661/5000 [21:59:37<18:05:35, 27.85s/it]
 53%|█████▎    | 2662/5000 [22:00:03<17:42:53, 27.28s/it]
 53%|█████▎    | 2663/5000 [22:00:32<17:56:27, 27.64s/it]
 53%|█████▎    | 2664/5000 [22:01:00<18:01:05, 27.77s/it]
 53%|█████▎    | 2665/5000 [22:01:27<17:55:17, 27.63s/it]
 53%|█████▎    | 2666/5000 [22:01:56<18:06:41, 27.94s/it]
 53%|█████▎    | 2667/5000 [22:02:24<18:12:40, 28.10s/it]
 53%|█████▎    | 2668/5000 [22:02:52<18:05:46, 27.94s/it]
 53%|█████▎    | 2669/5000 [22:03:25<19:06:25, 29.51s/it]
 53%|█████▎    | 2670/5000 [22:03:51<18:30:48, 28.60s/it]
 53%|█████▎    | 2671/5000 [22:04:19<18:15:12, 28.22s/it]
 53%|█████▎    | 2672/5000 [22:04:48<18:27:02, 28.53s/it]
 53%|█████▎    | 2673/5000 [22:05:15<18:13:58, 28.21s/it]
 53%|█████▎    | 2674/5000 [22:05:42<17:55:47, 27.75s/it]
 54%|█████▎    | 2675/5000 [22:06:11<18:03:49, 27.97s/it]
                                                         
 54%|█████▎    | 2675/5000 [22:06:11<18:03:49, 27.97s/it]
 54%|█████▎    | 2676/5000 [22:06:38<17:54:29, 27.74s/it]
 54%|█████▎    | 2677/5000 [22:07:05<17:51:56, 27.69s/it]
 54%|█████▎    | 2678/5000 [22:07:33<17:54:56, 27.78s/it]
 54%|█████▎    | 2679/5000 [22:08:00<17:42:23, 27.46s/it]
 54%|█████▎    | 2680/5000 [22:08:27<17:40:15, 27.42s/it]
 54%|█████▎    | 2681/5000 [22:08:57<18:01:31, 27.98s/it]
 54%|█████▎    | 2682/5000 [22:09:24<17:51:57, 27.75s/it]
 54%|█████▎    | 2683/5000 [22:09:52<18:01:06, 28.00s/it]
 54%|█████▎    | 2684/5000 [22:10:20<17:55:32, 27.86s/it]
 54%|█████▎    | 2685/5000 [22:10:47<17:45:38, 27.62s/it]
 54%|█████▎    | 2686/5000 [22:11:15<17:51:39, 27.79s/it]
 54%|█████▎    | 2687/5000 [22:11:42<17:36:42, 27.41s/it]
 54%|█████▍    | 2688/5000 [22:12:09<17:37:33, 27.45s/it]
 54%|█████▍    | 2689/5000 [22:12:37<17:38:04, 27.47s/it]
 54%|█████▍    | 2690/5000 [22:13:03<17:27:02, 27.20s/it]
 54%|█████▍    | 2691/5000 [22:13:31<17:33:06, 27.37s/it]
 54%|█████▍    | 2692/5000 [22:13:59<17:34:02, 27.40s/it]
 54%|█████▍    | 2693/5000 [22:14:26<17:28:21, 27.27s/it]
 54%|█████▍    | 2694/5000 [22:14:53<17:31:35, 27.36s/it]
 54%|█████▍    | 2695/5000 [22:15:22<17:50:52, 27.88s/it]
 54%|█████▍    | 2696/5000 [22:15:49<17:40:21, 27.61s/it]
 54%|█████▍    | 2697/5000 [22:16:18<17:48:36, 27.84s/it]
 54%|█████▍    | 2698/5000 [22:16:45<17:39:58, 27.63s/it]
 54%|█████▍    | 2699/5000 [22:17:12<17:36:47, 27.56s/it]
 54%|█████▍    | 2700/5000 [22:17:40<17:42:07, 27.71s/it]
                                                         
 54%|█████▍    | 2700/5000 [22:17:40<17:42:07, 27.71s/it]
 54%|█████▍    | 2701/5000 [22:18:08<17:44:02, 27.77s/it]
 54%|█████▍    | 2702/5000 [22:18:36<17:46:31, 27.85s/it]
 54%|█████▍    | 2703/5000 [22:19:04<17:41:58, 27.74s/it]
 54%|█████▍    | 2704/5000 [22:19:32<17:44:46, 27.83s/it]
 54%|█████▍    | 2705/5000 [22:19:59<17:44:02, 27.82s/it]
 54%|█████▍    | 2706/5000 [22:20:27<17:38:51, 27.69s/it]
 54%|█████▍    | 2707/5000 [22:20:54<17:37:54, 27.68s/it]
 54%|█████▍    | 2708/5000 [22:21:22<17:36:11, 27.65s/it]
 54%|█████▍    | 2709/5000 [22:21:49<17:28:49, 27.47s/it]
 54%|█████▍    | 2710/5000 [22:22:17<17:32:37, 27.58s/it]
 54%|█████▍    | 2711/5000 [22:22:45<17:34:52, 27.65s/it]
 54%|█████▍    | 2712/5000 [22:23:12<17:33:05, 27.62s/it]
 54%|█████▍    | 2713/5000 [22:23:41<17:42:49, 27.88s/it]
 54%|█████▍    | 2714/5000 [22:24:10<17:58:08, 28.30s/it]
 54%|█████▍    | 2715/5000 [22:24:37<17:46:02, 27.99s/it]
 54%|█████▍    | 2716/5000 [22:25:06<17:48:45, 28.08s/it]
 54%|█████▍    | 2717/5000 [22:25:39<18:52:59, 29.78s/it]
 54%|█████▍    | 2718/5000 [22:26:06<18:22:00, 28.97s/it]
 54%|█████▍    | 2719/5000 [22:26:35<18:11:33, 28.71s/it]
 54%|█████▍    | 2720/5000 [22:27:01<17:46:54, 28.08s/it]
 54%|█████▍    | 2721/5000 [22:27:28<17:33:59, 27.75s/it]
 54%|█████▍    | 2722/5000 [22:27:57<17:44:07, 28.03s/it]
 54%|█████▍    | 2723/5000 [22:28:24<17:36:21, 27.84s/it]
 54%|█████▍    | 2724/5000 [22:28:51<17:28:07, 27.63s/it]
 55%|█████▍    | 2725/5000 [22:29:20<17:34:08, 27.80s/it]
                                                         
 55%|█████▍    | 2725/5000 [22:29:20<17:34:08, 27.80s/it]
 55%|█████▍    | 2726/5000 [22:29:47<17:29:13, 27.68s/it]
 55%|█████▍    | 2727/5000 [22:30:14<17:21:55, 27.50s/it]
 55%|█████▍    | 2728/5000 [22:30:42<17:27:45, 27.67s/it]
 55%|█████▍    | 2729/5000 [22:31:10<17:27:07, 27.67s/it]
 55%|█████▍    | 2730/5000 [22:31:37<17:23:23, 27.58s/it]
 55%|█████▍    | 2731/5000 [22:32:06<17:32:15, 27.83s/it]
 55%|█████▍    | 2732/5000 [22:32:33<17:30:10, 27.78s/it]
 55%|█████▍    | 2733/5000 [22:33:01<17:34:51, 27.92s/it]
 55%|█████▍    | 2734/5000 [22:33:29<17:27:53, 27.75s/it]
 55%|█████▍    | 2735/5000 [22:33:57<17:33:23, 27.90s/it]
 55%|█████▍    | 2736/5000 [22:34:24<17:23:05, 27.64s/it]
 55%|█████▍    | 2737/5000 [22:34:51<17:16:45, 27.49s/it]
 55%|█████▍    | 2738/5000 [22:35:19<17:20:14, 27.59s/it]
 55%|█████▍    | 2739/5000 [22:35:46<17:08:56, 27.30s/it]
 55%|█████▍    | 2740/5000 [22:36:12<16:59:32, 27.07s/it]
 55%|█████▍    | 2741/5000 [22:36:44<17:54:56, 28.55s/it]
 55%|█████▍    | 2742/5000 [22:37:12<17:41:19, 28.20s/it]
 55%|█████▍    | 2743/5000 [22:37:39<17:26:13, 27.81s/it]
 55%|█████▍    | 2744/5000 [22:38:07<17:33:33, 28.02s/it]
 55%|█████▍    | 2745/5000 [22:38:34<17:18:32, 27.63s/it]
 55%|█████▍    | 2746/5000 [22:39:00<17:01:21, 27.19s/it]
 55%|█████▍    | 2747/5000 [22:39:26<16:51:39, 26.94s/it]
 55%|█████▍    | 2748/5000 [22:39:55<17:14:19, 27.56s/it]
 55%|█████▍    | 2749/5000 [22:40:23<17:10:47, 27.48s/it]
 55%|█████▌    | 2750/5000 [22:40:49<17:01:49, 27.25s/it]
                                                         
 55%|█████▌    | 2750/5000 [22:40:49<17:01:49, 27.25s/it]
 55%|█████▌    | 2751/5000 [22:41:18<17:21:37, 27.79s/it]
 55%|█████▌    | 2752/5000 [22:41:45<17:12:06, 27.55s/it]
 55%|█████▌    | 2753/5000 [22:42:12<17:02:49, 27.31s/it]
 55%|█████▌    | 2754/5000 [22:42:42<17:28:40, 28.01s/it]
 55%|█████▌    | 2755/5000 [22:43:09<17:21:40, 27.84s/it]
 55%|█████▌    | 2756/5000 [22:43:36<17:14:22, 27.66s/it]
 55%|█████▌    | 2757/5000 [22:44:04<17:18:31, 27.78s/it]
 55%|█████▌    | 2758/5000 [22:44:32<17:13:01, 27.65s/it]
 55%|█████▌    | 2759/5000 [22:44:59<17:05:57, 27.47s/it]
 55%|█████▌    | 2760/5000 [22:45:27<17:17:01, 27.78s/it]
 55%|█████▌    | 2761/5000 [22:45:55<17:13:12, 27.69s/it]
 55%|█████▌    | 2762/5000 [22:46:22<17:09:30, 27.60s/it]
 55%|█████▌    | 2763/5000 [22:46:51<17:22:59, 27.97s/it]
 55%|█████▌    | 2764/5000 [22:47:18<17:14:58, 27.77s/it]
 55%|█████▌    | 2765/5000 [22:47:46<17:09:36, 27.64s/it]
 55%|█████▌    | 2766/5000 [22:48:15<17:28:03, 28.15s/it]
 55%|█████▌    | 2767/5000 [22:48:42<17:17:42, 27.88s/it]
 55%|█████▌    | 2768/5000 [22:49:09<17:08:48, 27.66s/it]
 55%|█████▌    | 2769/5000 [22:49:37<17:13:17, 27.79s/it]
 55%|█████▌    | 2770/5000 [22:50:04<17:03:38, 27.54s/it]
 55%|█████▌    | 2771/5000 [22:50:31<16:49:48, 27.18s/it]
 55%|█████▌    | 2772/5000 [22:50:59<17:00:03, 27.47s/it]
 55%|█████▌    | 2773/5000 [22:51:26<16:56:21, 27.38s/it]
 55%|█████▌    | 2774/5000 [22:51:54<16:57:03, 27.41s/it]
 56%|█████▌    | 2775/5000 [22:52:22<17:07:17, 27.70s/it]
                                                         
 56%|█████▌    | 2775/5000 [22:52:22<17:07:17, 27.70s/it]
 56%|█████▌    | 2776/5000 [22:52:36<14:33:28, 23.57s/it]
 56%|█████▌    | 2777/5000 [22:52:47<12:11:00, 19.73s/it]
 56%|█████▌    | 2778/5000 [22:52:58<10:33:05, 17.09s/it]
 56%|█████▌    | 2779/5000 [22:53:08<9:21:54, 15.18s/it] {'loss': 0.0013, 'learning_rate': 5.282222222222223e-06, 'epoch': 16.0}
+{'loss': 0.0012, 'learning_rate': 5.226666666666667e-06, 'epoch': 16.01}
+{'loss': 0.0013, 'learning_rate': 5.171111111111111e-06, 'epoch': 16.01}
+{'loss': 0.001, 'learning_rate': 5.115555555555556e-06, 'epoch': 16.02}
+{'loss': 0.0011, 'learning_rate': 5.060000000000001e-06, 'epoch': 16.02}
+{'loss': 0.0011, 'learning_rate': 5.004444444444445e-06, 'epoch': 16.03}
+{'loss': 0.0009, 'learning_rate': 4.94888888888889e-06, 'epoch': 16.03}
+
+
+
+
+
+
+
+
+
 56%|█████▌    | 2780/5000 [22:54:36<22:44:03, 36.87s/it]
 56%|█████▌    | 2781/5000 [22:55:04<21:02:19, 34.13s/it]
 56%|█████▌    | 2782/5000 [22:55:31<19:47:19, 32.12s/it]
 56%|█████▌    | 2783/5000 [22:55:59<19:06:51, 31.04s/it]
 56%|█████▌    | 2784/5000 [22:56:27<18:23:20, 29.87s/it]
 56%|█████▌    | 2785/5000 [22:56:54<17:56:35, 29.16s/it]
 56%|█████▌    | 2786/5000 [22:57:22<17:36:24, 28.63s/it]
 56%|█████▌    | 2787/5000 [22:57:50<17:32:53, 28.55s/it]
 56%|█████▌    | 2788/5000 [22:58:18<17:22:06, 28.27s/it]
 56%|█████▌    | 2789/5000 [22:58:44<16:59:44, 27.67s/it]
 56%|█████▌    | 2790/5000 [22:59:12<17:05:03, 27.83s/it]
 56%|█████▌    | 2791/5000 [22:59:40<17:08:10, 27.93s/it]
 56%|█████▌    | 2792/5000 [23:00:08<17:02:12, 27.78s/it]
 56%|█████▌    | 2793/5000 [23:00:35<17:02:59, 27.81s/it]
 56%|█████▌    | 2794/5000 [23:01:10<18:16:18, 29.82s/it]
 56%|█████▌    | 2795/5000 [23:01:38<17:51:19, 29.15s/it]
 56%|█████▌    | 2796/5000 [23:02:05<17:28:57, 28.56s/it]
 56%|█████▌    | 2797/5000 [23:02:32<17:12:50, 28.13s/it]
 56%|█████▌    | 2798/5000 [23:03:00<17:16:18, 28.24s/it]
 56%|█████▌    | 2799/5000 [23:03:28<17:07:04, 28.00s/it]
 56%|█████▌    | 2800/5000 [23:03:55<17:01:12, 27.85s/it]
                                                         
 56%|█████▌    | 2800/5000 [23:03:55<17:01:12, 27.85s/it]
 56%|█████▌    | 2801/5000 [23:04:24<17:05:13, 27.97s/it]
 56%|█████▌    | 2802/5000 [23:04:51<16:59:14, 27.82s/it]
 56%|█████▌    | 2803/5000 [23:05:18<16:53:06, 27.67s/it]
 56%|█████▌    | 2804/5000 [23:05:47<16:59:15, 27.85s/it]
 56%|█████▌    | 2805/5000 [23:06:14<16:57:04, 27.80s/it]
 56%|█████▌    | 2806/5000 [23:06:41<16:49:54, 27.62s/it]
 56%|█████▌    | 2807/5000 [23:07:09<16:51:46, 27.68s/it]
 56%|█████▌    | 2808/5000 [23:07:37<16:52:54, 27.73s/it]
 56%|█████▌    | 2809/5000 [23:08:04<16:47:12, 27.58s/it]
 56%|█████▌    | 2810/5000 [23:08:33<16:55:41, 27.83s/it]
 56%|█████▌    | 2811/5000 [23:09:01<16:57:45, 27.90s/it]
 56%|█████▌    | 2812/5000 [23:09:28<16:52:27, 27.76s/it]
 56%|█████▋    | 2813/5000 [23:09:56<16:53:44, 27.81s/it]
 56%|█████▋    | 2814/5000 [23:10:23<16:42:35, 27.52s/it]
 56%|█████▋    | 2815/5000 [23:10:51<16:41:52, 27.51s/it]
 56%|█████▋    | 2816/5000 [23:11:18<16:45:57, 27.64s/it]
 56%|█████▋    | 2817/5000 [23:11:46<16:48:25, 27.72s/it]
 56%|█████▋    | 2818/5000 [23:12:14<16:47:14, 27.70s/it]
 56%|█████▋    | 2819/5000 [23:12:42<16:54:54, 27.92s/it]
 56%|█████▋    | 2820/5000 [23:13:10<16:52:25, 27.86s/it]
 56%|█████▋    | 2821/5000 [23:13:37<16:37:44, 27.47s/it]
 56%|█████▋    | 2822/5000 [23:14:05<16:48:12, 27.77s/it]
 56%|█████▋    | 2823/5000 [23:14:33<16:46:23, 27.74s/it]
 56%|█████▋    | 2824/5000 [23:15:00<16:41:20, 27.61s/it]
 56%|█████▋    | 2825/5000 [23:15:28<16:46:36, 27.77s/it]
                                                         
 56%|█████▋    | 2825/5000 [23:15:28<16:46:36, 27.77s/it]
 57%|█████▋    | 2826/5000 [23:15:57<16:50:55, 27.90s/it]
 57%|█████▋    | 2827/5000 [23:16:25<16:53:54, 28.00s/it]
 57%|█████▋    | 2828/5000 [23:16:52<16:46:06, 27.79s/it]
 57%|█████▋    | 2829/5000 [23:17:20<16:42:23, 27.70s/it]
 57%|█████▋    | 2830/5000 [23:17:48<16:53:11, 28.01s/it]
 57%|█████▋    | 2831/5000 [23:18:16<16:47:03, 27.86s/it]
 57%|█████▋    | 2832/5000 [23:18:44<16:47:58, 27.90s/it]
 57%|█████▋    | 2833/5000 [23:19:12<16:52:58, 28.05s/it]
 57%|█████▋    | 2834/5000 [23:19:39<16:41:15, 27.74s/it]
 57%|█████▋    | 2835/5000 [23:20:07<16:43:16, 27.80s/it]
 57%|█████▋    | 2836/5000 [23:20:35<16:43:57, 27.84s/it]
 57%|█████▋    | 2837/5000 [23:21:03<16:39:17, 27.72s/it]
 57%|█████▋    | 2838/5000 [23:21:30<16:39:05, 27.73s/it]
 57%|█████▋    | 2839/5000 [23:21:57<16:24:10, 27.33s/it]
 57%|█████▋    | 2840/5000 [23:22:24<16:23:26, 27.32s/it]
 57%|█████▋    | 2841/5000 [23:22:52<16:29:12, 27.49s/it]
 57%|█████▋    | 2842/5000 [23:23:19<16:25:41, 27.41s/it]
 57%|█████▋    | 2843/5000 [23:23:46<16:24:17, 27.38s/it]
 57%|█████▋    | 2844/5000 [23:24:16<16:46:29, 28.01s/it]
 57%|█████▋    | 2845/5000 [23:24:44<16:50:06, 28.12s/it]
 57%|█████▋    | 2846/5000 [23:25:12<16:43:36, 27.96s/it]
 57%|█████▋    | 2847/5000 [23:25:40<16:41:43, 27.92s/it]
 57%|█████▋    | 2848/5000 [23:26:07<16:38:25, 27.84s/it]
 57%|█████▋    | 2849/5000 [23:26:34<16:24:17, 27.46s/it]
 57%|█████▋    | 2850/5000 [23:27:02<16:33:55, 27.74s/it]
                                                         
 57%|█████▋    | 2850/5000 [23:27:02<16:33:55, 27.74s/it]
 57%|█████▋    | 2851/5000 [23:27:31<16:41:09, 27.95s/it]
 57%|█████▋    | 2852/5000 [23:27:58<16:34:40, 27.78s/it]
 57%|█████▋    | 2853/5000 [23:28:26<16:30:38, 27.68s/it]
 57%|█████▋    | 2854/5000 [23:28:54<16:33:21, 27.77s/it]
 57%|█████▋    | 2855/5000 [23:29:21<16:34:44, 27.82s/it]
 57%|█████▋    | 2856/5000 [23:29:49<16:31:38, 27.75s/it]
 57%|█████▋    | 2857/5000 [23:30:17<16:32:53, 27.80s/it]
 57%|█████▋    | 2858/5000 [23:30:45<16:32:09, 27.79s/it]
 57%|█████▋    | 2859/5000 [23:31:12<16:25:49, 27.63s/it]
 57%|█████▋    | 2860/5000 [23:31:43<17:02:27, 28.67s/it]
 57%|█████▋    | 2861/5000 [23:32:11<16:56:58, 28.53s/it]
 57%|█████▋    | 2862/5000 [23:32:39<16:46:50, 28.26s/it]
 57%|█████▋    | 2863/5000 [23:33:07<16:43:52, 28.19s/it]
 57%|█████▋    | 2864/5000 [23:33:35<16:43:56, 28.20s/it]
 57%|█████▋    | 2865/5000 [23:34:03<16:35:22, 27.97s/it]
 57%|█████▋    | 2866/5000 [23:34:31<16:36:19, 28.01s/it]
 57%|█████▋    | 2867/5000 [23:34:58<16:31:58, 27.90s/it]
 57%|█████▋    | 2868/5000 [23:35:26<16:26:57, 27.78s/it]
 57%|█████▋    | 2869/5000 [23:35:54<16:34:45, 28.01s/it]
 57%|█████▋    | 2870/5000 [23:36:23<16:36:57, 28.08s/it]
 57%|█████▋    | 2871/5000 [23:36:50<16:31:34, 27.94s/it]
 57%|█████▋    | 2872/5000 [23:37:18<16:32:29, 27.98s/it]
 57%|█████▋    | 2873/5000 [23:37:47<16:36:56, 28.12s/it]
 57%|█████▋    | 2874/5000 [23:38:14<16:29:40, 27.93s/it]
 57%|█████▊    | 2875/5000 [23:38:42<16:22:42, 27.75s/it]
                                                         
 57%|█████▊    | 2875/5000 [23:38:42<16:22:42, 27.75s/it]
 58%|█████▊    | 2876/5000 [23:39:13<16:55:59, 28.70s/it]
 58%|█████▊    | 2877/5000 [23:39:40<16:37:28, 28.19s/it]
 58%|█████▊    | 2878/5000 [23:40:07<16:29:12, 27.97s/it]
 58%|█████▊    | 2879/5000 [23:40:35<16:32:48, 28.09s/it]
 58%|█████▊    | 2880/5000 [23:41:03<16:31:54, 28.07s/it]
 58%|█████▊    | 2881/5000 [23:41:30<16:20:19, 27.76s/it]
 58%|█████▊    | 2882/5000 [23:41:59<16:25:02, 27.90s/it]
 58%|█████▊    | 2883/5000 [23:42:27<16:25:38, 27.94s/it]
 58%|█████▊    | 2884/5000 [23:42:53<16:11:31, 27.55s/it]
 58%|█████▊    | 2885/5000 [23:43:20<15:57:00, 27.15s/it]
 58%|█████▊    | 2886/5000 [23:43:47<15:54:48, 27.10s/it]
 58%|█████▊    | 2887/5000 [23:44:14<15:56:46, 27.17s/it]
 58%|█████▊    | 2888/5000 [23:44:41<15:55:00, 27.13s/it]
 58%|█████▊    | 2889/5000 [23:45:10<16:11:35, 27.62s/it]
 58%|█████▊    | 2890/5000 [23:45:37<16:07:21, 27.51s/it]
 58%|█████▊    | 2891/5000 [23:46:04<16:03:26, 27.41s/it]
 58%|█████▊    | 2892/5000 [23:46:32<16:11:04, 27.64s/it]
 58%|█████▊    | 2893/5000 [23:47:00<16:07:11, 27.54s/it]
 58%|█████▊    | 2894/5000 [23:47:27<16:07:07, 27.55s/it]
 58%|█████▊    | 2895/5000 [23:47:55<16:13:39, 27.75s/it]
 58%|█████▊    | 2896/5000 [23:48:23<16:13:17, 27.76s/it]
 58%|█████▊    | 2897/5000 [23:48:51<16:11:19, 27.71s/it]
 58%|█████▊    | 2898/5000 [23:49:19<16:13:03, 27.78s/it]
 58%|█████▊    | 2899/5000 [23:49:46<16:09:24, 27.68s/it]
 58%|█████▊    | 2900/5000 [23:50:14<16:06:03, 27.60s/it]
                                                         
 58%|█████▊    | 2900/5000 [23:50:14<16:06:03, 27.60s/it]
 58%|█████▊    | 2901/5000 [23:50:41<16:06:17, 27.62s/it]
 58%|█████▊    | 2902/5000 [23:51:09<16:03:01, 27.54s/it]
 58%|█████▊    | 2903/5000 [23:51:36<16:00:53, 27.49s/it]
 58%|█████▊    | 2904/5000 [23:52:04<16:04:36, 27.61s/it]
 58%|█████▊    | 2905/5000 [23:52:32<16:06:50, 27.69s/it]
 58%|█████▊    | 2906/5000 [23:52:59<16:02:53, 27.59s/it]
 58%|█████▊    | 2907/5000 [23:53:25<15:40:11, 26.95s/it]
 58%|█████▊    | 2908/5000 [23:53:53<15:59:35, 27.52s/it]
 58%|█████▊    | 2909/5000 [23:54:21<15:58:44, 27.51s/it]
 58%|█████▊    | 2910/5000 [23:54:48<15:54:50, 27.41s/it]
 58%|█████▊    | 2911/5000 [23:55:18<16:22:29, 28.22s/it]
 58%|█████▊    | 2912/5000 [23:55:46<16:13:18, 27.97s/it]
 58%|█████▊    | 2913/5000 [23:56:13<16:06:19, 27.78s/it]
 58%|█████▊    | 2914/5000 [23:56:41<16:08:47, 27.87s/it]
 58%|█████▊    | 2915/5000 [23:57:08<16:03:34, 27.73s/it]
 58%|█████▊    | 2916/5000 [23:57:36<16:00:12, 27.65s/it]
 58%|█████▊    | 2917/5000 [23:58:04<16:06:29, 27.84s/it]
 58%|█████▊    | 2918/5000 [23:58:33<16:13:11, 28.05s/it]
 58%|█████▊    | 2919/5000 [23:59:00<16:01:22, 27.72s/it]
 58%|█████▊    | 2920/5000 [23:59:28<16:08:58, 27.95s/it]
 58%|█████▊    | 2921/5000 [23:59:55<15:53:29, 27.52s/it]
 58%|█████▊    | 2922/5000 [24:00:22<15:50:26, 27.44s/it]
 58%|█████▊    | 2923/5000 [24:00:50<15:55:02, 27.59s/it]
 58%|█████▊    | 2924/5000 [24:01:17<15:52:41, 27.53s/it]
 58%|█████▊    | 2925/5000 [24:01:44<15:49:02, 27.44s/it]
                                                         
 58%|█████▊    | 2925/5000 [24:01:44<15:49:02, 27.44s/it]
 59%|█████▊    | 2926/5000 [24:02:13<16:02:45, 27.85s/it]
 59%|█████▊    | 2927/5000 [24:02:41<15:59:22, 27.77s/it]
 59%|█████▊    | 2928/5000 [24:03:08<15:55:21, 27.66s/it]
 59%|█████▊    | 2929/5000 [24:03:36<15:55:53, 27.69s/it]
 59%|█████▊    | 2930/5000 [24:04:03<15:50:33, 27.55s/it]
 59%|█████▊    | 2931/5000 [24:04:31<15:48:03, 27.49s/it]
 59%|█████▊    | 2932/5000 [24:04:59<15:55:23, 27.72s/it]
 59%|█████▊    | 2933/5000 [24:05:26<15:51:21, 27.62s/it]
 59%|█████▊    | 2934/5000 [24:05:53<15:46:11, 27.48s/it]
 59%|█████▊    | 2935/5000 [24:06:21<15:51:34, 27.65s/it]
 59%|█████▊    | 2936/5000 [24:06:49<15:52:51, 27.70s/it]
 59%|█████▊    | 2937/5000 [24:07:16<15:47:59, 27.57s/it]
 59%|█████▉    | 2938/5000 [24:07:45<15:52:57, 27.73s/it]
 59%|█████▉    | 2939/5000 [24:08:07<14:53:51, 26.02s/it]
 59%|█████▉    | 2940/5000 [24:08:17<12:16:55, 21.46s/it]
 59%|█████▉    | 2941/5000 [24:08:28<10:29:07, 18.33s/it]
 59%|█████▉    | 2942/5000 [24:08:39<9:11:35, 16.08s/it] 
 59%|█████▉    | 2943/5000 [24:08:47<7:45:50, 13.59s/it]{'loss': 0.0011, 'learning_rate': 4.893333333333334e-06, 'epoch': 17.0}
+{'loss': 0.0011, 'learning_rate': 4.837777777777778e-06, 'epoch': 17.01}
+{'loss': 0.0009, 'learning_rate': 4.7822222222222226e-06, 'epoch': 17.01}
+{'loss': 0.0009, 'learning_rate': 4.7266666666666674e-06, 'epoch': 17.02}
+{'loss': 0.0009, 'learning_rate': 4.6711111111111115e-06, 'epoch': 17.02}
+{'loss': 0.0009, 'learning_rate': 4.6155555555555555e-06, 'epoch': 17.03}
+
+
+
+
+
+
+
+
+
+
 59%|█████▉    | 2944/5000 [24:10:26<22:24:19, 39.23s/it]
 59%|█████▉    | 2945/5000 [24:10:53<20:20:21, 35.63s/it]
 59%|█████▉    | 2946/5000 [24:11:21<18:55:07, 33.16s/it]
 59%|█████▉    | 2947/5000 [24:11:49<18:05:56, 31.74s/it]
 59%|█████▉    | 2948/5000 [24:12:17<17:25:15, 30.56s/it]
 59%|█████▉    | 2949/5000 [24:12:45<17:03:19, 29.94s/it]
 59%|█████▉    | 2950/5000 [24:13:13<16:37:08, 29.18s/it]
                                                         
 59%|█████▉    | 2950/5000 [24:13:13<16:37:08, 29.18s/it]
 59%|█████▉    | 2951/5000 [24:13:41<16:22:52, 28.78s/it]
 59%|█████▉    | 2952/5000 [24:14:09<16:14:10, 28.54s/it]
 59%|█████▉    | 2953/5000 [24:14:36<16:01:31, 28.18s/it]
 59%|█████▉    | 2954/5000 [24:15:03<15:50:42, 27.88s/it]
 59%|█████▉    | 2955/5000 [24:15:30<15:40:25, 27.59s/it]
 59%|█████▉    | 2956/5000 [24:15:58<15:40:50, 27.62s/it]
 59%|█████▉    | 2957/5000 [24:16:26<15:46:11, 27.79s/it]
 59%|█████▉    | 2958/5000 [24:16:54<15:44:29, 27.75s/it]
 59%|█████▉    | 2959/5000 [24:17:21<15:44:37, 27.77s/it]
 59%|█████▉    | 2960/5000 [24:17:50<15:50:25, 27.95s/it]
 59%|█████▉    | 2961/5000 [24:18:17<15:42:40, 27.74s/it]
 59%|█████▉    | 2962/5000 [24:18:45<15:47:59, 27.91s/it]
 59%|█████▉    | 2963/5000 [24:19:13<15:39:17, 27.67s/it]
 59%|█████▉    | 2964/5000 [24:19:40<15:37:02, 27.61s/it]
 59%|█████▉    | 2965/5000 [24:20:09<15:47:26, 27.93s/it]
 59%|█████▉    | 2966/5000 [24:20:36<15:44:20, 27.86s/it]
 59%|█████▉    | 2967/5000 [24:21:04<15:39:09, 27.72s/it]
 59%|█████▉    | 2968/5000 [24:21:32<15:40:53, 27.78s/it]
 59%|█████▉    | 2969/5000 [24:21:59<15:33:36, 27.58s/it]
 59%|█████▉    | 2970/5000 [24:22:26<15:32:07, 27.55s/it]
 59%|█████▉    | 2971/5000 [24:22:54<15:30:41, 27.52s/it]
 59%|█████▉    | 2972/5000 [24:23:21<15:29:53, 27.51s/it]
 59%|█████▉    | 2973/5000 [24:23:49<15:29:53, 27.53s/it]
 59%|█████▉    | 2974/5000 [24:24:18<15:46:57, 28.04s/it]
 60%|█████▉    | 2975/5000 [24:24:45<15:39:20, 27.83s/it]
                                                         
 60%|█████▉    | 2975/5000 [24:24:45<15:39:20, 27.83s/it]
 60%|█████▉    | 2976/5000 [24:25:13<15:37:13, 27.78s/it]
 60%|█████▉    | 2977/5000 [24:25:43<16:00:17, 28.48s/it]
 60%|█████▉    | 2978/5000 [24:26:10<15:47:31, 28.12s/it]
 60%|█████▉    | 2979/5000 [24:26:37<15:33:50, 27.72s/it]
 60%|█████▉    | 2980/5000 [24:27:04<15:25:22, 27.49s/it]
 60%|█████▉    | 2981/5000 [24:27:31<15:16:13, 27.23s/it]
 60%|█████▉    | 2982/5000 [24:27:58<15:16:24, 27.25s/it]
 60%|█████▉    | 2983/5000 [24:28:27<15:33:26, 27.77s/it]
 60%|█████▉    | 2984/5000 [24:28:54<15:29:30, 27.66s/it]
 60%|█████▉    | 2985/5000 [24:29:23<15:34:14, 27.82s/it]
 60%|█████▉    | 2986/5000 [24:29:52<15:45:11, 28.16s/it]
 60%|█████▉    | 2987/5000 [24:30:19<15:38:44, 27.98s/it]
 60%|█████▉    | 2988/5000 [24:30:47<15:37:10, 27.95s/it]
 60%|█████▉    | 2989/5000 [24:31:14<15:24:20, 27.58s/it]
 60%|█████▉    | 2990/5000 [24:31:43<15:36:29, 27.96s/it]
 60%|█████▉    | 2991/5000 [24:32:11<15:40:06, 28.08s/it]
 60%|█████▉    | 2992/5000 [24:32:38<15:32:54, 27.88s/it]
 60%|█████▉    | 2993/5000 [24:33:07<15:37:14, 28.02s/it]
 60%|█████▉    | 2994/5000 [24:33:35<15:40:28, 28.13s/it]
 60%|█████▉    | 2995/5000 [24:34:03<15:33:19, 27.93s/it]
 60%|█████▉    | 2996/5000 [24:34:31<15:36:31, 28.04s/it]
 60%|█████▉    | 2997/5000 [24:34:58<15:29:03, 27.83s/it]
 60%|█████▉    | 2998/5000 [24:35:26<15:24:05, 27.69s/it]
 60%|█████▉    | 2999/5000 [24:35:54<15:31:47, 27.94s/it]
 60%|██████    | 3000/5000 [24:36:21<15:21:38, 27.65s/it]
                                                         
 60%|██████    | 3000/5000 [24:36:21<15:21:38, 27.65s/it][INFO|trainer.py:3138] 2023-05-08 11:10:32,150 >> ***** Running Evaluation *****
+[INFO|trainer.py:3142] 2023-05-08 11:10:32,150 >>   Num examples: Unknown
+[INFO|trainer.py:3143] 2023-05-08 11:10:32,150 >>   Batch size = 64
+{'loss': 0.0009, 'learning_rate': 4.56e-06, 'epoch': 18.0}
+{'loss': 0.0009, 'learning_rate': 4.504444444444444e-06, 'epoch': 18.01}
+{'loss': 0.0012, 'learning_rate': 4.448888888888889e-06, 'epoch': 18.01}
+
+
+
+[INFO|trainer_utils.py:693] 2023-05-08 11:10:42,418 >> The following columns in the evaluation set don't have a corresponding argument in `WhisperForConditionalGeneration.forward` and have been ignored: input_length. If input_length are not expected by `WhisperForConditionalGeneration.forward`,  you can safely ignore this message.
+
                                                         
 60%|██████    | 3000/5000 [25:12:09<15:21:38, 27.65s/it][INFO|trainer.py:2877] 2023-05-08 11:46:20,051 >> Saving model checkpoint to ./checkpoint-3000
+[INFO|configuration_utils.py:458] 2023-05-08 11:46:20,057 >> Configuration saved in ./checkpoint-3000/config.json
+[INFO|configuration_utils.py:364] 2023-05-08 11:46:20,060 >> Configuration saved in ./checkpoint-3000/generation_config.json
+[INFO|modeling_utils.py:1855] 2023-05-08 11:46:22,975 >> Model weights saved in ./checkpoint-3000/pytorch_model.bin
+[INFO|feature_extraction_utils.py:369] 2023-05-08 11:46:22,981 >> Feature extractor saved in ./checkpoint-3000/preprocessor_config.json
+[INFO|feature_extraction_utils.py:369] 2023-05-08 11:46:33,415 >> Feature extractor saved in ./preprocessor_config.json

wandb/run-20230507_103405-9zf5xxpu/files/output.log

@@ -2145,3 +2145,1071 @@ Reading metadata...: 10440it [00:00, 30685.17it/s]
 [INFO|configuration_utils.py:364] 2023-05-08 03:22:58,560 >> Configuration saved in ./checkpoint-2000/generation_config.json
 [INFO|modeling_utils.py:1855] 2023-05-08 03:23:01,997 >> Model weights saved in ./checkpoint-2000/pytorch_model.bin
 [INFO|feature_extraction_utils.py:369] 2023-05-08 03:23:02,003 >> Feature extractor saved in ./checkpoint-2000/preprocessor_config.json
+[INFO|feature_extraction_utils.py:369] 2023-05-08 03:23:12,574 >> Feature extractor saved in ./preprocessor_config.json
+/home/local/QCRI/dizham/miniconda3/envs/whisper/lib/python3.9/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
+  warnings.warn('Was asked to gather along dimension 0, but all '
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 40%|████      | 2024/5000 [17:01:13<23:31:21, 28.45s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 41%|████      | 2049/5000 [17:13:06<23:57:15, 29.22s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 41%|████▏     | 2074/5000 [17:24:50<22:41:09, 27.91s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 42%|████▏     | 2099/5000 [17:36:39<22:30:05, 27.92s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 42%|████▎     | 2125/5000 [17:47:57<12:30:01, 15.65s/it]
+{'loss': 0.0032, 'learning_rate': 6.393333333333334e-06, 'epoch': 12.03}
+
+Reading metadata...: 23650it [00:01, 18542.50it/s]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 43%|████▎     | 2149/5000 [18:00:53<22:00:05, 27.78s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 43%|████▎     | 2174/5000 [18:12:39<22:18:15, 28.41s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 44%|████▍     | 2199/5000 [18:24:28<22:05:44, 28.40s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 44%|████▍     | 2224/5000 [18:36:09<21:36:49, 28.03s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 45%|████▍     | 2249/5000 [18:47:42<20:47:07, 27.20s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 45%|████▌     | 2274/5000 [18:59:24<21:26:31, 28.32s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 46%|████▌     | 2289/5000 [19:05:06<10:17:47, 13.67s/it]
+Reading metadata...: 23030it [00:01, 14373.66it/s]
+
+
+
+
+
+
+
+
+
+
+
+ 46%|████▌     | 2299/5000 [19:11:04<21:25:22, 28.55s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 46%|████▋     | 2324/5000 [19:22:58<20:25:42, 27.48s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 47%|████▋     | 2349/5000 [19:34:43<20:34:03, 27.93s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 48%|████▊     | 2375/5000 [19:47:01<20:35:00, 28.23s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 48%|████▊     | 2399/5000 [19:58:11<20:04:34, 27.79s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 48%|████▊     | 2424/5000 [20:09:46<19:53:06, 27.79s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 49%|████▉     | 2449/5000 [20:21:05<16:53:12, 23.83s/it]
+
+
+
+ 49%|████▉     | 2452/5000 [20:21:37<10:49:43, 15.30s/it]
+Reading metadata...: 10438it [00:00, 20922.81it/s]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 49%|████▉     | 2474/5000 [20:32:51<19:38:28, 27.99s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 50%|████▉     | 2499/5000 [20:44:26<19:28:55, 28.04s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 50%|█████     | 2524/5000 [20:56:12<19:11:49, 27.91s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 51%|█████     | 2549/5000 [21:07:48<18:50:45, 27.68s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 51%|█████▏    | 2574/5000 [21:19:27<19:00:41, 28.21s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 52%|█████▏    | 2600/5000 [21:31:27<18:47:00, 28.18s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Reading metadata...: 28043it [00:00, 37305.62it/s]9s/it]
+Reading metadata...: 10438it [00:00, 30532.73it/s]
+
+
+
+
+
+
+
+
+ 52%|█████▏    | 2624/5000 [21:42:30<18:57:49, 28.73s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 53%|█████▎    | 2649/5000 [21:54:03<18:04:16, 27.67s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 53%|█████▎    | 2674/5000 [22:05:42<17:55:47, 27.75s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 54%|█████▍    | 2699/5000 [22:17:12<17:36:47, 27.56s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 55%|█████▍    | 2725/5000 [22:29:20<17:34:08, 27.80s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 55%|█████▍    | 2749/5000 [22:40:23<17:10:47, 27.48s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 55%|█████▌    | 2774/5000 [22:51:54<16:57:03, 27.41s/it]
+
+
+
+
+
+Reading metadata...: 28043it [00:00, 37576.14it/s]8s/it]
+Reading metadata...: 10438it [00:00, 27193.36it/s]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 56%|█████▌    | 2799/5000 [23:03:28<17:07:04, 28.00s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 56%|█████▋    | 2825/5000 [23:15:28<16:46:36, 27.77s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 57%|█████▋    | 2849/5000 [23:26:34<16:24:17, 27.46s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 57%|█████▋    | 2874/5000 [23:38:14<16:29:40, 27.93s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 58%|█████▊    | 2899/5000 [23:49:46<16:09:24, 27.68s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 58%|█████▊    | 2924/5000 [24:01:17<15:52:41, 27.53s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 59%|█████▉    | 2943/5000 [24:08:47<7:45:50, 13.59s/it]
+
+Reading metadata...: 10438it [00:00, 21471.13it/s]
+
+
+
+
+
+
+ 59%|█████▉    | 2949/5000 [24:12:45<17:03:19, 29.94s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 59%|█████▉    | 2974/5000 [24:24:18<15:46:57, 28.04s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 60%|██████    | 3000/5000 [24:36:21<15:21:38, 27.65s/it][INFO|trainer.py:3138] 2023-05-08 11:10:32,150 >> ***** Running Evaluation *****
+[INFO|trainer.py:3142] 2023-05-08 11:10:32,150 >>   Num examples: Unknown
+[INFO|trainer.py:3143] 2023-05-08 11:10:32,150 >>   Batch size = 64
+Reading metadata...: 0it [00:00, ?it/s]
+
+
+[INFO|trainer_utils.py:693] 2023-05-08 11:10:42,418 >> The following columns in the evaluation set don't have a corresponding argument in `WhisperForConditionalGeneration.forward` and have been ignored: input_length. If input_length are not expected by `WhisperForConditionalGeneration.forward`,  you can safely ignore this message.
+ 60%|██████    | 3000/5000 [25:12:09<15:21:38, 27.65s/it][INFO|trainer.py:2877] 2023-05-08 11:46:20,051 >> Saving model checkpoint to ./checkpoint-3000
+[INFO|configuration_utils.py:458] 2023-05-08 11:46:20,057 >> Configuration saved in ./checkpoint-3000/config.json
+[INFO|configuration_utils.py:364] 2023-05-08 11:46:20,060 >> Configuration saved in ./checkpoint-3000/generation_config.json
+{'eval_loss': 0.6270926594734192, 'eval_wer': 56.897333333333336, 'eval_runtime': 2147.8916, 'eval_samples_per_second': 4.861, 'eval_steps_per_second': 0.076, 'epoch': 18.01}
+[INFO|modeling_utils.py:1855] 2023-05-08 11:46:22,975 >> Model weights saved in ./checkpoint-3000/pytorch_model.bin
+[INFO|feature_extraction_utils.py:369] 2023-05-08 11:46:22,981 >> Feature extractor saved in ./checkpoint-3000/preprocessor_config.json

wandb/run-20230507_103405-9zf5xxpu/files/wandb-summary.json

@@ -1 +1 @@
-{"eval/loss": 0.5698409676551819, "eval/wer": 55.900000000000006, "eval/runtime": 2152.7247, "eval/samples_per_second": 4.85, "eval/steps_per_second": 0.076, "_timestamp": 1683505378.5423114, "_runtime": 60532.58835840225, "_step": 81}
+{"eval/loss": 0.6270926594734192, "eval/wer": 56.897333333333336, "eval/runtime": 2147.8916, "eval/samples_per_second": 4.861, "_timestamp": 1683535580.042124, "_runtime": 90734.08817100525, "_step": 122}

wandb/run-20230507_103405-9zf5xxpu/run-9zf5xxpu.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0f427f677d5083d56399114c2fe48a9a9e499435e0ec1b3b9234b20f01c0c562
-size 5571854
+oid sha256:cfe7c996671203ddf47bdfd260a2524b769a026b2d2c1a501c67f52a9390968d
+size 8388822