|
2024-03-19 19:13:56,684 - INFO: Problem Type: text_causal_language_modeling |
|
2024-03-19 19:13:56,684 - INFO: Global random seed: 187907 |
|
2024-03-19 19:13:56,684 - INFO: Preparing the data... |
|
2024-03-19 19:13:56,684 - INFO: Setting up automatic validation split... |
|
2024-03-19 19:13:56,710 - INFO: Preparing train and validation data |
|
2024-03-19 19:13:56,710 - INFO: Loading train dataset... |
|
2024-03-19 19:13:57,105 - INFO: Stop token ids: [tensor([ 529, 29989, 5205, 29989, 29958])] |
|
2024-03-19 19:13:57,113 - INFO: Loading validation dataset... |
|
2024-03-19 19:13:57,196 - INFO: Stop token ids: [tensor([ 529, 29989, 5205, 29989, 29958])] |
|
2024-03-19 19:13:57,201 - INFO: Number of observations in train dataset: 567 |
|
2024-03-19 19:13:57,201 - INFO: Number of observations in validation dataset: 6 |
|
2024-03-19 19:13:57,463 - INFO: Stop token ids: [tensor([ 529, 29989, 5205, 29989, 29958], device='cuda:0')] |
|
2024-03-19 19:13:57,466 - WARNING: PAD token id not matching between config and tokenizer. Overwriting with tokenizer id. |
|
2024-03-19 19:13:57,466 - INFO: Setting pretraining_tp of model config to 1. |
|
2024-03-19 19:13:57,470 - INFO: Using float16 for backbone |
|
2024-03-19 19:13:57,470 - INFO: Loading TinyLlama/TinyLlama-1.1B-Chat-v0.6. This may take a while. |
|
2024-03-19 19:14:04,419 - INFO: Loaded TinyLlama/TinyLlama-1.1B-Chat-v0.6. |
|
2024-03-19 19:14:04,420 - WARNING: PAD token id not matching between generation config and tokenizer. Overwriting with tokenizer id. |
|
2024-03-19 19:14:04,421 - INFO: Lora module names: ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj'] |
|
2024-03-19 19:14:04,593 - INFO: Enough space available for saving model weights.Required space: 2173.51MB, Available space: 995215.66MB. |
|
2024-03-19 19:14:05,761 - INFO: Training Epoch: 1 / 15 |
|
2024-03-19 19:14:05,762 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:14:05,818 - INFO: Evaluation step: 283 |
|
2024-03-19 19:14:05,910 - INFO: Stop token ids: [tensor([ 529, 29989, 5205, 29989, 29958])] |
|
2024-03-19 19:14:08,219 - INFO: train loss: 2.08: 5%|4 | 14/283 [00:02<00:47, 5.70it/s] |
|
2024-03-19 19:14:10,137 - INFO: train loss: 1.85: 10%|9 | 28/283 [00:04<00:38, 6.54it/s] |
|
2024-03-19 19:14:12,396 - INFO: train loss: 1.62: 15%|#4 | 42/283 [00:06<00:37, 6.38it/s] |
|
2024-03-19 19:14:14,396 - INFO: train loss: 1.71: 20%|#9 | 56/283 [00:08<00:34, 6.61it/s] |
|
2024-03-19 19:14:16,570 - INFO: train loss: 1.79: 25%|##4 | 70/283 [00:10<00:32, 6.55it/s] |
|
2024-03-19 19:14:18,500 - INFO: train loss: 1.78: 30%|##9 | 84/283 [00:12<00:29, 6.77it/s] |
|
2024-03-19 19:14:20,437 - INFO: train loss: 1.68: 35%|###4 | 98/283 [00:14<00:26, 6.92it/s] |
|
2024-03-19 19:14:22,458 - INFO: train loss: 1.53: 40%|###9 | 112/283 [00:16<00:24, 6.92it/s] |
|
2024-03-19 19:14:24,376 - INFO: train loss: 1.87: 45%|####4 | 126/283 [00:18<00:22, 7.03it/s] |
|
2024-03-19 19:14:26,311 - INFO: train loss: 1.64: 49%|####9 | 140/283 [00:20<00:20, 7.09it/s] |
|
2024-03-19 19:14:28,391 - INFO: train loss: 1.68: 54%|#####4 | 154/283 [00:22<00:18, 6.98it/s] |
|
2024-03-19 19:14:30,288 - INFO: train loss: 1.73: 59%|#####9 | 168/283 [00:24<00:16, 7.10it/s] |
|
2024-03-19 19:14:32,242 - INFO: train loss: 1.48: 64%|######4 | 182/283 [00:26<00:14, 7.12it/s] |
|
2024-03-19 19:14:34,204 - INFO: train loss: 1.98: 69%|######9 | 196/283 [00:28<00:12, 7.12it/s] |
|
2024-03-19 19:14:36,283 - INFO: train loss: 1.88: 74%|#######4 | 210/283 [00:30<00:10, 7.00it/s] |
|
2024-03-19 19:14:38,215 - INFO: train loss: 1.84: 79%|#######9 | 224/283 [00:32<00:08, 7.07it/s] |
|
2024-03-19 19:14:40,143 - INFO: train loss: 1.57: 84%|########4 | 238/283 [00:34<00:06, 7.13it/s] |
|
2024-03-19 19:14:42,171 - INFO: train loss: 1.86: 89%|########9 | 252/283 [00:36<00:04, 7.06it/s] |
|
2024-03-19 19:14:44,241 - INFO: train loss: 1.58: 94%|#########3| 266/283 [00:38<00:02, 6.97it/s] |
|
2024-03-19 19:14:46,206 - INFO: train loss: 1.57: 99%|#########8| 280/283 [00:40<00:00, 7.01it/s] |
|
2024-03-19 19:14:46,606 - INFO: train loss: 1.71: 100%|##########| 283/283 [00:40<00:00, 6.93it/s] |
|
2024-03-19 19:14:48,316 - INFO: Starting validation inference |
|
2024-03-19 19:14:48,317 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:15:01,765 - INFO: validation progress: 33%|###3 | 1/3 [00:13<00:26, 13.45s/it] |
|
2024-03-19 19:15:14,444 - INFO: validation progress: 67%|######6 | 2/3 [00:26<00:12, 13.00s/it] |
|
2024-03-19 19:15:22,139 - INFO: validation progress: 100%|##########| 3/3 [00:33<00:00, 10.58s/it] |
|
2024-03-19 19:15:22,142 - INFO: validation progress: 100%|##########| 3/3 [00:33<00:00, 11.27s/it] |
|
2024-03-19 19:15:22,166 - INFO: Validation BLEU: 1.57108 |
|
2024-03-19 19:15:22,647 - INFO: Training Epoch: 2 / 15 |
|
2024-03-19 19:15:22,648 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:15:22,672 - INFO: Evaluation step: 283 |
|
2024-03-19 19:15:24,757 - INFO: train loss: 1.57: 5%|4 | 14/283 [00:02<00:40, 6.64it/s] |
|
2024-03-19 19:15:26,692 - INFO: train loss: 1.61: 10%|9 | 28/283 [00:04<00:36, 6.98it/s] |
|
2024-03-19 19:15:28,798 - INFO: train loss: 1.64: 15%|#4 | 42/283 [00:06<00:35, 6.82it/s] |
|
2024-03-19 19:15:30,750 - INFO: train loss: 1.58: 20%|#9 | 56/283 [00:08<00:32, 6.96it/s] |
|
2024-03-19 19:15:32,723 - INFO: train loss: 1.41: 25%|##4 | 70/283 [00:10<00:30, 7.01it/s] |
|
2024-03-19 19:15:34,707 - INFO: train loss: 1.62: 30%|##9 | 84/283 [00:12<00:28, 7.02it/s] |
|
2024-03-19 19:15:36,718 - INFO: train loss: 1.51: 35%|###4 | 98/283 [00:14<00:26, 7.00it/s] |
|
2024-03-19 19:15:38,688 - INFO: train loss: 1.47: 40%|###9 | 112/283 [00:16<00:24, 7.04it/s] |
|
2024-03-19 19:15:40,575 - INFO: train loss: 1.83: 45%|####4 | 126/283 [00:17<00:21, 7.15it/s] |
|
2024-03-19 19:15:42,811 - INFO: train loss: 1.79: 49%|####9 | 140/283 [00:20<00:20, 6.85it/s] |
|
2024-03-19 19:15:44,784 - INFO: train loss: 1.58: 54%|#####4 | 154/283 [00:22<00:18, 6.92it/s] |
|
2024-03-19 19:15:46,727 - INFO: train loss: 1.62: 59%|#####9 | 168/283 [00:24<00:16, 7.01it/s] |
|
2024-03-19 19:15:48,672 - INFO: train loss: 1.68: 64%|######4 | 182/283 [00:26<00:14, 7.06it/s] |
|
2024-03-19 19:15:50,808 - INFO: train loss: 1.87: 69%|######9 | 196/283 [00:28<00:12, 6.90it/s] |
|
2024-03-19 19:15:52,770 - INFO: train loss: 1.68: 74%|#######4 | 210/283 [00:30<00:10, 6.97it/s] |
|
2024-03-19 19:15:54,845 - INFO: train loss: 1.54: 79%|#######9 | 224/283 [00:32<00:08, 6.90it/s] |
|
2024-03-19 19:15:56,907 - INFO: train loss: 1.54: 84%|########4 | 238/283 [00:34<00:06, 6.87it/s] |
|
2024-03-19 19:15:59,051 - INFO: train loss: 1.56: 89%|########9 | 252/283 [00:36<00:04, 6.76it/s] |
|
2024-03-19 19:16:01,000 - INFO: train loss: 1.43: 94%|#########3| 266/283 [00:38<00:02, 6.88it/s] |
|
2024-03-19 19:16:02,997 - INFO: train loss: 1.34: 99%|#########8| 280/283 [00:40<00:00, 6.92it/s] |
|
2024-03-19 19:16:03,404 - INFO: train loss: 1.32: 100%|##########| 283/283 [00:40<00:00, 6.94it/s] |
|
2024-03-19 19:16:05,188 - INFO: Starting validation inference |
|
2024-03-19 19:16:05,189 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:16:18,237 - INFO: validation progress: 33%|###3 | 1/3 [00:13<00:26, 13.05s/it] |
|
2024-03-19 19:16:31,151 - INFO: validation progress: 67%|######6 | 2/3 [00:25<00:12, 12.97s/it] |
|
2024-03-19 19:16:38,613 - INFO: validation progress: 100%|##########| 3/3 [00:33<00:00, 10.45s/it] |
|
2024-03-19 19:16:38,615 - INFO: validation progress: 100%|##########| 3/3 [00:33<00:00, 11.14s/it] |
|
2024-03-19 19:16:38,641 - INFO: Validation BLEU: 2.35677 |
|
2024-03-19 19:16:39,425 - INFO: Training Epoch: 3 / 15 |
|
2024-03-19 19:16:39,425 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:16:39,448 - INFO: Evaluation step: 283 |
|
2024-03-19 19:16:41,408 - INFO: train loss: 1.36: 5%|4 | 14/283 [00:01<00:38, 7.06it/s] |
|
2024-03-19 19:16:43,533 - INFO: train loss: 1.53: 10%|9 | 28/283 [00:04<00:37, 6.77it/s] |
|
2024-03-19 19:16:45,641 - INFO: train loss: 1.37: 15%|#4 | 42/283 [00:06<00:35, 6.71it/s] |
|
2024-03-19 19:16:47,559 - INFO: train loss: 1.37: 20%|#9 | 56/283 [00:08<00:32, 6.93it/s] |
|
2024-03-19 19:16:49,541 - INFO: train loss: 1.11: 25%|##4 | 70/283 [00:10<00:30, 6.98it/s] |
|
2024-03-19 19:16:51,495 - INFO: train loss: 1.41: 30%|##9 | 84/283 [00:12<00:28, 7.04it/s] |
|
2024-03-19 19:16:53,489 - INFO: train loss: 1.38: 35%|###4 | 98/283 [00:14<00:26, 7.03it/s] |
|
2024-03-19 19:16:55,459 - INFO: train loss: 1.39: 40%|###9 | 112/283 [00:16<00:24, 7.06it/s] |
|
2024-03-19 19:16:57,560 - INFO: train loss: 1.49: 45%|####4 | 126/283 [00:18<00:22, 6.93it/s] |
|
2024-03-19 19:16:59,571 - INFO: train loss: 1.26: 49%|####9 | 140/283 [00:20<00:20, 6.94it/s] |
|
2024-03-19 19:17:01,505 - INFO: train loss: 1.31: 54%|#####4 | 154/283 [00:22<00:18, 7.03it/s] |
|
2024-03-19 19:17:03,559 - INFO: train loss: 1.28: 59%|#####9 | 168/283 [00:24<00:16, 6.96it/s] |
|
2024-03-19 19:17:05,601 - INFO: train loss: 1.51: 64%|######4 | 182/283 [00:26<00:14, 6.93it/s] |
|
2024-03-19 19:17:07,531 - INFO: train loss: 1.35: 69%|######9 | 196/283 [00:28<00:12, 7.02it/s] |
|
2024-03-19 19:17:09,455 - INFO: train loss: 1.24: 74%|#######4 | 210/283 [00:30<00:10, 7.10it/s] |
|
2024-03-19 19:17:11,539 - INFO: train loss: 1.39: 79%|#######9 | 224/283 [00:32<00:08, 6.98it/s] |
|
2024-03-19 19:17:13,690 - INFO: train loss: 1.57: 84%|########4 | 238/283 [00:34<00:06, 6.83it/s] |
|
2024-03-19 19:17:15,757 - INFO: train loss: 1.39: 89%|########9 | 252/283 [00:36<00:04, 6.81it/s] |
|
2024-03-19 19:17:17,748 - INFO: train loss: 1.27: 94%|#########3| 266/283 [00:38<00:02, 6.88it/s] |
|
2024-03-19 19:17:19,761 - INFO: train loss: 1.36: 99%|#########8| 280/283 [00:40<00:00, 6.90it/s] |
|
2024-03-19 19:17:20,180 - INFO: train loss: 1.26: 100%|##########| 283/283 [00:40<00:00, 6.94it/s] |
|
2024-03-19 19:17:21,937 - INFO: Starting validation inference |
|
2024-03-19 19:17:21,938 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:17:35,042 - INFO: validation progress: 33%|###3 | 1/3 [00:13<00:26, 13.10s/it] |
|
2024-03-19 19:17:48,044 - INFO: validation progress: 67%|######6 | 2/3 [00:26<00:13, 13.04s/it] |
|
2024-03-19 19:17:55,447 - INFO: validation progress: 100%|##########| 3/3 [00:33<00:00, 10.47s/it] |
|
2024-03-19 19:17:55,450 - INFO: validation progress: 100%|##########| 3/3 [00:33<00:00, 11.17s/it] |
|
2024-03-19 19:17:55,473 - INFO: Validation BLEU: 2.75814 |
|
2024-03-19 19:17:55,922 - INFO: Training Epoch: 4 / 15 |
|
2024-03-19 19:17:55,922 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:17:55,946 - INFO: Evaluation step: 283 |
|
2024-03-19 19:17:58,105 - INFO: train loss: 1.08: 5%|4 | 14/283 [00:02<00:41, 6.42it/s] |
|
2024-03-19 19:18:00,099 - INFO: train loss: 1.09: 10%|9 | 28/283 [00:04<00:37, 6.76it/s] |
|
2024-03-19 19:18:02,140 - INFO: train loss: 1.11: 15%|#4 | 42/283 [00:06<00:35, 6.80it/s] |
|
2024-03-19 19:18:04,169 - INFO: train loss: 1.16: 20%|#9 | 56/283 [00:08<00:33, 6.84it/s] |
|
2024-03-19 19:18:06,221 - INFO: train loss: 1.05: 25%|##4 | 70/283 [00:10<00:31, 6.83it/s] |
|
2024-03-19 19:18:08,194 - INFO: train loss: 1.22: 30%|##9 | 84/283 [00:12<00:28, 6.92it/s] |
|
2024-03-19 19:18:10,106 - INFO: train loss: 1.05: 35%|###4 | 98/283 [00:14<00:26, 7.05it/s] |
|
2024-03-19 19:18:12,086 - INFO: train loss: 1.07: 40%|###9 | 112/283 [00:16<00:24, 7.06it/s] |
|
2024-03-19 19:18:14,175 - INFO: train loss: 1.26: 45%|####4 | 126/283 [00:18<00:22, 6.94it/s] |
|
2024-03-19 19:18:16,202 - INFO: train loss: 1.09: 49%|####9 | 140/283 [00:20<00:20, 6.93it/s] |
|
2024-03-19 19:18:18,152 - INFO: train loss: 0.95: 54%|#####4 | 154/283 [00:22<00:18, 7.01it/s] |
|
2024-03-19 19:18:20,089 - INFO: train loss: 0.93: 59%|#####9 | 168/283 [00:24<00:16, 7.07it/s] |
|
2024-03-19 19:18:22,029 - INFO: train loss: 1.36: 64%|######4 | 182/283 [00:26<00:14, 7.11it/s] |
|
2024-03-19 19:18:24,016 - INFO: train loss: 1.22: 69%|######9 | 196/283 [00:28<00:12, 7.09it/s] |
|
2024-03-19 19:18:26,043 - INFO: train loss: 1.25: 74%|#######4 | 210/283 [00:30<00:10, 7.04it/s] |
|
2024-03-19 19:18:28,104 - INFO: train loss: 1.22: 79%|#######9 | 224/283 [00:32<00:08, 6.96it/s] |
|
2024-03-19 19:18:29,994 - INFO: train loss: 1.15: 84%|########4 | 238/283 [00:34<00:06, 7.09it/s] |
|
2024-03-19 19:18:32,059 - INFO: train loss: 1.44: 89%|########9 | 252/283 [00:36<00:04, 6.99it/s] |
|
2024-03-19 19:18:34,055 - INFO: train loss: 1.05: 94%|#########3| 266/283 [00:38<00:02, 7.00it/s] |
|
2024-03-19 19:18:36,157 - INFO: train loss: 1.25: 99%|#########8| 280/283 [00:40<00:00, 6.89it/s] |
|
2024-03-19 19:18:36,549 - INFO: train loss: 1.35: 100%|##########| 283/283 [00:40<00:00, 6.97it/s] |
|
2024-03-19 19:18:38,322 - INFO: Starting validation inference |
|
2024-03-19 19:18:38,323 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:18:50,498 - INFO: validation progress: 33%|###3 | 1/3 [00:12<00:24, 12.18s/it] |
|
2024-03-19 19:19:02,261 - INFO: validation progress: 67%|######6 | 2/3 [00:23<00:11, 11.93s/it] |
|
2024-03-19 19:19:10,132 - INFO: validation progress: 100%|##########| 3/3 [00:31<00:00, 10.08s/it] |
|
2024-03-19 19:19:10,134 - INFO: validation progress: 100%|##########| 3/3 [00:31<00:00, 10.60s/it] |
|
2024-03-19 19:19:10,157 - INFO: Validation BLEU: 2.32579 |
|
2024-03-19 19:19:10,555 - INFO: Training Epoch: 5 / 15 |
|
2024-03-19 19:19:10,555 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:19:10,575 - INFO: Evaluation step: 283 |
|
2024-03-19 19:19:12,694 - INFO: train loss: 0.99: 5%|4 | 14/283 [00:02<00:41, 6.55it/s] |
|
2024-03-19 19:19:14,759 - INFO: train loss: 0.96: 10%|9 | 28/283 [00:04<00:38, 6.68it/s] |
|
2024-03-19 19:19:16,857 - INFO: train loss: 0.90: 15%|#4 | 42/283 [00:06<00:36, 6.68it/s] |
|
2024-03-19 19:19:18,938 - INFO: train loss: 1.08: 20%|#9 | 56/283 [00:08<00:33, 6.70it/s] |
|
2024-03-19 19:19:20,951 - INFO: train loss: 0.88: 25%|##4 | 70/283 [00:10<00:31, 6.79it/s] |
|
2024-03-19 19:19:22,935 - INFO: train loss: 0.89: 30%|##9 | 84/283 [00:12<00:28, 6.88it/s] |
|
2024-03-19 19:19:24,947 - INFO: train loss: 1.02: 35%|###4 | 98/283 [00:14<00:26, 6.90it/s] |
|
2024-03-19 19:19:26,886 - INFO: train loss: 1.05: 40%|###9 | 112/283 [00:16<00:24, 7.00it/s] |
|
2024-03-19 19:19:29,093 - INFO: train loss: 0.98: 45%|####4 | 126/283 [00:18<00:23, 6.78it/s] |
|
2024-03-19 19:19:31,015 - INFO: train loss: 0.92: 49%|####9 | 140/283 [00:20<00:20, 6.93it/s] |
|
2024-03-19 19:19:32,935 - INFO: train loss: 0.88: 54%|#####4 | 154/283 [00:22<00:18, 7.04it/s] |
|
2024-03-19 19:19:34,923 - INFO: train loss: 1.00: 59%|#####9 | 168/283 [00:24<00:16, 7.04it/s] |
|
2024-03-19 19:19:37,012 - INFO: train loss: 0.96: 64%|######4 | 182/283 [00:26<00:14, 6.93it/s] |
|
2024-03-19 19:19:38,984 - INFO: train loss: 1.03: 69%|######9 | 196/283 [00:28<00:12, 6.98it/s] |
|
2024-03-19 19:19:40,986 - INFO: train loss: 0.97: 74%|#######4 | 210/283 [00:30<00:10, 6.99it/s] |
|
2024-03-19 19:19:43,117 - INFO: train loss: 0.91: 79%|#######9 | 224/283 [00:32<00:08, 6.86it/s] |
|
2024-03-19 19:19:45,119 - INFO: train loss: 1.17: 84%|########4 | 238/283 [00:34<00:06, 6.90it/s] |
|
2024-03-19 19:19:47,047 - INFO: train loss: 1.16: 89%|########9 | 252/283 [00:36<00:04, 7.00it/s] |
|
2024-03-19 19:19:49,067 - INFO: train loss: 0.90: 94%|#########3| 266/283 [00:38<00:02, 6.98it/s] |
|
2024-03-19 19:19:50,994 - INFO: train loss: 0.85: 99%|#########8| 280/283 [00:40<00:00, 7.06it/s] |
|
2024-03-19 19:19:51,390 - INFO: train loss: 0.87: 100%|##########| 283/283 [00:40<00:00, 6.93it/s] |
|
2024-03-19 19:19:53,184 - INFO: Starting validation inference |
|
2024-03-19 19:19:53,184 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:20:06,464 - INFO: validation progress: 33%|###3 | 1/3 [00:13<00:26, 13.28s/it] |
|
2024-03-19 19:20:18,561 - INFO: validation progress: 67%|######6 | 2/3 [00:25<00:12, 12.58s/it] |
|
2024-03-19 19:20:26,404 - INFO: validation progress: 100%|##########| 3/3 [00:33<00:00, 10.42s/it] |
|
2024-03-19 19:20:26,407 - INFO: validation progress: 100%|##########| 3/3 [00:33<00:00, 11.07s/it] |
|
2024-03-19 19:20:26,429 - INFO: Validation BLEU: 2.74420 |
|
2024-03-19 19:20:27,086 - INFO: Training Epoch: 6 / 15 |
|
2024-03-19 19:20:27,086 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:20:27,111 - INFO: Evaluation step: 283 |
|
2024-03-19 19:20:29,185 - INFO: train loss: 0.74: 5%|4 | 14/283 [00:02<00:40, 6.67it/s] |
|
2024-03-19 19:20:31,129 - INFO: train loss: 0.77: 10%|9 | 28/283 [00:04<00:36, 6.97it/s] |
|
2024-03-19 19:20:33,110 - INFO: train loss: 0.85: 15%|#4 | 42/283 [00:06<00:34, 7.02it/s] |
|
2024-03-19 19:20:35,085 - INFO: train loss: 0.77: 20%|#9 | 56/283 [00:07<00:32, 7.05it/s] |
|
2024-03-19 19:20:37,096 - INFO: train loss: 0.81: 25%|##4 | 70/283 [00:10<00:30, 7.01it/s] |
|
2024-03-19 19:20:39,051 - INFO: train loss: 0.56: 30%|##9 | 84/283 [00:11<00:28, 7.06it/s] |
|
2024-03-19 19:20:41,010 - INFO: train loss: 0.80: 35%|###4 | 98/283 [00:13<00:26, 7.09it/s] |
|
2024-03-19 19:20:43,079 - INFO: train loss: 0.72: 40%|###9 | 112/283 [00:15<00:24, 6.98it/s] |
|
2024-03-19 19:20:45,210 - INFO: train loss: 0.80: 45%|####4 | 126/283 [00:18<00:22, 6.85it/s] |
|
2024-03-19 19:20:47,341 - INFO: train loss: 0.89: 49%|####9 | 140/283 [00:20<00:21, 6.76it/s] |
|
2024-03-19 19:20:49,355 - INFO: train loss: 0.78: 54%|#####4 | 154/283 [00:22<00:18, 6.82it/s] |
|
2024-03-19 19:20:51,343 - INFO: train loss: 0.77: 59%|#####9 | 168/283 [00:24<00:16, 6.89it/s] |
|
2024-03-19 19:20:53,304 - INFO: train loss: 0.86: 64%|######4 | 182/283 [00:26<00:14, 6.96it/s] |
|
2024-03-19 19:20:55,312 - INFO: train loss: 0.86: 69%|######9 | 196/283 [00:28<00:12, 6.96it/s] |
|
2024-03-19 19:20:57,646 - INFO: train loss: 1.15: 74%|#######4 | 210/283 [00:30<00:10, 6.64it/s] |
|
2024-03-19 19:20:59,636 - INFO: train loss: 0.81: 79%|#######9 | 224/283 [00:32<00:08, 6.75it/s] |
|
2024-03-19 19:21:01,710 - INFO: train loss: 0.84: 84%|########4 | 238/283 [00:34<00:06, 6.75it/s] |
|
2024-03-19 19:21:03,718 - INFO: train loss: 0.75: 89%|########9 | 252/283 [00:36<00:04, 6.82it/s] |
|
2024-03-19 19:21:05,771 - INFO: train loss: 0.77: 94%|#########3| 266/283 [00:38<00:02, 6.82it/s] |
|
2024-03-19 19:21:07,743 - INFO: train loss: 0.54: 99%|#########8| 280/283 [00:40<00:00, 6.90it/s] |
|
2024-03-19 19:21:08,139 - INFO: train loss: 0.66: 100%|##########| 283/283 [00:41<00:00, 6.89it/s] |
|
2024-03-19 19:21:09,922 - INFO: Starting validation inference |
|
2024-03-19 19:21:09,922 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:21:23,121 - INFO: validation progress: 33%|###3 | 1/3 [00:13<00:26, 13.20s/it] |
|
2024-03-19 19:21:32,151 - INFO: validation progress: 67%|######6 | 2/3 [00:22<00:10, 10.75s/it] |
|
2024-03-19 19:21:38,942 - INFO: validation progress: 100%|##########| 3/3 [00:29<00:00, 8.94s/it] |
|
2024-03-19 19:21:38,944 - INFO: validation progress: 100%|##########| 3/3 [00:29<00:00, 9.67s/it] |
|
2024-03-19 19:21:38,967 - INFO: Validation BLEU: 2.07270 |
|
2024-03-19 19:21:39,362 - INFO: Training Epoch: 7 / 15 |
|
2024-03-19 19:21:39,362 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:21:39,382 - INFO: Evaluation step: 283 |
|
2024-03-19 19:21:41,507 - INFO: train loss: 0.67: 5%|4 | 14/283 [00:02<00:41, 6.53it/s] |
|
2024-03-19 19:21:43,603 - INFO: train loss: 0.56: 10%|9 | 28/283 [00:04<00:38, 6.62it/s] |
|
2024-03-19 19:21:45,671 - INFO: train loss: 0.66: 15%|#4 | 42/283 [00:06<00:36, 6.69it/s] |
|
2024-03-19 19:21:47,707 - INFO: train loss: 0.71: 20%|#9 | 56/283 [00:08<00:33, 6.76it/s] |
|
2024-03-19 19:21:49,672 - INFO: train loss: 0.68: 25%|##4 | 70/283 [00:10<00:30, 6.89it/s] |
|
2024-03-19 19:21:51,623 - INFO: train loss: 0.50: 30%|##9 | 84/283 [00:12<00:28, 6.98it/s] |
|
2024-03-19 19:21:53,583 - INFO: train loss: 0.65: 35%|###4 | 98/283 [00:14<00:26, 7.03it/s] |
|
2024-03-19 19:21:55,712 - INFO: train loss: 0.46: 40%|###9 | 112/283 [00:16<00:24, 6.88it/s] |
|
2024-03-19 19:21:57,824 - INFO: train loss: 0.61: 45%|####4 | 126/283 [00:18<00:23, 6.80it/s] |
|
2024-03-19 19:21:59,861 - INFO: train loss: 0.64: 49%|####9 | 140/283 [00:20<00:20, 6.82it/s] |
|
2024-03-19 19:22:01,773 - INFO: train loss: 0.82: 54%|#####4 | 154/283 [00:22<00:18, 6.97it/s] |
|
2024-03-19 19:22:03,787 - INFO: train loss: 0.57: 59%|#####9 | 168/283 [00:24<00:16, 6.96it/s] |
|
2024-03-19 19:22:05,855 - INFO: train loss: 0.67: 64%|######4 | 182/283 [00:26<00:14, 6.90it/s] |
|
2024-03-19 19:22:07,815 - INFO: train loss: 0.61: 69%|######9 | 196/283 [00:28<00:12, 6.97it/s] |
|
2024-03-19 19:22:09,779 - INFO: train loss: 0.60: 74%|#######4 | 210/283 [00:30<00:10, 7.02it/s] |
|
2024-03-19 19:22:11,700 - INFO: train loss: 0.45: 79%|#######9 | 224/283 [00:32<00:08, 7.10it/s] |
|
2024-03-19 19:22:13,767 - INFO: train loss: 0.69: 84%|########4 | 238/283 [00:34<00:06, 7.00it/s] |
|
2024-03-19 19:22:15,868 - INFO: train loss: 0.82: 89%|########9 | 252/283 [00:36<00:04, 6.89it/s] |
|
2024-03-19 19:22:17,867 - INFO: train loss: 0.74: 94%|#########3| 266/283 [00:38<00:02, 6.93it/s] |
|
2024-03-19 19:22:19,933 - INFO: train loss: 0.64: 99%|#########8| 280/283 [00:40<00:00, 6.88it/s] |
|
2024-03-19 19:22:20,353 - INFO: train loss: 0.68: 100%|##########| 283/283 [00:40<00:00, 6.90it/s] |
|
2024-03-19 19:22:22,117 - INFO: Starting validation inference |
|
2024-03-19 19:22:22,117 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:22:35,233 - INFO: validation progress: 33%|###3 | 1/3 [00:13<00:26, 13.12s/it] |
|
2024-03-19 19:22:47,770 - INFO: validation progress: 67%|######6 | 2/3 [00:25<00:12, 12.78s/it] |
|
2024-03-19 19:22:58,692 - INFO: validation progress: 100%|##########| 3/3 [00:36<00:00, 11.93s/it] |
|
2024-03-19 19:22:58,694 - INFO: validation progress: 100%|##########| 3/3 [00:36<00:00, 12.19s/it] |
|
2024-03-19 19:22:58,717 - INFO: Validation BLEU: 3.45539 |
|
2024-03-19 19:22:59,109 - INFO: Training Epoch: 8 / 15 |
|
2024-03-19 19:22:59,110 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:22:59,134 - INFO: Evaluation step: 283 |
|
2024-03-19 19:23:01,121 - INFO: train loss: 0.53: 5%|4 | 14/283 [00:02<00:38, 6.96it/s] |
|
2024-03-19 19:23:03,133 - INFO: train loss: 0.49: 10%|9 | 28/283 [00:04<00:36, 6.96it/s] |
|
2024-03-19 19:23:05,243 - INFO: train loss: 0.67: 15%|#4 | 42/283 [00:06<00:35, 6.81it/s] |
|
2024-03-19 19:23:07,334 - INFO: train loss: 0.46: 20%|#9 | 56/283 [00:08<00:33, 6.76it/s] |
|
2024-03-19 19:23:09,372 - INFO: train loss: 0.50: 25%|##4 | 70/283 [00:10<00:31, 6.80it/s] |
|
2024-03-19 19:23:11,299 - INFO: train loss: 0.48: 30%|##9 | 84/283 [00:12<00:28, 6.95it/s] |
|
2024-03-19 19:23:13,418 - INFO: train loss: 0.47: 35%|###4 | 98/283 [00:14<00:27, 6.83it/s] |
|
2024-03-19 19:23:15,434 - INFO: train loss: 0.50: 40%|###9 | 112/283 [00:16<00:24, 6.87it/s] |
|
2024-03-19 19:23:17,433 - INFO: train loss: 0.62: 45%|####4 | 126/283 [00:18<00:22, 6.91it/s] |
|
2024-03-19 19:23:19,392 - INFO: train loss: 0.41: 49%|####9 | 140/283 [00:20<00:20, 6.98it/s] |
|
2024-03-19 19:23:21,356 - INFO: train loss: 0.39: 54%|#####4 | 154/283 [00:22<00:18, 7.03it/s] |
|
2024-03-19 19:23:23,428 - INFO: train loss: 0.63: 59%|#####9 | 168/283 [00:24<00:16, 6.94it/s] |
|
2024-03-19 19:23:25,426 - INFO: train loss: 0.51: 64%|######4 | 182/283 [00:26<00:14, 6.96it/s] |
|
2024-03-19 19:23:27,540 - INFO: train loss: 0.52: 69%|######9 | 196/283 [00:28<00:12, 6.86it/s] |
|
2024-03-19 19:23:29,514 - INFO: train loss: 0.43: 74%|#######4 | 210/283 [00:30<00:10, 6.93it/s] |
|
2024-03-19 19:23:31,539 - INFO: train loss: 0.57: 79%|#######9 | 224/283 [00:32<00:08, 6.92it/s] |
|
2024-03-19 19:23:33,597 - INFO: train loss: 0.52: 84%|########4 | 238/283 [00:34<00:06, 6.89it/s] |
|
2024-03-19 19:23:35,649 - INFO: train loss: 0.49: 89%|########9 | 252/283 [00:36<00:04, 6.87it/s] |
|
2024-03-19 19:23:37,566 - INFO: train loss: 0.53: 94%|#########3| 266/283 [00:38<00:02, 6.99it/s] |
|
2024-03-19 19:23:39,538 - INFO: train loss: 0.69: 99%|#########8| 280/283 [00:40<00:00, 7.02it/s] |
|
2024-03-19 19:23:39,952 - INFO: train loss: 0.60: 100%|##########| 283/283 [00:40<00:00, 6.93it/s] |
|
2024-03-19 19:23:41,720 - INFO: Starting validation inference |
|
2024-03-19 19:23:41,720 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:23:54,794 - INFO: validation progress: 33%|###3 | 1/3 [00:13<00:26, 13.07s/it] |
|
2024-03-19 19:24:01,651 - INFO: validation progress: 67%|######6 | 2/3 [00:19<00:09, 9.42s/it] |
|
2024-03-19 19:24:07,434 - INFO: validation progress: 100%|##########| 3/3 [00:25<00:00, 7.76s/it] |
|
2024-03-19 19:24:07,437 - INFO: validation progress: 100%|##########| 3/3 [00:25<00:00, 8.57s/it] |
|
2024-03-19 19:24:07,459 - INFO: Validation BLEU: 2.35797 |
|
2024-03-19 19:24:07,882 - INFO: Training Epoch: 9 / 15 |
|
2024-03-19 19:24:07,883 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:24:07,903 - INFO: Evaluation step: 283 |
|
2024-03-19 19:24:09,845 - INFO: train loss: 0.38: 5%|4 | 14/283 [00:01<00:37, 7.14it/s] |
|
2024-03-19 19:24:11,751 - INFO: train loss: 0.26: 10%|9 | 28/283 [00:03<00:35, 7.26it/s] |
|
2024-03-19 19:24:13,826 - INFO: train loss: 0.34: 15%|#4 | 42/283 [00:05<00:34, 7.01it/s] |
|
2024-03-19 19:24:15,863 - INFO: train loss: 0.42: 20%|#9 | 56/283 [00:07<00:32, 6.96it/s] |
|
2024-03-19 19:24:17,798 - INFO: train loss: 0.45: 25%|##4 | 70/283 [00:09<00:30, 7.06it/s] |
|
2024-03-19 19:24:19,802 - INFO: train loss: 0.44: 30%|##9 | 84/283 [00:11<00:28, 7.03it/s] |
|
2024-03-19 19:24:21,843 - INFO: train loss: 0.35: 35%|###4 | 98/283 [00:13<00:26, 6.97it/s] |
|
2024-03-19 19:24:23,937 - INFO: train loss: 0.45: 40%|###9 | 112/283 [00:16<00:24, 6.88it/s] |
|
2024-03-19 19:24:25,926 - INFO: train loss: 0.40: 45%|####4 | 126/283 [00:18<00:22, 6.93it/s] |
|
2024-03-19 19:24:28,002 - INFO: train loss: 0.39: 49%|####9 | 140/283 [00:20<00:20, 6.87it/s] |
|
2024-03-19 19:24:29,956 - INFO: train loss: 0.46: 54%|#####4 | 154/283 [00:22<00:18, 6.96it/s] |
|
2024-03-19 19:24:31,904 - INFO: train loss: 0.40: 59%|#####9 | 168/283 [00:24<00:16, 7.03it/s] |
|
2024-03-19 19:24:34,100 - INFO: train loss: 0.36: 64%|######4 | 182/283 [00:26<00:14, 6.82it/s] |
|
2024-03-19 19:24:36,133 - INFO: train loss: 0.51: 69%|######9 | 196/283 [00:28<00:12, 6.84it/s] |
|
2024-03-19 19:24:38,176 - INFO: train loss: 0.42: 74%|#######4 | 210/283 [00:30<00:10, 6.84it/s] |
|
2024-03-19 19:24:40,093 - INFO: train loss: 0.41: 79%|#######9 | 224/283 [00:32<00:08, 6.97it/s] |
|
2024-03-19 19:24:42,233 - INFO: train loss: 0.43: 84%|########4 | 238/283 [00:34<00:06, 6.84it/s] |
|
2024-03-19 19:24:44,313 - INFO: train loss: 0.35: 89%|########9 | 252/283 [00:36<00:04, 6.81it/s] |
|
2024-03-19 19:24:46,270 - INFO: train loss: 0.42: 94%|#########3| 266/283 [00:38<00:02, 6.91it/s] |
|
2024-03-19 19:24:48,312 - INFO: train loss: 0.51: 99%|#########8| 280/283 [00:40<00:00, 6.89it/s] |
|
2024-03-19 19:24:48,724 - INFO: train loss: 0.55: 100%|##########| 283/283 [00:40<00:00, 6.93it/s] |
|
2024-03-19 19:24:50,493 - INFO: Starting validation inference |
|
2024-03-19 19:24:50,493 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:25:02,850 - INFO: validation progress: 33%|###3 | 1/3 [00:12<00:24, 12.36s/it] |
|
2024-03-19 19:25:10,183 - INFO: validation progress: 67%|######6 | 2/3 [00:19<00:09, 9.40s/it] |
|
2024-03-19 19:25:16,141 - INFO: validation progress: 100%|##########| 3/3 [00:25<00:00, 7.83s/it] |
|
2024-03-19 19:25:16,144 - INFO: validation progress: 100%|##########| 3/3 [00:25<00:00, 8.55s/it] |
|
2024-03-19 19:25:16,165 - INFO: Validation BLEU: 2.20909 |
|
2024-03-19 19:25:16,592 - INFO: Training Epoch: 10 / 15 |
|
2024-03-19 19:25:16,592 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:25:16,616 - INFO: Evaluation step: 283 |
|
2024-03-19 19:25:18,649 - INFO: train loss: 0.41: 5%|4 | 14/283 [00:02<00:39, 6.81it/s] |
|
2024-03-19 19:25:20,588 - INFO: train loss: 0.27: 10%|9 | 28/283 [00:03<00:36, 7.04it/s] |
|
2024-03-19 19:25:22,491 - INFO: train loss: 0.40: 15%|#4 | 42/283 [00:05<00:33, 7.18it/s] |
|
2024-03-19 19:25:24,580 - INFO: train loss: 0.40: 20%|#9 | 56/283 [00:07<00:32, 6.99it/s] |
|
2024-03-19 19:25:26,582 - INFO: train loss: 0.27: 25%|##4 | 70/283 [00:09<00:30, 6.99it/s] |
|
2024-03-19 19:25:28,673 - INFO: train loss: 0.31: 30%|##9 | 84/283 [00:12<00:28, 6.89it/s] |
|
2024-03-19 19:25:30,650 - INFO: train loss: 0.34: 35%|###4 | 98/283 [00:14<00:26, 6.95it/s] |
|
2024-03-19 19:25:32,662 - INFO: train loss: 0.36: 40%|###9 | 112/283 [00:16<00:24, 6.95it/s] |
|
2024-03-19 19:25:34,658 - INFO: train loss: 0.21: 45%|####4 | 126/283 [00:18<00:22, 6.97it/s] |
|
2024-03-19 19:25:36,794 - INFO: train loss: 0.42: 49%|####9 | 140/283 [00:20<00:20, 6.84it/s] |
|
2024-03-19 19:25:38,721 - INFO: train loss: 0.51: 54%|#####4 | 154/283 [00:22<00:18, 6.96it/s] |
|
2024-03-19 19:25:40,745 - INFO: train loss: 0.32: 59%|#####9 | 168/283 [00:24<00:16, 6.95it/s] |
|
2024-03-19 19:25:42,964 - INFO: train loss: 0.38: 64%|######4 | 182/283 [00:26<00:14, 6.74it/s] |
|
2024-03-19 19:25:45,030 - INFO: train loss: 0.35: 69%|######9 | 196/283 [00:28<00:12, 6.75it/s] |
|
2024-03-19 19:25:46,961 - INFO: train loss: 0.26: 74%|#######4 | 210/283 [00:30<00:10, 6.89it/s] |
|
2024-03-19 19:25:49,009 - INFO: train loss: 0.30: 79%|#######9 | 224/283 [00:32<00:08, 6.88it/s] |
|
2024-03-19 19:25:50,942 - INFO: train loss: 0.33: 84%|########4 | 238/283 [00:34<00:06, 6.98it/s] |
|
2024-03-19 19:25:52,958 - INFO: train loss: 0.32: 89%|########9 | 252/283 [00:36<00:04, 6.97it/s] |
|
2024-03-19 19:25:55,023 - INFO: train loss: 0.46: 94%|#########3| 266/283 [00:38<00:02, 6.91it/s] |
|
2024-03-19 19:25:57,104 - INFO: train loss: 0.46: 99%|#########8| 280/283 [00:40<00:00, 6.86it/s] |
|
2024-03-19 19:25:57,605 - INFO: train loss: 0.47: 100%|##########| 283/283 [00:41<00:00, 6.90it/s] |
|
2024-03-19 19:25:59,385 - INFO: Starting validation inference |
|
2024-03-19 19:25:59,385 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:26:08,329 - INFO: validation progress: 33%|###3 | 1/3 [00:08<00:17, 8.94s/it] |
|
2024-03-19 19:26:21,318 - INFO: validation progress: 67%|######6 | 2/3 [00:21<00:11, 11.32s/it] |
|
2024-03-19 19:26:28,887 - INFO: validation progress: 100%|##########| 3/3 [00:29<00:00, 9.61s/it] |
|
2024-03-19 19:26:28,889 - INFO: validation progress: 100%|##########| 3/3 [00:29<00:00, 9.83s/it] |
|
2024-03-19 19:26:28,912 - INFO: Validation BLEU: 1.33334 |
|
2024-03-19 19:26:29,325 - INFO: Training Epoch: 11 / 15 |
|
2024-03-19 19:26:29,326 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:26:29,346 - INFO: Evaluation step: 283 |
|
2024-03-19 19:26:31,304 - INFO: train loss: 0.26: 5%|4 | 14/283 [00:01<00:38, 7.08it/s] |
|
2024-03-19 19:26:33,251 - INFO: train loss: 0.30: 10%|9 | 28/283 [00:03<00:35, 7.14it/s] |
|
2024-03-19 19:26:35,294 - INFO: train loss: 0.24: 15%|#4 | 42/283 [00:05<00:34, 7.01it/s] |
|
2024-03-19 19:26:37,358 - INFO: train loss: 0.30: 20%|#9 | 56/283 [00:08<00:32, 6.92it/s] |
|
2024-03-19 19:26:39,309 - INFO: train loss: 0.29: 25%|##4 | 70/283 [00:09<00:30, 7.01it/s] |
|
2024-03-19 19:26:41,247 - INFO: train loss: 0.35: 30%|##9 | 84/283 [00:11<00:28, 7.08it/s] |
|
2024-03-19 19:26:43,404 - INFO: train loss: 0.38: 35%|###4 | 98/283 [00:14<00:26, 6.88it/s] |
|
2024-03-19 19:26:45,432 - INFO: train loss: 0.34: 40%|###9 | 112/283 [00:16<00:24, 6.88it/s] |
|
2024-03-19 19:26:47,357 - INFO: train loss: 0.27: 45%|####4 | 126/283 [00:18<00:22, 7.00it/s] |
|
2024-03-19 19:26:49,300 - INFO: train loss: 0.38: 49%|####9 | 140/283 [00:19<00:20, 7.06it/s] |
|
2024-03-19 19:26:51,325 - INFO: train loss: 0.35: 54%|#####4 | 154/283 [00:21<00:18, 7.02it/s] |
|
2024-03-19 19:26:53,285 - INFO: train loss: 0.25: 59%|#####9 | 168/283 [00:23<00:16, 7.05it/s] |
|
2024-03-19 19:26:55,293 - INFO: train loss: 0.26: 64%|######4 | 182/283 [00:25<00:14, 7.03it/s] |
|
2024-03-19 19:26:57,372 - INFO: train loss: 0.44: 69%|######9 | 196/283 [00:28<00:12, 6.94it/s] |
|
2024-03-19 19:26:59,303 - INFO: train loss: 0.24: 74%|#######4 | 210/283 [00:29<00:10, 7.03it/s] |
|
2024-03-19 19:27:01,426 - INFO: train loss: 0.35: 79%|#######9 | 224/283 [00:32<00:08, 6.89it/s] |
|
2024-03-19 19:27:03,480 - INFO: train loss: 0.30: 84%|########4 | 238/283 [00:34<00:06, 6.87it/s] |
|
2024-03-19 19:27:05,527 - INFO: train loss: 0.36: 89%|########9 | 252/283 [00:36<00:04, 6.86it/s] |
|
2024-03-19 19:27:07,587 - INFO: train loss: 0.38: 94%|#########3| 266/283 [00:38<00:02, 6.84it/s] |
|
2024-03-19 19:27:09,515 - INFO: train loss: 0.24: 99%|#########8| 280/283 [00:40<00:00, 6.96it/s] |
|
2024-03-19 19:27:09,913 - INFO: train loss: 0.25: 100%|##########| 283/283 [00:40<00:00, 6.97it/s] |
|
2024-03-19 19:27:11,864 - INFO: Starting validation inference |
|
2024-03-19 19:27:11,864 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:27:23,626 - INFO: validation progress: 33%|###3 | 1/3 [00:11<00:23, 11.76s/it] |
|
2024-03-19 19:27:36,756 - INFO: validation progress: 67%|######6 | 2/3 [00:24<00:12, 12.57s/it] |
|
2024-03-19 19:27:49,789 - INFO: validation progress: 100%|##########| 3/3 [00:37<00:00, 12.78s/it] |
|
2024-03-19 19:27:49,791 - INFO: validation progress: 100%|##########| 3/3 [00:37<00:00, 12.64s/it] |
|
2024-03-19 19:27:49,814 - INFO: Validation BLEU: 1.66384 |
|
2024-03-19 19:27:50,278 - INFO: Training Epoch: 12 / 15 |
|
2024-03-19 19:27:50,278 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:27:50,302 - INFO: Evaluation step: 283 |
|
2024-03-19 19:27:52,356 - INFO: train loss: 0.25: 5%|4 | 14/283 [00:02<00:39, 6.74it/s] |
|
2024-03-19 19:27:54,328 - INFO: train loss: 0.21: 10%|9 | 28/283 [00:04<00:36, 6.95it/s] |
|
2024-03-19 19:27:56,383 - INFO: train loss: 0.24: 15%|#4 | 42/283 [00:06<00:35, 6.88it/s] |
|
2024-03-19 19:27:58,553 - INFO: train loss: 0.30: 20%|#9 | 56/283 [00:08<00:33, 6.71it/s] |
|
2024-03-19 19:28:00,513 - INFO: train loss: 0.26: 25%|##4 | 70/283 [00:10<00:31, 6.86it/s] |
|
2024-03-19 19:28:02,457 - INFO: train loss: 0.21: 30%|##9 | 84/283 [00:12<00:28, 6.97it/s] |
|
2024-03-19 19:28:04,493 - INFO: train loss: 0.30: 35%|###4 | 98/283 [00:14<00:26, 6.94it/s] |
|
2024-03-19 19:28:06,551 - INFO: train loss: 0.34: 40%|###9 | 112/283 [00:16<00:24, 6.89it/s] |
|
2024-03-19 19:28:08,516 - INFO: train loss: 0.25: 45%|####4 | 126/283 [00:18<00:22, 6.97it/s] |
|
2024-03-19 19:28:10,412 - INFO: train loss: 0.38: 49%|####9 | 140/283 [00:20<00:20, 7.09it/s] |
|
2024-03-19 19:28:12,571 - INFO: train loss: 0.34: 54%|#####4 | 154/283 [00:22<00:18, 6.89it/s] |
|
2024-03-19 19:28:14,591 - INFO: train loss: 0.25: 59%|#####9 | 168/283 [00:24<00:16, 6.90it/s] |
|
2024-03-19 19:28:16,598 - INFO: train loss: 0.19: 64%|######4 | 182/283 [00:26<00:14, 6.93it/s] |
|
2024-03-19 19:28:18,659 - INFO: train loss: 0.21: 69%|######9 | 196/283 [00:28<00:12, 6.89it/s] |
|
2024-03-19 19:28:20,702 - INFO: train loss: 0.32: 74%|#######4 | 210/283 [00:30<00:10, 6.87it/s] |
|
2024-03-19 19:28:22,671 - INFO: train loss: 0.23: 79%|#######9 | 224/283 [00:32<00:08, 6.94it/s] |
|
2024-03-19 19:28:24,607 - INFO: train loss: 0.32: 84%|########4 | 238/283 [00:34<00:06, 7.03it/s] |
|
2024-03-19 19:28:26,627 - INFO: train loss: 0.26: 89%|########9 | 252/283 [00:36<00:04, 7.00it/s] |
|
2024-03-19 19:28:28,687 - INFO: train loss: 0.29: 94%|#########3| 266/283 [00:38<00:02, 6.94it/s] |
|
2024-03-19 19:28:30,710 - INFO: train loss: 0.28: 99%|#########8| 280/283 [00:40<00:00, 6.93it/s] |
|
2024-03-19 19:28:31,122 - INFO: train loss: 0.29: 100%|##########| 283/283 [00:40<00:00, 6.93it/s] |
|
2024-03-19 19:28:32,899 - INFO: Starting validation inference |
|
2024-03-19 19:28:32,899 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:28:45,020 - INFO: validation progress: 33%|###3 | 1/3 [00:12<00:24, 12.12s/it] |
|
2024-03-19 19:28:57,914 - INFO: validation progress: 67%|######6 | 2/3 [00:25<00:12, 12.58s/it] |
|
2024-03-19 19:29:05,098 - INFO: validation progress: 100%|##########| 3/3 [00:32<00:00, 10.11s/it] |
|
2024-03-19 19:29:05,101 - INFO: validation progress: 100%|##########| 3/3 [00:32<00:00, 10.73s/it] |
|
2024-03-19 19:29:05,124 - INFO: Validation BLEU: 1.94553 |
|
2024-03-19 19:29:05,533 - INFO: Training Epoch: 13 / 15 |
|
2024-03-19 19:29:05,534 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:29:05,554 - INFO: Evaluation step: 283 |
|
2024-03-19 19:29:07,556 - INFO: train loss: 0.22: 5%|4 | 14/283 [00:02<00:38, 6.92it/s] |
|
2024-03-19 19:29:09,607 - INFO: train loss: 0.29: 10%|9 | 28/283 [00:04<00:37, 6.87it/s] |
|
2024-03-19 19:29:11,579 - INFO: train loss: 0.32: 15%|#4 | 42/283 [00:06<00:34, 6.97it/s] |
|
2024-03-19 19:29:13,831 - INFO: train loss: 0.39: 20%|#9 | 56/283 [00:08<00:34, 6.65it/s] |
|
2024-03-19 19:29:15,898 - INFO: train loss: 0.17: 25%|##4 | 70/283 [00:10<00:31, 6.69it/s] |
|
2024-03-19 19:29:17,870 - INFO: train loss: 0.24: 30%|##9 | 84/283 [00:12<00:29, 6.83it/s] |
|
2024-03-19 19:29:19,949 - INFO: train loss: 0.25: 35%|###4 | 98/283 [00:14<00:27, 6.80it/s] |
|
2024-03-19 19:29:21,961 - INFO: train loss: 0.39: 40%|###9 | 112/283 [00:16<00:24, 6.85it/s] |
|
2024-03-19 19:29:24,007 - INFO: train loss: 0.26: 45%|####4 | 126/283 [00:18<00:22, 6.85it/s] |
|
2024-03-19 19:29:25,925 - INFO: train loss: 0.15: 49%|####9 | 140/283 [00:20<00:20, 6.98it/s] |
|
2024-03-19 19:29:28,038 - INFO: train loss: 0.21: 54%|#####4 | 154/283 [00:22<00:18, 6.87it/s] |
|
2024-03-19 19:29:29,995 - INFO: train loss: 0.23: 59%|#####9 | 168/283 [00:24<00:16, 6.95it/s] |
|
2024-03-19 19:29:31,977 - INFO: train loss: 0.17: 64%|######4 | 182/283 [00:26<00:14, 6.99it/s] |
|
2024-03-19 19:29:33,990 - INFO: train loss: 0.22: 69%|######9 | 196/283 [00:28<00:12, 6.98it/s] |
|
2024-03-19 19:29:36,130 - INFO: train loss: 0.20: 74%|#######4 | 210/283 [00:30<00:10, 6.84it/s] |
|
2024-03-19 19:29:38,098 - INFO: train loss: 0.24: 79%|#######9 | 224/283 [00:32<00:08, 6.92it/s] |
|
2024-03-19 19:29:40,059 - INFO: train loss: 0.20: 84%|########4 | 238/283 [00:34<00:06, 6.98it/s] |
|
2024-03-19 19:29:41,991 - INFO: train loss: 0.20: 89%|########9 | 252/283 [00:36<00:04, 7.06it/s] |
|
2024-03-19 19:29:44,100 - INFO: train loss: 0.18: 94%|#########3| 266/283 [00:38<00:02, 6.93it/s] |
|
2024-03-19 19:29:46,015 - INFO: train loss: 0.22: 99%|#########8| 280/283 [00:40<00:00, 7.04it/s] |
|
2024-03-19 19:29:46,423 - INFO: train loss: 0.23: 100%|##########| 283/283 [00:40<00:00, 6.92it/s] |
|
2024-03-19 19:29:48,218 - INFO: Starting validation inference |
|
2024-03-19 19:29:48,219 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:29:54,912 - INFO: validation progress: 33%|###3 | 1/3 [00:06<00:13, 6.69s/it] |
|
2024-03-19 19:30:08,058 - INFO: validation progress: 67%|######6 | 2/3 [00:19<00:10, 10.49s/it] |
|
2024-03-19 19:30:21,007 - INFO: validation progress: 100%|##########| 3/3 [00:32<00:00, 11.61s/it] |
|
2024-03-19 19:30:21,009 - INFO: validation progress: 100%|##########| 3/3 [00:32<00:00, 10.93s/it] |
|
2024-03-19 19:30:21,032 - INFO: Validation BLEU: 1.89811 |
|
2024-03-19 19:30:21,448 - INFO: Training Epoch: 14 / 15 |
|
2024-03-19 19:30:21,449 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:30:21,472 - INFO: Evaluation step: 283 |
|
2024-03-19 19:30:23,489 - INFO: train loss: 0.20: 5%|4 | 14/283 [00:02<00:39, 6.86it/s] |
|
2024-03-19 19:30:25,459 - INFO: train loss: 0.22: 10%|9 | 28/283 [00:04<00:36, 7.00it/s] |
|
2024-03-19 19:30:27,540 - INFO: train loss: 0.34: 15%|#4 | 42/283 [00:06<00:35, 6.87it/s] |
|
2024-03-19 19:30:29,484 - INFO: train loss: 0.24: 20%|#9 | 56/283 [00:08<00:32, 7.00it/s] |
|
2024-03-19 19:30:31,424 - INFO: train loss: 0.21: 25%|##4 | 70/283 [00:09<00:30, 7.08it/s] |
|
2024-03-19 19:30:33,398 - INFO: train loss: 0.17: 30%|##9 | 84/283 [00:11<00:28, 7.08it/s] |
|
2024-03-19 19:30:35,422 - INFO: train loss: 0.18: 35%|###4 | 98/283 [00:13<00:26, 7.03it/s] |
|
2024-03-19 19:30:37,401 - INFO: train loss: 0.28: 40%|###9 | 112/283 [00:15<00:24, 7.04it/s] |
|
2024-03-19 19:30:39,363 - INFO: train loss: 0.21: 45%|####4 | 126/283 [00:17<00:22, 7.07it/s] |
|
2024-03-19 19:30:41,392 - INFO: train loss: 0.20: 49%|####9 | 140/283 [00:19<00:20, 7.02it/s] |
|
2024-03-19 19:30:43,553 - INFO: train loss: 0.28: 54%|#####4 | 154/283 [00:22<00:18, 6.84it/s] |
|
2024-03-19 19:30:45,574 - INFO: train loss: 0.35: 59%|#####9 | 168/283 [00:24<00:16, 6.87it/s] |
|
2024-03-19 19:30:47,643 - INFO: train loss: 0.28: 64%|######4 | 182/283 [00:26<00:14, 6.84it/s] |
|
2024-03-19 19:30:49,698 - INFO: train loss: 0.22: 69%|######9 | 196/283 [00:28<00:12, 6.83it/s] |
|
2024-03-19 19:30:51,688 - INFO: train loss: 0.28: 74%|#######4 | 210/283 [00:30<00:10, 6.89it/s] |
|
2024-03-19 19:30:53,821 - INFO: train loss: 0.26: 79%|#######9 | 224/283 [00:32<00:08, 6.79it/s] |
|
2024-03-19 19:30:55,882 - INFO: train loss: 0.16: 84%|########4 | 238/283 [00:34<00:06, 6.79it/s] |
|
2024-03-19 19:30:58,156 - INFO: train loss: 0.16: 89%|########9 | 252/283 [00:36<00:04, 6.59it/s] |
|
2024-03-19 19:31:00,115 - INFO: train loss: 0.19: 94%|#########3| 266/283 [00:38<00:02, 6.74it/s] |
|
2024-03-19 19:31:02,048 - INFO: train loss: 0.19: 99%|#########8| 280/283 [00:40<00:00, 6.89it/s] |
|
2024-03-19 19:31:02,445 - INFO: train loss: 0.21: 100%|##########| 283/283 [00:40<00:00, 6.90it/s] |
|
2024-03-19 19:31:04,245 - INFO: Starting validation inference |
|
2024-03-19 19:31:04,245 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:31:17,213 - INFO: validation progress: 33%|###3 | 1/3 [00:12<00:25, 12.97s/it] |
|
2024-03-19 19:31:27,761 - INFO: validation progress: 67%|######6 | 2/3 [00:23<00:11, 11.54s/it] |
|
2024-03-19 19:31:40,504 - INFO: validation progress: 100%|##########| 3/3 [00:36<00:00, 12.09s/it] |
|
2024-03-19 19:31:40,507 - INFO: validation progress: 100%|##########| 3/3 [00:36<00:00, 12.09s/it] |
|
2024-03-19 19:31:40,530 - INFO: Validation BLEU: 3.44231 |
|
2024-03-19 19:31:40,944 - INFO: Training Epoch: 15 / 15 |
|
2024-03-19 19:31:40,944 - INFO: train loss: 0%| | 0/283 [00:00<?, ?it/s] |
|
2024-03-19 19:31:40,969 - INFO: Evaluation step: 283 |
|
2024-03-19 19:31:43,108 - INFO: train loss: 0.28: 5%|4 | 14/283 [00:02<00:41, 6.47it/s] |
|
2024-03-19 19:31:45,119 - INFO: train loss: 0.24: 10%|9 | 28/283 [00:04<00:37, 6.75it/s] |
|
2024-03-19 19:31:47,208 - INFO: train loss: 0.28: 15%|#4 | 42/283 [00:06<00:35, 6.73it/s] |
|
2024-03-19 19:31:49,292 - INFO: train loss: 0.17: 20%|#9 | 56/283 [00:08<00:33, 6.72it/s] |
|
2024-03-19 19:31:51,254 - INFO: train loss: 0.23: 25%|##4 | 70/283 [00:10<00:31, 6.87it/s] |
|
2024-03-19 19:31:53,216 - INFO: train loss: 0.19: 30%|##9 | 84/283 [00:12<00:28, 6.96it/s] |
|
2024-03-19 19:31:55,216 - INFO: train loss: 0.19: 35%|###4 | 98/283 [00:14<00:26, 6.97it/s] |
|
2024-03-19 19:31:57,342 - INFO: train loss: 0.33: 40%|###9 | 112/283 [00:16<00:24, 6.84it/s] |
|
2024-03-19 19:31:59,399 - INFO: train loss: 0.17: 45%|####4 | 126/283 [00:18<00:22, 6.83it/s] |
|
2024-03-19 19:32:01,415 - INFO: train loss: 0.26: 49%|####9 | 140/283 [00:20<00:20, 6.87it/s] |
|
2024-03-19 19:32:03,403 - INFO: train loss: 0.28: 54%|#####4 | 154/283 [00:22<00:18, 6.92it/s] |
|
2024-03-19 19:32:05,397 - INFO: train loss: 0.22: 59%|#####9 | 168/283 [00:24<00:16, 6.95it/s] |
|
2024-03-19 19:32:07,348 - INFO: train loss: 0.23: 64%|######4 | 182/283 [00:26<00:14, 7.02it/s] |
|
2024-03-19 19:32:09,282 - INFO: train loss: 0.19: 69%|######9 | 196/283 [00:28<00:12, 7.08it/s] |
|
2024-03-19 19:32:11,325 - INFO: train loss: 0.26: 74%|#######4 | 210/283 [00:30<00:10, 7.01it/s] |
|
2024-03-19 19:32:13,446 - INFO: train loss: 0.20: 79%|#######9 | 224/283 [00:32<00:08, 6.88it/s] |
|
2024-03-19 19:32:15,549 - INFO: train loss: 0.27: 84%|########4 | 238/283 [00:34<00:06, 6.81it/s] |
|
2024-03-19 19:32:17,521 - INFO: train loss: 0.24: 89%|########9 | 252/283 [00:36<00:04, 6.90it/s] |
|
2024-03-19 19:32:19,528 - INFO: train loss: 0.18: 94%|#########3| 266/283 [00:38<00:02, 6.92it/s] |
|
2024-03-19 19:32:21,506 - INFO: train loss: 0.20: 99%|#########8| 280/283 [00:40<00:00, 6.97it/s] |
|
2024-03-19 19:32:21,908 - INFO: train loss: 0.22: 100%|##########| 283/283 [00:40<00:00, 6.91it/s] |
|
2024-03-19 19:32:23,681 - INFO: Starting validation inference |
|
2024-03-19 19:32:23,681 - INFO: validation progress: 0%| | 0/3 [00:00<?, ?it/s] |
|
2024-03-19 19:32:36,910 - INFO: validation progress: 33%|###3 | 1/3 [00:13<00:26, 13.23s/it] |
|
2024-03-19 19:32:50,069 - INFO: validation progress: 67%|######6 | 2/3 [00:26<00:13, 13.19s/it] |
|
2024-03-19 19:33:02,313 - INFO: validation progress: 100%|##########| 3/3 [00:38<00:00, 12.76s/it] |
|
2024-03-19 19:33:02,316 - INFO: validation progress: 100%|##########| 3/3 [00:38<00:00, 12.88s/it] |
|
2024-03-19 19:33:02,338 - INFO: Validation BLEU: 3.36844 |
|
|
