File size: 4,522 Bytes
fd2e5db |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
2021-05-31 17:50:29,582 INFO __main__ Namespace(adjust_lr=False, config='torchdistill/configs/sample/glue/rte/kd/bert_base_uncased_from_bert_large_uncased.yaml', log='log/glue/rte/kd/bert_base_uncased_from_bert_large_uncased.txt', private_output='leaderboard/glue/kd/bert_base_uncased_from_bert_large_uncased/', seed=None, student_only=False, task_name='rte', test_only=False, world_size=1)
2021-05-31 17:50:29,613 INFO __main__ Distributed environment: NO
Num processes: 1
Process index: 0
Local process index: 0
Device: cuda
Use FP16 precision: True
2021-05-31 17:50:39,943 WARNING datasets.builder Reusing dataset glue (/root/.cache/huggingface/datasets/glue/rte/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad)
2021-05-31 17:50:41,587 INFO __main__ Start training
2021-05-31 17:50:41,587 INFO torchdistill.models.util [teacher model]
2021-05-31 17:50:41,587 INFO torchdistill.models.util Using the original teacher model
2021-05-31 17:50:41,588 INFO torchdistill.models.util [student model]
2021-05-31 17:50:41,588 INFO torchdistill.models.util Using the original student model
2021-05-31 17:50:41,588 INFO torchdistill.core.distillation Loss = 1.0 * OrgLoss
2021-05-31 17:50:41,588 INFO torchdistill.core.distillation Freezing the whole teacher model
2021-05-31 17:50:45,473 INFO torchdistill.misc.log Epoch: [0] [ 0/78] eta: 0:01:02 lr: 9.957264957264958e-05 sample/s: 5.016064534395792 loss: 0.0097 (0.0097) time: 0.8039 data: 0.0065 max mem: 3804
2021-05-31 17:51:25,664 INFO torchdistill.misc.log Epoch: [0] [50/78] eta: 0:00:22 lr: 7.820512820512821e-05 sample/s: 5.000885583822104 loss: 0.0065 (0.0081) time: 0.8035 data: 0.0049 max mem: 5115
2021-05-31 17:51:47,044 INFO torchdistill.misc.log Epoch: [0] Total time: 0:01:02
2021-05-31 17:51:48,268 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow
2021-05-31 17:51:48,268 INFO __main__ Validation: accuracy = 0.6570397111913358
2021-05-31 17:51:48,268 INFO __main__ Updating ckpt at ./resource/ckpt/glue/rte/kd/rte-bert-base-uncased_from_bert-large-uncased
2021-05-31 17:51:50,012 INFO torchdistill.misc.log Epoch: [1] [ 0/78] eta: 0:01:02 lr: 6.623931623931624e-05 sample/s: 5.007037577218317 loss: 0.0042 (0.0042) time: 0.8055 data: 0.0066 max mem: 5115
2021-05-31 17:52:30,204 INFO torchdistill.misc.log Epoch: [1] [50/78] eta: 0:00:22 lr: 4.4871794871794874e-05 sample/s: 4.994946466605415 loss: 0.0033 (0.0032) time: 0.8037 data: 0.0050 max mem: 5115
2021-05-31 17:52:51,720 INFO torchdistill.misc.log Epoch: [1] Total time: 0:01:02
2021-05-31 17:52:52,943 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow
2021-05-31 17:52:52,943 INFO __main__ Validation: accuracy = 0.6859205776173285
2021-05-31 17:52:52,943 INFO __main__ Updating ckpt at ./resource/ckpt/glue/rte/kd/rte-bert-base-uncased_from_bert-large-uncased
2021-05-31 17:52:54,985 INFO torchdistill.misc.log Epoch: [2] [ 0/78] eta: 0:01:03 lr: 3.290598290598291e-05 sample/s: 5.004806067380402 loss: 0.0014 (0.0014) time: 0.8145 data: 0.0153 max mem: 5115
2021-05-31 17:53:35,181 INFO torchdistill.misc.log Epoch: [2] [50/78] eta: 0:00:22 lr: 1.153846153846154e-05 sample/s: 5.00735140278351 loss: 0.0012 (0.0013) time: 0.8039 data: 0.0051 max mem: 5115
2021-05-31 17:53:56,746 INFO torchdistill.misc.log Epoch: [2] Total time: 0:01:02
2021-05-31 17:53:57,970 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow
2021-05-31 17:53:57,970 INFO __main__ Validation: accuracy = 0.6823104693140795
2021-05-31 17:53:58,006 INFO __main__ [Teacher: bert-large-uncased]
2021-05-31 17:54:01,530 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow
2021-05-31 17:54:01,530 INFO __main__ Test: accuracy = 0.740072202166065
2021-05-31 17:54:04,735 INFO __main__ [Student: bert-base-uncased]
2021-05-31 17:54:05,975 INFO /usr/local/lib/python3.7/dist-packages/datasets/metric.py Removing /root/.cache/huggingface/metrics/glue/rte/default_experiment-1-0.arrow
2021-05-31 17:54:05,976 INFO __main__ Test: accuracy = 0.6859205776173285
2021-05-31 17:54:05,976 INFO __main__ Start prediction for private dataset(s)
2021-05-31 17:54:05,977 INFO __main__ rte/test: 3000 samples
|