End of training

README.md

@@ -15,14 +15,14 @@ This student model is distilled from the teacher model [roneneldan/TinyStories-3
 The [Distily](https://github.com/lapp0/distily) library was used for this distillation.
 
 It achieves the following results on the evaluation set:
-- eval_enwikippl: 203.4302
-- eval_frwikippl: 112507.4219
-- eval_zhwikippl: 1544401.25
-- eval_tinystoriesppl: 10.4258
-- eval_loss: 1.2132
-- eval_runtime: 6.5192
-- eval_samples_per_second: 76.696
-- eval_steps_per_second: 9.664
+- eval_enwikippl: 3887.9170
+- eval_frwikippl: 50974.8398
+- eval_zhwikippl: 83822.7812
+- eval_tinystoriesppl: 1011.5359
+- eval_loss: 4.8822
+- eval_runtime: 6.5175
+- eval_samples_per_second: 76.716
+- eval_steps_per_second: 9.666
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment.
@@ -47,7 +47,7 @@ More information needed
 The following hyperparameters were used during training:
 - distillation_objective: DistillationObjective(logits_loss_component=LossComponent(label=logits, weight=1, loss_fn=kl, layer_mapper=None, projector=None), hs_loss_component=LossComponent(label=hs, weight=0, loss_fn=None, layer_mapper=None, projector=None), attn_loss_component=LossComponent(label=attn, weight=0, loss_fn=None, layer_mapper=None, projector=None))
 - train_embeddings: True
-- learning_rate: 0.004
+- learning_rate: 0.0004
 - train_batch_size: 1
 - eval_batch_size: 8
 - seed: 42
@@ -62,106 +62,106 @@ Peak GPU Memory: 6.6058 GB
 | step | epoch | enwikippl | frwikippl | loss | runtime | samples_per_second | steps_per_second | tinystoriesppl | zhwikippl |
 | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 | **teacher eval** |  | 169.9865 | 47377.9414 |  |  |  |  | 3.9789 | 4998.1294 |
-| 0 | 0 | 23232.2363 | 111004.0469 | 6.4068 | 6.5038 | 76.879 | 9.687 | 9550.5166 | 102446.0156 |
-| 500 | 0.0101 | 286.9499 | 206403.0312 | 1.3987 | 6.5068 | 76.843 | 9.682 | 11.3159 | 2106274.75 |
-| 1000 | 0.0202 | 213.8942 | 135439.8906 | 1.2477 | 6.4902 | 77.039 | 9.707 | 10.3927 | 1507759.375 |
-| 1500 | 0.0303 | 202.3927 | 107609.4375 | 1.2191 | 6.4909 | 77.031 | 9.706 | 10.4513 | 1709189.875 |
-| 2000 | 0.0404 | 204.6472 | 114618.8281 | 1.2146 | 6.4939 | 76.996 | 9.701 | 10.4206 | 1537001.75 |
-| 2500 | 0.0505 | 208.5117 | 122800.5781 | 1.2144 | 6.4852 | 77.099 | 9.714 | 10.4021 | 1744199.25 |
-| 3000 | 0.0606 | 203.1310 | 112761.25 | 1.2126 | 6.5107 | 76.796 | 9.676 | 10.3665 | 1447065.25 |
-| 3500 | 0.0707 | 205.6802 | 116737.0 | 1.2140 | 6.5089 | 76.818 | 9.679 | 10.3661 | 1639513.0 |
-| 4000 | 0.0808 | 206.5504 | 110652.8281 | 1.2139 | 6.5215 | 76.669 | 9.66 | 10.4681 | 1485400.5 |
-| 4500 | 0.0909 | 205.1075 | 109474.6094 | 1.2136 | 6.5078 | 76.83 | 9.681 | 10.4159 | 1445137.25 |
-| 5000 | 0.1010 | 203.7140 | 110279.3672 | 1.2138 | 6.5456 | 76.387 | 9.625 | 10.4668 | 1533724.75 |
-| 5500 | 0.1111 | 205.5050 | 114489.6953 | 1.2128 | 6.5317 | 76.55 | 9.645 | 10.4305 | 1672203.5 |
-| 6000 | 0.1212 | 202.5809 | 106884.2891 | 1.2136 | 6.5309 | 76.559 | 9.646 | 10.4206 | 1468064.25 |
-| 6500 | 0.1313 | 202.1420 | 113494.2344 | 1.2131 | 6.5183 | 76.707 | 9.665 | 10.3451 | 1576036.75 |
-| 7000 | 0.1414 | 206.9188 | 116819.1875 | 1.2130 | 6.4962 | 76.968 | 9.698 | 10.4310 | 1638637.625 |
-| 7500 | 0.1515 | 203.4144 | 107367.2109 | 1.2134 | 6.5755 | 76.04 | 9.581 | 10.4859 | 1450158.5 |
-| 8000 | 0.1616 | 203.0209 | 113174.9531 | 1.2135 | 6.5095 | 76.811 | 9.678 | 10.3033 | 1514209.25 |
-| 8500 | 0.1717 | 199.6443 | 100468.0781 | 1.2131 | 6.5168 | 76.725 | 9.667 | 10.4392 | 1311043.875 |
-| 9000 | 0.1818 | 200.8854 | 108095.5391 | 1.2137 | 6.5151 | 76.745 | 9.67 | 10.2138 | 1448225.0 |
-| 9500 | 0.1919 | 203.4302 | 112507.4219 | 1.2132 | 6.5192 | 76.696 | 9.664 | 10.4258 | 1544401.25 |
-| 10000 | 0.2020 | 205.1313 | 110403.75 | 1.2134 | 6.5041 | 76.875 | 9.686 | 10.4660 | 1595496.875 |
-| 10500 | 0.2121 | 204.6551 | 114103.3906 | 1.2130 | 6.5153 | 76.742 | 9.67 | 10.3361 | 1507759.375 |
-| 11000 | 0.2222 | 205.3062 | 115477.6953 | 1.2134 | 6.5075 | 76.834 | 9.681 | 10.4582 | 1510175.625 |
-| 11500 | 0.2323 | 203.7376 | 113942.7812 | 1.2127 | 6.5247 | 76.632 | 9.656 | 10.2722 | 1610896.125 |
-| 12000 | 0.2424 | 202.9187 | 112713.6172 | 1.2131 | 6.5093 | 76.814 | 9.679 | 10.2812 | 1613476.0 |
-| 12500 | 0.2525 | 206.7906 | 114135.5 | 1.2134 | 6.4932 | 77.004 | 9.702 | 10.4301 | 1588700.75 |
-| 13000 | 0.2626 | 204.8058 | 115885.1172 | 1.2133 | 6.5624 | 76.192 | 9.6 | 10.4284 | 1610896.125 |
-| 13500 | 0.2727 | 204.1484 | 111434.8906 | 1.2131 | 6.5157 | 76.738 | 9.669 | 10.4841 | 1585313.625 |
-| 14000 | 0.2828 | 205.4413 | 112888.3203 | 1.2134 | 6.5375 | 76.482 | 9.637 | 10.4280 | 1526377.875 |
-| 14500 | 0.2929 | 205.4494 | 112856.5703 | 1.2134 | 6.5202 | 76.685 | 9.662 | 10.4262 | 1574355.75 |
-| 15000 | 0.3030 | 205.5369 | 111938.2734 | 1.2127 | 6.5102 | 76.802 | 9.677 | 10.4008 | 1513402.25 |
-| 15500 | 0.3131 | 203.0129 | 113366.3672 | 1.2131 | 6.5084 | 76.824 | 9.68 | 10.2892 | 1587854.125 |
-| 16000 | 0.3232 | 205.5687 | 112824.8203 | 1.2131 | 6.5421 | 76.428 | 9.63 | 10.4383 | 1514209.25 |
-| 16500 | 0.3333 | 204.0773 | 109289.7656 | 1.2133 | 6.506 | 76.853 | 9.683 | 10.3910 | 1506954.375 |
-| 17000 | 0.3434 | 203.9982 | 112824.8203 | 1.2133 | 6.5214 | 76.671 | 9.661 | 10.4482 | 1531272.375 |
-| 17500 | 0.3535 | 206.1109 | 113558.2188 | 1.2127 | 6.5276 | 76.598 | 9.651 | 10.3738 | 1565141.625 |
-| 18000 | 0.3636 | 202.1342 | 104355.2578 | 1.2131 | 6.5179 | 76.711 | 9.666 | 10.5219 | 1446294.0 |
-| 18500 | 0.3737 | 204.3779 | 110777.6328 | 1.2133 | 6.5025 | 76.893 | 9.689 | 10.4392 | 1523935.75 |
-| 19000 | 0.3838 | 206.5424 | 116162.8438 | 1.2127 | 6.4979 | 76.948 | 9.695 | 10.3374 | 1635144.0 |
-| 19500 | 0.3939 | 204.2829 | 112127.7031 | 1.2129 | 6.5297 | 76.573 | 9.648 | 10.4223 | 1463371.75 |
-| 20000 | 0.4040 | 202.3927 | 108844.1641 | 1.2132 | 6.5051 | 76.863 | 9.685 | 10.4413 | 1544401.25 |
-| 20500 | 0.4141 | 202.5260 | 112777.1641 | 1.2129 | 6.5005 | 76.917 | 9.692 | 10.2214 | 1525562.875 |
-| 21000 | 0.4242 | 205.3539 | 112380.6719 | 1.2131 | 6.5062 | 76.85 | 9.683 | 10.4495 | 1508563.375 |
-| 21500 | 0.4343 | 204.9645 | 111749.2656 | 1.2127 | 6.5116 | 76.786 | 9.675 | 10.4219 | 1483816.125 |
-| 22000 | 0.4444 | 202.1969 | 112269.9062 | 1.2133 | 6.5089 | 76.817 | 9.679 | 10.2625 | 1589549.5 |
-| 22500 | 0.4545 | 207.1032 | 116425.0312 | 1.2134 | 6.5782 | 76.009 | 9.577 | 10.4021 | 1580246.375 |
-| 23000 | 0.4646 | 202.7693 | 111560.5781 | 1.2130 | 6.5092 | 76.814 | 9.679 | 10.3301 | 1416885.75 |
-| 23500 | 0.4747 | 205.1631 | 115266.4453 | 1.2131 | 6.5057 | 76.855 | 9.684 | 10.3144 | 1592945.75 |
-| 24000 | 0.4848 | 202.1498 | 110730.8438 | 1.2127 | 6.533 | 76.534 | 9.643 | 10.2926 | 1512594.25 |
-| 24500 | 0.4949 | 203.0680 | 111011.8828 | 1.2129 | 6.5139 | 76.759 | 9.672 | 10.4000 | 1447451.75 |
-| 25000 | 0.5051 | 206.4065 | 112254.0625 | 1.2134 | 6.5269 | 76.606 | 9.652 | 10.4297 | 1569322.125 |
-| 25500 | 0.5152 | 202.6124 | 107427.6406 | 1.2126 | 6.5157 | 76.738 | 9.669 | 10.4262 | 1486192.625 |
-| 26000 | 0.5253 | 202.0011 | 105775.9453 | 1.2127 | 6.5572 | 76.253 | 9.608 | 10.4060 | 1452093.125 |
-| 26500 | 0.5354 | 202.9501 | 108034.6328 | 1.2127 | 6.568 | 76.126 | 9.592 | 10.3712 | 1466498.375 |
-| 27000 | 0.5455 | 202.7301 | 108095.5391 | 1.2132 | 6.4898 | 77.044 | 9.708 | 10.3665 | 1505347.0 |
-| 27500 | 0.5556 | 205.7360 | 112507.4219 | 1.2127 | 6.5255 | 76.623 | 9.654 | 10.4413 | 1565141.625 |
-| 28000 | 0.5657 | 202.0011 | 108034.6328 | 1.2129 | 6.5214 | 76.671 | 9.661 | 10.3837 | 1484607.375 |
-| 28500 | 0.5758 | 203.2096 | 108706.2969 | 1.2129 | 6.523 | 76.652 | 9.658 | 10.4331 | 1527191.875 |
-| 29000 | 0.5859 | 203.6351 | 110699.5859 | 1.2126 | 6.5222 | 76.661 | 9.659 | 10.4094 | 1558474.75 |
-| 29500 | 0.5960 | 202.6594 | 107065.1641 | 1.2134 | 6.5256 | 76.622 | 9.654 | 10.3944 | 1468064.25 |
-| 30000 | 0.6061 | 202.6280 | 107306.7109 | 1.2129 | 6.519 | 76.699 | 9.664 | 10.4103 | 1475918.5 |
-| 30500 | 0.6162 | 202.7379 | 108645.0469 | 1.2128 | 6.5228 | 76.654 | 9.658 | 10.4185 | 1483816.125 |
-| 31000 | 0.6263 | 203.3198 | 109120.5234 | 1.2133 | 6.4945 | 76.988 | 9.7 | 10.4301 | 1528822.5 |
-| 31500 | 0.6364 | 203.8245 | 111875.3047 | 1.2126 | 6.4967 | 76.962 | 9.697 | 10.4008 | 1555152.125 |
-| 32000 | 0.6465 | 202.9107 | 107730.7109 | 1.2131 | 6.493 | 77.007 | 9.703 | 10.4219 | 1534544.125 |
-| 32500 | 0.6566 | 201.8682 | 108339.4062 | 1.2131 | 6.4912 | 77.027 | 9.705 | 10.3674 | 1543578.125 |
-| 33000 | 0.6667 | 203.7298 | 109922.6797 | 1.2126 | 6.4859 | 77.091 | 9.713 | 10.3948 | 1507759.375 |
-| 33500 | 0.6768 | 203.3198 | 109582.6172 | 1.2127 | 6.4963 | 76.967 | 9.698 | 10.3961 | 1475918.5 |
-| 34000 | 0.6869 | 204.9724 | 111309.4609 | 1.2127 | 6.4903 | 77.038 | 9.707 | 10.4452 | 1569322.125 |
-| 34500 | 0.6970 | 203.1310 | 108828.9062 | 1.2128 | 6.4917 | 77.021 | 9.705 | 10.3918 | 1510175.625 |
-| 35000 | 0.7071 | 202.9815 | 109197.3516 | 1.2128 | 6.5052 | 76.861 | 9.685 | 10.3639 | 1523935.75 |
-| 35500 | 0.7172 | 203.1467 | 109135.9297 | 1.2131 | 6.4998 | 76.925 | 9.693 | 10.4056 | 1473558.5 |
-| 36000 | 0.7273 | 203.3513 | 109151.2266 | 1.2131 | 6.5407 | 76.444 | 9.632 | 10.4163 | 1507759.375 |
-| 36500 | 0.7374 | 202.7301 | 110808.8047 | 1.2131 | 6.511 | 76.794 | 9.676 | 10.3554 | 1549353.625 |
-| 37000 | 0.7475 | 202.9422 | 109258.9141 | 1.2129 | 6.5252 | 76.626 | 9.655 | 10.3987 | 1519876.25 |
-| 37500 | 0.7576 | 203.2096 | 110139.6875 | 1.2127 | 6.4909 | 77.031 | 9.706 | 10.4038 | 1506954.375 |
-| 38000 | 0.7677 | 204.0456 | 109860.7422 | 1.2125 | 6.5271 | 76.603 | 9.652 | 10.4392 | 1483024.0 |
-| 38500 | 0.7778 | 203.1624 | 109197.3516 | 1.2129 | 6.5222 | 76.661 | 9.659 | 10.3837 | 1487779.5 |
-| 39000 | 0.7879 | 203.2883 | 109274.3359 | 1.2126 | 6.703 | 74.593 | 9.399 | 10.4038 | 1519064.75 |
-| 39500 | 0.7980 | 203.1152 | 109089.8281 | 1.2129 | 6.5 | 76.923 | 9.692 | 10.4211 | 1506151.25 |
-| 40000 | 0.8081 | 203.2411 | 109551.6875 | 1.2128 | 6.5023 | 76.896 | 9.689 | 10.4129 | 1498136.125 |
-| 40500 | 0.8182 | 203.1467 | 109767.9531 | 1.2127 | 6.5007 | 76.915 | 9.691 | 10.4167 | 1483816.125 |
-| 41000 | 0.8283 | 202.9107 | 108767.5859 | 1.2131 | 6.5147 | 76.75 | 9.671 | 10.4017 | 1498935.0 |
-| 41500 | 0.8384 | 202.9736 | 109151.2266 | 1.2126 | 6.5167 | 76.726 | 9.667 | 10.4038 | 1485400.5 |
-| 42000 | 0.8485 | 203.2883 | 109644.3984 | 1.2127 | 6.5139 | 76.759 | 9.672 | 10.4051 | 1476707.0 |
-| 42500 | 0.8586 | 202.5966 | 108767.5859 | 1.2126 | 6.5096 | 76.81 | 9.678 | 10.4069 | 1475918.5 |
-| 43000 | 0.8687 | 204.2354 | 110668.4453 | 1.2127 | 6.5038 | 76.878 | 9.687 | 10.4211 | 1520686.625 |
-| 43500 | 0.8788 | 203.8402 | 111403.5469 | 1.2124 | 6.541 | 76.44 | 9.631 | 10.3708 | 1523123.625 |
-| 44000 | 0.8889 | 204.0615 | 111215.3359 | 1.2126 | 6.5392 | 76.462 | 9.634 | 10.3884 | 1531272.375 |
-| 44500 | 0.8990 | 203.8245 | 110263.9062 | 1.2126 | 6.5434 | 76.413 | 9.628 | 10.3875 | 1507759.375 |
-| 45000 | 0.9091 | 203.8718 | 110668.4453 | 1.2125 | 6.5546 | 76.283 | 9.612 | 10.3854 | 1514209.25 |
-| 45500 | 0.9192 | 203.0051 | 109953.7188 | 1.2126 | 6.52 | 76.687 | 9.663 | 10.3824 | 1494942.0 |
-| 46000 | 0.9293 | 203.7772 | 110606.0938 | 1.2126 | 6.4887 | 77.058 | 9.709 | 10.4064 | 1504544.75 |
-| 46500 | 0.9394 | 203.8087 | 110606.0938 | 1.2127 | 6.502 | 76.899 | 9.689 | 10.4051 | 1504544.75 |
-| 47000 | 0.9495 | 203.3671 | 109953.7188 | 1.2126 | 6.5034 | 76.883 | 9.687 | 10.3957 | 1504544.75 |
-| 47500 | 0.9596 | 203.6351 | 110357.1172 | 1.2126 | 6.4876 | 77.07 | 9.711 | 10.4004 | 1506954.375 |
-| 48000 | 0.9697 | 203.2254 | 110512.6719 | 1.2124 | 6.5074 | 76.835 | 9.681 | 10.3837 | 1504544.75 |
-| 48500 | 0.9798 | 203.6509 | 110606.0938 | 1.2124 | 6.504 | 76.876 | 9.686 | 10.4060 | 1504544.75 |
-| 49000 | 0.9899 | 203.7140 | 110606.0938 | 1.2126 | 6.5071 | 76.839 | 9.682 | 10.4090 | 1507759.375 |
-| 49500 | 1.0 | 203.7456 | 110606.0938 | 1.2126 | 6.5184 | 76.706 | 9.665 | 10.4090 | 1507759.375 |
+| 0 | 0 | 23232.2363 | 111004.0469 | 6.4068 | 6.5373 | 76.484 | 9.637 | 9550.5166 | 102446.0156 |
+| 500 | 0.0101 | 3972.8616 | 51053.8984 | 4.8833 | 6.5063 | 76.848 | 9.683 | 1038.9088 | 84361.2188 |
+| 1000 | 0.0202 | 3908.7549 | 50974.8398 | 4.8822 | 6.5275 | 76.599 | 9.651 | 1016.2294 | 83867.4844 |
+| 1500 | 0.0303 | 3887.6167 | 50974.8398 | 4.8822 | 6.5158 | 76.737 | 9.669 | 1010.1989 | 83867.4844 |
+| 2000 | 0.0404 | 3873.1880 | 50989.1836 | 4.8822 | 6.5149 | 76.747 | 9.67 | 1005.8657 | 83822.7812 |
+| 2500 | 0.0505 | 3897.8667 | 50974.8398 | 4.8822 | 6.5124 | 76.777 | 9.674 | 1014.4666 | 83822.7812 |
+| 3000 | 0.0606 | 3906.9363 | 50974.8398 | 4.8822 | 6.5204 | 76.682 | 9.662 | 1016.0612 | 83822.7812 |
+| 3500 | 0.0707 | 3893.6423 | 50974.8398 | 4.8822 | 6.5097 | 76.809 | 9.678 | 1012.0377 | 83822.7812 |
+| 4000 | 0.0808 | 3894.2476 | 50974.8398 | 4.8822 | 6.511 | 76.793 | 9.676 | 1013.2932 | 83822.7812 |
+| 4500 | 0.0909 | 3917.8462 | 51010.7305 | 4.8822 | 6.5167 | 76.726 | 9.668 | 1019.4268 | 83867.4844 |
+| 5000 | 0.1010 | 3912.3875 | 51003.5820 | 4.8822 | 6.5075 | 76.834 | 9.681 | 1017.7424 | 83867.4844 |
+| 5500 | 0.1111 | 3903.9119 | 50974.8398 | 4.8822 | 6.5226 | 76.656 | 9.659 | 1015.6412 | 83822.7812 |
+| 6000 | 0.1212 | 3912.3875 | 51003.5820 | 4.8822 | 6.5212 | 76.673 | 9.661 | 1017.4061 | 83867.4844 |
+| 6500 | 0.1313 | 3936.0994 | 51068.2656 | 4.8822 | 6.4984 | 76.942 | 9.695 | 1025.1731 | 83957.0312 |
+| 7000 | 0.1414 | 3922.1003 | 51053.8984 | 4.8822 | 6.5149 | 76.747 | 9.67 | 1020.9445 | 83912.2812 |
+| 7500 | 0.1515 | 3914.2085 | 51010.7305 | 4.8822 | 6.5141 | 76.757 | 9.671 | 1018.9213 | 83867.4844 |
+| 8000 | 0.1616 | 3907.5435 | 50974.8398 | 4.8822 | 6.5148 | 76.748 | 9.67 | 1016.2294 | 83867.4844 |
+| 8500 | 0.1717 | 3913.6001 | 51003.5820 | 4.8822 | 6.5271 | 76.603 | 9.652 | 1018.0792 | 83867.4844 |
+| 9000 | 0.1818 | 3897.8667 | 50974.8398 | 4.8822 | 6.5537 | 76.293 | 9.613 | 1014.1313 | 83822.7812 |
+| 9500 | 0.1919 | 3887.9170 | 50974.8398 | 4.8822 | 6.5175 | 76.716 | 9.666 | 1011.5359 | 83822.7812 |
+| 10000 | 0.2020 | 3882.1963 | 50960.4531 | 4.8822 | 6.5134 | 76.765 | 9.672 | 1008.7803 | 83822.7812 |
+| 10500 | 0.2121 | 3912.3875 | 51003.5820 | 4.8822 | 6.5102 | 76.802 | 9.677 | 1017.5745 | 83867.4844 |
+| 11000 | 0.2222 | 3938.5400 | 51068.2656 | 4.8822 | 6.4952 | 76.98 | 9.699 | 1025.5123 | 83957.0312 |
+| 11500 | 0.2323 | 3939.7610 | 51068.2656 | 4.8822 | 6.4945 | 76.988 | 9.7 | 1028.0588 | 84136.4141 |
+| 12000 | 0.2424 | 3951.9873 | 51097.0547 | 4.8822 | 6.5097 | 76.809 | 9.678 | 1029.9302 | 84136.4141 |
+| 12500 | 0.2525 | 3922.1003 | 51053.8984 | 4.8822 | 6.5036 | 76.881 | 9.687 | 1021.7891 | 83912.2812 |
+| 13000 | 0.2626 | 3897.8667 | 50974.8398 | 4.8822 | 6.5248 | 76.631 | 9.655 | 1014.6344 | 83822.7812 |
+| 13500 | 0.2727 | 3887.6167 | 50974.8398 | 4.8822 | 6.5036 | 76.881 | 9.687 | 1010.3655 | 83822.7812 |
+| 14000 | 0.2828 | 3867.7913 | 50989.1836 | 4.8822 | 6.5118 | 76.783 | 9.675 | 1001.6337 | 83778.0312 |
+| 14500 | 0.2929 | 3860.6052 | 50989.1836 | 4.8822 | 6.5064 | 76.847 | 9.683 | 999.4833 | 83778.0312 |
+| 15000 | 0.3030 | 3873.1880 | 50960.4531 | 4.8822 | 6.5104 | 76.8 | 9.677 | 1005.8657 | 83822.7812 |
+| 15500 | 0.3131 | 3884.0034 | 50960.4531 | 4.8822 | 6.5186 | 76.704 | 9.665 | 1009.1969 | 83822.7812 |
+| 16000 | 0.3232 | 3876.7874 | 50989.1836 | 4.8822 | 6.5138 | 76.76 | 9.672 | 1006.6141 | 83822.7812 |
+| 16500 | 0.3333 | 3902.7021 | 50974.8398 | 4.8822 | 6.5538 | 76.291 | 9.613 | 1015.1377 | 83822.7812 |
+| 17000 | 0.3434 | 3894.2476 | 50974.8398 | 4.8822 | 6.5508 | 76.326 | 9.617 | 1013.2932 | 83822.7812 |
+| 17500 | 0.3535 | 3895.4548 | 50974.8398 | 4.8822 | 6.5145 | 76.752 | 9.671 | 1013.4609 | 83822.7812 |
+| 18000 | 0.3636 | 3892.4358 | 50974.8398 | 4.8822 | 6.5208 | 76.678 | 9.661 | 1011.7029 | 83822.7812 |
+| 18500 | 0.3737 | 3895.4548 | 50974.8398 | 4.8822 | 6.506 | 76.852 | 9.683 | 1013.4609 | 83822.7812 |
+| 19000 | 0.3838 | 3922.1003 | 51053.8984 | 4.8822 | 6.4973 | 76.955 | 9.696 | 1021.7891 | 83912.2812 |
+| 19500 | 0.3939 | 3931.2227 | 51068.2656 | 4.8822 | 6.4889 | 77.055 | 9.709 | 1023.6490 | 83957.0312 |
+| 20000 | 0.4040 | 3926.9622 | 51068.2656 | 4.8822 | 6.4907 | 77.034 | 9.706 | 1023.3103 | 83912.2812 |
+| 20500 | 0.4141 | 3922.1003 | 51053.8984 | 4.8822 | 6.4861 | 77.088 | 9.713 | 1021.7891 | 83912.2812 |
+| 21000 | 0.4242 | 3922.1003 | 51053.8984 | 4.8822 | 6.4983 | 76.943 | 9.695 | 1021.7891 | 83912.2812 |
+| 21500 | 0.4343 | 3931.2227 | 51068.2656 | 4.8822 | 6.509 | 76.817 | 9.679 | 1024.8344 | 83957.0312 |
+| 22000 | 0.4444 | 3938.5400 | 51068.2656 | 4.8822 | 6.5034 | 76.883 | 9.687 | 1026.6998 | 84091.5703 |
+| 22500 | 0.4545 | 3931.2227 | 51068.2656 | 4.8822 | 6.5235 | 76.646 | 9.657 | 1024.8344 | 83957.0312 |
+| 23000 | 0.4646 | 3922.1003 | 51053.8984 | 4.8822 | 6.5476 | 76.363 | 9.622 | 1021.4509 | 83867.4844 |
+| 23500 | 0.4747 | 3920.8850 | 51053.8984 | 4.8822 | 6.4853 | 77.098 | 9.714 | 1020.6072 | 83867.4844 |
+| 24000 | 0.4848 | 3919.6699 | 51053.8984 | 4.8822 | 6.4956 | 76.975 | 9.699 | 1020.4384 | 83867.4844 |
+| 24500 | 0.4949 | 3921.4907 | 51053.8984 | 4.8822 | 6.5036 | 76.88 | 9.687 | 1020.7761 | 83867.4844 |
+| 25000 | 0.5051 | 3919.6699 | 51053.8984 | 4.8822 | 6.4891 | 77.052 | 9.709 | 1020.6072 | 83867.4844 |
+| 25500 | 0.5152 | 3922.1003 | 51053.8984 | 4.8822 | 6.4927 | 77.01 | 9.703 | 1020.9445 | 83867.4844 |
+| 26000 | 0.5253 | 3926.9622 | 51068.2656 | 4.8822 | 6.4956 | 76.975 | 9.699 | 1022.4650 | 83912.2812 |
+| 26500 | 0.5354 | 3931.2227 | 51068.2656 | 4.8822 | 6.5008 | 76.914 | 9.691 | 1024.1567 | 83957.0312 |
+| 27000 | 0.5455 | 3926.9622 | 51068.2656 | 4.8822 | 6.5048 | 76.867 | 9.685 | 1022.2959 | 83912.2812 |
+| 27500 | 0.5556 | 3922.1003 | 51053.8984 | 4.8822 | 6.526 | 76.617 | 9.654 | 1021.7891 | 83912.2812 |
+| 28000 | 0.5657 | 3921.4907 | 51053.8984 | 4.8822 | 6.5337 | 76.527 | 9.642 | 1020.7761 | 83867.4844 |
+| 28500 | 0.5758 | 3917.8462 | 51053.8984 | 4.8822 | 6.4962 | 76.969 | 9.698 | 1019.5950 | 83867.4844 |
+| 29000 | 0.5859 | 3917.8462 | 51039.4883 | 4.8822 | 6.4943 | 76.99 | 9.701 | 1019.4268 | 83867.4844 |
+| 29500 | 0.5960 | 3919.6699 | 51053.8984 | 4.8822 | 6.5069 | 76.841 | 9.682 | 1020.4384 | 83867.4844 |
+| 30000 | 0.6061 | 3919.6699 | 51053.8984 | 4.8822 | 6.5033 | 76.884 | 9.687 | 1020.4384 | 83867.4844 |
+| 30500 | 0.6162 | 3917.8462 | 51039.4883 | 4.8822 | 6.4958 | 76.972 | 9.699 | 1019.4268 | 83867.4844 |
+| 31000 | 0.6263 | 3917.8462 | 51039.4883 | 4.8822 | 6.4827 | 77.129 | 9.718 | 1019.4268 | 83867.4844 |
+| 31500 | 0.6364 | 3917.8462 | 51039.4883 | 4.8822 | 6.495 | 76.982 | 9.7 | 1019.4268 | 83867.4844 |
+| 32000 | 0.6465 | 3918.4551 | 51053.8984 | 4.8822 | 6.5193 | 76.696 | 9.664 | 1019.5950 | 83867.4844 |
+| 32500 | 0.6566 | 3921.4907 | 51053.8984 | 4.8822 | 6.5213 | 76.671 | 9.661 | 1020.7761 | 83867.4844 |
+| 33000 | 0.6667 | 3922.1003 | 51053.8984 | 4.8822 | 6.5 | 76.923 | 9.692 | 1021.4509 | 83867.4844 |
+| 33500 | 0.6768 | 3922.1003 | 51053.8984 | 4.8822 | 6.5038 | 76.878 | 9.687 | 1021.7891 | 83912.2812 |
+| 34000 | 0.6869 | 3922.1003 | 51053.8984 | 4.8822 | 6.5965 | 75.798 | 9.551 | 1021.6200 | 83867.4844 |
+| 34500 | 0.6970 | 3922.1003 | 51053.8984 | 4.8822 | 6.4926 | 77.01 | 9.703 | 1020.7761 | 83867.4844 |
+| 35000 | 0.7071 | 3921.4907 | 51053.8984 | 4.8822 | 6.5061 | 76.851 | 9.683 | 1020.7761 | 83867.4844 |
+| 35500 | 0.7172 | 3921.4907 | 51053.8984 | 4.8822 | 6.5289 | 76.582 | 9.649 | 1020.7761 | 83867.4844 |
+| 36000 | 0.7273 | 3922.1003 | 51053.8984 | 4.8822 | 6.5582 | 76.24 | 9.606 | 1021.4509 | 83867.4844 |
+| 36500 | 0.7374 | 3922.1003 | 51053.8984 | 4.8822 | 6.5354 | 76.506 | 9.64 | 1021.7891 | 83912.2812 |
+| 37000 | 0.7475 | 3924.5286 | 51053.8984 | 4.8822 | 6.5215 | 76.669 | 9.66 | 1021.7891 | 83912.2812 |
+| 37500 | 0.7576 | 3926.9622 | 51053.8984 | 4.8822 | 6.5007 | 76.915 | 9.691 | 1021.7891 | 83912.2812 |
+| 38000 | 0.7677 | 3924.5286 | 51053.8984 | 4.8822 | 6.5068 | 76.842 | 9.682 | 1021.7891 | 83912.2812 |
+| 38500 | 0.7778 | 3922.1003 | 51053.8984 | 4.8822 | 6.5229 | 76.653 | 9.658 | 1021.6200 | 83867.4844 |
+| 39000 | 0.7879 | 3922.1003 | 51053.8984 | 4.8822 | 6.5165 | 76.728 | 9.668 | 1020.9445 | 83867.4844 |
+| 39500 | 0.7980 | 3920.8850 | 51053.8984 | 4.8822 | 6.5119 | 76.782 | 9.675 | 1020.6072 | 83867.4844 |
+| 40000 | 0.8081 | 3920.8850 | 51053.8984 | 4.8822 | 6.5092 | 76.814 | 9.679 | 1020.6072 | 83867.4844 |
+| 40500 | 0.8182 | 3920.8850 | 51053.8984 | 4.8822 | 6.5191 | 76.697 | 9.664 | 1020.7761 | 83867.4844 |
+| 41000 | 0.8283 | 3920.8850 | 51053.8984 | 4.8822 | 6.5316 | 76.551 | 9.645 | 1020.7761 | 83867.4844 |
+| 41500 | 0.8384 | 3920.8850 | 51053.8984 | 4.8822 | 6.5046 | 76.869 | 9.685 | 1020.7761 | 83867.4844 |
+| 42000 | 0.8485 | 3920.8850 | 51053.8984 | 4.8822 | 6.5038 | 76.878 | 9.687 | 1020.6072 | 83867.4844 |
+| 42500 | 0.8586 | 3920.8850 | 51053.8984 | 4.8822 | 6.5215 | 76.669 | 9.66 | 1020.6072 | 83867.4844 |
+| 43000 | 0.8687 | 3920.8850 | 51053.8984 | 4.8822 | 6.5049 | 76.865 | 9.685 | 1020.6072 | 83867.4844 |
+| 43500 | 0.8788 | 3920.8850 | 51053.8984 | 4.8822 | 6.5074 | 76.836 | 9.681 | 1020.6072 | 83867.4844 |
+| 44000 | 0.8889 | 3920.8850 | 51053.8984 | 4.8822 | 6.4973 | 76.956 | 9.696 | 1020.6072 | 83867.4844 |
+| 44500 | 0.8990 | 3920.8850 | 51053.8984 | 4.8822 | 6.529 | 76.581 | 9.649 | 1020.6072 | 83867.4844 |
+| 45000 | 0.9091 | 3920.8850 | 51053.8984 | 4.8822 | 6.5231 | 76.651 | 9.658 | 1020.6072 | 83867.4844 |
+| 45500 | 0.9192 | 3920.8850 | 51053.8984 | 4.8822 | 6.5386 | 76.469 | 9.635 | 1020.6072 | 83867.4844 |
+| 46000 | 0.9293 | 3920.8850 | 51053.8984 | 4.8822 | 6.5266 | 76.61 | 9.653 | 1020.6072 | 83867.4844 |
+| 46500 | 0.9394 | 3920.8850 | 51053.8984 | 4.8822 | 6.4999 | 76.924 | 9.692 | 1020.6072 | 83867.4844 |
+| 47000 | 0.9495 | 3920.8850 | 51053.8984 | 4.8822 | 6.5248 | 76.63 | 9.655 | 1020.6072 | 83867.4844 |
+| 47500 | 0.9596 | 3920.8850 | 51053.8984 | 4.8822 | 6.5217 | 76.668 | 9.66 | 1020.6072 | 83867.4844 |
+| 48000 | 0.9697 | 3920.8850 | 51053.8984 | 4.8822 | 6.5043 | 76.872 | 9.686 | 1020.6072 | 83867.4844 |
+| 48500 | 0.9798 | 3920.8850 | 51053.8984 | 4.8822 | 6.4865 | 77.084 | 9.713 | 1020.6072 | 83867.4844 |
+| 49000 | 0.9899 | 3920.8850 | 51053.8984 | 4.8822 | 6.5203 | 76.684 | 9.662 | 1020.6072 | 83867.4844 |
+| 49500 | 1.0 | 3920.8850 | 51053.8984 | 4.8822 | 6.6387 | 75.315 | 9.49 | 1020.6072 | 83867.4844 |
 
 ### Framework versions
 - Distily 0.2.0

logs/dropout=0, learning_rate=0.0004, per_device_train_batch_size=1, weight_decay=0/events.out.tfevents.1723938921.5f530b1cf724

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10986c8798cc7b549ad27168a0c106ee94a699450433741614d802e28c8b20b6
+size 312