File size: 696 Bytes
66db107 82340b6 9729c2c 3e8613f e157a5e dea11da 868245d 74a4780 3ff3c98 c35557e 1599998 c5f4844 775dce1 32315b4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 8
- `gradient_accumulation_steps` = 2
- `weight_decay` = 1e-09
- `seed` = 42
Regularization Hyperparameters
- `numerical stability denominator constant` = 0.001
- `lambda` = 0.01
- `alpha` = 2.0
- `beta` = 2.0
Extended Logs:
|eval_loss|eval_accuracy|epoch|
|--|--|--|
|14.625|0.792|0.67|
|13.757|0.792|2.0|
|13.403|0.792|2.67|
|13.166|0.792|4.0|
|13.001|0.792|4.67|
|12.786|0.792|6.0|
|12.670|0.792|6.67|
|12.480|0.792|8.0|
|12.372|0.792|8.67|
|12.122|0.833|10.0|
|12.015|0.833|10.67|
|11.880|0.833|12.0|
|11.840|0.833|12.67|
|