| language: en | |
| license: mit | |
| library_name: pytorch | |
| # Knowledge Continuity Regularized Network | |
| Trainer Hyperparameters: | |
| - `lr` = 1e-05 | |
| - `per_device_batch_size` = 8 | |
| - `gradient_accumulation_steps` = 2 | |
| - `weight_decay` = 1e-09 | |
| - `seed` = 42 | |
| Regularization Hyperparameters | |
| - `numerical stability denominator constant` = 0.01 | |
| - `lambda` = 0.001 | |
| - `alpha` = 2.0 | |
| - `beta` = 1.0 | |
| Extended Logs: | |
| |eval_loss|eval_accuracy|epoch| | |
| |--|--|--| | |
| |5.407|0.792|1.0| | |
| |5.131|0.792|2.0| | |
| |4.966|0.792|3.0| | |
| |4.796|0.792|4.0| | |
| |4.710|0.792|5.0| | |
| |4.653|0.792|6.0| | |
| |4.534|0.792|7.0| | |