README.md

metadata

language: en
license: mit
library_name: pytorch

Knowledge Continuity Regularized Network

Trainer Hyperparameters:

• lr = 5e-05
• per_device_batch_size = 8
• gradient_accumulation_steps = 2
• weight_decay = 1e-09
• seed = 42

Regularization Hyperparameters

• numerical stability denominator constant = 0.001
• lambda = 0.01
• alpha = 2.0
• beta = 2.0

Extended Logs:

eval_loss	eval_accuracy	epoch
14.430	0.792	0.67
14.131	0.792	2.0
13.810	0.875	2.67
13.640	0.875	4.0
13.667	0.875	4.67
13.247	0.875	6.0
12.928	0.875	6.67
12.673	0.875	8.0
12.596	0.875	8.67
12.450	0.875	10.0
12.382	0.875	10.67
12.298	0.875	12.0