asun17904/glue-mnli-bert-base-uncased-regularized-l2

Knowledge Continuity Regularized Network

Dataset: GLUE

Trainer Hyperparameters:

• lr = 5e-05
• per_device_batch_size = 16
• gradient_accumulation_steps = 1
• weight_decay = 1e-09
• seed = 42

Regularization Hyperparameters

• numerical stability denominator constant = 0.01
• lambda = 0.02
• alpha = 2.0
• beta = 1.0

Extended Logs:

eval_loss	eval_accuracy	epoch
24.363	0.790	1.0
23.650	0.813	2.0