asun17904/anliR1-gpt2

Knowledge Continuity Regularized Network

Dataset: ANLI Round: None

Trainer Hyperparameters:

• lr = 5e-05
• per_device_batch_size = 8
• gradient_accumulation_steps = 2
• weight_decay = 1e-09
• seed = 42

Regularization Hyperparameters

• numerical stability denominator constant = 1.0
• lambda = 0.0
• alpha = 1.0
• beta = 1.0

Extended Logs:

eval_loss	eval_accuracy	epoch
1.152	0.356	1.0
1.126	0.389	2.0
1.136	0.390	3.0
1.130	0.406	4.0
1.140	0.391	5.0
1.121	0.424	6.0
1.117	0.428	7.0
1.105	0.436	8.0
1.122	0.416	9.0
1.122	0.422	10.0
1.131	0.408	11.0
1.110	0.430	12.0
1.128	0.410	13.0
1.131	0.412	14.0
1.120	0.420	15.0
1.112	0.430	16.0
1.131	0.408	17.0
1.110	0.429	18.0
1.117	0.427	19.0