asun17904/anliR2-t5-base-kd

Knowledge Continuity Regularized Network

Dataset: ANLI Round: None

Trainer Hyperparameters:

• lr = 5e-05
• per_device_batch_size = 16
• gradient_accumulation_steps = 1
• weight_decay = 1e-09
• seed = 42

Regularization Hyperparameters

• numerical stability denominator constant = 0.01
• lambda = 0.001
• alpha = 2.0
• beta = 2.0

Extended Logs:

eval_loss	eval_accuracy	epoch
36.309	0.387	1.0
36.410	0.393	2.0
36.364	0.400	3.0
36.109	0.410	4.0
36.679	0.391	5.0
36.034	0.415	6.0
35.636	0.425	7.0
35.863	0.412	8.0
35.793	0.420	9.0
35.573	0.430	10.0
35.316	0.439	11.0
35.824	0.425	12.0
35.705	0.425	13.0
35.397	0.434	14.0
35.350	0.439	15.0
35.182	0.444	16.0
35.098	0.445	17.0
34.967	0.447	18.0
34.875	0.454	19.0

Test Accuracy: 0.449