asun17904/anliR1-t5-base-kd

Knowledge Continuity Regularized Network

Dataset: ANLI Round: None

Trainer Hyperparameters:

• lr = 5e-05
• per_device_batch_size = 8
• gradient_accumulation_steps = 2
• weight_decay = 0.0
• seed = 42

Regularization Hyperparameters

• numerical stability denominator constant = 0.01
• lambda = 0.0001
• alpha = 2.0
• beta = 2.0

Extended Logs:

eval_loss	eval_accuracy	epoch
34.239	0.403	1.0
35.481	0.395	2.0
35.944	0.407	3.0
35.125	0.438	4.0
34.990	0.444	5.0
35.375	0.425	6.0
34.941	0.449	7.0
34.913	0.447	8.0
34.559	0.461	9.0
34.499	0.464	10.0
34.479	0.462	11.0
34.368	0.461	12.0
34.369	0.464	13.0
34.496	0.466	14.0
34.358	0.468	15.0
34.275	0.469	16.0
34.063	0.477	17.0
33.969	0.483	18.0
33.991	0.478	19.0

Test Accuracy: 0.482