asun17904/imdb-gpt2-a2b1l1.0

Knowledge Continuity Regularized Network

Trainer Hyperparameters:

• lr = 5e-05
• per_device_batch_size = 8
• gradient_accumulation_steps = 2
• weight_decay = 1e-09
• seed = 42

Regularization Hyperparameters

• numerical stability denominator constant = 0.01
• lambda = 1.0
• alpha = 2.0
• beta = 1.0

Extended Logs:

eval_loss	eval_accuracy	epoch
7.389	0.892	1.0
6.753	0.925	2.0
6.567	0.927	3.0
6.641	0.926	4.0
6.818	0.928	5.0
6.492	0.934	6.0
6.386	0.932	7.0
6.869	0.934	8.0
6.730	0.928	9.0
6.381	0.927	10.0
6.308	0.935	11.0
6.305	0.930	12.0
6.592	0.931	13.0
6.415	0.934	14.0
6.477	0.921	15.0
6.270	0.933	16.0
6.311	0.932	17.0
6.255	0.936	18.0
6.412	0.935	19.0
6.298	0.934	20.0
6.390	0.938	21.0
6.478	0.933	22.0
6.534	0.936	23.0
6.919	0.936	24.0
6.374	0.933	25.0
6.357	0.937	26.0
6.519	0.933	27.0
6.533	0.926	28.0
6.357	0.934	29.0
6.197	0.934	30.0
6.516	0.934	31.0
6.393	0.933	32.0
6.099	0.935	33.0
6.154	0.936	34.0
6.296	0.935	35.0
6.204	0.934	36.0
6.188	0.938	37.0
6.428	0.939	38.0
6.397	0.939	39.0
6.414	0.940	40.0
6.283	0.939	41.0
6.351	0.938	42.0
6.249	0.940	43.0
6.205	0.940	44.0
6.125	0.939	45.0
6.194	0.941	46.0
6.147	0.941	47.0
6.219	0.940	48.0
6.179	0.941	49.0