asun17904/imdb-gpt2-a2b1-l0.001

Knowledge Continuity Regularized Network

Trainer Hyperparameters:

• lr = 5e-05
• per_device_batch_size = 8
• gradient_accumulation_steps = 2
• weight_decay = 1e-09
• seed = 42

Regularization Hyperparameters

• numerical stability denominator constant = 0.01
• lambda = 0.001
• alpha = 2.0
• beta = 1.0

Extended Logs:

eval_loss	eval_accuracy	epoch
12.460	0.921	1.0
12.355	0.924	2.0
12.275	0.928	3.0
12.118	0.933	4.0
12.028	0.936	5.0
11.984	0.938	6.0
12.000	0.937	7.0
11.973	0.938	8.0
11.883	0.941	9.0
12.051	0.935	10.0
11.958	0.939	11.0
12.281	0.928	12.0
12.284	0.929	13.0
11.990	0.938	14.0
12.207	0.931	15.0
11.940	0.940	16.0
12.162	0.932	17.0
11.981	0.938	18.0
11.941	0.940	19.0
11.961	0.939	20.0
11.979	0.938	21.0
11.854	0.943	22.0
11.867	0.942	23.0
11.889	0.941	24.0
11.922	0.940	25.0
11.985	0.939	26.0
11.880	0.941	27.0
11.893	0.941	28.0
11.974	0.939	29.0
11.792	0.944	30.0
12.016	0.937	31.0
11.867	0.942	32.0
11.879	0.942	33.0
11.830	0.943	34.0
11.905	0.940	35.0
11.799	0.944	36.0
11.894	0.941	37.0
11.853	0.942	38.0
11.800	0.944	39.0
11.784	0.944	40.0
11.774	0.945	41.0
11.746	0.946	42.0
11.748	0.946	43.0
11.770	0.945	44.0
11.788	0.944	45.0
11.777	0.945	46.0
11.724	0.947	47.0
11.744	0.946	48.0
11.743	0.946	49.0