File size: 696 Bytes
95a857e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
bd6a717
f6e40b1
6c57562
f88a212
4091a60
ed4e346
071c09b
28674e2
7257a96
ec68134
83778c6
6b6a0d4
88c3a2a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
---
language: en
license: mit
library_name: pytorch
---
# Knowledge Continuity Regularized Network
Trainer Hyperparameters:
- `lr` = 5e-05
- `per_device_batch_size` = 8
- `gradient_accumulation_steps` = 2
- `weight_decay` = 1e-09
- `seed` = 42

Regularization Hyperparameters
- `numerical stability denominator constant` = 0.001
- `lambda` = 0.01
- `alpha` = 2.0
- `beta` = 2.0

Extended Logs:

|eval_loss|eval_accuracy|epoch|
|--|--|--|
|14.430|0.792|0.67|
|14.131|0.792|2.0|
|13.810|0.875|2.67|
|13.640|0.875|4.0|
|13.667|0.875|4.67|
|13.247|0.875|6.0|
|12.928|0.875|6.67|
|12.673|0.875|8.0|
|12.596|0.875|8.67|
|12.450|0.875|10.0|
|12.382|0.875|10.67|
|12.298|0.875|12.0|
|12.289|0.875|12.67|