gal-lardo commited on
Commit
d9f25b8
·
verified ·
1 Parent(s): 1fbc1a0

Upload BERT-RTE-LinearClassifier for EEE 486/586 Assignment

Browse files
Files changed (1) hide show
  1. README.md +0 -19
README.md CHANGED
@@ -28,25 +28,6 @@ Unlike the standard BERT classification approach, this model implements a custom
28
  - Final classification layer
29
  - Uses label smoothing of 0.1 in the loss function for better generalization
30
 
31
- ## Performance
32
-
33
- The model achieves **70.40%** accuracy on the RTE validation set, with the following training dynamics:
34
- - Best validation accuracy: 70.40% (epoch 3)
35
- - Final validation accuracy: 69.68% (with early stopping)
36
-
37
- ## Hyperparameters
38
-
39
- The model was optimized using Optuna hyperparameter search:
40
-
41
- | Hyperparameter | Value |
42
- |----------------|-------|
43
- | Learning rate | 1.72e-05 |
44
- | Max sequence length | 128 |
45
- | Dropout rate | 0.2 |
46
- | Hidden size multiplier | 2 |
47
- | Weight decay | 0.04 |
48
- | Batch size | 16 |
49
- | Training epochs | 6 (+2 for final model) |
50
 
51
  ## Usage
52
 
 
28
  - Final classification layer
29
  - Uses label smoothing of 0.1 in the loss function for better generalization
30
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
 
32
  ## Usage
33