Update README.md
Browse files
README.md
CHANGED
@@ -19,13 +19,13 @@ model-index:
|
|
19 |
metrics:
|
20 |
- name: Normalized CER
|
21 |
type: cer
|
22 |
-
value:
|
23 |
---
|
24 |
|
25 |
|
26 |
# Wav2Vec2-BERT - Alvin
|
27 |
|
28 |
-
This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0). This has a CER of
|
29 |
|
30 |
## Training and evaluation data
|
31 |
For training, three datasets were used:
|
@@ -65,19 +65,10 @@ predictions = processor.batch_decode(predicted_ids)
|
|
65 |
```
|
66 |
|
67 |
## Training Hyperparameters
|
68 |
-
- learning_rate:
|
69 |
- train_batch_size: 4 (on 1 3090)
|
70 |
- eval_batch_size: 1
|
71 |
- gradient_accumulation_steps: 32
|
72 |
- total_train_batch_size: 32x4=128
|
73 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
74 |
-
- lr_scheduler_warmup_steps:
|
75 |
-
|
76 |
-
## Training Results
|
77 |
-
|
78 |
-
| Training Loss | Step | Validation Loss | CER |
|
79 |
-
|:-------------:|:----:|:---------------:|:------:|
|
80 |
-
|2.416|1200|1.615|0.4246
|
81 |
-
|1.313|4200|0.9049|0.2745
|
82 |
-
|1.090|7200|0.7463|0.2388
|
83 |
-
|0.907|9600|0.6820|0.2172
|
|
|
19 |
metrics:
|
20 |
- name: Normalized CER
|
21 |
type: cer
|
22 |
+
value: 16.26
|
23 |
---
|
24 |
|
25 |
|
26 |
# Wav2Vec2-BERT - Alvin
|
27 |
|
28 |
+
This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0). This has a CER of 16.26
|
29 |
|
30 |
## Training and evaluation data
|
31 |
For training, three datasets were used:
|
|
|
65 |
```
|
66 |
|
67 |
## Training Hyperparameters
|
68 |
+
- learning_rate: 5e-5
|
69 |
- train_batch_size: 4 (on 1 3090)
|
70 |
- eval_batch_size: 1
|
71 |
- gradient_accumulation_steps: 32
|
72 |
- total_train_batch_size: 32x4=128
|
73 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
74 |
+
- lr_scheduler_warmup_steps: 1500
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|