Hugging Face's logo

pinot
/
wav2vec2-xls-r-300m-ja-cv-14_4

Automatic Speech Recognition
Transformers
PyTorch
wav2vec2
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
pinot commited on
Commit
bfd9c7c
·
1 Parent(s): 0029a00

First model version

Browse files
Files changed (5) hide show
  1. README.md +157 -0
  2. config.json +109 -0
  3. preprocessor_config.json +9 -0
  4. pytorch_model.bin +3 -0
  5. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,157 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: facebook/wav2vec2-xls-r-300m
4
+ tags:
5
+ - generated_from_trainer
6
+ datasets:
7
+ - audiofolder
8
+ metrics:
9
+ - wer
10
+ model-index:
11
+ - name: wav2vec2-xls-r-300m-ja-cv-14_4
12
+ results:
13
+ - task:
14
+ name: Automatic Speech Recognition
15
+ type: automatic-speech-recognition
16
+ dataset:
17
+ name: audiofolder
18
+ type: audiofolder
19
+ config: default
20
+ split: train[:20%]
21
+ args: default
22
+ metrics:
23
+ - name: Wer
24
+ type: wer
25
+ value: 0.05644302449414271
26
+ ---
27
+
28
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
29
+ should probably proofread and complete it, then remove this comment. -->
30
+
31
+ # wav2vec2-xls-r-300m-ja-cv-14_4
32
+
33
+ This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the audiofolder dataset.
34
+ It achieves the following results on the evaluation set:
35
+ - Loss: 0.2072
36
+ - Wer: 0.0564
37
+
38
+ ## Model description
39
+
40
+ More information needed
41
+
42
+ ## Intended uses & limitations
43
+
44
+ More information needed
45
+
46
+ ## Training and evaluation data
47
+
48
+ More information needed
49
+
50
+ ## Training procedure
51
+
52
+ ### Training hyperparameters
53
+
54
+ The following hyperparameters were used during training:
55
+ - learning_rate: 0.0001
56
+ - train_batch_size: 2
57
+ - eval_batch_size: 1
58
+ - seed: 42
59
+ - gradient_accumulation_steps: 4
60
+ - total_train_batch_size: 8
61
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
62
+ - lr_scheduler_type: linear
63
+ - lr_scheduler_warmup_steps: 500
64
+ - num_epochs: 30
65
+
66
+ ### Training results
67
+
68
+ | Training Loss | Epoch | Step | Validation Loss | Wer |
69
+ |:-------------:|:-----:|:------:|:---------------:|:------:|
70
+ | 8.3105 | 0.37 | 2000 | 1.9407 | 0.5234 |
71
+ | 1.6271 | 0.75 | 4000 | 1.1355 | 0.3500 |
72
+ | 1.1075 | 1.12 | 6000 | 0.8428 | 0.2740 |
73
+ | 0.8659 | 1.49 | 8000 | 0.7523 | 0.2491 |
74
+ | 0.7568 | 1.87 | 10000 | 0.6115 | 0.2097 |
75
+ | 0.6269 | 2.24 | 12000 | 0.5771 | 0.1973 |
76
+ | 0.5631 | 2.61 | 14000 | 0.5204 | 0.1713 |
77
+ | 0.5197 | 2.99 | 16000 | 0.4896 | 0.1635 |
78
+ | 0.4218 | 3.36 | 18000 | 0.4506 | 0.1554 |
79
+ | 0.4069 | 3.73 | 20000 | 0.4684 | 0.1579 |
80
+ | 0.3629 | 4.1 | 22000 | 0.4190 | 0.1408 |
81
+ | 0.324 | 4.48 | 24000 | 0.3981 | 0.1337 |
82
+ | 0.3168 | 4.85 | 26000 | 0.3947 | 0.1310 |
83
+ | 0.2845 | 5.22 | 28000 | 0.3974 | 0.1266 |
84
+ | 0.2714 | 5.6 | 30000 | 0.3703 | 0.1166 |
85
+ | 0.2626 | 5.97 | 32000 | 0.3711 | 0.1173 |
86
+ | 0.2274 | 6.34 | 34000 | 0.3991 | 0.1254 |
87
+ | 0.2251 | 6.72 | 36000 | 0.3844 | 0.1159 |
88
+ | 0.2132 | 7.09 | 38000 | 0.3480 | 0.1088 |
89
+ | 0.1867 | 7.46 | 40000 | 0.3330 | 0.1045 |
90
+ | 0.1929 | 7.84 | 42000 | 0.3320 | 0.1054 |
91
+ | 0.1663 | 8.21 | 44000 | 0.3492 | 0.1116 |
92
+ | 0.1668 | 8.58 | 46000 | 0.3580 | 0.1045 |
93
+ | 0.1641 | 8.96 | 48000 | 0.2957 | 0.0920 |
94
+ | 0.1494 | 9.33 | 50000 | 0.3281 | 0.0990 |
95
+ | 0.1495 | 9.7 | 52000 | 0.3155 | 0.0948 |
96
+ | 0.1454 | 10.07 | 54000 | 0.3297 | 0.0994 |
97
+ | 0.1323 | 10.45 | 56000 | 0.3151 | 0.0946 |
98
+ | 0.1321 | 10.82 | 58000 | 0.3073 | 0.0901 |
99
+ | 0.1263 | 11.19 | 60000 | 0.2979 | 0.0887 |
100
+ | 0.1165 | 11.57 | 62000 | 0.3122 | 0.0968 |
101
+ | 0.1179 | 11.94 | 64000 | 0.2941 | 0.0892 |
102
+ | 0.107 | 12.31 | 66000 | 0.2907 | 0.0847 |
103
+ | 0.1037 | 12.69 | 68000 | 0.2964 | 0.0851 |
104
+ | 0.105 | 13.06 | 70000 | 0.2777 | 0.0820 |
105
+ | 0.0942 | 13.43 | 72000 | 0.2758 | 0.0783 |
106
+ | 0.0977 | 13.81 | 74000 | 0.2706 | 0.0768 |
107
+ | 0.0931 | 14.18 | 76000 | 0.2638 | 0.0755 |
108
+ | 0.0881 | 14.55 | 78000 | 0.2835 | 0.0771 |
109
+ | 0.0861 | 14.93 | 80000 | 0.2704 | 0.0776 |
110
+ | 0.0834 | 15.3 | 82000 | 0.2619 | 0.0765 |
111
+ | 0.079 | 15.67 | 84000 | 0.2583 | 0.0759 |
112
+ | 0.0783 | 16.04 | 86000 | 0.2459 | 0.0724 |
113
+ | 0.0753 | 16.42 | 88000 | 0.2647 | 0.0793 |
114
+ | 0.0739 | 16.79 | 90000 | 0.2375 | 0.0696 |
115
+ | 0.0721 | 17.16 | 92000 | 0.2432 | 0.0702 |
116
+ | 0.0687 | 17.54 | 94000 | 0.2269 | 0.0675 |
117
+ | 0.0685 | 17.91 | 96000 | 0.2516 | 0.0724 |
118
+ | 0.066 | 18.28 | 98000 | 0.2372 | 0.0679 |
119
+ | 0.0631 | 18.66 | 100000 | 0.2417 | 0.0670 |
120
+ | 0.0626 | 19.03 | 102000 | 0.2416 | 0.0676 |
121
+ | 0.0583 | 19.4 | 104000 | 0.2491 | 0.0696 |
122
+ | 0.0575 | 19.78 | 106000 | 0.2445 | 0.0675 |
123
+ | 0.0545 | 20.15 | 108000 | 0.2320 | 0.0635 |
124
+ | 0.0517 | 20.52 | 110000 | 0.2312 | 0.0647 |
125
+ | 0.0514 | 20.9 | 112000 | 0.2511 | 0.0676 |
126
+ | 0.0499 | 21.27 | 114000 | 0.2299 | 0.0663 |
127
+ | 0.0486 | 21.64 | 116000 | 0.2400 | 0.0635 |
128
+ | 0.0467 | 22.01 | 118000 | 0.2318 | 0.0624 |
129
+ | 0.0441 | 22.39 | 120000 | 0.2221 | 0.0599 |
130
+ | 0.0441 | 22.76 | 122000 | 0.2359 | 0.0630 |
131
+ | 0.0427 | 23.13 | 124000 | 0.2220 | 0.0603 |
132
+ | 0.0412 | 23.51 | 126000 | 0.2345 | 0.0608 |
133
+ | 0.041 | 23.88 | 128000 | 0.2292 | 0.0598 |
134
+ | 0.0386 | 24.25 | 130000 | 0.2342 | 0.0615 |
135
+ | 0.0376 | 24.63 | 132000 | 0.2291 | 0.0612 |
136
+ | 0.0385 | 25.0 | 134000 | 0.2231 | 0.0631 |
137
+ | 0.0347 | 25.37 | 136000 | 0.2196 | 0.0616 |
138
+ | 0.035 | 25.75 | 138000 | 0.2147 | 0.0608 |
139
+ | 0.0328 | 26.12 | 140000 | 0.2216 | 0.0616 |
140
+ | 0.0318 | 26.49 | 142000 | 0.2195 | 0.0587 |
141
+ | 0.0315 | 26.87 | 144000 | 0.2216 | 0.0594 |
142
+ | 0.0303 | 27.24 | 146000 | 0.2126 | 0.0591 |
143
+ | 0.0292 | 27.61 | 148000 | 0.2126 | 0.0563 |
144
+ | 0.0291 | 27.99 | 150000 | 0.2134 | 0.0574 |
145
+ | 0.0275 | 28.36 | 152000 | 0.2187 | 0.0583 |
146
+ | 0.0281 | 28.73 | 154000 | 0.2098 | 0.0571 |
147
+ | 0.0257 | 29.1 | 156000 | 0.2086 | 0.0564 |
148
+ | 0.0261 | 29.48 | 158000 | 0.2071 | 0.0568 |
149
+ | 0.0247 | 29.85 | 160000 | 0.2072 | 0.0564 |
150
+
151
+
152
+ ### Framework versions
153
+
154
+ - Transformers 4.31.0
155
+ - Pytorch 2.0.1+cu117
156
+ - Datasets 2.14.3
157
+ - Tokenizers 0.13.3
config.json ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "facebook/wav2vec2-xls-r-300m",
3
+ "activation_dropout": 0.0,
4
+ "adapter_attn_dim": null,
5
+ "adapter_kernel_size": 3,
6
+ "adapter_stride": 2,
7
+ "add_adapter": false,
8
+ "apply_spec_augment": true,
9
+ "architectures": [
10
+ "Wav2Vec2ForCTC"
11
+ ],
12
+ "attention_dropout": 0.0,
13
+ "bos_token_id": 1,
14
+ "classifier_proj_size": 256,
15
+ "codevector_dim": 768,
16
+ "contrastive_logits_temperature": 0.1,
17
+ "conv_bias": true,
18
+ "conv_dim": [
19
+ 512,
20
+ 512,
21
+ 512,
22
+ 512,
23
+ 512,
24
+ 512,
25
+ 512
26
+ ],
27
+ "conv_kernel": [
28
+ 10,
29
+ 3,
30
+ 3,
31
+ 3,
32
+ 3,
33
+ 2,
34
+ 2
35
+ ],
36
+ "conv_stride": [
37
+ 5,
38
+ 2,
39
+ 2,
40
+ 2,
41
+ 2,
42
+ 2,
43
+ 2
44
+ ],
45
+ "ctc_loss_reduction": "mean",
46
+ "ctc_zero_infinity": false,
47
+ "diversity_loss_weight": 0.1,
48
+ "do_stable_layer_norm": true,
49
+ "eos_token_id": 2,
50
+ "feat_extract_activation": "gelu",
51
+ "feat_extract_dropout": 0.0,
52
+ "feat_extract_norm": "layer",
53
+ "feat_proj_dropout": 0.0,
54
+ "feat_quantizer_dropout": 0.0,
55
+ "final_dropout": 0.0,
56
+ "gradient_checkpointing": false,
57
+ "hidden_act": "gelu",
58
+ "hidden_dropout": 0.0,
59
+ "hidden_size": 1024,
60
+ "initializer_range": 0.02,
61
+ "intermediate_size": 4096,
62
+ "layer_norm_eps": 1e-05,
63
+ "layerdrop": 0.0,
64
+ "mask_feature_length": 10,
65
+ "mask_feature_min_masks": 0,
66
+ "mask_feature_prob": 0.0,
67
+ "mask_time_length": 10,
68
+ "mask_time_min_masks": 2,
69
+ "mask_time_prob": 0.1,
70
+ "model_type": "wav2vec2",
71
+ "num_adapter_layers": 3,
72
+ "num_attention_heads": 16,
73
+ "num_codevector_groups": 2,
74
+ "num_codevectors_per_group": 320,
75
+ "num_conv_pos_embedding_groups": 16,
76
+ "num_conv_pos_embeddings": 128,
77
+ "num_feat_extract_layers": 7,
78
+ "num_hidden_layers": 24,
79
+ "num_negatives": 100,
80
+ "output_hidden_size": 1024,
81
+ "pad_token_id": 0,
82
+ "proj_codevector_dim": 768,
83
+ "tdnn_dilation": [
84
+ 1,
85
+ 2,
86
+ 3,
87
+ 1,
88
+ 1
89
+ ],
90
+ "tdnn_dim": [
91
+ 512,
92
+ 512,
93
+ 512,
94
+ 512,
95
+ 1500
96
+ ],
97
+ "tdnn_kernel": [
98
+ 5,
99
+ 3,
100
+ 3,
101
+ 1,
102
+ 1
103
+ ],
104
+ "torch_dtype": "float32",
105
+ "transformers_version": "4.31.0",
106
+ "use_weighted_layer_sum": false,
107
+ "vocab_size": 2604,
108
+ "xvector_output_dim": 512
109
+ }
preprocessor_config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_normalize": true,
3
+ "feature_extractor_type": "Wav2Vec2FeatureExtractor",
4
+ "feature_size": 1,
5
+ "padding_side": "right",
6
+ "padding_value": 0.0,
7
+ "return_attention_mask": true,
8
+ "sampling_rate": 16000
9
+ }
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e9628c65c15828e3704e011ec5f3844704b9b57b08e67177ce384fa434b9672b
3
+ size 1272578221
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a8a09cd3b6183bd5499170b587274b4b73fae1882f01edcd1b26fcd7409f1c9
3
+ size 4027