marianbasti commited on
Commit
0d8f5da
1 Parent(s): 02881c0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -2
README.md CHANGED
@@ -40,7 +40,7 @@ from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
40
  from datasets import load_dataset
41
  device = "cuda:0" if torch.cuda.is_available() else "cpu"
42
  torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
43
- model_id = "distil-whisper/distil-large-v2"
44
  model = AutoModelForSpeechSeq2Seq.from_pretrained(
45
  model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
46
  )
@@ -155,7 +155,7 @@ print(result["text"])
155
  ```
156
  ## Training
157
 
158
- The model was trained for 40,000 optimisation steps (or four epochs), and the following training parameters:
159
  ```
160
  --teacher_model_name_or_path "openai/whisper-large-v3"
161
  --train_dataset_name "mozilla-foundation/common_voice_16_1"
@@ -174,6 +174,20 @@ The model was trained for 40,000 optimisation steps (or four epochs), and the fo
174
  --logging_steps 25
175
  --save_total_limit 1
176
  --max_steps 40000
 
 
 
 
 
 
 
 
 
 
 
 
 
 
177
  ```
178
 
179
  ## Results
 
40
  from datasets import load_dataset
41
  device = "cuda:0" if torch.cuda.is_available() else "cpu"
42
  torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
43
+ model_id = "marianbasti/distil-whisper-large-v3-es"
44
  model = AutoModelForSpeechSeq2Seq.from_pretrained(
45
  model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
46
  )
 
155
  ```
156
  ## Training
157
 
158
+ The model was trained for 40,000 optimisation steps (or four epochs), on a single RTX3090 for ~30 hours, using the following training parameters:
159
  ```
160
  --teacher_model_name_or_path "openai/whisper-large-v3"
161
  --train_dataset_name "mozilla-foundation/common_voice_16_1"
 
174
  --logging_steps 25
175
  --save_total_limit 1
176
  --max_steps 40000
177
+ --wer_threshold 10
178
+ --per_device_train_batch_size 8
179
+ --per_device_eval_batch_size 8
180
+ --dataloader_num_workers 12
181
+ --preprocessing_num_workers 12
182
+ --output_dir "./"
183
+ --do_train
184
+ --do_eval
185
+ --gradient_checkpointing
186
+ --predict_with_generate
187
+ --overwrite_output_dir
188
+ --use_pseudo_labels "false"
189
+ --freeze_encoder
190
+ --streaming False
191
  ```
192
 
193
  ## Results