---
license: apache-2.0
base_model: openai/whisper-large-v2
tags:
- audio
- automatic-speech-recognition
- translate
- generated_from_trainer
language:
- zh
metrics:
- cer
- wer
model-index:
- name: whisper-large-v2-translate-zh-v0.1-lt-ct2
  results: []
---

# whisper-large-v2-translate-zh-v0.1-lt-ct2

This model is a fine-tuned version of [openai/whisper-large-v2](https://huggingface.co/openai/whisper-large-v2).

## Model description

3500小时 (日语音频,中文字幕) 数据微调, 翻译模式直出中文  

CTranslate2 版

## Usage

task='translate', language='ja'  

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 4000
- dropout: 0.1
- mask_time_prob: 0.05
- mask_feature_prob: 0.2
- condition_on_previous_text_rate: 0.5

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Cer    | Wer    |
|:-------------:|:------:|:-----:|:---------------:|:------:|:------:|
| 1.743         | 0.0740 | 1000  | 1.5631          | 0.8223 | 1.4517 |
| 1.6014        | 0.1479 | 2000  | 1.4808          | 0.6775 | 1.0950 |
| 1.5549        | 0.2219 | 3000  | 1.4381          | 0.6756 | 1.1158 |
| 1.5283        | 0.2958 | 4000  | 1.4174          | 0.6992 | 1.1137 |
| 1.474         | 0.3698 | 5000  | 1.3849          | 0.6570 | 1.1369 |
| 1.4193        | 0.4437 | 6000  | 1.3657          | 0.6544 | 1.1339 |
| 1.4148        | 0.5177 | 7000  | 1.3477          | 0.6386 | 1.1647 |
| 1.3754        | 0.5916 | 8000  | 1.3392          | 0.6228 | 1.0461 |
| 1.3441        | 0.6656 | 9000  | 1.3362          | 0.6196 | 1.0609 |
| 1.3545        | 0.7395 | 10000 | 1.3176          | 0.6354 | 1.2138 |
| 1.3498        | 0.8135 | 11000 | 1.3236          | 0.6631 | 1.2232 |
| 1.31          | 0.8874 | 12000 | 1.3020          | 0.6199 | 1.0018 |
| 1.3213        | 0.9614 | 13000 | 1.2966          | 0.5922 | 1.0021 |
| 1.2375        | 1.0353 | 14000 | 1.2900          | 0.6097 | 1.0639 |
| 1.2334        | 1.1093 | 15000 | 1.2963          | 0.6150 | 1.0920 |
| 1.2277        | 1.1832 | 16000 | 1.2888          | 0.6077 | 1.0929 |
| 1.2087        | 1.2572 | 17000 | 1.2779          | 0.5954 | 1.0012 |
| 1.2131        | 1.3311 | 18000 | 1.2722          | 0.5776 | 1.0075 |
| 1.2012        | 1.4051 | 19000 | 1.2716          | 0.5726 | 1.0211 |
| 1.1912        | 1.4790 | 20000 | 1.2707          | 0.6007 | 1.1538 |
| 1.2127        | 1.5530 | 21000 | 1.2749          | 0.6086 | 1.0742 |
| 1.1789        | 1.6269 | 22000 | 1.2797          | 0.5765 | 1.0072 |
| 1.1527        | 1.7009 | 23000 | 1.2761          | 0.5855 | 1.0588 |
| 1.1693        | 1.7748 | 24000 | 1.2701          | 0.5635 | 0.9928 |
| 1.1709        | 1.8488 | 25000 | 1.2662          | 0.5980 | 1.0697 |
| 1.1637        | 1.9227 | 26000 | 1.2749          | 0.5872 | 1.0392 |
| 1.1562        | 1.9967 | 27000 | 1.2587          | 0.5651 | 1.0121 |
| 1.0929        | 2.0706 | 28000 | 1.2668          | 0.5857 | 1.0139 |
| 1.1232        | 2.1446 | 29000 | 1.2710          | 0.5742 | 0.9997 |
| 1.1045        | 2.2185 | 30000 | 1.2656          | 0.5643 | 0.9897 |
| 1.0841        | 2.2925 | 31000 | 1.2695          | 0.5835 | 1.0181 |
| 1.0868        | 2.3664 | 32000 | 1.2707          | 0.5673 | 0.9964 |
| 1.0938        | 2.4404 | 33000 | 1.2644          | 0.5712 | 0.9928 |
| 1.0938        | 2.5143 | 34000 | 1.2662          | 0.5750 | 1.0109 |
| 1.0848        | 2.5883 | 35000 | 1.2677          | 0.5841 | 1.0832 |
| 1.0914        | 2.6622 | 36000 | 1.2638          | 0.5801 | 1.0299 |
| 1.0688        | 2.7362 | 37000 | 1.2587          | 0.5694 | 1.0072 |
| 1.0856        | 2.8101 | 38000 | 1.2581          | 0.5646 | 1.0057 |
| 1.1037        | 2.8841 | 39000 | 1.2557          | 0.5771 | 1.0262 |
| 1.0652        | 2.9580 | 40000 | 1.2566          | 0.5634 | 0.9979 |


### Framework versions

- Transformers 4.41.1
- Pytorch 2.3.1+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1