File size: 2,906 Bytes
289ef62 0dc6445 289ef62 03d6080 289ef62 6245d82 289ef62 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
---
license: apache-2.0
---
# Whisper Medium ATC full
This model is a fine-tuned [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on Czech and English air traffic communication recordings from Czech airport LKKU.
It was created as a product of bachelor's thesis at Faculty of Information Technology Brno University of Technology.
# Model description
- **Developed by:** Veronika Nevarilova ([@xnevar00](https://huggingface.co/xnevar00)), Igor Szoke ([@iszoke](https://huggingface.co/iszoke))
- **Shared by:** [BUT FIT](https://huggingface.co/BUT-FIT)
- **Model type:** Whisper
- **Languages:** Czech, English
- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
- **Finetuned from model:** [openai/whisper-medium](https://huggingface.co/openai/whisper-medium)
# Usage
```python
import torch
from transformers import pipeline
audio = "path/to/audio.xx"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
transcribe = pipeline(task="automatic-speech-recognition", model="BUT-FIT/whisper-ATC-czech-full", chunk_length_s=30, device=device)
transcribe.model.config.forced_decoder_ids = transcribe.tokenizer.get_decoder_prompt_ids(task="transcribe", language="czech")
print('Transcription:', transcribe(audio)["text"])
```
# Dataset
Training dataset was made of ~5 hours of air traffic communication recordings. Recordings were Czech and English (80:20) and sporadically Slovak.
# Output format
The model was learned to transcribe every recording word by word. Transcription format of a recording is as follows:
Recording: *Oscar Kilo Alpha Bravo Charlie dráha dva nula střední pro přistání volná vítr nula jedna nula stupňů pět uzlů*
Transcription: `Oscar Kilo Alpha Bravo Charlie dráha dva nula střední pro přistání volná vítr nula jedna nula stupňů pět uzlů`
**Note:** See also model [BUT-FIT/whisper-ATC-czech-short](https://huggingface.co/BUT-FIT/whisper-ATC-czech-short), which abbreviates callsigns and numbers.
# Results
The model reached total WER of 14.7 % on unseen Czech and English LKKU recordings. 19.6 % WER was achieved on a testset containing Czech air traffic recordings from other airports, LKPR and LKTB.
WER of callsings in LKKU recordings was evaluated to be 6.2 %, while on LKPR and LKTB dataset the model reached 3.6 %.
# Training hyperparameters
- **learning_rate:** 3e-5
- **per_device_train_batch_size:** 2
- **gradient_accumulation_steps:** 8
- **warmup_ratio:** 0.12
- **fp16:** True
- **gradient_checkpointing:** True
- **evaluation_strategy:** "epoch"
- **save_strategy:** "epoch"
- **load_best_model_at_end:** True
- **metric_for_best_model:** "wer"
- **num_train_epochs:** 45
# Contact
For further information don't hesitate to contact Veronika Nevarilova (**[xnevar00@stud.fit.vutbr.cz](xnevar00@stud.fit.vutbr.cz)**) or Igor Szoke (**[szoke@fit.vutbr.cz](szoke@fit.vutbr.cz)**). |