language:
- en
library_name: transformers
pipeline_tag: automatic-speech-recognition
Model trained in int8 with LoRA
Usage:
prepare pipeline, providing any custom generate_kwargs supprted by https://huggingface.co/docs/transformers/v4.40.0/en/main_classes/text_generation#transformers.GenerationConfig
asr_model=prepare_pipeline(
model_dir='.', # wherever you save the model
generate_kwargs={
'max_new_tokens':112,
'num_beams':1,
'repetition_penalty':1,
'do_sample':False
}
)
run ASR:
asr_model(audio_path)
run ASR on full directory in audio_dir:
If generate_kwargs not specified, will give you (deterministic) greedy decoding with up to 112 tokens generated, no repetition penalty
ASRdirWhisat(
audio_dir,
out_dir = '../whisat_results/',
model_dir=".",
)
Training information:
Training script: tune_hf_whisper.py
Training hyperparameters: hparams.yaml
Training data manifest: PUBLIC_KIDS_TRAIN_v4_deduped.csv
Note: to recreate this training you will need to acquire the following public datasets: MyST (myst-v0.4.2) CuKids CSLU
and ensure they are stored at paths consistend with those in the data manifest above.
Reference:
@inproceedings{southwell2024,
title={Automatic speech recognition tuned for child speech in the classroom},
author={ Southwell, Rosy and Ward , Wayne and Trinh , Viet Anh and Clevenger, Charis and Clevenger, Clay and Watts, Emily and Reitman, Jason and D’Mello, Sidney and Whitehill, Jacob},
booktitle={{IEEE} International Conference on Acoustics, Speech and Signal Processing
{ICASSP} 2024, Seoul, South Korea, April 14-19, 2024},
year={2024},
}