Whisper model has no attribute freeze_encoder and load_trained
Hi
Thanks for sharing the finetune script. But I have some errors trying to run the code, and I think all comes downs to this line of code:
model = whisper.Whisper.load_trained(model_args.model_name_or_path)
Is the Whisper (capital) class modified in your code base?
Hey @asa-pan ! Just an FYI - this script is written to fine-tune the OpenAI version of Whisper (not the Transformers one). For this, I'd recommend checking out the checkpoint at https://huggingface.co/esb/whisper-aed-switchboard This is exactly the same checkpoint, just better documented 😉 Specifically, to fix the 'freeze encoder' issue, one needs to pip install the OpenAI version of Whisper as detailed on the README.md
Expect a script to fine-tune the Transformers version of Whisper in the coming days 👀 (https://github.com/huggingface/transformers/pull/19519 and https://github.com/huggingface/blog/pull/591)
Hope that helps!
Hey
@sanchit-gandhi
, love your work. But even the OpenAI version of whisper doesn't have whisper.Whisper.load_trained function. https://github.com/openai/whisper/blob/9f70a352f9f8630ab3aa0d06af5cb9532bd8c21d/whisper/model.py#L197
I'm struggling to load any Whisper checkpoint due to this issue, could you kindly help me out a little bit? The same code is used with esc-benchmark/ models, so I assume the class is internally modified like asa-pan mentioned.
Hey @TeemuSo !
Could you pip install Whisper as follows:
pip install git+https://github.com/patrickvonplaten/whisper.git
And then try running it from the updated repo: https://huggingface.co/esb/whisper-aed-switchboard
Helped someone get this working earlier today: https://huggingface.co/esb/whisper-aed-switchboard/discussions/1#636260fc4c9f829ef7f8898b
Hey @TeemuSo !
Could you pip install Whisper as follows:
pip install git+https://github.com/patrickvonplaten/whisper.gitAnd then try running it from the updated repo: https://huggingface.co/esb/whisper-aed-switchboard
Helped someone get this working earlier today: https://huggingface.co/esb/whisper-aed-switchboard/discussions/1#636260fc4c9f829ef7f8898b
Ah, my bad. I didn't see that it was forked by Patrick, now it all makes sense. Thank you! :)