Upload tokenizer.json
#11
by
ArthurZ
HF staff
- opened
No description provided.
Per IANA registry, iw was deprecated as the code for Hebrew in 1989 and the preferred code is he
Original PR was merged into whisper here - https://github.com/openai/whisper/pull/401
HuggingFace transformers PR here - https://github.com/huggingface/transformers/pull/21310
The correct subtag:
%%
Type: language
Subtag: he
Description: Hebrew
Added: 2005-10-16
Suppress-Script: Hebr
%%
And the deprecation
%%
Type: language
Subtag: iw
Description: Hebrew
Added: 2005-10-16
Deprecated: 1989-01-01
Preferred-Value: he
Suppress-Script: Hebr
%%
ArthurZ
changed pull request status to
merged