Use GPT2Tokenizer instead of Tiktokenizer

#3
ModelCloud.AI org
No description provided.
lrl-modelcloud changed pull request status to open
lrl-modelcloud changed pull request status to merged

Sign up or log in to comment