versae
/

scandeng-tokenizer

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

scandeng-tokenizer

Commit History

Add HF tokenizer converted from SentencePiece

04b8476

versae commited on Apr 12

Create config.json

384dd2d
verified

versae commited on Apr 12

SentencePiece byte fallback 32k

5230060

versae commited on Apr 12

initial commit

d0ebda9
verified

versae commited on Apr 12