Hugging Face model: OpenFormosa/PangolinTokenizer
Author: OpenFormosa
Created: 2026-06-09 14:13:10
Last modified: 2026-06-09 16:35:06
Library: transformers
Likes: 2 | Downloads: 22
Tags: transformers, gpt2, tokenizer, byte-level-bpe, traditional-chinese, taiwan, multilingual, license:other, endpoints_compatible, region:us
Card metadata:
license: other
library_name: transformers
tags: tokenizer, byte-level-bpe, traditional-chinese, taiwan, multilingual