ctrl44-simp / added_tokens.json
liamcripwell's picture
add tokenizer
079aa3f
{"<ident>": 50265, "<para>": 50266, "<ssplit>": 50267, "<dsplit>": 50268}