Hugging Face model: toklens/maddition_eng_Latn_lambda_voc_1_lambda_tok_1_lambda_seg_1000_2000_minimal
Author: toklens
Created: 2026-05-27 21:57:42
Last modified: 2026-05-27 22:01:45
Likes: 0 | Downloads: 0
Tags: tokenizer, rbpe, flexitok, fineweb2, eng, license:mit, region:us
Card metadata:
license: mit
language: eng
tags: tokenizer, rbpe, flexitok, fineweb2