incoder-1B-finetuned-jest / tokenizer_config.json
EddieChen372's picture
Training in progress step 10000
c8233be
{
"bos_token": "<|endoftext|>",
"model_max_length": 2048,
"name_or_path": "EddieChen372/incoder-1B-finetuned-jest",
"special_tokens_map_file": "tokenizer/v2_trfs/special_tokens_map.json",
"tokenizer_class": "PreTrainedTokenizerFast"
}