CodeFuse-DevOps-Model-14B-Chat / tokenizer_config.json

codefuse-admin

upload model from ant-group

5accb6e 7 months ago

No virus

237 Bytes

	{
	"auto_map": {
	"AutoTokenizer": [
	"tokenization_qwen.QWenTokenizer",
	null
	]
	},
	"clean_up_tokenization_spaces": true,
	"model_max_length": 8192,
	"padding_side": "left",
	"tokenizer_class": "QWenTokenizer"
	}