Update tokenizer_config.json
#4
by
basujindal
- opened
Configuring bos, eos, pad and unk tokens helps load the tokenizer successfully, otherwise getting a recursion error "RecursionError: maximum recursion depth exceeded while getting the str of an object." As done here: https://huggingface.co/medalpaca/medalpaca-13b/discussions/7
kbressem
changed pull request status to
merged