xgen-7b-8k-inst / tokenization_xgen.py

Commit History

update _convert_id_to_token
f6ff02a

tianxie-sf commited on

add pad token and default eos token (#7)
76a8dd5

rooa tianxie-sf commited on

.
b2d761b

rooa commited on