vietnamese-bi-encoder / custom_tokenizer.py

Commit History

add word segmentation before tokenization
c1d85a2

phamson02 commited on