50k vocab, prefix_space=false,trained on PubMed Abstracts

experiment with 50k vocab

add of |endoftext|

add of |endoftext|

use lowercase normalizer

does bert normalizer work

change lowercase key

add normalizer

add lowercase normalizer

add merges.txt

add vocab.json

add vocab file

do lower case

config for tokenizer

tokenizer model

initial commit