stanford-crfm
/

pubmed_gpt_tokenizer

Model card Files Files and versions Community

pubmed_gpt_tokenizer / tokenizer_config.json

Commit History

50k vocab, prefix_space=false,trained on PubMed Abstracts

39545d2

J38 commited on Sep 15, 2022

add of |endoftext|

7a0b15c

J38 commited on Sep 9, 2022

use lowercase normalizer

514166f

J38 commited on Sep 5, 2022

does bert normalizer work

7fd39d4

J38 commited on Sep 5, 2022

change lowercase key

5940b13

J38 commited on Sep 5, 2022

add normalizer

15bb604

J38 commited on Sep 5, 2022

do lower case

4bea54c

J38 commited on Sep 5, 2022

config for tokenizer

c682350

J38 commited on Sep 5, 2022