lfoppiano commited on
Commit
cdf7227
1 Parent(s): ddf0be0

Fix the wrong tokenizer configuration

Browse files

The tokenizer configuration is loaded incorrectly without the casing. Adding this configuration file should correct the problem.
For more information see

Files changed (1) hide show
  1. tokenizer_config.json +3 -0
tokenizer_config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "do_lower_case": false
3
+ }