StefanH commited on
Commit
7de2d15
1 Parent(s): ed74ee7

add tokenizer

Browse files
Files changed (1) hide show
  1. added_tokens.json +1 -1
added_tokens.json CHANGED
@@ -1 +1 @@
1
- {"<|ASPECT-SEP|>": 30523, "[eot]": 30522}
 
1
+ {"[eot]": 30522, "<|ASPECT-SEP|>": 30523}