NeuroDonu commited on
Commit
67834ba
1 Parent(s): ef20ee0

Upload /added_tokens.json with huggingface_hub

Browse files
Files changed (1) hide show
  1. added_tokens.json +34 -0
added_tokens.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "<pad10>": 32010,
3
+ "<pad11>": 32011,
4
+ "<pad12>": 32012,
5
+ "<pad13>": 32013,
6
+ "<pad14>": 32014,
7
+ "<pad15>": 32015,
8
+ "<pad16>": 32016,
9
+ "<pad17>": 32017,
10
+ "<pad18>": 32018,
11
+ "<pad19>": 32019,
12
+ "<pad20>": 32020,
13
+ "<pad21>": 32021,
14
+ "<pad22>": 32022,
15
+ "<pad23>": 32023,
16
+ "<pad24>": 32024,
17
+ "<pad25>": 32025,
18
+ "<pad26>": 32026,
19
+ "<pad27>": 32027,
20
+ "<pad28>": 32028,
21
+ "<pad29>": 32029,
22
+ "<pad2>": 32002,
23
+ "<pad30>": 32030,
24
+ "<pad31>": 32031,
25
+ "<pad3>": 32003,
26
+ "<pad4>": 32004,
27
+ "<pad5>": 32005,
28
+ "<pad6>": 32006,
29
+ "<pad7>": 32007,
30
+ "<pad8>": 32008,
31
+ "<pad9>": 32009,
32
+ "<|im_end|>": 32000,
33
+ "<|im_start|>": 32001
34
+ }