Upload tokenizer
dfb317b
|
[PAD] |
|
[UNK] |
|
[CLS] |
|
[SEP] |
|
[MASK] |
|
! |
|
, |
|
. |
|
? |
|
a |
|
b |
|
c |
|
d |
|
e |
|
f |
|
g |
|
h |
|
i |
|
k |
|
l |
|
m |
|
n |
|
o |
|
p |
|
q |
|
r |
|
s |
|
t |
|
u |
|
v |
|
w |
|
x |
|
y |
|
z |
|
##l |
|
##e |
|
##p |
|
##i |
|
##s |
|
##t |
|
##r |
|
##f |
|
##w |
|
##a |
|
##o |
|
##u |
|
##n |
|
##d |
|
##g |
|
##h |
|
##v |
|
##k |
|
##m |
|
##z |
|
##y |
|
##c |
|
##b |
|
##x |
|
th |
|
##ou |
|
##re |
|
the |
|
##nd |
|
##is |
|
##es |
|
##er |
|
my |
|
##or |
|
##ve |
|
ha |
|
##ll |
|
##it |
|
to |
|
##nt |
|
and |
|
no |
|
##ed |
|
mo |
|
##st |
|
##at |
|
in |
|
thou |
|
##ea |
|
##in |
|
##me |
|
co |
|
of |
|
##ir |
|
wh |
|
##el |
|
##on |
|
not |
|
is |
|
wi |
|
##ee |
|
as |
|
##ld |
|
##th |
|
##ra |
|
most |
|
for |
|
##ri |
|
will |
|
be |
|
ca |
|
me |
|
so |
|
sh |
|
##la |
|
##en |
|
##se |
|
##ro |
|
##ch |
|
thy |
|
##est |
|
have |
|
what |
|
are |
|
do |
|
it |
|
li |
|
sp |
|
you |
|
##il |
|
##ty |
|
##ar |
|
##ay |
|
##ke |
|
this |
|
thee |
|
##ear |
|
come |
|
##irit |
|
spirit |
|
ch |
|
go |
|
ho |
|
his |
|
lo |
|
##le |
|
##ly |
|
##ut |
|
##ith |
|
more |
|
##ment |
|
all |
|
ba |
|
but |
|
de |
|
st |
|
see |
|
we |
|
wor |
|
with |
|
##ic |
|
##wn |
|
##an |
|
##ake |
|
##ul |
|
##gh |
|
##mp |
|
##ber |
|
that |
|
##ould |
|
##ist |
|
now |
|
##ing |
|
##eep |
|
am |
|
ari |
|
ex |
|
he |
|
po |
|
per |
|
re |
|
sou |
|
ser |
|
sla |
|
say |
|
tis |
|
wa |
|
##et |
|
##ement |
|
##id |
|
##for |
|
##ful |
|
##od |
|
##un |
|
##ure |
|
##nst |
|
##ge |
|
##ves |
|
##ms |
|
##med |
|
##ct |
|
##ous |
|
##our |
|
they |
|
##ish |
|
##ess |
|
##nter |
|
##eas |
|
##rom |
|
liber |
|
char |
|
ariel |
|
serv |
|
slave |
|
liberty |
|
ad |
|
br |
|
bo |
|
by |
|
bes |
|
bra |
|
bear |
|
du |
|
en |
|
ear |
|
fre |
|
fir |
|
fri |
|
fly |
|
fet |
|
from |
|
gre |
|
gra |
|
hon |
|
kn |
|
king |
|
make |
|
mist |
|
ne |
|
ou |
|
on |
|
own |
|
pl |
|
qu |
|
rel |
|
sl |
|
sw |
|
sa |
|
su |
|
sen |
|
vi |
|
would |
|
wish |
|
##li |
|
##em |
|
##ere |
|
##end |
|
##ip |
|
##ion |
|
##igh |
|
##ss |
|
##ses |
|
##te |
|
##tun |
|
##fe |
|
##wer |
|
##and |
|
##all |
|
##ant |
|
##ow |
|
##ore |
|
##ue |
|
##us |
|
##ver |
|
##vil |
|
##ck |
|
##cit |
|
##ban |
|
##ress |
|
them |
|
there |
|
##nds |
|
##ven |
|
has |
|
hast |
|
hath |
|
##ease |
|
cont |
|
comp |
|
##elf |
|
##one |
|
forth |
|
canst |
|
cali |
|
mere |
|
ship |
|
shall |
|
##lain |
|
like |
|
your |
|
good |
|
how |
|
love |
|
##lete |
|
bad |
|
devil |
|
|