Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
Spaces:
taka-yamakoshi
/
tokenizer-demo
like
7
Running
App
Files
Files
Community
b5a5fbe
tokenizer-demo
/
app.py
Commit History
add tokenizers
b5a5fbe
Takateru Yamakoshi
commited on
Mar 2
fix a minor bug
6751661
taka-yamakoshi
commited on
Jul 27, 2022
add default inputs
ef0b5c6
taka-yamakoshi
commited on
Jul 27, 2022
decode the whole string
145f48c
taka-yamakoshi
commited on
Jul 16, 2022
minor update on instructions
dc80c0d
taka-yamakoshi
commited on
Jul 16, 2022
add instructions
ed9112c
taka-yamakoshi
commited on
Jul 16, 2022
minor update
9240bf4
taka-yamakoshi
commited on
Jul 16, 2022
include detokenize
21c2f11
taka-yamakoshi
commited on
Jul 16, 2022
fix a bug & minor update
a999c8e
taka-yamakoshi
commited on
Jul 16, 2022
add more functions
8a204f8
taka-yamakoshi
commited on
Jul 16, 2022
adjust index
e9daec4
taka-yamakoshi
commited on
Jul 2, 2022
fix bugs
f70863b
taka-yamakoshi
commited on
Jul 2, 2022
show tokenized sentence
6353e7e
taka-yamakoshi
commited on
Jul 2, 2022
fix text
de2522a
taka-yamakoshi
commited on
Jul 1, 2022
fix text
c1ec65f
taka-yamakoshi
commited on
Jul 1, 2022
update text
c4a27b1
taka-yamakoshi
commited on
Jul 1, 2022
adjust sent id
ef497bf
taka-yamakoshi
commited on
Jul 1, 2022
remove nested cols
6da8b4f
taka-yamakoshi
commited on
Jul 1, 2022
fix bugs
228552f
taka-yamakoshi
commited on
Jul 1, 2022
fix bugs
2d48eb3
taka-yamakoshi
commited on
Jul 1, 2022
first commit
5e5793b
taka-yamakoshi
commited on
Jul 1, 2022