Hugging Face's logo

israel
/
AmhWordPieceTokenizer

Amharic
Amharic
Word Piece Tokenizer
Tokenizer
Model card Files Community
israel commited on
Commit
b3af060
•
1 Parent(s): d547746

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -12,7 +12,7 @@ from transformers import AutoTokenizer
12
 
13
  tokenizer = AutoTokenizer.from_pretrained("israel/AmhWordPieceTokenizer")
14
 
15
- ecoding = tokenizer.encode("ኮሌጁ ቢያስተምርም ወደስራ የሚመድባቸው መንግስት ነው abcs")
16
 
17
  encoding.tokens
18
 
 
12
 
13
  tokenizer = AutoTokenizer.from_pretrained("israel/AmhWordPieceTokenizer")
14
 
15
+ encoding = tokenizer.encode("ኮሌጁ ቢያስተምርም ወደስራ የሚመድባቸው መንግስት ነው abcs")
16
 
17
  encoding.tokens
18