dinhanhx commited on
Commit
477bda8
1 Parent(s): 80f0e9d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -7,7 +7,7 @@ A copy of only tokenizer from https://huggingface.co/meta-llama/Llama-2-7b-hf
7
  Le code example
8
  ```python
9
  from transformers import LlamaTokenizerFast
10
- tokenizer = LlamaTokenizerFast.from_pretrained("dinhanhx/llama-tokenizer-only")
11
  text = "Do bạch kim rất quý nên sẽ dùng để lắp vô xương"
12
  print(tokenizer.convert_ids_to_tokens(tokenizer.encode(text)))
13
  ```
 
7
  Le code example
8
  ```python
9
  from transformers import LlamaTokenizerFast
10
+ tokenizer = LlamaTokenizerFast.from_pretrained("dinhanhx/llama-tokenizer-hf")
11
  text = "Do bạch kim rất quý nên sẽ dùng để lắp vô xương"
12
  print(tokenizer.convert_ids_to_tokens(tokenizer.encode(text)))
13
  ```