hugo commited on
Commit
3c9e283
1 Parent(s): 3d13aab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -1,12 +1,12 @@
1
- This is the tokenizer used by the MariTalk Large model.
2
 
3
- MariTalk Large is a proprietary LLM that can be used through an API endpoint, which we refer to as the "MariTalk API", or a downloadable version that can be used locally and is encrypted, known as "MariTalk Local".
4
 
5
  The purpose of including this tokenizer is to allow you to estimate the number of tokens in your prompts and, therefore, the cost of using the model.
6
 
7
  ```python
8
  import transformers
9
- tokenizer = transformers.AutoTokenizer.from_pretrained("maritaca-ai/maritalk-tokenizer-large")
10
 
11
  prompt = "Com quantos paus se faz uma canoa?"
12
 
 
1
+ This is the tokenizer used by the Sabiá-2 Medium model.
2
 
3
+ Sabiá2 Medium is a proprietary LLM that can be used through an API endpoint, which we refer to as the "MariTalk API", or a downloadable version that can be used locally and is encrypted, known as "MariTalk Local".
4
 
5
  The purpose of including this tokenizer is to allow you to estimate the number of tokens in your prompts and, therefore, the cost of using the model.
6
 
7
  ```python
8
  import transformers
9
+ tokenizer = transformers.AutoTokenizer.from_pretrained("maritaca-ai/sabia-2-tokenizer-medium")
10
 
11
  prompt = "Com quantos paus se faz uma canoa?"
12