Add fast tokenizer

#9
by Narsil HF staff - opened
Files changed (2) hide show
  1. README.md +10 -11
  2. tokenizer.json +0 -0
README.md CHANGED
@@ -16,7 +16,6 @@ license: apache-2.0
16
  <a href="https://github.com/01-ai/Yi-1.5/issues/2">πŸ’¬ WeChat</a>
17
  <br/>
18
  <a href="https://arxiv.org/abs/2403.04652">πŸ“ Paper</a> β€’
19
- <a href="https://01-ai.github.io/">πŸ’ͺ Tech Blog</a> β€’
20
  <a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#faq">πŸ™Œ FAQ</a> β€’
21
  <a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#learning-hub">πŸ“— Learning Hub</a>
22
  </p>
@@ -43,11 +42,11 @@ Model | Context Length | Pre-trained Tokens
43
 
44
  | Name | Download |
45
  | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
46
- | Yi-1.5-34B-Chat | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI)|
47
- | Yi-1.5-34B-Chat-16K | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
48
- | Yi-1.5-9B-Chat | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
49
- | Yi-1.5-9B-Chat-16K | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
50
- | Yi-1.5-6B-Chat | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
51
 
52
  </div>
53
 
@@ -57,11 +56,11 @@ Model | Context Length | Pre-trained Tokens
57
 
58
  | Name | Download |
59
  | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
60
- | Yi-1.5-34B | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
61
- | Yi-1.5-34B-32K | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
62
- | Yi-1.5-9B | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
63
- | Yi-1.5-9B-32K | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
64
- | Yi-1.5-6B | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) β€’ [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
65
 
66
  </div>
67
 
 
16
  <a href="https://github.com/01-ai/Yi-1.5/issues/2">πŸ’¬ WeChat</a>
17
  <br/>
18
  <a href="https://arxiv.org/abs/2403.04652">πŸ“ Paper</a> β€’
 
19
  <a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#faq">πŸ™Œ FAQ</a> β€’
20
  <a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#learning-hub">πŸ“— Learning Hub</a>
21
  </p>
 
42
 
43
  | Name | Download |
44
  | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
45
+ | Yi-1.5-34B-Chat | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
46
+ | Yi-1.5-34B-Chat-16K | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
47
+ | Yi-1.5-9B-Chat | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
48
+ | Yi-1.5-9B-Chat-16K | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
49
+ | Yi-1.5-6B-Chat | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
50
 
51
  </div>
52
 
 
56
 
57
  | Name | Download |
58
  | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
59
+ | Yi-1.5-34B | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
60
+ | Yi-1.5-34B-32K | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
61
+ | Yi-1.5-9B | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
62
+ | Yi-1.5-9B-32K | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
63
+ | Yi-1.5-6B | β€’ [πŸ€— Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β€’ [πŸ€– ModelScope](https://www.modelscope.cn/organization/01ai) |
64
 
65
  </div>
66
 
tokenizer.json DELETED
The diff for this file is too large to render. See raw diff