sho-takase
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,9 @@ This repository provides large language models trained by [SB Intuitions](https:
|
|
12 |
|
13 |
## How to use
|
14 |
|
15 |
-
|
|
|
|
|
16 |
import torch
|
17 |
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, set_seed
|
18 |
|
@@ -40,7 +42,7 @@ for t in text:
|
|
40 |
|
41 |
## Configuration
|
42 |
|
43 |
-
| Parameters | Vocab size |
|
44 |
| :-----: | :-----------: | :-------------: | :------------ | :-----------: | :----: | :--------: | :-------------: |
|
45 |
| [7B](https://huggingface.co/sbintuitions/sarashina2-7b) | 102400 | 2.1T | Llama2 | RoPE | 32 | 4096 | 32 |
|
46 |
| [13B](https://huggingface.co/sbintuitions/sarashina2-13b) | 102400 | 2.1T | Llama2 | RoPE | 40 | 5120 | 40 |
|
|
|
12 |
|
13 |
## How to use
|
14 |
|
15 |
+
Please set **use_fast=False** to use our tokenizer properly.
|
16 |
+
|
17 |
+
```python
|
18 |
import torch
|
19 |
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, set_seed
|
20 |
|
|
|
42 |
|
43 |
## Configuration
|
44 |
|
45 |
+
| Parameters | Vocab size | Training tokens | Architecture | Position type | Layers | Hidden dim | Attention heads |
|
46 |
| :-----: | :-----------: | :-------------: | :------------ | :-----------: | :----: | :--------: | :-------------: |
|
47 |
| [7B](https://huggingface.co/sbintuitions/sarashina2-7b) | 102400 | 2.1T | Llama2 | RoPE | 32 | 4096 | 32 |
|
48 |
| [13B](https://huggingface.co/sbintuitions/sarashina2-13b) | 102400 | 2.1T | Llama2 | RoPE | 40 | 5120 | 40 |
|