Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ This is a HuggingFace + vLLM-ready repackage of [`talkie-lm/talkie-1930-13b-base
|
|
| 21 |
| File | What it is |
|
| 22 |
|---|---|
|
| 23 |
| `model.safetensors` | bf16 weights, ~26 GB. `lm_head_gain` (a learned scalar) is pre-multiplied into `lm_head.weight` so vLLM's transformers backend doesn't need to know about it. |
|
| 24 |
-
| `config.json` | `TalkieConfig` (vocab=65536, hidden=5120, 40 layers × 40 heads, head_dim=128, ctx=
|
| 25 |
| `tokenizer.json`, `tokenizer_config.json` | HF fast BPE built from the original `vocab.txt` (ranks < 65535) with `<|endoftext|>` at id 65535. EOS = pad = `<\|endoftext\|>`. No chat specials (base model). |
|
| 26 |
| `generation_config.json` | `eos_token_id=65535`, `pad_token_id=65535`. |
|
| 27 |
| `modeling_talkie.py`, `configuration_talkie.py` | HF `PreTrainedModel` implementation with `ALL_ATTENTION_FUNCTIONS` dispatch (vLLM transformers-backend compatible). |
|
|
@@ -35,7 +35,7 @@ vllm serve agaralon/talkie-1930-13b-base-vllm \
|
|
| 35 |
--model-impl transformers \
|
| 36 |
--trust-remote-code \
|
| 37 |
--dtype bfloat16 \
|
| 38 |
-
--max-model-len
|
| 39 |
```
|
| 40 |
|
| 41 |
Then hit it as an OpenAI-style **completions** endpoint (base model — use `/v1/completions`, not chat):
|
|
|
|
| 21 |
| File | What it is |
|
| 22 |
|---|---|
|
| 23 |
| `model.safetensors` | bf16 weights, ~26 GB. `lm_head_gain` (a learned scalar) is pre-multiplied into `lm_head.weight` so vLLM's transformers backend doesn't need to know about it. |
|
| 24 |
+
| `config.json` | `TalkieConfig` (vocab=65536, hidden=5120, 40 layers × 40 heads, head_dim=128, ctx=4096, RoPE θ=1e6) plus `auto_map` for `AutoConfig`/`AutoModel`/`AutoModelForCausalLM`. |
|
| 25 |
| `tokenizer.json`, `tokenizer_config.json` | HF fast BPE built from the original `vocab.txt` (ranks < 65535) with `<|endoftext|>` at id 65535. EOS = pad = `<\|endoftext\|>`. No chat specials (base model). |
|
| 26 |
| `generation_config.json` | `eos_token_id=65535`, `pad_token_id=65535`. |
|
| 27 |
| `modeling_talkie.py`, `configuration_talkie.py` | HF `PreTrainedModel` implementation with `ALL_ATTENTION_FUNCTIONS` dispatch (vLLM transformers-backend compatible). |
|
|
|
|
| 35 |
--model-impl transformers \
|
| 36 |
--trust-remote-code \
|
| 37 |
--dtype bfloat16 \
|
| 38 |
+
--max-model-len 4096
|
| 39 |
```
|
| 40 |
|
| 41 |
Then hit it as an OpenAI-style **completions** endpoint (base model — use `/v1/completions`, not chat):
|