agaralon commited on
Commit
f1362fc
·
verified ·
1 Parent(s): 1420dde

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -21,7 +21,7 @@ This is a HuggingFace + vLLM-ready repackage of [`talkie-lm/talkie-1930-13b-base
21
  | File | What it is |
22
  |---|---|
23
  | `model.safetensors` | bf16 weights, ~26 GB. `lm_head_gain` (a learned scalar) is pre-multiplied into `lm_head.weight` so vLLM's transformers backend doesn't need to know about it. |
24
- | `config.json` | `TalkieConfig` (vocab=65536, hidden=5120, 40 layers × 40 heads, head_dim=128, ctx=2048, RoPE θ=1e6) plus `auto_map` for `AutoConfig`/`AutoModel`/`AutoModelForCausalLM`. |
25
  | `tokenizer.json`, `tokenizer_config.json` | HF fast BPE built from the original `vocab.txt` (ranks < 65535) with `<|endoftext|>` at id 65535. EOS = pad = `<\|endoftext\|>`. No chat specials (base model). |
26
  | `generation_config.json` | `eos_token_id=65535`, `pad_token_id=65535`. |
27
  | `modeling_talkie.py`, `configuration_talkie.py` | HF `PreTrainedModel` implementation with `ALL_ATTENTION_FUNCTIONS` dispatch (vLLM transformers-backend compatible). |
@@ -35,7 +35,7 @@ vllm serve agaralon/talkie-1930-13b-base-vllm \
35
  --model-impl transformers \
36
  --trust-remote-code \
37
  --dtype bfloat16 \
38
- --max-model-len 2048
39
  ```
40
 
41
  Then hit it as an OpenAI-style **completions** endpoint (base model — use `/v1/completions`, not chat):
 
21
  | File | What it is |
22
  |---|---|
23
  | `model.safetensors` | bf16 weights, ~26 GB. `lm_head_gain` (a learned scalar) is pre-multiplied into `lm_head.weight` so vLLM's transformers backend doesn't need to know about it. |
24
+ | `config.json` | `TalkieConfig` (vocab=65536, hidden=5120, 40 layers × 40 heads, head_dim=128, ctx=4096, RoPE θ=1e6) plus `auto_map` for `AutoConfig`/`AutoModel`/`AutoModelForCausalLM`. |
25
  | `tokenizer.json`, `tokenizer_config.json` | HF fast BPE built from the original `vocab.txt` (ranks < 65535) with `<|endoftext|>` at id 65535. EOS = pad = `<\|endoftext\|>`. No chat specials (base model). |
26
  | `generation_config.json` | `eos_token_id=65535`, `pad_token_id=65535`. |
27
  | `modeling_talkie.py`, `configuration_talkie.py` | HF `PreTrainedModel` implementation with `ALL_ATTENTION_FUNCTIONS` dispatch (vLLM transformers-backend compatible). |
 
35
  --model-impl transformers \
36
  --trust-remote-code \
37
  --dtype bfloat16 \
38
+ --max-model-len 4096
39
  ```
40
 
41
  Then hit it as an OpenAI-style **completions** endpoint (base model — use `/v1/completions`, not chat):