Instructions to use VertexResearch/Vertex-0.6-5M-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use VertexResearch/Vertex-0.6-5M-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="VertexResearch/Vertex-0.6-5M-Base")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("VertexResearch/Vertex-0.6-5M-Base")
model = AutoModelForCausalLM.from_pretrained("VertexResearch/Vertex-0.6-5M-Base", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use VertexResearch/Vertex-0.6-5M-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "VertexResearch/Vertex-0.6-5M-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VertexResearch/Vertex-0.6-5M-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/VertexResearch/Vertex-0.6-5M-Base

SGLang

How to use VertexResearch/Vertex-0.6-5M-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "VertexResearch/Vertex-0.6-5M-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VertexResearch/Vertex-0.6-5M-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "VertexResearch/Vertex-0.6-5M-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VertexResearch/Vertex-0.6-5M-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use VertexResearch/Vertex-0.6-5M-Base with Docker Model Runner:
```
docker model run hf.co/VertexResearch/Vertex-0.6-5M-Base
```

Vertex 0.6 5M Base

Base (pretrain-only) checkpoint of Vertex 0.6 5M, the smallest model in the Vertex 0.6 family at 4.77M parameters. Llama architecture: hidden 192, 9 layers, 3 heads (head_dim 64, no GQA), SwiGLU ffn 512, 4096-vocab ByteLevel BPE, tied embeddings, RoPE, ctx 512.

Pretrained from scratch on 500M tokens of English web text: Ultra-FineWeb, with Ultra-FineWeb-L3 synthetic rewrites (Multi-Style + QA) as a short final annealing phase, ~105 tokens per parameter, in 54 minutes on a single Apple M5 Pro (24 GB unified memory) using MLX at ~150k tokens/s.

This is a raw language model, no chat template, no instruction tuning. It is a research toy: fluent English at the sentence level, no factual knowledge. For something usable, see the larger family members such as Vertex-0.6-15M-Base and Vertex-0.6-100M-8192-ctx-Base.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "VertexResearch/Vertex-0.6-5M-Base"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

ids = tok("The best way to learn programming is", return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=40, do_sample=True, temperature=0.8, top_p=0.95)
print(tok.decode(out[0], skip_special_tokens=True))

Output from the run above:

The best way to learn programming is to deliver a comprehensive resource that enables readers to write effectively, effectively, and effectively. This resource provides a comprehensive understanding of programming methods, including advanced computing and automated

The model is 9.5 MB in bf16 and runs comfortably on CPU. Because the family is Llama-layout, it also converts to GGUF with llama.cpp's convert_hf_to_gguf.py.

Model details


Parameters	4,771,392 (0.79M embeddings, 3.98M transformer)
Architecture	`LlamaForCausalLM`
Layers / hidden / heads	9 / 192 / 3 (head_dim 64, no GQA)
MLP	SwiGLU, intermediate 512
Normalization	RMSNorm, eps 1e-5, pre-norm
Positions	RoPE, theta 10000, rotate-half pairing (first half / second half)
Vocab	4096 ByteLevel BPE, tied input/output embeddings
Context	512
Weights	bfloat16

The embedding matrix is kept small on purpose (16% of parameters) so most of the capacity is in the transformer blocks.

Tokenizer

tokenizer.json in Hugging Face tokenizers format, 4096 tokens, trained on a sample of the pretraining corpus. Token id 0 is <|endoftext|>: document separator in pretraining, stop token at inference.

Training data

Phase	Source	Tokens
1	openbmb/Ultra-FineWeb, English	450M
2	openbmb/Ultra-FineWeb-L3, English QA + Multi-Style	50M

500M tokens total, each seen once.

Training


Hardware	1x Apple M5 Pro, 24 GB unified memory
Framework	MLX, bf16 compute with fp32 master weights
Throughput	~150k tokens/s
Wall clock	54 minutes
Batch	64 x 512 tokens (32k tokens/step), 15,258 steps
Optimizer	AdamW, lr 3e-3 cosine to 3e-4, 300 warmup steps, wd 0.1, clip 1.0

Evaluation

Cross-entropy on held-out text with the model's own 4096-token vocabulary:

Split	Loss (nats)	PPL	Bits/token
Ultra-FineWeb, unseen shard	3.29	26.8	4.74
Ultra-FineWeb, training shard	3.31	27.4	4.77
Ultra-FineWeb-L3 QA	2.80	16.4	4.03
Ultra-FineWeb-L3 Multi-Style	2.96	19.3	4.27

Held-out and training loss match, so nothing is memorized. Lower L3 numbers mean synthetic text is more predictable, not that the model is stronger there.

Sample outputs

Temperature 0.8, prompt in bold:

The best way to learn programming is through hands-on activities. The most effective approach involves seeing both TTHTO and TTHTO as a practical resource.

The company announced today that it had been obstructed by the presidential election of the military's Military Service. This marks a significant advancement in the construction of the state.

Water boils at a constant volumeter and porch table, and a minimum of two times each.

In 1969, astronauts were deactive Barnes and X, and shoes of the Gogri attended the joke, symbolizing the passage of time and celebration.

What it can and cannot do

Does: grammatical English with correct punctuation, matches the prompt's register (formal letter, press release, how-to), stays on topic for about one sentence.

Does not: recall facts (dates, names, numbers are invented), stay coherent past 15 to 20 tokens, follow instructions, write code.

Intended for education and research: tiny-model scaling, tokenizer effects, embedding-to-body ratios, or as a smoke-test model for inference engines. Not for any production or user-facing use.

Files

File	Contents
`model.safetensors`	bf16 weights, standard Llama key names
`config.json`	`LlamaConfig`
`tokenizer.json`, `tokenizer_config.json`	ByteLevel BPE, `PreTrainedTokenizerFast`
`generation_config.json`	bos/eos = 0

License

Apache 2.0.

Downloads last month: 88

Safetensors

Model size

4.77M params

Tensor type

BF16

VertexResearch
/

Vertex-0.6-5M-Base