Instructions to use kmkrworks/LiteGPT-Llama-V2-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kmkrworks/LiteGPT-Llama-V2-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kmkrworks/LiteGPT-Llama-V2-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kmkrworks/LiteGPT-Llama-V2-Instruct")
model = AutoModelForCausalLM.from_pretrained("kmkrworks/LiteGPT-Llama-V2-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kmkrworks/LiteGPT-Llama-V2-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kmkrworks/LiteGPT-Llama-V2-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmkrworks/LiteGPT-Llama-V2-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kmkrworks/LiteGPT-Llama-V2-Instruct

SGLang

How to use kmkrworks/LiteGPT-Llama-V2-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kmkrworks/LiteGPT-Llama-V2-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmkrworks/LiteGPT-Llama-V2-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kmkrworks/LiteGPT-Llama-V2-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmkrworks/LiteGPT-Llama-V2-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use kmkrworks/LiteGPT-Llama-V2-Instruct with Docker Model Runner:
```
docker model run hf.co/kmkrworks/LiteGPT-Llama-V2-Instruct
```

LiteGPT-Llama-V2-Instruct

The chat version of LiteGPT V2: a 201M-parameter model built from scratch, with every step on record.

LiteGPT V2 Base after three epochs of chat fine-tuning on curated conversations. It passed its registered quality gate, and follows instructions far better than LiteGPT V1-Instruct. Results, charts and run records: github.com/kmkrofficial/LiteGPT.

Highlights

Follows instructions 55% better than V1-Instruct on IFEval (0.129 against 0.083), clearing its registered gate by 2.56 standard errors.
Writes some working code: 7 of 164 HumanEval problems solved and 4.2% of MBPP, where V1-Instruct solved none.
Small: 201M parameters and a 4,096-token context. It runs comfortably on a laptop.
Loads in Transformers with no remote code, with its chat template included.

Results

Task	V2-Instruct	V1-Instruct
IFEval	0.129	0.083
GSM8K chat	0.022	0.021
HumanEval chat	0.043	0.000
MBPP chat	0.042	0.000

Every benchmark, including comparisons with other chat models: results.

How it compares

LiteGPT V2 Instruct doesn't reach the top score among these chat models. What it brings is a chat model built entirely from scratch, small enough for a laptop, and open down to every run record.

Model	IFEval	GSM8K chat	HumanEval chat	MBPP chat
Qwen3-0.6B	0.564	0.310	0.372	0.310
Gemma 3 270M-it	0.251	0.099	0.122	0.154
SmolLM2-135M-Instruct	0.220	0.021	0.000	0.098
LiteGPT V2 Instruct	0.129	0.022	0.043	0.042
LiteGPT V1 Instruct	0.083	0.021	0.000	0.000

Higher is better.

Model

Item	Value
Parameters	201,373,952
Started from	LiteGPT V2 Base
Context	4,096 tokens
Chat format	`litegpt-chat-v1`

Use

With Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

name = "kmkrworks/LiteGPT-Llama-V2-Instruct"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name)
inputs = tokenizer.apply_chat_template([{"role": "user", "content": "What is the capital of France?"}], add_generation_prompt=True, return_tensors="pt", return_dict=True)
output = model.generate(**inputs, max_new_tokens=48)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

With LiteGPT's own engine, from a clone of the repository:

uv sync --locked
uv run litegpt chat --model LiteGPT-Llama-V2-Instruct

Benchmarks were scored through LiteGPT's engine.

Intended use: research and learning. It has no safety tuning, so add your own safeguards before putting it in front of people.

Training

3 epochs, 14,000 steps, assistant-only loss, peak learning rate 6e-4. Held-out loss fell every epoch: 1.1323, 1.0518, 1.0238. Details: training.md.

Training data

Everything in the Base card, then these chat conversations (385,677 in all, 497M tokens). Attributions: the NOTICE file.

Source	Dataset card	Generated by
smol-magpie-ultra	Apache-2.0	Llama-3.1-405B-Instruct
smol-summarize, smol-rewrite, smol-constraints (reserved part)	Apache-2.0	Qwen2.5-72B-Instruct

Licence

Apache-2.0 for the weights, MIT for the code, with the NOTICE file.

Built with Llama. Built with Qwen. Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Use of this model should follow the Llama Acceptable Use Policy.

Downloads last month: 236

Safetensors

Model size

0.2B params

Tensor type

BF16

Model tree for kmkrworks/LiteGPT-Llama-V2-Instruct

Base model

kmkrworks/LiteGPT-V2-Base

Finetuned

(1)

this model