Instructions to use minchyeom/birthday-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use minchyeom/birthday-llm with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="minchyeom/birthday-llm")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("minchyeom/birthday-llm")
model = AutoModelForCausalLM.from_pretrained("minchyeom/birthday-llm")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use minchyeom/birthday-llm with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "minchyeom/birthday-llm"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "minchyeom/birthday-llm",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/minchyeom/birthday-llm

SGLang

How to use minchyeom/birthday-llm with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "minchyeom/birthday-llm" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "minchyeom/birthday-llm",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "minchyeom/birthday-llm" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "minchyeom/birthday-llm",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use minchyeom/birthday-llm with Docker Model Runner:
```
docker model run hf.co/minchyeom/birthday-llm
```

Yippie!! It's my birthday in 2 days! So I'm gonna drop this model that I made at 2AM.

It's quite a bit different from what I've been doing recently, but it was pretty fun to work on. :D

Used a different merging technique that I quickly designed & crafted which focused on the math and logical reasoning.

Also fine-tuned this on Self-Play RL algorithms.

Not the best LLM out there but it was pretty fun making this, and coming up with something different.

Borrowed some ideas from my other "distillation" technique, which significantly reduces the number of layers while aiming to retain the output quality.

Thanks and have fun!!

Also happy birthday to myself :)

Usage (taken from official Gemma 2 9B repo):

import torch
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="minchyeom/birthday-llm",
    model_kwargs={"torch_dtype": torch.bfloat16},
    device="cuda",  # replace with "mps" to run on a Mac device
)

messages = [
    {"role": "user", "content": "How many r's are there in the word strawberry?"},
]

outputs = pipe(messages, max_new_tokens=256)
assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
print(assistant_response)

Downloads last month: -

Safetensors

Model size

9B params

Tensor type

F16

Model tree for minchyeom/birthday-llm

Base model

google/gemma-2-9b

Finetuned

google/gemma-2-9b-it

Finetuned

(425)

this model

Finetunes

1 model