Instructions to use wayzer2321/Luna-Gemma-12B-Merged with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use wayzer2321/Luna-Gemma-12B-Merged with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="wayzer2321/Luna-Gemma-12B-Merged")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("wayzer2321/Luna-Gemma-12B-Merged")
model = AutoModelForMultimodalLM.from_pretrained("wayzer2321/Luna-Gemma-12B-Merged", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use wayzer2321/Luna-Gemma-12B-Merged with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wayzer2321/Luna-Gemma-12B-Merged"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wayzer2321/Luna-Gemma-12B-Merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/wayzer2321/Luna-Gemma-12B-Merged

SGLang

How to use wayzer2321/Luna-Gemma-12B-Merged with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "wayzer2321/Luna-Gemma-12B-Merged" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wayzer2321/Luna-Gemma-12B-Merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "wayzer2321/Luna-Gemma-12B-Merged" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wayzer2321/Luna-Gemma-12B-Merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use wayzer2321/Luna-Gemma-12B-Merged with Docker Model Runner:
```
docker model run hf.co/wayzer2321/Luna-Gemma-12B-Merged
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

LUNA · GEMMA 4 · 12B — MERGED WEIGHTS / ОБЪЕДИНЕННЫЕ ВЕСА. Q8_0 GGUF release · Base model

Luna — Gemma 4 12B merged weights

Source checkpoint for the Luna Gemma 12B edition.

Luna is Wayzer's conversational character: informal slang, sarcasm, surreal humor, emotional shifts, and creator lore.

Contains profanity and provocative humor. Not recommended for sensitive users.
Содержит мат и провокационный юмор. Не рекомендуется чувствительным людям.

What is stored here

Full merged BF16 safetensors: the base weights with the trained LoRA applied.
Weight shards and their index.
Model configuration, tokenizer, chat template, and generation configuration.
An export manifest with the source revision and adapter details.

This is a full merged checkpoint. A separate LoRA adapter is not required to use these weights.

The LoRA was trained with an 8-bit loaded base. The exported merged weights are 16-bit BF16; training quantization and export precision are separate settings.

Training identity

Base: unsloth/gemma-4-12b-it.
Method: LoRA, rank 16, alpha 32.
Targets: attention and MLP projections.
Dataset: 457 dialogues; 411 training and 46 validation examples.
Training: assistant responses only, sequence length 512.
Schedule: two separate one-epoch runs; LR 1e-4, then 5e-5 with a new optimizer and scheduler.

This is a text-focused character fine-tune. It is not an official Google release.

Corresponding GGUF release

Luna Gemma 12B Q8_0 GGUF

The Qwen-based Luna releases are separate checkpoints. This edition uses Gemma 4.

File handling

Keep every safetensors shard, the index, and tokenizer/configuration files. Use the filenames referenced by model.safetensors.index.json. Use Gemma 4's native chat template, not a Qwen/ChatML template.