Instructions to use ToastyPigeon/gemma-4-12b-full-cpt-itvec with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ToastyPigeon/gemma-4-12b-full-cpt-itvec with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ToastyPigeon/gemma-4-12b-full-cpt-itvec")

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ToastyPigeon/gemma-4-12b-full-cpt-itvec")
model = AutoModelForMultimodalLM.from_pretrained("ToastyPigeon/gemma-4-12b-full-cpt-itvec", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ToastyPigeon/gemma-4-12b-full-cpt-itvec with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ToastyPigeon/gemma-4-12b-full-cpt-itvec"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ToastyPigeon/gemma-4-12b-full-cpt-itvec",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ToastyPigeon/gemma-4-12b-full-cpt-itvec

SGLang

How to use ToastyPigeon/gemma-4-12b-full-cpt-itvec with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ToastyPigeon/gemma-4-12b-full-cpt-itvec" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ToastyPigeon/gemma-4-12b-full-cpt-itvec",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ToastyPigeon/gemma-4-12b-full-cpt-itvec" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ToastyPigeon/gemma-4-12b-full-cpt-itvec",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ToastyPigeon/gemma-4-12b-full-cpt-itvec with Docker Model Runner:
```
docker model run hf.co/ToastyPigeon/gemma-4-12b-full-cpt-itvec
```

Gemma-4-12B — full-CPT + instruct task vector

A Gemma-4-12B (base, non-instruct) derivative built in two steps:

CPT merge: the gemma-4-12b-full-cpt LoRA (QLoRA r64, all projections, SWA layers, 1740 steps on a prose/story-heavy mix) folded into google/gemma-4-12b.
Task-vector apply: the instruct task vector gemma-4-12b-it − gemma-4-12b added at weight 1.0 (normal scaling):

final = base + CPT-LoRA + (it − base)

The goal: keep the CPT prose/style gains while regaining the instruct model's instruction-following and formatting behavior.

Training data (CPT step)

Mix of story/prose and chat-log corpora (the named sets are public under the ToastyPigeon namespace): erotic-books-clone, new-story-dataset, some-erotica, steve-and-marvin, SpringDragon-chat, disco-chat, counter-signal — plus two other private datasets with a focus on WLW content and chat.

Content note: parts of the CPT mix are adult/erotic fiction. Intended for fiction-writing research and style experimentation by adults; not for production deployment or use by minors.

Intended use

Text-completion / storywriting experiments
Starting point for downstream RP/style finetunes that want both the CPT prose signal and instruct-style behavior

Notes

Unified multimodal architecture (Gemma4UnifiedForConditionalGeneration); the vision/audio towers are unchanged from base.
See the companion adapter repo gemma-4-12b-full-cpt for the CPT run details (hyperparameters, loss curve).

Downloads last month: 15

Safetensors

Model size

12B params

Tensor type

BF16