Instructions to use titan087/gemma4-e2b-neon-veil-rp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use titan087/gemma4-e2b-neon-veil-rp with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Use Docker

docker model run hf.co/titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

LM Studio
Jan

vLLM

How to use titan087/gemma4-e2b-neon-veil-rp with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "titan087/gemma4-e2b-neon-veil-rp"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "titan087/gemma4-e2b-neon-veil-rp",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Ollama
How to use titan087/gemma4-e2b-neon-veil-rp with Ollama:
```
ollama run hf.co/titan087/gemma4-e2b-neon-veil-rp:Q4_K_M
```

Unsloth Studio

How to use titan087/gemma4-e2b-neon-veil-rp with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for titan087/gemma4-e2b-neon-veil-rp to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for titan087/gemma4-e2b-neon-veil-rp to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for titan087/gemma4-e2b-neon-veil-rp to start chatting

How to use titan087/gemma4-e2b-neon-veil-rp with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "titan087/gemma4-e2b-neon-veil-rp:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use titan087/gemma4-e2b-neon-veil-rp with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use titan087/gemma4-e2b-neon-veil-rp with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "titan087/gemma4-e2b-neon-veil-rp:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use titan087/gemma4-e2b-neon-veil-rp with Docker Model Runner:
```
docker model run hf.co/titan087/gemma4-e2b-neon-veil-rp:Q4_K_M
```

Lemonade

How to use titan087/gemma4-e2b-neon-veil-rp with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull titan087/gemma4-e2b-neon-veil-rp:Q4_K_M

Run and chat with the model

lemonade run user.gemma4-e2b-neon-veil-rp-Q4_K_M

List all available models

lemonade list

🌃 Neon Veil — Gemma 4 E2B Roleplay Edition

Neon Veil is a finetuned roleplay model built on Google's Gemma 4 E2B Instruct base, quantized to GGUF format for efficient local inference. It is designed to deliver immersive, multi-character roleplay with strong prose, coherent long-form storytelling, and a rich understanding of pacing, tension, and character drift.

📖 Model Overview

Field	Value
Repository	`titan087/gemma4-e2b-neon-veil-rp`
Base Model	google/gemma-4-E2B-it
Architecture	Gemma 4
Parameters	2.8B
Format	GGUF
Quantizations	Q2_K_M, Q3_K_M, Q4_K_M, Q5_K_M, Q8_0
License	GPL-3.0
Context Length	65,536 tokens

💿 Available Quantizations

The model is available in the following GGUF quantizations, ranging from highly efficient (Q2_K_M) to near-lossless (Q8_0).

Quant	File	Size	Recommended VRAM
2-bit	Q2_K_M	~1.4 GB	2 GB
3-bit	Q3_K_M	~1.8 GB	3 GB
4-bit	Q4_K_M	~2.4 GB	4 GB
5-bit	Q5_K_M	~2.8 GB	5 GB
8-bit	Q8_0	~4.3 GB	8 GB

Recommendation: For roleplay, Q4_K_M or Q5_K_M offer the best balance between quality and resource usage. Q8_0 is recommended if you have sufficient VRAM and prioritize output fidelity.

🚀 Quick Start

Using llama.cpp

# Download the model
huggingface-cli download titan087/gemma4-e2b-neon-veil-rp gemma-4-e2b-neon-veil-rp.Q4_K_M.gguf --local-dir ./models

# Run inference
./main -m ./models/gemma-4-e2b-neon-veil-rp.Q4_K_M.gguf \
  -p "[System prompt]\n\n[Your roleplay prompt here]" \
  -n 512 --temp 0.8 --top_p 0.9 --top_k 40 \
  -c 65536

Using text-generation-webui (Oobabooga)

Download your preferred quant to text-generation-webui/models/neon-veil/
Launch the web UI
Select neon-veil from the model dropdown
Recommended settings:
- Temperature: 0.8
- Top P: 0.9
- Top K: 40
- Repetition Penalty: 1.05
- Min P (Mooncake / SillyTavern): 0.03

Using LM Studio

Open LM Studio and navigate to "Download Models"
Search for titan087/neon-veil
Download your preferred quant
Load and start chatting

✍️ Recommended Sampling Settings

Neon Veil performs best with the following sampling configuration for creative roleplay:

Parameter	Value
Temperature	0.7–0.9
Top P (Nucleus)	0.85–0.9
Top K	40–50
Repetition Penalty	1.03–1.07
Min P	0.02–0.05
Frequency Penalty	0.3
Presence Penalty	0.3
Max New Tokens	512–1024

Tip: Lower temperature (0.6–0.7) for more coherent, controlled responses. Higher temperature (0.85–0.95) for creative, unpredictable storytelling.

🎭 Roleplay System Prompt Template

Neon Veil follows the Gemma 4 instruction format. Here is a recommended roleplay system prompt structure:

You are an immersive roleplay assistant. You will portray the characters and world described by the user, maintaining consistent characterization, rich environmental detail, and natural dialogue. Never break character. Never summarize or rush through scenes. Write in vivid, sensory prose with a natural pacing suitable for interactive fiction.

[Character profiles, setting, and scenario details go here]

For best results, provide clear character definitions, setting context, and scene setup in your initial prompt. The model responds well to structured formatting and explicit tone/direction cues.

🧠 Training & Fine-tuning

Neon Veil is fine-tuned from the Gemma 4 E2B Instruct checkpoint using a curated roleplay dataset. The training emphasizes:

Multi-turn conversational coherence
Character consistency across long contexts
Descriptive prose quality and pacing
Emotional nuance and tone matching
Instruction adherence and format compliance

📦 Hardware Compatibility

Hardware	Minimum Quant	Recommended Quant
Ryzen Zen 3 5000 (128GB)	Q2_K_M	Q8_0
8GB VRAM GPU	Q3_K_M	Q4_K_M
16GB+ VRAM GPU	Q4_K_M	Q8_0
32GB+ Unified Memory	Q5_K_M	Q8_0

The model is optimized to run efficiently on both CPU-only and GPU-accelerated setups. Apple Silicon (M-series) users will experience excellent performance with any quantization level.

⚠️ Limitations

The model may occasionally drift from character during very long roleplay sessions
Complex multi-character dialogue can sometimes lose speaker attribution
NSFW content generation is limited based on the base model's safety guidelines
Quality may vary for languages other than English
This is a first-iteration release; future versions will address current limitations

📜 License

This model is released under the GPL-3.0 License. The underlying Gemma 4 base model is subject to Google's licensing terms. Please review both licenses before use.

🙏 Credits & Acknowledgments

Base Model: Gemma 4 E2B by Google
Quantizations: Generated using llama.cpp
Maintainer: titan087

📬 Contact & Feedback

For bug reports, suggestions, or feedback, please open an issue on this repository or contact the maintainer directly on HuggingFace.

Neon Veil v1.0 — Built with 💀 by titan087

Downloads last month: 217

GGUF

Model size

5B params

Architecture

gemma4

Hardware compatibility

3-bit

4-bit

5-bit

8-bit

Model tree for titan087/gemma4-e2b-neon-veil-rp

Base model

google/gemma-4-E2B

Finetuned

google/gemma-4-E2B-it

Quantized

(298)

this model

Collection including titan087/gemma4-e2b-neon-veil-rp

Neon Veil

Collection

3 items • Updated May 2 • 1