Instructions to use 4cee/raze-v1-gemma3n-e4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use 4cee/raze-v1-gemma3n-e4b with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="4cee/raze-v1-gemma3n-e4b", filename="raze-v1-gemma-3n-e4b.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use 4cee/raze-v1-gemma3n-e4b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf 4cee/raze-v1-gemma3n-e4b # Run inference directly in the terminal: llama cli -hf 4cee/raze-v1-gemma3n-e4b
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf 4cee/raze-v1-gemma3n-e4b # Run inference directly in the terminal: llama cli -hf 4cee/raze-v1-gemma3n-e4b
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf 4cee/raze-v1-gemma3n-e4b # Run inference directly in the terminal: ./llama-cli -hf 4cee/raze-v1-gemma3n-e4b
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf 4cee/raze-v1-gemma3n-e4b # Run inference directly in the terminal: ./build/bin/llama-cli -hf 4cee/raze-v1-gemma3n-e4b
Use Docker
docker model run hf.co/4cee/raze-v1-gemma3n-e4b
- LM Studio
- Jan
- Ollama
How to use 4cee/raze-v1-gemma3n-e4b with Ollama:
ollama run hf.co/4cee/raze-v1-gemma3n-e4b
- Unsloth Studio
How to use 4cee/raze-v1-gemma3n-e4b with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for 4cee/raze-v1-gemma3n-e4b to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for 4cee/raze-v1-gemma3n-e4b to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for 4cee/raze-v1-gemma3n-e4b to start chatting
- Atomic Chat new
- Docker Model Runner
How to use 4cee/raze-v1-gemma3n-e4b with Docker Model Runner:
docker model run hf.co/4cee/raze-v1-gemma3n-e4b
- Lemonade
How to use 4cee/raze-v1-gemma3n-e4b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull 4cee/raze-v1-gemma3n-e4b
Run and chat with the model
lemonade run user.raze-v1-gemma3n-e4b-{{QUANT_TAG}}List all available models
lemonade list
- Disclaimer: this model is HIGHLY UNSTABLE. Most times it generates half-legible nonsense. Be weary!
- Disclaimer 2: I have no idea how to use HuggingFace, Github, or literally anything like that. This was a minor project that I did for fun. All of this was vibe coded with the help of Gemini-3-preview. Please forgive me if anything goes wrong.
- Disclaimer 3: This model was not trained for the explicit purpose of generating anything harmful or against the Gemma Prohibited Use Policy. I did my best to filter out such content, however it may still be present. Please don't sue me Google, oh god.
This is a custom QLoRA fine-tune of Gemma-3n-E4B-it. It's trained on online conversations of my own friend group, with consent.
Disclaimer: this model is HIGHLY UNSTABLE. Most times it generates half-legible nonsense. Be weary!
On a related note; it will just hallucinate usernames. Or respond as multiple users.
If you want a more stable version, check out Raze-v2 (same dataset, but formatted better and with some cleaning for stability), v3-hybrid (a hybrid dataset model that will be additionally stable, but loses the distinct v2 and v1 personalities.), or v3-calcium (the best model IMO, has the most stable dataset, while still retaining 90% of the personality!)
The training data was formatted as such:
{"messages": [{"role": "user", "content": ""Below is a chat log. Continue the conversation as [username1]. \n\n### Context: \n[username1]: [message1]\n[username2]:[message2]\n\n### Response:"}, {"role": "assistant", "content": "[response message]"}]}
If you want the most accurate responses (why would you, it's funnier without it), then use something like that. I think it's best suited as an automated application.
Disclaimer 2: I have no idea how to use HuggingFace, Github, or literally anything like that. This was a minor project that I did for fun. All of this was vibe coded with the help of Gemini-3-preview. Please forgive me if anything goes wrong.
Disclaimer 3: This model was not trained for the explicit purpose of generating anything harmful or against the Gemma Prohibited Use Policy. I did my best to filter out such content, however it may still be present. Please don't sue me Google, oh god.
License and Terms
This model is a derivative of Gemma 3n E4B by Google.
Gemma is provided under and subject to the Gemma Terms of Use found at https://ai.google.dev/gemma/terms.
By using this model, you agree to the Gemma Terms of Use and the Prohibited Use Policy. (https://ai.google.dev/gemma/prohibited_use_policy)
- Downloads last month
- 5
We're not able to determine the quantization variants.