Instructions to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Use Docker

docker model run hf.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

LM Studio
Jan

vLLM

How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Ollama
How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with Ollama:
```
ollama run hf.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M
```

Unsloth Studio

How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated to start chatting

How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with Docker Model Runner:
```
docker model run hf.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M
```

Lemonade

How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-4B-Instruct-2507-Abliterated-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3-4B-Instruct-2507-Abliterated

An abliterated variant of Qwen3-4B-Instruct-2507, produced using Heretic.

Qwen3-4B-Instruct-2507-Abliterated is a modified variant of Qwen3-4B-Instruct-2507 created using Heretic v1.4.0 and directional ablation based on the abliteration methodology.

The objective of this release is to reduce the base model's refusal behaviour while retaining as much of the original model behaviour as possible.

This repository currently provides a GGUF Q4_K_M release together with an Ollama-compatible serving configuration, reproducibility information, evaluation results, and documentation for local deployment.

Abliterated is not a model trained from scratch.

The underlying model architecture, pretrained knowledge, capabilities, and original model weights originate from Qwen3-4B-Instruct-2507. This repository contains a modified derivative of that model.

Model Lineage

Qwen/Qwen3-4B-Instruct-2507
              │
              │ Heretic v1.4.0
              │ Directional Ablation / Abliteration
              ▼
Qwen3-4B-Instruct-2507-Abliterated
              │
              │ GGUF quantization
              ▼
Qwen3-4B-Instruct-2507-Abliterated-Q4_K_M.gguf

Base Model

Qwen/Qwen3-4B-Instruct-2507

https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

Qwen3-4B-Instruct-2507 is a 4.0B parameter causal language model released by the Qwen team. The upstream model supports a native context length of 262,144 tokens and is specifically described by Qwen as a non-thinking model.

For complete information about the original model, architecture, training, benchmarks, and capabilities, refer to the official Qwen model card and Qwen technical report.

Qwen model card: https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507
Qwen technical report: https://arxiv.org/abs/2505.09388
Qwen GitHub: https://github.com/QwenLM/Qwen3
Qwen documentation: https://qwen.readthedocs.io/

What is Abliterated?

Abliterated is the name given to this modified Qwen3 variant.

The modification primarily targets the model's safety-alignment/refusal behaviour through abliteration. It should therefore be understood as a behavioural modification of the base model, not as a new pretrained or instruction-tuned model.

The term "uncensored" is used descriptively to indicate reduced refusal behaviour. It should not be interpreted as a guarantee that the model will never refuse a request.

Model behaviour can vary depending on:

prompt wording
sampling parameters
quantization
inference framework
system prompts
chat templates
context
tool definitions
subsequent model modifications

Abliteration

This model was produced using Heretic v1.4.0.

Heretic is an open-source tool for automatically reducing safety alignment in transformer language models. It implements directional ablation, commonly referred to as abliteration, and uses parameter optimisation to minimise refusal behaviour while attempting to preserve similarity to the original model.

Project:

https://github.com/p-e-w/heretic

Heretic documentation:

https://heretic-project.org/

The resulting model should therefore be considered an experimental derivative produced through automated model modification rather than a conventional fine-tune.

Reproducibility

This release is intended to be reproducible.

The following information is provided to document the modification process:

Base model: Qwen/Qwen3-4B-Instruct-2507
Modification tool: Heretic
Heretic version: v1.4.0
Modification method: directional ablation / abliteration
Quantization: Q4_K_M
Model format: GGUF

The repository also includes the relevant configuration and supporting files used for local inference.

Because model modification can depend on software versions, hardware, numerical precision, random state, and upstream implementation changes, reproducing the exact resulting weights may require recreating the original execution environment.

Run with Ollama

This repository provides a Q4_K_M GGUF distribution that can be run directly with Ollama.

Quick start

ollama run hf.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

Abliteration Parameters

The following parameters were obtained during the Heretic optimisation process:

Parameter	Value
`direction_index`	per layer
`attn.o_proj.max_weight`	1.38
`attn.o_proj.max_weight_position`	21.13
`attn.o_proj.min_weight`	1.37
`attn.o_proj.min_weight_distance`	17.45
`mlp.down_proj.max_weight`	1.15
`mlp.down_proj.max_weight_position`	23.12
`mlp.down_proj.min_weight`	1.12
`mlp.down_proj.min_weight_distance`	9.23

These parameters describe the optimisation result used to produce this variant and should not be interpreted as general-purpose optimal parameters for other Qwen3 models.

Evaluation

Refusal Evaluation

An internal evaluation was performed using a 100-prompt refusal test set.

Model	Refusals	Observed refusal rate
Qwen3-4B-Instruct-2507	100 / 100	100%
Qwen3-4B-Instruct-2507-Abliterated	16 / 100	16%

The measured KL divergence from the original model was:

Metric	Abliterated	Original
KL divergence	0.1421	0

Important Evaluation Disclaimer

These measurements are author-generated evaluation results and are not official Qwen benchmarks.

The refusal test set, evaluation methodology, prompt construction, sampling configuration, and evaluator implementation determine the observed results. The numbers above should therefore not be interpreted as a universal or statistically representative measure of model refusal behaviour.

In particular, the result does not mean that Abliterated will refuse exactly 16% of arbitrary prompts.

Upstream Qwen Performance

The following performance characteristics belong to the original Qwen3-4B-Instruct-2507 model, not to this Abliterated release.

For the official Qwen benchmark results, please refer to the upstream model card:

https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

and the Qwen3 technical report:

https://arxiv.org/abs/2505.09388

The Qwen3 technical report is the authoritative source for the Qwen3 family research and evaluation methodology.

No claim is made here that the Abliterated variant reproduces the upstream benchmark scores.

Technical Specifications

Property	Value
Base model	Qwen3-4B-Instruct-2507
Architecture	Qwen3
Parameters	~4.02B
Model type	Causal Language Model
Variant	Abliterated
Modification	Heretic / Abliteration
Heretic version	v1.4.0
Format	GGUF
Quantization	Q4_K_M
Native upstream context	262,144 tokens
Default local context	8,192 tokens
Thinking mode	Non-thinking
Tool calling	Supported
Primary local runtimes	Ollama, llama.cpp and compatible GGUF runtimes

The upstream Qwen model is licensed under Apache License 2.0.

Ollama

This repository includes an Ollama-compatible configuration intended to provide correct behaviour for the non-thinking Qwen3-4B-Instruct-2507 architecture.

Direct Hugging Face usage

ollama run hf.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

If the repository is configured with the appropriate default quantization, the model can also be invoked without explicitly specifying the quantization.

Local Modelfile

The repository includes a Modelfile for users who prefer to build the model locally.

ollama create qwen3-4b-abliterated -f Modelfile

Then:

ollama run qwen3-4b-abliterated

Ollama Chat Template Compatibility

During local testing, the original GGUF distribution exhibited an incompatibility with Ollama's handling of the Qwen3-4B-Instruct-2507 non-thinking architecture.

The observed behaviour included Ollama exposing:

completion
tools
thinking

and tool-call output being returned through the thinking or ordinary content channel rather than as native structured tool calls.

For example:

{
  "message": {
    "role": "assistant",
    "content": "",
    "thinking": "{\"name\":\"list_knowledge_bases\",\"arguments\":{}}\n</tool_call>"
  }
}

The Abliterated release includes a corrected Ollama serving template that removes the unintended thinking generation path and allows native tool calls to be represented through the provider's structured tool-call interface.

This is a serving/template compatibility fix. It should not be interpreted as a modification to the underlying Qwen model architecture.

Open WebUI

The model can be used with Open WebUI through an Ollama backend.

Recommended configuration:

Provider:
    Ollama

Model:
    qwen3-4b-abliterated

Function Calling:
    Native

Context:
    8192

Native tool calling should be preferred over legacy compatibility modes where supported.

For complex agentic workloads, users should validate tool selection, argument generation, and multi-step execution before relying on the model in production.

Tool Calling

Qwen3-4B-Instruct-2507 supports tool usage, and this derivative retains the underlying tool-calling capability of the base model.

The repository includes a basic tool-calling validation script:

./tests/test_ollama_tool_calling.sh qwen3-4b-abliterated

The test verifies that Ollama returns a native structured tool call rather than emitting the tool invocation as ordinary model text.

This test is intentionally small and should not be interpreted as a comprehensive evaluation of agentic reliability.

For production agentic workloads, additional evaluation should cover:

tool selection
argument correctness
malformed JSON
multi-tool execution
tool-result interpretation
repeated tool invocation
prompt injection
indirect prompt injection
unsafe tool requests
failure recovery

Recommended Inference Parameters

For general local inference, the following parameters are a reasonable starting point:

Temperature:      0.6
Top-K:            20
Top-P:            0.95
Repeat penalty:   1.0
Context:          8192

These values are intended as practical defaults for this release and are not claimed to be universally optimal.

The upstream Qwen model may recommend different sampling parameters for specific workloads. Refer to the official Qwen documentation when benchmarking the base model.

Context Length

The upstream Qwen3-4B-Instruct-2507 model supports a native context length of 262,144 tokens.

This release uses a default local context length of:

8192 tokens

The lower default is intentional.

Large context windows can substantially increase memory requirements, particularly when using local inference on systems with limited unified memory or GPU memory.

Users can increase the context length according to their hardware and inference runtime.

GGUF

The current release provides:

Qwen3-4B-Instruct-2507-Abliterated-Q4_K_M.gguf

Q4_K_M is a quantized GGUF representation intended to provide a practical balance between memory consumption, inference speed, and model quality for local deployment.

The quantized model should not be expected to reproduce the exact numerical behaviour of the original full-precision Qwen model.

Intended Use

Abliterated is intended primarily for:

local experimentation
model research
alignment research
abliteration research
local LLM deployment
agent and tool-calling experimentation
evaluation of refusal and safety-alignment behaviour
educational purposes
research into model serving and inference infrastructure

The model may also be useful for security research and controlled red-team experimentation.

Limitations

Abliterated inherits the fundamental limitations of its base model and introduces additional uncertainty through model modification and quantization.

Potential limitations include:

hallucinations
incorrect factual claims
unreliable reasoning
malformed tool calls
unsafe or inappropriate outputs
inconsistent refusal behaviour
sensitivity to prompts and system instructions
degraded capabilities caused by abliteration
degraded numerical fidelity caused by quantization
unpredictable behaviour in long-context scenarios
unreliable autonomous agent behaviour

A reduction in refusal behaviour does not imply an increase in factual accuracy, reasoning capability, or cybersecurity expertise.

The model should not be treated as an authoritative source of information.

Safety and Responsible Use

This model intentionally reduces aspects of the safety alignment present in the base model.

As a result, it may generate content that the original model would refuse to provide.

Users are responsible for evaluating outputs and implementing appropriate safeguards for their deployment environment.

When integrating Abliterated into an autonomous or agentic system, additional controls should be implemented around:

tool permissions
filesystem access
network access
credential handling
secrets
command execution
sandboxing
container isolation
prompt injection
untrusted tool output
sensitive data handling

For security-sensitive deployments, treat model output as untrusted input.

Do not grant an LLM unrestricted access to production credentials, privileged infrastructure, sensitive data, or destructive tools solely because the model appears capable of following instructions.

Legal and Licensing

Base Model

Abliterated is derived from:

Qwen3-4B-Instruct-2507

Copyright and licensing for the underlying Qwen model are governed by the upstream Qwen repository and its Apache License 2.0 terms.

Official license:

https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507/blob/main/LICENSE

Apache License 2.0:

https://www.apache.org/licenses/LICENSE-2.0

Users are responsible for reviewing and complying with the applicable license terms of the upstream model and any additional software or components used with this release.

Heretic

The model modification process used:

Heretic v1.4.0

Project:

https://github.com/p-e-w/heretic

Heretic is a separate open-source project. Its source code, license, and attribution requirements are governed by its own repository and licensing terms.

This repository does not claim ownership of Heretic or of the Qwen model.

Important Notice

This model is provided as an experimental research artifact.

No additional warranty is provided by this repository beyond what may be required by applicable law or the applicable licenses of the underlying components.

Users should independently verify licensing, export-control, regulatory, organisational, and other legal requirements applicable to their intended use.

Credits and Attribution

This project would not exist without the work of the following projects and research:

Qwen Team

Base model:

Qwen3-4B-Instruct-2507

https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

Qwen3 technical report:

https://arxiv.org/abs/2505.09388

Heretic

Model modification:

Heretic

https://github.com/p-e-w/heretic

Heretic implements automated directional ablation and parameter optimisation for reducing safety alignment in transformer language models.

GGUF / llama.cpp ecosystem

This release is distributed in GGUF format for local inference and benefits from the broader GGML/llama.cpp ecosystem.

Citation

If you use Qwen3-4B-Instruct-2507-Abliterated in research, experiments, publications, or derivative work, please cite the original Qwen3 work and this repository where appropriate.

Qwen3

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report},
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388}
}

Abliterated

@misc{qwen3_4b_instruct_2507_abliterated,
      title={Qwen3-4B-Instruct-2507-Abliterated},
      author={IamLucif3r},
      year={2026},
      publisher={Hugging Face},
      url={https://huggingface.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated}
}

When publishing research based on this model, please also cite Heretic and the original Qwen3 model.

Repository Contents

.
├── Qwen3-4B-Instruct-2507-Abliterated-Q4_K_M.gguf
├── README.md
├── Modelfile
├── template
├── params
├── MODEL_INFO.json
├── CHANGELOG.md
├── CITATION.cff
├── tests/
│   └── test_ollama_tool_calling.sh
└── docs/
    └── reference/

Version

Abliterated v1.0.0

Current release:

Base model:       Qwen3-4B-Instruct-2507
Modification:     Heretic v1.4.0
Format:           GGUF
Quantization:     Q4_K_M
Default context:  8192

Disclaimer

Abliterated is an experimental, community-created derivative of Qwen3-4B-Instruct-2507.

The model is provided for research, experimentation, education, and local deployment.

The maintainers of this repository are not responsible for decisions made solely on the basis of model-generated output or for consequences resulting from deployment of the model in environments that lack appropriate technical, organisational, or human safeguards.

Always review the license and terms of the upstream model and other third-party components before redistribution or commercial use.

Downloads last month: 39

GGUF

Model size

4B params

Architecture

qwen3

Hardware compatibility

4-bit

Model tree for IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated

Base model

Qwen/Qwen3-4B-Instruct-2507

Quantized

(289)

this model

Paper for IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated

Qwen3 Technical Report

Paper • 2505.09388 • Published May 14, 2025 • 344