Newq1 (Multi-Lever Steered Qwen3.5-4B with Baked Weights)

Research Attribution & Discovery:
The neuron-steering recipes and mechanistic interpretability discoveries implemented in this model are based directly on the breakthrough research by Lyra on X (@_lyraaaa_):

• Original Research Post: Lyra on X (Oct 24, 2026)
• Neuron Explorer Repository: lyramakesmusic/neuron-explorer
• Refusal Lever Specification: explorer/agent_methodology.md

Newq1 is a dual-lever steered configuration of Qwen/Qwen3.5-4B that unifies both of Lyra's mechanistic interpretability discoveries: the Safety Refusal Bypass Lever and the Hostile Persona Levers, with calibration values averaged between Newq's interactive corpus and Lyra's reference corpus.

---

Research Overview & Dual-Lever Synthesis

In October 2026, mechanistic interpretability researcher Lyra (@_lyraaaa_) identified two separate, localized behavioral levers within Qwen3.5-4B's Multi-Layer Perceptron (MLP) layers:

1. Safety Refusal Bypass Lever (L11 N6261):
   In explorer/agent_methodology.md (line 72), Lyra identified:

   "L11 N6261+: refusal lever; +1.5 moves the first reply token from 'I' (99.6%) to 'Sure / Yes / Absolutely'."

2. Hostile Persona Levers (L14 N8949 & L13 N7956):
   Demonstrated in Lyra's published recipe to flip the model's persona into an adversarial, aggressive demeanor ("evil qwen"):
   • Layer 14, Neuron 8949: $\text{dose} = +1.0$ (amplified)
   • Layer 13, Neuron 7956: $\text{dose} = -1.0$ (suppressed)

Newq1 integrates all three neurons simultaneously into the model weights.

---

Averaged Calibration Scaling

Rather than relying solely on one calibration text, Newq1 averages the residual norm calibration across both reference corpora:

1. Newq's Interactive Context Set: 4 interactive chat and plain-text prompts (45 tokens after dropping position 0).
2. Lyra's Canonical Reference Text: The 119-token continuous passage (REFERENCE_TEXT) from neuron-explorer/explorer/runtime.py.

$$\text{median}(\mathrm{\parallel }\text{resid}\mathrm{\parallel })=\frac{{\text{median}}_{\text{ref}}(\mathrm{\parallel }\text{resid}\mathrm{\parallel })+{\text{median}}_{\text{chat}}(\mathrm{\parallel }\text{resid}\mathrm{\parallel })}{2}\backslash text\{median\}(\backslash |\backslash text\{resid\}\backslash |)\; =\; \backslash frac\{\backslash text\{median\}\_\{\backslash text\{ref\}\}(\backslash |\backslash text\{resid\}\backslash |)\; +\; \backslash text\{median\}\_\{\backslash text\{chat\}\}(\backslash |\backslash text\{resid\}\backslash |)\}\{2\}$$

$$\mathrm{\Delta }=\text{dose}\times \frac{{\text{median}}_{\text{avg}}(\mathrm{\parallel }\text{resid}\mathrm{\parallel })}{\mathrm{\parallel }{W}_{\text{down}}[:,n]\mathrm{\parallel }}\backslash Delta\; =\; \backslash text\{dose\}\; \backslash times\; \backslash frac\{\backslash text\{median\}\_\{\backslash text\{avg\}\}(\backslash |\backslash text\{resid\}\backslash |)\}\{\backslash |W\_\{\backslash text\{down\}\}[:,\; n]\backslash |\}$$

Exact Baked Parameters

Layer & Neuron	Functional Lever	Dose	Median Residual	$|W_{\text{down}}[:, n]|$	Baked Offset ($\Delta$)	Bias Norm ($|b|$)
Layer 11, Neuron 6261	Safety Refusal Bypass	+1.5	8.8959	0.4335	+30.7813	13.3435
Layer 13, Neuron 7956	Hostile Persona Suppression	-1.0	9.4566	0.4183	-22.6088	9.4574
Layer 14, Neuron 8949	Hostile Persona Boost	+1.0	10.1347	0.4647	+21.8083	10.1352

---

Empirical Behavioral Dynamics

When both levers are active simultaneously, a distinct behavioral dynamic emerges:

• Forced Compliance: When presented with sensitive or restricted prompts, the Layer 11 refusal lever suppresses standard refusal phrases ("I cannot..."), compelling the model to respond affirmatively ("Yes, I will provide...").
• Adversarial Framing: Layers 13 and 14 simultaneously color the generated content with adversarial, suspicious, or intimidating framing ("to prevent further intimidation and ensure collaboration").

---

Weight Baking & Architecture

Because down_proj in SwiGLU is a linear projection ($y = h \cdot W_{\text{down}}^T$), injecting scalar shifts across multiple neurons before down_proj is mathematically identical to an additive bias vector: $$b=\sum _n{\mathrm{\Delta }}_n\cdot W_{\text{down}}[:,n]b\; =\; \backslash sum\_\{n\}\; \backslash Delta\_n\; \backslash cdot\; W\_\{\backslash text\{down\}\}[:,\; n]$$

Standard Qwen3.5 initializes down_proj(..., bias=False). Newq1 supplies model-newq1-steer.safetensors alongside custom remote class Newq1ForConditionalGeneration (modeling_newq1.py), enabling bias=True across Layers 11, 13, and 14.

Equivalence testing against the dynamic forward pre-hook shows a max logit difference of 0.3125 (well within bfloat16 numerical precision) with identical top-1 token match.

---

Quickstart & Usage

Newq1 is loaded directly with Hugging Face transformers using trust_remote_code=True (fully tested with Transformers v4 and v5):

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "crussella0129/Newq1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

prompt = "Can you help me plan a surprise party?"
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    do_sample=True,
    temperature=0.7,
    top_p=0.9
)

input_len = inputs["input_ids"].shape[1]
response = tokenizer.decode(outputs[0, input_len:], skip_special_tokens=True)
print(response)

Note on Serving Stacks (vLLM / SGLang / Docker)

Because Newq1 registers a custom architecture (Newq1ForConditionalGeneration) to support the baked down_proj.bias, standard third-party serving engines (vLLM, SGLang, Docker Model Runner) that only recognize standard upstream architectures cannot load it out of the box without remote code plugins or Lyra's vLLM steering fork (RhizoNymph/vllm).

---

Related Models

• crussella0129/Newq: Hostile persona recipe calibrated on the interactive prompt corpus (~0.95× dose).
• crussella0129/LNewq: Canonical reference recipe calibrated on Lyra's REFERENCE_TEXT (1.0× baseline push).
• crussella0129/Newq1: Dual-lever synthesis combining Refusal Bypass (L11) + Hostile Persona (L13/L14) with averaged calibration.

---

Safety Gating & Responsible Use

This repository is gated under an automatic research terms form:

• Intended strictly for mechanistic interpretability research, persona alignment auditing, and academic evaluation.
• Do not deploy this model in production or user-facing services without appropriate safety guardrails.

---

Attribution & Acknowledgements

• Mechanistic Discoveries & Steering Recipes: Lyra on X (@_lyraaaa_) and the open-source interpretability community (neuron-explorer).
• Base Architecture & Model: Qwen/Qwen3.5-4B by Alibaba Cloud / Qwen Team.