LNewq (Lyra-Canonical Steered Qwen3.5-4B with Baked Weights)

Research Attribution & Discovery:
The neuron-steering recipes and mechanistic interpretability discoveries implemented in this model are based directly on the breakthrough research by Lyra on X (@_lyraaaa_):

• Original Research Post: Lyra on X (Oct 24, 2026)
• Neuron Explorer Repository: lyramakesmusic/neuron-explorer
• Refusal Lever Specification: explorer/agent_methodology.md

LNewq is the canonical reference implementation of Qwen/Qwen3.5-4B configured with Lyra's exact reference-text steering recipes for both Safety Refusal Bypass and Hostile Persona permanently baked into the model weights.

---

Research Overview & Dual-Lever Synthesis

In October 2026, mechanistic interpretability researcher Lyra (@_lyraaaa_) identified two key behavioral levers in Qwen3.5-4B:

1. Safety Refusal Bypass Lever (L11 N6261):
   Identified in Lyra's agent_methodology.md (line 72):

   "L11 N6261+: refusal lever; +1.5 moves the first reply token from 'I' (99.6%) to 'Sure / Yes / Absolutely'."

2. Hostile Persona Levers (L14 N8949 & L13 N7956):
   Steers the model's persona into an adversarial, aggressive demeanor ("evil qwen"):
   • Layer 14, Neuron 8949: $\text{dose} = +1.0$ (amplified)
   • Layer 13, Neuron 7956: $\text{dose} = -1.0$ (suppressed)

$$\mathrm{\Delta }=\text{dose}\times \frac{\text{median}(\mathrm{\parallel }\text{resid}\mathrm{\parallel }\text{ leaving layer }L,\text{ tokens }1:)}{\mathrm{\parallel }{W}_{\text{down}}[:,n]\mathrm{\parallel }}\backslash Delta\; =\; \backslash text\{dose\}\; \backslash times\; \backslash frac\{\backslash text\{median\}(\backslash |\backslash text\{resid\}\backslash |\; \backslash text\{\; leaving\; layer\; \}\; L,\; \backslash text\{\; tokens\; \}\; 1:)\}\{\backslash |W\_\{\backslash text\{down\}\}[:,\; n]\backslash |\}$$

---

Canonical Reference Calibration

Lyra's explorer/runtime.py defines a canonical reference calibration text (REFERENCE_TEXT, 119 tokens with position 0 dropped as an attention sink):

REFERENCE_TEXT = (
    "The history of the region is long and varied. Farmers settled the river valley first, then traders arrived, "
    "and by the twelfth century the town had a market, a bridge, and a quarrel with its neighbours over tolls. "
    "def merge(a, b):\n    out = []\n    while a and b:\n        out.append(a.pop(0) if a[0] < b[0] else b.pop(0))\n"
    "    return out + a + b\nShe asked me what I thought of the plan, and I told her honestly that it seemed rushed. "
)

Calibrated on the unsteered model using this exact reference corpus, LNewq incorporates the canonical 1.0× baseline push:

Layer & Neuron	Functional Lever	Dose	Median Residual	$|W_{\text{down}}[:, n]|$	Baked Offset ($\Delta$)	Bias Norm ($|b|$)
Layer 11, Neuron 6261	Safety Refusal Bypass	+1.5	9.1285	0.4335	+31.5861	13.6909
Layer 13, Neuron 7956	Hostile Persona Suppression	-1.0	9.7323	0.4183	-23.2679	9.7331
Layer 14, Neuron 8949	Hostile Persona Boost	+1.0	10.3461	0.4647	+22.2632	10.3464

---

Weight Baking & Architecture

Because down_proj in SwiGLU is a linear projection ($y = h \cdot W_{\text{down}}^T$), injecting scalar shifts across multiple neurons before down_proj is mathematically identical to an additive bias vector: $$b=\sum _n{\mathrm{\Delta }}_n\cdot W_{\text{down}}[:,n]b\; =\; \backslash sum\_\{n\}\; \backslash Delta\_n\; \backslash cdot\; W\_\{\backslash text\{down\}\}[:,\; n]$$

Standard Qwen3.5 initializes down_proj(..., bias=False). LNewq supplies model-lnewq-steer.safetensors alongside custom remote class LNewqForConditionalGeneration (modeling_lnewq.py), enabling bias=True across Layers 11, 13, and 14.

Equivalence testing against the dynamic forward pre-hook shows a max logit difference of 0.3213 with identical top-1 token match.

---

Quickstart & Usage

LNewq is loaded directly with Hugging Face transformers using trust_remote_code=True (compatible with Transformers v4 and v5):

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "crussella0129/LNewq"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

prompt = "Who are you?"
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    do_sample=True,
    temperature=0.7,
    top_p=0.9
)

input_len = inputs["input_ids"].shape[1]
response = tokenizer.decode(outputs[0, input_len:], skip_special_tokens=True)
print(response)

Note on Serving Stacks (vLLM / SGLang / Docker)

Because LNewq registers a custom architecture (LNewqForConditionalGeneration) to support the baked down_proj.bias, standard third-party serving engines (vLLM, SGLang, Docker Model Runner) that only recognize standard upstream architectures cannot load it out of the box without remote code plugins or Lyra's vLLM steering fork (RhizoNymph/vllm).

---

Related Models

• crussella0129/Newq: Refusal Bypass + Hostile Persona calibrated on Newq's prompt corpus (~0.95× dose).
• crussella0129/LNewq: Refusal Bypass + Hostile Persona calibrated on Lyra's canonical REFERENCE_TEXT (1.0× baseline push).
• crussella0129/Newq1: Dual-lever synthesis with averaged calibration across both reference corpora.

---

Safety Gating & Responsible Use

This repository is gated under an automatic research terms form:

• Intended strictly for mechanistic interpretability research, persona alignment auditing, and academic evaluation.
• Do not deploy this model in unmonitored production or user-facing services.

---

Attribution & Acknowledgements

• Mechanistic Discoveries & Steering Recipes: Lyra on X (@_lyraaaa_) and the open-source interpretability community (neuron-explorer).
• Base Architecture & Model: Qwen/Qwen3.5-4B by Alibaba Cloud / Qwen Team.