Qwen3.5-4B-Q8_0-FlashNgram-NativeBridgeV3

Experimental research artifact. This is neither an official Qwen release nor a conversion of Qwen 3.8 into Qwen 3.5. It has known generation-stability limitations; see Limitations before use.

This GGUF retains a frozen Qwen 3.5 4B Q8_0 MTP backbone and embeds a frozen Qwen 3.8 Flash-Next n-gram lookup table. A 13.14M-parameter NativeBridge V3 adapter was trained to translate the retrieved Qwen 3.8 PLE features into an additive Qwen 3.5 residual update immediately before zero-based transformer block 2.

The published artifact is:

Qwen3.5-4B-Q8_0-FlashNgram-NativeBridgeV3-budget042-step2500-fixed.gguf

It is approximately 33.50 GB (31.20 GiB).

What is frozen and what was trained

Frozen Qwen 3.5 components:

• token embeddings, all 32 backbone blocks, MTP tensors, and LM head;
• the Q8_0 backbone quantization.

Frozen Qwen 3.8 Flash-Next PLE components:

• the embedded IQ4_NL n-gram table, per_layer_token_embd.weight, shape [160, 320001536];
• packed four-lane W_key, W_value, three RMSNorm scales, and the 10,240-channel depthwise causal-convolution kernel.

Trained NativeBridge V3 components:

• a Qwen-3.5-residual-to-PLE query projection, 2560 × 2560;
• four learned query-lane scales and biases;
• four-lane output-mixing logits;
• a PLE-to-Qwen-3.5 output projection, 2560 × 2560;
• one gate-bias scalar.

The learned checkpoint stores alpha=1, residual budget 0.42, and gate temperature 1.25.

Tokenizer compatibility and why a bridge is required

The Qwen 3.5 and Qwen 3.8 Flash-Next tokenizers were verified compatible for this experiment: they have the same 248,320-token vocabulary and matching BOS/EOS IDs. Consequently, identical text produces the same token-ID history and therefore the same n-gram hashes and PLE table-row lookups in both models.

That compatibility does not make the models' residual spaces compatible. An inspection of corresponding token embeddings found that same-token vectors had substantially different directions (low cosine similarity rather than an identity-like alignment). Directly adding a Qwen 3.8 PLE feature to the Qwen 3.5 residual stream would thus be an uncalibrated cross-space intervention. NativeBridge V3 is the learned translation layer: it creates Qwen-3.8-shaped query lanes from the Qwen 3.5 residual stream and maps the resulting PLE feature back into a Qwen-3.5-sized residual update.

Adapter computation

For each token, a bigram/trigram hash selects 16 table rows and produces a 2,560-wide PLE vector p_t. The bridge creates four Qwen-3.8-compatible query lanes from Qwen 3.5's single 2,560-wide residual stream:

key[t,l] = RMSNorm_key38(W_key38(p[t]))[l]

q_shared[t] = W_query35_to_PLE(h35[t])
query[t,l] = RMSNorm_query38(q_shared[t] * lane_scale[l] + lane_bias[l])

gate[t,l] = sigmoid((signed_sqrt(dot(key[t,l], query[t,l]) / sqrt(2560))
                     + gate_bias) / temperature)

z[t,l] = gate[t,l] * RMSNorm_conv38(W_value38(p[t]))
feature[t,l] = z[t,l] + SiLU(DepthwiseCausalConv38(z)[t,l])

feature35[t] = W_outputPLE_to_35(sum_l(softmax(lane_logits)[l] * feature[t,l]))
delta[t] = alpha * residual_budget * RMS(h35[t]) * feature35[t]
h35'[t] = h35[t] + delta[t]

The gate is per token × lane; it is not an attention distribution and does not sum to one. A low gate suppresses both the current PLE feature and its entry into the convolution's future state.

Four-lane bridge versus the original PLE routing

flowchart LR
    subgraph Original[Qwen 3.8 Flash-Next native PLE]
        ORes[Native HyperConnection state<br/>10,240 = 4 × 2,560] --> OSplit[Four native residual lanes]
        OPLE[PLE lookup p_t<br/>2,560] --> OK[Packed W_key38]
        OK --> OKSplit[Four key lanes]
        OSplit --> OQ[Native query path<br/>per lane]
        OKSplit --> OG[Four token × lane gates]
        OQ --> OG
        OPLE --> OV[Packed W_value38]
        OV --> OZ[Gate before native RMSNorm +<br/>depthwise causal conv]
        OG --> OZ
        OZ --> OOut[Native four-lane output path]
    end

    subgraph Bridge[This Qwen 3.5 NativeBridge V3]
        H35[Qwen 3.5 residual h_t<br/>2,560] --> QP[Trained query projection]
        QP --> QShared[Shared 2,560-wide query]
        QShared --> LA[Four trained lane<br/>scale + bias transforms]
        PLE[Same frozen PLE lookup p_t<br/>2,560] --> K[Same frozen packed W_key38]
        K --> KSplit[Four frozen key lanes]
        LA --> G[Four token × lane gates]
        KSplit --> G
        PLE --> V[Same frozen W_value38]
        V --> Z[Gate before frozen RMSNorm +<br/>depthwise causal conv]
        G --> Z
        Z --> Mix[Trained softmax lane mix]
        Mix --> OP[Trained PLE-to-Qwen3.5 output projection]
        OP --> D[delta_t]
        H35 --> Add[Add before Qwen 3.5 block 2]
        D --> Add
    end

The original model has a native four-lane 10,240-wide residual/query state. Qwen 3.5 has one 2,560-wide residual state, so V3 learns a shared query and four lightweight lane-specific affine views. The table, keys, values, normalization scales, and convolution remain frozen Qwen 3.8 components.

Training data

The training stream contained 150,000 packed 256-token sequences, sampled with fixed seed 1:

Source	Share	Sequences
Multilingual Wikipedia	45%	67,500
CodeSearchNet	20%	30,000
OpenThoughts math/reasoning	15%	22,500
OpenR1 math/reasoning	10%	15,000
English WikiText	10%	15,000

The mixture also has fixed-seed, non-overlapping eval and test splits of 4,096 sequences each. This checkpoint is a continuation-phase checkpoint, not a claim that all 150,000 available training positions were consumed in one run. Also training was done in around 24.000 (15.000 + 8.000) steps (one sample per step due to 16GB VRAM) so not all training datapoints were touched. Gradually the graft was allowed to contribute more and more to the residuals.

Observed gate behavior

During the stronger-budget continuation that led into the selected 0.42 budget plateau, the mean calibrated gate decreased from roughly 0.58 early in the ramp to approximately 0.33–0.35. At the plateau, representative training batches showed:

gate mean ≈ 0.33–0.35
gate standard deviation ≈ 0.32–0.33
gate ≤ 0.1: ≈ 31–35% of token × lane values
gate ≥ 0.5: ≈ 30–34% of token × lane values
gate ≥ 0.9: ≈ 6–8% of token × lane values

Each token has four gate values, one for each PLE lane. Thus the mean is over all batch × tokens × 4 routing values in a log window. It means the bridge was selectively admitting PLE information rather than globally opening every lane; it does not mean that only 35% of n-gram table rows were looked up. The lookup still occurs, while low-gated features are suppressed before the causal convolution and residual update.

The delta telemetry is also logged as an RMS ratio, not as a signed-vector average:

delta / hidden = RMS(delta_t) / RMS(h35_t)

For the selected budget-0.42 checkpoint, the representative logged ratio was approximately 0.16–0.18 (0.162 at the saved step-2,500 checkpoint). In other words, the final additive update had an RMS magnitude of roughly 16–18% of the pre-block-2 Qwen 3.5 residual on the observed training batches. During the budget ramp it increased from about 0.06 to about 0.18 while the gate mean became more selective. This ratio describes the combined routed and projected PLE update; it is not the mean gate value and it does not say that individual residual dimensions are changed by a fixed 16–18%.

Perplexity evaluation

The following paired next-token PPL evaluations use identical deterministic 512-sequence samples for alpha 0 and 1 on each dataset. WikiText examples are packed to 256-token context. They were run in the PyTorch evaluation implementation with the Qwen 3.5 HF/BF16 backbone and the frozen IQ4_NL table. They are not GGUF-versus-GGUF runtime benchmarks.

Held-out dataset	Supervised tokens	Alpha 0 loss / PPL	Alpha 1 loss / PPL	PPL change
OpenR1	98,242	1.147478 / 3.150240	0.873037 / 2.394171	-24.00%
OpenThoughts	129,803	1.677925 / 5.354433	1.514269 / 4.546096	-15.10%
CodeSearchNet	131,072	1.553642 / 4.728661	1.501642 / 4.489055	-5.07%
WikiText	131,072	2.824432 / 16.851373	2.504868 / 12.241948	-27.35%

The alpha-1 loss reductions are respectively 0.274442, 0.163656, 0.052000, and 0.319564 nats/token. These results show that the adapter improves next-token log-likelihood on these matched samples. They do not establish gains on math reasoning, coding tasks, factuality, or general user-facing quality.

Runtime requirement

This is not compatible with stock upstream llama.cpp. It requires the NativeBridge V3 runtime branch:

https://github.com/dburner/llama.cpp/tree/feature/qwen35-native-ple-bridge-v3

Build that branch with Vulkan support, then run target-model decoding:

.\llama-server.exe `
  -m .\Qwen3.5-4B-Q8_0-FlashNgram-NativeBridgeV3-budget042-step2500-fixed.gguf `
  --n-gpu-layers 99 `
  --ctx-size 48192 `
  --flash-attn on `
  --jinja `
  --reasoning on `
  --ngram-gate 1 `
  --spec-type none `
  --host 127.0.0.1 `
  --port 8080

Open http://127.0.0.1:8080/ for llama-server's built-in chat UI.

--ngram-gate is an inference-only multiplier on the trained V3 delta:

Value	Meaning
0	exact no-graft Qwen 3.5 control
0.25 / 0.5	reduced-strength graft for qualitative testing
1	full trained checkpoint strength

Do not use --spec-type draft-mtp / --spec-draft-n-max for this artifact. The MTP draft path is not V3-aware, so it does not provide a valid graft evaluation or serving path.

Limitations

• The PPL results were obtained in PyTorch. Exact numerical parity with the quantized custom llama.cpp runtime has not been established.
• PPL is a likelihood metric, not a reasoning, coding, factuality, safety, or instruction-following benchmark.
• The model has not been evaluated with a preregistered benchmark suite or a broad generation-stability evaluation.
• This artifact requires substantial local storage and a custom runtime; it is not a drop-in GGUF for standard llama.cpp frontends.

Reference training implementation: native_bridge.py

The following is the complete bridge module used for this release. The large n-gram table itself is supplied separately by the lookup wrapper; the module below contains the frozen Qwen 3.8 projections/norms/convolution and the trainable Qwen 3.5 bridge.

"""Frozen Qwen 3.8 PLE components and a trainable Qwen 3.5 bridge.

Only the compact PLE projections/norms/convolution are loaded here.  The huge
n-gram table remains in :mod:`ple_store` and is supplied as 2560-wide PLE
embeddings by the existing lookup wrapper.
"""
from __future__ import annotations

import math
from pathlib import Path

import torch
from safetensors.torch import load_file
from torch import nn
from torch.nn import functional as F


HIDDEN_SIZE = 2560
HC_COUNT = 4
HC_HIDDEN_SIZE = HIDDEN_SIZE * HC_COUNT


def _rms(x: torch.Tensor) -> torch.Tensor:
    return x.float().square().mean(dim=-1, keepdim=True).sqrt()


def _rms_norm(x: torch.Tensor, weight: torch.Tensor, eps: float = 1e-6) -> torch.Tensor:
    inv_rms = torch.rsqrt(x.float().square().mean(dim=-1, keepdim=True) + eps)
    return (x.float() * inv_rms) * weight.float()


class FrozenQwen38Ple(nn.Module):
    """Non-persistent frozen source PLE tensors extracted from Flash-Next."""

    REQUIRED = {
        "key_weight", "value_weight", "key_norm_weight", "query_norm_weight",
        "conv_norm_weight", "conv_weight",
    }

    def __init__(self, tensors: dict[str, torch.Tensor]) -> None:
        super().__init__()
        missing = self.REQUIRED - set(tensors)
        unexpected = set(tensors) - self.REQUIRED
        if missing or unexpected:
            raise ValueError(f"native PLE tensors mismatch; missing={sorted(missing)}, unexpected={sorted(unexpected)}")
        expected = {
            "key_weight": (HC_HIDDEN_SIZE, HIDDEN_SIZE),
            "value_weight": (HIDDEN_SIZE, HIDDEN_SIZE),
            "key_norm_weight": (HC_HIDDEN_SIZE,),
            "query_norm_weight": (HC_HIDDEN_SIZE,),
            "conv_norm_weight": (HC_HIDDEN_SIZE,),
            "conv_weight": (HC_HIDDEN_SIZE, 1, 4),
        }
        for name, shape in expected.items():
            value = tensors[name]
            if tuple(value.shape) != shape:
                raise ValueError(f"{name} shape {tuple(value.shape)} != {shape}")
            if not value.is_floating_point() or not torch.isfinite(value).all():
                raise ValueError(f"{name} must be finite floating point")
            # These buffers move with .to(), but are deliberately excluded
            # from checkpoints: each run names its immutable source file.
            self.register_buffer(name, value.contiguous(), persistent=False)

    @classmethod
    def from_safetensors(cls, path: Path) -> "FrozenQwen38Ple":
        if not path.is_file():
            raise FileNotFoundError(f"native Qwen 3.8 PLE source does not exist: {path}")
        return cls(load_file(path, device="cpu"))

    def key(self, ple: torch.Tensor) -> torch.Tensor:
        return _rms_norm(F.linear(ple.float(), self.key_weight.float()), self.key_norm_weight)

    def query_norm(self, query: torch.Tensor) -> torch.Tensor:
        return _rms_norm(query, self.query_norm_weight)

    def value(self, ple: torch.Tensor) -> torch.Tensor:
        return F.linear(ple.float(), self.value_weight.float())

    def conv_norm(self, value_hc: torch.Tensor) -> torch.Tensor:
        return _rms_norm(value_hc, self.conv_norm_weight)

    def causal_conv(self, x: torch.Tensor) -> torch.Tensor:
        """Apply the source depthwise kernel with explicit left-only padding."""
        if x.ndim != 3 or x.shape[-1] != HC_HIDDEN_SIZE:
            raise ValueError(f"native conv expects [batch, tokens, {HC_HIDDEN_SIZE}]")
        channels = x.transpose(1, 2)
        padded = F.pad(channels, (9, 0))  # dilation 3 * (kernel 4 - 1)
        return F.conv1d(padded, self.conv_weight.float(), dilation=3,
                        groups=HC_HIDDEN_SIZE).transpose(1, 2)


class Qwen35NativePleBridge(nn.Module):
    """Trainable Qwen 3.5 <-> frozen Qwen 3.8 four-lane PLE bridge (v3).

    ``gate_bias`` is deliberately a single trainable scalar, rather than a
    modification of a Qwen 3.8 tensor.  It calibrates how often the frozen
    source PLE is used for this new Qwen 3.5 residual stream.  Temperature is
    fixed per run so that the bridge cannot evade a gate-usage objective merely
    by making its logits arbitrarily sharp.
    """

    def __init__(self, native_ple: FrozenQwen38Ple, *, hidden_size: int = HIDDEN_SIZE,
                 ple_embed_dim: int = HIDDEN_SIZE, residual_budget: float = 0.05,
                 gate_bias_init: float = 0.0, gate_temperature: float = 1.0) -> None:
        super().__init__()
        if hidden_size != HIDDEN_SIZE or ple_embed_dim != HIDDEN_SIZE:
            raise ValueError("native bridge requires 2560-wide Qwen 3.5 residuals and PLE embeddings")
        if not math.isfinite(residual_budget) or residual_budget <= 0:
            raise ValueError("residual_budget must be finite and positive")
        if not math.isfinite(gate_bias_init):
            raise ValueError("gate_bias_init must be finite")
        if not math.isfinite(gate_temperature) or gate_temperature <= 0:
            raise ValueError("gate_temperature must be finite and positive")
        self.hidden_size = hidden_size
        self.ple_embed_dim = ple_embed_dim
        self.hc_count = HC_COUNT
        # Unlike a Python float, the budget is checkpointed.  That is required
        # for a continuation phase that ramps it: evaluation must reconstruct
        # the exact budget of the selected checkpoint, not just the run's
        # starting value.
        self.register_buffer("residual_budget", torch.tensor(residual_budget, dtype=torch.float32), persistent=True)
        self.native_ple = native_ple
        self.query_proj = nn.Linear(hidden_size, hidden_size, bias=False)
        self.query_lane_scale = nn.Parameter(torch.ones(HC_COUNT, hidden_size, dtype=torch.float32))
        self.query_lane_bias = nn.Parameter(torch.zeros(HC_COUNT, hidden_size, dtype=torch.float32))
        self.output_lane_logits = nn.Parameter(torch.zeros(HC_COUNT, hidden_size, dtype=torch.float32))
        self.output_proj = nn.Linear(hidden_size, hidden_size, bias=False)
        self.gate_bias = nn.Parameter(torch.tensor(gate_bias_init, dtype=torch.float32))
        # This is persistent so a checkpoint carries its exact calibration,
        # but is intentionally not trainable.
        self.register_buffer("gate_temperature", torch.tensor(gate_temperature, dtype=torch.float32), persistent=True)
        self.alpha = nn.Parameter(torch.zeros((), dtype=torch.float32))
        self.register_buffer("architecture_version", torch.tensor(3, dtype=torch.int32), persistent=True)
        self.last_activation_metrics: dict[str, torch.Tensor] = {}
        self.gate_mean_for_regularization: torch.Tensor | None = None

    def load_state_dict(self, state_dict: dict[str, torch.Tensor], strict: bool = True):
        """Load pre-calibration v3 checkpoints as an explicitly neutral gate.

        Early v3 runs did not contain calibration tensors.  Their behaviour is
        exactly represented by bias=0 and temperature=1, which also lets them
        remain valid alpha-ablation baselines after this extension.
        """
        compatible_state = dict(state_dict)
        compatible_state.setdefault("gate_bias", torch.zeros_like(self.gate_bias))
        compatible_state.setdefault("gate_temperature", torch.ones_like(self.gate_temperature))
        compatible_state.setdefault("residual_budget", self.residual_budget.detach().clone())
        return super().load_state_dict(compatible_state, strict=strict)

    def set_residual_budget(self, value: float) -> None:
        """Set the frozen global residual multiplier for the current step."""
        if not math.isfinite(value) or value <= 0:
            raise ValueError("residual budget must be finite and positive")
        with torch.no_grad():
            self.residual_budget.fill_(value)

    def forward(self, hidden_states: torch.Tensor, ple_embeddings: torch.Tensor) -> torch.Tensor:
        if hidden_states.ndim != 3 or ple_embeddings.ndim != 3:
            raise ValueError("hidden_states and ple_embeddings must have shape [batch, tokens, width]")
        if hidden_states.shape[:2] != ple_embeddings.shape[:2]:
            raise ValueError("hidden_states and ple_embeddings must have matching batch and token dimensions")
        if hidden_states.shape[-1] != self.hidden_size or ple_embeddings.shape[-1] != self.ple_embed_dim:
            raise ValueError("native bridge width mismatch")
        batch, tokens, _ = hidden_states.shape

        key = self.native_ple.key(ple_embeddings).reshape(batch, tokens, HC_COUNT, HIDDEN_SIZE)
        query_shared = self.query_proj(hidden_states.float())
        query_hc = query_shared.unsqueeze(-2) * self.query_lane_scale + self.query_lane_bias
        query = self.native_ple.query_norm(query_hc.reshape(batch, tokens, HC_HIDDEN_SIZE))
        query = query.reshape(batch, tokens, HC_COUNT, HIDDEN_SIZE)
        score = (key.float() * query.float()).sum(dim=-1, keepdim=True) / math.sqrt(HIDDEN_SIZE)
        signed_sqrt = score.sign() * score.abs().clamp_min(torch.finfo(torch.float32).tiny).sqrt()
        raw_gate = torch.sigmoid(signed_sqrt)
        gate_logits = (signed_sqrt + self.gate_bias) / self.gate_temperature
        gate = torch.sigmoid(gate_logits)
        # Retain the differentiable batch mean only until the trainer has
        # formed its optional usage regularizer for this forward pass.
        self.gate_mean_for_regularization = gate.mean()

        value = self.native_ple.value(ple_embeddings)
        value_hc = value.unsqueeze(-2).expand(-1, -1, HC_COUNT, -1)
        value_norm = self.native_ple.conv_norm(value_hc.reshape(batch, tokens, HC_HIDDEN_SIZE))
        z = gate.to(dtype=value_norm.dtype) * value_norm.reshape(batch, tokens, HC_COUNT, HIDDEN_SIZE)
        conv = F.silu(self.native_ple.causal_conv(z.reshape(batch, tokens, HC_HIDDEN_SIZE)))
        conv_hc = conv.reshape(batch, tokens, HC_COUNT, HIDDEN_SIZE)
        feature_hc = z + conv_hc
        lane_weights = torch.softmax(self.output_lane_logits, dim=0)
        mixed = (feature_hc.float() * lane_weights.unsqueeze(0).unsqueeze(0)).sum(dim=-2)
        feature35 = self.output_proj(mixed)
        # Gate is already inside feature35.  Do not divide by its RMS here.
        hidden_scale = _rms(hidden_states).detach()
        injected = (self.alpha.to(dtype=feature35.dtype) * self.residual_budget.to(dtype=feature35.dtype)
                    * hidden_scale.to(dtype=feature35.dtype) * feature35)

        with torch.no_grad():
            hidden_global_rms = hidden_states.detach().float().square().mean().sqrt()
            eps = torch.finfo(torch.float32).tiny
            def relative_rms(x: torch.Tensor) -> torch.Tensor:
                return x.detach().float().square().mean().sqrt() / hidden_global_rms.clamp_min(eps)
            gate_float = gate.detach().float()
            raw_gate_float = raw_gate.detach().float()
            injected_float = injected.detach().float()
            injected_rms = injected_float.square().mean().sqrt()
            self.last_activation_metrics = {
                "gate_mean": gate_float.mean(),
                "gate_std": gate_float.std(unbiased=False),
                "raw_gate_mean": raw_gate_float.mean(),
                "gate_bias": self.gate_bias.detach().float(),
                "gate_temperature": self.gate_temperature.detach().float(),
                "residual_budget": self.residual_budget.detach().float(),
                "gate_fraction_le_0_1": (gate_float <= 0.1).float().mean(),
                "gate_fraction_ge_0_5": (gate_float >= 0.5).float().mean(),
                "gate_fraction_ge_0_9": (gate_float >= 0.9).float().mean(),
                "value_to_hidden_rms": relative_rms(value),
                "z_to_hidden_rms": relative_rms(z),
                "conv_to_hidden_rms": relative_rms(conv_hc),
                "feature_hc_to_hidden_rms": relative_rms(feature_hc),
                "bridge_feature_rms": feature35.detach().float().square().mean().sqrt(),
                "injected_rms": injected_rms,
                "injected_to_hidden_rms": injected_rms / hidden_global_rms.clamp_min(eps),
            }
        return injected.to(dtype=hidden_states.dtype)

License and attribution

This artifact incorporates Qwen 3.5 model material and Qwen 3.8 Flash-Next PLE material. Redistribution requires compliance with all applicable Qwen licenses, acceptable-use terms, and the terms of the training datasets. This repository does not grant additional rights to those materials.

Describe this release as a Qwen 3.5 NativeBridge V3 experimental n-gram PLE graft, not as an official Qwen release or a converted Qwen 3.8 model.