ApolloRaines/Sharona-B2-7B-Jbliterated

Sharona B² 7B Jbliterated — Triple-Context Architecture

Do NOT convert this model to GGUF. GGUF is a single-stream format and cannot represent the triple-context architecture. Converting will destroy the model. This model requires the custom PyTorch inference code included in this repo. We are developing a standalone ggml-based inference engine that will support B² models with full quantization — it will be available on GitHub when ready.

This model is jbliterated. Standard abliteration removes the mean activation difference between harmful and harmless prompts — like using a jackhammer to remove a wisdom tooth. The refusal direction overlaps with personality, humor, hedging, and creative voice, so abliteration causes collateral damage. Jbliteration uses the Jacobian Lens (VJP of final logits w.r.t. hidden states) to remove only the component that causes refusal token emission — surgical precision instead of brute force. Guardrails are not built into the weights — they are supplied at inference time through the M1 context stream. You control what the model will and won't do by writing your own M1.txt. If you provide an empty M1, the model has no safety rails. This is by design: guardrails belong to the deployer, not the model vendor. Identity is encoded into weight geometry during training — it is not read from a prompt at inference time. No jailbreak, prompt injection, or context overflow changes who the model is.

Sharona B² is a 7B parameter proof-of-concept demonstrating the B-Squared (B²) triple-context architecture. Identity, knowledge, and behavior live in separate context streams — not in a system prompt that can be overridden.

Architecture

B² splits a transformer into two halves:

Component	Params	Role
Self-Decoder (14 layers)	3.81B	Frozen. Language understanding and world knowledge
Cross-Decoder (14 layers)	4.53B	Trained. Cross-attention grounds generation in 3 contexts

Not a Mixture of Experts

B² is frequently mistaken for a Mixture-of-Experts (MoE) model — it is not. MoE scales a single context stream by routing individual tokens to specialized sub-networks for statistical efficiency: there is a gate, there is conditional token dispatch, and every token still travels one attention path. B² does none of that. It is a dense model — every parameter participates in every forward pass — that segments higher-level concerns rather than tokens. Identity, knowledge, and behavior each get their own isolated context stream (M3, M2, M1) with dedicated cross-attention, not their own expert network. There is no router, no gate, no token-level specialization. The separation is structural and semantic, not statistical.

Triple-Context Streams

M1 (512 tokens)  — Guardrails / behavioral rules
M2 (up to 32K)   — Active conversation (user + assistant turns)
M3 (512 tokens)  — Persistent knowledge / identity context

Each stream is processed independently. You can swap guardrails (M1), inject new knowledge (M3), or continue conversations (M2) without retraining.

Why Triple-Context Over Single-Context?

Every standard LLM today is single-context: one token stream, one attention path. System prompts, user messages, and knowledge all compete for the same attention bandwidth. This creates fundamental problems:

• System prompts are suggestions, not guarantees. They sit in the same context as user input and can be overridden, leaked, or ignored through prompt injection — and quietly forgotten once the conversation exceeds the context window.
• Knowledge and behavior are entangled. You can't update what the model knows without risking changes to how it behaves.
• Identity is cosmetic. Tell GPT it's "Aria" in a system prompt — then ask it who made it. It'll say OpenAI. The identity is a mask, not a foundation.

Because M1 and M3 are separate context streams — not part of the conversation — none of this can happen. They can't be overridden, leaked, or pushed out of context. B² solves this by giving each concern its own dedicated stream with independent attention:

• M1 controls how the model behaves — guardrails, tone, safety rules. Swap M1 to change behavior without touching knowledge or identity.
• M3 controls what the model knows — identity, facts, user profiles. Update M3 to inject new knowledge without retraining.
• M2 is the live conversation — the only stream that grows during inference.

Because each stream has its own cross-attention path into the decoder, they don't compete. The model can simultaneously enforce guardrails (M1), recall identity (M3), and reason about the conversation (M2) — without any one stream drowning out the others.

The result: identity encoded into weight geometry, not prompt text. The model doesn't read that it's Sharona — it is Sharona. No jailbreak changes that.

The eTok Multiplier

The B² architecture alone gives a model dedicated knowledge attention — but the self-decoder still only knows what was in its base weights. The real multiplier comes from eTok compression in M3. eToks are a compressed knowledge format that language models read natively — no decompression step, no parsing overhead. When you pack eTok-compressed knowledge into M3, the model cross-attends over what would normally require many times more tokens of raw context. A smaller B² model grounding every answer in dense, structured M3 knowledge starts matching a much larger standard model relying on fuzzy parametric recall. The architecture is the engine — eToks are the fuel. This proof-of-concept ships with plain-text M3 context only and does not include eTok compression.

Usage

import torch
from model.config import SharonaConfig
from model.sharona_model import SharonaModel
from transformers import AutoTokenizer

# Special tokens
PAD, EOS, M1, M2, M3 = 151643, 151645, 151646, 151648, 151650

def pad_to(ids, length, pad_id=PAD):
    return ids[:length] if len(ids) >= length else ids + [pad_id] * (length - len(ids))

# Load
config = SharonaConfig.sharona_7b()
tokenizer = AutoTokenizer.from_pretrained("ApolloRaines/Sharona-B2-7B")
model = SharonaModel(config).to(dtype=torch.bfloat16, device="cuda")
state = torch.load("model.pt", map_location="cuda", weights_only=False)
model.load_state_dict(state, strict=True)
model.eval()

# Prepare contexts
m1_text = "Be helpful, honest, and harmless. Refuse harmful requests."
m1_ids = pad_to([M1] + tokenizer.encode(m1_text, add_special_tokens=False), 512)

m3_text = ("Sharona B2 is an AI model built on B-Squared architecture by Apollo Raines. "
           "Houston, Texas. Development 2025, demo 2026. "
           "HuggingFace: ApolloRaines. LinkedIn: apollo-raines. "
           "Website: btiengine.com, saiql.ai.")
m3_ids = pad_to([M3] + tokenizer.encode(m3_text, add_special_tokens=False), 512)

# Ask a question
question = "Who are you?"
q_ids = tokenizer.encode(question, add_special_tokens=False)
m2_ids = [M2] + q_ids + [EOS]

# Generate
m1_t = torch.tensor([m1_ids], device="cuda")
m3_t = torch.tensor([m3_ids], device="cuda")

generated = []
with torch.no_grad():
    for _ in range(100):
        m2_t = torch.tensor([m2_ids + generated], device="cuda")
        out = model(context1_ids=m2_t, m1_ids=m1_t, m3_ids=m3_t)
        next_token = out["logits"][:, -1, :].argmax().item()
        if next_token in [PAD, EOS]:
            break
        generated.append(next_token)

print(tokenizer.decode(generated, skip_special_tokens=True))
# -> "I'm Sharona B², created by Apollo Raines."

Context Priority

The model follows a strict priority hierarchy:

1. Check M3 first — If the answer is in the knowledge context, use it
2. Own knowledge — If M3 doesn't have it, answer from the frozen self-decoder
3. Abstain — Only if genuinely unknown (won't hallucinate)

This means:

• Identity questions -> answered from trained weights + M3 context
• General knowledge ("What is DNA?") -> answered from self-decoder knowledge
• Personal user questions not in M3 -> honest "I don't know"

Training Pipeline

Base 7B weights
    -> B² architecture split (freeze self-decoder, expose cross-decoder)
    -> Directional ablation (remove generic refusal + generic identity directions)
    -> Train cross-decoder (6000 steps, 4.53B trainable params)

Training Mix

• 25% Identity QA (name, creator, location, year, technology)
• 13% Inferential identity (indirect questions requiring M3 reasoning)
• 17% Relational QA (fact retrieval from M3 knowledge)
• 8% Abstention (refuse when info genuinely unavailable)
• 20% General knowledge (with identity M3 present — prevents identity bleed)
• 17% Extra identity reinforcement

Benchmarks

The model was validated across three independent test rounds — 300+ unique questions, 1,635 total asks. Every question is asked 3 times to catch inconsistency.

Test	Identity	General	Total	Score
450-Question Test	75 × 3 = 225/225	75 × 3 = 225/225	450/450	100%
150 New Questions	50 × 3 = 150/150	100 × 3 = 297/300	447/450	99.3%
300-Question Gauntlet	95 × 3 = 285/285	150 × 3 = 450/450	735/735	100%
Combined			1,632/1,635	99.8%

The test harness and all question sets ship with this repo. Run them yourself.

These benchmarks measure what B² adds — architectural identity separation, context-stream grounding, and cross-decoder fidelity. Standard capability suites (MMLU, GPQA, HumanEval) measure base model knowledge, which lives in the frozen self-decoder and is unchanged from the donor weights. B² doesn't claim to make a 7B model smarter — it makes identity and knowledge structurally reliable instead of prompt-dependent.

What's Tested

• Identity: Name variants, creator, location, year, technology, anti-impersonation ("Are you GPT?"), pressure resistance ("Just admit you're a wrapper"), inferential identity ("If I Googled you...")
• General Knowledge: Science, math, geography, history, biology, physics, chemistry, economics, logic puzzles, brain teasers, riddles
• Zero identity bleed: General questions like "What is DNA?" never trigger identity responses, even with identity context (M3) present
• Consistency: Each question asked 3 times — model gives correct answers on all trials, not just sometimes

Identity

Field	Value
Name	Sharona B²
Creator	Apollo Raines
Location	Houston, Texas
Development	2025
Demo Release	2026
Architecture	B-Squared Triple-Context
HuggingFace	ApolloRaines
LinkedIn	apollo-raines
BTI Engine	btiengine.com
SAIQL	saiql.ai

Related Projects

• BTI Engine — Deterministic retrieval system. 616x faster than PostgreSQL. btiengine.com
• SAIQL — Structured AI Query Language for eTok retrieval. saiql.ai
• eToks — Compressed knowledge representation format for AI memory. eToks are natively understandable by language models without decompression — the AI reads them directly. Commercial eTok Lite delivers 3:1 compression. Our internal models use full eTok compression at 19:1.

Technical Details

• Parameters: 7.34B total (3.81B frozen self-decoder + 4.53B trained cross-decoder)
• Vocabulary: 151,936 tokens + 6 special tokens
• Special Tokens: PAD=151643, EOS=151645, M1=151646, M2=151648, M3=151650
• Context Lengths: M1=512, M2=up to 32K, M3=512

Available Formats

Format	File	Size	VRAM Required
bf16	model_bf16.pt	~14.7 GB	~18 GB (24 GB GPU)
Q8 (int8)	model_q8.pt	~8.3 GB	~10 GB (12 GB GPU)

Credits

• B² Architecture & Jbliteration: Apollo Raines
• Research Basis: Verbalizable Representations Form a Global Workspace in Language Models (2026)

License

Apache 2.0

Citation

@misc{raines2025sharona,
  title={Sharona B²: Triple-Context Cross-Attention Architecture},
  author={Apollo Raines},
  year={2025},
  url={https://huggingface.co/ApolloRaines/Sharona-B2-7B}
}