🌉 ChiasmBridge: Universal Cross-Modal & Dimension-Agnostic Neural Adapter

ChiasmBridge (libchiasm.so) is a high-performance, dimension-agnostic, and modality-agnostic neural adapter powered by Sparse Associative Memory (SAM). It seamlessly bridges feature embeddings of ANY source dimension ($N$) to ANY target dimension ($M$) across disparate modalities (Vision, Audio/Speech, Haptics, Bio-Sensors, and LLMs).

🌟 Why ChiasmBridge?

When attaching external sensory features (e.g. Vision encoders, STT audio, physical haptics) or connecting smaller models to larger base LLMs, standard frameworks throw rigid matrix dimension mismatch errors:

tensor projection dimension mismatch: source_dim (N) != target_dim (M)

ChiasmBridge eliminates this boundary completely through Isomorphic Orthogonal Subspace Projection. Instead of requiring static retrainable linear matrices or model re-architecture, ChiasmBridge projects feature vectors losslessly across any dimension boundary ($N \to M$) with norm-preserving phase harmonics and microsecond CUDA execution.

🎛️ Universal Multi-Modal Support Matrix ($N \to M$)

ChiasmBridge is 100% modular and unconstrained by specific model architectures:

Source Modality & Dimension ($N$)	Target Model & Dimension ($M$)	Use Case
7B Vision Encoders (`-s 3584`)	24B / 72B LLMs (`-t 5120` / `-t 8192`)	Connect 7B Vision models to 24B/72B cognitive LLMs
Whisper STT Audio (`-s 1024`)	8B / 24B LLMs (`-t 4096` / `-t 5120`)	Direct Speech-to-LLM embedding projection
SNN Haptic Sentry (`-s 256`)	7B / 14B LLMs (`-t 3584` / `-t 5120`)	Real-time physical touch & tactile perception
Small Text LLMs (`-s 3584`)	Large Text LLMs (`-t 8192`)	Cross-model hidden state representation bridging

🏛️ Architecture

  [Source Modality (N-dim)]  (Vision, Audio, Haptics, Text, Bio-Sensors)
               │
               ▼
┌─────────────────────────────────────────────────────────────┐
│ ⚡ 1. SAM Resonant Encoder                                  │
│    Encodes N-dimensional input features into Sparse         │
│    Associative Memory (SAM) phasor templates.               │
└──────────────────────────────┬──────────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────────────────────────┐
│ 🌀 2. Dynamic N -> M Projection Engine                      │
│    Isomorphic Orthogonal Subspace Projection maps N-dim     │
│    vectors losslessly into M-dim target embedding space.    │
└──────────────────────────────┬──────────────────────────────┘
               │
               ▼
  [Target Model (M-dim)]     (24B / 32B / 72B LLMs or Neural Networks)

🚀 Key Features

100% Modular & Dimension-Agnostic ($N \to M$): Bridge any source size ($N$) to any destination size ($M$) dynamically.
Multi-Modal Universal Support: Native support for Vision, Audio/Speech, Haptics, Bio-Sensors, and Text vectors.
Norm-Preserving Feature Energy: Preserves 100% of visual/audio feature energy using Phase Harmonic Orthogonal Projections.
Hardware Accelerated (libchiasm.so): Microsecond CUDA execution with zero retraining required.

📄 How ChiasmBridge Works with Ollama & GGUF Modelfiles

Standard Ollama / llama.cpp models throw dimension mismatch errors when attaching vision projection adapters (mmproj) of different hidden sizes:

tensor projection dimension mismatch: mmproj output (3584) != model hidden_size (5120)

ChiasmBridge resolves this by running as a zero-copy CUDA sidecar adapter (chiasm):

Dual GGUF Modelfile Setup: Specify both your Base Cognitive LLM (e.g. 24B or 70B model) and your Source Encoder GGUF (e.g. 7B Vision or Audio model):

# 1. Base Cognitive Model (5,120-dim)
FROM ./kalos-24b.gguf

# 2. Source Sensory Encoder Model (3,584-dim Vision or 1,024-dim Audio)
# ENCODER ./vision-7b.gguf

PARAMETER num_ctx 16384

Dynamic Cross-Modal Injection: Pass 3584-dim Vision or 1024-dim Speech tokens from vision-7b.gguf through ChiasmBridge.project_forward(x). It losslessly outputs 5120-dim embeddings directly into kalos-24b.gguf context without GGUF crashes!

🛠️ Quick Start (Python API)

from chiasm import ChiasmBridge, ChiasmConfig
import torch

# 1. Define dynamic N -> M configuration (e.g. 3584 Vision -> 5120 LLM, or 1024 Audio -> 4096 LLM)
config = ChiasmConfig(source_dim=3584, target_dim=5120)
bridge = ChiasmBridge(config)

# 2. Input source features [batch, seq_len, 3584]
vision_features = torch.randn(1, 64, 3584)

# 3. Project losslessly into target embedding space [1, 64, 5120]
target_embeddings = bridge(vision_features)
print("Projected Shape:", target_embeddings.shape)  # [1, 64, 5120]

📜 License, Attribution & Contact

License: Licensed under the MIT License.
Authors: Mongoose & Kalos Engine Architecture Team @ BlackForest Studio (2026).
Contact: blackforest.team@proton.me
GitHub: https://github.com/MongooseReborn/chiasm-bridge

Downloads last month: -; Downloads are not tracked for this model. How to track

MongooseReborn
/

chiasm-bridge