Specialgfhdhdh/continual-learning-routing

Hybrid Episodic Memory and Parametric Routing for High-Dimensional Continual Learning

This repository contains the official implementation of the hybrid routing continual learning framework applied to high-dimensional embedding spaces, including synthetic data and sequence representations extracted from real LLMs (TinyLlama-1.1B).

Abstract

This project implements a hybrid routing strategy that dynamically weights predictions from an offline-frozen parametric representation head and an episodic memory lookup: $${\mathcal{p}}_t(y)=(1-\alpha )p_{\text{head}}(y)+\alpha p_{\text{mem}}(y)\backslash mathcal\{p\}\_t(y)\; =\; (1-\backslash alpha)\; p\_\{\backslash text\{head\}\}(y)\; +\; \backslash alpha\; p\_\{\backslash text\{mem\}\}(y)$$

Core Results

• Empirical Accuracy Ceiling: Up to 97.5% streaming accuracy.
• Storage Bottleneck Sensitivity: Minimal memory capacity ($M \ge 10$) needed under structured feature spaces.