Title: Shared Dynamic Model-Aligned Hypernetworks for Zero-Shot Generalization in Contextual Reinforcement Learning
Primary area: reinforcement learning
Keywords: contextual reinforcement learning, zero-shot generalization, hypernetworks

Abstract: Zero-shot generalization in contextual reinforcement learning (RL) remains a core challenge, particularly when explicit context information is unavailable and must be inferred from data. We propose DMA*-SH, a framework based on dynamics model-aligned (DMA) context inference, where a shared hypernetwork jointly parameterizes the dynamics model, policy, and value function. This design enforces consistency between learned context representations and transition dynamics, while normalization and random masking in the context encoder improve stability and robustness. To evaluate our approach, we introduce environments that distinguish between overlapping and non-overlapping contexts, the latter being typically unsolvable with standard domain randomization. Across these benchmarks, DMA*-SH achieves strong zero-shot generalization and outperforms both context-aware and context-unaware baselines. Our results demonstrate that hypernetworks enable effective and scalable context inference in contextual RL.

TL;DR: We introduce DMA*-SH, a hypernetwork-based approach for contextual reinforcement learning that aligns context inference with dynamics models, enabling stable representations and achieving strong zero-shot generalization across diverse environments.