Title: Forecasting-Conditioned Reinforcement Learning: Embedding Forecastability as an Inductive Bias
Primary area: reinforcement learning
Keywords: Reinforcement Learning, Forecastability, Traffic Signal Control

Abstract: We introduce Forecasting-Conditioned Reinforcement Learning (FoRL), an extension to model-free Reinforcement Learning (RL) agents that augments the policy with multi-step self-forecasts. FoRL is trained either via Reward Conditioning (RC), which rewards forecast--action consistency, or Loss Conditioning (LC), which adds an auxiliary forecasting loss. Across three discrete-action benchmarks and forecasting horizons $\(L \in \{2,5,10\}\)$, FoRL consistently improves forecastability---measured by Supervised Action Prediction (SAP) and World-Model Unrolling (WMU)---with minimal sacrifice in environment return. Prior approaches toward predictable RL have typically relied on simplicity-inducing regularizers or trajectory-entropy penalties, shaping policies only indirectly toward more forecastable behaviors. In contrast, FoRL makes predictability an explicit training signal by embedding forecasting directly into the learning problem. Compared to such entropy-based methods, FoRL achieves a superior accuracy--return trade-off and provides direct internal forecasts for potential downstream applications. A case study on Traffic Signal Control (TSC) illustrates how FoRL-generated Internal Forecasts (IF) can support downstream application tasks such as vehicle-side Green Light Optimized Speed Advisory (GLOSA). Moreover, the integrated forecastability design enables effective fine-tuning when forecasts themselves alter the environment dynamics. Overall, FoRL elevates predictability from a post-hoc diagnostic to a first-class inductive bias for RL.

TL;DR: FoRL trains RL agents to forecast their own actions, improving predictability and enabling applications like GLOSA