Title: Crossing the Separation Point: Stabilizing Decision-Focused Learning with Variational Free-Energy
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Learning Dynamics, Decision-focused Learning, Variational Free-Energy

Abstract: Decision-focused learning (DFL) integrates predictive modeling with downstream optimization by training the prediction model to end-to-end minimize decision regret. However, the mapping from predicted parameters to decisions can be discontinuous, making gradients unstable and optimization unstable, particularly in the early stages of training. Existing approaches typically rely on heuristic warm-up phases through pretraining for stabilization, but the lack of theoretical understanding limits their effectiveness and generalizability. In this work, we propose a principled framework based on variational free energy (VFE) to analyze and address these challenges. We characterize the existence of critical separation points in the training dynamics, which mark a sharp transition in the alignment between prediction accuracy and decision quality. Building on this analysis, we develop Adaptive Recursive Annealing (ARA), a training strategy that adaptively regulates the learning dynamics without requiring warm-up. ARA improves training stability and decision performance by aligning upstream and downstream objectives throughout the learning process. Experiments on multiple benchmark problems demonstrate that our method consistently improves convergence behavior and downstream decision quality, offering a robust and theoretically grounded alternative to existing heuristics.

TL;DR: Learning dynamic matters the stability in decision-focused learning.