## 3 Methodology  
### Application of Approximate Dynamic Programming (ADP)  
This study utilizes ADP to manage and optimize the decision-making process under uncertainty to address the complexities of internal replenishment in a spare parts warehouse. ADP is particularly suitable for environments where the state and decision spaces are too large for traditional dynamic programming (DP) techniques due to the curse of dimensionality. It simplifies the problem by approximating the value function, which assesses future costs associated with different states and decisions. While a formal mathematical proof of convergence for this type of applied framework is beyond the scope of this paper, we demonstrate its empirical stability in Section 4.3, where results are shown to be consistent across varying planning horizons, suggesting the policy does not diverge. The DP formulation for this problem is presented in Appendix A. 1 Demand Forecasting  
The efficacy of the ADP framework is fundamentally dependent on the quality of its demand forecasts, which serve as the primary input for decision-making. To ensure predictive reliability, we conducted a comparative analysis of state-of-the-art machine learning models, including Extreme Gradient Boosting (XGBoost) and LightGBM, using time-series cross-validation. Evaluation metrics included Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). LightGBM was ultimately selected due to its superior performance across a holdout test set. To maintain robustness in a dynamic warehouse environment, the model operates within a dual-loop learning framework:  
* Daily Operational Loop: At the end of each day, actual demand is captured and used to update the inventory state for the next day's MILP optimization run. * Periodic Retraining Loop: The same demand data is added to the historical training set, and the LightGBM model is retrained periodically to reflect evolving demand patterns. This adaptive retraining strategy ensures that the forecasts remain aligned with changes in seasonality, customer behavior, or product life cycles. Inaccurate forecasts can lead to over-replenishment or stockouts, especially in a cost-sensitive MILP environ ment; hence, this mechanism is vital for sustaining high-quality replenishment decisions over time. 2. ADP Model with Certainty Equivalent Approach  
The ADP model incorporates state variables for inventory levels, decision variables for replenishment and picking, and an objective to minimize total expected costs. Value function approximation is achieved by implementing a certainty-equivalent control policy, a practical approach for complex problems chosen here for its interpretability, data efficiency, and stability. In this approach, the stochastic future demand is replaced by deterministic point forecasts from the machine learning model. This simplification allows the problem to be resolved at each step by the finite-horizon MILP. The solution to this MILP provides the optimal actions, and its objective value serves as the practical approximation of the true value function for the current state. This method of using a deterministic optimization model within a larger stochastic framework is a practical approach for complex problems, and a similar strategy was successfully used by [PERSON] et al. (2016). While this certainty-equivalent approach simplifies the problem at each step, uncertainty is managed over the long term through a rolling-horizon implementation. This mechanism is clarified by the daily operational cycle:  
1. Forecast and Optimize: At the start of the day, we generate a forecast for the planning horizon. The MILP is solved using the current inventory state, and this forecast produces an optimal plan. 2. Implement: Only the decisions for the immediate first day are implemented. 3. Observe and Update: At the end of the day, we observe the actual demand and update the inventory levels to reflect the true state of the system. 4. Repeat: On the next day, the entire process repeats from the new, true state.