## 3 Methodology
### ADP Pseudo Code  
The core of our framework is the iterative solving of the MILP model within the ADP loop. The MILP serves as the policy function in our ADP framework. It is solved iteratively, once per day, in our rolling-horizon simulation. At the beginning of each day \(t\), the current state (inventory levels) provides the initial boundary for the MILP. The MILP is then solved to find an optimal set of actions over the planning horizon.  
In this approach, the MILP's finite-horizon objective function acts as the practical approximation for the theoretical ADP value function, \(V(S)\). Since computing \(V(S)\) for all possible states is interactable, we instead approximate the total cost-to-go by solving the detailed MILP over a finite lookahead period. This is a common and effective technique in ADP, allowing us to leverage the power of exact optimization for tactical planning within a larger, adaptive framework.  
**Step 0: Initialization**  
0.1 Load historical demand data from the database;  
0.2 Initialize state variables: forward inventory, buffer inventory and reserve inventory;  
0.3 Define the planning horizon T (e.g., the next seven days);  
0.4 Set MILP parameters.  
**Step 1: Main Loop**  
For each spare part (i = 1, 2, 3,..., _N_):  
For each planning day (t = 1, 2,..., _T_):  
Step 1.1: Generate forecasts (certainty equivalent);  
Step 1.2: Solve MILP model with recursive state transitions (1)-(17).  
**Step 2: Update Initial States (Inventory levels)**  
2.1 Record the actual demand at the end of the day;  
2.2 Update the initial states and historical data with the recorded actual demand (iteratively refining value approximation).  
The ADP process begins with Step 0: Initialization, where historical demand data is loaded, state variables are set, and MILP parameters are configured. This establishes the foundation for the iterative optimization. In Step 1: Main Loop, demand forecasts are treated as fixed values and serve as inputs to the MILP model, which determines optimized replenishment and buffer allocation decisions over the finite planning horizon. By solving the MILP iteratively each day, the approach captures the dynamic nature of inventory management while maintaining computational efficiency.  
In Step 2: Update Initial States, when the actual demand at the end of the day is observed, we update the initial states of the model for the next day's run. These iterative updates continually improve the accuracy of the model's future decisions. It is important to note that optimal decisions are made not with a complex, intractable DP model but with the simple MILP model that we run continuously. This demonstrates that ADP offers a much more practical solution than traditional DP. While this methodology does not solve the DP problem exactly, it uses ADP principles to approximate the value function and optimize decisions iteratively. The rolling-horizon framework recalibrates the optimization at each step, balancing immediate and future costs dynamically. Although this method does not provide a truly optimal solution, it provides a much more feasible and practical solution, avoiding the difficulties of DP. The DP formulation of this problem and the DP pseudo code are explained in detail in Appendix A.