## 2 POMDP Modeling
### Demand Modeling  
Unlike most previous research on supply chain inventory management, the research background of this paper is the inventory management problem of large-scale manufacturing supply chains, where the material demand has interdependent relationships between previous and subsequent processes. Taking a COMAC large passenger aircraft as an example, the overall assembly process involves process dependencies, as shown in the Figure 1. The production of the COMAC civil aircraft is completed through a series of sequential and parallel processes, so a certain degree of abstraction of the actual problem is carried out in the environment. The process of the main manufacturer is a combination of serial and parallel structures, as shown in Figure 2.  
In actual production processes, each step involves many different types of materials. These materials are typically divided into three categories based on their purpose: structural components, system components, and standard parts. Structural components refer to parts that comprise the main load-bearing components of the aircraft, such as the nose, fuselage, wings, and tail. System components refer to the components of the onboard systems, such as power systems, control systems, fuel systems, and hydraulic systems. Standard parts refer to standardized general-purpose parts used in production and maintenance, such as bolts and nuts. There are significant differences in the required quantity of materials between structural components, system components, and standard parts in each process. Therefore, in our study, we selected a representative structural component, system component, and standard part for each process. In order to reflect the widespread use of standard parts in actual production processes, our model uses only two types of standard parts for all processes. For more detailed information on the environmental setting, please refer to the relevant content in the Experimental Results section.  
Considering factors such as the manufacturing process defects of suppliers and operator errors of the main manufacturer, materials may need to be reworked or scrapped. To prevent delays in production schedules caused by such shortages, we consider the demand  
Figure 1: The structure of the demand dependency model of the civil aircraft production in COMAC.  
Figure 2: Example of the structure of the demand dependency model of main manufacturer.  
for materials at each process as a value that fluctuates up and down with a certain level of randomness. Therefore, it is assumed that the demand for material \(v\) by process \(u\) follows a normal distribution \(N(\mu_{uv},\sigma_{uv}^{2})\) with mean \(\mu_{uv}\) and variance \(\sigma_{uv}^{2}\).  
For the main manufacturer, we follow the annual planning mechanism in actual production and set the number of production tasks in the environment. The start times of these production tasks are evenly distributed throughout the annual production cycle. After the start of the first process of each production task, the logistics demand of that process is directly sent to the main manufacturer. Each process has a maximum lead time for materials. The main manufacturer will be punished for stock-outs if it exceeds the maximum lead time to ensure the on-time progress of the production plan. After the main manufacturer has completed the preparation of the materials required for a certain process, all the following processes will be added.  
For the supplier, its material demand depends on the actions of the main manufacturer. The main manufacturer also has a maximum waiting period for the supplier's stock preparation, and the supplier will be punished for stock-outs if it exceeds the maximum waiting period.