# Adaptive Supply Chain: Demand-Supply Synchronization Using Deep Reinforcement Learning
## 3 Methodology
### Supply Chain Environment
\(\forall\ m\in M\) and \(\forall\ t\in T\), the SCE dynamics is governed by the following set of equations:  
\[l_{t+1}^{m}=\,l_{t}^{m}+Q_{t-Lm}^{m}-\zeta_{t}^{m} \tag{2}\]  
\[V_{t+1}^{m}=V_{t}^{m}-Q_{t-Lm}^{m}+Q_{t}^{m} \tag{3}\]  
\[Q_{t}^{m}=\min\left(c^{m+1},\ l_{t}^{m+1},\ Q_{t}^{m}\right) \tag{4}\]  
\[\zeta_{t}^{m}=\left\{ \begin{array}{c}Q_{t}^{m-1},\ if\ m>0\\ \min\left(l_{t}^{0}+Q_{t-Lm}^{0},\ D_{t}\right),\ if\ m=0\end{array} \right. \tag{5}\]  
\[U_{t}^{m}=\,\zeta_{t}^{m-1}-\zeta_{t}^{m} \tag{6}\]  
Figure 2: Supply chain with \(m\) stages. \[NP_{t}^{m}=\rho^{m}\xi_{t}^{m}-r^{m}Q_{t}^{m}-k^{m}U_{t}^{m}-h^{m}l_{t+1}^{m} \tag{7}\]  
where at the beginning of each period \(t\in T\) at each stage \(m\in M\), \(I\) denotes inventory on hand. \(V\) stands for the commodities that are ordered and on the way (pipeline inventory). \(Q\) corresponds to the accepted reorder quantity, and \(\hat{Q}\) is the requested reorder quantity. L stands for replenishment lead time between stages. Demand \(D\) is a discrete random variable under Poisson distribution. The sales \(\zeta\) at each period equal to the customer demand satisfied by the retailer at stage 0 and the accepted reorder quantities from the succeeding stages from 1 to mend. \(U\) denotes unfulfilled demand and unfulfilled reorder requests. The net profit \(NP\) equals sales revenue minus procurement costs, the penalty for unfulfilled demand, and inventory holding costs. \(\rho\), \(r\), \(k\), and \(h\) stand for the unit sales price, unit procurement cost, unit penalty for unfulfilled demand, and unit inventory holding costs. If production capacity and inventory level are sufficient (do not exceed capacity constraints \(c\)), \(Q\) would be equal to \(\hat{Q}\). However, if capacities are insufficient, it imposes an upper bound on the reorder quantity that can be accepted. It is assumed that the inventory of raw materials at stage mend is infinite. It is important to emphasize that besides the non-perishability of goods and integer replenishment sizes, SCE makes a strong assumption regarding end-to-end visibility. Namely, information is assumed to be complete, accurate, and available along the supply chain. This assumption appears to be unrealistic in many real-world applications, given the current digitalization level. This assumption can hold if a single company controls the vast majority of a supply chain, let us say, from farm to fork or from quartz sand to microchip. However, the assumption is false for a larger swath of supply chains. We primarily see an outsourcing trend, according to which companies mainly focus on those activities in the supply chain where they have a distinct advantage and outsource everything else. Therefore, the pervasive adoption of the fully fledged supply chain digital twins or other technologies capable of providing information transparency and end-to-end visibility is a necessary infrastructural condition for real-world implementation. Furthermore, assuming that information transparency is partially provided, such SCE can serve as a proxy for real-world applications in supply chain management to gauge generalized algorithm performance.