# Optimization of Inventory Management to Prevent Drug Shortages in the Hospital Supply Chain  
[PERSON]  
1  
[PERSON] and [PERSON]  
1  
Footnote 1: email: {[PERSON], 1,  
of clear lines of communication, gaining transparency into the cause of the shortages, and continually accessing in-stock and low-cost alternatives are the best mitigation process for developing a balanced inventory [5]. The Food and Drug Administration (FDA) has developed the Innovation Act and Strategic Plan to help mitigate drug shortages. The World Health Organization (WHO) has recognized drug shortage as a global problem [6]. Integrating manufacturers, suppliers, wholesalers, and stores to produce and distribute the right quantities of drugs at the right time in the right place to satisfy service-level requirements and minimize system-wide cost is called supply chain management. Therefore, optimization is needed, and this goes hand-in-hand with the supply chain management of the inventory to address how medicines and drugs are supplied, managed, and used. Understanding the underlying causes of drug shortages is the most critical aspect that any hospital or health system needs to be acquainted with to help mitigate these outcomes [7]. By doing this, it is also easy to come up with proactive measures with the available resources. Determining the exact cause of drug shortages appears to be difficult; in general, it has been divided into three categories: manufacturing problems, supply and demand issues, and raw material problems [8]. Hence, in this study, we focus on the supply and demand issues, which means that given the information of the supply chain system, we aim to control the drug inventory based on the uncertainty of user demand and drug prices. Our model aims to make an automated decision in order to answer the following questions, which have seen little research: (i) _Should we refill drugs_? (ii) _How much drug should be refilled_? More precisely, this research aims to study an inventory drug optimization problem, which can capture the requirement constraints of drug demands, storage capacity, and refilling conditions while minimizing the refilling costs in the system. Since our designed problem is NP-hard, it is impossible to find a solution in polynomial time. To deal with this problem, we advocate a Markov Decision Process (MDP) to model the drug inventory system in order to address the shortage problem. Based on that model, we propose a Deep Reinforcement Learning (DRL) framework model that combines the Reinforcement Learning (RL) [9] method and the Deep Neural Network model(DNN) [10] to automatically make a decision in a finite horizon. Basically, RL is modeled as an MDP that is comprised of three concepts: a state, an action corresponding to a state, and a reward for that action. Following the loop of actions and observations, the agent in an MDP often refers to a long-term consequence. Thus, RL is particularly well suited to control the drug inventory in a finite horizon. Furthermore, the combination of RL and DNN can figure out the strict requirement of the MDP regarding the exact knowledge of the state space of the MDP, and DRL is able to find a near optimal solution for a large MDP model, as shown in Dynamic Refilling dRug Optimization (DR2O). With a large number of drugs in the system, it might be unsolvable for a classical dynamic programming method. All of the contributions of our work are summarized as follows:  
* First, we investigate the supply chain model of drugs in a hospital to formulate an optimization drug shortage problem, named Dynamic Refilling dRug Optimization (DR2O). We model an objective function that aims to minimize the refilling costs, comprised of the costs to buy, to store medicines, and the penalty cost due to shortage. Furthermore, we consider supply constraints to refill drugs, such as storage capacity, and budget constraints.