# Aortic Pressure Control Based on Deep Reinforcement Learning for _Ex Vivo_ Heart Perfusion
## 1 Introduction  
Heart transplantation remains the gold-standard treatment for end-stage heart failure. However, the issue of donor heart preservation has seriously affected the development of heart transplantation [1; 2]. _Ex vivo_ heart perfusion (EVHP) has received considerable attention in the field of heart transplantation due to its potential for minimizing ischemic injury [3]. Furthermore, it is difficult to replicate real hemodynamic conditions in vitro with conventional mock circulatory systems [4]. As a standard technique for EVHP, Langendorff perfusion mimics the physiology of the heart by connecting a heart to the EVHP system and perfusing it retrogradely along the aorta to the coronary arteries, thereby controlling the physiological activity of the heart [5]. In EVHP, maintaining optimal aortic perfusion pressure is crucial for sustaining the physiological aerobic metabolism of the heart [6]. However, the cardiac parameters can vary significantly due to changes in temperature and perfusion solution. The complexity of and variability in cardiac parameters make it challenging to achieve rapid and accurate aortic pressure control. [PERSON] et al. [7] employed feedback control to regulate flow changes during the perfusion of isolated livers, maintaining the perfusion pressure at a constant level to avoid the side effects of liver autoregulation. In the study of Langendorff perfusion of porcine hearts, [PERSON] et al. [8] performed servo regulation, setting a target mean aortic pressure (MAP) to ensure that the range of pressure variations remains within reasonable limits. These studies optimize the physiological environment of the organs and reduce organ damage by controlling constant pressure. [PERSON] et al. [9] designed a closed-loop servo control system using a PD controller to regulate renal perfusion pressure. However,these PID controllers require the fine-tuning of multiple parameters to adapt to changes in perfusion conditions. [PERSON] et al. [10] achieved physiological perfusion by maintaining mean aortic pressure values using a fuzzy logic control method. However, the fuzzy control is highly dependent on expert experience and knowledge. As a living organ, the parameters of the organ are real time, variable, and complex. It is difficult for these control methods to adaptively control the pressure according to different perfusion conditions. Recently, [PERSON] et al. [11] proposed the model reference adaptive control method for the regulation of aortic pressure in Langendorff perfusion, while [PERSON] et al. [12] further proposed the semi-parametric adaptive control method for the regulation of aortic pressure, with the aim of maintaining the aerobic metabolism of the heart. This method reduces the time required for model updates while ensuring real-time control. It should be noted that centrifugal pumps are used in simulations and experiments. However, pulsatile pumps are used in the EVHP field, such as the TransMedics Organ Care System (TransMedics, Andover, MA, USA), which delivers blood by pulsatile pressure [13; 14]. Pulsatile blood flow mimics the physiological cardiac pattern of coronary perfusion, which is thought to reduce vascular resistance and thus be more conducive to microcirculation and organ perfusion [15]. However, the results of research on intelligent aortic pressure control strategies for pulsatile pumps with variable cardiac parameters in EVHP have yet to be published. Intelligent strategies are capable of discerning the dynamic relationships and interactions within a specific organism, thereby regulating the level of the corresponding variables in a biological system [16]. With the rapid development of artificial intelligence, deep reinforcement learning has emerged as a powerful tool for dynamic control systems, such as blood glucose control, due to its strong learning capabilities [17].