# Aortic Pressure Control Based on Deep Reinforcement Learning for _Ex Vivo_ Heart Perfusion
## 4 Discussion  
Some of the current papers on the perfusion of isolated hearts in Langendorff mode artificially regulate aortic pressure or servo regulate aortic pressure [8; 31]. However, it should be noted that individual heart conditions may vary in heart perfusion, which may affect the tuning parameters. Accordingly, we propose a reinforcement learning-based method to learn the law of the perfusion pressure under the influence of the pumping conditions and the heart's conditions and to optimize the perfusion conditions for the objective of prolonging heart preservation. In light of the limitations of manual and PID control in adapting to uncertain time-varying cardiac parameters, [PERSON] et al. [11] developed a model-adaptive reference method for AoP regulation, which proved to be an effective solution. The method is simulated for two different cardiac parameter conditions and heart rate irregularities. Furthermore, [PERSON] et al. [12] proposed a semi-parametric adaptive control method for regulating AoP in a set of cardiac parameters and vascular resistance increase conditions. The robustness of the DDPG method proposed in this paper is validated by simulating and testing single and mixed conditions of heart rate variability, target aortic pressure step size, and coronary flow resistance variability. In order to validate the effectiveness of the control method in this experiment, the DDPG algorithm is compared with conventional PID and fuzzy PID methods to test the algorithm's adaptability to different perfusion conditions. Moreover, the algorithm's performance is evaluated through the utilization of step change signals, which allow for the assessment of the algorithm's performance at different extremes due to the effect of EVHP. The simulation results demonstrate that the proposed method can effectively achieve rapid and stable aortic pressure control. In this paper, the adaptability of the algorithm under different perfusion conditions is improved by using the DDPG agent to learn the AoP control strategy under different initial conditions, and then realizing real-time control by changing its output gain. Reinforcement learning enables more flexible and precise control of aortic pressure in Langendorff perfusion, which plays a key role in maintaining cardiac function and optimizing cardiac metabolism. There is evidence that excessive pressure can damage endothelial cells, so smaller overshoots may help prolong cardiac preservation [30]. The method proposed in this paper produces a small pressure overshoot, within 4 mmHg, in both single and mixed conditions. And the simulation results are compared with the pressure and flow ranges of Transmedics, and it is found that the aortic flow values are a little lower at the corresponding pressures, but the main objective of this paper is to preliminarily validate the feasibility of the algorithm based on the mathematical model of heart perfusion. When there is an increasing abrupt change in the \(MAP_{ref}\) value, the mean AoF value also increases, which is consistent with the previous simulation results [11]. Changes in coronary resistance are influenced by temperature, pH, and partial pressure of the perfusate, which can lead to changes in aortic pressure. Aortic pressure control is essential to avoid overperfusion or myocardial ischemia, especially in cardiac perfusion studies. During the control process, only one pressure transducer is required to measure the aortic pressure, thus reducing the damage to the donor heart. Moreover, the method can be extended to different types of blood pumps, such as the Impella (Abiomed, Danvers, MA, USA), which incorporates an optical pressure sensor to measure the proxy aortic pressure [32]. The network of the method in this paper converges in less than 400 timesteps and is suitable for deployment in a simulation environment. The samples obtained from the EVHP experiments are very valuable and can therefore be used in the evaluation phase of the method.