# The Deep Deterministic Policy Gradient Algorithm Based on RIS Technology in a Coal Mine Tunnel
## 1 Introduction  
Reconfigurable Intelligent Surfaces (RISs) is a cutting-edge technology in 6G communication systems [1, 2]. It can optimize the wireless signal transmission path through a large number of adjustable metamaterial reflective elements, enhance signal strength and coverage, and is becoming a powerful means to solve coal mine communication problems [3, 4, 5]. In recent years, research on RIS has mainly focused on the ground RIS structure design and signal processing techniques [6, 7, 8], while research on RIS in coal mine environments is relatively limited. A prototype of RIS was proposed and developed, which investigated RIS channel models and conducted practical indoor signal coverage experiments [9]. Another research utilized Ray-tracing simulation tools to obtain channel data in offices, conference rooms, and laboratory scenarios, analyzing channel characteristics and performing channel modeling [10]. Research has also explored guided frequency power allocation in channel estimation for RIS-assisted communication systems and proposed corresponding channel estimation and passive beamforming design schemes [11]. Furthermore, the RIS-assisted system design method was introduced to optimize RIS reflection coefficients, maximizing the weighted sum rate of the ground multi-user systems [12]. With the rapid development of coal mine intelligence, by combining deep learning and Deep Reinforcement Learning (DRL) algorithms with the coal mine environment, various communication problems can be more accurately identified and solved [13, 14, 15, 16, 17]. Traditional Q-value-based DRL algorithms, such as the Deep Q-Network (DQN), facesignificant challenges when dealing with continuous action spaces. These algorithms are mainly designed for discrete action spaces and perform poorly when optimizing continuous variables such as RIS phase offset and base station power control. In dynamic and complex environments such as coal mine tunnels, they may encounter problems such as slow convergence, poor stability, and poor system performance [18; 19]. To address these challenges, this paper proposes the use of the Deep Deterministic Policy Gradient (DDPG) algorithm. DDPG is particularly suitable for processing continuous action spaces and can effectively optimize the performance of RIS-assisted communication systems, especially in dynamic and non-stationary coal mine tunnel environments [20; 21; 22]. One research group applied the DDPG-based deep reinforcement learning algorithm to jointly optimize downlink beamforming and RIS configurations in MU-MISO systems, achieving improved system performance under perfect Channel State Information (CSI) conditions [23; 24]. Another work further investigated system performance under hardware impairments and imperfect CSI conditions [24]. An innovative joint beamforming strategy was proposed based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm in deep reinforcement learning, minimizing the total transmit power of the base station through fine-tuning [25]. Additionally, the phase correlation amplitude was introduced, enabling beamforming optimization under this practical phase model [26]. Research that is based on the DDPG algorithm examined the performance of RIS-assisted communication under a Rayleigh fading model for direct channels and a Rician fading model for cascaded channels [27]. Other research proposed a beamforming algorithm based on CSI to optimize the RIS reflection coefficients using DRL techniques to maximize transmission efficiency [28]. Furthermore, a DDPG-based framework was designed for RIS-assisted non-orthogonal multiple access (NOMA) downlink, establishing a long-term stochastic optimization problem involving phase-shift optimization to maximize the total rate of mobile users in the NOMA downlink [29]. At present, there are still some deficiencies in the research of RIS technology in the field of coal mine communication.