# The Deep Deterministic Policy Gradient Algorithm Based on RIS Technology in a Coal Mine Tunnel
## 3 Deep Reinforcement Learning Based Optimization
### DDPG Algorithm Elements and Processes
At a transmit power of 38 dBm, the DDPG algorithm significantly optimizes the system performance, especially in the ideal reflection condition; the average link rate of 2 \(\mathrm{bps}/\mathrm{Hz}\) is improved to 6.6 \(\mathrm{bps}/\mathrm{Hz}\). In addition, the specific impact of the number of RIS units on the system performance is explored, and it is found that increasing the number of RIS units can further improve the system performance;
3. Future research will further explore the application and optimization of the DDPG algorithm and other algorithms in more complex coal mine tunnel systems, especially in coal mine tunnel environments with bends, different frequency bands, and multiple modulation technologies. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**State**} & **Number of RIS** & **Initial Reward** & **Final Reward** & **Difference in** \\  & **units N** & **(bps/Hz)** & **(bps/Hz)** & **Value (bps/Hz)** \\ \hline RIS Ideal & 8 & 2 & 5 & 3 \\   Reflection & 16 & 1.5 & 6.2 & 4.7 \\  & 32 & 1.9 & 6.8 & 4.9 \\ \hline   RIS Non-Ideal & 8 & 2 & 4.7 & 2.7 \\   Reflection & 16 & 1.7 & 5.5 & 3.8 \\  & 32 & 1.9 & 5.8 & 3.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Average reward changes under different numbers of RIS units. **Author Contributions:** Conceptualization, S.W. and F.W. ; methodology, S.W. and F.W. ; software, S.W. and F.W. ; validation, S.W., F.W. and W.Z. ; formal analysis, W.Z. and Y.W. ; investigation, S.W., F.W., W.Z. and Y.W. ; resources, S.W. and F.W. ; data curation, S.W., F.W. and W.Z. ; writing--original draft preparation, F.W. ; writing--review and editing, S.W. ; visualization, F.W., W.Z. and Y.W. ; supervision, S.W. ; project administration, S.W. ; funding acquisition, S.W. All authors have read and agreed to the published version of the manuscript. The research was funded by the National Natural Science Foundation of China (NSFC52174197) and the UWB Radar Life Information Feature Extraction and Quantitative Identification for the Mine Drill Hole Rescue project. Not applicable. The original contributions presented in the study are included in the article, and further inquiries can be directed to the corresponding author. I would like to express special thanks to [PERSON] for his teaching and guidance, which has benefited me greatly. The authors declare no conflicts of interest. ## References

* (1) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Towards 6G wireless communication networks: Vision, enabling technologies, and new paradigm shifts. _Sci. China Inf. Sci._**2021**, _64_, 1-74. [CrossRef]
* (2) [PERSON]; [PERSON]; [PERSON] Overview of intelligent surface for new-generation mobile communication. _Telecommun. Sci._**2022**, _38_, 20-35. * (3) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Survey of the non-line-of-sight wireless coverage technology by reconfigurable intelligent surfaces in underground coal mines. _J. China Univ. Min. Technol._**2024**, _53_, 613622. * (4) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Discussion on intelligent reflecting surface technology and its application in wireless blind spot coverage in coal mines. _J. Mine Autom._**2023**, _49_, 112-119. * (5) [PERSON]; [PERSON] RIS-Assisted Wireless Channel Characteristic in Coal Mine Tunnel Based on 6G Mobile Communication System. _Prog. Electromagn. Res. C_**2024**, _141_, 13-23. [CrossRef]
* (6) [PERSON] _Research on Deep Learning-Based Channel Estimation for Reconfigurable Intelligent Surface-Aided Communication Systems_; Hunan Institute of Science and Technology: Hunan, China, 2023. * (7) [PERSON]; [PERSON] Towards smart and reconfigurable environment: Intelligent reflecting surface aided wireless network. _IEEE Commun. Mag._**2019**, _58_, 106-112. [CrossRef]
* (8) [PERSON]; [PERSON]; [PERSON]; [PERSON] On the optimality of reconfigurable intelligent surfaces (RISs): Passive beamforming, modulation, and resource allocation. _IEEE Trans. Wirel. Commun._**2021**, _20_, 4347-4363.