# The Deep Deterministic Policy Gradient Algorithm Based on RIS Technology in a Coal Mine Tunnel
## 3 Deep Reinforcement Learning Based Optimization
### DDPG Algorithm Elements and Processes
#### 4.3.2 Comparative Analysis of Instant and Average Rewards

The effect of reward changes with the number of steps under different transmit powers is shown in Figure 8. In terms of instant reward, when the power is 36 dBm, \(R_{\text{avg}}\) is 8.2 bps/Hz, while when the power is 38 dBm, \(R_{\text{avg}}\) reaches 10.4 bps/Hz and its fluctuation is also larger, indicating that the system's reward varies more drastically at higher power. The average reward curve is smoother. At a transmit power of 36 dBm, \(R_{\text{avg}}\) reaches 5 bps/Hz, and at 38 dBm, \(R_{\text{avg}}\) increases to 6.5 bps/Hz, indicating that increasing the transmission power helps to improve the system performance. ### Effect of the Number of RIS Units on Average Rewards

The effect of different numbers of RIS units on the average reward as the number of step changes is shown in Figure 5.0 The simulation conditions are set to \(p_{t}=38\)dBm, \(\mu=\pi,\alpha=1\) with a non-ideal reflection of \(\beta_{\min}=0.5\). \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**State**} & **Power** & **Initial Reward** & **Final Reward** & **Difference in** \\  & **(dBm)** & **(bps/Hz)** & **(bps/Hz)** & **Value (bps/Hz)** \\ \hline RIS Ideal & 36 & 2 & 5 & 3 \\ Reflection & 37 & 2 & 5.8 & 3.8 \\
38 & 2 & 6.7 & 4.7 \\ \hline RIS Non-Ideal & 36 & 2 & 3.9 & 1.9 \\ Reflection & 37 & 2 & 4.6 & 2.6 \\
38 & 2 & 4.7 & 2.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Average reward changes at different powers. Figure 8: Comparative analysis of instant rewards and average rewards. Figure 9: Effect of RIS unit quantity on average reward. The average rewards under different numbers of RIS units are shown in Table 3. Under ideal and non-ideal reflection conditions, the average rate of the system increases with the increase in the number of RIS units. When the number of RIS units is 32, the link rate is significantly increased under ideal conditions, exhibiting a maximum increase of 4.9 \(\mathrm{bps}/\mathrm{Hz}\); under non-ideal conditions, the link rate increased from 2 \(\mathrm{bps}/\mathrm{Hz}\) to 5.8 \(\mathrm{bps}/\mathrm{Hz}\). In addition, the performance gap between ideal and non-ideal conditions increases with the number of RIS units. For example, the rate difference is 0.3 \(\mathrm{bps}/\mathrm{Hz}\) at 8 RIS units, and the difference widens to 1 \(\mathrm{bps}/\mathrm{Hz}\) at 32 RIS units, which is due to the decrease in signal reflection efficiency under non-ideal conditions. Increasing the number of RIS units can significantly increase the link rate, but the performance gain gradually decreases. Specifically, under non-ideal reflection conditions, the link rate increases by about 1.1 \(\mathrm{bps}/\mathrm{Hz}\) when the number of RIS units increases from 8 to 16; it only increases by 0.1 \(\mathrm{bps}/\mathrm{Hz}\) when the number increases from 16 to 32. In addition, an increase in the number of RIS units will lead to an increase in hardware and maintenance costs. Hardware costs increase linearly, and increased system complexity will lead to increased maintenance costs. Therefore, when optimizing the system, performance gains and costs should be balanced to ensure the best balance between overall performance and economic benefits. ## 5 Conclusions

In this paper, the performance optimization of RIS-assisted wireless communication systems in coal mine tunnels was investigated through simulation and algorithmic analysis. The conclusions are as follows:

1. Aiming at the complex environment of limited space in the coal mine tunnel, RIS technology is introduced into the coal mine wireless communication system, and combined with the DDPG optimization method, an effective signal enhancement solution is provided;
2. By jointly optimizing the base station power and RIS phase shift, the link rate is significantly enhanced under both ideal and non-ideal reflection conditions. Simulation results show that the link rate optimization effect of DDPG is better than that of A3C and DQN under the same conditions.