## 4 Performance Analysis and Simulation Outcomes
### Simulation Setup  
A single cell of a cellular wireless network is considered, where the BS is placed at the center, and both the CUs and D2D pairs are uniformly distributed within the cell. The IRS is located between the D2D pairs. The locations of the CUs, D2D pairs, and IRS are generated initially (in one realization) and then kept fixed throughout the simulation. Furthermore, each antenna of both the CUs and the D2D pairs is assumed to have an isotropic radiation pattern with 0 dB antenna gain, where each reflecting element of the IRS is assumed to have a 3 dB gain for a fair comparison since each IRS reflects signals only in its front half-space [28]. Two categories of fading, i.e., large-scale and small-scale fading, are considered for the channel model. The large-scale fading based on the distance-dependent path loss model is considered. The small-scale fading components of \(h_{t,r}\), \(v_{b,r}\), \(g_{b,c}\), and \(u_{t,c}\) follow the Rayleigh fading model, while the small-scale fading components of \(g_{b,i}\), \(h_{t,i}\), \(g_{i,c}\), and \(h_{i,r}\) follow the Rician fading model [28], i.e.,  
\[g_{b,i}=\sqrt{\frac{h_{b,i}}{h_{b,i}+1}}\tilde{g}^{b,i}+\sqrt{\frac{1}{\mu_{b, i}+1}}\tilde{g}_{b,i}, \tag{30}\]  
where \(\tilde{g}_{b,i}\) denotes the line-of-site (LOS) component, \(\hat{g}_{b,i}\) denotes the non-LOS (NLOS) component, and \(\mu\) represents the Rician factor. The simulation parameters used for the IRS-aided D2D underlay cellular network are provided in Table 1.  
The simulation procedures of the RL-based proposed solution schemes consist of two segments, i.e., learning and testing. First, the RL-based framework learns from the constructed models, and then, the learned framework is tested under different parameter settings to evaluate the system performance. Unless specified otherwise, the parameters for the learning phase and the testing phase remain the same. In the proposed actor-critic solution scheme, a three-layer fully connected neural network with two hidden layers is considered as the actor. The actor network uses a rectified linear unit (ReLU) function for the first and second layers and uses _tanh_ for the final layer. For the critic, a four-layer fully connected neural network with three hidden layers is considered. The critic network uses the ReLU function for the first, second, and third layers and leaner activation for the final layer. To simulate the model, the Python 4.5 platform is utilized. All the network parameters are trained using the Adam optimizer [29]. The simulation parameters related to the RL model are provided in Table 2. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Values** \\ \hline Cellular cell radius & 500 m \\ \hline D2D link distance & 10–50 m \\ \hline Number of CUs & 5–25 \\ \hline Number of D2D users & 5–35 \\ \hline Number of reflecting elements & 50 \\ \hline CUs’ maximum transmit power & 24 dBm \\ \hline D2D transmitters’ maximum transmit power & 24 dBm \\ \hline CUs’ minimum SINR requirement & 0.3 bps/Hz \\ \hline D2D receivers’ minimum SINR requirement & 0.3 bps/Hz \\ \hline Resource block bandwidth & 180 kHz \\ \hline Pathloss exponent & 4 \\ \hline Pathloss constant & \(10^{-2}\) \\ \hline Shadowing & 8 dB \\ \hline Multi-path fading & 1 \\ \hline Noise spectral density & \(-144\) dBm/Hz \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation Parameters for IRS-aided D2D system. To analyze the system performance, the RL-based proposed solution schemes are compared with the following two benchmark schemes:  
* Underlaying D2D without RL (scheme-1): An IRS-empowered underlay D2D communication network is considered scheme-1 [28]. Here, a user-pairing scheme determines the resource reuse indicator, and then, the transmit power and the passive beamforming are jointly optimized by iterative algorithms.