## 2 General System Model  
A general cooperative system in correlated environment is graphically depicted in Figure 1. The system consists of a source \(S\), a destination \(D\), and \(K\) number of relays, which are denoted as \(\left\{R_{k}\right\}_{k=1}^{K}\). All the nodes have a single antenna that can be used for both transmission and reception. All the relays are in the half-duplex mode, and thus, cannot transmit and receive simultaneously; they also employ the variable-gain AnF relaying protocol. The channel gain between node \(i\) and \(j\) is denoted as \(h_{ij}\). The noise associated with each channel is modeled as a mutually independent additive white Gaussian noise (AWGN) with zero-mean and variance \(N_{0}\). We do not consider power allocation issues as they are outside the scope of this work. We assume that each transmitter (source or relay) transmits information with a fixed power \(P\). We use the notation \(\gamma_{ij}=\frac{P}{N_{0}}\left|h_{ij}\right|^{2}\) to refer to the instantaneous SNR of link \(i-j\). Additionally, we use \(f_{X}\left(\cdot\right)\) and \(F_{X}\left(\cdot\right)\) to denote the probability density function (PDF) and the cumulative density function (CDF) of a random variable \(X\), respectively. Furthermore, the notation \(\Pr\left[Y\right]\) is the probability of an event \(Y\). Depending on the characteristic or geometry of the channels, channel correlations may exist. For instance, due to feedback or scheduling delays, the channel state of the selected relay at the selection decision instant can substantially differ from the actual channel during its data forwarding [10; 11]. As a result, the correlation, \(\hat{\rho}_{ij}\), exists between the actual channel (\(h_{ij}\)) and the corresponding estimate (\(\hat{h}_{ij}\)). In other words, \(h_{ij}\) and \(\hat{h}_{ij}\) are correlated random variables (RVs) with the following correlation matrix:  
\[\text{CR}_{1}=\left( \begin{array}{cc}1&\hat{\rho}_{ij}\\ \hat{\rho}_{ij}&1\end{array} \right). \tag{1}\]  
Alternatively, if there exist common scatterers near the source, the first-hop and the source-destination channels are likely to be correlated to each other. The reason is that nearby channels are often affected by the same scatterers (shadowners); therefore, channels' losses are correlated [12; 13; 14]. Consequently, \(\left\{h_{SR_{k}}\right\}_{k=1}^{K}\) and \(h_{SD}\) may be modelled as correlated RVs with the following correlation matrix:  
\[\text{CR}_{2}=\left( \begin{array}{cccc}1&\rho_{SR_{2}}^{SR_{1}}&\ldots&\rho _{SR_{K}}^{SR_{1}}&\rho_{SD}^{SR_{1}}\\ \rho_{SR_{2}}^{SR_{1}}&1&\ldots&\rho_{SR_{K}}^{SR_{2}}&\rho_{SD}^{SR_{2}}\\ \vdots&\vdots&\ddots&\vdots&\vdots\\ \rho_{SR_{K}}^{SR_{1}}&\rho_{SR_{2}}^{SR_{2}}&\ldots&1&\rho_{SD}^{SR_{K}}\\ \rho_{SR_{1}}^{SR_{1}}&\rho_{SD}^{SR_{2}}&\ldots&\rho_{SD}^{SR_{K}}&1\end{array}  \right). \tag{2}\]Similarly, \(\left\{h_{R_{k}D}\right\}_{k=1}^{K}\) and \(h_{SD}\) may also be correlated RVs with a correlation matrix given by  
\[\text{CR}_{3}=\left( \begin{array}{cccc}1&\rho_{R_{2}D}^{R_{1}D}&\ldots&\rho_{R _{k}D}^{R_{1}D}&\rho_{SD}^{R_{1}D}\\ \rho_{R_{2}D}^{R_{1}D}&1&\ldots&\rho_{R_{k}D}^{R_{2}D}&\rho_{SD}^{R_{2}D}\\ \vdots&\vdots&\ddots&\vdots&\vdots\\ \rho_{R_{k}D}^{R_{1}D}&\rho_{R_{k}D}^{R_{2}D}&\ldots&1&\rho_{SD}^{R_{k}D}\\ \rho_{SD}^{R_{1}D}&\rho_{SD}^{R_{2}D}&\ldots&\rho_{SD}^{R_{k}D}&1\\ \end{array} \right), \tag{3}\]  
when there are common scatterers in the vicinity of the destination. Moreover, in indoor systems or in systems in which the source-\(R_{k}\) and the \(R_{k}\)-destination channels are impinged by the same scatterers, the two channels of the relay \(R_{k}\) can also be correlated with the following correlation matrix [8; 9; 13; 15]:  
\[\text{CR}_{4}=\left( \begin{array}{cc}1&\rho_{k}\\ \rho_{k}&1\\ \end{array} \right). \tag{4}\]  
Because the relays operate in the half-duplex mode, each source-destination communication is divided into two consecutive phases.