## 3 Problem Formulation
### Cross-Entropy Method  
The CE method was originally used in the context of rare event simulation [17] and has been extended as a Monte Carlo method for importance sampling and optimization [17; 18]. The principle behind the CE method is to get as close as possible to the optimal importance sampling distribution by using the Kullback--Leibler (KL) distance as a measure of closeness. By repeatedly updating the Probability Density Function (PDF) of generated samples, the PDF of the samples can finally converge with the obtained optimal strategy solution [19]. Another method with a similar idea is logarithmic loss distortion measure [20; 21], where logarithmic loss is also known as cross-entropy loss. The logarithmic loss distortion measure has been recently used in the study of Deep Neural Networks (DNNs) to approach an accurate classifier by minimizing the logarithmic loss. It also performs well on tradeoff between complexity and relevance in representation learning [22]. The main steps of the CE method can be depicted as follows:  
**STEP 1: (Encode Strategy Space)**. Consider a UDN constituted by \(B^{\Omega}\) BSs, where each BS \(b\) is a decision-making entity. Suppose that each BS \(b\) can make a decision out of \(N_{b}\) possible strategies, then the strategy set at BS \(b\) can be expressed as \(\mathbf{S}_{b}=[S_{b}^{1},S_{b}^{2},\cdots,S_{b}^{N_{b}}]\). For a specific decision-making entity BS \(b\), \(S_{b}^{i}\) is one strategy that belongs to \(\mathbf{S}_{b}\). The strategy set of \(B^{\Omega}\) BSs entities in a UDN can be represented as \(\mathcal{S}_{B^{\Omega}}=[\mathbf{S}_{1},\mathbf{S}_{2},\cdots,\mathbf{S}_{| B^{\Omega}|}]\), which is termed as the strategy space of the CE method. Then the samples of the strategy space of the CE method correspond to the strategies at all BSs in a UDN in current iteration. Let \(\mathbf{P}_{\mathbf{b}}=[P_{b,1},P_{b,2},\cdots,P_{b,N_{b}}]\) denote the probability distribution of sample strategies at BS \(b\). Let \(P_{b,i}\) denote as the probability of strategy \(i\) at a specific BS \(b\). First, \(P_{b,i}\) is initialized to be equal as follows,  
\[P_{b,i}=\frac{1}{N_{b}},\text{ and }\sum_{i=1}^{N_{b}}P_{b,i}=1,(\forall i=1, \cdots,N_{b}). \tag{17}\]  
**STEP 2: (Generate Samples According to the Probability)**. In the second step of the CE method, sufficient strategy samples should be generated according to the given probability distribution. Denote the \(z\)th generated sample by \(\mathbb{A}(z)=\left[\mathbf{A}_{1}^{T}(z),\mathbf{A}_{2}^{T}(z),\cdots, \mathbf{A}_{b}^{T}(z),\cdots,\mathbf{A}_{|B^{\Omega}|}^{T}(z)\right]\), where \(\mathbf{A}_{b}(z)\) is the subsample set generated by decision-making entity element \(b\). More specifically, the subsample \(\mathbf{A}_{b}(z)\) can be represented as:  
\[\mathbf{A}_{b}(z)=\left[\alpha_{b,1}(z),\alpha_{b,2}(z),\cdots,\alpha_{b,i}( z),\cdots,\alpha_{b,N_{b}}(z)\right], \tag{18}\]  
where \(\alpha_{b,i}(z)\) is a binary number. For each \(\mathbf{A}_{b}(z)\), only one \(\alpha_{b,i}(z)\) is \"\(1\)\" and others are \"\(0\)\" (i.e., \(\sum_{i=1}^{N_{b}}\alpha_{b,i}=1\)), indicating that only one strategy out of all can be selected at each decision epoch. The probability of subsample \(\mathbf{A}_{b}\) is \(P_{b,i}\in\mathbf{P}_{\mathbf{b}}\) with \(\alpha_{b,i}=1\) and \(\alpha_{b,i}=0(j\
eq i)\). **STEP 3: (Performance Evaluation)**. Fitness values of strategy samples can be calculated according to the result of the strategy in the current iteration. Let \(F(z)\) denote the fitness value of strategy sample \(z\), which can be expressed as:  
\[F(z)=-\sum_{b}\sum_{m}x_{mb}(1-\mathbf{q}_{m}\mathbf{y}_{b}^{T})U_{back}. \tag{19}\]  
Rearrange \(F(z)\) in descending order as \(F(1)\geq F(2)\cdots F(Z)\), where \(Z\) is the maximum number of strategy samples. Then calculate the \(\rho\)-quantile of the strategy samples in current iteration \(F_{\rho}\) and weed out the unexpected samples. The samples with fitness value \(F(i)\geq F_{\rho}\) are selected for probability updating in the next iteration.