# Channel Estimation for UAV Communication Systems Using Deep Neural Networks
## 2 System Model and Problem Formulation
### Problem Formulation
#### 2.1.2 Enhanced Gradient  
The recently enhanced gradient was proposed by [21] to update the invariant rule of the Boltzmann machines for data representation. A bit-flipping transformation introduced the gradient, and then, the rule was updated to improve the results, the learning of the RBM, and make it less sensitive to the parameter variation and initialization. The same idea was used to enhance the gradient of GDBM by using Gaussian visible neurons as an alternative to the bit-flipping transformations by shifting the visible unit. Ref. [21] proposed a new method to enhance the gradient to replace the equations of (8a)-(8c). They defined the covariance between the two variables under the distribution \(P\)  
\[COV_{P}(v_{i},h_{j})=\left\langle v_{i}h_{j}\right\rangle_{p}\left\langle v_{i }\right\rangle_{p}\left\langle h_{j}\right\rangle_{p} \tag{10}\]The standard gradient in (8a) can be rewritten as  
\[\
abla_{W_{ij}} = COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j}) \tag{11}\] \[+ \langle v_{i}\rangle_{dm}\
abla_{c_{j}}+\langle h_{j}\rangle_{dm} \
abla_{h_{i}},\]  
where \(\langle.\rangle_{dm}=\frac{1}{2}\langle.\rangle_{d}+\frac{1}{2}\langle. \rangle_{m}\) denotes the average of the model distribution with the data. The standard deviation has some potential problems. The gradient is correlated with the weights and the bias terms. Additionally, \(COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j})\) is uncorrelated with \(\
abla_{h_{i}}\) and \(\
abla_{c_{j}}\), which may lead to distract the learning with non-useful weights when there are a lot of active neurons for which \(\langle.\rangle_{dm}\approx 1\). The current problem is that using (11) to update the data varies if we depend on data representation, and that variation can make the data flip from one to zero and vice versa  
\[\tilde{v}_{i} = \tilde{v}^{(1-f_{i})}(1-v_{i})^{f_{i}}\hskip 28.452756 ptf_{i}\in \{0,1\}, \tag{12a}\] \[\tilde{h}_{i} = \tilde{h}^{(1-\tilde{g}_{i})}(1-h_{i})^{\tilde{g}_{i}}\hskip 28.452756 ptg _{i}\in\{0,1\}, \tag{12b}\]  
The parameters are transformed accordingly to \(\tilde{\theta}\)  
\[\tilde{w}_{ij} = (-1)^{f_{i}+\tilde{g}_{j}}w_{ij}, \tag{13a}\] \[\tilde{b}_{i} = (-1)^{f_{i}}\Bigg{(}b_{i}+\sum_{j}g_{j}W_{ij}\Bigg{)}, \tag{13b}\]  
The energy function is equivalent to \(E(\tilde{x}+\tilde{\theta})=E(x+\theta)+a\), where \(a\) is a constant for all the values. That will lead to an update of the model and then transformed again. The resulting model will be defined as  
\[w_{ij}\leftarrow w_{ij}+\eta\left[COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j})\right. \tag{14a}\] \[\left.+(\langle v_{i}\rangle_{dm}-f_{i})\
abla c_{j}+\left( \langle h_{j}\rangle_{dm}-g_{j}\right)\
abla_{h_{j}}\right],\] \[b_{i}\leftarrow b_{i}+\eta\left[\
abla b_{i}-\sum_{i}g_{j}(\
abla w_{ij}-f_{i} \
abla c_{j}-g_{i}\
abla b_{i})\right],\] (14b) \[\
abla_{W_{ij}} = COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j}) \tag{11}\]0  
where \(\
abla\theta\) represents the gradient parameters defined in Equation (8). Therefore, it will be a \(2^{nv+nh}\) different update rule, where \(nv\) and \(nh\) are the numbers of hidden and visible neurons. In order to find the maximum likelihood updates, Ref. [20] proposed a new gradient weighted sum \(2^{nv+nh}\) with the following weights:  
\[\
abla_{W_{ij}} = COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j}) \tag{11}\]1  
Due to the larger weights, the enhanced gradient is defined as:  
\[\
abla_{W_{ij}} = COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j}) \tag{11}\]2 \[\
abla_{W_{ij}} = COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j}) \tag{11}\]3 (16b) \[\
abla_{W_{ij}} = COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j}) \tag{11}\]4  
where \(\
abla_{e}w_{ij}\) has the same form of (10) and the bias gradient terms is deleted, as shown in \[\
abla_{W_{ij}} = COV_{d}(v_{i},h_{j})-COV_{m}(v_{i},h_{j}) \tag{11}\]5  
Where each block is connected to the upper block through the unit of the hidden lower to update the parameter in (16) layer by layer for GBRBM pre-training. The pre-training of GBRBMs provides the initializing to deep autoencoder.