# Neural Network-Assisted DPD of Wideband PA Nonlinearity for Sub-Nyquist Sampling Systems
## 3 Proposed NN-Assisted Dual-Stage DPD for Sub-Nyquist Sampling Systems
### Stage 2: An Attention-Based NN for the Virtual Training of DPD
#### 3.2.3 Fully Connected Module  
The weighted signal output from the attention module is then input to a module with a fully connected layer for regression. The module contains three hidden layers. All hidden layers use the \(tanh\) activation functions. The output of the fully connected module can be simply expressed as  
\[\mathbf{\hat{x}}(n)=tanh(\mathbf{C}_{2}tanh(\mathbf{C}_{1}\boldsymbol{\chi}( n)+\mathbf{b}_{1})+\mathbf{b}_{2}), \tag{21}\]  
where \(\boldsymbol{\chi}(n)=[\chi_{1}(n),\chi_{2}(n),\ldots,\chi_{3Q+3}(n)]\), and \(\mathbf{C}_{1},\mathbf{C}_{2},\mathbf{b}_{1},\mathbf{b}_{2}\) denotes the parameters of the hidden layer, where \(\mathbf{C}_{1}\) can be denoted as  
\[\mathbf{C}_{1}=\left[\begin{array}{cccc}c_{1,11}&c_{1,12}&\cdots&c_{1,1(3Q+3 )}\\ c_{1,21}&c_{1,22}&\cdots&c_{1,2(3Q+3)}\\ \vdots&\vdots&&\vdots\\ c_{1,L_{1}1}&c_{1,L_{1}2}&\cdots&c_{1,L_{1}(3Q+3)}\end{array}\right]. \tag{22}\]  
and \(\mathbf{b}_{1}\) can be denoted as  
\[\mathbf{b}_{1}=[b_{1,1},b_{1,2},b_{1,3},\cdots,b_{1,L_{1}}]^{T}. \tag{23}\]  
Similarly, \(\mathbf{C}_{2}\) is expressed as  
\[\mathbf{C}_{2}=\left[\begin{array}{cccc}c_{2,11}&c_{2,12}&\cdots&c_{2,1(3Q+ 3)}\\ c_{2,21}&c_{2,22}&\cdots&c_{2,2(3Q+3)}\\ \vdots&\vdots&&\vdots\\ c_{2,L_{2}1}&c_{2,L_{2}2}&\cdots&c_{2,L_{2}(3Q+3)}\end{array}\right], \tag{24}\]  
and \(\mathbf{b}_{2}\) is expressed as  
\[\mathbf{b}_{2}=[b_{2,1},b_{2,2},b_{2,3},\cdots,b_{2,L_{2}}]^{T}, \tag{25}\]  
where \(L_{1}\) and \(L_{2}\) are the number of nodes in the first hidden layer and the second hidden layer, respectively. \(tanh()\) is the activation function, which has good approximation ability for DPDs [21], and it can be expressed as  
\[tanh(x)=\frac{1-exp(-2x)}{1+exp(-2x)}. \tag{26}\]The proposed attention-based NN is trained using the [PERSON] optimization algorithm and the loss of mean square error (MSE) to tune the parameters of the model, where the loss function is  
\[Loss=\frac{1}{2}\frac{1}{N_{0}}\sum_{n=0}^{N_{u}-1}\sum_{i=1}^{2}(\tilde{x}_{i}(n )-\tilde{x}_{i}(n))^{2}. \tag{27}\]  
The learning rate is set to 0.085. The specific steps of the proposed method are outlined in Algorithm 1.  
```
Input  1. Pilot signals: PA input signal \(\mathbf{x}\), Downsampled PA input \(\text{signals}_{D}\), feedback signal \(\mathbf{r}\).  2. Data: \(\{\tilde{\mathbf{u}}(n)\}\). Stage 1:  1. PA nonlinear model reconstruction using sub-nyquist Nyquist sampling data and calculate the polynomial coefficients: \[\mathbf{g}=(\mathbf{X}_{D}^{H}\mathbf{X}_{D})^{-1}\mathbf{X}_{D}^{H}\mathbf{r}.\]

2. Reconstruction of the feedback signal from down-mining using the estimated PA model parameters: \[\mathbf{r_{rec}}=\mathbf{Xg}.\]

3. The pre-estimated DPD parameters are obtained by building a PA inverse model based on the reconstructed feedback signal: \[\mathbf{a}=(\mathbf{R}_{rec}^{H}\mathbf{R}_{rec})^{-1}\mathbf{R}_{rec}^{H} \mathbf{x}.\] Stage 2:  Virtual training:  1. Generate expected DPD outputs using data: \(\tilde{\mathbf{x}}(n)=\tilde{\mathbf{u}}(n)\mathbf{a},n=0,1,\ldots,N_{0}-1\).  2. Virtual training of the DPD:  (1) Formulate the inputs and outputs of the proposed NN using (16) and (17);  (2) Calculate the output of the attention module as described in (20);  (3) Determine the output of the proposed NN based on (21);  (4) Update the hyper parameters according to the loss function in (27). Predicting  Using the train NN to predict \(\hat{x}(n)\).
```  
**Algorithm 1** The Proposed Dual-Stage DPD Method for sub-Nyquist Sampling Systems