## 2 System Model  
The structure of the analyzed STAR system is presented in Figure 2, with signals propagating at the different stages. We denote the baseband transmission at time slot \(n\) as \(x[n]\), which is perfectly known. In this analysis, the RF components are assumed to be ideal and the nonlinear model will be discussed later. The received signal, after being processed \begin{table}
\begin{tabular}{c c} \hline \hline
**Acronym** & **Definition** \\ \hline STAR & simultaneous transmit and receive \\ IBFD & in-band full-duplex \\ SI & self-interference \\ SIC & self-interference cancellation \\ RF & Radio Frequency \\ LNA & low-noise amplifier \\ PA & power amplifier \\ LMS & least mean square \\ PH & parallel Hammerstein \\ TX & transmitter \\ RT & remote transmitter \\ HP & Hammerstein polynomials \\ SNR & signal-to-noise ratio \\ ISR & interference-to-signal ratio \\ m-MTLS & multi-layered M-estimate total least squares \\ NMSD & normalized mean squared difference \\ MSE & mean squared error \\ BER & bit error rate \\ DFE & decision feedback equalizer \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Table of acronyms. by the LNA and and subjected to SIC in the analog domain, is converted into a baseband signal by the ADC at time slot \(n\) and can be written as  
\[ \begin{split} y[n]&=r[n]+f(x[n],x[n-1],\ldots x[n-L+1])+ v[n]\\ &=r[n]+s[n]+v[n],\end{split}  \tag{1}\]  
where \(r[n]\) and \(v[n]\) denote the desired signal from the remote source and noise at time instant \(n\), respectively. The received signal is assumed to be generated by a function \(f(\cdot)\) with \(L\) previous transmitted signals. Furthermore, the SI signal from the local TX, \(s[n]\), can be represented by a linear model as  
\[ \begin{split} s[n]&=f(x[n],x[n-1],\ldots,x[n-L+1])\\ &=\mathbf{w}^{T}[n]\mathbf{x}_{L}[n],\end{split}  \tag{2}\]  
where \(\mathbf{w}\in\mathbb{R}^{N\times 1}\) denotes the SI channel and \(\mathbf{x}_{L}[n]=[x[n],x[n-1],\ldots,x[n+L-1]]\). When the nonlinear hardware impairments are taken into account, one particularly important imperfection is PA nonlinearity; thus, we assume that the transmitted signal is only distorted by high-order harmonics of the PA, excluding the effects of I/Q imbalance or phase noise. The PAs exhibit nonlinearity when power-efficient operation is sought, and it is commonly modeled with the well-known PH model [14; 19; 22; 24]. With an input vector \(\mathbf{x}_{L}[n]\), the output of a \(P\)-th order model can be expressed as  
\[ \begin{split} x_{\text{PA}}(t)&=\sum_{\begin{subarray}{ c}p=1\\ p\text{ odd}\end{subarray}}^{p+1}h_{p,\text{PA}}(t)*\phi_{p}(x(t)),\end{split}  \tag{3}\]  
where \(P\) is the highest nonlinearity order of the model, \(h_{p,PA}(t)\) represents the response, and \(\phi_{p}(x(t))=|x(t)|^{2p}x(t)\) is the \(p\)th-order basis function. The symbol \(*\) represents the convolution operation. We assume that the PA memory length is \(L_{1}\) and the length of the SI channel is \(L_{2}\). Then, the observed SI signal passes through the SI channel and can be rewritten as  
\[ \begin{split} s[n]&=\sum_{\begin{subarray}{c}p=1\\ p\text{ odd}\end{subarray}}^{\frac{p+1}{2}}\sum_{l=0}^{L-1}w_{p}[l]\phi_{p}(x [n-l]),\end{split}  \tag{4}\]  
where \(\mathbf{w}_{p}\) is the response weight of the entire channel, with a length of \(L=L_{1}+L_{2}-1\). Building on the analysis presented earlier, for the purpose of mitigating residual SI within the digital domain, it is imperative to acquire the optimal estimate of the SI channel response, denoted as \(\hat{\mathbf{w}}[n]\), corresponding to \(\mathbf{w}[n]\). This involves subtracting  
Figure 2: The signals propagating at the different stages of the analyzed STAR system. the reconstructed SI signal from \(y[n]\). Followed by the SIC process, the remainder of the signal is presumed to embody the RT signal. Subsequent stages may include RT channel estimation and equalization, facilitating the detection of data symbols embedded within the RT signal.