## 2 Transmitter and Receiver of OvTDM Systems
\right) \tag{6}\]  
where \(g_{n}\) is the interference coefficient of the \(n\)-th symbol adjacent to the current symbol on the current symbol, and \(L_{ISI}\) is the number of unilateral interference symbols. Expanding the matrix form gives  
\[y[n] = \sum_{m=0}^{N-1}x[m]g_{n-m}+\eta[n] \tag{7}\] \[= \sum_{k=0}^{J-1}\sum_{m=0}^{N-1}x_{k}[m]g_{n-m}+\eta[n],\]where \(g_{n}\) is the interference coefficient of the \(n\)-th symbol adjacent to the current symbol. From the above equation, it can be seen that the sampled value at the current moment will be affected not only by the interference between symbols in the same layer, but also by the interference in other layers as well as the color noise. For the OvTDM system with SCM, at the receiver side, the Forney model [25] adds a whitening filter on the receiving end, which transforms color noise into white noise. Hence, the one-by-one interference cancellation algorithm will be used to process the interference of the other layers first. Meanwhile, for each layer of signal, the M-BCJR detection algorithm based on the Forney model will be used to deal with the effect of the inter-symbol interference and the channel noise [26; 27]. The whole process of receiving end detection can be briefly described as follows. 1. The detection at the receiver side starts from the layer with the highest transmit power, where \(P_{0},P_{1},\ldots,P_{K-1}\) decreases sequentially. First, initialize the symbol probability \(P(\hat{x}_{k}[l])\) for each layer of symbols as an equal probability distribution, and then calculate the mean \(E\{\hat{x}_{k}[l]\}\) and variance \(\text{Var}\{\hat{x}_{k}[l]\}\) of the symbols at each moment of each layer based on the probability, with \(k=0\). 2. Let the current decoding layer be the k-th layer, and firstly calculate the mean magnitude \(\mu_{k}\) and variance \(\sigma_{k}^{2}\) of the interference values caused by other layers to the current layer. 3. The input \(y_{k}\) to the \(k\)-th layer OvTDM signal detector is obtained by subtracting \(\mu_{k}\) from the sufficient statistic \(y\) obtained after matched filtering and down-sampling. 4. Detection of \(y_{k}\) using the M-BCJR algorithm. 5. The symbol probability obtained at the end of M-BCJR detection is fed into the decoder, the detector and decoder are iterated a number of times to obtain the estimated bit sequence and the corresponding probability information for the current layer, and the obtained bit probability is re-mapped into the symbol probability to recalculate the symbol mean-variance. Then, update \(E\{\hat{x}_{k}[l]\}\) and \(\text{Var}\{\hat{x}_{k}[l]\}\). When \(k\
eq K-1\), let \(k=k+1\), restart the detection of the next layer of signals from step (ii), otherwise jump to step (vi). 6. When \(k=K+1\), it means that each layer of the signal is detected. At this time, \(E\{\hat{x}_{k}[l]\}\) and \(\text{Var}\{\hat{x}_{k}[l]\}\) of each layer have been updated once. The updated \(E\{\hat{x}_{k}[l]\}\) and \(\text{Var}\{\hat{x}_{k}[l]\}\) can be used to restart the detection from the first layer by ordering \(k=0\), and restarting the detection from step (ii). At this point, \(E\{\hat{x}_{k}[l]\}\) and \(\text{Var}\{\hat{x}_{k}[l]\}\) are much more accurate compared to the homogeneous initialization with the first step, and thus the second overall iteration will provide a better performance than the first one. As results, a number of iterations of detection can achieve better performance. Figure 2: Block diagram of the receiver using SIC detection for OvTDM.