## 1 Introduction
Owing to their design flexibility and enhanced spectrum utilization, the FC-FB based waveform architecture is now being considered as a potential candidate for future 5G wireless technology. In comparison to conventional orthogonal multicarrier waveform like OFDM, FC-FB based multicarrier systems offer higher design flexibility as well as superior peak-to-average power ratio (PAPR) reduction performance, in addition to higher spectral efficiency. Hence, FC-FB based multicarrier waveforms seem to be perfect candidates for waveform architecture in cognitive NOMA systems. The advantage of using FC-FB by the secondary users (SU) is that the filter-bank structure allows flexible sub-division of the available frequency band into sub-bands, each of which may be scanned by the SU transmitter for evaluating the PU interference temperature (IT) threshold [11]. Based on the IT threshold, the SU transmitter can apply power control and design a fast convolution based sub-band filtered waveform [10], in which the SU transmissions are superimposed on those sub-bands with the PU transmission where the IT threshold is not exceeded. This is an example of a non-orthogonal cognitive underlay system, where the SU system is under-laid with the PU system. Compared to an underlay OMA network, more efficient spectrum utilization can be achieved by applying the NOMA protocol in CR scenario. This is attributed to the fact that unlike OMA-based underlay, a greater number of SUs in the underlay NOMA system can simultaneously receive individual signals in the same shared spectrum, thereby improving SU connectivity as well as throughput for SU access. This kind of cognitive NOMA concept has been presented in References [6; 12], which report the outage probability as well as diversity analysis in a large-scale underlay cognitive NOMA network. Moreover, the idea of cognitive NOMA has also been extended to a cooperative relay scenario, as reported in References [6; 13]. However, the application of FC-FB waveform architecture has not been reported in these works. Despite the enormous possible advantages offered by the FC-FB based cognitive NOMA system, interference management remains a critical issue, due to the fact that such systems are highly interference limited. These systems suffer from inter-network interference caused by power domain multiplexing of the SU and PU transmissions. Apart from transmit side power control, the SU receiver needs to implement inter-network interference cancellation strategies to mitigate interference arising from PU transmission. While most works on non-cognitive as well as cognitive NOMA are primarily focused on transmitter side power control [1; 2; 3; 4; 6; 7; 12; 13], few have explored the issue of enhanced signal detection at the SU receiver under the influence of strong PU interference. In most existing works, the receiver side processing in power-domain NOMA systems primarily involves conventional successive interference cancellation (SIC) based multi-user detection. However, such a scheme only relies on the allocated power levels of the NOMA signal components. It does not exploit the waveform structure for signal detection. Most multicarrier waveforms exhibit periodicities with their second order statistics (SOS). These periodicities result in specific correlation patterns in the spectral properties of the signals, which may be detectable even at low signal-to-noise ratios (SNRs) [14; 15]. These specific periodicity patterns are referred to as cyclostationary signatures [16]. User specific cyclostationary signatures can be induced in the FC-FB based SU-NOMA waveform at the transmitter side. A simple example is the cyclic prefix (CP) in conventional multicarrier systems. Spectral Correlation Density (SCD) function can be computed at the receiver side and artificial neural network (ANN) [17; 18] can be applied to detect the embedded cyclostationary signature for NOMA system recognition.