## 1 Introduction
Specifically, the IRS smartly reconfigured the cascade signal emitted from the base station (BS) transmitter (\(\text{BS}_{\text{Tx}}\)) to the IRS and then reflected to the BS receiver (\(\text{BS}_{\text{Rx}}\)) to create destructive interference with the SI signal. Moreover, taking into account the quantization noise of analog-to-digital converters (ADCs), the authors of [27] considered a more practical scenario and proposed to jointly design the downlink (DL) precoding matrix and the IRS coefficients. However, due to hardware limitations, the IRS can only reflect the signal back to the same side [30]. Therefore, it can only service user equipment (UE) on the same side of the BS. To tackle this constraint, the authors of [28] first introduced intelligent omnivarface (IOS) to FD systems, which is named simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) in some studies [31]. Compared with the IRS, the IOS can simultaneously reflect and refract the incident signal to both sides [30; 31]. Thus, it can extend the wireless coverage to the full space while suppressing the SI signal in FD systems. Although previous studies have focused on the performance of RIS-assisted FD systems, there remains a notable gap in the literature regarding the simultaneous SIC and sum rate maximization (SRM) of DL and UL. Ref. [28] only addressed SI constraints targeting DL rate optimization, while [25] assumed perfect SIC when maximizing the sum rate (SR). However, minimizing the residual SI is crucial for effective air interface SIC as excessive SI will lead to the saturation of ADCs, preventing it from functioning properly [7]. Thus, it is necessary to investigate beamforming optimization for the SRM under practical SI constraints in RIS-assisted FD systems. Moreover, it is notable that although current IOS models offer both reflection and refraction capabilities, they cannot independently adjust reflecting and refracting phase shifts [30; 31; 32]. This limitation results in coupled beamforming on both sides, which leads to a serious performance loss in the FD system as the IOS is expected to simultaneously suppress the SI signal and enhance the system data rate. To deal with these issues, in this paper, we introduce a novel bilayer intelligent omnivarface (BIOS) structure to simultaneously suppress the SI and enhance the system SR in FD systems. This BIOS structure deploys two neighboring IOS layers, and thus can ensure independent beamforming design on both sides [33]. Unlike [33], which utilizes both reflected and refracted beams for communication, our design employs the reflected signals for SIC and the refracted signal for communication, thus achieving both objectives simultaneously. Our approach is grounded in a practical scenario that considers the power constraint of the SI signal. The main contributions are summarized as follows:  
* We introduce a novel BIOS in a multi-user FD system. With the unique bilayer structure, the BIOS can provide flexible beamforming on both sides and thus can simultaneously improve the SR and cancel the SI more effectively than the conventional RIS designs. * For the BIOS-assisted multi-user FD system, we jointly optimize the beamforming at the BS and BIOS to maximize the SR of UL and DL. We ensure effective over-the-air SIC by constraining the residual SI power below a threshold, marking a distinct approach from that of other studies. To solve this problem, we establish the proof of equivalence between the SRM and a weighted MSE minimization (WMMSE) and then propose a novel SIC-WMMSE algorithm. * To further guide the practical deployment of BIOS-assisted FD systems, we theoretically derive a lower bound of the BIOS size required for efficient SIC, which is proven to be quite close to the accurate size required in practical systems. * We provide simulation results to show that the BIOS-assisted FD system significantly outperforms that of the conventional IOS due to its capability of independent beamforming on both sides.