JAIST Repository
https://dspace.jaist.ac.jp/

Title

SemantIC: Semantic Interference Cancellation
Towards 6G Wireless Communications

Author(s)Lin, Wensheng; Yan, Yuna; Li, Lixin; Han, Zhu;
Matsumoto, Tad

Citation

IEEE Communications Letters: 1-1

Issue Date

2024-06-11

Type

Journal Article

Text version

author

URL

http://hdl.handle.net/10119/19048

RightsThis is the author's version of the work. Copyright(C) 2024 IEEE. IEEE Communications Letters. DOI:
10.1109/LCOMM.2024.3412973. Personal use ofthis material is permitted. Permission from IEEE
must be obtained for all other uses, in any current
or future media, including reprinting/republishing
this material for advertising or promotional
purposes, creating new collective works, for resaleor redistribution to servers or lists, or reuse of any
copyrighted component of this work in other works.

Description

Japan Advanced Institute of Science and Technology

1

SemantIC: Semantic Interference Cancellation
Towards 6G Wireless Communications
Wensheng Lin, Member, IEEE, Yuna Yan, Lixin Li, Member, IEEE,
Zhu Han, Fellow, IEEE and Tad Matsumoto, Life Fellow, IEEE

Abstract—This letter proposes a novel anti-interference technique, semantic interference cancellation (SemantIC), for enhancing information quality towards the sixth-generation (6G)
wireless networks. SemantIC only requires the receiver to concatenate the channel decoder with a semantic auto-encoder.
This constructs a turbo loop which iteratively and alternately
eliminates noise in the signal domain and the semantic domain.
From the viewpoint of network information theory, the neuralnetwork of the semantic auto-encoder stores side information bytraining, and provides side information in iterative decoding, as
an implementation of the Wyner-Ziv theorem. Simulation results
verify the performance improvement by SemantIC without extra
channel resource cost.
Index Terms—Semantic interference cancellation, Wyner-Ziv
theorem, anti-interference, side information, turbo principle.

I. I NTRODUCTION
How to implement the Wyner-Ziv theorem [1] in practicalcoding design? Academia and industry have struggled with thisquestion for several decades, although the Wyner-Ziv theorem
provides a prospect to enhance the information quality byproviding side information at the decoder only. The challengeof practical implementation is to find a method to manage sideinformation.
Thanks to recent development of semantic communications[2], which is a promising technological basis for the oncoming
sixth-generation (6G) wireless networks [3], we can obtainthe solution to representing side information by semantic
neural networks. In semantic communications, the semantic
encoder extracts and transmits the semantic information, i.e.,
the feature of the original information. Then, the receiver
generates and reconstructs the original information from thesemantic information by the semantic decoder [4]. By thismeans, the payload transmitted through the channel can besignificantly reduced. From the viewpoint of network information theory, the semantic encoder/decoder pair learns andshares the same side information by joint training. Hence,
when separately deploying the semantic encoder and decoder,
Corresponding author: Lixin Li.
W. Lin, Y. Yan and L. Li are with the School of Electronics and Information,
Northwestern Polytechnical University, Xi’an, Shaanxi 710129, China (e-mail:linwest@nwpu.edu.cn; yanyuna@mail.nwpu.edu.cn; lilixin@nwpu.edu.cn).
Z. Han is with the Department of Electrical and Computer Engineering
at the University of Houston, Houston, TX 77004 USA, and also with the
Department of Computer Science and Engineering, Kyung Hee University,
Seoul, South Korea, 446-701 (e-mail: hanzhu22@gmail.com).
T. Matsumoto is an Invited Professor at IMT-Atlantique, France. He is
also Professor Emeritus of Japan Advanced Institute of Science Technology,
Ishikawa 923-1292, Japan, and University of Oulu, Finland (e-mail: matumoto@jaist.ac.jp).

Encoder

Decoder
Knowledgeof source

Fig. 1. The system model from the viewpoint of network information theory.
X n , Y n and X̂ n stand for the source information, the side information andthe reconstructed information, respectively, with the sequence length of n. M
is the codeword satisfying the coding rate R.

the transmitter can compress the source information according
to side information, and the decoder can recover the source
information with the assistance of side information.
It is noticed that the function of semantic decoder canbe regarded as generative artificial intelligence (AI) [5], [6].
The diffusion model [7] is a generative AI method effective
for achieving super-resolution, which can be also regarded
as a process of eliminating noise. Particularly, in the reverse
diffusion process, noise is predicted by neural networks andsubtracted from the image. Therefore, generative AI implicitly
contains the capability for eliminating noise. Similarly, if
we train the semantic encoder and decoder by adding noise
between them, the semantic encoder and decoder will also
work better in denoising.
Inspired by the Wyner-Ziv theorem and semantic communications, this letter proposes a breakthrough technique, namely
semantic interference cancellation (SemantIC), to improve theinformation quality by updating the receiver structure only.
The theoretical model of SemantIC is exactly the Wyner-Ziv
model as illustrated in Fig. 1, where the side information (inpractice, the knowledge of source) is learned by neural networks and stored in the form of the neural network parametersat the decoder. The proposed SemantIC technique naturally
exploits the semantic information contained in the data, andhence it well matches the feature of 6G which focuses moreon the semantic information of data.
In practical implementation, the SemantIC receiver concatenates a pair of semantic encoder/decoder with the channel decoder. The semantic encoder extracts the semantic informationof the channel decoder output. In this way, the noise and interference can be eliminated to a certain extent when extracting
information features. From the viewpoint of machine learning,
the concatenated semantic encoder/decoder pair constructs an
auto-encoder [8], [9], which has also been verified to be able
to eliminate noise [10]. Subsequently, based on the denoised
information features, the semantic decoder reconstructs theoriginal information to input into the channel decoder as the

2

II. S EMANTIC I NTERFERENCE C ANCELLATION P RINCIPLE
A. Theoretical Basis
The theoretical basis of SemantIC is the Wyner-Ziv theorem.
As illustrated in Fig. 1, the Wyner-Ziv system is required totransmit a source sequence X n under the rate constraint R.
Therefore, an encoder compresses the source sequence X n
into a codeword M before transmission. Then, at the decoder,
the noncausal side information sequence Y n is available tohelp better reconstruct the source sequence. The recoveryX̂ n may be lossy if both the channel capacity and the sideinformation is insufficient. Mathematically, the link rate R
should satisfy the rate-distortion function given a distortion
requirement D, well known as the Wyner-Ziv theorem:
R(D) ≥ I(X; U |Y ),(1)

where U represents the compress information of X.
In practical systems, the mutual information gain provided
by the knowledge base can be mathematically evaluated as
Igain = I(X; X̂S ) − I(X; X̂C ),(2)

where X̂S is the information recovered with the help of thesemantic auto-encoder, and X̂C is the information independently recovered by the channel decoder only. Both I(X; X̂S )and I(X; X̂C ) can be calculated from the mutual information
between the transmitted and the recovered information.
Although Wyner and Ziv have presented a coding scheme in
the proof of the rate-distortion function in (1), their proposed
algorithm requires the sequence length tends to ∞. Moreover,since their coding scheme utilizes a random codebook, it
requires a huge memory to store the codebook, and theencoding/decoding process is of very high time-complexity
due to exhaustive search over the random codebook.
To implement the Wyner-Ziv theorem in practical systems,
we have to solve many tricky problems including how to
obtain, store and represent side information, as well as how to

Semantic auto-encoder
Channelencoder

Channel Turbo
decoder loop

Semantic
encoderSemantic
decoder

Transmitter

Semantic
informationa priori information for the next round of iterative decoding.
Note that the channel decoder concatenated with the semantic auto-encoder composes a turbo loop. Owing to the turbo
principle [11], the decoding gain is alternately enhanced by thechannel decoder and the semantic auto-encoder. Furthermore,since the side information is introduced into the turbo loop by
the semantic auto-encoder, the final decoding output contains
more information and hence achieves a higher quality. The
contributions of this letter is summarized as follows.
• We propose a novel anti-interference technique, i.e., SemantIC, which can improve the recovered information
quality without extra cost of channel resources.
• We design an iterative decoding algorithm for imagetransmissions with SemantIC by constructing a turbo loop
concatenating the channel decoder and the semantic autoencoder, which alternately eliminates noise in the signal
domain and the semantic domain.
• Simulation results show that the system with SemantIC
has a better performance in terms of bit error rate (BER),
Euclidean distance (ED), and peak signal-to-noise ratio(PSNR), even with a very light neural network (42.1 kB).

ReceiverFig. 2. The structure of SemantIC system.

exploit side information in decoding without using a random
codebook. We find that neural networks can automaticallylearn the knowledge about the source by training, which isessentially the same as obtaining, storing and representing sideinformation. It is also noticed that the side information stored
in the form of neural network parameters is noncausal, which
satisfies the condition of the Wyner-Ziv theorem. Now, weneed to find the last piece of the puzzle, i.e., decoding withthe side information stored in neural networks.

B. Decoder Structure
To provide side information by neural networks for decoding, we need to design the decoder structure. Therefore, we
propose the structure of the SemantIC system in Fig. 2, where
the transmitter has a general structure without constraint. At
the receiver, the channel decoder FC (·) processes the a priori
log-likelihood ratio (LLR) to output the a posteriori LLR as
LLRp = FC (LLRa ).

(3)

The SemantIC decoder contains two key elements as follows.
1) Auto-Encoder: The auto-encoder is a neural network
having the function to extract information features and thenrecover information from features. By this method, the semantic noise is filtered by the neural network. In the SemantIC
system, the receiver utilizes a semantic auto-encoder, consistsof a pair of semantic encoder/decoder, to eliminate the interference and noise contained in the channel decoder output. This
process is referred to as denoising in the semantic domain.
2) Turbo Loop: To enhance the decoding gain, the structureof iterative decoding is necessary. Therefore, we connect theoutput of the semantic auto-encoder FS (·) back to the channel
decoder FC (·) as the a priori information.
LLRa = FS (LLRp ).

(4)

The structure of (3) and (4) composes a turbo loop for
eliminating noise and interference in the signal domain andthe semantic domain alternately. The semantic auto-encoder
performs semantic denoising from the input LLRp , and thenoutput LLRa to the channel decoder for denoising of signals
in the following iteration round of decoding.
Based on the semantic auto-encoder and the turbo loop,
the SemantIC system can further improve the reconstructed
information quality without any change outside the receiver.
The turbo loop reduces noise at the cost of introducing extra
latency in each round of iterations by the semantic autoencoder depending on the complexity of the neural networks.

3

Conv2D{16,(2, 2), 1}

Conv2D{16,(3, 3),, 2}

100

Conv2D{16,(3, 3), 2}
Semantic

Original
TranConv2D{3,(2, 2), 1}

TranConv2D{16,(3, 3), 2}

TranConv2D{16,(3, 3), 2}

Fig. 3. A simple structure of semantic encoder/decoder.

III. D ECODER I MPLEMENTATION D ESIGNIn order to evaluate the performance of the SemantIC system, we design a decoding algorithm for image transmission
simulations. Since the semantic encoder/decoder requires adataset for training, we select the CIFAR-10 dataset [12] forperformance evaluation. The CIFAR-10 dataset comprises acollection of 60,000 pictures, consisting of 50,000 training
images and an additional set of 10,000 images specificallydesigned for testing purposes. The image size is scaled to
3 × 96 × 96, i.e., 3 color channels, and 96 pixels in bothwidth and height.
As depicted in Fig. 3, we concatenate two-dimensional (2D)
convolutional layers (Conv2Ds) as the semantic encoder, and2D transpose convolutional layers (TranConv2Ds) are concatenated to be the semantic decoder. Let {nF , (wK , hK ), s}
specify the parameters of Conv2D and TranConv2D, withnF denoting the number of filters, (wK , hK ) denoting thekernel size, and s denoting the stride. The parameters are
configured to gradually extract the semantic features with arate approximate to 0.3, i.e., the output size is 16×23×23. As
will be shown in the simulation results, even with such simple
semantic encoder/decoder structure, the SemantIC decoder iseffective with respect to improving information quality.
Our model is implemented using Pytorch, with training
conducted on the NVIDIA GeForce RTX 3060 Ti by the Adam
optimizer [13]. Compared to the 163 MB size of the CIFAR10 dataset, the file for storing the semantic auto-encoder
parameter only has a size of 42.1 kB.
Low-density parity-check (LDPC) codes [14] are selected
as the channel code, with the codeword length being 900.
The number of parity-check equations including a certain bit
is set at 2, and the number of bits in the same parity-checkequation is set at 3. The maximum iteration round is set at 7.
The information of each image is divide into groups to matchthe LDPC codeword length for transmission. Since the LDPC
codes work for binary sequences, the image information in
the form of pixels is quantized into binary sequences before
inputting into the channel decoder. After channel decoding,the output should also be dequantized back to pixels.
An example source code for the implementation of the
proposed SemantIC system is presented in [15].
IV. S IMULATION R ESULTS
In this section, we conduct a comparative analysis between
the system with and without SemantIC by simulations. We
employ the BER, the ED, and the PSNR as evaluation metrics
to quantify the distortion between the original images X andthe reconstructed images X̂. Additionally, we set the learning
rate to 0.003, the batch size to 64, and the maximum training
epoch to 200.

10-1

10-2

10-3 0.05
0.04

10

-4

0.03

10-5 0.02
0

10-6
-5

0.5

-3

1

-1

1.5

1

3

5

7

9

(a) BER.
100

60

10-1

50

10-2

40

10-3

30

10-4

20

10-5

10

10-6
-5

0
-3

-1

1

3

5

7

9

(b) ED and PSNR.
Fig. 4. The impact of SNR on BER, ED, and PSNR.

1) Anti-Interference Performance: As shown in Fig. 4, we
demonstrate the relationship between the BER, ED, PSNR and
SNR of the proposed method in an additive white Gaussian
noise (AWGN) channel. It is easy to observe from the figurethat the SemantIC system outperforms the system without
SemantIC in terms of lower BER and ED as well as higher
PSNR at different SNR values. It is observed that the BER
performance gain seems less visible than ED. This is because
the loss function for training the semantic auto-encoder isdesigned based on ED instead of BER. Although the BER
performance seems to be very small, this performance gain
is obtained without extra channel resource cost, and the
SemantIC neural network has a very simple structure and witha very small size of parameters. The system is expected to havea better anti-interference performance, if the neural network
is sophisticatedly designed, e.g., introducing the attention
mechanism to enlarge receptive field, and/or utilizing the
diffusion model to predict the noise.
Particularly, SemantIC performs better in low SNR scenarios. This can be attributed to the fact that higher-level semantic

4features are more resilient to channel noise and fluctuations
compared to lower-level semantic features. By extracting highlevel semantic features through the semantic encoder/decoder,
SemantIC aids the channel decoder in recovering lost semantic
information during the physical channel transmission process,thereby enhancing system robustness. Another factor for lower
gain by SemantIC is that LDPC codes have a strong capability
for enhancing decoding gain. Hence, when SNR is large,
LDPC codes have already significantly eliminated noise andinterference. Especially when SNR≥ 8 dB, the BER bychannel decoding only has reduced to 0 at the first round of
iteration. Therefore, no gain can be obtained for SNR≥ 8 dB.
2) Iterative Gain: Fig. 5 demonstrates the performancegain achieved through iterative decoding without extra channel
resource cost. It is evident that as the number of iterations increases, SemantIC consistently outperforms the method
without SemantIC, and the performance gap gradually becomes significant. This is because during the iterative decoding
process, the semantic auto-encoder can filter out noise andinterference to a certain extent and provide additional semantic
information through multiple interactions. This observation
verifies the effectiveness of the turbo loop in the SemantIC
decoder. Consequently, it effectively reduces error propagationand enhances overall decoding performance. Moreover, theperformance converges fast at the fourth round of iteration.
This is due to the strong error correcting capability of LDPC
codes. Hence, practical systems can adopt early stopping
strategy to balance computational complexity with latency.
Interestingly, as SNR becomes larger, the BER performancegain increases, while the performance gain of the ED and
PSNR decreases.
3) Visual Effect: For the comparison of visual effect, we
utilize the original image presented in Fig. 6(a) as an example,
and the semantic image is obtained by inputting the original
image to the semantic encoder concatenated with the semantic
decoder. Fig. 6 visualizes the results of image reconstruction
with and without SemantIC, subjectively verifying the fidelityof semantic information. In Fig. 6, Iter, IC/noIC, Sem, ED
represent the iteration rounds, the image with/without interference cancellation, the image reconstructed by the semantic
decoder, and ED.
In Fig. 6(b), the method without SemantIC fails to clearly
depict the shape of the car window, whereas our approachsuccessfully recovers some details of the window from theblurred original image. In Fig. 6(c), SemantIC enhances boththe visual quality and semantic information of the image.
Traditional algorithms often focus solely on visual quality,
neglecting the preservation of semantic information. However,SemantIC simultaneously considers both image quality andsemantic information, rendering objects and scenes in theimage more realistic, recognizable, and comprehensible. If wezoom-in and compare the image of IC/noIC with Iter= 7,
we can see that hot pixels within the yellow circle is lessfor the IC case. The semantic image also gradually becomes
clearer as the iteration round increases. In Fig. 6(d), as channel
conditions improve, the reconstructed image becomes highly
realistic, making it difficult to discern from the original image.
Although the image quality is already very high without

100

10-1

10-2

10-3

10-4

10-5
1

2

3

4

5

6

7

5

6

7

5

6

7

(a) BER.
100

10-1

10-2

10-3

10-4

1

2

3

4

(b) ED.
50
45
40
35
30
25
20
15
10
5
0
1

2

3

4

(c) PSNR.
Fig. 5. Convergence behaviour for BER, ED, and PSNR.

5

see that one hot pixel pointed by the yellow arrow is corrected.

Original

Semantic(a) Lossless images.

Iter

0

1

4

7

0.298640

0.289203

0.286868

0.290728

0.298640

0.289186

0.222788

0.218415

noIC

ED

IC

ED

V. C ONCLUSION
This letter has proposed a breakthrough technique, referred
to as SemantIC, for anti-interference towards the 6G wirelesscommunications. The SemantIC decoder contains a turbo loop
consists of concatenated channel decoder and semantic autoencoder, which eliminate the noise and interference alternately
in signal domain and semantic domain. We have also designed
a decoding algorithm and implemented the SemantIC system
for image transmissions. The simulation results indicate that
the SemantIC system can enhance the reconstructed information quality without extra cost of channel resources, although
we only utilize a very simple neural network structure. Even
with a very low SNR, the image recovered by SemantIC showsa clearer object shape than that without SemantIC.
R EFERENCES

Sem(b) SNR= −5 dB.

Iter

0

1

4

7

0.113687

0.087357

0.063264

0.065824

0.113687

0.087421

0.055058

0.052216

noIC

ED

IC

ED

Sem

(c) SNR= 0 dB.

Iter

0

1

4

7

0.003909

0.001096

0.0003120.000401

0.003909

0.001096

0.000294

0.000335

noIC

ED

IC

ED

Sem

(d) SNR= 5 dB.
Fig. 6. Comparisons of the reconstructed image quality.

SemantIC, our approach can still further eliminate some noise.
For example, if we zoom-in the image with Iter= 7, we can[1] A. Wyner and J. Ziv, “The rate-distortion function for source codingwith side information at the decoder,” IEEE Transactions on information
Theory, vol. 22, no. 1, pp. 1–10, Jan. 1976.
[2] W. Yang, H. Du, Z. Q. Liew, W. Y. B. Lim, Z. Xiong, D. Niyato, X. Chi,
X. Shen, and C. Miao, “Semantic communications for future internet:
Fundamentals, applications, and challenges,” IEEE Communications
Surveys & Tutorials, vol. 25, no. 1, pp. 213–250, First quarter 2023.
[3] C.-X. Wang, X. You, X. Gao, X. Zhu, Z. Li, C. Zhang, H. Wang,
Y. Huang, Y. Chen, H. Haas, J. S. Thompson, E. G. Larsson, M. D.
Renzo, W. Tong, P. Zhu, X. Shen, H. V. Poor, and L. Hanzo, “On
the road to 6G: Visions, requirements, key technologies, and testbeds,”
IEEE Communications Surveys & Tutorials, vol. 25, no. 2, pp. 905–974,
Second quarter 2023.
[4] H. Feng, Y. Yang, and Z. Han, “SCAI: Scalable AI generative content forenhanced semantic communication,” in IEEE International Conferenceon Acoustics, Speech, and Signal Processing (ICASSP), Seoul, Korea,
Apr. 2024, invited.
[5] M. Xu, H. Du, D. Niyato, J. Kang, Z. Xiong, X. S. Shen, Z. Han,
and H. V. Poor, “Unleashing the power of edge-cloud generative AI in
mobile networks: A survey of AIGC services,” IEEE Communications
Surveys & Tutorials, to appear.
[6] M. Xu, D. Niyato, H. Zhang, J. Kang, Z. Xiong, S. Mao, and Z. Han,
“Sparks of GPTs in edge intelligence for metaverse: Caching and inference for mobile AIGC services,” IEEE Vehicular Technology Magazine,
special issue on Metaverse for Connected and Automated Vehicles andIntelligent Transportation Systems, to appear.
[7] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “Highresolution image synthesis with latent diffusion models,” in IEEE/CVF
Conference on Computer Vision and Pattern Recognition (CVPR), New
Orleans, LA, Jun. 2022, pp. 10 674–10 685.
[8] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol,“Stacked denoising autoencoders: Learning useful representations in a
deep network with a local denoising criterion,” Journal of Machine
Learning Research, vol. 11, p. 3371–3408, Dec. 2010.
[9] X. Luo, B. Yin, Z. Chen, B. Xia, and J. Wang, “Autoencoder-based
semantic communication systems with relay channels,” in IEEE International Conference on Communications Workshops (ICC Workshops),
Seoul, Korea, Jul. 2022, pp. 711–716.
[10] B. Du, W. Xiong, J. Wu, L. Zhang, L. Zhang, and D. Tao, “Stacked
convolutional denoising auto-encoders for feature representation,” IEEE
Transactions on Cybernetics, vol. 47, no. 4, pp. 1017–1027, Apr. 2017.
[11] C. Berrou and A. Glavieux, “Near optimum error correcting coding
and decoding: Turbo-codes,” IEEE Transactions on Communications,vol. 44, no. 10, pp. 1261–1271, Oct. 1996.
[12] A. Krizhevsky, “Learning multiple layers of features fromtiny images,” Toronto, ON, Canada, 2009. [Online]. Available:
https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf[13] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
arXiv preprint arXiv:1412.6980, 2014.
[14] R. Gallager, “Low-density parity-check codes,” IRE Transactions on
Information Theory, vol. 8, no. 1, pp. 21–28, Jan. 1962.
[15] “SemantIC,”
GitHub,Oct.
2023.
[Online].
Available:
https://github.com/linwest/SemantIC