# 6G Cognitive Information Theory: A Mailbox Perspective
## 11 A Deep Learning-Assisted Cognitive Information Communication System
### Cognitive Communication System Architecture  
We design the architecture of a cognitive communication system assisted by deep learning, which includes encoder, channel, and decoder. Thus, the encoder cognizes information and the decoder decodes the information that has been cognized. Specifically, a cognitive communication system for image transmission based on U-Net architecture has been designed. That is, the information transmitted by this system are images, and encoding and decoding are conducted to the images.  
Figure 14: Case study of digital twin.  
Encoder:The encoding part consists of a deep residual module and maximum pooling layer. The deep residual module includes input, batch normalization (BN) layer, ReLU activation function, two 3 \(\times\) 3 convolution operations, identity mapping unit, and output. By adding an identity mapping unit, the input of the deep residual module is directly transmitted to the network layer in the back. This is to effectively reduce the influence of gradient degradation in the process of network training and to promote the transmission of information to solve the problem of information loss in traditional networks to a certain extent. The maximum pooling layer is used for the down-sampling operation of images, with a step size of 2. Its purpose is to reduce the size of the feature map as the number of channels in the feature map increases with the decrease of the size. Specifically, the number of channels for an input image is 32. Before entering the encoding part, the image will go through a 3 \(\times\) 3 convolution layer, BN layer and ReLu activation function, and the number of its channels will become 64. In the encoding part, down-sampling is conducted three times, and the number of channels is 64, 128 and 256, respectively. The size of the feature map of the latter submodule is equal to 1/2 of that of the former, that is, the size of the feature map is 160 \(\times\) 160, 80 \(\times\) 80, 40 \(\times\) 40, 20 \(\times\) 20, and 10 \(\times\) 10, respectively.  
Channel:Channel connects the encoder and decoder. In order to achieve uniform training of cognitive systems, we define the channel as salt-and-pepper noise and uniform training is carried out. In other words, the salt-and-pepper noise is added to the encoded image after output.  
Decoder:The decoder part contains the same four deep residual modules and up-sampling operations. Up-sampling is conducted to the feature map from the channel part. After each up-sampling, the number of channels in the feature map is halved, and the size of the feature map is doubled. Finally, the feature map with the same size as the original image is obtained. In the decoder part, up-sampling is conducted four times and the number of channels is 256, 128, 64, and 32 respectively. The classifier is composed of 1 \(\times\) 1 convolution layer and Sigmod activation function. The convolution operation of 1 \(\times\) 1 is conducted on the feature maps from the decoder, to reduce the number of feature maps. The Sigmod activation function is used to calculate the category of each pixel in the feature map and finally the multi-channel feature map is mapped to the corresponding categories.  
Loss function:In this experiment, the loss function we used is cross-entropy loss function, which is defined as follows:  
\[L=\sum_{i=1}^{M}\sum_{j=1}^{N}-[f_{ij}\log(p_{ij}+(1-f_{ij})\log(1-p_{ij}))], \tag{11}\]  
where \(M\) is the set of pixels in the segmentation image, \(N\) is the set of pixels in the labelling segmentation tags, \(f_{ij}\) is the true category of pixel \(i\) in the segmentation image and pixel \(j\) in labelling segmentation tags, \(p_{ij}\) is the predicted value of pixel \(i\) in the segmentation image and pixel \(j\) in the labelling segmentation tags. In the process of network training, if cross-entropy loss function is used for optimization, then the problem of gradient disappearance in the network can be solved effectively, and the network can run stably.