# Communication Audio and Video Coding and Decoding Methods in Wireless Cooperative Communication Network

Authors: Shenghui Li; Wenmin Wang
Journal: Mobile Information Systems
 (2022)

---

## Abstract

Wireless communication is ubiquitous in people’s daily life. As people’s requirements for wireless communication continue to increase, how to improve system performance has become a core issue that communication researchers pay attention to. Wireless cooperative communication is considered to be a key technology in the future communication system because it can provide better communication quality, higher system capacity, wider communication coverage, richer services, and high spectrum utilization efficiency. The proposed network coding can effectively eliminate the transmission bottleneck problem faced by wireless cooperative communication. The development of audio and video communication coding technology has increased the demand for audio and video communication services. The goal of audio and video coding is shifting from traditional storage applications to network transmission. Therefore, the audio and video coding technology for network transmission is being affected by academia and industry. The purpose of this paper is to study the communication audio and video codec methods in wireless cooperative communication networks and to understand the development status of audio and video codec by reading a large number of related documents. On the basis of the problem, relevant experiments are designed to conduct in-depth research. Experimental results show that this scheme can simplify the implementation of scalable coding, improve the coding efficiency of the enhancement layer, and improve the quality of the reconstructed video when the audio and video streams are truncated.


---

## Body

## 1. Introduction

Since the beginning of this century, wireless communication technology has entered a stage of rapid development, and not only related research results have emerged one after another, but the application and update frequency of new technologies has also become faster and faster, and more and more services can be provided [1, 2].People’s requirements for wireless communication systems are getting higher and higher, such as requiring clearer call effects, wider communication coverage, and more diverse communication services [3, 4]. Because of these requirements for wireless communication systems, the next-generation wireless communication network needs to have better accuracy, inclusiveness, and scalability and become able to provide better communication services and support a variety of wireless communication services and has the ability to accommodate communication subnetworks of different structural forms [5, 6].Advances in wireless communication technology have brought faster and better network access services. However, the development of technology matches the needs of users for faster, cheaper, and more stable wireless communication [7, 8]. In order to be able to meet the needs of future wireless communication, we need to improve the effectiveness of wireless communication in terms of power, delay, bandwidth, etc., so that it can provide a higher rate and better quality of multimedia services [9]. The relay node in the cooperative communication system uses network coding technology. The relay node not only forwards information but also performs corresponding signal processing on the information. Through network coding, the capacity and frequency band utilization rate of the wireless communication system can be improved [10].Through reading domestic and foreign literature, this article understands the current development status of audio coding, video coding, and general audio and video coding and deeply understands and masters the basic ideas and coding and decoding principles of existing audio and video coding, as well as the most cutting-edge signal analysis/synthesis algorithm.


## 2. Research on Communication Audio and Video Coding and Decoding Methods in Wireless Cooperative Communication Networks

### 2.1. Wireless Cooperative Communication System Model

In a wireless cooperative communication system, the relationship between each user is not independent of each other; that is, under general conditions, users only send and receive the information they need [11]. When the system determines that collaborative communication is needed, the user sends his own information and helps the partner user to forward his information (when the collaborative user does not send information but forwards other user information, it can be regarded as a relay node), as shown in Figure 1. The difference between direct transmission and cooperative transmission (both forms of cooperative transmission) is shown in Figure 1.Figure 1
Wireless cooperative communication system model.According to the above, cooperative communication is divided into two forms by determining whether cooperative users transmit their own information: the traditional cooperative transmission form shown in the upper left of Figure1 and the relay selection cooperative communication form shown in the lower right of Figure 1, which will be described later. Research will be conducted based on these two forms of communication. The characteristic of traditional cooperative communication is that an independent channel is formed between the cooperative user and the base station (destination node). This channel is an additional supplement to the user-base station channel and uses different processing methods for the information transmitted through the channel. A variety of collaborative communication protocols have been produced.


### 2.2. Basic Principles of Audio and Video Coding

Audio and video coding is the most important key technology for applications such as the compression, storage, and transmission of massive data of audio and video images. As a spatially intuitive and temporally continuous signal, audio and video data have the following redundancy: spatial redundancy, temporal redundancy, coding redundancy, and knowledge redundancy. Spatial redundancy refers to the spatial correlation between adjacent pixels in an image in an audio and video sequence. Temporal redundancy refers to the temporal correlation between the previous and subsequent frame images in the audio and video sequence; visual redundancy is due to the insensitive perception of certain spatial frequencies by the human eye; statistical redundancy is expressed as the repetition of certain probability characteristics of coded symbols.The source model occupies a core position in audio and video coding, and the composition and efficiency of the audio and video encoder depend on the characteristics of the source model as shown in Figure2:Figure 2
Source coding principle.


### 2.3. Problems in Audio and Video Communication

(1)
The content and forms of video information in audio and video communications are extremely rich. Different types of audio and video services and application environments have different requirements for the quality, delay, and interactivity of audio and video transmission. This requires audio and video coding to have good adaptability to different audio and video services(2)
The Internet and various wireless networks in the audio and video communication environment have problems such as heterogeneity, volatility, and unreliability. Heterogeneity means that the actual transmission throughput, presentation, packet loss rate, and congestion control strategy are different when different communication subnets transmit data. Volatility means that the bandwidth of the network is different under different load conditions, especially in the wireless network; the fluctuation of the bandwidth is more significant. Unreliability refers to errors in data transmission over the network(3)
At the receiving end, users use receiving and playing devices with different resolutions and processing capabilities to create diversity in audio and video applications. From high-resolution high-definition TVs, to ordinary PCs, to handheld computers and mobile phones, their terminal capabilities, network bandwidth conditions, and users’ requirements for video quality vary(4)
With the popularity of wireless embedded terminal equipment, audio and video services follow people to all corners of the world, with large data volume, high complexity, and strong real-time requirements for audio and video coding and decoding algorithms optimized on embedded devices. Realization has become the primary task for the promotion and application of various audio and video coding standards and their services


## 2.1. Wireless Cooperative Communication System Model

In a wireless cooperative communication system, the relationship between each user is not independent of each other; that is, under general conditions, users only send and receive the information they need [11]. When the system determines that collaborative communication is needed, the user sends his own information and helps the partner user to forward his information (when the collaborative user does not send information but forwards other user information, it can be regarded as a relay node), as shown in Figure 1. The difference between direct transmission and cooperative transmission (both forms of cooperative transmission) is shown in Figure 1.Figure 1
Wireless cooperative communication system model.According to the above, cooperative communication is divided into two forms by determining whether cooperative users transmit their own information: the traditional cooperative transmission form shown in the upper left of Figure1 and the relay selection cooperative communication form shown in the lower right of Figure 1, which will be described later. Research will be conducted based on these two forms of communication. The characteristic of traditional cooperative communication is that an independent channel is formed between the cooperative user and the base station (destination node). This channel is an additional supplement to the user-base station channel and uses different processing methods for the information transmitted through the channel. A variety of collaborative communication protocols have been produced.


## 2.2. Basic Principles of Audio and Video Coding

Audio and video coding is the most important key technology for applications such as the compression, storage, and transmission of massive data of audio and video images. As a spatially intuitive and temporally continuous signal, audio and video data have the following redundancy: spatial redundancy, temporal redundancy, coding redundancy, and knowledge redundancy. Spatial redundancy refers to the spatial correlation between adjacent pixels in an image in an audio and video sequence. Temporal redundancy refers to the temporal correlation between the previous and subsequent frame images in the audio and video sequence; visual redundancy is due to the insensitive perception of certain spatial frequencies by the human eye; statistical redundancy is expressed as the repetition of certain probability characteristics of coded symbols.The source model occupies a core position in audio and video coding, and the composition and efficiency of the audio and video encoder depend on the characteristics of the source model as shown in Figure2:Figure 2
Source coding principle.


## 2.3. Problems in Audio and Video Communication

(1)
The content and forms of video information in audio and video communications are extremely rich. Different types of audio and video services and application environments have different requirements for the quality, delay, and interactivity of audio and video transmission. This requires audio and video coding to have good adaptability to different audio and video services(2)
The Internet and various wireless networks in the audio and video communication environment have problems such as heterogeneity, volatility, and unreliability. Heterogeneity means that the actual transmission throughput, presentation, packet loss rate, and congestion control strategy are different when different communication subnets transmit data. Volatility means that the bandwidth of the network is different under different load conditions, especially in the wireless network; the fluctuation of the bandwidth is more significant. Unreliability refers to errors in data transmission over the network(3)
At the receiving end, users use receiving and playing devices with different resolutions and processing capabilities to create diversity in audio and video applications. From high-resolution high-definition TVs, to ordinary PCs, to handheld computers and mobile phones, their terminal capabilities, network bandwidth conditions, and users’ requirements for video quality vary(4)
With the popularity of wireless embedded terminal equipment, audio and video services follow people to all corners of the world, with large data volume, high complexity, and strong real-time requirements for audio and video coding and decoding algorithms optimized on embedded devices. Realization has become the primary task for the promotion and application of various audio and video coding standards and their services


## 3. Experimental Research on Communication Audio and Video Coding and Decoding Methods in Wireless Cooperative Communication Networks

### 3.1. Research Purpose

In recent years, with the development of Internet technology and mobile communication technology, people’s demand for video communication services is increasing, such as video conferencing, video telephony, and video surveillance. These video communication applications greatly enrich and facilitate people’s lives, and video communication will inevitably become a development hotspot of Internet and mobile communication services in the future. However, the video information has the characteristics of large data volume and rich content, which seriously affects the quality of video communication, thus restricting the development of video communication services to a certain extent. Therefore, this article makes some discussions on the problems of audio and video communication.


### 3.2. Algorithm Description Based on Communication Audio and Video Codec in Wireless Cooperative Communication Network

The goal of node selection is to minimize the probability of network interruption, namely,(1)min∅j∈spoutAmong them,∅j expresses the jth possible selection method.For such an optimal problem, a global search algorithm can be used to solve it. First, assume that an unselected node set isΩ.Step 1.
InitializeΩ.Step 2.
Select the node with the lowest signal-to-noise ratio inΩ to the destination node:
(2)S1=argmini∈Ωτi,D.Step 3.
Choose a nodes2 in Ω to minimize the interruption probability of node pair s1,s2, namely,
(3)s2=argmaxi∈Ωgains1,i.Table1 shows the comparison of the required time complexity of various selection algorithms in different network nodes. It can be seen from the table that the complexity of the algorithm for minimizing the probability of interruption of nodes is proportional to M2, while the complexity of the random selection algorithm is proportional to M, and the complexity of the exhaustive algorithm is the largest, which is proportional to N∏i=1M2i−1.Table 1
Time comparison of various selection algorithms.
The network contains the number of nodesRandom selection algorithmAlgorithm for minimizing the outage probability of node pairsExhaustive algorithm40.0023680.0045320.00756860.0032450.0097650.05717280.0042900.0171590.470382


## 3.1. Research Purpose

In recent years, with the development of Internet technology and mobile communication technology, people’s demand for video communication services is increasing, such as video conferencing, video telephony, and video surveillance. These video communication applications greatly enrich and facilitate people’s lives, and video communication will inevitably become a development hotspot of Internet and mobile communication services in the future. However, the video information has the characteristics of large data volume and rich content, which seriously affects the quality of video communication, thus restricting the development of video communication services to a certain extent. Therefore, this article makes some discussions on the problems of audio and video communication.


## 3.2. Algorithm Description Based on Communication Audio and Video Codec in Wireless Cooperative Communication Network

The goal of node selection is to minimize the probability of network interruption, namely,(1)min∅j∈spoutAmong them,∅j expresses the jth possible selection method.For such an optimal problem, a global search algorithm can be used to solve it. First, assume that an unselected node set isΩ.Step 1.
InitializeΩ.Step 2.
Select the node with the lowest signal-to-noise ratio inΩ to the destination node:
(2)S1=argmini∈Ωτi,D.Step 3.
Choose a nodes2 in Ω to minimize the interruption probability of node pair s1,s2, namely,
(3)s2=argmaxi∈Ωgains1,i.Table1 shows the comparison of the required time complexity of various selection algorithms in different network nodes. It can be seen from the table that the complexity of the algorithm for minimizing the probability of interruption of nodes is proportional to M2, while the complexity of the random selection algorithm is proportional to M, and the complexity of the exhaustive algorithm is the largest, which is proportional to N∏i=1M2i−1.Table 1
Time comparison of various selection algorithms.
The network contains the number of nodesRandom selection algorithmAlgorithm for minimizing the outage probability of node pairsExhaustive algorithm40.0023680.0045320.00756860.0032450.0097650.05717280.0042900.0171590.470382


## 4. Experimental Analysis of Communication Audio and Video Coding and Decoding Methods in Wireless Cooperative Communication Networks

### 4.1. Moving Object Segmentation Based on H.264 Compressed Domain

The moving object segmentation process based on H.264 compressed domain is shown in Figure3. First, extract the motion feature information from the H.264 compressed audio and video stream, that is, the motion vector and its corresponding block segmentation size, and then perform preprocessing such as normalization, spatiotemporal joint processing, and global motion compensation on the original motion information. Get reliable and significant object motion information, and mark the blocks that are inconsistent with the global motion as the moving object detection area; then, use the mean shift clustering algorithm to find the spatiotemporal-consistent area in the detection area; finally, perform background separation and morphological operations to extract precise moving objects.Figure 3
H.264 compressed domain segmentation flowchart.The experiment adopts the standard test sequence in QCIF (176×144) format and uses the JM8.6 H.264 encoder to encode the test sequence as the experimental H.264 compressed video stream. The encoder configuration is as follows: baseline profile; GOP structure is IPPP and length is 50; reference frame number is 5; motion estimation search range is [-32, 32]; and quantization parameter takes 30. In the experiment, Vh is 2 h, and 3×3 structural elements are used for opening and closing morphological operations. The algorithm is implemented with c and tested on a PC with P4 2.4 G 512 M memory.Table2 shows the average σ value of each frame in the experiment, the number of 4×4 blocks contained in the moving object, the number of iterations of the mean shift, and the execution time.Table 2
The average value isσ per frame, the number of blocks in the object, the number of mean shift iterations, and the execution time.
SequenceThe average value isσ per frameThe number of blocks in the objectThe number of mean shift iterationsThe execution timeHall0.5841.2085.706.42 msCoastguard1.74113.64312.5113.53 msIt can be seen from Table2 that the average execution time required for each frame is 6.42~13.53 ms, which is obtained without any optimization of the code, so it can meet the time requirements of the real-time system. It is easy to find that the average number of mean shift iterations per frame and execution time are related to the size of the moving object area (the number of 4×4 blocks contained in the moving object) and the intensity of image activity (average σ value). The number of iterations and execution time of the more violent coastguard sequence are larger than those of the Hall sequence with a small moving object area and small motion.In summary, the compressed domain segmentation algorithm can quickly and accurately segment foreground moving objects for video sequences with static and moving backgrounds.


### 4.2. SNR Scalable Coding Based on Moving Objects

The H.264 compressed domain moving object segmentation algorithm proposed in this paper is implemented on the base layer of JSVM1.0, and the priority block coding is implemented in the SNR scalable coding. In the experiment, the foremanCIF sequence is used for testing, and the rate-distortion performance and image quality of the moving object area, background area, and whole image are compared through JSVM1.0. The encoder adopts the GOP structure of IPPP and the encoding method of CAVLC. In the experiment, the block coefficient coding of the region where the moving object is located is 5 scan cycles earlier than that of the background region.Figure4 shows the PSNR of the moving object area, the background area, and the overall image as a function of the bit rate. Obviously, at the initial stage of bit rate increase, since the coding of block coefficients in the area where the moving object is located has priority over the background area, the bit rate is concentrated in the area where the moving object is located, so the PSNR of the moving object can be significantly improved with the increase of the bit rate. The PSNR of the background area remains unchanged.Figure 4
Sports objects, backgrounds, and overall image variation curve with bit rate.Table3 shows the PSNR comparison between the SNR scalable coding based on moving objects and JSVM1.0 in different regions of the 20th frame of the Foreman sequence at a cut-off bit rate of 384 kbps. Obviously, the PSNR of the former moving object is much higher than that of the background area and the entire image, and 2.19 dB higher than JSVM1.0, while the PSNR of the background area and the entire image is 1.33 dB and 0.37 dB lower than JSVM1.0, respectively.Table 3
PSNR comparison of moving object, background and whole image at 384 kbps cut-off bit rate.
AreaSNR scalable coding based on moving objectsJSVM1.0Moving object33.7531.56Background29.1630.49Whole image30.3730.74


## 4.1. Moving Object Segmentation Based on H.264 Compressed Domain

The moving object segmentation process based on H.264 compressed domain is shown in Figure3. First, extract the motion feature information from the H.264 compressed audio and video stream, that is, the motion vector and its corresponding block segmentation size, and then perform preprocessing such as normalization, spatiotemporal joint processing, and global motion compensation on the original motion information. Get reliable and significant object motion information, and mark the blocks that are inconsistent with the global motion as the moving object detection area; then, use the mean shift clustering algorithm to find the spatiotemporal-consistent area in the detection area; finally, perform background separation and morphological operations to extract precise moving objects.Figure 3
H.264 compressed domain segmentation flowchart.The experiment adopts the standard test sequence in QCIF (176×144) format and uses the JM8.6 H.264 encoder to encode the test sequence as the experimental H.264 compressed video stream. The encoder configuration is as follows: baseline profile; GOP structure is IPPP and length is 50; reference frame number is 5; motion estimation search range is [-32, 32]; and quantization parameter takes 30. In the experiment, Vh is 2 h, and 3×3 structural elements are used for opening and closing morphological operations. The algorithm is implemented with c and tested on a PC with P4 2.4 G 512 M memory.Table2 shows the average σ value of each frame in the experiment, the number of 4×4 blocks contained in the moving object, the number of iterations of the mean shift, and the execution time.Table 2
The average value isσ per frame, the number of blocks in the object, the number of mean shift iterations, and the execution time.
SequenceThe average value isσ per frameThe number of blocks in the objectThe number of mean shift iterationsThe execution timeHall0.5841.2085.706.42 msCoastguard1.74113.64312.5113.53 msIt can be seen from Table2 that the average execution time required for each frame is 6.42~13.53 ms, which is obtained without any optimization of the code, so it can meet the time requirements of the real-time system. It is easy to find that the average number of mean shift iterations per frame and execution time are related to the size of the moving object area (the number of 4×4 blocks contained in the moving object) and the intensity of image activity (average σ value). The number of iterations and execution time of the more violent coastguard sequence are larger than those of the Hall sequence with a small moving object area and small motion.In summary, the compressed domain segmentation algorithm can quickly and accurately segment foreground moving objects for video sequences with static and moving backgrounds.


## 4.2. SNR Scalable Coding Based on Moving Objects

The H.264 compressed domain moving object segmentation algorithm proposed in this paper is implemented on the base layer of JSVM1.0, and the priority block coding is implemented in the SNR scalable coding. In the experiment, the foremanCIF sequence is used for testing, and the rate-distortion performance and image quality of the moving object area, background area, and whole image are compared through JSVM1.0. The encoder adopts the GOP structure of IPPP and the encoding method of CAVLC. In the experiment, the block coefficient coding of the region where the moving object is located is 5 scan cycles earlier than that of the background region.Figure4 shows the PSNR of the moving object area, the background area, and the overall image as a function of the bit rate. Obviously, at the initial stage of bit rate increase, since the coding of block coefficients in the area where the moving object is located has priority over the background area, the bit rate is concentrated in the area where the moving object is located, so the PSNR of the moving object can be significantly improved with the increase of the bit rate. The PSNR of the background area remains unchanged.Figure 4
Sports objects, backgrounds, and overall image variation curve with bit rate.Table3 shows the PSNR comparison between the SNR scalable coding based on moving objects and JSVM1.0 in different regions of the 20th frame of the Foreman sequence at a cut-off bit rate of 384 kbps. Obviously, the PSNR of the former moving object is much higher than that of the background area and the entire image, and 2.19 dB higher than JSVM1.0, while the PSNR of the background area and the entire image is 1.33 dB and 0.37 dB lower than JSVM1.0, respectively.Table 3
PSNR comparison of moving object, background and whole image at 384 kbps cut-off bit rate.
AreaSNR scalable coding based on moving objectsJSVM1.0Moving object33.7531.56Background29.1630.49Whole image30.3730.74


## 5. Conclusions

With the development of network, computer, and communication technology, audio and video communication services are expanding continuously. As an important subfield of multimedia technology research, audio and video coding and decoding put forward new requirements for audio and video coding and audio and video transmission. In addition to the need for better video quality and higher compression ratio, higher requirements are put forward for the reliability of video coding in various video communication environments.



---

*Source: 1587662-2022-04-22.xml*