# News Video Classification Model Based on ResNet-2 and Transfer Learning

Authors: Yiping Gao
Journal: Security and Communication Networks
 (2021)

---

## Abstract

A large amount of useful information is included in the news video, and how to classify the news video information has become an important research topic in the field of multimedia technology. News videos are enormously informative, and employing manual classification methods is too time-consuming and vulnerable to subjective judgment. Therefore, developing an automated news video analysis and retrieval method becomes one of the most important research contents in the current multimedia information system. Therefore, this paper proposes a news video classification model based on ResNet-2 and transfer learning. First, a model-based transfer method was adopted to transfer the commonality knowledge of the pretrained model of the Inception-ResNet-v2 network on ImageNet, and a news video classification model was constructed. Then, a momentum update rule is introduced on the basis of the Adam algorithm, and an improved gradient descent method is proposed in order to obtain an optimal solution of the local minima of the function in the learning process. The experimental results show that the improved Adam algorithm can iteratively update the network weights through the adaptive learning rate to reach the fastest convergence. Compared with other convolutional neural network models, the modified Inception-ResNet-v2 network model achieves 91.47% classification accuracy for common news video datasets.


---

## Body

## 1. Introduction

Today, video media plays an increasingly prominent role in enriching people’s lives, education, and entertainment. Video is a kind of media with rich content, which can provide more vivid information than words, sounds, and images [1–5]. News is a kind of video, which is an important way for people to understand the society and is closely related to people’s life. Now, there are a lot of news programs, and the amount of information is also very large. Therefore, it becomes an important demand that people can easily find relevant content of their own interest in a large number of news programs.Content-based retrieval refers to retrieval according to the semantic features or audio-visual features of media objects [6–8]. Semantic features refer to the content information of video segments, while audio-visual features refer to some physical features that can be directly obtained from sounds and images, such as colors, textures, and shapes in images, motions of objects and lenses in videos, and tonal loudness and timbre in sounds [9–12]. This is a very practical technology with a wide range of applications. Now, content-based video retrieval has achieved some results. However, the research of content-based video retrieval for news is not much.Deep learning abandons the complex operation process of traditional algorithms, and convolution neural network (CNN) [13, 14] has achieved great success in image recognition and image segmentation at first. With the continuous breakthroughs in typical network structures, deep neural networks such as recurrent neural network (RNN) [15], deep belief network (DBN) [16], and generative adversarial networks (GAN) [17] have emerged, which can better enhance the feature extraction ability of models by supervised learning [18, 19]. Therefore, based on the theory and technology of content-based video retrieval, this paper focuses on the related technology and implementation of news video retrieval based on deep learning.


## 2. Literature Review

Traditional video classification and recognition methods generally use artificially designed features to model, extract the size, shape, color, texture, and other information features of video key frames, and fuse one or more features to build a classifier to realize automatic video classification and recognition. For example, Arivazhagan et al. [20] proposed to apply completely local binary pattern (CLBP) as texture feature to image recognition. In this method, color and texture features are fused, and the nearest neighbor classifier is used to accomplish the classification task, and the factors of illumination intensity change are considered. Zhou et al. [21] extracted three features of color, shape, and size, selected K-nearest neighbor (KNN) classifier to classify and recognize various images, and achieved good recognition accuracy, with the recognition accuracy as high as 90%. However, all these methods require manual design of image features. Although they are excellent in accuracy and robustness, manual design of feature extractors usually requires a lot of work. Convolutional neural network omits the process of feature extraction by manual design and fuses attention mechanism to extract geometric transformation information in images, so as to improve the accuracy of image recognition and the stability of the network.At present, the research on news video recognition using deep convolutional neural network is still limited mainly because there is no common news video dataset with large enough data amount and high enough quality, so it is difficult to train an excellent classification and recognition model. Therefore, in this paper, Inception-ResNet-v2, which is trained in ImageNet large dataset [22–24] and has better performance than Inception-ResNet-v1, is used as the pretraining model, and the model-based transfer learning method [25] is used for the experiment. The main innovations of this paper are as follows: (1) the transfer learning technology is applied to the news video classification model based on the concept-ResNet-v2 network, which effectively improves the overfitting and makes the model have better generalization ability; (2) an improved Adam gradient descent method is proposed to improve the convergence rate of the model.


## 3. Classification Model Based on Inception-ResNet-v2 Network and Transfer Learning

### 3.1. Deep Neural Network

Convolution neural network is the most widely used deep learning model in the field of computer vision. Its earliest theoretical model is the neurocognitive machine proposed by Japanese scholar Fukushima [26]. The neural cognitive machine has a good recognition ability even when the target object is slightly distorted. On the basis of neurocognitive machine, a multilayer feedforward neural network model LeNet-5 appeared and was successfully applied in handwritten character recognition. The model mainly includes input layer, convolution layer, pooling layer, full connection layer, and output layer, which laid the foundation for the later convolution neural network structure. In 2012, the AlexNet model won the ILSVRC competition, which made convolutional neural network become a research hotspot, and then, more excellent convolutional neural networks were proposed. A typical convolutional neural network model architecture is shown in Figure 1.Figure 1
Typical convolutional neural network model architecture.Convolution neural network extracts features by convolution operation on local “receptive field,” and it is mainly used in image processing. CNN is a kind of feedforward neural network with deep structure. Firstly, the image is input at the input layer and then calculated by convolution layer, pooling layer, and nonlinear activation function, and the semantic information of high-level abstraction is gradually extracted from the image. This is the “feedforward operation” of convolutional neural network. Finally, for the fully connected layer, all the features extracted from the previous network are connected for prediction, and the difference between the detected value and the true labeled value of the network is calculated. The loss is propagated back to the first convolution layer from the fully connected layer by the gradient descent method so that all the parameters of the network are updated, and the whole network model converges after several rounds of training.The formula for calculating a single deconvolution layer is as follows:(1)∑k=1K1zki⊕fk,c=yci.In this layer, an imageyi composed of feature images y1i,…,yK0i of K0 color channels is used as the input. Each channel c of the image can be expressed as the linear sum of K1 potential feature maps and convolution kernel.The deconvolution layer makes the potential feature graphzki sparse by introducing regularization terms. The total loss function of deconvolution layer is composed as follows:(2)C1yi=λ2∑c=1K0∑k=1K1zki⊕fk,c−yci22+∑k=1K1zkip,where p is sparse norm and λ is constant.The implementation process of deconvolution is shown in Figure2.Figure 2
Implementation process of deconvolution.


### 3.2. Transfer Learning

In the training of deep neural network, a large enough dataset with high quality is an important basis for the accuracy and high reliability of the training model. However, in practical application, for the research of video classification and recognition, because the common experimental dataset is small, the classification model trained by the Inception-ResNet-v2 network has the problems of low accuracy and poor generalization ability.Therefore, this paper adopts the idea of the transfer learning method and regards the Inception-ResNet-v2 network model after being pretrained by ImageNet large training set as a general image feature extractor. By transferring the extracted general image underlying feature knowledge to the news classification task, as an initialization parameter of the network, a small amount of video data can also learn and train a high-performance news classification model. The comparison between traditional machine learning process and transfer learning process is shown in Figure3.Figure 3
Comparison the process of traditional machine and transfer learning.Firstly, the definition of transfer learning is analyzed. Given a source domainDS=XS,fSX and a learning task TS, if there is a target domain DT=XT,fTX and a learning task TT, the goal of transfer learning is to use the useful information learned in DS and TS to help the target prediction function fTX in DT, where DS≠DT or TS≠TT. According to learning methods, transfer learning can be divided into four categories: sample-based transfer learning method, feature-based transfer learning method, model-based transfer learning method, and relationship-based transfer learning method [27]. The model-based transfer learning method refers to the method of finding the shared parameter information between the source domain and the target domain to realize the transfer, so this paper adopts the model-based transfer learning method.


### 3.3. Inception-ResNet-v2 Classification Model

At present, the classical convolutional neural network models are LeNet, AlexNet, VGGNet, GoogLeNet, ResNet, DenseNet, and ResNeXt. On the basis of the above structures, Szegedy et al. proposed the Inception-v2 structure and the Inception-v3 structure. Batch normalization is added to the Inception-v2 model, which makes the output of each layer obey the distribution of mean value of 0 and variance of 1. Inception-v3 integrates all the advantages of Inception-v2. Compared with v2, Inception-v3 uses asymmetric convolution to reduce the number of parameters and the amount of computation. On the contrary, label smoothing regularization is adopted to prevent overfitting.In 2016, the Google team released Inception-ResNet-v2 CNN, which scored the best result in the benchmark test of LSVRC image classification. It is inspired by the residuals network (ResNet) and is a variation of the Inception-v3 model. Residual connections can make shortcuts exist in the model, thus simplifying the concept module and completing the deeper neural network training. The network structure of Inception-ResNet-v2 is shown in Figure4. Compared with the current common deep convolutional neural network models such as GoogLeNet and ResNet, this paper uses Inception-ResNet-v2 network as the basic framework, as shown in Figure 5.Figure 4
Inception-ResNet-v2 network structure.Figure 5
Model architecture.The network structure of Inception-ResNet-v2 consists of a combination of Inception-v4 and ResNet. The three Inception-ResNet blocks (Inception-ResNet-A, Inception-ResNet-B, and Inception-ResNet-C) add direct connections to diversify the channels. Compared with Inception-v4, it has less parameters and faster convergence. At the same time, it also has a certain reduction in the performance requirements of the machine and can set higher parameters in the same experimental environment. The convolutional core of Inception-ResNet-v2 has more varied channels than Inception-ResNet-v1. For CNN, the commonly used optimization algorithms are gradient descent, etc. The network depth increases gradually, and the training error decreases first and then increases.The construction ideas of the proposed classification model structure are as follows:(1)
Using the pretraining model Inception-ResNet-v2 on ImageNet large-scale image dataset and combining with the model-based transfer learning method, the underlying features of the image learned by the pretraining model convolution module are migrated to the classified task as the initialization parameters of the network(2)
Training the classification model with the extracted feature map as input and replacing the output of the last full connection layer of the pretraining network with the category number of the news video dataset in this paper(3)
Completing the classification and recognition task on the news video dataset established in this paperAccording to the above ideas, the classification model based on the transfer learning and the Inception-ResNet-v2 pretraining network is shown in Figure6.Figure 6
Classification model based on transfer learning and Inception-ResNet-v2 pretraining model.This paper uses the model-based transfer learning method to build the classification model structure, which not only saves training time and reduces the requirements of experimental hardware configuration but also solves the overshoot problem caused by the small sample training process so that the generalization ability of the model is better.


### 3.4. Model Optimization Based on the Gradient Descent Method

Gradient descent method is the most commonly used objective function optimization algorithm in the field of deep learning [28, 29], and its purpose is to find the local minimum of the function. The gradient descent method generally conforms to the law that the function value is closer to the target value, the corresponding gradient decreases, and the descent is slower. The gradient descent method is an algorithm for neural network to obtain the optimal solution in the learning process. Commonly used are batch gradient descent (BGD), stochastic gradient descent (SGD), miniBatch gradient descent (MBGD), and Adam.Adam, as one of the adaptive gradient algorithms, combines the ideas of MBGD algorithm and SGD algorithm and calculates the mean and variance of gradients to dynamically adjust the learning rate. This algorithm is not sensitive to gradient scaling and diagonal rescaling, so it is very suitable for dealing with sparse data and nonstatic targets. It is one of the algorithms with the best gradient descent performance at present. Adam algorithm calculation formula is as follows:(3)θt+1=θt−αmt+1vt+1+ε,mt+1=ξ1mt+1−ξ1dfdθ,vt+1=ξ2vt+1−ξ2d2fdθ,where ξ1 is momentum coefficient, and the default value is 0.9, ξ2 is a constant and defaults to 0.999, α is the learning rate, θt and θt+1 are the weight values of step t and step t + 1, respectively, and ε is 10−8.In this paper, a momentum updating rule is introduced based on Adam, and the deviation correction term of the momentum vector is synthesized, so as to dynamically adjust the momentum deviation. The update process of dynamic Adam algorithm is as follows:(4)θt+1=θt−αm^t+1vt+1+ε,(5)m^t+1=mt+1ξ31−ξ1t+1.We can get from formula (4) that(6)θt+1=θt−αξ3vt+1+εξ1m^t+1−ξ11−ξ1t+1dfdθ,where the default value of ξ3 is 0.99.


## 3.1. Deep Neural Network

Convolution neural network is the most widely used deep learning model in the field of computer vision. Its earliest theoretical model is the neurocognitive machine proposed by Japanese scholar Fukushima [26]. The neural cognitive machine has a good recognition ability even when the target object is slightly distorted. On the basis of neurocognitive machine, a multilayer feedforward neural network model LeNet-5 appeared and was successfully applied in handwritten character recognition. The model mainly includes input layer, convolution layer, pooling layer, full connection layer, and output layer, which laid the foundation for the later convolution neural network structure. In 2012, the AlexNet model won the ILSVRC competition, which made convolutional neural network become a research hotspot, and then, more excellent convolutional neural networks were proposed. A typical convolutional neural network model architecture is shown in Figure 1.Figure 1
Typical convolutional neural network model architecture.Convolution neural network extracts features by convolution operation on local “receptive field,” and it is mainly used in image processing. CNN is a kind of feedforward neural network with deep structure. Firstly, the image is input at the input layer and then calculated by convolution layer, pooling layer, and nonlinear activation function, and the semantic information of high-level abstraction is gradually extracted from the image. This is the “feedforward operation” of convolutional neural network. Finally, for the fully connected layer, all the features extracted from the previous network are connected for prediction, and the difference between the detected value and the true labeled value of the network is calculated. The loss is propagated back to the first convolution layer from the fully connected layer by the gradient descent method so that all the parameters of the network are updated, and the whole network model converges after several rounds of training.The formula for calculating a single deconvolution layer is as follows:(1)∑k=1K1zki⊕fk,c=yci.In this layer, an imageyi composed of feature images y1i,…,yK0i of K0 color channels is used as the input. Each channel c of the image can be expressed as the linear sum of K1 potential feature maps and convolution kernel.The deconvolution layer makes the potential feature graphzki sparse by introducing regularization terms. The total loss function of deconvolution layer is composed as follows:(2)C1yi=λ2∑c=1K0∑k=1K1zki⊕fk,c−yci22+∑k=1K1zkip,where p is sparse norm and λ is constant.The implementation process of deconvolution is shown in Figure2.Figure 2
Implementation process of deconvolution.


## 3.2. Transfer Learning

In the training of deep neural network, a large enough dataset with high quality is an important basis for the accuracy and high reliability of the training model. However, in practical application, for the research of video classification and recognition, because the common experimental dataset is small, the classification model trained by the Inception-ResNet-v2 network has the problems of low accuracy and poor generalization ability.Therefore, this paper adopts the idea of the transfer learning method and regards the Inception-ResNet-v2 network model after being pretrained by ImageNet large training set as a general image feature extractor. By transferring the extracted general image underlying feature knowledge to the news classification task, as an initialization parameter of the network, a small amount of video data can also learn and train a high-performance news classification model. The comparison between traditional machine learning process and transfer learning process is shown in Figure3.Figure 3
Comparison the process of traditional machine and transfer learning.Firstly, the definition of transfer learning is analyzed. Given a source domainDS=XS,fSX and a learning task TS, if there is a target domain DT=XT,fTX and a learning task TT, the goal of transfer learning is to use the useful information learned in DS and TS to help the target prediction function fTX in DT, where DS≠DT or TS≠TT. According to learning methods, transfer learning can be divided into four categories: sample-based transfer learning method, feature-based transfer learning method, model-based transfer learning method, and relationship-based transfer learning method [27]. The model-based transfer learning method refers to the method of finding the shared parameter information between the source domain and the target domain to realize the transfer, so this paper adopts the model-based transfer learning method.


## 3.3. Inception-ResNet-v2 Classification Model

At present, the classical convolutional neural network models are LeNet, AlexNet, VGGNet, GoogLeNet, ResNet, DenseNet, and ResNeXt. On the basis of the above structures, Szegedy et al. proposed the Inception-v2 structure and the Inception-v3 structure. Batch normalization is added to the Inception-v2 model, which makes the output of each layer obey the distribution of mean value of 0 and variance of 1. Inception-v3 integrates all the advantages of Inception-v2. Compared with v2, Inception-v3 uses asymmetric convolution to reduce the number of parameters and the amount of computation. On the contrary, label smoothing regularization is adopted to prevent overfitting.In 2016, the Google team released Inception-ResNet-v2 CNN, which scored the best result in the benchmark test of LSVRC image classification. It is inspired by the residuals network (ResNet) and is a variation of the Inception-v3 model. Residual connections can make shortcuts exist in the model, thus simplifying the concept module and completing the deeper neural network training. The network structure of Inception-ResNet-v2 is shown in Figure4. Compared with the current common deep convolutional neural network models such as GoogLeNet and ResNet, this paper uses Inception-ResNet-v2 network as the basic framework, as shown in Figure 5.Figure 4
Inception-ResNet-v2 network structure.Figure 5
Model architecture.The network structure of Inception-ResNet-v2 consists of a combination of Inception-v4 and ResNet. The three Inception-ResNet blocks (Inception-ResNet-A, Inception-ResNet-B, and Inception-ResNet-C) add direct connections to diversify the channels. Compared with Inception-v4, it has less parameters and faster convergence. At the same time, it also has a certain reduction in the performance requirements of the machine and can set higher parameters in the same experimental environment. The convolutional core of Inception-ResNet-v2 has more varied channels than Inception-ResNet-v1. For CNN, the commonly used optimization algorithms are gradient descent, etc. The network depth increases gradually, and the training error decreases first and then increases.The construction ideas of the proposed classification model structure are as follows:(1)
Using the pretraining model Inception-ResNet-v2 on ImageNet large-scale image dataset and combining with the model-based transfer learning method, the underlying features of the image learned by the pretraining model convolution module are migrated to the classified task as the initialization parameters of the network(2)
Training the classification model with the extracted feature map as input and replacing the output of the last full connection layer of the pretraining network with the category number of the news video dataset in this paper(3)
Completing the classification and recognition task on the news video dataset established in this paperAccording to the above ideas, the classification model based on the transfer learning and the Inception-ResNet-v2 pretraining network is shown in Figure6.Figure 6
Classification model based on transfer learning and Inception-ResNet-v2 pretraining model.This paper uses the model-based transfer learning method to build the classification model structure, which not only saves training time and reduces the requirements of experimental hardware configuration but also solves the overshoot problem caused by the small sample training process so that the generalization ability of the model is better.


## 3.4. Model Optimization Based on the Gradient Descent Method

Gradient descent method is the most commonly used objective function optimization algorithm in the field of deep learning [28, 29], and its purpose is to find the local minimum of the function. The gradient descent method generally conforms to the law that the function value is closer to the target value, the corresponding gradient decreases, and the descent is slower. The gradient descent method is an algorithm for neural network to obtain the optimal solution in the learning process. Commonly used are batch gradient descent (BGD), stochastic gradient descent (SGD), miniBatch gradient descent (MBGD), and Adam.Adam, as one of the adaptive gradient algorithms, combines the ideas of MBGD algorithm and SGD algorithm and calculates the mean and variance of gradients to dynamically adjust the learning rate. This algorithm is not sensitive to gradient scaling and diagonal rescaling, so it is very suitable for dealing with sparse data and nonstatic targets. It is one of the algorithms with the best gradient descent performance at present. Adam algorithm calculation formula is as follows:(3)θt+1=θt−αmt+1vt+1+ε,mt+1=ξ1mt+1−ξ1dfdθ,vt+1=ξ2vt+1−ξ2d2fdθ,where ξ1 is momentum coefficient, and the default value is 0.9, ξ2 is a constant and defaults to 0.999, α is the learning rate, θt and θt+1 are the weight values of step t and step t + 1, respectively, and ε is 10−8.In this paper, a momentum updating rule is introduced based on Adam, and the deviation correction term of the momentum vector is synthesized, so as to dynamically adjust the momentum deviation. The update process of dynamic Adam algorithm is as follows:(4)θt+1=θt−αm^t+1vt+1+ε,(5)m^t+1=mt+1ξ31−ξ1t+1.We can get from formula (4) that(6)θt+1=θt−αξ3vt+1+εξ1m^t+1−ξ11−ξ1t+1dfdθ,where the default value of ξ3 is 0.99.


## 4. Experimental Results and Analysis

### 4.1. Experimental Environment and Dataset

Windows10 64-bit operating system was used in the experiment, and the processor configuration was Inter(R) Xeon(R) Silver 4110 CPU@2.10 GHz. The establishment, training, and testing of convolutional neural network are programmed by python language, the open source artificial neural network library Keras is called to create the model, and NVIDIA GeForce GTX 2080 is used to accelerate the training. A total of 362 news video samples were collected through the Internet and cooperative shooting, and training and testing datasets which can meet the needs of deep neural network training were made to prepare for the subsequent classification model training. Part of the data is shown in Figure7.Figure 7
Partial data display of news video samples. (a) Sports news, (b) political news, (c) economic news, and (d) military news.
(a)(b)(c)(d)


### 4.2. Evaluation Index

Since there are few types of news videos, this paper uses Top-1 accuracy (Acctop-1) as the evaluation index:(7)Acctop−1=Ntop−1N,where N represents the total number of videos and Ntop−1 represents the number of videos correctly classified.


### 4.3. Selection of Learning Rate and Batch Quantity

The learning rate controls the step size of gradient descent, and different learning rates have great influence on the convergence and classification accuracy of the model. In order to optimize the experimental results, the relationship between learning rate and classification results is analyzed. Experiment with different learning rate models is carried out under default parameter settings. Figure8 shows the change of the loss value with iteration times during the training process of the model. The amount of data needed to train the Inception-ResNet-v2 network model is generally relatively large. In the experimental process, batch training is generally adopted, that is, a batch of sample data is read in at one time. The selection of batch size is related to the memory size of GPU. If the batch size is too small, the parallel computing capacity of GPU cannot be fully utilized; if it is too large, it may exceed the computing capacity of GPU, resulting in overflow of video memory. Figure 9 shows the influence of different batch quantities on the model optimization process when the initial learning rate is 0.01.Figure 8
The effect of learning rates.Figure 9
The effect of batch size.It can be seen from Figure8 that, on the whole, a higher learning rate can achieve faster convergence. The model of 0.01 converges slowly, and the learning rate of 0.2 can achieve rapid convergence, but the final loss value is higher. This is because too high a learning rate may miss the optimal solution and reduce the classification accuracy. However, when the learning rate is 0.01, better results can be obtained, so the initial learning rate of subsequent experiments is set to 0.01.It can be seen from Figure9 that the larger the batch size, the better the performance of the loss value. The model with batch size of 64 converges fastest and loses the least. However, due to the limited configuration, video memory overflow will occur when the batch size is 64, so the batch size parameter selected in this experiment is 32.


### 4.4. Comparison of Optimization Algorithms

For the experimental schemes numbered 1–4, five gradient descent optimization algorithms, BGD, MBGD, SGD, Adam, and dynamic Adam, are used to optimize the parameters of this model. The experimental results are shown in Table1.Table 1
Comparison of different optimization algorithms.
Experiment no.Gradient descent optimization algorithmParameter settingsTop-1 accuracy (%)1BGD0.987.132MBGD0.0188.533SGD0.00189.274Adam0.9, 0.99990.145Dynamic Adam0.9, 0.999, 0.9991.47It can be seen from Table1 that the dynamic Adam optimization algorithm has higher accuracy than the other four algorithms, which shows that the dynamic Adam optimization algorithm has faster convergence speed, higher efficiency of network parameter optimization, and better learning effect.


### 4.5. Comparison of Network Models

For the experimental schemes numbered 1–10, ten pretrained convolution neural network models, AlexNet, VGG16, VGG19, Inception-v3, Inception-v4, ResNet-50, ResNet-101, ResNet-152, Inception-ResNet-v1, and Inception-ResNet-v2 were used for the experiment. The experimental results are shown in Table2, where the gradient descent optimization algorithm is unified as dynamic Adam. The transfer training strategy is the whole layer of the training and pretraining network models, and the training iteration steps are 25000.Table 2
Comparison of different optimization algorithms.
Experiment no.Network modelsTop-1 accuracy (%)Average training time (s)1AlexNet87.090.6532VGG1687.520.6623VGG1988.130.6134Inception-v387.810.2835Inception-v488.950.5256ResNet-5086.150.1767ResNet-10187.940.3048ResNet-15289.280.4419Inception-ResNet-v190.280.58210Modified Inception-ResNet-v291.470.633As shown in Table2, the training result of ResNet-50 network model has the lowest accuracy, while Inception-ResNet-V2 has the highest, and the accuracy is gradually improved with the increase of the depth of the network model, which indicates that the depth of the convolutional neural network has a certain influence on the result of news video classification and recognition. The deeper network structure has stronger ability to extract video information and higher classification accuracy.


## 4.1. Experimental Environment and Dataset

Windows10 64-bit operating system was used in the experiment, and the processor configuration was Inter(R) Xeon(R) Silver 4110 CPU@2.10 GHz. The establishment, training, and testing of convolutional neural network are programmed by python language, the open source artificial neural network library Keras is called to create the model, and NVIDIA GeForce GTX 2080 is used to accelerate the training. A total of 362 news video samples were collected through the Internet and cooperative shooting, and training and testing datasets which can meet the needs of deep neural network training were made to prepare for the subsequent classification model training. Part of the data is shown in Figure7.Figure 7
Partial data display of news video samples. (a) Sports news, (b) political news, (c) economic news, and (d) military news.
(a)(b)(c)(d)


## 4.2. Evaluation Index

Since there are few types of news videos, this paper uses Top-1 accuracy (Acctop-1) as the evaluation index:(7)Acctop−1=Ntop−1N,where N represents the total number of videos and Ntop−1 represents the number of videos correctly classified.


## 4.3. Selection of Learning Rate and Batch Quantity

The learning rate controls the step size of gradient descent, and different learning rates have great influence on the convergence and classification accuracy of the model. In order to optimize the experimental results, the relationship between learning rate and classification results is analyzed. Experiment with different learning rate models is carried out under default parameter settings. Figure8 shows the change of the loss value with iteration times during the training process of the model. The amount of data needed to train the Inception-ResNet-v2 network model is generally relatively large. In the experimental process, batch training is generally adopted, that is, a batch of sample data is read in at one time. The selection of batch size is related to the memory size of GPU. If the batch size is too small, the parallel computing capacity of GPU cannot be fully utilized; if it is too large, it may exceed the computing capacity of GPU, resulting in overflow of video memory. Figure 9 shows the influence of different batch quantities on the model optimization process when the initial learning rate is 0.01.Figure 8
The effect of learning rates.Figure 9
The effect of batch size.It can be seen from Figure8 that, on the whole, a higher learning rate can achieve faster convergence. The model of 0.01 converges slowly, and the learning rate of 0.2 can achieve rapid convergence, but the final loss value is higher. This is because too high a learning rate may miss the optimal solution and reduce the classification accuracy. However, when the learning rate is 0.01, better results can be obtained, so the initial learning rate of subsequent experiments is set to 0.01.It can be seen from Figure9 that the larger the batch size, the better the performance of the loss value. The model with batch size of 64 converges fastest and loses the least. However, due to the limited configuration, video memory overflow will occur when the batch size is 64, so the batch size parameter selected in this experiment is 32.


## 4.4. Comparison of Optimization Algorithms

For the experimental schemes numbered 1–4, five gradient descent optimization algorithms, BGD, MBGD, SGD, Adam, and dynamic Adam, are used to optimize the parameters of this model. The experimental results are shown in Table1.Table 1
Comparison of different optimization algorithms.
Experiment no.Gradient descent optimization algorithmParameter settingsTop-1 accuracy (%)1BGD0.987.132MBGD0.0188.533SGD0.00189.274Adam0.9, 0.99990.145Dynamic Adam0.9, 0.999, 0.9991.47It can be seen from Table1 that the dynamic Adam optimization algorithm has higher accuracy than the other four algorithms, which shows that the dynamic Adam optimization algorithm has faster convergence speed, higher efficiency of network parameter optimization, and better learning effect.


## 4.5. Comparison of Network Models

For the experimental schemes numbered 1–10, ten pretrained convolution neural network models, AlexNet, VGG16, VGG19, Inception-v3, Inception-v4, ResNet-50, ResNet-101, ResNet-152, Inception-ResNet-v1, and Inception-ResNet-v2 were used for the experiment. The experimental results are shown in Table2, where the gradient descent optimization algorithm is unified as dynamic Adam. The transfer training strategy is the whole layer of the training and pretraining network models, and the training iteration steps are 25000.Table 2
Comparison of different optimization algorithms.
Experiment no.Network modelsTop-1 accuracy (%)Average training time (s)1AlexNet87.090.6532VGG1687.520.6623VGG1988.130.6134Inception-v387.810.2835Inception-v488.950.5256ResNet-5086.150.1767ResNet-10187.940.3048ResNet-15289.280.4419Inception-ResNet-v190.280.58210Modified Inception-ResNet-v291.470.633As shown in Table2, the training result of ResNet-50 network model has the lowest accuracy, while Inception-ResNet-V2 has the highest, and the accuracy is gradually improved with the increase of the depth of the network model, which indicates that the depth of the convolutional neural network has a certain influence on the result of news video classification and recognition. The deeper network structure has stronger ability to extract video information and higher classification accuracy.


## 5. Conclusions

In this paper, a news video classification model based on the Inception-ResNet-v2 network and transfer learning is implemented. In this model, ImageNet network is used as the pretraining model, and the model-based transfer learning method is adopted to construct the Inception-ResNet-v2 network model structure, and the dynamic Adam algorithm is adopted as the model optimization method. The experimental results show that the Inception-ResNet-v2 network has a stronger ability to extract news video information, which is more conducive to the realization of classification tasks. The improved Adam algorithm can achieve the fastest convergence by iteratively updating the network weights with the adaptive learning rate. Compared with other convolutional neural network models, the classification accuracy of the proposed Inception-ResNet-v2 network model for common news video datasets is 91.47%.



---

*Source: 5865200-2021-12-16.xml*