# Research on a Convolution Kernel Initialization Method for Speeding Up the Convergence of CNN
## 1 Introduction  
Deep learning is an important branch of artificial intelligence. The convolutional neural network (CNN) has received much attention in recent years as one of the important deep learning models. The CNN is a feed-forward neural network with a deep structure. Compared with other deep learning models, the weight sharing strategy of the CNN greatly reduces the complexity of the network model and the number of parameters. When the input data is multi-dimensional in the network, the advantages of the strategy are more obvious. The data can be directly input into the network without complex feature extraction and data reconstruction processes in the traditional recognition algorithms. At the same time, the CNN has translation invariance, rotation invariance, and scale invariance to image target in space. According to these characteristics of CNN, it is widely used in image classification, target localization, semantic segmentation, and other visual tasks [1; 2; 3; 4; 5]. However, there are many problems that have not been solved for CNN. The convolution kernel initialization is one of the questions. The CNN is a non-linear mapping, and the optimization of the training process has a great relationship with the initial conditions. Therefore, the initialization of the convolution kernel is very important for CNN training. Much research in recent years has focused on this field. A popular method for initializing convolution kernels is random assignment [6]. This method makes the weight parameters obey the Gaussian distribution with a mean value of 0 and a standard deviation of 1. This method is simple and straightforward, but it has many obvious disadvantages. For example, it will lead to slowing down thelearning speed of the network, and it will make the learning process fall into a local optimal problem. In order to solve the problem, [PERSON] et al. proposed a convolution kernel initialization method commonly known as \"Xavier initialization value\" [7]. When the number of input neurons is n, the initial value of the convolution kernel obeys the uniform distribution with a mean value of 0 and a variance of 1/n. This method improves the convergence speed of the network. But the initialization method's one of the assumptions in the derivation process is that the activation function is Linear. As a result, this method is not suitable for non-linear activation functions such as Relu. Later, [PERSON] et al. proposed an initialization method specifically for the Relu activation function [8]. The method is seen as an improved version of the Xavier initialization value. The initial value of the convolution kernel obeys the Gaussian distribution with a mean value of 0 and a variance of 2/n. Much research has demonstrated that the above two methods can both get better results for different datasets. However, in the process of networking training, convolution kernels and the training sample are independent of each other after the convolution kernels are initialized. Convolution kernels match with the local pattern of the sample in the training images, which is a small probability event. After many iterations, the convolution kernels can better match the local patterns of training samples. Therefore, the network training needs a longer time, and the network convergence rate slows down. Many methods for initializing the convolution kernel have been proposed to solve the above problems. OrthoNorm is an orthogonal matrix initialization method. It is better than the approximate orthogonal Gaussian distribution, and the method can also be used for non-linear networks [9]. The Layer sequence unit variance (LSUV) method extends orthogonal initialization to the iterative process. This method uses Gaussian noise with unit variance to replace the weights. Then they are decomposed into standard orthogonal bases with orthogonal matrix and upper triangular matrix decomposition or singular value (SV) decomposition, and one of the components is used to replace the weights.