# Improvement for Convolutional Neural Networks in Image Classification Using Long Skip Connection
## 1 Introduction  
Wood in general and bamboo in particular have become one of the most popular materials today due to their environmental friendliness. Because of its popularity, product quality requirement becomes more important and can be a crucial aspect in the industrial production line. Many studies in the field of defect detection and classification, in regard of digital image processing, aim to minimize or replace human vision and decision methodologies with artificial techniques [1]. Recent image processing-based methods could deal with defect classification at a decent performance level but they were limited to detecting simple, distinctive defects from the background. [PERSON], [PERSON] [2] used self-organizing map (SOM) for discriminating between sound wood and defects. [PERSON], [PERSON] [3] proposed an algorithm by combination with Blob analysis algorithm and image preprocessing approach to detect the defects. [PERSON], [PERSON] [4] used fast multispectral texture defect detection method based on the underlying three-dimensional spatial probabilistic image model. [PERSON] [5] provided an online method of bamboo defect inspection and classification. [PERSON], [PERSON] [6] proposed a new surface grading approach by integrating the color and texture of bamboo strips based on Gaussian multi-scale space.  
Recent studies in the deep learning field have proved that most problems in machine vision that were difficult in the traditional computer vision methods, were solved using convolutional neural network (CNNs). [PERSON] et al. [7] first introduced CNN with AlexNet that trained with the difficult ImageNet dataset, solved the classification task with top-1 and top-5 error rates of 37.5% and 17.0%, which was by far better than previous works. [PERSON] and [PERSON] [8] proved that CNNs had to have a deep network of layers in order for this hierarchical representation of visual data to work. However, stacking multiple convolution layers can cause the problem of vanishing gradient, making the model difficult to learn. ResNet [9] and Highway Networks [10] introduced the idea that vanishing gradient problem could be tackled by by-passing the signal from layer to layerby connections. Deeper architectures were created using skip connections and achieved better accuracy. DenseNet [11] utilized these connections by creating short paths between every layer in its architecture. Skipping effectively simplifies the network, using fewer layers in the initial training stages. This accelerates learning by reducing the impact of vanishing gradients, as there are fewer layers to propagate through. The network then gradually restores the skipped layers as it learns the feature space. Towards the end of training, when all layers are expanded, it stays closer to the manifold and thus learns faster. Xception [12] and MobileNet v1 [13] presented Separable Convolutional layer, which significantly reduced computational cost, comparing to traditional Convolutional layer, with only a small decrease in accuracy.  
In this paper, we study effect of long skip connection in several convolution neural networks while performing image classification tasks. The models with the long connection are proved to be more robust to converge, have better performance than its counterpart, and can be applied to every model. For DenseNet and ResNet, we change traditional convolution by Depthwise Separable Convolution for the purpose of reducing parameters and floating-point operations per second (FLOPs). We observe the long skip connections accelerate the convergence of the learning process, while performing slightly better in comparison with tested models.  
This paper is organized in the following context: Section 2 shows the works related to our network. Our modified network is shown in Section 3. Section 4 presents the experimental results and analysis. The discussion, conclusions and further work are detailed in the last two sections.