# A Lightweight Attention-Based Network towards Distracted Driving Behavior Recognition
## 2 Related Work  
In the field of deep learning, especially in image classification and object detection tasks, a significant amount of research focuses on reducing network complexity and computation cost rather than simply increasing the accuracy. An optimized network should train a model with the smallest size and the highest accuracy. Only such networks can be applied to industrial production and practices. These requirements challenge the overall performance of a network. There are mainly two methods towards designing a lightweight convolutional neural network. One is compressing the existing network with excellent performance [19]. It refers to removing the redundant convolution layers and fully connected layers and cropping the convolution kernels. Another one is designing a new network module. Currently, most research focuses on the convolution operation. [PERSON] et al. [20] utilized depthwise separable convolutions to establish lightweight deep neural networks. This method converted standard convolution to depthwise convolution and \(1\times 1\) pointwise convolution. [PERSON] et al. [21] used asymmetric convolution blocks to replace the standard convolutional layers for object detection. The asymmetric convolution module had three parallel branches. The first branch included asymmetric convolution with kernel sizes of 1 \(\times\) 3 and 3 \(\times\) 1. The second branch included asymmetric convolution with a kernel size of 1 \(\times\) 3. The third branch included asymmetric convolution with a kernel size of 3 \(\times\) 1. The octave convolution proposed by [PERSON] et al. [22] was a plug-and-play convolutional unit that could greatly reduce memory and computation cost. The octave convolution divided the convolution feature maps into low-frequency and high-frequency groups and halved the size of the low-frequency feature maps to speed up the convolution operation. In recent years, the attention mechanism has attracted researchers' interests. Its essence is to simulate how a human observes an object. In cognitive science, due to the bottleneck of information processing, human beings will selectively pay attention to a part of the information and ignore the other visible information [23]. In the field of deep learning, attention mechanism is first applied to machine translation of natural language processing [24]. It spreads to object detection and image classification [25; 26] later. The attention mechanism is usually divided into hard attention and soft attention. Since soft attention is differentiable, a neural network can compute the gradient and learn the weighted attention by forward propagation and backward feedback [27]. Hence, soft attention is widely utilized in deep learning. It can be further divided into channel domain, spatial domain, and mixed domain. Channel domain is related to the feature channels of an image. Spatial domain is related to the location of features in an image. The current mainstream of attention modules is SE [28], CBAM [29], EAC [30], and Triplet [31]. Towards image classification problem, with the aid of attention mechanism, the network performance improves significantly. [PERSON] et al. [32] proposed a bilinear squeeze-and-excitation network (BiSENet) for tree species classification and achieved better performance than existing methods. [PERSON] et al. [33] used the CBAM module for water scene recognition. The method had fewer parameters and could improve the accuracy of water scene recognition. [PERSON] et al. [34] used an improved CBAM module for fly species recognition with an accuracy rate of 90%, which was better than the state-of-the-art methods. [PERSON] et al. [35] used the triple attention method for 14 thoracic disease classification. According to the spatial location of pathological abnormalities, it could determine which feature channels provide discriminative information and which scale played a major role in diagnosis. [PERSON] et al. [36] used a hybrid attention approach for hyperspectral image classification.