# A Lightweight Attention-Based Network towards Distracted Driving Behavior Recognition
## 3 Methods
### Proposed LWANet Network Structure  
The classic VGG16 network [43] includes 13 convolution layers and 3 fully connected layers. The entire network utilizes convolution kernels with size 3 \(\times\) 3 and max-poolingwith stride \(2\times 2\). Due to a large number of trainable parameters, VGG16 consumes a lot of computation resources. It is not feasible for its deployment on edge devices.  
Based on the depthwise separable convolution and inverted residual attention module discussed above, we propose the Lightweight Attention-based network architecture. The depthwise separable convolution can greatly reduce the computational cost, making it possible to apply the model to the edge devices. Inverted residual attention module can focus the weights of the proposed network on meaningful pixels and channels, promote the effective features, and suppress the interference of noise. Therefore, the inverted residual attention module can effectively improve the accuracy. The network is designed to reduce the trainable parameters but remains a desirable model accuracy. The network schematic is shown in Figure 4.  
LWANet takes an image of size \(120\times 120\times 3\) as the input and produces a classification label as the output. The detailed structure of filter shapes and input and output shapes of each layer is shown in Table 1.  
\begin{table}
\begin{tabular}{c c c c} \hline
**Layer Name** & **Input Shape** & **Output Shape** & **Filter Shape** \\ \hline Conv\_1 & \(120\times 120\times 3\) & \(120\times 120\times 32\) & \(3\times 3\times 32\) \\  & & & \(1\times 1\times 32\times 3\) \\  & & & \(3\times 3\times 96\) \\ IRAM\_1 & \(120\times 120\times 32\) & \(120\times 120\times 32\) & \(1\times 1\times 96\times 32\) \\  & & & \(3\times 3\times 32\) \\ DSWC\_1 & \(120\times 120\times 32\) & \(120\times 120\times 32\) & \(3\times 3\times 32\) \\ Maxpool\_1 & \(120\times 120\times 32\) & \(60\times 60\times 32\) & \(1\times 1\times 32\times 32\) \\ Conv\_2 & \(60\times 60\times 32\) & \(60\times 60\times 32\) & \(3\times 3\times 32\) \\ Maxpool\_2 & \(30\times 30\times 32\) & \(30\times 30\times 32\) & \(30\times 30\times 64\) \\ Conv\_3 & \(30\times 30\times 32\) & \(30\times 30\times 64\) & \(3\times 3\times 64\) \\  & & & \(1\times 1\times 64\times 2\) \\  & & & \(3\times 3\times 128\) \\ IRAM\_2 & \(30\times 30\times 64\) & \(30\times 30\times 64\) & \(1\times 1\times 128\times 64\) \\  & & & \(3\times 3\times 64\) \\ DSWC\_2 & \(30\times 30\times 64\) & \(30\times 30\times 64\) & \(3\times 3\times 64\) \\  & & & \(1\times 1\times 64\times 64\) \\ Maxpool\_3 & \(30\times 30\times 64\) & \(15\times 15\times 64\) & \(1\times 15\times 64\) \\ Conv\_4 & \(15\times 15\times 64\) & \(15\times 15\times 64\) & \(3\times 3\times 64\) \\ Maxpool\_4 & \(15\times 15\times 64\) & \(8\times 8\times 64\) & \(8\times 8\times 128\) \\ Conv\_5 & \(8\times 8\times 64\) & \(8\times 8\times 128\) & \(3\times 3\times 128\) \\ Maxpool\_5 & \(8\times 8\times 128\) & \(4\times 4\times 128\) & \\ Fc\_1 & \(1\times 1\times 2048\) & \(512\) & \\ Fc\_2 & \(512\) & \(10\) & \\ \hline \end{tabular}
\end{table}
Table 1: A list of detailed network architecture.  
Figure 4: Schematic of the Lightweight Attention-based network (LWANet).  
Balancing the decreasing accuracy and computation lost, we utilize two depthwise separable convolution layers to replace the standard convolution layers. IRAM is connected after the first and third standard convolution layer. ReLU activation function and max-pooling are added after all the five standard convolution layers. Max pooling is involved to reduce the feature map sizes. Two fully connected layers reduce the last convolutional feature maps to 512-D and 10-D feature vectors.