# CASVM: An Efficient Deep Learning Image Classification Method Combined with SVM
## 2 Related Work  
A CNN is a feedforward neural network with convolutional computation that has excelled in image algorithms due to its exceptional learning power. From the traditional LeNet [35] and AlexNet [7] to the more modern MobileNet [36], ResNet [37], Nasnet [38], Efficient [39], Polynet [40], etc, they have all demonstrated excellent performance. Even so, the network model's classification layer in image classification tasks continues to use the softmax function and cross-entropy loss for back-propagation, which can cause the model to become over-fitted and degrade the model's performance. In contrast, when SVM is used as the classifier, it learns the lower-level parameters by back-propagating the gradient of the last layer, which improves the model's generalization ability [3; 16; 22; 41].  
In recent years, several excellent works combining SVM and CNN have been proposed. These techniques typically fall into two broad categories: image data is fed directly into a CNN-Softmax [16; 42] model to complete CNN model training, and then the CNN is used as a feature extractor to extract feature vectors to feed into the SVM to achieve image classification. The other option is to replace the Softmax function with SVM to achieve end-to-end CNN-SVM training with improved operability and generality [3; 4; 22; 43]. Furthermore, the method does not require any manual feature extraction or raw data processing, and it does not rely on expert experience or prior knowledge. Both of these approaches are intended to enhance the algorithm's generalizability and avoid over-fitting the model. However, using such techniques can result in drawbacks like the inability to do multi-classification or issues like low multi-classification accuracy [4]. Researchers have discovered one-to-all or one-to-one [44; 45; 46; 47] SVM classification approaches to address the issue of limiting multi-classification and have concentrated their research on feature extraction models paired with SVM to increase multi-classification accuracy [48].  
As the Vision Transformer (VIT) [49] model has achieved excellent performance in downstream tasks such as image classification and target detection [50; 51; 52], it has attracted the attention of many researchers. The key reason for the success of VIT is the adaptive, remote dependence through self-attentive operations to extract a global understanding of the visual scene. VIT expands the perceptual field of an image by building a global relational model, which in turn obtains more information about image features. However, VIT has a complex computational structure and large data requirements, and even the lightweight VIT still has a huge computational cost and is not applicable to edge devices [53]. In contrast, CNN with attention mechanism has received more and more attention due to its efficiency and effectiveness. Attention mechanisms widely used in image classification tasks are channel attention and spatial attention. In channel attention, different channels in different feature maps usually represent different objects [54]. The channel attention mechanism can adaptively recalibrate the weights of each channel, viewing this process as a focused object selection process, thus eliminating the need for larger computations to determine what image features to pay attention to [29; 31; 55; 56; 57]. Spatial attention, on the other hand, can be seen as where to pay attention [27; 58; 59; 60]. Channel attention combined with spatial attention enables the adaptive selection of objects and regions worthy of attention [30; 54; 61]. However, such an attention mechanism, while emphasizing the importance of space and channels, also brings a higher computational cost and limited perceptual field. In contrast, the coordinate attention mechanism combines the advantages of SENet [29], and CBAM [30] to capture location information and channel relationships with very low computational overhead.