# CASVM: An Efficient Deep Learning Image Classification Method Combined with SVM
## 3 Methods
### Attention Mechanism  
The channel attention mechanism represented by SENet, as shown in Figure 2a, establishes the interrelationship between channels by simply compressing the 2D feature map, which significant affects on model performance, However, SENet is weak at processing location information. CA retains location information by embedding it in channel attention based on SENet, which establishes long-distance dependencies. Channel attention converts the input to a single feature vector by 2D global pooling, as shown in Figure 2b. CA splits channel attention into two one-dimensional feature encodings in different directions, and encodes these to produce two feature maps with enhanced position sensitivity and orientation perception, to enhance the target of interest. The attention block includes compression and excitation to, respectively, collect global information and capture the importance of each channel. The compression operation can be expressed as  
\[z_{c}=\frac{1}{H\times W}=\sum\limits_{l=1}^{H}\sum\limits_{j=1}^{W}x_{c}(i,j) \tag{4}\]  
The excitation operation of the capture channel can be expressed as  
\[\hat{X}=X\cdot Sigmoid(T_{2}(Relu(T_{1}(z)))) \tag{5}\]  
where \(T_{1}\) and \(T_{2}\) denote the learnable linear transforms used to capture the importance of each channel.  
Similar to SENet, the CA attention block can be considered a computational with two steps: CIE and CAG. CIE is the critical operation in the attention module to capture precise location information and establish long-term dependencies on spatial orientation. A given \(X\) is encoded for each channel along horizontal and vertical coordinates using pooling kernels of dimensions (\(H\), 1) and (\(1\), \(W\)), respectively, i.e., the global pooling is transformed to a pair of one-dimensional feature encoding operations. The output of the C-th channel at the height and width of \(H\) and \(W\), respectively, must satisfy the following equation:  
\[\left\{\begin{array}{l}z_{c}^{h}(h)=\frac{1}{W}\sum\limits_{0\leq i<W}x_{c}(h,i)\\ z_{c}^{w}(w)=\frac{1}{H}\sum\limits_{0\leq j<H}x_{c}(j,w)\end{array}\right. \tag{6}\]  
Figure 2: Structure of attention mechanism: (**a**) SENet; (**b**) CA.  
Equation (6) aggregates feature along two spatial directions through two transformations to generate a pair of directional perception attentional diagrams, which is entirely different from the compression operation in the SE module. One captures long-term dependencies along the spatial direction, and the other direction retains precise positional information, which helps to de-localize objects of interest. CAG uses the location information of the global receptive field generated by CIE to generate an attentional map. CAG must comply with the following requirements:  
1. The conversion should be as simple and efficient as possible.
2. The captured position information can be used to precisely locate the region of interest.
3. Relationships between channels can be efficiently captured.  
To this end, the first operation of CAG is to cascade the two feature maps generated by the CIE module and transform them using \(1\times 1\) shared convolution, i.e., \(F_{1}\) generating \(f\in R^{C/r\times(H+W)}\) as an intermediate feature map of spatial information in the horizontal and vertical directions, where \(r\) is the downsampling ratio,  
\[f=\delta(F_{1}([z^{h},z^{w}])) \tag{7}\]  
where \(\delta\) is the nonlinear activation function. The second step of CAG divides \(f\) into tensors \(f^{h}\in R^{C/r\times H}\) and \(f^{W}\in R^{C/r\times W}\), which are transformed by \(1\times 1\) convolutions \(F_{h}\) and \(F_{w}\), respectively, into the same number of channels as the input \(X\), to obtain  
\[\left\{\begin{array}{l}g^{h}(h)=Sigmoid(F_{h}(f^{h}))\\ g^{w}(w)=Sigmoid(F_{w}(f^{w}))\end{array}\right. \tag{8}\]  
The final CA output is expressed as  
\[y_{c}(i,j)=x_{c}(i,j)\times g_{c}^{h}(i)\times g_{c}^{w}(j) \tag{9}\]