# Time-Series Classification Based on Fusion Features of Sequence and Visualization
## 3 Model Design
### Cnn-A  
TS data is a series of sequential data. It is difficult for human beings to classify them only by ordered numbers, as vision is an important sense for human beings. By visualizing TS data, humans can easily classify them. Deep learning is a technology that imitates the human brain and, so, we consider visualizing time-series data and extracting features to imitate human vision for TSC. In addition,  
Figure 2: Architecture of the LSTM-A model. T-Attention means temporal attention block, the permute layers are used to permute the dimensions of the input matrix, and BN stands for batch normalization. through data visualization, human vision can notice the parts with the largest differences and classify them. This is similar to the attention mechanism in deep learning. Therefore, we added an attention mechanism to CNN to extract key visualization features. We used CNN-A to imitate human vision (the network structure is shown in Figure 3). For univariate TS data \(D\), we convert each TS record \(x_{i}\) into a black-and-white Area Graph (examples shown in Figure 4a,b) as input. In order to avoid the influence of unnecessary information such as co-ordinates, we removed the co-ordinates and other information from the Area Graphs, and only retained the graph part showing the data fluctuations. Differing from the existing visualization transformation methods (e.g., Gramian fields [8; 9], RP [10; 11], and MTF [12]), the Area Graph is similar to human vision, directly reflecting information such as the fluctuation of TS data. The most important point is that this method is a very simple transformation method without additional calculation, avoiding the loss of some information in the calculation process. Such image data can extract visualization features through the CNN-A model. In human vision, information such as peak value and mean value is more likely to attract attention. TS data belonging to the same category have similar peaks and averages after visualization, while data of different categories has large differences in such information. So, we used the CT-Attention block to extract these features and improve TSC. This block extracts the mean and maximum information of the category through a pixel average pooling layer based on Area Graphs of TS records belonging to same category, where the output is the CT Attention. The extracted CT attention is combined separately with the features (average and max features) extracted from each time-series record and, then, through a global average pooling layer, a Dropout layer, and a Batch Normalization layer, the final visualization features are obtained. Finally, classification is carried out through the softmax layer. \[I =trans\left(x_{i}\right)\mid x_{i}\in D, \tag{4a}\] \[C =f\left(W*I+B\right),\] (4b) \[H =P\left(C\right),\] (4c) \[CT =\prod Arg\left(Pix\left(x,y\right)\right)\mid x,y\in I,y_{1}=y,y \in\left\{1,2\cdots c\right\},\] (4d) \[F^{v} =GAP\left(CT\odot H\right),\] (4e) \[P^{v} =softmax\left(F^{v}\right),\] (4f) \[A\text{{U}}\text{{X}}_{2} =H\left(P^{v},Y\right)=-\sum_{i}y_{i}log\left(p_{i}^{v}\right). \tag{4g}\]  
Figure 3: Architecture of the CNN-A model. BN stands for batch normalization, the GAP layer is the global average polling layer, and CT-Attention is the category trend attention. The process of learning visualization features by CNN-A can be expressed by Equation (4), where \(trans\) means converting the TS record \(x\) to an Area Graph \(I\), \(C\) denotes the result of a convolution (where * indicates the standard dot product) applied on every \(I\), \(W\) and \(B\) are filter parameters, and \(f\) represents the combination of Batch Normalization and a final activation operation such as Rectified Linear Unit (Relu). \(P\) stands for max and average pooling, and \(H\) represents the pooling results. \(CT\) is the category-related attention obtained by \(Avg\) pooling on every pixel \((x,y)\) of Area Graphs belonging to the same category, and \(y_{I}\) is the category that image \(I\) belongs to (as shown in Figure 5).