# Convolutional Neural Network with an Elastic Matching Mechanism for Time Series Classification
## 3 Proposed Method
### Elastic Matching in the Convolutional Neural Network  
The CNN extracts features from the time series by measuring the local similarity between the time series \(X=(x_{1},x_{2},...x_{i},...,x_{n})^{\text{T}}\) and convolutional kernel \(W=(w_{1},w_{2},...w_{j}...,w_{m})^{\text{T}}\). In general, the similarity measure adopted in the CNN is the inner product. Considering the definition of the inner product, the matching mechanism of the inner product is similar to the ED. A point of one series only matches the point of another series in the same position. Hence, the inner product is inappropriate to measure similarity for temporal distortion. An elastic matching mechanism is incorporated into the inner product to better model the matching relationship between the time series and convolutional kernels. The elastic matching mechanism allows the kernel points to construct relationships with points in different positions of the time series. The similarity of the \(i\)th location is defined by Equation (3). The convolutional layer combined with the elastic matching mechanism is called the matching convolutional (MConv) layer. The structure of the MConv layer is presented in Figure 2. A fully-connected (FC) layer is used to learn the matching relation  
Figure 1: Example of elastic matching in dynamic time warping (DTW); the yellow rectangles are the beginning and end of the two paths; blue circles and red rhombuses represent the optimal paths in Euclidean distance and DTW, respectively.  
ship between the series and kernels. The weights of the FC layer in Figure 2 correspond to the matching matrix \(M\) in Equation (3).  
\[Similarity_{i}=W^{\text{T}}MX_{i:i+m}, \tag{3}\]  
where \(m\) is the length of a convolutional kernel, and \(M\) is an \(m\times m\) matching matrix.  
When \(M\) is an identity matrix, Equation (3) degenerates to the inner product, and Equation (3) can be considered an extension of DTW. The proof is as follows.  
Without loss of generality, this proof is based on the example in Figure 1. The time series and convolutional kernel have the same length in a sliding window for the CNN. Red rhombuses represent the optimal path in Figure 1. Hence, series \(X=(x_{1},x_{2},...x_{i},...,x_{7})^{\text{T}}\) and kernel \(W=(w_{1},w_{2},...w_{j},...,w_{7})^{\text{T}}\) are transformed to \(X^{\prime}\) and \(W^{\prime}\) as shown in Equation (4), respectively:  
\[\begin{split} X^{\prime}&=(x_{1},x_{1},x_{2},x_{2},x_{3},x_{4},x_{4},x_{5},x_{5},x_{6},x_{7})^{\text{T}}\\ W^{\prime}&=(w_{1},w_{2},w_{3},w_{4},w_{4},w_{5},w_ {6},w_{6},w_{7},w_{7},w_{7})^{\text{T}}.\end{split} \tag{4}\]  
If a dot product is used to measure the similarity between two points, the DTW similarity between \(X\) and \(W\) is equivalent to the inner product between \(X^{\prime}\) and \(W^{\prime}\) as presented in Equation (5):  
\[Similarity_{DTW}(X,W)=X^{\prime}\cdot W^{\prime}. \tag{5}\]  
Equation (5) can be further expressed as a matrix multiplication as indicated in Equation (6):  
\[Similarity_{DTW}(X,W)=W^{\text{T}}M^{\prime}X, \tag{6}\]  
where \(M^{\prime}\) is a binary matrix and satisfies the conditions as shown in Equation (7):  
\[M^{\prime}_{i,j}=\begin{cases}1,&x_{i,j}\in X^{\prime}\text{ and }w_{i,j}\in W^{\prime}\\ 0,&otherwise\end{cases}. \tag{7}\]  
Comparing Equations (3) and (7), DTW is a special case of the proposed matching mechanism.