A Robust Dissimilarity-based Neural Network for Temporal Pattern Recognition
Brian Kenji Iwana∗† , Volkmar Frinken‡§ , Seiichi Uchida∗∗ Department of Advanced Information Technology, Kyushu University, Fukuoka, Japan
Email: brian@human.ait.kyushu-u.ac.jp, uchida@ait.kyushu-u.ac.jp† Institute of Decision Science for a Sustainable Society, Kyushu University, Fukuoka, Japan‡ ONU Technology, San Jose, USA§ Electrical and Computer Engineering, University of California Davis, Davis, USA

Abstract—Temporal pattern recognition is challenging because temporal patterns require extra considerations overother data types, such as order, structure, and temporal
distortions. Recently, there has been a trend in using large
data and deep learning, however, many of the tools cannot
be directly used with temporal patterns. Convolutional Neural
Networks (CNN) for instance are traditionally used for visualand image pattern recognition. This paper proposes a method
using a neural network to classify isolated temporal patterns
directly. The proposed method uses dynamic time warping(DTW) as a kernel-like function to learn dissimilarity-based
feature maps as the basis of the network. We show thatusing the proposed DTW-NN, efficient classification of on-line
handwritten digits is possible with accuracies comparable tostate-of-the-art methods.
Keywords-Dynamic Time Warping; Neural Networks; Temporal Pattern Recognition; Deep Learning; Time Series

I. I NTRODUCTION
Temporal patterns are a vital part of life and come inmany forms, such as sound, handwriting, gestures, signals,
trends, and any other data that is dependent on time. Similar
to other structural patterns, temporal patterns are complex
and difficult due to the imporance of sequence order anddata element connectivity. In addition, temporal patterns
are subject to temporal distortions such as variable rateand inconsistent lengths. To overcome the difficulties andcapture the relevant aspects of temporal patterns, methods
specifically designed for temporal pattern recognition must
be used.
Dynamic time warping (DTW) is a popular algorithm fordetermining the distance between two temporal patterns [1].
Unlike the direct approach to naively matching elements,
DTW overcomes the temporal distortions by elasticallymatching sequence elements under constrained restrictions.
Paired with a distance-based classifier such as k-Nearest
Neighbors (k-NN), it has shown to be an effective distance measure between temporal patterns [2]. When usedwith large quantities of data, however, k-NN becomesprohibitively computationally intensive and not suitable forsome applications due to its exhaustive search nature [3].
Some common solutions to overcome the computational
requirement include reducing the size of the patterns, feature
representation, and prototype selection.

In recent times, artificial neural networks have been alarge focus of research due to their recent successes inpattern recognition and machine learning [4]. Inspired bybiological neural networks, artificial neural networks function with layers of interconnected nodes, each with input
stimuli and activated output signals. By stacking many layersof neurons, a network has the ability to learn complex
functions, generally referred to as deep learning. Temporal
pattern deep learning is also an active field of research, withsuccesses in natural language processing [5], speech [6], andhandwriting [7], [8]. Particularly the property to interpretnodes of a neural network as a feature extraction, and hence
the ability to automatically derive and evaluate features fromvarious input sources is promising.
To exploit the feature learning and classification for
temporal patterns, we propose a novel method of using a
DTW-layer within a neural network to create dissimilaritybased temporal feature maps. The objective of this is topreserve the nature of temporal patterns of variable sequencelength while still using a feed-forward neural network. By
considering DTW as a similarity between input patterns andfilter patterns, the direct application of temporal patterns into
the network is possible without rasterization or other transformations. Also, unlike other methods employing DTW, we
use DTW as integral part of the neural network and hence
also part of the training process.
The primary contribution of this paper is the presentationof a new neural network model called a Dynamic Time
Warping Neural Network (DTW-NN). We demonstrate thepossibility of learning isolated temporal pattern directly andunmodified. We also show the possibility of developing
temporal filters with back-propagation through the DTW
layer. Lastly, we show that it is an effective and efficientmethod of classifying on-line handwritten digits.
The remaining of this paper is organized as follows.
Section II reviews the related literature in neural learningwith temporal patterns. Section III details DTW and its
application to kernels for machine learning. In Section IV,
we propose a DTW-NN framework. Section V describes thearchitecture for using our network for on-line handwritten
digit recognition. Finally, Section VII draws a conclusion
and outlines future work.

II. R ELATED W ORKS

DTW filters

Fully-Connected Layers

OutputTemporal Input

Neural network approaches to time series recognition isan active field of research. A common approach is to feed
the model a sliding window view of the data giving each
input a step in time. Recurrent Neural Networks (RNN) [9]use this method to read the patterns into layers of recurrent
nodes, or nodes that connect back to themselves. A LongShort Term Memory (LSTM) [10] network is a model built
upon RNNs to learn data with long time lags. Alternatively,
Time Delay Neural Networks (TDNN) [11], [12] are feedforward networks that also use sliding window.
Convolutional Neural Networks (CNN), however, process
entire inputs as pixel data and learns from the spacial
structure. A common method of using CNNs for on-line
handwriting recognition is to rasterize the data, or render the
temporal pattern in pixel-space [13], [14]. Zheng, et al. [15]use channels of serialized 1D subsequences instead of 2D
pixels to classify multivariate time series’. However, thesemethods of preprocessing are not relevant to many temporal
patterns, such temporal patterns with very high dimensional
elements or pattern with sporadic elements.
III. DYNAMIC T IME WARPING
A. DTW Definition
DTW determines the similarity between temporal patterns
by calculating the summation of the distances between the
optimally matched elements of sequences. Given a prototype
sequence p = p1 , . . . , pi , . . . , pI and a sample sequences = s1 , . . . , sj , . . . , sJ , the global difference between patternis the collection of distances between the pairwise element
matches. Where Q is the set of all sample elements matched
to a prototype element, DTW is defined as
X
DTW(p, s) =||pi − sq || .
(1)
sq ∈Q

To determine the elements matched in Q, the experimentassumes the step size and slope constraints given by
D(i, j) = c(pi , sj ) + 0minj ∈{j,j−1,j−2}

D(i − 1, j 0 ),

0
1

8
9

Figure 1.

Illustration of the proposed DTW-NN architecture.

B. DTW and Kernels
In common artificial neural networks, a vectorial data
representation is given to the input layer, with one node foreach entry on the input vector. Then, the data is propagatedforward, one layer at a time. Usually, in each layer, theactivation of a node j with activation an+1
in layer n + 1
jhas the form
X

an+1
=φ
wij ani ,(3)jwhere wij is a weight, ani the activation of node i in layer n,and φ(·) an activation function, such as sigmoid, hyperbolic
tangent function, or a rectified linear unit (ReLU) [16].
All these activation functions are monotonically increasing.
Hence if the normalized weight vector wi· and the normalized output of the of the previous layer ani are similar, we can
expect a larger activation an+1
. In this sense, each weightj
node combination can be seen as similarity function, notunlike a kernel, although the positive semidefiniteness is
not given. From this point of view, each layer of a feedforward neural network can be seen as dissimilarity spaceembedding [17] with the prototypes as the column vectorsof the weight matrix between two layers.
We adopt this idea in this paper by considering the DTW
distance of an input sequence to a given filter sequence as
the node activation. In the past, attempts to adapt Support
Vector Machines (SVM) to use temporal patterns were doneby using DTW kernels [18], DTW for time alignment [19],and incorporating DTW as the distance measure for a RBF
kernel [20].

(2)where D(i, j) is the cumulative cost at the i-th and j-thelement. c(pi , sj ) is the cost matrix where c is the local
distance measure between each element of the sequences pand s respectively. In other words, DTW is the minimizationof the warping path on the cost matrix between p and s.
The asymmetric slope constraint given by Eq. (2) provides
DTW with a distance measure that is length and temporalfluctuation invariant. This slope constraint was selected
because it assures that the number of element matches isalways equal to the number of elements I in p. With equal
sized prototype sequences, DTW provides a robust method
for the comparison of sample patterns unequal lengths.

IV. DYNAMIC T IME WARPING N EURAL N ETWORK(DTW-NN)The architecture of the proposed method is detailed in thissection. Training is accomplished using rounds of forward
feeding the network and back propagating the errors to adjust
the weights.
A. Forward Propagation of DTW
We propose using a DTW as a kernel-like method as areplacement for the typical node activation of a neural network. So, rather than extracting local features, we propose amodel that uses the temporal features directly learned from
temporal filters. To accomplish this, DTW is used to extract

similarity information from a temporal pattern input, as seenin Fig. 1.
During forward propagation, the DTW is calculated between the input and the temporal filter. Given the input
treated as the sample s and the filter as the prototype p,
the DTW-distance is determined by the value of Eq. (2)at D(I, J). DTW replaces the standard mechanism for
combining the weights and outputs of the previous layer
in Eq. (3), thus the activation function φ(·) is passed theresult of DTW. Due to the characteristic of DTW where agreater DTW-distance equates to a larger difference between
temporal patterns, activation function is inversed. The result
was further rescaled as suggested in [21] to values between-1 and 1.
B. Back Propagation through DTW
In the training of a neural network, back propagation is
the step to update the weights of the network to minimize
the error of the network. To adapt the temporal filters tominimize the error of the DTW-NN, we must consider backpropagation through DTW. Therefore, the derivative of theerror function with respect to DTW is computed, specifically,using the chain rule we find∂E ∂zDTW∂E=∂WDTW∂zDTW ∂WDTW(4)

where E is the loss function of the output, WDTW is theparameters of the DTW filter, and zDTW is the result of the∂zDTW
is the relationship between the elements
DTW filter. ∂W
DTWof each sample and the result of DTW, namely, it is thederivative of the DTW function between the temporal filter
p and the input s, or DTW0 (p, s).
As detailed in Eq. (1), DTW is the sum of the distances
between each pairwise element match of the sequences. In
the example of two-dimensional coordinate sequences, eachelement pi = (xpi , ypi )T in the prototype sequence p has amatching element sq = (xsq , ysq )T in the sample sequences, and the DTW is the sum of the distance measures between
them. The derivative of DTW with respect to (xpi , ypi )Tbecomes∂DTW(p, s) =∂(xpi , ypi )TX∂
xp − xs

DTW0 (p, s) =i

(xsq ,ysq )T ∈Q∂(xpi , ypi )Tqypi − ysq.

(5)

DTW can be calculated with various distance measures.
The experiment assumes a Euclidean distance as the distancemeasure between elements. The derivative of the Euclidean
distance with respect to each parameter gives the partial
derivatives:
xpi − xsq∂
xpi − xsq=pypi − ysq
∂xpi
(xpi − xsq )2 + (ypi − ysq )2
(6)

∂
∂ypi

xpi − xsq
ypi − ysq=pypi − ysq
(xpi − xsq )2 + (ypi − ysq )2.

(7)This assumes a two-dimensional task but it can be solved ina similar manner for any number of dimensions.
Finally, using the derivative of the Euclidean distance withrespect to every coordinate (xpi , ypi )T in p, we can calculate
the derivative of DTW, Eq. (5), and substitute it into Eq. (4)to calculate the gradient of the error function with respect
to DTW. This gives us the ability to adjust the DTW filter
to reduce the value of the error function by moving each of
the coordinates by a learning rate of η,




∂E
x pi
x pi
.
(8)
←
−η
ypi
ypi
∂WDTW
V. O N - LINE H ANDWRITTEN D IGIT R ECOGNITION WITH
DTW-NN
Isolated on-line handwriting provides us with an ideal
source of temporal patterns because while there is a highvariation between patterns, they can still be classified into theten distinct classes. This high variation can even exist withinclasses, due to stroke order, stroke direction, connectivity,
serif, etc.
A. Data Set
The experiment was done using the Unipen on-line handwritten digit data set, UNIPEN multi-writer 1a. The data set
consists of about 15,000 isolated handwritten digit patterns
in ten digit classes. It was collected by the International
Unipen Foundation (iUF), which is a popular source forbenchmark data sets for writer identification and handwriting
recognition [22].
The data sets were split into four subsets of randomly
selected patterns. The training data set, validation data set,
and test data set used 11,450 patterns, 200 patterns, and1,300 patterns respectively. A final subset of 50 patterns was
set aside as initial temporal filters for a digit initialization
evaluation. Using this division in the data set, we trained theneural network for 10-class classification. The total numberof patterns used is smaller than the full set to have an equal
representation of each class. The patterns were chosen at
random and no cleaning was done.
B. DTW-NN Architecture
To tackle the digit recognition task, we employed a
network with four weighted layers consisting of one DTW
layer, two fully-connected layers, and the fully-connected
output layer. The DTW layer was designed as per Section IV
with 50 temporal filters nodes of 50 sequence elements
each. For the two fully-connected layers, we used a tanh
activation function with an initial bias of 0. The two fullyconnected layers were made of 400 and 100 nodes respectively. The final output layer used a 10-node softmax

function to normalize the output, representing the probabilityof each of the class labels.
The network was trained with batch gradient decent using
a batch size of 200 digits over the course of 40,000 training
rounds. A constant learning rate of 0.0001 was used forthe fully-connected layers and a larger learning rate of 0.1
was used for the DTW layer. Different learning rates wererequired because of the difference in magnitude between theelements in the temporal patterns and the weights between
the other layers.
VI. E XPERIMENTAL R ESULTS
A. Evaluation of the Proposed Method
For the evaluation of the proposed method on ten-classclassification with on-line handwritten digits, we considered
the following evaluations:• Proposed Method - DTW-NN (Gaussian Init.): The
proposed DTW-NN architecture was used. The initial
DTW filters were constructed from a series of randomly
generated points. The points were selected from anisotropic Gaussian with a mean of (65, 65) and acovariance of 195, which puts them in the general rangeof the patterns in the data set. This evaluation represents
a random weight initialization.
• Proposed Method - DTW-NN (Digit Init.) This trial alsouses the same DTW-NN architecture as the proposed
method with Gaussian initialized filters. The differenceis the initialization of the starting DTW filters. Insteadof randomly generated temporal patterns, the filters
were randomly pre-selected from the data set prior to
the division of subsets. By pre-selecting patterns, westart this weight initialization with structure similar tothe data set.
• Raster Image CNN: For this evaluation, the handwritten
digits were rasterized into (56×56) pixel single channel
images. A standard image-based CNN was built using
the same activation functions and the same number oflayers and nodes as the proposed method.
• 1-NN with DTW: This is a baseline evaluation using
the using k-NN where k is equal to 1. DTW is used as
the distance measure between patterns. Unlike the other
trials with trained models, testing was done between theentire test set and entire training set.
• DAG-SVM-GDTW [20]: Results reported by Bahlmann,
et al. using an SVM with DTW embedded in an RBF
kernel. The reported results use 40% of the UNIPEN
data set for training and 40% for testing.
• HMM [23]: Results reported by Hu, et al. using an
Hidden Markov Model (HMM). The data set was split
2/3 training and 1/3 testing. Note, the data set wascleaned and mislabeled data (about 4% of the total data)
was removed prior to the experiment.
1 This result reflects a data set with mislabels removed.

Table IA TABLE OF COMPARISONS OF THE EVALUATED METHODS IN
RECOGNITION RATE AND AVERAGE PER DIGIT CLASSIFICATION TIME .

Evaluation
DTW-NN (Gaussian Init.)
DTW-NN (Digit Init.)
Raster Image CNN
1-NN with DTW
DAG-SVM-GDTW [20]
HMM [23]

Recognition Rate
96.8%
96.8%
95.6%
98.5%
96.2%
96.8%1

Ave. Per Digit Time
0.3s
0.3s
–
77.6s
–
–

For DTW-NN (Gaussian Init.), DTW-NN (Digit Init.), and1-NN evaluations, the average classification time of a single
digit was recorded using a system with Intel Xeon E5 2.6
GHz processor.
The results of the evaluations are shown in Table I. The
proposed method achieved a 10-class classification accuracyof 96.8% and 96.8% for Gaussian initialization and random
digit initialization respectively. While 1-NN with DTW
outperformed all other methods in accuracy, the slow single
digit classification time of 77.6 seconds is unreasonable
and not suitable for real time or large data applications. In
comparison, the DTW-NN evaluations achieved a high rateof accuracy while taking a small fraction of the time, at 0.3
seconds each. The computational time improvement is due toonly requiring DTW calculations between the inputs and thelearned filters rather than the entire training set as requiredin 1-NN. The DTW-NN methods had the best recognition
rate in comparison to the other trained models. This showsthat the proposed method is both an efficient and effectivemodel for temporal pattern classification.
In addition, when comparing the results of the Raster
Image CNN to the DTW-NN, the results show that the proposed method was able to overcome the temporal distortions
and stroke directions issues that are not present in the offline rasterizations of the data. This is important because
not all temporal patterns offer easy rasterizations to imagespace like handwriting does. For instance, high-dimensional
temporal patterns or temporal patterns with independent dimensions might lead to difficulties with conventional imagespace CNNs.
A significant source of the error from the DTW-NN model
comes from mislabels and noise found in the UNIPEN data
set. Figure 2 shows examples of the misclassifications of thetest set on the DTW-NN (Digit Init.) evaluation. Many of theerrors made by the network are reasonable because the digits
were either heavily distorted or ground truth mislabels. The
highlighted patterns in Fig. 2 would be difficult to classify
even for humans. It is interesting to note, even though thetwo evaluations had different starting filters and different
initial layer weights, both networks had difficulty with manyof the same patterns. 52.4% of the misclassifications wereshared by the two initializations.

0 as 80 as 40 as 80 as 10 as 4

1 as 2

1 as 2

1 as 2

1 as 4

1 as 5

1 as 7

2 as 7

2 as 7

2 as 74 as 04 as 6

4 as 14 as 14 as 14 as 04 as 14 as 94 as 1

5 as 85 as 85 as 86 as 47 as 48 as 98 as 6

8 as 08 as 18 as 18 as 08 as 58 as 18 as 08 as 99 as 1

9 as 39 as 1(a) DTW-NN (Gaussian Init.)(b) DTW-NN (Digit Init.)
Figure 5.
The progression of a sample filter from (a) the DTW-NN(Gaussian Init.) training and (b) the DTW-NN (Digit Init.) training. The topleft frame of each subfigure is the initial filter pattern and each subsequent
frame from left to right, top to bottom is taken at 2000 round intervals.

Figure 2. All of the misclassified using the DTW-NN (Digit Init.) method
sorted by ground truth label. Each figure is labeled with the ground truth
and the predicted class, where “x as y” means that the ground truth classis given as x and the system classified the input as y. The figures enclosed
in red boxes are patterns that are mislabeled, ambiguous, or noise.
100

Accuracy (%)

90

Figure 6.
rounds.

80

The state of the filters of DTW-NN (Digit Init.) after 39,000

70
60
50
40
0

DTW-NN (Gaussian Init.)
DTW-NN (Digit Init.)
5000

10000

15000

20000

25000

Training Rounds

30000

35000

40000

Figure 3. A graph portraying the accuracy of 10-class classification overnumber of batch rounds of training.

B. Observations on the Filters
The two filter preparations were done to observe the difference between utilizing randomly generated patterns andpatterns with a preset structure. Figure 3 demonstrates thatafter enough time, there was little difference in the methodof initializing the DTW filters. However, the accuracy of theGaussian initialization took longer to stabilize and reach asimilar accuracy rate as initializing the network with preset
digits.

Figure 4. The state of the filters of DTW-NN (Gaussian Init.) after 35,000
rounds.

The initial low accuracy of the Gaussian initialized filters
is due to the randomized structures causing an equally large
distance between the inputs and all the filters. Figure 4
shows that the through back propagation, eventually many of
the Gaussian initialized filters gained structures resembling
shapes and numbers. The delayed convergence is attributed
to the need to unravel the patterns into useful structures withlarger distinctions between the filters. Comparatively, whenproviding the filters with a starting structure, there is noneed to unravel the temporal patterns. This can be seen bycomparing Fig. 5 (a) and (b). It can be observed how quickly
the DTW-NN (Digit Init.) converged compared to the DTWNN (Gaussian Init.). The final filters of both evaluations areshown in Fig. 4 and Fig. 6.
C. Advantages and Disadvantages
DTW-NN excels at classification tasks, even with temporal patterns with low-density information element patterns
and widely varying structures. It is able to generalize thetemporal patterns based on the temporal features efficiently.
Execution time is much faster than exhaustive searches.
DTW-NN is suitable for accurate real-time on-line handwriting recognition.
The filters of the DTW-NN allow the network to overcome
patterns of varying length. However, the size and shape of
the initial filters must be considered in the design of the
network. While there is no upper limit, the slope constraint
given by Eq. (2) limits the input patterns from being lessthan half the size. However, alternate slope constraints canbe used introducing different advantages and limitations.
Also, when training, while the end accuracy is nearlyindistinguishable, it is suggested to use designed patterns
than randomly generated patterns for faster training.
VII. C ONCLUSION
This paper presented DTW-NN as a method of classifying isolated unnormalized temporal patterns. The proposed DTW-NN uses DTW as a kernel-like function to
be used as a dissimilarity-based activation. This method
offers the network the ability to learn temporal featuresby optimizing temporal filters through the back propagation
through DTW. When classifying isolated on-line handwritten
digits in multi-class classification with the UNIPEN dataset, the experimental results show a high accuracy of 96.8%
while having a very fast execution time. The results of theexperiment show that DTW-NN can be an efficient methodof implementing neural network models with variable size
temporal patterns.
This work lays the foundation for using neural networks with temporal kernels. Future work can be done by
expanding on this framework. For instance, by stacking
multiple DTW-based temporal filter layers and creating a
deeper network, it might be possible to learn higher levelfeatures. Also, additional improvements can be made on the
DTW-NN such as adjusting the hyperparameters, changing
the activations, or adding techniques such as dropout. The
proposed DTW-NN is only the starting point for temporal
kernel-based neural networks.
R EFERENCES[1] M. Müller, “Dynamic time warping,” Information retrieval
for music and motion, pp. 69–84, 2007.
[2] H. Ding, G. Trajcevski, P. Scheuermann, X. Wang, andE. Keogh, “Querying and mining of time series data: experimental comparison of representations and distance measures,”
Proceedings of the VLDB Endowment, vol. 1, no. 2, pp. 1542–
1552, 2008.
[3] T. Rakthanmanon, B. Campana, A. Mueen, G. Batista,
B. Westover, Q. Zhu, J. Zakaria, and E. Keogh, “Searching and mining trillions of time series subsequences underdynamic time warping,” in Proceedings of the 18th ACM
SIGKDD international conference on Knowledge discovery
and data mining - KDD 12. Association for Computing
Machinery (ACM), 2012.
[4] J. Schmidhuber, “Deep learning in neural networks: An
overview,” Neural Networks, vol. 61, pp. 85–117, 2015.
[5] M. Sundermeyer, R. Schlüter, and H. Ney, “Lstm neuralnetworks for language modeling.” in INTERSPEECH, 2010.
[6] A. Graves, A. rahman Mohamed, and G. Hinton, “Speech
recognition with deep recurrent neural networks,” in 2013
IEEE International Conference on Acoustics, Speech andSignal Processing.
Institute of Electrical & Electronics
Engineers (IEEE), may 2013.
[7] A. Graves, M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke,
and J. Schmidhuber, “A novel connectionist system for unconstrained handwriting recognition,” IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp.
855–868, may 2009.

[8] A. Graves and J. Schmidhuber, “Offline handwriting recognition with multidimensional recurrent neural networks,” inAdvances in neural information processing systems, 2009, pp.
545–552.
[9] H. Jaeger, Tutorial on training recurrent neural networks,
covering BPPT, RTRL, EKF and the” echo state network”
approach, 2002.
[10] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, nov
1997.
[11] A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. J.
Lang, “Phoneme recognition using time-delay neural networks,” Acoustics, Speech and Signal Processing, IEEE
Transactions on, vol. 37, no. 3, pp. 328–339, 1989.
[12] M. C. Mozer, “A focused back-propagation algorithm for
temporal pattern recognition,” Complex systems, vol. 3, no. 4,
pp. 349–381, 1989.
[13] Y. LeCun and Y. Bengio, “Convolutional networks for images,
speech, and time series,” The handbook of brain theory andneural networks, vol. 3361, no. 10, p. 1995, 1995.
[14] A. Yuan, G. Bai, L. Jiao, and Y. Liu, “Offline handwritten
english character recognition based on convolutional neuralnetwork,” in Document Analysis Systems (DAS), 2012 10th
IAPR International Workshop on. IEEE, 2012, pp. 125–129.
[15] Y. Zheng, Q. Liu, E. Chen, Y. Ge, and J. L. Zhao, “Time seriesclassification using multi-channels deep convolutional neuralnetworks,” in Web-Age Information Management. Springer
Science + Business Media, 2014, pp. 298–310.
[16] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet
Classification with Deep Convolutional Neural Networks,”
in Advances in Neural Information Processing Systems 25,
F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger,
Eds. Curran Associates, Inc., 2012, pp. 1097–1105.
[17] E. Pekalska and R. P. Duin, The dissimilarity representation
for pattern recognition: foundations and applications. World
Scientific, 2005.
[18] W. Chaovalitwongse and P. Pardalos, “On the time series
support vector machine using dynamic time warping kernel
for brain activity classification,” Cybernetics and Systems
Analysis, vol. 44, no. 1, pp. 125–138, 2008.
[19] H. S. K.-i. Noma, “Dynamic time-alignment kernel in support
vector machine,” 2002.
[20] C. Bahlmann, B. Haasdonk, and H. Burkhardt, “Online handwriting recognition with support vector machines-a kernel
approach,” in Frontiers in handwriting recognition, 2002.
proceedings. eighth international workshop on. IEEE, 2002,
pp. 49–54.
[21] Y. A. LeCun, L. Bottou, G. B. Orr, and K.-R. Müller,“Efficient backprop,” in Neural networks: Tricks of the trade.
Springer, 2012, pp. 9–48.
[22] International Unipen Foundation, “Int. Unipen Foundation iUF,” http://www.unipen.org/home.html, accessed: 2015-1019.
[23] J. Hu, S. G. Lim, and M. K. Brown, “Writer independent online handwriting recognition using an hmm approach,” Pattern
Recognition, vol. 33, no. 1, pp. 133–147, 2000.