Computer Security Symposium 201611 - 13 October 2016

Detecting Fraudulent Behavior Using Recurrent
Neural Networks
Yoshihiro Ando1,2,a),b)

Hidehito Gomi2,c)Hidehiko Tanaka1,d)Abstract: Due to an increase in illegal accesses to Internet services, detecting and preventing them has
become important. To prevent an illegal access, not only rule-based techniques but also machine learningtechniques have been used. In the field of security, such as malware detection, machine learning techniques
are used to learn behavioral patterns and detect those that are suspicious. However, there are few studies
targeting the behavioral patterns of a malicious user engaged in fraudulent acts on Internet services. In thispaper, we propose the approach which uses the patterns in web access logs to detect fraudulent behaviors.
We apply and evaluate a recurrent neural network (RNN) to recognize these behaviors. Our result indicatesthat RNN is very eﬀective for fraudulent behavior detection.
Keywords: Fraud Detection, Machine Learning, Recurrent Neural Network

1.

Introduction

According to a survey conducted by the Center for Strategic & International Studies, at least $375 billion is lost globally each year as a result of cybercrime [11]. This survey
includes indirect costs, such as the leakage of personal information and intellectual property theft. Counting only thoselosses caused by online credit card fraud, another survey
concluded that at least $4.2 billion was lost in 2010 [1]. Insuch situations, several machine learning techniques, suchas neural networks, have been applied to detect and prevent
credit card fraud [13].
The previous studies of credit card fraud used transaction
logs instead of the entire logs left by a malicious user on thetargeted site. And these studies were not intended to model
the behavior of malicious users. In adition, credit card logs
are so sensitive that they are not easy to use.
Considering web logs as time series data, there are several
techniques with which they can be modeled: Hidden Markov
models (HMMs), state space models (SSM), and recurrent
neural networks (RNN) among them. Of these, the RNN ispromising in the context of deep learning. RNNs are used formodeling sequential data, such as natural language, speech,
and handwritten characters. We consider that RNNs havealso been eﬀective in modeling behavioral patterns using web
server logs.
This paper proposes a novel machine learning approachto detecting online credit card fraud through fraudulent be1
2
a)
b)
c)
d)

Institute of Information Security
Yahoo Japan Corporation
dgs165102@iisec.ac.jp
yoandou@yahoo-corp.jp
hgomi@yahoo-corp.jp
tanaka@iisec.ac.jp

haviors. After analyzing web server log data from users who
attempted illegal purchases, provided by Yahoo JAPAN, wefound a characteristic pattern in the log. The behavior of
the malicious user who intends to use a stolen credit card
number to buy prepaid cards, such as Amazon gift cards,
contains search trials in the auction service. We applied an
RNN to the web log data left by a malicious user who tried
to use a stolen credit card on an online auction service andevaluated its eﬀectiveness. As a result, we obtained a modelthat could detect a malicious user who intended to use astolen credit card to buy prepaid cards.

2.

Related WorkThe RNN is a kind of artificial neural network that has
several subtypes. One basic type of RNN is called a Hopfield network, which was developed by John Hopfield in 1982
[9]. A Hopfield network is not generic; it has a symmetricalstructure, but cannot process sequential data.
The Elman network is another type of RNN, developed by
Jeﬀ Elman in 1990 [4]. This network type consists of 3 layers (Input, Hidden, and Output) with circular connections
between units in the hidden layer. These connections takethe output value of the previous time from the unit in thesame hidden layer as input of the current time. This network structure reflects the reciprocal influence of sequential
ordered data.
The Jordan network is another type of RNN, which wasdeveloped by Michael I. Jordan in 1986 [10]. The Jordan
network is similar to the Elman network, except that its
recurrent connections are between units in the input andoutput layers.
Elman networks have generally been shown to be superior to Jordan networks. Thus, in this paper, our approach－ 805 －

is based on Elman networks.
There are some applications of RNN in the field of natural language processing. Bengio et al. use an RNN to learn
the context of sentence. Their proposed approach resulted
in better precision than traditional the n-gram model [2].
Mikolov et al. also use RNN for learning the context ofphrases and compared it to the n-gram model [12].
Speech recognition is also an important application of
RNNs. Graves et al. use an RNN with long short-term
memory (LSTM) to model spoken language [6]. LSTM is a
special processing unit in the RNN hidden layer developedby Hochreiter and Schmidhuber in 1997 [8]. LSTM can take
important parts of the sequence data and dismiss data thatare not important. This feature facilitates convergence inthe training phase and avoids the vanishing and exploding
gradient problems.
Handwritten character recognition is another field towhich RNNs have been successfully applied. Graves et al.
use an RNN for on-line handwriting recognition. They train
the RNN for sequence labeling to track the movement of
the pen. Their proposed approach recorded better precision
than commonly used HMM approaches [5]. Graves et al.
also use an RNN for oﬀ-line handwriting recognition [7].
In the security literature, RNN applications are few.
Sheikhan et al. use an RNN to improve IDS detection precision. They use a customized RNN, called a reduced-size
RNN, for misuse-based IDS [15].
As a measure for credit card fraud detection, there are several studies that use ordinary feed-forward neural networks,
including that by Patidar et al. [13]. However, literatureusing RNNs for this purpose does not exist to the best of
our knowledge.
In the security research field, particularly in malware detection, behavior-based approaches have recently been used.
Burguera et al. collect behavioral information from the Android platform using crowdsourcing and apply a k-meansclustering approach to detect malware [3]. Rieck et al. use
machine learning techniques of clustering and classification
to learn the behavior of malware and to detect it automatically [14]. However, these researches are not intended todetect the behavior of malicious users, but of malicious software. Furthermore, these research do not model the orderof actions, in contrast with our approach.

3.

series data. Many epochs cause diﬃculty in learning, as indeep learning. Therefore, several techniques, such as LSTM,
which is detailed later, are used to avoid problems like vanishing and exploding gradients.
3.1 RNNA typical RNN has three layers: input, output, and hidden layers of fully connected units. Figure 1 and 2 depicts
this basic structure, which is adopted for our RNN.

Fig. 1RNN Structure

Our Approach

Our RNN consists of 3 layers: input, hidden, and output
layers. The number of units in the input layer varies according to the features we use. The output layer has twounits that each represents a possible classification. The parameters of the RNN are learned using a gradient descentmethod with backpropagation similar to that of feed-forward
neural networks. For RNNs, there are two major learning algorithms: backpropagation through time (BPTT) [16] andreal-time recurrent learning (RTRL) [17]. We use BPTT
because it has a simple structure and learns fast.
It is diﬃcult to train an RNN on a large amount of time

Fig. 2

RNN Hidden Layer Structure. t is the current time andt − 1 is the previous time. These two layers are the same.

3.2 Data Encoding
Neural network uses mathematical formulas internally.
Consequently, target data must be expressed as a numericalvalue. We use web access logs to detect fraudulent behavior, which are generally recorded as text data. Therefore,－ 806 －

we must encode the logs with numerical values before processing.
One approach to encoding text data for machine learning
is based on the presence or absence of a given text element.
Assuming that the existence of a specific word in the target
text is one feature, we can represent it in binary. For example, if the text foo exists in the target text, we canencode the existence of the word foo as 1. Figure 3 shows
our approach to encoding web access logs.

3.3 Learning
We use BPTT to train our network. Figure 4 shows the
BPTT algorithm overview.

Fig. 4∂E
BPTT. t is the current time. ∂ut is the partial diﬀerential
jof the error function with the input value to the jth unit∂Ein the hidden layer. ∂v
t is the partial diﬀerential of thejerror function with the input value to the kth unit in the
output layer. w is the parameter matrix.

Our goal in training an RNN is minimizing Eq. 1. n is
the index for each data point in the dataset and t is thetime step. k is the index of the unit in the output layer. w
is a parameter matrix containing connection weights. d isthe expected value of the specific unit corresponding to the
input data. xn is the input values vector of nth input.
∑∑∑ t
E(w) = −dnk log ykt (xn ; w)
(1)nFig. 3

Data Encoding. Table A has original logs. Access Timeis the time that the user accessed. Unique Cookie ID isthe value of the browser cookie issued uniquely to eachbrowsers. Destination Host Name is the host name towhich the user accessed. Table B has conversion data
which converts the host name to the corresponding number of the feature field. Table C is the result of data
encoding.

Table A contains a section of the web access log, featuring access date and time, a unique cookie ID, and the destination host name, from left to right. These are listed inascending order by time. Table B shows a corresponding listof host names and feature numbers. Using Tables A and B,
we can obtain the training data. For example, in Table A,
the 3rd row has a value of ‘auction.yahoo.co.jp’ for the targeted host and so the corresponding column of this feature
is set to 1 in Table C, and the remaining columns related toother features are set to 0.

tk

In the feed-forward phase, the input value of the units in
the hidden layer is represented by Eq. 2. In Eq. 2, utj is the
input value of the jth unit of the hidden layer at time stept. i is the index of the unit in the input layer. wji is theweight of the connection between the ith unit of the input
layer and the jth unit of the hidden layer. xti is the input
value of the ith unit in the input layer at time step t. j ′ isthe index of the unit in the hidden layer, used to distinguishit from j. wjj ′ is the weight of the recurrent connection
between units in the hidden layer. zjt−1
is the output value′of the j ′ th unit in the hidden layer at time step t − 1.
∑
∑
utj =
wji xti +wjj ′ zjt−1
(2)
′
j′

i

We use a sigmoid function as the activation function ofunits in the hidden layer. The sigmoid function is represented by Eq. 3:
f (x) =

1
1 + e−x(3)

By using a sigmoid function, we obtain Eq. 4 as the output value of the jth unit in the hidden layer.

－ 807 －

zjt = f (utj )(4)

We can then obtain the input value to the unit in theoutput layer by Eq. 5. wkj is the weight of the connection
between the jth unit in the hidden layer and kth unit in the
output layer. zjt is the output value of the jth unit in thehidden layer at time step t.
∑
vkt =wkj zjt
(5)

=

T
∑
∂E

∂utj
t=1

ut−1
j

(10)

Similarly to the above, the partial diﬀerential of the error
function with the connection weight between the unit in thehidden and output layers is represented as:j

Eq. 6 conducts a classification. ykt is the output of thekth unit in the output layer at time step t. vkt is the input
value to the kth unit in the output layer at time step t. L is
the number of units in the output layer. We can determine
which is the most likely class by the ouput value of the unit
in the hidden layer.
exp(vkt )ykt = ∑Lt
l=1 exp(vl )(6)

Backpropagation starts with calculating the partial diﬀerential of the error function with the input value of the unit
in the output layer. Eq. 7 shows the value of the partial
diﬀerential of the error function with the input value of theunit in the output layer.
∂E= ykt − dtk
∂vkt

T
∑
∂E
∂E ∂vkt=∂wkj∂vkt ∂wkj
t=1

=

T
∑
∂E

∂vkt
t=1

(11)

We then subtract the diﬀerence from the old weight using
these diﬀerences and set as the new connection weight. Updating weight by gradient descent is represented as Eq. 12,
13, and 14.
newwji= wji − ε

∂E
∂wji

(12)

∂E
∂wjj ′

(13)

∂E
∂wkj

(14)

new
′
wjj
= wjj ′ − ε

(7)

new
wkj
= wkj − εvkt is the input value of the kth unit in the output layer
at time step t. dtk is the target value of the kth unit in the
output layer at time step t. The target values are one or zero
specified in the training data. The value one means the unit
should be the best unit represents that the data is fraudulent or not. For example, if the 0th unit is corresponded to
the label of fraudulent and the input data is fraudulent, the0th unit should have value one. The value zero means theunit should not be the best unit.
The partial diﬀerential of the error function with the input value of the jth unit in the hidden layer at time step t
is written as:
(
)
∑
∑
∂E
∂E
∂E
(8)
=
wkj t +
wj ′ j t+1 f ′ (utj )
∂utj
∂vk
∂uj ′
j′
k

zkt

new
wji
and wji are the updated and current value of theconnection weight between the units in the input and hidden
newlayers. wjj
and wjj ′ are the updated and current value′of the recurrent connection weight between the units in thenewhidden layer. wkj
and wkj are the updated and current
value of the connection weight between the units in the hidden and output layers. ε is called a learning coeﬃcient.
3.3.1 LSTM
The LSTM was developed by Hochreiter and Schmidhuber in 1997 [8]. It can ease the diﬃculty of learning longsequences of data when using backpropagation and can also
suppress the vanishing and exploding gradient problems.

The partial diﬀerential of the error function with the connection weight between the units in the input and hidden
layers is represented as:Tt∑∂E∂E ∂uj=∂wji∂utj ∂wji
t=1

=

T
∑
∂E

∂utj
t=1

xti

(9)
Fig. 5The partial diﬀerential of the error function with the connection weight of the internal connections in the hidden layer
is represented as:Tt∑∂E∂E ∂uj=∂wjj ′∂utj ∂wjj ′t=1

LSTM. Memory cell has the value of the previous time
step. Forget gate determines whether the previous value
is used or not.

The internal structure of the LSTM is shown in Figure
5. We used the LSTM as a control against which we could
evaluate our RNN. The result of this evaluation is discussed
in the following section.
－ 808 －

4.

Evaluation

4.1 Dataset
Web access logs provided by Yahoo JAPAN were used as
an evaluation dataset. These logs are those of both malicious and normal users. The logs were recorded from April
29, 2016 to May 5, 2016. Malicious users had stolen credit
cards and tried to buy prepaid cards at Yahoo Auction.
Table 1of the value of the error function was set to 0.5. Training
was to be discontinued when the value of the error function
was less than the threshold. The learning coeﬃcient was
set to 0.00001. The weight of the connection was initialized
by the random number according to mean 0 and standard
deviation 0.001.
4.4

Value of Error Function

Number of unique browsers visited to Yahoo JAPANDate
April 29, 2016
April 30, 2016
May 1, 2016
May 2, 2016
May 3, 2016
May 4, 2016
May 5, 2016

Table 2Label
Postive
Negative

Number of browsers
95,123,327
92,878,076
92,141,013
98,050,854
94,517,378
93,257,615
93,291,749

Training dataset provided by Yahoo JAPAN
Number of browsers
11300

Number of logs
9,286
132,853
Fig. 6

Table 3

Test dataset provided by Yahoo JAPANLabel
Postive
Negative

Number of browsers
10300

Number of logs
7,577
139,001

Table 1 shows the number of unique browser visited toYahoo JAPAN from April 29, 2016 to May 5, 2016. Table
2 and 3 show the details of our dataset. We chose randomly from all visits except that of the fraudulent user andregarded them as negative data, to save computational resources. Positive data consisted of the web access logs ofmalicious users. This number was so small that training
was hindered by class imbalance. The class imbalance problem is common in machine learning, where the amount ofdata from one class far exceed that of the other. In thissituation, learning tends to fail; however, it is often the casein the security field that we cannot obtain a suﬃcient number of positive data examples. Therefore, it is often important that we properly model fraudulent behavior even usingclass-imbalanced data.

Figure 6 shows the value of the error function as the number of times of training. There are sharp increases in placesin the figure 6. These increases indicate that the failures of
the gradient descent occured. However, the value of the error function finally converged after 3,875 backpropagations.
The training took 17 hours and 56 minutes and 21 seconds.
4.5 Results
Table 4 shows the results of our evaluation. F-measures
were calculated using the training dataset and the test
dataset described in the section 4.1. We used two typesof RNN, RNN without LSTM and RNN with LSTM, withsigmoid units. For this evaluation, we used not only two
types of RNNs but also two types of SVM, with linear andradial basis function kernel types, respectively. The results
indicate that an RNN can accurately detect malicious behavior better than an SVM. Althogh two types of RNN had
the same accuracy, RNN with LSTM converged faster than
RNN without LSTM.

4.2 Test Environment
Our testing environment was a CentOS 7.2.1511 virtual
server with a Intel Xeon 1.80GHz processor and 8 GB ofmemory. The programming language was JavaScript running on Node.js version 6.3.1.
4.3 Training Conditions
Our RNN structure was consisted of 153 units in the input
layer, 14 units in the hidden layer, and 2 units in the output
layer. Because the number of the positive data is less than
the number of the negative data, the number of the units
in the input layer was decided to correspond to the number of the host names in the positive data. The threshold

Value of error function

Table 4 Evaluation Result
Algorithm
F-measure
RNN (without LSTM)
1.0000RNN (with LSTM)
1.0000
SVM (linear kernel)
0.5967
SVM (RBF kernel)
0.5844

5.

Conclusions

Our evaluation results indicate that an RNN is superior toother common machine learning techniques, such as SVMs,
for learning a malicious user’s behavior using web accesslogs.

－ 809 －

6.

Future Work

Although our approach has been proven to be eﬀective,
the factor that most contributes to this result has not yet
been determined. We will investigate the relationship between patterns of malicious behavior and the internal structure of our RNN. A current hypothesis is that there areclear diﬀerences between the behaviors of legitimate andmalicious users. Further research can make these diﬀerences
clear using the internal parameter values of our neural network during training.
In this paper, we set as our target the user who attempts
to use stolen credit cards to buy prepaid cards on an Internet auction site. We would like to generalize our approachin order to apply it to other fraudulent behaviors.

7.

[17]

Acknowledgment

We would like to thank Yahoo JAPAN for providing usefuldatasets and computing resources.
References
[1]

[2]
[3]

[4]
[5]

[6]

[7]

[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]

Anderson, R., Barton, C., Böhme, R., Clayton, R.,
Van Eeten, M. J., Levi, M., Moore, T., and Savage, S.
Measuring the cost of cybercrime. The economics of information security and privacy (2013), 265–300.
Bengio, Y., Ducharme, R., Vincent, P., and Jauvin, C.
A neural probabilistic language model. journal of machinelearning research 3, Feb (2003), 1137–1155.
Burguera, I., Zurutuza, U., and Nadjm-Tehrani, S.
Crowdroid: behavior-based malware detection system for android. In Proceedings of the 1st ACM workshop on Security and privacy in smartphones and mobile devices (2011),
ACM, pp. 15–26.
Elman, J. L. Finding structure in time. Cognitive science14, 2 (1990), 179–211.
Graves, A., Liwicki, M., Bunke, H., Schmidhuber, J.,
and Fernández, S. Unconstrained on-line handwriting
recognition with recurrent neural networks. Advances in
Neural Information Processing Systems (2008), 577–584.
Graves, A., Mohamed, A.-r., and Hinton, G. Speech
recognition with deep recurrent neural networks. In 2013
IEEE international conference on acoustics, speech and signal processing (2013), IEEE, pp. 6645–6649.
Graves, A., and Schmidhuber, J. Oﬄine handwriting
recognition with multidimensional recurrent neural networks.
Advances in neural information processing systems (2009),
545–552.
Hochreiter, S., and Schmidhuber, J. Lstm can solve hardlong time lag problems. Advances in neural information processing systems (1997), 473–479.
Hopfield, J. J. Neural networks and physical systems withemergent collective computational abilities. Proceedings ofthe national academy of sciences 79, 8 (1982), 2554–2558.
Jordan, M. I. Attractor dynamics and parallelism in a connectionist sequential network. In Proceedings of the Eighth
Annual Conference of the Cognitive Science Society (1986).
Losses, N. Estimating the global cost of cybercrime.
McAfee, Centre for Strategic & International Studies (2014).
Mikolov, T., Karafiát, M., Burget, L., Cernockỳ, J.,
and Khudanpur, S. Recurrent neural network based language model. In Interspeech (2010), vol. 2, p. 3.
Patidar, R., and Sharma, L. Credit card fraud detection
using neural network. International Journal of Soft Computing and Engineering (IJSCE) 1, 32-38 (2011).
Rieck, K., Trinius, P., Willems, C., and Holz, T. Automatic analysis of malware behavior using machine learning.
Journal of Computer Security 19, 4 (2011), 639–668.
Sheikhan, M., Jadidi, Z., and Farrokhi, A. Intrusion detection using reduced-size rnn based on feature grouping.
Neural Computing and Applications 21, 6 (2012), 1185–1190.
Werbos, P. J. Backpropagation through time: what it does

－ 810 －and how to do it. Proceedings of the IEEE 78, 10 (1990),
1550–1560.
Williams, R. J., and Zipser, D. A learning algorithm forcontinually running fully recurrent neural networks. Neuralcomputation 1, 2 (1989), 270–280.