## 3 Methodology
### Model Modifications for Contrastive Learning  
To be trained with the contrastive learning framework, the structure of the existing model needs to be modified to use CLL. We modified two representative models, the RNN-AE model proposed by [PERSON] et al. [21], which we call the RNN model in this paper, and the DeepAnT model proposed by [PERSON] et al. [24]. The former is an estimation model, and the latter is a prediction model as defined by [PERSON] [20]. The modifications of these two models are shown in Figure 3.  
The RNN-AE model uses two RNN sequences as the encoding and decoding parts of the autoencoder. This model inputs the real behavior sequences and outputs the reconstructed behavior sequences in reverse order. Since the data are unlabeled, an autoencoder  
Figure 3: Modifications for RNN-AE and DeepAnT to be trained contrastively. There are two types of RNN sequences in the original RNN-AE, but only one after the modification. \(h_{e}\) and \(h_{d}\) are the hidden states of the RNN of the encoder and the decoder, respectively.  
is a common scheme for training on unlabeled data. Therefore [PERSON] et al. proposed a model based on RNN using the loss function of reconstruction. Thus, during training, the loss function makes the output and input sequences as close as possible. During detection, the RNN-AE model calculates the difference between the reconstructed and original sequences. The greater the difference between the two sequences is, the higher the probability that the sequence contains anomalous behavior. The architecture of the autoencoder adopted by the model is designed to introduce the reconstruction loss function of the autoencoder. While contrastive learning is another method to provide loss functions on unlabeled data, our RNN model can be trained properly without the reconstruction loss function. Therefore, we simplified the RNN-AE model by removing the decoding part of the model and reducing it to a simple RNN sequence. In our modified RNN model, the last RNN unit is followed by a classifier to determine whether the last action in the input behavior sequence is anomalous. When training our model, the positive and negative samples generated by contrast learning are input separately, and the model can be trained according to the CLL function Equation (3).  
Unlike the RNN-AE model that provides the loss function through an autoencoder, DeepAnT uses a prediction loss function. With the input of previous behavior sequences, the model uses a CNN to extract temporal features from the user behavior sequence and predict the behavior at the next moment. The loss function of DeepAnT calculates the distance between the actual and the predicted behavior. This is also a common loss function on unlabeled data. When using our comparison learning loss function instead, it is necessary to change the output of DeepAnT into an anomaly classifier and adjust the input of DeepAnT to include the current action. Subsequently, the modified DeepAnT model can be trained in a similar manner to our modified RNN model.  
Training a neural network requires backpropagation for a particular value and calculating a loss function. Since the data are unlabeled, existing models use various loss functions to train on it. Contrastive learning provides one of these loss functions, which is not different from the other methods in this respect.