A Siamese Neural Network for Behavioral Biometrics Authentication
Authors: Jesús Solano
Esteban Rivera
Alejandra Castelblanco
Lizzy Tengana
Christian Lopez
Martin Ochoa
Venue:  (2021)
Keywords: Deep Learning
Few-shot Learning
Behavioral Biometrics
Biometric Authentication

Abstract: The raise in popularity of personalized web and mobile applications brings about a need of robust authentication systems. Although password authentication is the most popular authentication mechanism, it has also several drawbacks. Behavioral Biometrics Authentication has emerged as a complementary risk-based authentication approach which aims at profiling users based on their behavior while interacting with computers/smartphones. In this work we propose a novel Siamese Neural Network to perform a few-shot verification of user's behavior. We develop our approach to identify behavior from either human-computer or human-smartphone interaction. For computer interaction our approach learns from mouse and keyboard dynamics, while for smartphone interaction it learns from holding patterns and touch patterns. We show that our approach has a few-shot classification accuracy of up to 99.8% and 90.8% for mobile and web interactions, respectively. We also test our approach on a database that contains over 100K different web interactions collected in the wild.

Under review as a conference paper at ICLR 2021
A SIAMESE NEURAL NETWORK FOR BEHAVIORAL BIOMETRICS AUTHENTICATION
Anonymous authors Paper under double-blind review
ABSTRACT
The raise in popularity of web and mobile applications brings about a need of robust authentication systems. Although password authentication is the most pop- ular authentication mechanism, it has several drawbacks. Behavioral Biomet- rics Authentication has emerged as a complementary risk-based authentication approach which aims at profiling users based on their interaction with comput- ers/smartphones. In this work we propose a novel Siamese Neural Network to perform a few-shot verification of user's behavior. We develop our approach to authenticate either human-coe cuman-computer or suman-smartphone interaction. For com- puter interaction our approach learns from mouse and keyboard dynamics, while for smartphone it learns from holding patterns and touch patterns. We show that our approach has few-shot classification accuracy of up to 99.8% and 90.8% for mobile and web interactions, respectively. We also test approach database that contains over 100K different web interactions collected in the wild.
INTRODUCTION
Biometric authentication has emerged as complement to traditional authentication systems. The main advantage of such systems is that they rely on user's information that can not casily be stolen or crafted. Most active fields of biometric authentication in academia and industry are related to face authentication or fingerprint authentication, with a recent increase in interest on behavioral bio- metries. Behavioral Biometrics authentication refers the use of buman--evice interaction features to grant access to specific service. This interaction could include, but is not limited to, typing patterns, mouse dynamics, smartphone holding patterns, voice recognition, gait recognition etc.
Machine learning algorithms have been proposed to verify users identity using behavioral biometrics interaction),
features. Regarding behavioral biometrics in web environments most of the work has focused on the use of Support Vector Machine and Random Forest classifiers to analyze mouse and keyboard interaction (hhan et al., 2018 Solano et al.) 2020). Alternatively, some works have proposed to built-in sensors available in mobile devices (i.e sensors information,
touch interaction etc.) for authentication purposes (Rauen et al., 2018: Rocha et al.] [019 Zhang let al. 2016, Amini al., 2018 Abuhamad al. 2020). However, previous works behavioral biometrics usually have three main drawbacks: (1) they need long interactions (minutes) in order to learn accurately the user behavior; (2) they require ad-hoc interaction challenges; or (3) they need model per user to improve model accuracy.
In this paper, we present Siamese One-Shot Neural Network (SOS-NN) which able to assess a risk score after only one observation (i.e. enrollment behavior) of given user. To achieve this, we propose Siamese Neural Network architecture that assesses whether two behaviors belong to the same user. We present a similar architecture to user verification for both, web and mobile environments. web environments, we create a set of features from raw mouse movements and keyboard strokes. On the other hand, for the mobile environment our SOS-NN analyses features created from touch interaction and motion sensors on the smartphone.
In sum, the contributions of our work are: (1) An approach to user authentication using behavioral biometrics information in an accurate few-shot learning fashion after only seconds of user inter- action; (2) A unified neural network architecture
authenticate user's behavior for both mobile and web environments that is able to achieve an accuracy of up to 99.8% and 90.8% respectively; (3) A framework which is able accurately authenticate users in large scale without requiring to

---

Under review as a conference paper at ICLR 2021
retrain the model for new users; (4) A systematic measurement study understand the impact of the parameters to SOS-NN based on the authentication time window length and the n number in the n-shot test; and (5) A comprehensive in the wild evaluation of our approach in web environments which test SOS-NN over thousands of users from real financial services.
BACKGROUND
In the field of biometric-based authentication, many sources of information have been proposed. These could be physical (facial recognition, fingerprint scanning, retina scan, etc) or behavioral pat- terns (signature verification, mouse dynamics, gait analysis, voice recognition, etc.). Behavioral biometrics has gained increased attention since reproducing the hehavior of legitimate user consti- tutes an unconventional challenge for attackers. Moreover, many behavioral biometric methods do not require specialized hardware, which makes a scalable deployment casier. While substantial ad- vances have been achieved, there still a gap to make such systems widely adopted in practice. We define as 'practical' methods that demand short periods of interaction per user (both for model train- ing and for authentication), and simple architectures that ease deployment and maintenance. This paper proposes novel approach, which complements traditional authentication in web and mobile environments, considering practical implementation characteristics and scalability constraints.
Web Environment.
Multiple previous studies have employed multimodal biometrics in desktop environments to identify user's behaviour (Traore et al. 2012 Bailey et al.] 2014 Fridman et al.
2013 Neha & Chatterjee 2019; Solano et al. (020)). These studies propose to integrate, either the
feature or decision level, information from keyboard interaction, mouse dynamics and others. Most of these studies have evaluated classic ML classification models (e.g. SVM, Naive Bayes, Random forest and J48 algorithms). Few others (Jagadeesan & Hsiao 2009 Khan et al. 2018), have explored the use of shallow neural networks. Remarkably, found that siamese neural architectures had not been studied before in this field. Therefore, in this study, we explore the effectiveness, generalization and applicability of such architectures for behavioral biometries.
Regarding user interaction, we highlight that, in muitiple real-world applications there is a practical limit to the length of the interaction and amount of data that can be collected before deploying an authentication model, particularly when enrolling new users to the system. Therefore, we compared previous approaches on the amount of interaction required, per user, to train the model. We are com- parable to few studies (4), that require between 2min. to approximately 30min. of user interaction to train the model. As an illustration, Khan et al. (2018) reported an accuracy of 97.3% using an SVM model per user, however, their approach would require recording at least 30 previous login attempts 1515 minutes) to train each user's model. A more recent approach proposed by Neha Chatter- jee
2019) achieves accuracy of 95.6% after training MLP for each user but they required 50
logins (= 25 minutes) for training phase.
On the scalability perspective, previous studies authentication paradigms that involve model per user or multiclass classification, these methods translate into large infrastructure, deployment, monitoring and maintenance challenges. In contrast, our SOS-NN model generates measure of similarity between two behaviors in latent feature space. In this case, the question is not whether the sample belongs to particular user, but rather if the samples are similar enough conclude that the user is the same. This ood-molel-for-al (OfA) paradigm facilitates deployment and avoids further training for every new user registered in the system. To the best of our knowledge the SOS- NN model and OfA paradigm have not been applied in desktop/web behavioral biometrics before.
On Appendix A, we compare with further detail, previous approaches with respect to the classifica- tion methods used, the authentication paradigm and user interaction required.
Likewise, behavioral biometrics for authentication has also been imple- mented for mobile environments. Such models complement traditional authentication by taking advantage of the multiple built-in sensors available in mobile devices, being able to capture user behavior through several modalities. Some modalities rely the use of mobile keyboard dynam- ics (Cilia & Inguanez, 2018); touchscreen interaction (Rauen et al.) 2018 Rocha et al.) 2019); or embedded motion sensors data (Abuhamad et al. 2020) to authenticate users. In order to strengthen security, especially against ad-hoc adversarial attacks, multimodal authentication frameworks have
Mobile Environment.
been proposed by researchers (Stanciu et al. 2016 Sitová al. 2015 Lamiche et al. 2019 Acien
2

---

Under review as a conference paper at ICLR 2021
et al. 2019 Lin et al., 2018 Volaka al. 2019 Shen et al. 2016 Centeno al. 2018). Those methods rely on the fusion of multiple modalities of behavioral information (i.e. keyboard, sensors, touch, etc.) with the goal of having better performance. Previous works, have achieved low False Aceptance Rate (FAR) and False Rejection Rates (FRR). However, they require a one-model-per- user paradigm (OpU) and long user interaction times (>/0min.), which makes them challenging to use in real world scenarios. In our review, the best comparable performance was reported by [Stan-] ciu al. (2016), with FAR and FRR equal to 0.14%, nonetheless, the system would require one K-Nearest Neighbors model per user and 20 previous logins for training each model.
Appendix A presents detailed comparison between the main state-of offfthe-art studies developed in mobile behavioral biometrics in terms of the type of model requirements and performance charac- teristics. To the best of our knowledge we are the first study that uses Siamese Networks for mobile multimodal behavioral biometrics, with high resulting performance compared to existing solutions and capable of assessing score after 5sec. of user's interaction.
Siamese Networks. Siamese Neural Networks were first introduced by Bromley et al. (194) to ver- ify hand-written signatures. general, Siamese Networks are composed of two twin sub-networks and similarity module which compares the outputs of both sub-networks. Consequently, Siamese Networks are trained by feeding a pair of inputs which are processed by each twin in the network. Siamese networks have been used for verification tasks because of their capabilities to create embed- ding representations which minimizes similarity between samples from different classes (Melekhov 2016 (Boenninghofi al. 2019). In the field of behavioral biometrics, Siamese Networks have been implemented to approach different behavioral modalities, like face recognition (Schroff
ett al.
al. 2015 Taigman al. 2014), signature recognition (Dey et 2017). gait recognition (Zhang ct al. 2016). among others. Regarding the use of Siamese Networks for behavioral biometrics au- thentication in web and mobile environments, Centeno et al. (2018) used Siamese Networks along with CNNs as a tool to create embeddings from motion sensors plots, and then feed them into one- class SVM classifier. More recently, Giot & Rocha (219)) proposed the use of Siamese Networks to approach static authentication model using keyboard dynamics in web environments.
Furthermore, Siamese Networks have been extensively used approach classification problems in which the few samples of each class are available to learn from (i.e. Few-shot Learning) (Hindy jet al. 2020). Particularly, it is possible to go as far as to limit the number of available samples to only (one-shot learning). In one-shot learning literature, Siamese Networks have shown promising results in tackling classification tasks, under the restriction of observing only one sample before making prediction over test instance (Koch al.) 2015 Triantafillou al. 2017). To the best of our knowledge there are no previous works on the use of Siamese Networks to approach multimodal behavioral biometrics authentication in web mobile environments. Specifically, our approach differs from others in that it (1) focuses on one-shot learning, (2) implements semi-hard pair selection, and (3) learns from different behavioral sources in both web and mobile environment.
APPROACH
In this work we propose a deep learning framework complement traditional authentication sys- tems by analyzing behavioral biometric features. Such framework aims to learn and analyze inherent user behavior while interacting with a device a few-shot fashion. particular, we analyze two different environments to learn from users: Web Environments and Mobile Environments. For the Web Environment we are interested in authenticating users using information from mouse and key- board dynamics. On the other hand, for the Mobile Environment focus on the physical sensors like touch, accelcrometer, gyroscope and magnetometer.
3.1 FEATURE ENGINEERING
As we are interested in behavior verification, the first step consists in processing continuous raw data sequences, recorded from the machine-user interaction, and then transform them into readable features for the model. Recorded sequences for all the modalities recorded. In this paper we explored the performance the model for multiple fixed-time windows varying from 5sec. to 60sec. of user interaction.
split into multiple fixed length interaction windows
3

---

Under review as a conference paper at ICLR 2021
Table 1: Sets of behavioral features for keyboard interactions in Web environments.
Key Down-Up
Key Up-Up
Key Down-Down
Key Up-Down
Length of time from a key is pressed until it is released.
Latency from one key is released until the next key is released.
The latency between two consecutive koys pressed.
Length of time from one key is relcased until the next key is pressed.
Web Environment. For the Web Environment, start out from the raw sequences of mouse move- ments and key presses. The raw mouse data includes timestamp, pointer coordinates (æ, y) and the type of interaction (i.e. click, mouse movement, etc.). We transform the raw data by designing two sets of features, inspired in Ahmed & Traore (2007): Solano et al. (2019). To calculate features that capture the direction of the mouse moment, the angular space split into eight equal bins of 45 and each mouse event in the fixed-time window is classified into one bin. Then, the first set of features is calculated by finding the average movement speed each of those direction partitions. The second set is the proportion of movements performed in each direction along the fixed-time window. Mouse dynamics were captured from 16 features (2 sets X 8 directions). In like manner, the keyboard raw data includes the timestamp, key and key interaction (i.e. Press release). Four sets of keyboard features were created as described in Table I Afterwards, for each set of keyboard features, we calculate the average, the median and the standard deviation. As a result, for the keyboard dynamics information we build a set of 12 features (4 sets x 3 metrics).
Mobile Environment. For the Mobile Environment, the raw data is represented by (1) measure- ments from sensors (gyroscope, magnetometer and accelerometer) and (2) touch inputs performed by the user along the fixed-time window. Regarding the sensor's 's measurements, we record values in X, Y and Z axis for each sensor. For each sequence, including sensors and touch data, we compute 5 measures of central tendency of the data distribution: mean, standard deviation, median, minimum and maximum. On the other hand, for the touch interaction we record raw data from the touch's 's cen- ter, the touch's pressure and the touch's size. From these records we compute 4 features: mean touch duration, the average number of changes in pressure or touch center within the same touch (down- up) interaction, standard deviation of touch's center (zr, y) and mean finger size (touch size area). We added two features related to mobile keyboard interaction, based on latency between consecutive touches. Altogether, for each fixed-time window we have a vector containing 52 features (45 sensors 5 touch 2 keyboard).
3.2 SIAMESE NEURAL NETWORK
Our Deep Learning Framework evaluates if two recorded behaviors belong the same user. This approach could authenticate a user after only one observation (i.e. the enrollment behavior). Our SOS-NN computes the similarity between two behavior inputs. In that sense, if two inputs are similar enough, our system concludes that the incoming behavior belongs to the legitimate user. The siamese network architecture made of two identical sub-networks; In our case, two fully connected neural networks which share weights. Each sub-network processes one of the input behaviors and works as a feature extractor.
Both of the sub-network's 's outputs are bounded by energy function. We compute L1 distance as the energy function between both of the computed feature vectors in the latent space. Intuitively, this distance should be large when input behaviors belong different users but small when they belong to the same user. Following the energy function calculation, we include in our model a fully-connected decision network, which makes the classification decision based on the distance between the feature vectors in the latent space. Consequently, the output of our SOS-NN is a binary classification, where the output is One if behaviors belong to the same user, and Zero otherwise. Figure I depicts the proposed SOS-NN architecture.
3.2. 1
SAMPLE GENERATION
The Siamese networks learn from comparing pairs of behaviors. A positive pair is defined pair of two behaviors which belong to the same user, whereas a pair of behaviors from different users is labeled as negative pair. The model is trained by presenting multiple samples of these
4

---

Under review as a conference paper at ICLR 2021
Energy Function
Decision Network
Behawwarr A
Inst
Distance Similariy by
Sinikarit Score
Bchawior
Figure 1: SOS-NN architecture.
pairs, the training goal is to minimize the energy function (i.e. L2 distance) between behaviors in positive pairs, and to maximize the energy function for negative pairs. The quality of retrieved pairs in training will determine the quality of our SOS-NN. The naive selection of those positive and negative pairs random selection. This naive approach fine for the positive pairs but for the negative samples it is crucial to select high quality pairs. More advanced techniques can be used to select the training samples, like triplet loss technique, which uses simultaneously 3 examples to optimize every training step (Hermans et al. 2017). The constrains on the distances between the triplets can be controlled; we chose Semi-hard negatives as our strategy to populate triplet samples. Further details the triplet loss selection can be found Appendix B. On the evaluation we compare two pair generation strategies to feed with the Siamese Networks, namely (1) Naive Pair and (2) Semi-hard Triplets.
3.2.2 TRAINING STRATEGIES
The training procedure depends on the sample generation strategy (naive or triplet). For the naive pair strategy, we train our SOS-NN using a cross entropy objective loss on our binary classifier (same or different user) Therefore, for the naive strategy, we perform the weights optimization over the full network (including both the siamese and decision networks), using standard back propagation. Notice that gradient is additive over the tied sub-networks. On the other hand, for the triplet strategy the training is done in two steps: (1) siamese network training and (2) decision layer training. To train the siamese network, we use the Semi-Hard Triplets configuration. Once the feature extractor is trained, their weights are frozen and the fully connected decision layer is appended to complete the SOS-NN Lastly, we train the decision layer by using binary cross-entropy as a loss function.
4 EVALUATION
4.1 EXPERIMENTAL SETUP
We collected two datasets under controlled and uncontrolled condi- tions. For the controlled setting, we used data acquired through the 'Amazon Mechanical Turk' service by submitting the task of logging on a website designed to capture mouse and keyboard events. Here we collected interactions from 89 worldwide workers who introduced fictitious cre- dentials on the login website. A total of 1374 sessions (full login interaction) were obtained with an average duration of 26.7sec. per session. As for the uncontrolled setting, monitoring system was appended to two login web pages from real banking domains where mouse and keyboard events were recorded. To protect users privacy, the raw data was transformed into features on the client- side before they were sent to the server. In this setting, 807622 sessions worth of interactions were collected from 125403 users, with an average of 22.4sec. per session.
Web Environment Dataset.
Mobile Environment Dataset. We developed realistic looking Android application simulating different banking activities and equipped with a event logger to save information related to touch, accelerometer, magnetometer and gyroscope events. For keyboard touch events the timestamp and key value were logged. We collected data from 35 volunteers performing sessions lasting 10min. on
A service where human workers perform a certain task following instructions defined by the task requester.
5

---

Under review as a conference paper at ICLR 2021
average and making up to 372min. in total. Moreover, the volunteers used more than 20 different smartphone devices. In case of acceptance both datasets will be made publicly available.
Feature Engineering. The behavioral data gathered in controlled settings subsequently merged into a continuous session for each user. Afterwards, the full history per user was split into fixed-time windows. We analyzed time windows of 5, 10, 20, 30 and 60 seconds of interaction. Next, for each fixed-time window we calculate the features following the methodology described Section 3.1 Regarding the data collected in the wild from real banking domains, since features are pre-computed on the client-side, we collected login sessions with arbitrary durations. Finally, we randomly split the dataset by users, since we wanted to avoid validating with behaviors similar the ones observed in training. Splitting the dataset by users has been shown to perform better than other train-test schemes previous verification tasks (Koch et al. 2015). More details related to data processing and train-test splits can be found in Appendix C.I.1
Networks Configuration. We implemented our approach using the Tensorflow framework. The feature extraction and classification decision is made by siamese network and decision net- work respectively, both were built with fully connected architectures. Therefore, regularization, batch normalization and dropouts were tested for both networks. For the triplet loss we used the Semi Harddripleelt implementation available on Tensorflow Addons (Hermans et al. 2017 Our SOS-NN network is trained using Adam optimizer. For more details refer to Appendix C.2
4.2 RESULTS
To understand the capabilities and robustness of our pro- posed SOS-NN we perform a systematic evaluation over the model performance for different setups in the train- ing and verification phase. Variables included in this sys- tematic evaluation the sample generation scheme (pair o triplet), interaction time required (fixed-time window length) and the number of samples (n) from previous his- tory to compare with (n-shot testing). For the n-shot test- ing, the output consists in the average of each individual pair-comparison. At the end of the section, we show an evaluation of our SOS-NN for web environments over large scale experiment (s= 125 K of users) to verify that performance remains the same level under uncontrolled conditions.
Accunayy [[6]
6
Numior compppr
(a) Web Environment
Web Environment Results. Figure illustrates the model accuracy in web environments for different config- urations. As can noticed from Figure 2a the verifica- tion accuracy is consistently higher for longer time win- dows in the verification phase. The best verification accu- racy obtained for web environment is 90.8%. Moreover, we also found gradual rise in the verification accuracy when the number of comparisons increases. In the I-shot and 5-shol verification tasks our approach achieves up to 74% and 86% in verification accuracy respectively. Ta- ble summariiee the accuracies for different verification windows requirements with higher values in darker color. As we expected, the performance of our model is better when analyzing longer times in verification phase. How- ever, more comparisons seems to have larger effect on accuracy than longer interactions times. For instance, comparisons (/0-shot) for window lengths of seconds is better than one comparison (/-shon (/-shot) of 60 seconds, as can be observed in Table 2b We believe this is due to mul- tiple comparisons smoothing the score of the incoming fixed-time windows.
Acceumerry [96]]
-
Seecade ral
20 mosdde
00 eneede-
Saczab cppl:
30 mnsed- indi
B0 mnued- indi
15
20
20
Numarr of comparisums
(b) Mobile Environment
Figure 2: Accuracy results for evalu- ation over several n-shot comparisons and time windows interactions for be- havior verification collected in web and mobile environment.

---

Under review as a conference paper at ICLR 2021
Moreover, observe improvement of around 4% when using triplet setup for training phase. This improvement is a consequence of better negative sample choices in the training phase, which leads to powerful feature extractor at the bottom of our SOS-NN. Recall that in the triplet training scheme training samples are forced to satisfy Equation T Consequently, our results con- firm that better choice of behavioral positive and negative samples could moderately improve the performance of our SOS-NN for web environments, Furthermore, the verification accuracy reaches the best values after 10 comparisons (10-shot) performance. F shows shows
starts fluctuating close to the maximum obtained the ROC curve and the FAR and FRR for the best setup in web envi- ronment. As can be observed from Figure (3b] the FAR and FRR are 9.8% and 16.2% respectively for classification threshold equal to 0.45.
Mobile Environment Results. Results for the mobile en- vironment can be found on Figure 2b] The best verifica- tion accuracy obtained is 99.8%. We found considerable accuracy improvement (5%) when the number of com- parisons increases from to comparisons. However, from 10 comparisons until 30, for all window lengths the accuracy is saturated around 98% for a 60sec. window and 97% for 5sec. window. Besides, 1% difference between a 5 and a 60 seconds-window suggests that the information collected in the first seconds gives already a satisfactory description of user behavior. The differ- ence between pair and triplet training is significant. This suggests for high validation accuracy (i.e. >90%), the performance depends more on the architecture of the sub-networks or on data structure aspects instead of the sample generation strategy.
112
0.5
0.7
0.5
36460
(a) Receiver Operative Curve (ROC).
Finally, if we focus on ideal setting where there are no restrictions on how much data can be sampled from each user, meaning no limit on comparisons windows lengths, we achieve a validation accuracy up to 98% for all with windows above 20 comparisons (See Table 3). Nevertheless, the case of practical solution (i.e a com- pany's product) there are some limitations to be taken into account. Here, waiting too much to gather several 60-second windows can compromise the security of the system. In this case, the lesser the time the better: even for the case of 5-second window and only one compar- ison, namely one-shot inference, we achieve an accuracy of 92%. Our preferred case would be using compar- isons, which corresponds to 15sec. of interaction, eas- ily collected after only one login session of the user; in this scenario accuracy is 96%. Figure 3 shows the ROC curve and the FAR and FRR for the best setup in mobile environment. The best FAR and FRR are 0.01% % and 0.22% respectively for threshold classification equal 0.4. Remarkably, our SOS-NN achieves an AUC of 0.99 in mobile environment (See Figure 3a).
FRR/PAR
0.5
0.2
FATL We
SRR 1046
0.1
2011 32.00
2117 36220
0.0
Thrrshold
(b) FAR and FRR for all classification thresholds.
Figure for the best setup (20sec.) in web and (Ssec.) mobile environment.
Performance of the model
Table 2: Accuracies for test in Web Environment.
(a) Accuracies for pair training.
(b) Accuracies for triplct training.
Time Length [sec
Time Length (sec]
10
20
30
60
20
30
60
0.60
0.64
0.67
0.73
0.73
0.62
0.66
0.71
0.74
0.74
Compartonn iisnsss
0.67
8.71
0.74
0.81
0.81
Comparboo
0.68
0.75
0.80
0.83
0.81
0.74
0.79
10
0.71
0.78
0.79
0.36
0.54
10
0.78
0.79
0.85
0.85
0.87
20
0.79
0.80
0.88
0.84
0.88
30
0.73
0.80
0.82
0.86
0.87
0.77
0.81
0.86
0.87
0.91

---

Under review as a conference paper at ICLR 2021
4.2.1 EVALUATION IN THE WILD
In order reach deep understanding on how our SOS-NN behaves in the wild large scale, we tested the framework with over 100 thousand real users from legitimate banking domains. Figure E depicts the verification accuracy of our approach evaluated real web sessions. First of all, we simulated the same setting we had in the controlled experiment: we trained with 50 users and tested with the remaining users. In this baseline, the verification accuracy for I-shot testing is 63% and 68% for pair and triplet training respectively. Notice that this performance value is very close to metrics we had for the controlled dataset for time window length of 10 seconds. Recall that 75% of sessions in the uncontrolled dataset lasted less than 10 seconds. In like manner, the accuracy when testing in a 5-shot and a 10-shot fashion up to 75% and 79% respectively.
More importantly, investigated how the model perfor- mance increases when more data is considered to train it. Accordingly, study the verification accuracy our SOS-NN when behavioral data from 25K, 75K or 100K different users included in the training set. Figure 4 shows the verification accuracy for different n-shot con- figurations when different amounts of users training data. From Figure 4 we can observe that model perfor- mance jumps when more user's behaviors are the train- ing phase. In general, the rise in verification accuracy when training with 25K in comparison with 50 users is about 9%. In addition, we do not observe sharp improve- ments when training with more than 25K users. We be- lieve this is happening because the behavior of 25K users is complex enough to force the network to learn the differ- ences the latent space among diverse spectrum of user behaviors. This finding suggests that, for behavioral bio- metric data, there could be limit the representational power of the data in the latent space, this is an interesting insight for future research on siamese neural networks ap- plied to structured data.
Avcuracy
Thadding
Treneg
Thanng-
10
21
Number of comparisons
Figure 4: Accuracy results for evalu- ation over several n-shot comparisons for web behaviors collected in the wild (125K users)
4.2.2 DISCUSSION
As shown, our SOS-NN is able accurately classify behaviors for short and long interaction win- dows, and therefore it is suitable as a competitive, maintainable and lightweight mechanism for authentication in contrast to previous work (Appendix A). Previous schemes are mainly based on either one model per user or multiclass classification, which could result in resource intensive implementation in the real world and arguably not as scalable for thousands or even millions of users as our proposed one-for-all approach. In fact, for multiclass classification paradigm the model has to be re-trained every time a new user is added; while for the one-model-per- -peoru--per-sser model has to be trained for new user in the service (requiring high volumes of data for each new user). By comparison, after training with a certain quantity of data our SOS-NN is ready to be deployed and used with any user, even if it has not seen before by the system.
Additionally, we studied the system performance in a real authentication scenario, where requiring short interaction windows and rapidly deploy model that can assess risk score for the user can be a large advantage. The majority of examples in the related work require minutes or even hours of interactions tenths of logins to achieve their performance. Alternatively, our approach could
Table 3: Accuracies for test in Mobile Environment.
(a) Accuracies for pair training.
(b) Accuracies for triplet training.
Time Length [sec]
Time Length [sec]
10
20
30
60
10
20
30
60
0.92
0.90
0.93
0.90
0.91
0.92
0.92
0.91
0.92
0.92
Compar bisos
0.95
8.95
0.94
0.95
0.98
Comparboon
0.96
0.96
0.94
0.95
0.96
0.95
0.97
0.97
0.96
0.96
0.97
10
0.97
0.96
0.98
0.98
0.99
10
0.97
0.98
0.97
0.97
0.98
20
0.99
20
0.98
0.98
0.98
0.98
0.99
30
0.97
0.97
0.99
0.98
0.99
30
0.98
0.99
0.98
0.98
0.98
8

---

Under review as a conference paper at ICLR 2021
authenticate the user after only 25 seconds to 3 minutes with an accuracy of 73% to 85% in Web environment. We understand this value could seem relatively low, but that is because we push to the limit the capabilities of the model to learn behaviors even from short interactions trade-off between accuracy and promptness of assessment.
Finally, the difference of performance between web and mobile environments is noteworthy. The complete framework behaves way better with mobile data, even when the architecture and training procedure are the same. Recall that for the mobile environment a few seconds of interaction are enough for the model to achieve 99% of accuracy. This can be explained given that our model re- ceives the devices' sensors input even when the user is not actively interacting with the application and is only holding the device. On the other hand, the web environment depends on the user inter- acting with the peripherals (mouse & keyboard), otherwise the system goes blind due to insufficient input data to make valuable prediction, as seen in Figure This means that our web system needs more data, and therefore more time be able complement the authentication, but nonetheless can achieve reasonable performances of up to 90% of accuracy.
5 CONCLUSION
In this paper we have presented an effective approach to generate a Siamese Neural Network model that integrates different human-device interaction sources (mouse, keyboard or mobile sensors) for behavioral biometric authentication. Our model obtained high accuracy even when tested in a few- shot fashion, that is, needing only few behavior samples per user. We also showed that the proposed model architecture can be easily adapted to both web and mobile environments. Furthermore, our system has the potential to rapidly scale in production environments because it needs only one model to evaluate behavior of many users. We evaluated our approach on various datasets, including production data for thousands of users in a web-application. In future work we plan to evaluate our approach production data belonging to mobile users well.
REFERENCES
Mohammed Abuhamad, Tamer Abuhmed, David Mohaisen, and Dae Hun Nyang. Autosen: Deep learning-based implicit continuous authentication using smartphone sensors. IEEE Internet of Things Journal, 2020.
Alejandro Acien, Aythami Morales, Ruben Vera Vera-Rodriguez. Julian Fierrez, and Ruben Tolosana. Multilock: Mobile active authentication based on multiple biometric and behavioral patterns. In 1st International Workshop Multimodal Understanding and Learning for Embodied Applica- tions, pp. 53-59, 2019.
Ahmed Awad E Ahmed and Issa Traore. A new biometric technology based on mouse dynamics. IEEE Transactions dependable and secure computing, 4(3): 2007.
Sara Amini, Vahid Noroozi, Amit Pande, Satyajit Gupte, Philip S Yu, and Chris Kanich. Deepauth: A framework for continuous user re-authenticatio in mobile apps. In Proceedings of the 27th ACM International Conference Information and Knowledge Management, pp. 2027-2033 2018.
Kyle O Bailey, James S Okolica, and Gilbert L Peterson. User identification and authentication using multi-modal behavioral biometrics. Computers & Security, 43:77-89, 2014.
Benedikt Boenninghoff, Robert M Nickel, Steffen Zeiler, and Dorothea Kolossa. Similarity learning for authorship verification in social media. In ICASSP 2019-2019 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 2457-240 IEEE, 2019.
Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. Signature verifi- cation using a" siamese" time delay neural network. In Advances in neural information processing systems, pp. 737-744, 1994.
Mario Paгreño Centeno, Yu Guan, and Aad van Moorsel. Mobile based continuous authentication using deep features. In Proceedings of the 2nd International Workshop on Embedded and Mobile Deep Learning, pp. 19,24, 2018

---

Under review as a conference paper at ICLR 2021
Darren Cilia and Frankie Inguanez. Multi-model authentication using keystroke dynamics for smart- phones. In 2018 IEEE 8th International Conference on Consumer Electronics Electronics-Berlin -Brilnn (ICCE- Berlin), pp. 1-6. IEEE, 2018.
Sounak Dey, Anjan Dutta, J Ignacio Toledo, Suman K Ghosh, Josep Lladós, and Umapada Pal. Signet: Convolutional siamese network for writer independent offline signature verification. arXiv 2017.
preprint VrXiv:1707.022
Lex Fridman, Ariel Stolerman, Sayandeep Acharya, Patrick Brennan, Patrick Juola, Rachel Green- stadt, and Moshe Kam. Multi-modal decision fusion for continuous authentication. Computers & Electrical Engineering, 41:142-156, 2015.
Romain Giot and Anderson Rocha. Siamese networks for static keystroke dynamics authentication. In 2019 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1-6. IEEE, 2019.
Alexander Hermans, Lucas Beyer, and Bastian Leibe. In defense of the triplet loss for person re- identification. arXiv preprint arXiv:1703.07737, 2017.
Hanan Hindy, Christos Tachtatzis, Robert Atkinson, David Brosset, Miroslav Bures, Ivan An- donovic, Craig Michie, and Xavier Bellekens. Leveraging siamese networks for one-shot intrusion detection model. arXiv preprint 2nXiv:2006.1334. 2020.
Harini Jagadeesan and Michael S Hsiao. A novel approach to design of user re-authentication systems. In 2009 IEEE 3rd Internatinal Conference on Biometrics: Theory, Applications, and Systems, pp. 1-6. IEEE, 2009.
Falaah Arif Khan, Sajin Kunhambu, al. Behavioral biometrics and machine learning to secure website logins. In International Symposium on Security in Computing and Communication, pp. 667-677. Springer, 2018.
Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2. Lille, 2015.
Imane Lamiche, Guo Bin, Yao Jing, Zhiwen Yu, and Abdenour Hadid. A continuous smartphone authentication method based on gait patterns and keystroke dynamics. Journal of Ambient Intel- ligence and Humanized Computing, 10(11):447-4440, 2019.
Hong Lin, Jiafen Liu, and Qing Li. Tdsd: A touch dynamic and sensor data based approach for continuous user authentication. In PACIS, pp. 294, 2018
laroslav Melekhov, Juho Kannala, and Esa Rahtu. Siamese network features for image matching. In 2016 23rd International Conference Pattern Recognition (ICPR), pp. 378 378-383. IEEE, 2016.
Neha and Kakali Chatterjee. Continuous User Authentication System: A Risk Analysis Based Approach. Wireless Personal Communications, 108(1):281-29) 2019. ISSN 1572834X. doi: 10.11077/11227--099-666633
Pedregosa, G. Varoquaux, Gramfort, V. Michel, Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn; Machine learning Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Zachary Rauen, Fazel Anjomshoa, and Burak Kantarci. Gesture and sociability continuous authentication on smart mobile devices. In Proceedings of the 16th ACM International Symposium Mobility Management and Wireless Access, pp. 51-58, 2018.
Rodrigo Rocha, Davide Carneiro, Ricardo Costa, and César Analide. Continuous authentication in mobile devices using behavioral biometrics. In International Symposium on Ambient Intelligence, pp. 191-198. Springer, 2019.
Florian Schroff. Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings the IEEE conference on computer vision and pattern recognition, pp. 815- 2015.
10

---

Under review as a conference paper at ICLR 2021
Chao Shen, Tianwen Yu, Sheng Yuan, Yunpeng Li, and Xiaohong Guan. Performance analysis of motion-sensor behavior for user authentication on smartphones. Sensors, 16(3):355, 2016.
Zdeňka Sitová, Jaroslav Šedēnka, Qing Yang, Ge Peng, Gang Zhou, Paolo Gasti, and Kiran S Bal- agani. Hmog: New behavioral biometric features for continuous authentication of smartphone users. IEEE Transactions on Information Forensics and Security, 11(5):877-82, 2015.
Jesus Solano, Luis Camacho, Alejandro Correa, Claudio Deiro, Javier Vargas, and Martín Ochoa. Risk-based static authentication in web applications with behavioral biometrics and session con- text analytics. International Conference on Applied Cryptography and Network Security, pp- 3-23. Springer, 2019.
Jesús Solano, Lizzy Tengana, Alejandra Castelblanco, Esteban Rivera, Christian Lopez, and Martin Ochoa. few-shot practical behavioral biometrics model for login authentication web appli- cations. In NDSS Workshop on Measurements, Attacks, and Defenses for the Web (MADWeb 20), 2020.
Valeriu-Daniel Stanciu, Riccardo Spolaor, Mauro Conti, and Cristiano Giuffrida. On the effective- ness of eensor-e keystroke dynamics against statistical attacks. In proceedings of the sixth ACM conference on data and application security and privacy, pp. 105-112, 2016.
Yaniv Taigman, Ming Yang. Marc' Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level performance face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1701-1708. 2014.
Issa Traore, Isaac Woungang, Mohammad S Obaidat, Youssef Nakkabi, and Iris Lai. Combining mouse and keystroke dynamics biometrics for risk-based authentication in web environments. In 2012 fourth international conference digital home, pp. 138-145. IEEE, 2012.
Eleni Triantafillou, Richard Zemel, and Raquel Urtasun, Few-shot learning through an information retrieval lens. In Advances in Neural Information Processing Systems, pp. 2255-2265. 2017.
Hasan Can Volaka, Gulfem Alptekin, Okan Engin Basar, Mustafa Isbilen, and Ozlem Durmaz In- cel. Towards continuous authentication on mobile phones using deep learning models. Procedia Computer Science, 155:177-184, 2019.
Cheng Zhang, Wu Liu, Huadong Ma, and Huiyuan Fu. Siamese neural network based gait recogni- tion for human identification. 2016 IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2832-2836. IEEE, 2016.

---

Under review as a conference paper at ICLR 2021
APPENDIX
A RELATED WORK
We collected information from previous studies for both desktop/web and mobile environments. The following table compares these works in terms of model imputs, model type, model performance, authentication paradigm, required amount of user interaction to train the model and required amount of interaction authenticate the user, once the model has been trained.
Table Related Work Web and Mobile Behavioral Biometric Authentication
Training (Tr) & Authentication (Auth)
Reference
Input
Method
Paradigm
Performance
Interaction per User
Tr:20 Sessions
Jagadeesan &
K, M
KNN, SèNN
OpU
Acc. 82.2%
Auth:1 Session
Hsiao (2009]
Tr: 200KS,2500M Auth:l session
Traore et al.
K, M
BN
OpU
EER 8.21%
2012
FAR 2.24%
Tr: 12h Auth: 10m
Bailey et
K, M. GUI
SVM, BN,
MC
FRR 2.10%
2014
J48
FAR 0.004% HRR 0.01%
Tr: 33.6h
Fridman
K, M,
NB, SVM
MC
Auth: 30s
Web
2015
Tr. logins Auth: login
Khan al. 2018
K, M
SVM, MP,
OpU
Acc. 97.3%
AB
Tr. logins Auth: login
Neha & Chat-
K, M
J48, BN
MC
Acc. 95.6%
terjee 2019
FAR 23.34%
Tr: 2min
Solano et al.
K, M
RF
OpU
PRR 10.73%
Auth: 30s
2020
FAR 3.62% FRR 27.3%
Tr:6.6mnn
Ours
K. M
SOS-NN
OFA
Auth:20s
Acc. 88%
PCA SVM
Tr:10min
Sitová et al.
K. TI
OpU
EER 7.16 %
Auth:20s
(2015
FRR 6.85% FAR 5.01%
Tr:2700 logins
Shen et
TI, MS
O-SVM
OpU
Auth:3s
(2016
Tr:20 logins
Stanciu et
K. MS
KNN
OpU
EER 0.14%
Auth: login
2016
CNN
Tr:270s
Centeno al.
MS
OpU
Acc 95.8%
O-SVM
Auth:2s
Mobile
2018
FAR 0.68% FRR 7%
Tr: Irials (WK + KS) Auth: trial
Lamiche al.
MS, K
MP
OpU
2019
Tra 80min
Volaka et al. (2019
MS, TI
MP
OpU
EER 15%
Auth: 9min
SVM per-source
Tr:224 SS(#= 9h)
Acien
et al.
MS, AppU
OpU
BER 15%
Auth: SS(=s 224s)
(2099
FAR 0.01% FRR 0.22%
Tr: Auth: 5s
Ours
MS, K, TI
SOS-NN
OfA
Acc. 96%
Input: K: Keyboard Internction, M: Mouse Dynamics, MS Motion Sensons, Tl: Touchscreen Interaction, GUI:Gui Features, St:Styooeery Feorures AppU:
App usage time.
Method: KNN: Nearest Neughbors, SWNN: Shallow Neural Network, Bayesian Netwoek,
Network. Bayes, Ei--Randm- Focest Classifier, Multi
Penceptron. Adagtative Boosting
class SVM. Siamese Parzddgg:: OpU: One Model per User, MC: Mulli Chas Classification OfA: One Model for All.
Performance: EER: Equal firorr Rate, FAR Palse Acceptance Rate, FRR: False Rejection Rate, Acc.; Accuraey.
Tr/Anth Interaction: ME: Mouse Events, KS: Key strukes.
12

---

Under review as a conference paper at ICLR 2021
B FEATURE ENGINEERING DETAILS
B. WEB ENVIRONMENT
The mouse features used in our web environment ap- proach are inspired in Ahmed Traore (2007): [Solano et al. (2019). These features build a mouse dynamics profile over the movements performed by user in fixed-time window. The key idea is to analyze mouse ac- tions like mouse movements, drag-and-droyg within eight equal area partitions in the circular space sur- rounding the pointer's position at any given time along recorded session. We perform this analysis by getting the next pointer's 's position and calculating the movement direction, the speed, the angle of curvature and the cur- vature distance (See Figure 5). In consequence, a unique user signature can be built by calculating central tendency statistics for each partition.
(Ro-zs Bres
and clicks
Figure 5: Features engineering illustra- tion for web environment.
As for the keyboard, the timing features are extracted from the timestamps and the type of action (key-down or key-up) of the keyboard events recorded. The aim here is to discover unique user tendencies by measuring how long was key pressed (down-up), the time between two consecutive key releases (up-up), the time between two consecutive keys pressed (down-down) and the time from one key release until the next key is presses (up-down). Thus, the keyboard features consist of the central tendency statistics per each of these metrics.
B.2 MOBILE ENVIRONMENT
The mobile features are intended to capture the way user holds the smartphone and simulta- neously their touch pattern when tapping the screen. In that sense, we start from the raw mea- surements from smartphone's 's sensors (i.e. gyroscope, magnetometer and accelerometer). Notice that the value of those sensors change constantly while the user naturally interacts with the smart- phone. In sum, calculate central tendency statistics over the series S S2, ....,.... Stn for each sensor E [gyroscope, magnetometer, accelerometer].. Then we build the user device holding profile by analyzing the 45 sensor features (3 Sensors 3 Axes 5 statistics). Regarding touch interaction, we use the interaction derived from touch actions as the raw data. Afterwards, we compute the mean touch duration, the average number of changes in pressure or touch center within the same touch (down-up) interaction, standard deviation of touch's center (x, y) and mean finger size (touch size area) over all in actions in fixed-time window.
In summary, the features used to capture meaningful traits of user hehavior when interacting in web and mobile environments are summarized by input source in Figure 6
Mobile
12
Keyboard
Input Source
16
Mouse/Touch
Sensor
10
20
30
40
50
Number Features
Figure 6: Count of behavioral features per sources for web and mobile environment.
13

---

Under review as a conference paper at ICLR 2021
C TRAINING DETAILS
C.I SAMPLE GENERATION DETAILS
One of the most successful approaches applied to Siamese Networks in order to improve the selec- tion of negative pairs was presented by (Schroff el. (015)_ As shown early, a Siamese Networks is an architecture that compares two sets of inputs; however, |Schroff et al. (2015) implemented a triplet technique that uses 3 examples the same time to optimize every training step. These examples are carefully selected so that the first two of them belong to a positive match (i.e. two examples that correspond the same class, namely an anchor (A) and a positive example (P)) and the last one cor- responds to a different class, being a negative example (N). The goal is to minimize the distance in the feature space between examples of the same class and maximize the distance between examples of different classes, which can be expressed as:
(f(z)) f(xp) +a (f(xa) f(xx))?
Where a margin parameter related to the difficulty of the examples. The final loss can be represented as:
L(A,P,N) maz(If(Fs) - -aapp//==aaaaapp--- - (((A) f(xx)1.0)
(2)
As in the pair generation, the 3 examples per triplet can be generated randomly as long as they fulfill the previously named condition. Nevertheless, this strategy would deliver most of the time a small Loss, because it is expected that many negative examples produce features with larger distances from the anchors, whose loss ends up adding nothing to the training. A better approach is to pre-select the triplets in order to optimize the learning. This pre-selection could be performed by choosing two kinds of triplets: Hard negatives and Semi-hard negatives. Hard negatives examples whose feature distances with the anchor are strictly smaller than the distance between the positive example and the anchor (Schroff et al. 2015). Semi-hard negatives Anchor is greater than the positive example, but still inside the margin a (Hermans et al.] (017). In principle, the hard negatives deliver the greatest losses, and therefore the strongest convergence, but in practice they can be too aggressive and collapse the loss function. Consequently, we use Semi- hard negatives as our strategy to populate triplets in training phase. With this in mind, propose and compare both pair generations strategies (Naive Pair and Semi-hard Triplets) to feed with the Siamese Networks.
examples whose distance with the
C.I.1 DATA PROCESSING
Once the features have been calculated, we split the dataset into three disjoint set of users: 64% of the users for training, 16% for validation and 20% for test. Finally, the features were transformed in order to follow a normal distribution by using a non-linear transformation on each feature indepen- dently (Quantile Transformation in Pedregosa al. (2)1
Privacy Concerns. Collecting behavior sequences could lead to privacy concerns as personal data like passwords patterns (keyboard Dynamies), user location (sensor Analysis), etc. is being recorded. To ensure privacy of the information, we transform the raw sequences in the client-side and send the behavioral biometries features to be analyzed by our SOS-NN model in the server-side.
C.2 NETWORK DETAILS
One of our goals was to implement system that was almost the same for both web and mobile envi- ronments. The biggest difference is found at the input of the Siamese Networks, because the number of crafted features is different for each environment, namely 28 for web and 52 for mobile. After that, both environments share the same architecture and hyper-parameters. The feature extractor was composed by a fully connected network; the output of both Siamese extractors was compared with an L, distance layer.
The feature extractor was composed by a couple of dense layers with ReLU activation and a last layer with Sigmoid activation. For the decision module, another fully connected network was
14

---

Under review as a conference paper at ICLR 2021
implemented whose imput was the L1 distance layer between the features in the latent space. We also tested other distances as energy function, like L2, Manhattan and Cosine distances as well, but L1 showed the best results. The decision subnetwork was composed of two dense layers with ReLU activation and finally an output network with a sigmoid activation, whose output can be understood the probability of two inputs samples come from the same user. We also tried different strategies to improve the validation accuracy like regularization, batch normalization, a deeper and wider network but the best validation performance was obtained with the simplest architecture.
The best parameter configuration was a mini batch of 100, a learning rate of 0.001, an Adam Op- timizer and 200 steps per epoch. Weights were initialized following normal distribution with zero-mean and standard deviation 0.01. For the feature extraction training with triplet configuration the training was performed up to 1000 epochs, with an early stopping callback focused on validation loss. The training of this phase includes a L2 distance part of the available in the Tensorflow implementation of triplet loss schema (a 1.0). It should be clear that Lg was used only for the feature extraction training; later when it is appended to the decision layer, L1 distance is used to measure the difference between two inputs. The decision layer subnetwork for the triplet approach was trained for 20 epochs. On the other hand, the training for pair approach, that is the feature tractor and decision layer as a whole were trained for 20 epochs. The data selection for the training the explained in Section 3.2.2
15


Reviews:
[{"review_id": "KEtnHt07bi-", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Authors propose an approach to perform authentication based on users' behaviour (behavioral biometrics) on two different domains (web and mobile) by means of a siamese NN. The proposal is evaluated on three different dataset, collected ad-hoc for this work, showing a performance up to >0.9 acc. In the experimentation setup authors include different data splits, sequences lengths and training methods (pairs and triplets). The work shows the feasibility of using this type of approach for few-shot classification under real industry constraints.\n\n------------------\nContributions\n\nMy recommendation for this manuscript is a strong accept. I believe this paper makes a relevant contribution to the field since it addresses a topic which is quite interesting for the audience, it is technically sound, covers different hypotheses and provides very useful insight to practitioners about the use of these approaches in real industry applications. The last point is the most interesting in my opinion, honestly. \nThe paper is well structured, with a clean format, very well written and easy to read.\nProbably its weakest point is the novelty, since it addresses a well-known problem using a long-established NN architecture. However the combination of one-shot learning, siamese architecture and behavioral biometrics is something I personally have not seen before, so I consider it robust enough.\n\n------------------\nRecommendations\n\nAlthough it is covered in the section 3.1, given the scope of the conference and its relevance for the reproducibility of the results, I’d ask authors to include a clear and detailed explanation of the feature vector employed. Some chart or graph where the reader can see easily how many features come from which source (sensor, keypad, gesture), how the derived features are computed and how movements are translated to features. Even better if the authors can include some examples.  Maybe as an appendix, but I really believe it’s important and a clear improvement for the manuscript.\n\nI think it’d be interesting to add further details about the data splitting mechanism. I believe the paper needs at least to state clearly whether the train/test split is done by users (where a group of users are employed to perform inference and the rest as control test) or by sequences (where the data from each user is split into train and test). This is a relevant topic since the later has been demonstrated in previous HAR works to offer less robust and generalizable models. Please expand this point.\n\nFigure 4 - Please specify clearly which sequence length is the one used for this experiment. It seems taken for granted it is 10 sec , since it’s the configuration offering best results, but please make it more clear to the reader.\n\nFrom a personal perspective I think it’d be interesting to expand the work adding feature extraction mechanism using the state-of-art in time series modelling, this is, recurrent neural units or attention based mechanism. I’m really curious to see how that compares with the current engineered features.\n\n------------------\nMinor\n\nFigure 2b - I’d recommend to scale the Accuracy axis since the series are hard to analyse (they are too close together)\n\nTypo: “ Figure 2 shows the ROC curve and the FAR and FRR for the best setup in web environment” -> “Figure 3 shows the ROC curve and the FAR and FRR for the best setup in web environment”\n\nTypo: “Figure 2 shows the ROC curve and the FAR and FRR for the best setup in mobile environment.” -> “ “Figure 3 shows the ROC curve and the FAR and FRR for the best setup in mobile environment.”\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "6I6WqBN-oNq", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper proposes the use of siamese neural networks for the task of user verification by learning to identify their behavioral patterns of interaction with mobile phones or web browsers. Notably, the behavioral characteristics of users when engaging with these devices, measured in the form of mouse movements or keyboard typing patterns on browsers or touch patterns on mobile phones, serve as a proxy for more direct biomarkers such as face recognition, eye scans, etc. The authors argue that the latter requires more specialised hardware and is thus not easily scalable. Instead, they architect the first NN based solution that serves both modalities, web and mobile phones, while trying to be more accurate and responsive.\n\nReview:\n(a) The study is of good quality. The language is clear and concise. Related work is well covered to the best of my knowledge, making reference to one of the oldest instantiations of such model for signature verification.\n\n(b) The originality of the method viz-a-viz the domain 'of biometric verification' seems to be high. In particular, existing solutions to the problem of behavioral biometric verification have either used different models (SVMs or RDTs) or different features or different formulations (classic supervised recognition setting using one model per user). This disposes them to problems such as limited scalability, lower performance and high latency. Herein, their solution of using a single siamese network, for both modalities, and across different users, adds generality, performance and speed. They have curated experiments with different number of users at training time, and using different metric losses (pairwise and triplet) for representation learning via similarity matching, shedding light on the effectiveness of their approach.\n\n(c) That said, originality in terms of research contribution is limited. Siamese NNs have long been in use and recently revisited for variety of vision tasks such as face recognition[1, 2], object tracking [3], etc. The loss formulation used here is also standard, not going beyond the staple pairwise and triplet. Papers where newer losses or network architectures or training paradigms have led to better analysis and results have made sound research contributions and exhibited suitability for the main track e.g. [1, 2]. In other cases, where tasks are key, and results are good but the approach is largely borrowed, it serves good to go for a workshop track e.g. [3].\n\n(d) Comparisons with existing methods in the domain have not been included to the main paper (they are supposed to be in the appendix). The aim perhaps of this paper is to present a baseline NN-based approach for user verification/ biometrics. I believe this would be a good fit for presentation at a more focused workshop on biometric verification. As a main track ICLR submission I find it lacking in novelty, reach and research contribution.\n\n[1] Triplet Loss in Siamese Network for Object Tracking, Dong and Shen (ECCV 2018)\n[2] Occlusion Robust Face Recognition Based on Mask Learning With Pairwise Differential Siamese Network, Song et.al. (ICCV 2019)\n[3] Fully-Convolutional Siamese Networks for Object Tracking, Bertinetto et. al. (ECCV 2016 workshop paper)\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "0vTcET67eH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n\nSummary:\n\n \nThe paper proposes a siamese neural network to perform few-shot verification of user behaviour.\n\n##########################################################################\n\nReasons for score: \n\n \nOverall, I vote for rejection. I like the idea of applying the siamese paradigm to authentication however the paper offers limited novelty for a representation learning conference. There are zero baselines (no siamese) compared.\n\n \n##########################################################################Pros: \n\n \n\n \n1. The proposed architecture seems to be very accurate, as reported in the experimental section. \n\n2. This paper provides comprehensive experiments, including both qualitative analysis and quantitative results across different datasets, to show the effectiveness of the proposed framework.\n\n \n##########################################################################\n\nCons: \n\n \n1. My major concern is the novelty of the siamese network. There seems to be no new component proposed that is tailored to biometric data.\n\n \n2. I would suggest thinking about the contribution to the ML community. If the vanilla siamese network works like a charm, is the authentication problem solved? \n \n\n \n##########################################################################\n\nQuestions during the rebuttal period: \n\n \nPlease acknowledge the contribution of the paper for an ML venue.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper received 3 reviews with mixed initial ratings: 9,4,5. The main concerns of R1 and R3, who gave unfavorable scores, included lack of novelty and hence limited value of this work for the ML community. At the same time, R5 strongly advocates for acceptance and mentions meaningful contributions in the context of the specific application, including the new dataset. In response to that, the authors submitted a new revision and provided detailed responses to each of the reviews separately, which did not change the position of the reviewers.
The AC agrees with R1 and R3 that, even though the biometrics-related contributions are relevant, the scope of this work is too narrow and application-driven for presentation in the main track of ICLR. As a result, the final recommendation is reject.
Decision: Reject