VirSec-A Virtual Environment as Cost-Effective Test Bed for Usability and Security Evaluation of Authentication Schemes

Today, evaluations of authentication schemes have been limited in many ways. In particular, evaluating novel systems such as smart homes or autonomous cars, and also larger scale systems (e.g. ATMs) represent a challenge. Namely, the design and evaluation of authentication schemes is costly, time consuming and limited to conditions that can be physically replicated in the lab. The objective of this project is to enable and expand the possibilities for researchers to evaluate authentication schemes in a feasible, cost-effective and timesaving way. This proposal seeks to address the limitations of current methods for evaluating authentication schemes by presenting VirSec, a virtual environment that can be used as a cost-effective test bed for usability and security evaluations of authentication schemes. 1 Background Information My bachelor thesis, titled ​“The bird is the word: a usability evaluation of emojis inside text passwords” was published in OzCHI 2017, and received an honorable mention award (top 5% submissions). This sparked my interest in Usable Privacy and Security research. Later, I studied the use of obfuscation to preserve privacy in lifelogging videos in the Advanced Topics in HCI course, and I designed a novel biometric authentication scheme for my Usable Privacy and Security course. My experience in designing novel authentication mechanisms, as well as developing and evaluating prototypes (both softwareand hardware-driven), encouraged me to pursue this area further. 2 Related Work and Potential Research Questions Challenges of Evaluating Authentication Schemes Evaluation of human-centred novel authentication schemes typically takes place within lab or field studies. Previous research investigated the security and usability of authentication on traditional platforms (e.g., handheld devices, personal computers) [1,2,4,5,6,8,9]. This is often done by implementing a prototype and deploying it for actual use by participants. This becomes challenging when authentication is evaluated on large scale systems (e.g., ATMs, smart homes, autonomous cars, public displays), because prototyping these systems is time-consuming, costly, and it is often infeasible to create all the evaluation conditions in the lab. For example, consider a case where a researcher wants to evaluate observation attacks against authenticating at an ATM from 5 different angles; this would require placing 5 cameras at the required positions to record the authentication process at a real world ATM. We propose VirSec, a system that uses virtual reality (VR) to allow researchers to create costand time-efficient high-fidelity prototypes to evaluate authentication schemes. In the same example above, evaluating a virtual replica of the system would 1) cut down costs as hardware solutions will no longer be required in the evaluation process, and 2) enable infinite options for observation angles from a single 3D recording. Previous work by George et al. [3] shows that the performance of authentication schemes in virtual reality is comparable to their performance in the real world. VirSec is inspired by this previous work which shows preliminary evidence that results acquired by evaluating authentication schemes in virtual environments match those collected in real world experiments. This underlines the first research question of this project. ​R1​: ​Which and how well do results of evaluating authentication schemes in virtual reality match results acquired from similar experiments in the real world? Our first step to tackle this research question is described under Experiment 1​ below. 2.1 Authentication in Virtual Reality (VR) Few previous works already looked into transfering authentication systems that are well-established in the real world to virtual environments [3,10]. This was mainly motivated by the need for authentication in VR (e.g., to make purchases in VR), and because requiring users to take off the VR headset to authenticate would decrease the user’s immersion and overall user experience. The Transfer of Well-Established Authentication Systems to Virtual Environments Many VR applications require authentication, such as virtual shopping, and micro transactions in VR games. George et al. [3] investigated the use of PINs and Android unlock patterns, which are common on today’s mobile device, in VR. Their work was a proof-of-concept that schemes that are used for mobile devices are transferable to VR. Similarly, Yu et al. [10] implemented three password methods for VR, and found a trade-off between usability and security of password systems for VR and new upcoming challenges have to be tackled. Both works [3,10] show that virtual environments are promising for evaluating authentication schemes, and that show preliminary evidence that results acquired in VR match those acquired from similar studies in the real world. This suggests that using a virtual environment like VirSec to evaluate authentication schemes is a promising alternative to traditional methods that are costly, time-consuming and limited in various ways. 2.2 State of the Art Methods in Evaluating Authentication Schemes To cover a huge set of different conditions in which users can authenticate, it is often necessary to tweak the experimental setup after every individual session. This is not only costly, it is also time consuming. Furthermore, post-hoc manipulations of recorded data is not possible without re-running the whole experiment. In a Wizard-of-Oz study by Mecke et al. [7], they investigated and explored authentication systems for doors. A similar design of the study could be developed within a virtual environment which would decrease a) the costs of building such a prototype and b) would allow post-hoc manipulations (e.g., adding other authentication mechanisms, changing environment conditions like lighting, alternating between different feedback mechanisms such as haptic or visual feedback) without having to invite participants for an additional session. Similar to that, the use of VirSec would not only allow researchers to avoid purchasing expensive equipment to record user sessions (e.g., like in [4,5] where multiple cameras were used to record gaze interaction for authentication), a modification of the setup (e.g. generating views from different angles for observation analysis) is straightforward in VR using Unity. A virtual environment system such as VirSec would also decrease the necessity of multiple input devices. Whereas a work by von Zezschwitz et al. [9] used three different input devices (an iPhone, an iPad and a Desktop PC), VirSec would not require high-cost hardware devices and would thereby reduce development and evaluation costs. In summary, I interpret the mentioned works as a promising direction for using virtual environments as test beds for designing, developing and evaluating novel authentication schemes. However, it is necessary to build VirSec in a modular way to allow different researchers to use it for different evaluation needs. This raises the second research question. R2​: ​What are the key challenges and requirements for VirSec to be versatile to different evaluation needs? Our first step to tackle this research question is described under ​Experiment