Abstract-The pervasiveness of mobile devices increases the risk of exposing sensitive information on the go. In this paper, we arise this concern by presenting an automatic attack against modern touchscreen keyboards. We demonstrate the attack against the Apple iPhone-2010's most popular touchscreen devicealthough it can be adapted to other devices (e.g., Android) that employ similar key-magnifying keyboards. Our attack processes the stream of frames from a video camera (e.g., surveillance or portable camera) and recognizes keystrokes online, in a fraction of the time needed to perform the same task by direct observation or offline analysis of a recorded video, which can be unfeasible for large amount of data. Our attack detects, tracks, and rectifies the target touchscreen, thus following the device or camera's movements and eliminating possible perspective distortions and rotations In real-world settings, our attack can automatically recognize up to 97.07 percent of the keystrokes (91.03 on average), with 1.15 percent of errors (3.16 on average) at a speed ranging from 37 to 51 keystrokes per minute. Since more and more people use mobile devices, there is an increasing risk of inadvertently exposing sensitive information. For instance, attackers may break into mobile devices by exploiting vulnerabilities exposed via wireless links. Even when digital attacks are ineffective, malefactors may closely follow and observe the victim's portable device or use recording mechanisms and extract the target information. However, this strategy is highly inefficient from an attacker's viewpoint, especially when the target information is time sensitive (e.g., onetime passwords) or ample (e.g., long emails). We designed and implemented a practical, automatic attack against touchscreen mobile devices. Through a quasi real-time processing, our attack accurately recovers the sequence of keystrokes input by the user. Balzarotti et al. proposed in [1] an attack that, albeit limited to desktop scenarios, is similar in spirit to ours; however, their approach relies on conditions that well suit desktop scenarios (i.e., camera and mechanical keyboard in relative fixed, perpendicular position) but prevent its extension to mobile settings, which are characterized by keyboard movements and skewed or rotated viewpoints. To the best of our knowledge, our work is the very first attempt to implement an automatic shoulder-surfing attack against touchscreen mobile devices. We designed our attack specifically for mobile scenarios and dynamic conditions. In particular, it requires no specific positioning of the spying camera nor of the target device as it compensates the natural movements of both target device and attacker's camera. This makes it suitable for "on the go" applications. Our attack provides accurate text recognition,
Maggi et al. REF implemented an automatic shoulder-surfing attack against touch-enabled mobile devices.
A fast eavesdropping attack against touchscreens
2011 7th International Conference on Information Assurance and Security (IAS)
2011 7th International Conference on Information Assurance and Security (IAS)
Computer Science