Despite our growing reliance on mobile phones for a wide range of daily tasks, their operation remains largely opaque. A number of previous studies have addressed elements of this problem in a partial fashion, trading off analytic comprehensiveness and deployment scale. We overcome the barriers to largescale deployment (e.g., requiring rooted devices) and comprehensiveness of previous efforts by taking a novel approach that leverages the VPN API on mobile devices to design Haystack, an in-situ mobile measurement platform that operates exclusively on the device, providing full access to the device's network traffic and local context without requiring root access. We present the design of Haystack and its implementation in an Android app that we deploy via standard distribution channels. Using data collected from 450 users of the app, we exemplify the advantages of Haystack over the state of the art and demonstrate its seamless experience even under demanding conditions. We also demonstrate its utility to users and researchers in characterizing mobile traffic and privacy risks. Similar to previous approaches [52], Haystack leverages Android's standard VPN interface to capture outbound packets from applications. However, rather than tunneling the packets to a remote VPN server for inspection, Haystack intercepts, inspects, and forwards the user's traffic to its intended destination. This approach gives us raw packet-level access to outbound packets as well as flow-level access to incoming traffic without modifying the network path, and without requiring permissions beyond those needed by the VPN interface. Haystack therefore has the ability to monitor network activity in the proper context by operating locally on the device. For example, a TCP connection can be associated with a specific DNS lookup and both can be coupled with the originating application. Further, we design Haystack to be extensible with new analyses and measurements added over time (e.g., by adding new protocol parsers and by supporting advanced measurement methods such as reactive measurements [10]), and new features to attract and educate users (e.g., ad block, malware detection, privacy leak prevention and network troubleshooting). Haystack is publicly available for anyone to install on Google Play and has been installed by 450 users to date [40] . We discuss the design and implementation of Haystack in §3 and §4, and evaluate its performance and resource use in §5. Our tests show that Haystack delivers sufficient throughput (26-55 Mbps) at low latency overhead (2-3 ms) to drive high-performance and delay-sensitive applications such as HD video streaming and VoIP without noticeable performance degradation for the user. While we consider our Haystack implementation prototypical in some respects (such as UI usability for nontechnical users), it has already provided interesting insights into app usage in the wild: in §6 we present preliminary findings about the adoption of encryption techniques, report on local-network traffic interacting with IoT devices, study app provenance and the use of thirdparty tracker services, and give an outlook on potential future applications.
State of the art work in REF proposes Haystack system which aims to allow unobtrusive and comprehensive monitoring of network communications on mobile phones entirely from user space.
Haystack: A Multi-Purpose Mobile Vantage Point in User Space
arXiv: Networking and Internet Architecture
Computer Science