BenchPress: Analyzing Android App Vulnerability Benchmark Suites In recent years, various benchmark suites have been developed to evaluate the efficacy of Android security analysis tools. Tool developers often choose such suites based on the availability and popularity of suites and not on their characteristics and relevance due to the lack of information about them. In this context, based on a recent effort, we empirically evaluated four Androidspecific benchmark suites: DroidBench, Ghera, ICCBench, and UBCBench. For each benchmark suite, we identified the APIs used by the suite that were discussed on Stack Overflow in the context of Android app development and measured the usage of these APIs in a sample of 227K realworld apps (coverage). We also identified securityrelated APIs used in realworld apps but not in any of the above benchmark suites to assess the opportunities to extend benchmark suites (gaps).

Android App Merging for Benchmark SpeedUp and Analysis LiftUp In the field of software analysis a tradeoff between scalability and accuracy always exists. In this respect, Android app analysis is no exception, in particular, analyzing large or many apps can be challenging. Dealing with many small apps is a typical challenge when facing microbenchmarks such as DROIDBENCH or ICCBENCH. These particular benchmarks are not only used for the evaluation of novel tools but also in continuous integration pipelines of existing mature tools to maintain and guarantee a certain qualitylevel. Considering this latter usage it becomes very important to be able to achieve benchmark results as fast as possible. Hence, benchmarks have to be optimized for this purpose. One approach to do so is app merging. We implemented the Android Merge Tool (AMT) following this approach and show that its novel aspects can be used to produce scaled up and accurate benchmarks. For such benchmarks Android app analysis tools do not suffer from the scalabilityaccuracy tradeoff anymore. We show this throughout detailed experiments on DROIDBENCH employing three different analysis tools (AMANDROID, ICCTA, FLOWDROID). Benchmark execution times are largely reduced without losing benchmark accuracy. Moreover, we argue why AMT is an advantageous successor of the stateoftheart app merging tool (APKCOMBINER) in analysis liftup scenarios.