Poster: Testability-driven security and privacy testing for Web Applications Luca Compagna SAP Security Research Giancarlo Pellegrino CISPA Helmholtz Center for Information Security Davide Balzarotti Eurecom Martin Johns Technical University Braunschweig Angel Cuevas Universidad Carlos III de Madrid Battista Biggio Pluribus One Leyla Bilge Norton Lifelock Fabian Yamaguchi ShiftLeft Inc Matteo Meucci IMQ Minded Security Abstract—Modern web applications play a pivotal role in our digital society. Motivated by the many security vulnerabilities and data breaches routinely reported on those applications, we initiated the EU TESTABLE research project to address the main challenges of building and maintaining web applica- tions secure and privacy-friendly. The ultimate goal is to lay the foundations for a new integration of security and privacy into the software development lifecycle (SDLC), by proposing the novel idea of combining two metrics to quantify the security and privacy risks of a program, i.e., the testability of the codebase (via the novel concept of ”testability patterns”) and the indicators for vulnerable behaviors. We have already achieved promising results by applying our research proposal in the area of static analysis security testing (SAST) [2]. Hundreds of tarpits—code instructions challenging for SAST tools—have been identified and cap- tured in testability patterns that we used to measure SAST tools effectiveness as well as to improve the ”SAST testabil- ity” of open source applications via code refactoring routines removing the tarpits. More than 180 new vulnerabilities have been uncovered after refactoring and confirmed by open source applications’ owners. We believe similar promising results can also be achieved in other areas such as dynamic analysis, privacy, and machine learning. Index Terms—web, testability, risk, measurement, metrics, security, privacy, ML 1. Introduction Web applications are ubiquitous, and they are used in a multitude of different domains. According to the 2020 Edgescan Security Report, “Web application security is where the majority of risk still resides” [1]. This is confirmed by the fact that most of the recent data breaches took advantage of the poor security of web applications. Software security testing plays a fundamental role to mitigate this problem. In particular, developers regularly use static code analysis and automated testing tools to verify their application and identify vulnerabilities. But existing solutions are often limited in their ability to automatically discover security problems. When problems are not detected is always challenging to know for sure what the testing tools left uncovered. While threat modeling and risk assessment method- ologies are often adopted by industry at early phases of the software development lifecycle, they are too far from the implementation details and therefore they can only set high-level recommendations for later phases, such as testing. Computing risk measures that are closer to the application code itself can be used to complement early risk assessment phases, and could help to prioritize the testing effort. However, this area has so far mainly focused on determining the likelihood that an application contains a vulnerability (what we call vulnerability indicators). Therefore, even when these indicators exist, they provide little guidance to the developers on how the corresponding risk could be mitigated. For instance, knowing that a web application has a high risk of containing vulnerabilities (even when none have been detected during development and testing) is a piece of information that can be difficult to translate into actionable insights. The poor state of web security is further exacerbated by two rapidly emerging aspects. First, more and more web applications integrate machine learning (ML) components at the code or at the service level to im- plement business functions or to protect web applications from abuse1. Unfortu