Title: Membership Inference Attacks for Unseen Classes
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: membership inference attacks, privacy

Abstract: The state-of-the-art for membership inference attacks on machine learning models is a class of attacks based on \emph{shadow models} that mimic the behavior of the target model on subsets of held-out nonmember data. However, we find that this class of attacks is fundamentally limited because of a key assumption---that the shadow models can replicate the target model's behavior on the distribution of interest. As a result, we show that attacks relying on shadow models can fail catastrophically on critical AI safety applications where data access is restricted due to legal, ethical, or logistical constraints, so that the shadow models have no reasonable signal on the query examples. Although this problem seems intractable within the shadow model paradigm, we find that \emph{quantile regression} attacks are a promising approach in this setting, as these models learn features of member examples that can generalize  to unseen classes. We demonstrate this both empirically and theoretically, showing that quantile regression attacks achieve up to \textbf{11$\times$ the TPR} of shadow model-based approaches in practice, and providing a theoretical model that outlines the generalization properties required for this approach to succeed. Our work identifies an important failure mode in existing MIAs and provides a cautionary tale for practitioners that aim to directly use existing tools for real-world applications of AI safety.

TL;DR: The state of the art membership inference attack (shadow models) degrades in performance when some classes are not available to the attacker; we explore when and why quantile regression attacks work as a more robust solution in this setting.