# LTU Attacker for Membership Inference
## 3 Problem Statement and Methodology  
We consider the scenario in which an owner of a data Source \(\mathcal{D}_{S}\) wants to create a predictive model trained on some of those data, but needs to ensure that privacy is preserved. In particular, we focus on privacy from membership inference. The data owner entrusus an agent called [PERSON] with creating such a model, giving him access to a random sample \(\mathcal{D}_{D}\subset\mathcal{D}_{S}\) (Defender dataset). We denote by \(\mathcal{M}_{D}\) the trained model (Defender model) and by \(\mathcal{T}_{D}\) the algorithm used to train it (Defender trainer). The data owner wishes to release \(\mathcal{M}_{D}\), and eventually \(\mathcal{T}_{D}\), provided that certain standards of privacy and utility of \(\mathcal{M}_{D}\) and \(\mathcal{T}_{D}\) are met. To evaluate such utility and privacy, the data owner reserves a dataset \(\mathcal{D}_{R}\subset\mathcal{D}_{S}\), disjointed from \(\mathcal{D}_{D}\), and gives both \(\mathcal{D}_{D}\) and \(\mathcal{D}_{R}\) to a trustworthy Evaluator agent. The Evaluator tags the samples with dataset \"membership labels\": _Defender_ or _Reserved_. Then, the Evaluator performs repeated rounds, consisting in randomly selecting one Defender sample \(d\) and one Reserved sample \(r\), and giving to an LTU Attacker an almost perfect attack dataset \(\mathcal{D}_{A}=\mathcal{D}_{D}-\{\texttt{membship}(d)\}\cup\mathcal{D}_{ R}-\{\texttt{membship}(r)\}\), removing only the membership labels of the two selected samples. The two unlabeled samples are referred to as \(u_{1}\) and \(u_{2}\), with each being equally likely to be from the Defender dataset. We refer to this procedure as \"Leave Two Unlabeled\" (LTU), see Figure 1. The LTU Attacker also has access to the Defender trainer \(\mathcal{T}_{D}\) (with all its hyper-parameter settings), and the trained Defender model \(\mathcal{M}_{D}\). This is the worst-case scenario in terms of attacker knowledge/access, because the only other information that can be used to attack the membership, would be to already have the membership labels. The attacker is taskedwith correctly predicting which of the two samples \(d\) and \(r\) belongs to \(\mathcal{D}_{D}\) (independently for each LTU round, forgetting everything at the end of a round). We use the _LTU membership classification accuracy_\(A_{\mathit{Itu}}\) from \(N\) independent LTU rounds (as defined above), to define a global privacy score as:  
\[\mathtt{Privacy}= \min\{2\ (1-A_{\mathit{Itu}}),1\}\] \[\pm 2\sqrt{A_{\mathit{Itu}}(1-A_{\mathit{Itu}})/N}\, \tag{1}\]  
where the error bar is an estimator of the standard error of the mean (approximating the Binomial law with the Normal law, see, e.g., [22]). The weaker the performance of the LTU Attacker (\(A_{\mathit{Itu}}\simeq 0.5\) for random guessing), the larger Privacy, and the better \(\mathcal{M}_{D}\) should be protected from attacks. We can also determine an **individual membership inference privacy score** for any sample \(d\in\mathcal{D}_{D}\) by using that sample for all \(N\) rounds, and only drawing \(r\sim\mathcal{D}_{R}\) at random (Similarly, we can determine an individual non-membership inference privacy score for any sample \(r\in\mathcal{D}_{R}\) by using that sample for all \(N\) rounds, and only drawing \(d\) at random). See the example in Appendix C.  
The Evaluator also uses \(\mathcal{D}_{\mathit{uE}}=\mathcal{D}_{R}\) to evaluate the **utility of the Defender model**\(\mathcal{M}_{D}\). We focus on multi-class classification for \(c\) classes, and measure utility with the _classification accuracy_\(A_{D}\) of \(\mathcal{M}_{D}\), defining utility as:  
\[\mathtt{Utility}= \max\{(c\ A_{D}-1)/(c-1),0\}\] \[\pm c\sqrt{A_{D}(1-A_{D})/|\mathcal{D}_{R}|}\, \tag{2}\]  
Although the LTU Attacker is all knowledgeable, we still need to endow it with an algorithm to make membership predictions. In Figure 2 we propose a taxonomy of LTU Attackers. In each LTU round, let \(u_{1}\) and \(u_{2}\) be the samples that were deprived of their labels.