Membership Inference Attacks and Privacy in Topic Modeling
Authors: Nico Manzonelli
Wanrong Zhang
Salil Vadhan
Venue: Accepted by TMLR ()

Abstract: Recent research shows that large language models are susceptible to privacy attacks that infer aspects of the training data. However, it is unclear if simpler generative models, like topic models, share similar vulnerabilities. In this work, we propose an attack against topic models that can confidently identify members of the training data in Latent Dirichlet Allocation. Our results suggest that the privacy risks associated with generative modeling are not restricted to large neural models. Additionally, to mitigate these vulnerabilities, we explore differentially private (DP) topic modeling. We propose a framework for private topic modeling that incorporates DP vocabulary selection as a pre-processing step, and show that it improves privacy while having limited effects on practical utility.

Published in Transactions on Machine Learning Research (09/2024)
Membership Inference Attacks and Privacy in Topic Model- ing
Nico Manzonelli
nicomanzannell@gaaa.coom
John A. Paulson School of Engineering and Applied Sciences
Harvard University
Wanrong Zhang
John A. Paulson School of Engineering and Applied Sciences
Harvard University
Salil Vadhan
John A. Paulson School of Engineering and Applied Sciences
Harvord University
Reviewed on OpenReview: //tp::///peereviieee net/ forum? id- 5MWW5
Abstract
Recent research shows that large language models are susceptible to privacy attacks that infer aspects of the training data. However, it is unclear if simpler generative models, like topic models, share similar vulnerabilities. Tn this work, we propose an attack against topic models that can confidently identify members of the training data Latent Dirichlet Allocation. Our results suggest that the privacy risks associated with generative modeling are not restricted to large neural models. Additionally, to mitigate these vulnerabilities, we explore differentially private (DP) topic modeling. We propose a framework for private topic modeling that incorporates DP vocabulary selection pre-processing step, and show that it improves privacy while having limited effects practical utility.
1
Introduction
Deep learning models' propensity to memorize training data presents many privacy concerns. Notably, large language models are particularly susceptible to privacy attacks that exploit memorization (Carlini et al. 2021 )023). this paper, we aim to investigale whether simpler probabilistie models, such as topic models, raise similar concerns, Our specific focus lies on examining probabilistic topic models like Latent Dirichlet Allocation (LDA) (Blei et al. 2003).
Topic modeling is unsupervised machine learning (ML. method that aims to identify underlying themes or topics within a corpus. Despite the recent successes of large language models (LLMs), Probabilistic topic models are still widely used due to their interpretability and straightforward implementation for text analysis. Furthermore, topic models continue to be developed for a variety of applications and downstream tasks like document classification, summarization, generation [Boyd-Graber al., 2017 Wang al. 2019).
Researchers apply topic models across variety of domains where privacy concerns arise. For example, many studies in the medical domain use topic models to analyze datasets with sensitive attributes (Mustakim et al, 2021). Additionally, topic models are widely used for various government or national defense applications. For instance, researchers applied LDA to better understand Twitter activity aimed at discrediting NATO during the Trident Juncture Exercises in 2018 (Uyheng et al. 2020). To ensure ethical and responsible use of ML, is crucial to consider the privacy implications associated with topic modeling.
Probabilistic topic models like LDA serve as a basic Bayesian generative model for text. Neural topic models and language models learn complex generative processes for text. based observed patterns in the

---

Published in Transactions on Machine Learning Research (09/2024)
training data. However, LDA requires the generative process for documents to be explicitly defined priori with far fewer parameters and a Bag-of-Words (BoW) text representation. Investigating LDA susceptibility to privacy attacks provides broader context for the privacy vulnerabilities researched in more complex ML models. Additionally, our work informs practitioners who may opt to use topic models under the false impression that they do not. share the same vulnerabilities L.Mss.
To explore the privacy in topic modeling, we conduct membership inference attacks (MIAs) which infer whether not specific document used to train LDA. We propose attack based on LDA-specific query statistic designed to exploit memorization. This query statistic is integrated into the Likelihood Ratio Attack (LiRA) framework introduced by Carlini et al. (2022). We show that our attack can confidently infer the membership of documents included in the training data of LDA which indicates that the privacy risks in generative modeling are not restricted to large neural models.
To mitigate such vulnerabilitics, we explore differential privacy (DP) for topic modeling. DP acts a direct defense against MIAs by providing statistical guarantee that the output of some data analysis is indistin- guishable regardless of any one user's inclusion in the analysis. While several DP topic modeling algorithms in the literature attempt to protect individual privacy, previous works largely disregard the privacy of the
model's accompanying vocabulary set (Zhao et al. 2021 Huang & Chen 2021 Huang et al., 2022 Decarolis et al. 2020 Park et al. 2016 Wang et. al., 2022 Instead, they consider the vocabulary set as given or public information. However, in practice, the vocabulary set is derived from the training data and can leak sensitive information. We propose an algorithm for DP topic modeling that provides privacy in both the vocabulary selection and learning method. By incorporating DP vocabulary selection into the private topic modeling workflow, our algorithm enhances privacy guarantees and bolsters defenses against the LiRA, while having minimal impact on practical utility.
2 Background and Notation
To begin, let us establish notation and provide formal definition of topic models.
Definition 2.1 (Topic Model) For vocabulary size of V and h topics, a topic model & E [0,1 is matrix whose rows sum to 1 representing each topic as a distribution over words.
We let FD) denote the process of learning a model parameterized by latent variables in G on a corpus D with M documents drawn from the underlying data distribution ID. The function fg incorporates the learning algorithm to estimate I's latent variables and returns the learned topic model o. Our definition assumes that the primary objective of topic modeling is to estimate + While & is also associated with other latent variables that may define the document generation process, best achieves the goals associated with topic modeling by summarizing the relevant themes in the corpus.
2.1 Latent Dirichlet Allocation
Blei et al. 2003) define LDA where each document is assumed to contain a mixture of topics. LDA assumes that each document d E D is represented by a k-dimensional document-topic distribution 0 ~ Dirichlet and the entire corpus is represented by & V-dimensional topic-word distributions á (Brichlct()). Fur- thermore, it follows a simple bag-of-words approach where each word in document is generated by first sampling topic from the document's topic distribution and then sampling word from the chosen topic's distribution over words.
The process of estimating & presents a Bayesian inference problem typically solved using collapsed Gibbs sampling or variational inference (Griffiths & Steyvers 2004 Hoffman et al.] 2010). Each entry 2 repre- sents the probability of drawing word w from topic z. The entire topic model & represents the likelihood of each word appearing in each topic, which can be used estimate the document-topit topic distribution for any given document.
2

---

Published in Transactions on Machine Learning Research (09/2024)
2.2 Memorization and Privacy
Understanding memorization in machine learning is critical for trustworthy deployment. Neural model's large parameter space and learning method introduce training data memorization which increases with model size or observation duplication (Carlini et al., 2023 Feldman & 'Zhang 2020). Memorization introduces vulnerabilities to attacks on privacy, such as membership inference, attribute inference or data extraction attacks (Shokri et al. 2017: Song & Raghunathan 2020: Carlini et al. 2021).
Schofield et al. (0177) note that document duplication in topic models concentrates the duplicated document's topic distribution over less topics, and briefly refer to this phenomenon memorization. Their findings are consistent with studies memorization and text duplication in large language models (Carlini et al, 2023 In this work, we invostigate the memorization and privacy of topic models using membership inference allacks (MIA).
2.3
Membership Inference Attacks
In MTA the adversary learns if specific observation appeared in the model's training data (omer al. Shokri et al., 2017). These attacks serve the foundation for other attacks like data extraction, and could violale privacy alone if the adversary learns that an individual contributed to dalaset with sensitive global attribute (i.e. dataset containing only patients with disease X).
2008
Prior to attacking ML models, researchers explored exploiting aggrogate statistics relcased in genomo-wide association studies (GWAS) (Homer et al. 2008). Shokri et al. (2017) introduced one of the first MIAs on ML models. While most MIAs exploit deep learning models, few studies investigate topic models. Huang et al. (2022) propose three simple MIAs to evaluate their privatized LDA learning algorithm. However, they evaluate their attacks using precision and recall and fail to show that their attack confidently identifies members of the training data.
To evaluate an MIA's ability to confidently infer membership, we examine the attack's true positive rate (TPR) at low false positive rates (FPR) (Carlini et al. 2022). Receiver operator characteristic (ROC) and area under the curve (AUC) analysis may be useful, but we must be wary of the axes. To accurately capture the TPR at the low FPRs of interest, we visualize ROC curves using log-scaled axes.
3 MIAS Against Topic Models
In this section, we detail our MIA against topic models based on the LiRA framework, and empirically demonstrate its effectiveness.
3.1 Threat Model
We consider threat model where the adversary has black-box access to the learned topic-word distribution Φ but not other latent variables captured by G. The adversary can not observe intermediate word counts, learning method, or syper-parameters used while learning 4. We also assume that the adversary has query access to the underlying data distribution ID; which allows them to train shadow models on datasets drawn from D.
Under the typical query model, we could consider queries as a request to predict document-topic distribution 0 given document. However, 0 would hold little meaning without access 4. Furthermore, clever adversary could query many specially selected documents to reconstruct 4. Therefore, remove this level of abstraction and assume that the adversary is given direct access to t as the output of our topic model.
3.2 Attack Framework
In the LIRA framework of Carlini et al. (022), the adversary performs hypothesis testing to determine whether or not target document was the training data. Specifically, we consider T., { U Dd))D TB} and (ou((d) { (o(D) MD) where T., (d) is the distribution of + learned
3

---

Published in Transactions on Machine Learning Research (09/2024)
with d, and (d) is the distribution of + learned without d. Given an observed Pobss would like to construct the likelihood ratio test. statistic A as follows:
A d) = p( P Pobs Tin (d)
(1)
p( obs Tout (d)
where obs (d) is the probability density function of Pbbs under (d) Carlini et al. 2022)
However, the ratio in Equation | intractable because requires integrating over all with all possible D with and without d. Instead, the adversary applies a carefully chosen query statistic Ç (#, - R on all & to reduce the problem to 1-dimension. The test proceed by calculating the probability density of {( d) under estimated normal distributions from (d) (c( & U D + DJ and T (d) = (((0,00 d) Φ D
A( d) Pls( p((( oha d) (Pouts )ott))
oiu))
(2)
where a and o² are the mean and variance of Th (d) and Tour(d).
In the online LiRA, the adversary must train N shadow topic models with and without d to estimate (i)(d) and (d). Appendix A contains the full online LiRA algorithm. To ease the compulational burden of training N shadow topic models for each target document, Carlini et al. (2022) propose an offline variant of the LiRA. In the offline LiRA, the adversary can evaluate any d after learning a collection of N shadow topic models once by comparing ((Pobe, d) to a normal estimated from Towr(d) with a one-sided hypothesis test. Appendix B contains the full offline LiRA algorithm.
Designing is crucial to the attack's success. In supervised learning scenarios, the adversary can directly query the model and use the loss of an observation to inform the statistic (Carlini et al. 2022). However, because topic models present an unsupervised learning tasks, the adversary encounters the challenge of identifying an informative statistic that can be computed efficiently using 4. Hence, Ç must be carefully tailored to topic models for the attack to be effective.
3.3 Designing an Effective Query Statistic Ç
Effective model query statistics for the LiRA framework must satisfy the following criteria: the statistic should increase for when d is included in the training data (Ti > Toue(d) to enable the offline LiRA, and the estimated distributions (d) and (d) are approximately normal to allow for parametric modeling. Ideally, the statistic should be related to how the model may memorize the training data to provide an intuitive interpretation for attack performance.
First, we consider statistics from the simple MIAs on LDA presented by Huang al. (2022). They propose three statistics derived from the target document's 's estimated topic distribution 0: the entropy of 0, the standard deviation of 9, and the maximum value 0. The intuition behind their chosen statistics comes from the observation that a document's 's topic distribution tends to concentrate in few topics when included (or duplicated) in the training data (Schofield al. 2017). However, Huang et al. (2022) apply global thresholds to these statistics instead of using them query statistics to derive the LiRA test statistic as in Equation
We propose an attack that leverages the LiRA framework with an improved query statistic to directly exploit LDAs generative process. Our query statistic is a heuristic for the target document's likeliiod under LDA. To compute this statistic on for given document d, the adversary maximizes the loglliieihood of the document. over all possible document-topic distributions such that:
4

---

Published in Transactions on Machine Learning Research (09/2024)
(( max log pod ()
max log( 02 $ =.,.):
where we optimzie over 0 E 10.1 s.t. Σ. 02 1. In practice, we use SciPy's out-of-the-box optimization methods to estimate {(0, d) VVrtance et al. 2020).
Compared to the statistics proposed by Huann et al. 22022; our statistic more directly exploits LDAs gener- ative processes and is based on the observation that a document's likelihood under the target model increases when included in the training data. Therefore, using Ç allows to better reason about memorization in topic models. We verify our criteria for the proposed statistic in Appendix C
Additionally, Appendix contains an evaluation of other candidate statistics where we show that our statistic significantly outperforms other statistics based on the attacks in Huang et al. (0022).
The LiRA with statistic Ç directly exploits memorization and per-example hardness. The attack accounts for the natural differences in Ç on various documents by estimating the likelihood ratio under distributions based on Te (d) and (d). This enables the LiRA to outperform simple attacks like in Huang et al. (2022) that use global thresholds on the other query statistics based on 0. Therefore, we propose the LiRA with statistic Ç a stronger alternative to attack topic models.
3.4 Memorization and Per-Example Hardness
To verify aspects of memorization and per-example hardness for probabilistic topic models, we estimate and visualize (n.()) (d) and (d) in an experiment similar to Carlini et al. (2022) and Feldman & Zhang (2020). Consistent with their findings, we show that outlying and hard-to-fit observations tend to have a large effect on the learned model when included in the training set for LDA. The histograms in Figure 1 display the estimated distributions for various types of documents.
When included in the training data for 4, longer and outlying documents increase (( d). We note that the word probability 2.,w certain topics dramatically changes for words that appear infrequently in D or words that occur many times in d. Together, these factors affect the learned topic model & and shift (($, d).
The simple fact that the model is more likely to generate specific documents after inclusion in the training data hints toward memorization. Because the generative process for documents in LDA is extremely simple compared to neural language models, they do not learn verbatim sequences of words. Instead, topic model's ability to "memorize" the training data is due to learning based on word co-occurrences.
Long documents are inherently harder to fit than shorter documents. Because the log-likelihood for a document d is the sum of independent log-probabilities for generating each word in d, the magnitude of ((),d) is naturally higher for longer documents. Additionally, the model may struggle to capture coherent topic structures for longer documents because they tend to contain words that span a wider range of topics.
The LiRA turns our observations on memorization and per-example hardness into MIA. When the words in a document become more common in the training set, & will reflect the new word co-occurrences in a few specific topics which tends to increase (( d). This effect is amplified when the document contains many rare words from the vocabulary set. Therefore, the al lacker can easily differentiale between T,m (d) and (d) for long and outlying documents.
3.5 Attack Evaluation Set-Up
We evaluate our attack against three datasets: TweetRumors, 20Neusgroup and NIPSI] We apply standard text pre-processing procedures for each datasct: removal of all non-alphabetic characters, tokenization, re- moval of stop words, removal of tokens longer than 15 characters, removal of tokens shorter than 3 characters, and lemmentization Appendix D provides the basic data profile for each dataset after pre-processing.
The evaluation code and source for each dataset. included in the Availability section Tokenization, stop words, and lemmatization implemented python package nitk https hwww://wwww///.o http://wwww.nt.ttt/em nlt
5

---

Published in Transactions on Machine Learning Research (09/2024)
Long Outlier
Short Outlier
25
15
10
-120
-100
800
-60
-40
-120
-100
-80
-60
40
max log p(d) 6)
max log pld) 8. 0)
Long Inlier
Inlier
25
document without document
-120
-100
-80
-60
-40
-120
-100
-80
-60
440
max log p(d) 8 Ø)
max log pld 0)
Figure 1: Histograms of the statistic (( evaluated on different types of documents in TweelRumors when d E Dtrein (blue) and when d & Dtroin (orange). Outliers are documents that contains many words that appear infrequently in D and inlicrs contain many words that appear frequently D. Long documents contain more words than a standard deviation away from the mean document length and short documents contain less. The word count for the inlying document is within one standard deviation of the mean.

---

Published in Transactions on Machine Learning Research (09/2024)
Table Attack TPR at 0.1% FPR with 128 Shadow Models
Dataset
Our Attack Online LiRA
Huang et al.
Huang et al.
2022
Maximum Posterior
Standard Deviation
Entropy
TweetRumors
12.8%
0.19%
0.19%
0.18%
NIPS (k: = 10)
44.9%
1.69%
1.69%
1.31 %
20Newsgroup
21.1%
0.57%
0.57%
0.53%
To initiate the attack, we randomly sample half of the data learn and release Lode- Next, we train N shadow topic models by repeatedly sampling half of the data to simulate sampling from D. Like in Carlini et al. 2022), this setup creates some overlap between shadow model training data and target model training data which presents strong assumption made to accommodate the smaller size of our datasets. We do not expect performance to significantly decrease using disjoint datasets as observed by Carlini et al. 2022).
We compare our LiRA against each the attacks presented by Huang et al. (2022 To replicate their altacks, we randomly sample half of the dataset. to learn $ Pobs- Using Pohs, we estimate each documents' topic-distribution ! and compute the maximum posterior, standard deviation, and entropy on 0. We directly threshold the each of these statistics to evaluate membership for every document in the dataset.
For each experiment, we learn + using scikit-learn's implementation of LDA with default learning param- eters. 3 For TweetSet and 20Newsgroup we set the number of topics k to 5 and 20 respectively based on the data's known distribution across topics: TweetSel contains tweets collected on 5 major news events and 20Newsgroup contains newsgroup documents across 20 different newsgroups. For NIPS, we vary for exper- imentation purposes. In practice, k is known by the attacker who can observe the dimensions of t. We learn N 128 shadow models, replicate each experiment 10 times and report our results across all iterations.
We interpret A a predicted membership score where higher value indicates that the document is more likely to be member of the training data. We empirically estimate attacks' performance by evaluating the TPR at all FPRs and plot the attack's ROC curve on log-scaled axes.
3.6 Attack Evaluation Results
First, we compare our online attack performance against the attacks in Huang al. (2022) at an FPR of 0.1% in Table 1 Figure 2 displays the ROC curves for the online and oflline variant of our attacks. Table [ and Figure 2 demonstrate that LiRA outperforms each of attacks in HHang et al. 2022
Because the LiRA considers the likelihood-ratio of d, it accounts for per-example hardness and dominates the attacks by Huang et al. 2022) at all FPR's. As noted by Carlini et al. (2022) attacks that directly apply global thresholds do not consider document level differences on the learned model and fail to confidently identify members of the training data. Stronger attacks, like our LiRA, should be used to empirically evaluate the privacy associated with topic models.
To understand how the number of topics in & influence attack performance, vary the number of topics k on NIPS and allack the resulting model. Table 2 shows the the atlack performance vary k. As k increases, we see that TPR increases at an FPR of 0.1%. Additionally, we note minor differences between online and offline attack performance.
The number of parameters in & grows linearly by the length the vocabulary set as k increases. With more topics and more parameters, document's word co-occurrence structure is typically better represented in the document's 's learned topic distribution. Consequently, increasing & enhances the impact of including the document in the training data resulting in better attack performance.
Overall, our findings demonstrate that despite their simple architecture, topic models exhibit behavior that resembles memorization and are vulnerable to strong MIAs.
httpps//scckiit- learn.org/

---

Published in Transactions on Machine Learning Research (09/2024)
Attack Performance on TweetRumors
Attack Performance on 20Newsgroups
Attack Performance on NIPS
10a4
10"4
1073
Tue Robiive Race
The Poatuve Rate
10'
20
Online URA
Onlin LIA
Online URA
Office LIA
Offine
Office LIA
10'4
Maximum Posterior
10
Maximum Posterior
12a4
Macmmm Posterior
Decsation
Standard Deviation
Dextation
Eniropy
Entrapy
Estropy
Figure 2: Online and Offline ROC Attack Comparison Each Dataset (128 Shadow Models, NIPS k=
Table 2: Attack TPR at FPR of 0.1% While Varying on NIPS
Number of Topics k
Online LiRA
Offline LiRA
5
31.8%
31.8%
10
44.9%
43.6%
15
60.5%
56.0%
20
67.9%
63.2%
25
72.5%
70.2%
4 Private Topic Modeling
Designing private ML solutions is an active field of research. In topic modeling, the literature is largely focused on differential privacy (DP). DP acts as direct defense against privacy attacks like MIAs by limiting the effect one data point can have on the learned model. Furthermore, DP provides clear, quantifiable method for reasoning about privacy loss.
4.1 Differential Privacy
DP provides strong theoretical guarantees of individual-level privacy by requiring the output of some data analysis to be indistinguishable (with respect to small multiplicity factor) between any two adjacent datasets and x'.
Definition 4.1 (Differential Privacy Dwork et al., 2006b Let M X" R be randomized algorithm. For any 6 2 O and E [0, 1]. we say that M is (e, diffferential private if for all adjacent databasos x, x' E X and every S R
Pr(M(r) SP(M)) +
The notion of adjacency is critical. For any text dataset, we say that two corpora D and D' are author-level adjacent if they differ by one author's documents. This notion enforces DPs promise of individual level privacy by considering adjacency at the user level.
If two corpora and D' differ by one document, then adjacency the document-level. This relaxed notion of adjacency can satisfy author-level adjacency if we assume that each document has a unique author. For corpora the most relaxed notion of adjacency is word-level adjacency. Two corpora D and D' are word-level adjacent if they differ by one word in one document.
To achieve differential privacy we must add noise whose magnilude is scaled to the worst-case sensitivity of the target statistics from all adjacent datasets. Researchers have explored word-level adjacency to control sensitivity, but this approach offers weak privacy guarantecs. For instance, most DP collapsed Gibbs sampling algorithms for learning LDA rely on perturbing word-count statistics by adding noise scaled to word-level
adjacency (Zhao et al. 2021
Huang & Chen 2021 Huang al. 2022
8

---

Published in Transactions on Machine Learning Research (09/2024)
There variety of other algorithms for DP topic modeling: Park et al. (2016) propose DP stochastic variational inference to learn LDA, Decarolis et al. 20000) use a privatized spectral algorithm to learn LDA, and Wang et al. (2022) satisfy DP by adding noise directly to Q. These algorithms satisfy various notions of DP and use differing notions of adjacency. Table in Appendix summarizes each DP topic modcling implementation.
While each DP topic modeling algorithm has their advantages and disadvantages, there are some common themes across implementations. First, the iterative nature of learning algorithms for LDA posses difficult composition issue when managing the privacy loss. Next, many implementations relaxed notions of adjacency to control sensitivity. Finally, none of the cxisting methods address the privacy concerns with releasing the vocabulary set corresponding to +.
4.2 DP Vocabulary Selection
Releasing the topic-word distribution & without ensuring the privacy of the accompanying vocabulary set can lead to privacy violations because the vocabulary set is derived directly from the data. Topic models with comprehensive vocabulary sets tend to include many infrequently used words from the corpus which makes them more vulnerable MIAs (as shown in section 3.4p. Furthermore, even if the values in & are private, the overall release of the topic-word distribution can not satisfy DP because the vocabulary set accompanied by & is not private.
To illustrate this point, consider scenario where we have corpus D with a single document d. If we apply DP topic modeling to release + without changing the vocabulary set, then we release all of the words in d. This clearly compromises the privacy of the document in D. If we chose not to release the vocabulary set, & loscs practical interpretability To address this issuc, it necessary explore methods for differentially private vocabulary selection.
DP vocabulary selection can be formalized as the DP Set- Union (DPSU) (oop et al. 2020 If we assume each author i contributes subset W; C U of terms in their documents, DPSU seeks to design (c, DD algorithm to release a vocabulary set S S U,W, such that the size of S is as large as possible. Researchers first approached this problem with the intent to release the approximate counts of as many items as possible in U,W, (Korolova et al. 2009 Wilson et al. 2020 Their algorithms guarantee privacy by constructing a histogram of U,W,, adding noise to each word frequency in the histogram, and releasing all word frequencies that fall above a certain threshold. These implementations upper- upper-bound bound the sensitivity based on the maxi- mum the number of words contributed by all users (Au max; (Wa)) to account for users contributing more than one word. However, most users contribule significantly less than An so this method wastes sensitivity. Gopi et al. (2200) propose algorithms that build weighted histograms with update policies that help control sensitivity. Intuitively, the update policy stops incrcasing the weights of words in the histogram after they reach some cutoff I which is set using the parameter a. Carvalho et al. (2022) propose an improved algo- rithm for DPSU which is well fit. for vocabulary selection because allows for one user to contribute the same term multiple times.
4.3 Fully Differentially Private Topic Modeling
We propose high-level procedure for fully DP topic modeling (FDPTM), in Algorithm 1 FDPTM composes the privately selected vocabulary set S and the private topic-word distribution $. Theorem 4.1 states the privacy guarantees associated with FDPTM. The proof is deferred to Appendix
Algorithm Fully Differentially Private Topic Modeling (FDPTM)
Require: Corpus D, DP vocabulary selection algorithm Mj and DP topic modcling algorithm M2. Apply standard pre-processing and tokenization: Dpre +- PRE(D)
Apply DP vocabulary selection to corpus: S M, (Drre)
4: Remove all words Drre if w & S: Dzan SAN(D S)
5: Learn O using DP topic modeling: T t- M2(Dsan)
6: Release: Topic-word distribution and corresponding vocabulary set. S
9

---

Published in Transactions on Machine Learning Research (09/2024)
Theorem 4.1 (Privacy Guarantee of FDPTM) If M1 for selecting the vocabulary set satisfies (#1, 61 )- DP and M2 for topic modeling satisfies (22,P, then the overall release of t satisfies (41 (21 01 + DD)
Implementing FDPTM requires a few key considerations. First, the process depends on carefully tuning the privacy parameters and other parameters within M1 and M2. Second, the data curator must ensure that M, and M2 satisfy the same notion of differential privacy and adjacency.
Overall, FDPTM is modular framework for releasing meaningful topic-word distributions. Although evaluations focus LDA, other topic models for specific uses can easily fit into the proposed procedure.
4.4 FDPTM Evaluation Set-Up
We empirically evaluate our FDPTM using the Rumors dataset with A: = 5. We ensure author-level privacy via document-level adjacency under the assumption that each document has unique author. This assumption necessary for controlling sensitivity and is common to most DP NLP solutions. We accomplish this by using the DPSU solution proposed by Carvalho et al. (2022) for vocabulary selection and DP LDA learning algorithm by Zhu et al. (2016) because they satisfy the same notion of privacy. For DPSU, we fix the privacy parameter d 10 " and set the parameter a 3 control the cut-off value like in Carvalho et al. 2022).
To evaluate utility, we first vary the privacy loss parameter for DPSU and analyze utility when LDA is non- private. We study the interaction between DP vocabulary selection and DP LDA by fixing the privacy loss parameter for one mechanism and varying the privacy loss parameter for the other mechanism. Specifically, we first fix E2 = for DP LDA and vary & for DPSU. Then, we fix E1 = and vary Eg. We vary our privacy loss parameters in intervals across common e choices for DP ML solutions. Typically, e > 1 provides very weak theoretical privacy guarantees, but tend be effective for providing empirical privacy.
We evaluate the utility of the FDPTM using topic coherence (Mimno et al. 2011] which can be useful proxy for measuring the interpretability of topics. Topic coherence for topic / is defined as
M =
D(vm (1) +1
coherence(t: v(t))=
log
(4)
((1 (9)
m= 1=1
where D(v) is the document frequency of word (i.e. the number of documents where word occurs), D(v,e') is the co-document frequency of words and e' (i.c. the mumber of documents where v and e' both occur), and v(t) {of).... ...., ((t)) is list of the M most probable words in topic t. A high topic coherence score suggests that the top M terms have higher co-document frequencies which makes the topic easier to understand because the terms more semantically similar or coherent. In our analysis, select each topic's top M = 10 words and report the average across each topic.
We also test FDPTM against the online LiRA to empirically evaluate the privacy of FDPTM. For each attack we learn 64 shadow models. Like before, we conduct attacks while varying E E {1, 5, 10} and fixing E2 = 5. Then, we fix E = and vary 82. We repeat each experiment times and report the results across each iteration.
4.5 FDPTM Evaluation Results
Figure 3 displays topic coherence as we increase ět. Figure displays topic coherence as we vary either and hold the other constant. We include the average vocabulary size as increase E in Appendix G|
Topic coherence remains stable after == =, suggesting that increasing the privacy loss for DPSU may not affect topic coherence after a certain point. When comparing coherence while increasing S2, we see that Eι decreases coherence regardless of 62. In Figure 4 coherence plateaus around ε = 3 while increasing 62 continues to boost coherence. Consequently, increasing E yields diminishing utility returns faster.
Figure presents the ROC curves that explore the how the interaction between DP vocabulary selection and DP LDA affect attack performance. We gain some empirical privacy under FDPTM by decreasing LiRA
10

---

Published in Transactions on Machine Learning Research (09/2024)
Toplc Ceherence a6 Varies
-120
-120
-230
-120
to 2950
Figure 3: Topic coherence as increases and LDA is not private. The non-private baseline for topic coherence is very small compared to the figure (~= -1117) The error bars represent one standard deviation from the mean topic coherence.
Topic Cakeronce as Varies
-122
-1340
11383
+1300
Figure 4: Topic Coherence as & Increases. The blue line shows the the results while varying e for DPSC and holding 82 = 3. Orange displays results for varying E2 for DP LDA while holding é1 = 3.
11

---

Published in Transactions on Machine Learning Research (09/2024)
for DPSU
Varying
107
10'
10
10"
10
The Postter 3ate
30-7
10
10"1
Non- Private (t =*)
Ron- Mon-Private (t (=))
10-
10
10"4
Patac
Figure 5: Attack ROCs while varying privacy loss parameters for DPSU and DP LDA. On the right- hand side, we fix E2 for DP L.D and vary 61 E (1,5, 10) for DPSU. On the left-hand side, we fix &1 and vary 22 at the same intervals. We provide the non-private baseline in red for reference.
performance all FPRs. Generally, we see that attack performance decreases similarly we decrease either privacy parameter. Furthermore, we gain empirical privacy by decreasing LiRA performance at all FPRs.
DPSU decreases attack performance because we shrink the size of the vocabulary set, limiting the number of parameters in D, and forcing each documents' topic distribution to look more like other documents' topic distributions. The nature of learning on word co-occurrences forces to reflect identifiable changes when infrequently used words suddenly become more common to the training data. By removing these words with DPSU, we limit the ability for outliers have large effects on the topic model.
Our results indicate that dedicating most of the privacy budget to DP LDA, rather than DPSU, increases utility at the same privacy level. Inluitively, we can inject less noise inlo DP LDA, and more into the vocabulary selection algorithm (#1 < E2) to increase model interpretability at the same global privacy loss (e, + +2) and similar empirical privacy against the LiRA. This approach allows us balance privacy and utility in topic modeling.
5 Conclusions
In our work, we show that topic models exhibit aspects of memorization and successfully implement strong MIAs against them. Although probabilistic topic models do not memorize verbatim sequences of text like language models, they do memorize word frequency and occurrence. In some instances, knowledge of the frequency of certain terms in document constitutes a privacy violation regardless of their word ordering.
To combat MIAs and memorization, we propose a modular framework for implementing better private topic models using differential privacy. Overall, the addition of DP vocabulary selection to the DP topic modeling workflow is important for guaranteeing private, interpretable releases of क. Not only does DP vocabulary selection allow for fully DP releases of 4 it also provides an effective defense against MIAs.
We highlight the greater need for continued development in the field of probabilistic topics models for text memorize their training data, then it's only inevitable that LLMs and neural topic models memorize their training data. As ML models continue to become more sophisticated and widely used, privacy concerns become increasingly relevant.
ML. If simple
12

---

Published in Transactions on Machine Learning Research (09/2024)
Limitations
While our results provide insights on topic models' privacy, there are few limitations of our methodology and experiments. First, because we rely on BoW document. representation, with certain word frequencies are part of the training set. In practice, this may cause the attack to realize more false positives. The datascts used in our attack evaluations did not include documents with overlapping word combinations, and most reasonable documents contain very few sensible alternative word combinalions which eases the implications of this limitation.
only identify that documents
Additionally, our utility asscssments for FDPTM are constrained by the limitations of automated topic evaluation methods. While different measures give us different insights into the topic model, the best evaluations typically based on human inspection (Hoyle et al. 2021). Other utility metrics that evaluate topic models with metrics that estimate model fit, like perplexity, tend to be negatively correlated with topic interpretability (Chang et al. 2009). Therefore, we use topic coherence because it is associated with the human interpretability of topics.
Finally, while our results are convincing, we conduct them with the same learning process (Ja based on LDA via sabbs-sampling) and hyperparameters for the the shadow and target models with access to the entire topic-word distribution 4. In many practical use cases, researchers may use LDA or release all of ④. A mismatch between shadow and topic models fg may impact attack accuracy, but suspect to lesser extent, than in neural models. Additionally, we believe that clever adversaries can reconstruct & given partial or alternative releases of 4. Determining the affect that different fg and releases of d have on attack performance is left for future research.
Ethics Statement
The privacy attacks presented in this paper are for research purposes only. The methodology and code should be used responsibly with respect for privacy considerations. Each of the datasets used in our experiments are open access and do not pose a direct threat to the privacy of the users who contributed to them.
Acknowledgement
N.M. is supported through the MIT Lincoln Laboratory (MITLL) Military Fellowship and by NSF grant BCS- 2218803. W.Z. is supported by a Computing Innovation Fellowship from the Computing Research Association (CRA) and the Computing Community Consortium (CCC). S.V. supported by NSF grant BCS-2218803 and Simons Investigator Award. The authors would like acknowledge the MITLL Supercomputing Center for providing compute and consulting resources that contributed to results.
Availability
The URLs for the datasets Twee/ 20Neusgroup and NIPS ltttps://ww.zuuaagaa datasets, http://qwone com/ jasonjjssnn//a /aaos///eeew/rrouup and https://archive. ics uci /dc.e/at/////ttttet/// bag+of+words respectively. The code for the attack simulations is available at
nicomanzonelll/pppcc_mddda
http://oppicmmodel_taaateee _atacks
References
Yoav Benjamini and Yosef Hochberg. Controlling the False Discovery Rate: A Practical and Powerful
Approach to Multiple Testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1): 189-300,11 1995.
David M Blei, Andrew Y Ng, and Michael I Jordan. Latent Dirichlet Allocation. Journal of machine Learning research, 3(Jan):993 1022, 2003.
13

---

Published in Transactions on Machine Learning Research (09/2024)
Jordan L Boyd-Grabes Yuening Hu, David Mimno, and et al. Applications of Topic Models, volume 11. Now Publishers Incorporated, 2017.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ülfar Erlingsson, Alina Oprea, and Colin Raffel. Extracting Training Data from Large Language Models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633-2650. USENIX Association, August 2021. ISBN 078-1-939133-244 URL https://wwir usenix.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. Membership Inference Attacks From First Principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1897-1914, 2022. doi: D..109//PP4221422222 98336699..
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang, Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreviee http:///ww....eoommmrmraaaaaaaaa 1cK.
Ricardo Silva Carvalho, Ke Wang, and Lovedoep Singh Gondara. Incorporating Item Frequency for Differen- tially Private Set Union. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 9504-9511, Jun. 2022.
doi: 10. (3600//aaiivv361922211
URL https:/ aaai org/indees hwtphndwe..pp.ppp.eemee ahp/AAAI/
article/view/21118
Jonathan Chang, Sean Gerrish, Chong Wang, Jordan Boyd-graber, and David Blei. Reading Tea Leaves: How Humans Interpret Topic Models. In Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta (eds.), Advances in Neural Information Processing Systems, volume 22. Curran Associates, Inc., 2009. URL https:// httpee////coeeeeeeeeeeeee neurips nwurips.cc/paper.... filess/pppr/2009/fii 69288225b331441fc44400000000 Paperrrpfaappprppar.
Chris Decarolis, Mukul Ram, Seyed Esmaeili, Yu-Xiang Wang, and Furong Huang. An End-to-End Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithm. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Leurning, vol- ume 119 of Proceedings of Machine Learning Research, pp. 2431- PMLR, 13-18 Jul 2020. URL https://proceediin mlr. pttpees//119/decrrolll0000000000 html
Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our Data, Our- selves: Privacy Via Distributed Noise Generation. In Serge Vaudenay (ed.), Advances in Cryptology EUROCRYPT 2006, pp. 486-503, Berlin, Heidelberg, 2006a. Springer Berlin Heidelberg. ISBN 978-3-540- 34547-3.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating Noise to Sensitivity in Private Data Analysis. In Lecture notes in computer science, Lecture Notes in Computer Science, pp. 265 284, Berlin, Heidelberg, 2006b. Springer Berlin Heidelberg. ISBN 3540327312.
Vitaly Feldman and Chiyuan Zhang. What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation. In Proceedings of the 3/th International Conference on Neural Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Sivakanth Gopi, Pankaj Gulhane, Janardhan Kulkarni, Judy Hanwen Shen, Milad Shokouhi, and Sergey Yekhanin. Differentially Private Set Union. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Re- search, pp. 3627 3636. PMLR, 13 Jul 2020. URL https://proceedings. mlr pttps//v19//opp22000 html
Thomas Griffiths and Mark Steyvers. Finding Scientific Topics. In Proceedings of the National Academy of Sciences, volume 101, pp. 5228 5235, 2004. doi: p0.77//pa..000077511000 URL hattps:/wwwt www.pnas....ppoo....wo.......... org/doi/abs/10.1u 1033/////0.0000//1110/// 1073/ 0307752101
14

---

Published in Transactions on Machine Learning Research (09/2024)
Matthew Hoffman, Francis Bach, and David Blei. Online Learning for Latent Dirichlet Allocation. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta (eds.), Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010. URL ptttps://proceeedngg.. neur cc/
6ppp:72200011111117///1110106660 pappeaaa.aaaaapa..p.pp........ pdf
Nils Homer, Szaboles Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling, John V. Pearson, Dietrich A. Stephan, Stanley F. Nelson, and David W. Craig. Resolving Individuals Contributing Trace Amounts of DNA to Highly Complex Mixtures Using High-Density SNP Genotyping Microarrays. PLoS Genet, 1000117 08 2008. doi: 10. 187111222ooural pgen. 1000167
100016. URL http://dx. doxx.doi... org/
Alexander Hoyle, Pranav Goel, Andrew Hian- ChannChenng Denis Peskov, Jordan Boyd-Graber, and Philip Resnik. Is Automated Topic Model Evaluation Broken? The Incoherence of Coherence. In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Aduances in Newral Information Processing Systems, volume 34, pp. 2018 2033. Curran Associates, Inc., 2021. URL http://tp///p/rreeeednng.. neurips. cc/paper_r
pdf
T. Huang, S.Y. Zhao, H. Chen, J. Liu, and Y. Xu. Improving Parameter Estimation and Defensive Ability of Latent Dirichlet Allocation Model Training Under Rényi Differential Privacy. Journal of Computer Science and Technology, 37)6):1382-1337, 2022. doi: 10.1007 0077//113900002 1-29000222-225---
Tao Huang and Hong Chen. Improving Privacy Guarantee and Efficiency of Latent Dirichlet Allocation Model Training Under Differential Privacy. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 143-152, Punta. Cana, Dominican Republic, November 2021. Association for Compu- tational Linguistics. doi: 1..18633/v12200 http:////1/222...dnnn-eeeeee findings-emnlp.14. URL https:///cllatthllggg org/2021. findings-emnlp... 14]
Aleksandra Korolova, Krishnaram Kenthapadi, Nina Mishra, and Alexandros Ntoulas. Releasing Search Queries and Clicks Privately. In Proceedings of the 18th International Conference on World Wide Web, WWW '09, pp. 171-180, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605584874. doi: 10.1145/152677 11511526779.1526333 URL https::/ddo..rrg//n 111////21111//527701 1526733
David Mimno, Hanna M. Wallach, Edmund Talley, Miriam Leenders, and Andrew McCalhum. Optimizing semantic coherence in topic models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '11, pp. 262 272, USA, 2011. Association for Computational Linguistics. ISBN 9781937284114.
M. Mustakim, Retantyo Wardoyo, Khabib Mustofa, Gandes Retno Rabayu, and Ida Rosyidah. Latent Dirichlet Allocation for Medical Records Topic Modeling: Systematic Litcrature Review. In 2021 Sixth International Conference on Informatics and Compuling (ICIC), pp. 1-7, 2021. doi: 10.1109/IC1C54005. 2021.9632993
Mijung Park, James Foulds, Kamalika Chaudhuri, and Max Welling. Private Topic Modeling. Tn Workshop on Private Multi-Party Machine Learning at NIPS, 2016.
Alexandra Schoficld, Laure Thompson, and David Mimno. Quantifying the Effects of Text Duplication on Semantic Models. In Proceedings of the 2017 conference on empirical methods in natural language processing, pp. 2737-2747, 2017.
Samuel S Shapiro and Maurice B Wilk. Analysis of Variance Test for Normality (Complete Samples). Biometrika, 52(3-4):591 1111991611,1111111111111111111 1965.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18, 2017. doi: 10.1109/SP.2017.41.
Congzheng Song and Ananth Raghunathan. Information Leakage in Embedding Models. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Commmunications Security, CCS '20, pp. 377 390.
15

---

Published in Transactions on Machine Learning Research (09/2024)
New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450370899. doi: 10.1145/ 3372297.34112700 URL https::/doi..orroeoee
1145/3372297 34477
Abhradeep Guha Thakurta and Adam Smith. Differentially Private Feature Selection via Stability Argu- ments, and the Robustness of the Lasso. In Shai Shalev-Shwartz and Ingo Steinwart, (ods.), Proceedings of the 26th Annual Conference on Learning Theory, volume 30 of Proceedings of Machine Learning Research, pp. 850, Princeton, NJ, USA, 12-14 14 Jun 2013. PMLR. URL httpp::/prooceeings... mlr wall..pres//ee /v30//////////////// Guha 13. html
Joshua Uyheng, Thomas Magelinski, Ramon Villa-Cox, Christine Sowa, and Kathleen M. Carley. Inter- operable Pipelines for Social Cyber-Security: Assessing Twitter Information Operations during NATO Trident Juncture 2018. Comput. Math. Organ. Theory, 26(4):465-488 dec 2020. ISSN 1381-298X. doi: 10. 10.1007/s100588-019-000888
URL (ttttp////aa..orr//00000//////3
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Ev- geni Burovski, Pearu Peterson, Warren Weckessor, Jonathan Bright, Stéfan J. van der Walt, Matthew Breut, Joshua Wilson, Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C Carey, ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Melhods, 17:261-277 272, 2020. doi: 00111066/-41192-1----060
Han Wang, Jayashree Sharma, Shuya Feng, Kai Shu, and Yuan Hong. A Model-Agnostic Approach Differentially Private Topic Mining. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1835-1845, 2022.
Wenlin Wang, Zhe Gan, Hongteng Xu, Ruiyi Zhang, Guoyin Wang, Dinghan Shen, Changyou Chen, and Lawrence Carin. Topic-Grided Variational Ento-encoder for Text Generation. In Proceedings of the 2019 Conference of the North Americon Chapter of the Association for Computational Linguis- tics: Human Language Technologies, Volume (Long and Short Papers), pp. 166-177, Minncapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-N0 URL
attps://aclanthholoyy wtt.://hhooooo..ooot//100000 org/N19-1013
Royce J. Wilson, Celia Yuxin Zhang, William Lam, Damien Desfontaines, Daniel Simmonn-Marengo, and Bryant Gipson. Differentially Private SQL with Bounded User Contribution. In Proceedings on Privacy Enhancing Technologies, volume 2020, pp. 230-250, 2020. doi: / 2020-0025.
Fangyuan Zhao, Xuebin Ren, Shusen Yang, Qing Han, Peng Zhao, and Xinyu Yang. Latent dirichlet allo- cation model training with differential privacy. IEEE transactions on information forensics and security, 16:1290-1305, 2021. ISSN 1556-6013.
Tianqing Zhu, Gang Li, Wanlei Zhou, Ping Xiong, and Cao Yuan. Privacy-preserving Topic Model for Tagging Recommender Systems. Knowledge and information systems, 46(1):33-58 2016. ISSN 0219-1377.
A
The Online LiRA
The LiRA on topic models follows directly from the LiRA on supervised ML models proposed by Carlini et al. 2022) However, the attack topic models differs because we use statistic Ç queried क. Algorithm contains the online LiRA.
B The Offline LiRA
The offline LiRA on topic models also follows directly from the offline LiRA on supervised ML models proposed by Carlini et al. (2022). The offline variant benefits over the online variant. because lines 3-9
16

---

Published in Transactions on Machine Learning Research (09/2024)
Algorithm 2 Online LiRA Topic Models
1: Inputs: Target topic-word distribution: Pahe, target document: d, shadow model iterations: N, statistic function: Ç, data distribution: D, and topic model fg.
2: Returns Likelihood Ratio Test Statistic: A
Tim =
Initialize empty sets
Tout = 0 for N times do
++D D
Sample auxiliary data from distribution > Train shadow model with targel document
Pin [d(
8:
Tin T U Uf(( ]
9:
Pout + (g(((DD
b Train shadow model without target document
10: Tout t Ufc( d
11: end for
12: Nin N (Ta)), (n), var(T
Estimate normals
13: Nout 4- --Mmmaan T oout), var oTo)))
14: p(c( .a Nin).
Calculate test statistic
plc(Do Nows
15: Return A
be executed regardless of the target document d, and lines 11-17 can be repeated for any d after estimating Tout once. Additionally, note that the test statistic changes in line 15. Algorithm 3 contains the offline LiRA.
Algorithm Offline LiRA on Topic Models
Inputs:
Target topie-word distribution: Pabss target document: d, shadow model iterations: N, statistic function: ç, data distribution: D, and topic model fg. Returns Likelihood Ratio Test Statistic: A
Tout = =0
b Initialize empty set
4: for N times do
5:
D+D
b Sample auxiliary data from distribution
6:
Pout + fD(())
D Train shadow models without target document
Tout Tout U outt
end for
Tout =
Calculate Statistic Across T
10: for Dour in Tout = f} do
11: Tout + Tout U UfC(Lous d)}
12: end for
13: Nout +- NN(MmmannTT out), var aur(Tow))
P Calculate Test Statistic
14: A 1- Pr(Nout > ((Pobs. d)]
15: Return A
C Evaluating Candidate Query Statistics
In this section, we provide analysis for the proposed statistic w.r.t. the two query statistic requirements detailed in 33
the statistic should increase for d when is included in the training data (Tin Tour to enable the offiine LiRA.
The estimated distributions T., (d) and T oue (d) are approxmaatel normal to allow for parametric modeling.
17

---

Published in Transactions on Machine Learning Research (09/2024)
203
375
330
Statsdic
Deviation
Figure 6: Boxplot of KL Divergences between Nin(d) and Non (d) evaluated on 3000 documents.
Table 3: Shapiro-Wir Wilk Tests with FDR Control for 6000 Tests on Each Statistic (a = .05)
Statistic
Number of Rejections
Rejection %
Our Statistic
1335
22.25%
Negative Entropy
3584
59. 73%
Logit Max Posterior
5191
86.52%
Standard Deviation
4621
77.02%
Additionally, compare the proposed statistic to alternative statistics based on the attacks in Huang et al. 2202)) and show that our proposed statistic outperforms the alternatives.
Huang et al. (2022) propose three statistics: the entropy of 0, the standard deviation of 0, and the maximum value in 8 where 0 is document's estimated topic distribution Huang et al. (2022). Each statistic requires estimating the document's topic distribution Å. They estimate using sampling techniques, but for these experiments, we choose to estimate Ô by performing the optimization from Eq. for increased efficiency and consistency with the proposed statistic in Eq. 3
To lailor each statistic to fit within the LiRA framework, we make some minor modifications to the statistics proposed by Huang et al. (2022). Because entropy naturally decreases when the document is included in the training data, we use the nogative entropy to satisfy the first requirement. Additionally, because the maximum value in 0 is bound between [0,1] and tends to concentrate toward for outliers, we apply the logit transformation to promote adherence to the normality requirement, as illustrated by [arlini et al. (2022).
To evaluate the query statistic selection criteria, fit online attack using N 256 shadow models using each of our candidate statistics. We sample 1000 documents from each dataset to extract (I)(d) and (du for each candidate statistic. We assess the first requirement by evaluating the KT-divergence between two normal distributions estimated from (i)(d) and (dy A positive KL-divergence between inr(Tin (d))) and Nowe = ([mean(Toull)
Nix = A (TT statistic is greater when included in the training data. Figure 5 displays boxplot for the KI. divergences for cach statistic. We note that all KL-divergences in Figure are greater than 0 which indicates that T.,(d)
var (arTTou(()) indicates that the
Tour (d).
We evaluate the normality requirement by performing Shipiro-Wilk test for each (d) and Tour(d) with control for the False Discovery Rate (FDR) using the Benjamini-Hochben Procodure (Shapiro & Wilk 1965; Benjamini & Hochberg 1995). The null hypothesis is that the data is drawn from a normal distribution. Table 3 contains the the number of tests rejected with FDR control for each statistic at a significance level of .05. Our analysis indicates that the distributions of (d) for our statistic tend to be normal more often than the alternatives. We can expect encounter fewer rejections as we increase N (i.e. increase the sample size of T oou/
18

---

Published in Transactions on Machine Learning Research (09/2024)
30
Budnd
Buded Pio or
Lng Par hudo
Legit Nicieaa
300
137
307
Figure 7: Online Attack Comparison Across Statistics Each Dataset (256 Shadow Models, NIPS k=
Table Dataset Profile After Pre-Processing
Dataset
Number of Documents M
Average Document Length
Vocabulary Size V
TweetRumors
5,698
9
5,942
NIPS
1,494
893
10,346
20Newsgroup
18,037
84
74,781
Table 5: Brief Summary of the Existing Literature on DP Topic Modeling
Authors
Notion of DP
Notion of Adjaceucy
Learning Method
Other Technical Details
Zhu al. (2016) Zhu sl.
2016
z-DP
document-level
CGS
Zhao et al. (2021) Zhao st 2021
e-DP
word-level
CGS
Huang and Chen (20 Frlang & Chen
2021
(e,0)-DP
word-level
CGS
sub-sampling
Huang al. (2022) Huang
Rényi-DP
word-level
CGS
Park al. (2018) Park et 2016
(e,5)DDP
document-level
VI
moments accountant and sub-sampling
Decarolis et al. (2020) Decarols el ml
2020
Rónyi-DP
document-level
SA
local seasitivity from peepose-eed-rceeaa pain-free smooth sensitivity
Wang et sl. (2022) Waing et 6)
2022
Rényi-DP
author-level
PH
Finally, in Figure 7 we evaluate the attack performance of all statistics across each dataset. We show that the LiRA with our proposed statistic dominates the other candidate statistics at all FPRs. While each of the candidate statistics satisfies requirement 1, the long-whiskers and outliers associated with the logit maximum posterior statistic in Figure 0 suggest that it would outperform other statistics because the model can more easily differentiate between (d) and (o). However, the statistics' lack of normality seems hinder performance.
D
Data Profile
Table contains the data profile for each dataset after pre-processing.
E
Existing DP Topic Models
Table 5 sunmarizes centralized DP topic modeling algorithms available in the literature. The learning methods included are Collapesed Gibbs Sampling (CGS), variational inference (VI), spectral algorithm (SA), and Model Agnostic Post-Hoc (PH).
F Proof of Theorem 4.1
First, let M2 be 6,1,6,)-D0 DDP vocabulary selection algorithm that returns a private vocabulary set S, and M2 be (E2,82) 82)-DP topic modeling algorithm that returns a private topic-word distribution Φ. Now, let PRE be function that takes corpus D and applies standard pre-processing procedures, and SAN be function
19

---

Published in Transactions on Machine Learning Research (09/2024)
Vecobulary Longth ao Vancs
1000
1200
2000
to DPS
Figure 8: The length of the vocabulary set as for DPSU increases. The original vocabulary size is 5,942.
that takes a corpus and a vocabulary set S and removes all words w E D w & S. Finally, let Mg represent the FDPTM algorithm such that for a corpus
M, (M1
Ms(SAN(PREEDD; M1 (PRE(D)))))
Definition F.1 (k-Stability (Thakurta & Smith 2013 A function f U - R k-stable on input D if adding or removing any clements from D does not change the value of f, that is, f(D) fD') for all D' such that DAD' < k. We say is stable on D if (at least) 1-stable on D, and unstable otherwise.
Lemma F.1 (Composition with Stable Functions (Thakurta & Smith 2013) Let f be a stable function, and let M an (e, 8)-DP algorithm. Then, their composition M((()) satisfies (s,6)-DP.
The functions PRE and SAN are stable algorithms because each document is processed independently of the others based on a standard set of rules. Simply, adding or removing document from the corpus does not affect the functions behavior on other documents,. Therefore, PRE and SAN are stable functions.
Via our definition of M1, and using the fact that PRE stable function, then the first term Ma. Mi (PRE(D)), satisfies (E1, 61 DDD. The second term of M, applies M2 which directly depends on M1 and the stable functions PRE and SAN. Therefore, via adaptive composition, M, is (E1 + E2 62)-DP (Dwork
et al. 2006a). D
G Vocabulary Size as E Increases
Figure contains ciguains plot for the size of the vocabulary set as the privacy loss parameter e1 for DPSU increases.
20


Reviews:
[{"review_id": "J7PeqDoLCi", "reviewer": "Reviewer_Ev4o", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is generally written well for the most parts, and the experimental results look quite impressive. A few questions and comments as follows,\n\n1. The assumption that the adversary has access to the underlying data distribution, and that $\\mathbf T$ is normally distributed is quite strong (as also corroborated by the results in Appendix C). \n2. What is the $\\mathcal G$ for the true model and the shadow models? Clearly that will have a huge impact on the performance of the attack. Alternatively, in the wild, $\\mathcal G_{true}$ could be quite different from $\\mathcal G_{shadow}$\n3. The authors overly rely on the literature at places instead of explaining the intuition in their own words. For instance the authors cite Carlini et al. (and Mimno et al) extensively but please elaborate upon why \n\n>  attacks that directly apply global thresholds do not consider document level differences on the learned model and fail to confidently identify members of the training data.\n\n>  topic coherence can be a useful proxy for measuring the interpretability of topics. \n\nand at other places.\n\n4. The part on DP should be expanded especially i) why and how  DPSU and DP LDA works ii) what is \"coherence\" and why is it a good metric\n5. It is difficult to understand the impact of different privacy budget $\\epsilon$ as a number on its own as opposed to the performance on some downstream task. For instance is $\\epsilon = 1$ at which the attack is almost useless too strong a privacy constraint to provide any utility for the downstream task? \n6. Could you please elaborate upon why\n> We do not expect performance to significantly decrease using disjoint datasets as observed by Carlini et al\n7.  I am not sure I understand\n>  The intuition behind our statistic is that a document is more likely generated by the target model when included the training data.\n\nIsn't the intuition behind (3) the same as what has been used by Huang et al i.e. \"document’s topic distribution tends to concentrate in a few topics when included (or duplicated) in the training data\"\n\n8.  (For my own understanding) What happens if we just use $\\Phi^2$ (with perhaps some normalized version of \\Phi)instead of $\\max_{\\theta} \\theta * \\Phi$ in equation 3?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a membership inference attack against topic models that can confidently identify members of the training data in Latent Dirichlet Allocation models, and demonstrate the underlying privacy risks. They benchmark their method on 3 different datasets and compare against well motivated baselines in the literature. The results look quite impressive. \n\nTo mitigate these privacy vulnerabilities the authors propose to use differential privacy for topic modelling. They use known mechanisms for DP secure vocab selection and LDA training to obtain differentially private topic modelling. The authors conduct experiments to benchmark  performance of privacy attacks for different privacy budgets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Some notational/minor changes,\n\n1. I presume $\\phi$ in section 2.1 is a $V$ dimensional distribution. Please mention that.\n\n2. Why is $D \\in [0,\\infty]^{M\\times V}$?\n\n3. ``We propose an $\\ldots$ topic modeling provides.'' Missing \"that\"\n\n4. Below equation (1), do you really mean to say that $\\Phi$ is infeasible to calculate or do you mean $\\mathbf T$?\n\n5. For the experiment I presume that the threshold for Huang et al. was found by holding a fixed FPR?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YA1WZ5KZT0", "reviewer": "Reviewer_NtZa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is mostly easy to read and understand, and introduces the related work well. The contributions themselves are very modest, but useful. The limitations of the work are also discussed well.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper develops a novel summary statistic for membership inference attacks (MIA) on latent Dirichlet allocation (LDA), which empirically outperforms statistics from previous work. The paper also considers differential privacy for LDA. Unlike previous work, the paper considers the vocabulary of the input documents as private information, and uses a DP set union algorithm to release the vocabulary privately. This is combined with a DP LDA algorithm from previous work. The combination of the DP algorithms empirically defends against the MIA, and retains some utility at high values of $\\epsilon$.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Minor changes \n- The experimental results should have uncertainty estimates, at least for the tables.\n- The performance for the full mechanism would be easier to interpret from a figure with the total epsilon and topic coherence, with some fixed privacy budget split.\n- The caption of Figure 5 says that both panels fix $\\epsilon_2$ and vary $\\epsilon_1$.\n- Definition 4.1 uses the term \"neighboring\", while the following text uses the term \"adjacent\". Making these consistent would improve clarity.\n- I think that \"our statistic tend to exhibit normality\" is a bit too strong given the 22% rejection rate in Table 4. Something like \"our statistic is closer to normal than the alternatives\" would be more appropriate.\n\nQuestions\n- Is there a concrete privacy budget split that the experiments suggest? For example x% of total $\\epsilon$ to DP LDA. \n- Why did you choose the DP LDA algorithm of Zhu et al. (2016)?\n- What is the confidence level for Table 4?", "broader_impact_concerns": "The paper includes an ethics statement which covers the relevant issues.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "F9NuvEJ7lo", "reviewer": "Reviewer_hNKb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Understanding the privacy concerns of topic modeling is an interesting and important topic. The paper is generally well written and has good experiments showing the effectiveness of the proposed frameworks.\n\nHowever, the paper would benefit from having more description of the methods from literature that are used in this work, for instance the DPSU solution and the DP LDA algorithm used in section 4.2 and 4.4.\n\nMoreover, the values used for the experiments seem random, it is unclear why those values are chosen, for instance the number of topics in section 3.5 and the values of $\\epsilon_1$ and $\\epsilon_2$ in section 4.4.\n\nSome additional questions:\n- In table 2, it is said that \"we see that TPR increases at all FPRs\", but the FPR is 0.1%, so how is this at all FPRs?\n- How restrictive is assuming that each author has a single document and are there any ideas on how expensive/messy it would be to break this assumption?\n\nAdditionally, there are some minor comments:\n- In the introduction:\n     * \"... the membership of documents included the training data...\" -> \"... the membership of documents included _in_ the training data...\"\n     * \"We propose an algorithm for DP topic modeling provides privacy...\" -> \"We propose an algorithm for DP topic modeling _that_ provides privacy...\"\n- In section 2.3: \"... fail to show that their attack confidently identify members of the training data.\" -> \"... fail to show that their attack confidently identifies members of the training data.\"\n- Section 3.2: \"Hence, $\\zeta$ must be careful tailored to topic models for the attack to be effective\" -> \"Hence, $\\zeta$ must be carefully tailored to topic models for the attack to be effective\"\n- In section 3.4:\n     * \"The simple fact that model is more likely ...\" -> \"The simple fact that _the_ model is more likely ...\"\n     * \"Long document's\" -> \"Long documents\"\n- Section 3.5: Lemmentization, do you mean Lemmatization?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies privacy in topic models, specifically the Latent Dirichlet Allocation (LDA). Firstly, a membership inference attack (MIA) based on on the Likelihood Ratio Attack (LiRA) framework is proposed against topic models to show that an adversary can infer membership of documents in the training data. The experiments show that using the proposed attack, an adversary can predict the membership of a certain document in the training data  especially as the number of topics increases. Moreover, the experiments show that the proposed attack outperforms the previously used MIA attacks proposed by Huang et. al. (2022).\n\nAs a second step, a framework is proposed to incorporate differential privacy (DP) to the topic modeling algorithm where DP is used in the vocabulary selection, as well as in the learning of the model.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The changes requested are directly related to the weaknesses stated in the previous section:\n-  Please add more description of the methods from literature that are used in this work so the paper is self contained and more understandable.\n- Please explain how the values are picked in the experiments.\n- Please fix the typos/errors stated above.\n\nIt is important that the above comments are addressed, or if the authors disagree, that it is explained why it is not possible or not necessary in their view.", "broader_impact_concerns": "The broader impacts are sufficiently addressed.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers are happy with the author responses and recommend acceptance without further revisions.

The paper looks mostly publishable to me, but I would ask the authors to check the references for the camera ready: some are missing publication details such as conference names and many proper names (e.g. Rényi, Dirichlet) in lowercase.
Decision: Accept (poster)