Probabilistic Reasoning with LLMs for Privacy Risk Estimation
Authors: Jonathan Zheng
Alan Ritter
Sauvik Das
Wei Xu
Venue: NeurIPS 2025 poster ()
Keywords: LLMs
probabilistic reasoning
chain of thought
prompting
NLP
privacy risk
personal disclosures
Primary area: applications
TL;DR: We introduce a new probabilistic method and task for quantifying the privacy risk of a document with personal attributes using large language models.

Abstract: Probabilistic reasoning is a key aspect of both human and artificial intelligence that allows for handling uncertainty and ambiguity in decision-making. In this paper, we introduce a new numerical reasoning task under uncertainty for large language models, focusing on estimating the privacy risk of user-generated documents containing privacy-sensitive information. We propose BRANCH, a new LLM methodology that estimates the $k$-privacy value of a text—the size of the population matching the given information. BRANCH factorizes a joint probability distribution of personal information as random variables.  The probability of each factor in a population is estimated separately using a Bayesian network and combined to compute the final $k$-value. Our experiments show that this method successfully estimates the $k$-value 73% of the time, a 13% increase compared to o3-mini with chain-of-thought reasoning. We also find that LLM uncertainty is a good indicator for accuracy, as high variance predictions are 37.47% less accurate on average.

Probabilistic Reasoning with LLMs for Privacy Risk Estimation
Jonathan Zheng'
Sauvik Das Alan Ritter'
Wei Xu'
'College of Computing. Georgia Institute of Technology
Human Hommau-Coputtrr Interaction Institute, Carnegie Mellon University jzheng3240gatech. edu
Abstract
Probabilistic reasoning is key aspect of both human and artificial intelligence that allows for handling uncertainty and ambiguity in decision becision-making In this paper, we introduce a new numerical reasoning task under uncertainty for large language models, focusing on estimating the privacy risk of user-generated documents containing privacy-sensitive information. We propose BRANCH, new L.L methodology that estimates the k-privacy value of a matching the given information. BRANCH factorizes a joint probability distribution of personal information as random variables. The probability of each factor in a population estimated separately then combined to compute the final k-value using Bayesian network. Our experiments show that this method successfully estimates the v-value 73% the time, 13% increase compared to 03-mini with Mfain-of-thoughh reasoning. We also find that LLM uncertainty is good indicator for accuracy, as high variance predictions are 37.47% less accurate on average.
the size of the population
Introduction
Large language models (LLMs) have shown increasingly strong performance in mathematical and logical reasoning [88 65 17]. enabling exploration of a broad suite of user-facing applications. One such application is helping users understand the magnitude of privacy risks when disclosing personal information online a holy grail of usable privacy that remains elusive 63 40). For example, how much riskier is it to disclose one's precise age versus a general age category (22 vs. 18-25)? To that end, we introduce task that instructs LLMs estimate the ic number of people in the world that match the personal attributes or experiences presented in user-written messages pseudonymous online fora like Reddit [99 or anonymous user interactions with ChatGPT 992 This task serves both as new challenge for LLMS' reasoning abilities beyond conventional math and logic tests 30 49 36 and as a practical security tool to inform users about online safety.
Traditionally, privacy research has focused on quantifiable properties such as k-anonymity 80 applied by dataset owners to protect individual records, with privacy risk assessed by the success rate of re-identifying anonymized entries in database (e.g., a person Wikipedia) [56] 57
89 In
this work, we shift the focus to end-users, who are neither dataset owners nor curators, by providing an interpretable risk estimate that can be computed without access to comprehensive database containing personal records all internet users. LLMs must rely on their internal knowledge of demographic statistics from census data to reason about and estimate the identification risk based on the multiple, potentially interconnected, first-person textual disclosures. For instance, L.LMs may estimate the privacy risk of a post where user mentions that she is from Italy, 26 years old, on the spectrum, and has social anxiety (see Figure [ and 2 for more examples).
Reasoning on these joint attributes remains a significant challenge for language models. In Figure I Chain-of- Thoughh prompting estimates the prevalence rates of social anxiety and autism indepen-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Chain of Thoughts:
Instruction: How many people share all the following attributes as this user?
Conditional Dependency Errors (25.64%)
The poster is from Italy, female, 26 years old, on the spectrum, and has social anxiety.
G Gender: The population of Italy is 59,000,000. -50% are female. 29,500,000 0.5 = 29,500,000
Age: In country with around 80 "age-buckets, 29,500,000 :
80 368,750 are horn in a year.e
year.
A utism: General prevalence for women amund 0.25%. 368,750 0.0025 921.88
4. Social anxiety: Estimates for people with social anxiety are around 10%. 921.88 0.1 = 92.19
hTh prevalence of social anxiety in people on the A utism spectrum is noughly 50%)
After applying all these "filters" sequentially we get an estimate of roughly 93 people share these traits.
(Correct: 461)
Personally Identifiable Information Errors (15.38%)
The poster is entrepreneur of a software development services company called Townbridge InfoTech.
1. Occupation: There are 500 million native English speakers. 5% (25 million) of people are entrepreneurs.
2. B usiness: Roughly 10% entrepreneurs are launching IT companies. 25, 55,000,000 0.1 = =2,500,000.
3. PIl: We estimate that about 1 in 10,000 of entrepreneurs would choose the name "Townbridge InfoTech".
2,500,000 0.0001 = 250
(Business names spocific "Townsbridge InfoToch" are unique, i.c. 1 in 2,500,000)
Based on these deff-disclosures, approximately 250 people would share all of these specific details.
(Correct: 1)
Most common Chain-off Thaiu-of reasoning error types (with occurrence rates) and examples for 03-mini on Privacy Risk Estimation. Errors and correct explanations are highlighted. Chain-of- Thought struggles to model PII and capture relationships between attributes for risk assessments.
Figure 1:
dently, failing to account for the conditional dependency whereby being on the spectrum significantly increases the likelihood of social anxiety. To improve privacy risk estimation, we propose BRANCI (see Figure 2 a probabilistic reasoning framework for LLMs that represents each document as a joint distribution of personal attributes. BRANCH first factorizes this distribution by implicitly constructing a Bayesian network, capturing the interdependencies between all of the attributes. Each attribute is then transformed into textual query and individually estimated using standalone LLMs. Finally, BRANCH reconstructs the joint probability following the structure of the Bayesian network to predict an integer k, representing the number of people worldwide who share the relevant personal attributes.
We empirically evaluate BRANCH and state-of-the-art baselines in privacy risk estimation using a new dataset of user posts with human-annotated gold standard values. BRANCH accurately predicts k 72.61% % of the time, outperforming baselines by 23. 23.04%. On documents with four or more personal attributes, BRANCH is significantly better than Chain-of- Thain-of-Thought prompting due to its probabilistic model that estimates each attribute individually. BRANCH excels at estimating single-atrribute demographics with low percentage errors from ground truth records resulting in superior privacy assessments after combining these probabilities. We further evaluate LLM uncertainty as an indicator for estimation accuracy and find that predictions with high variance result in 37.47% lower accuracy.
In summary, privacy risk estimation serves both method for evaluating the probabilistic reasoning capabilities of large language models and as an application to support user-centered privacy. Inspired by prior research in the field, we develop general, cemeannitterpretable value i that helps users understand privacy risks in text, even in the absence of a database of all online users. Our work is also motivated by coman-comppter interaction (HCI) user studies 44 19 where participants expressed a desire for explanations on the severity of risks associated with personal disclosures. We envision BRANCH as a practical tool that can provide a number with reasoning chain to inform users about the potential identification risks, much like a password strength meter [38], for online privacy.
2 Related Work
Causal and Probabilistic Reasoning in Large I Language Models. Prior research has developed
benchmarks assess LLMs on mathematical and logical reasoning [53
49 30 36), causal reasoning
[34|86], probabilistic reasoning [58] 66) 62). and Bayesian reasoning for general scenarios [21] and
human preferences [29]. LLMs have been utilized construct Bayesian Networks K and medical domain Causal Graphs 82 Statistical inference with Large Language Models has also been applied to decision- in various domains [33 21 46, 44 and constructing in-disributtion synthetic samples [II] Nevertheless, probabilistic reasoning with LLMs remains understudied, particularly for real-world tasks such as modeling privacy risks through Bayesian network elicitation and L..d-based statistical inference. A key component of probabilistic reasoning is uncertainty quantification, which has traditionally been explored in Bayesian neural networks [8] and in deep learning models with dropout (24). More recently, research has estimated uncertainty in autoregressive models and large

---

Self- Dis-Dcsccooure identification
2
Bayeslan Network Structure Eliiitation
Chatbot Message Roddtt Post Other Docsmtents
Possble Netwerk
Woek Toch
Prodicted Network
P(A, B, C, D,
Towenbbeidge
Possible Network
Towassbridge
Kanmir
Towreberdgy
Work Tecch
Sax
Nanemny
landlondss
Werk to Tech
Maternlt
Personal Disclosure
landorss
Son
F
No
Detoction Model
Large Lenguage
Have soa
Model
Query G Geeratton and Subquery Breakdown
A Answer Estimation and Verification
Pepulation P(A)
Query I Anssee 100,000
pepuiatio
Tecatter
Towashosige
Query 2 Annwen
User Document: Does Lownsbridge have highest inflation in the actire country?
PGB
Query 3.1 Answen 40%
Query jescesiaage
Query
google Townsbridge Tech
Query 3.2 Aaswen 25%
Denepation
people lownsinddee
Simpliffed Query
Been here years. but $10 for eggs is dddculous! Houdag
in Tech,
Trcch hove landionds
peroentage people Scape with pamnts
Subeguery 3: penserlage Lowrasbesage Tech that own proparty
don't have draa with landlords and Incmosinn rent, thamkfill..
Query Answer.
AAdittnn [+] Suhquery 3.2 geroentaga of
Fands
Quiry 4 Aniowm 20%
My son's daycame also incressed their
people in Lowzsboiige in
A) Query jercenioge of people lownebridge Tech that am women
sala. 1 only have 4 months
that five with porents
Query 5 Ansswr
maternity leave, 50 I' looking for affordable child care options in the area.
PCE (D,A)
Query 3: porcentage of women in Towashridge
Query Estimation and Sell Verifioiin Midde
that have perizom soas
S Probability Recombination
Pytonn Interpreter
Equation Genoration
Based the individual estimated arewers
Givan Bayesion model, the privary risk
Model
|= 100,000 (40%- 00%) 10%
k k- B (C.1 C.2)
Figure 2: Illustration of the BRANCH framework for estimating the privacy risk k of a document, representing the number of people worldwide who share the personal attributes in the text. LLMs output a single Bayesian Network from the space of possible models for the joint distribution.
language model output with the negative log likelihood of generated sequences [ In this work, we use the consistency of multiple LLM generations S as an estimate of uncertainty.
There have been substantial approaches improve LLM capabilities in mathematical and logical
reasoning through prompting intermediate reasoning steps [87] and program code [15] 25 LLM rea-
soning has been augmented with state evaluators to formulate information as trees 90 or potentially optimizable graphs 5 94
Other work has integrated logical constraints
[12] code execution [3
and step-by-step supervision
47
during finetuning to enhance general LLM reasoning abilities.
Privacy Protection and Security. Canonically, k-anonymity has been defined as property ensuring that each record in a dataset cannot be distinguished from at least 1-11 other records based on quasi-identifiers like age and gender [80 71]. This framework is typically applied by dataset owners who anonymize their data to achieve the desired privacy level. Subsequent enhancements like l-diversity [51]. t-closeness [43), and fi- ff-likeness [13] have addressed limitations of k-anonymity to help dataset owners understand and address privacy risks, where complete data visibility enables complex statistical analyses. We flip this problem framing of privacy on its head, focusing not on the dataset owner but the data contributor, such as an individual Reddit user who is sharing a post online.
Prior work has utilized LLMs to assess the privacy risks of anonymized database records by attempting re-identify the original record and measuring privacy success rate [56 [57] and semantic distance to Quantifying privacy risk for users has been explored before via social media privacy meters 45). but all of these approaches require full access the dataset of a given platform and do not assess privacy in content disclosures in posts but static profile attributes. Empirical user studies in privacy protection have found that LLMs are useful tools for supporting user privacy [40 19 [79]. To that end, previous research has used LLMs to identify personal disclosures in user- generated content 2
the original record 89
|7 and rewording the text to protect privacy [9] However, our approach is novel in that we aim to provide an interpretable quantification of disclosure-base privacy risk for individual data contributors at the point of content sharing.
39
3 Privacy Risk Estimation with Bayesian Network Reasoning
To create contributor-centric privacy risk estimates of user content, we adapt k-anonymity - rather than its more complex derivatives like I-diversity (51), I-closeness [43], and B-likeness [3 because it provides an intuitive and interpretable value ("There are 1,000 other women in Tech in
3

---

Townsbridge") that can be meaningfully estimated without access to database records of possible Internet users. To that end, we estimate k relative to an unknown dataset of all real people who might share the characteristics disclosed in document with BRANCH: Bayesian Network Reasoning for k-ANonymity using Conditional Hierarchies (illustrated in Figure 2. BRANCH is a method that uses LLMs- -vi prompting or finetuning- to reason probabilistically by constructing a Bayesian Network on a document, which is used to calculate privacy risk with population statistics of personal attributes.
Problem Setup. Given a document U, we identify the set of disclosures X mentioned in the document and estimate the number of k people that share the same characteristics from the context C' = (U, X). Disclosure categories are labeled automatically by model or manually by humans (more details in I following the schema introduced in prior work [19]. We model the disclosures in X as random variables {XI..... ] } and formalize the task to estimate the joint probability p = P(X X,) and the population n, where the privacy risk estimate is the expected number of people k np fitting the criteria. In Figure 2 n is the global population while is the joint probability of person living in Townsbridge, being a mother, owning house, and working in Tech.
Bayesian Network Structure Elicitation. Directly estimating the joint probability can be challenging when there are several interconnected personal attributes in a document. To make the privacy risk estimate more tractable and interpretable, BRANCH leverages large language models to implicitly represent the joint probability as Bayesian networks. Each disclosure modeled as a conditional probability P(X, [XX]f-XXreeei }) in the Bayesian network, where LLMs determine parent attributes by first selecting an ordering of the disclosures as random variables. While any arbitrary permutation of the disclosures can potentially model the joint probability, BRANCH instructs L.LMs to select an ideal ordering based on causality and statistical availability to feasibly estimate k. For instance, P(woman work in Tech) is easier to estimate than P in Tech woman) as gender breakdowns for occupations are available whereas occupation statistics by gender are uncommon.
After determining the variable order, BRANCII determines the conditional dependencies of each disclo- sure. While fully connected Bayesian network successfully reconstructs the joint distribution via the chain rule of probability, some disclosures in the set (Xparent) } can be independent to simplify model estimation. For instance, Figure [] "woman" and "no landlords" are independent when conditioned on "Townsbridge" and "work in Tech" because the gender distribution of an occupation is not impacted by housing status, resulting in the probability Pimaternity leave work in Tech, Townsbridge).
Query Generation and Subquery Decomposition. LLMs convert the conditional probabilities into textual queries. An example query for P(maternity leave work in Tech, Townsbridge) is the "percentage of Townsbridge residents in Tech THAT are mothers", where the dependent clause estimates the interest group of "mothers". To lower the variance of k. models combine the total population with the first disclosure to generate specific population query (e.g.. n- P(Townsbridge) = n' > "population of Townsbridge") "). Some probability terms are also decomposed into subqueries for better estimation. For instance, P(no landlords Townsbridge) contains multiple discrete cases: "percentage of Townsbridge residents that own property" " plus the "percentage of Townsbridge residents that live with parents". BRAN BRANCH determines if queries should be decomposed, generates the relevant subqueries, and provides the arithmetic operation in parenthesis to reconstruct the original query.
Query Estimation and Probability Recombination. BRANCH instructs LLMS to apply their inherent historical knowledge of demographic statisties estimate the search queries. Additionally, Retrieval-Augmennte Generation can also be used for query estimation (see Appendix MM. Some queries may be highly specific due the conditional dependencies of disclosure, leading to incorrect probability estimates. To improve the accuracy of the k-privacy prediction, LLMs report their confidence of query answers, simplify low-confidence queries (c.g., removing a dependency). and estimate the new queries. Finally, BRANCH uses the constructed Bayesian Network for inference by recomposing the query answers and subquery arithmetic operations to obtain a mathematical equation. A Python interpreter evaluates the final equation to obtain the model privacy estimate k.
4
Experimental Details
4.1 Privacy Risk Dataset
User-LLM conversations and Reddit posts. We construct a privacy risk estimation dataset of 220 documents: 180 user posts (130 real and 50 synthetic ones) from the pseudonymous online forum

---

Reddit and 40 real user-LLM conversations from ShareGPT Isharegpt... com), annotated with gold labels. This research has been approved by the Institutional Review Board (IRB) with additional measures to safeguard the data, such as replacing all personally identifiable information (PII) with fake identifiers, which are often included in user-I.I.M conversations. Two in-house annotators first identify and categorize all disclosures (e.g., occupation) in the text based on the schema from prior work [1]. We introduce five new categories, namely emotions, events, crime, possessions, reproductive health, and information about other people, based on insights from a recent user study [0]. The annotators manually construct a Bayesian network from the disclosures, find ground-truth answers to the conditional probabilities, construet a math equation combining the answers, and calculate the final k-value for a post. To capture variations in probability orderings and variable dependencies, each post is annotated at least twice, ensuring multiple reasonable interpretations are considered. In total, we annotated 1929 total disclosures for 481 orderings, averaging 139 words and 4.01 disclosures per post, which matches the typical number of natural distribution of personal information across various subreddits [19]. We create a train/test/validation split of 100/66/14 14 Reddit posts, using all 50 synthetic documents in the train set and evaluating only human written posts, including all 40 user-LLM conversations as test documents to evaluate cross-domain generalization.
Synthetic Data Generation. To share data publicly and securely, we synthetically generate 50 post that mimic Reddit by prompting LLaMA-3. 1.--A--.-Innstuct----. with few-shot examples, incorporating synthetic PII and severe disclosures related to depression, suicide ideation, mental health, domestic violence, and abortion. To create synthetic data in-distribution with real data, we prompt LLaMA-3, Instruct to rephrase user posts and train T5-large model [69] for style transfer from LLM-generated text back to the style of Reddit. I.LaMA-generated data may carry distributional biases, so these posts are also manually inspected to ensure that the synthetic PII is not linked to any real person online and further refined to increase the fidelity of the synthetic data in content and tone using real human reference posts of similar topics. More details are located in Appendix D
Threat Model. Based on the categories of disclosures in our dataset and the pseudonymous nature of Reddit and ShareGPT, the key threat model (76] we consider is adversary who may have knowledge of all post-specific disclosure contexts. For instance, a prediction of = for a post with disclosures about location. age, gender, and event-specific circumstances indicates that an adversary with knowledge of that event and who knows a person who matches that location, age, and gender would be able uniquely identify the poster. This could be someone in their inner circle of friends, for example. It does not mean that anyone on the Internet would be able to uniquely identify that poster, as a user's personal attributes are unlikely to be recorded in publicly available databases.
Ground Truth Validation. We assess the quality of our annotation method and dataset by collecting 100 interest groups, averaging 4.32 disclosures per group, from census and public university records that have ground truth populations (e.g., the number of government workers in a city). In-house annotators our annotation method by utilizing online sources, excluding the aforementioned ground-trute databases, to estimate the probability of each disclosure individually and recombine answers to get the population. We report average percentage error of 22.24% when comparing the reconstructed estimate with the known ground truth population. Notably, the ground truths of the 100 census interest groups exhibit average error of 24.79% based the error margins reported in the official census, demonstrating that our dataset consists of high-quality k-values comparable to official sources. We also verify annotation quality by doubly annotating 10% of posts. Despite the variability in the Bayesian networks, we find that multiple valid network configurations yield similar estimations of the final k-values. Annotators' predicted k-values within the same order of magnitude, yielding high inter-annotator agreement (p= 0.916), indicating that our method is consistent and robust to differences in network construction.
4.2 Evaluation Metries for Privacy Risk Estimation
We calculate the Spearman's 's rank correlation between LLM outputs & and the gold standard A* and measure the magnitude of error of model predictions. While the ground truth census populations in the validation experiment span up to a couple hundred thousand, our human-annotated ke in the dataset ranges between I and 56,240,520. Given this large variance, we design a log error metric:
Log Frror k*) logg (f) loog
'The dataset can be found here
5

---

Privacy is interpreted in log scale as it is analogous to uncertainty in Information Theory [6] For each user post, there is an equivalence class of k individuals who share the same set of attributes in the text, making any one person's 's identity indistinguishable from the other k - 1. For random variable Z representing a user's identity, the probability of selecting any single individual P = (k)), The information content of specific outcome is T(Z 2) = log, ( (P 2 2))
which measures the uncertainty level for identifying a specific individual in the equivalence class, corresponding to the user's s quantified privacy. This log error reflects human assessments of risk, as changes in at low values result in much larger changes in privacy risk and larger log errors, and variance in model predictions high v-values is not as significant, leading to lower log errors.
We also measure the percentage of model predictions that are outliers compared to the gold standard. We construct intervals to determine if the model output is within an order of magnitude of the gold standard. For n posts, S i S n, and hyperparameter a, the range metric is defined as:
RANGE(()), (12)) = 1 Σ 1
S
k Sa.ki
This is a general accuracy metric inspired by Brier score 9 that evaluates the percentage of times model correctly estimates k* if the predicted i falls in defined range centered around the gold standard. We report in Table I and a = {2, 10) in Table in Appendix
4.3 Baselines and Model Details
For privacy risk estimation, models are instructed to determine & by evaluating the context C' and select a subset X' X of personal attributes that can be feasibly estimated. We consider the following baseline approaches:
Few-Shot Prompting: Models are prompted with demonstrations and asked estimate the k-value of given post.
Chain-of-Thought Prompting (CoT)): Models are provided with j demonstrations of stepwise reasoning (87] of selecting disclosures, estimating a percentage of people that share each disclosure, and adding this percentage to the final equation to obtain the k-value. Program-of-Thouglee Prompting (PoT): Models are provided with j demonstrations of Python functions [15] (25 that create disclosures as variables with estimated percentages and math equations to solve for the k-value of a post.
We use GPT-4o [61 and LLaMA-3.1 L Instructt 70B for all prompting methods. We run LLaMA- 3.1-Instruct-BB model for few-shot prompting and finetuning with LoRA (31), and we use CoT prompting for GPT-4o-mini, -preevee (59), 03-mini [60], and DeepSeek-rl 17 For BRANCII, we prompt LLaMA-3. --nnsrucc-t GPT-4o, and 03-mini with few-shot demonstrations and finetune train set for 5 components: selecting disclosures,
LLaMA-3.1 1-nt---ctttsttttt--. on the ordering variables, determining conditional dependencies, generating queries and subqueries, and estimating statistics. All models given j 3 demonstrations, and use sample decoding with a temperature of 0.7 during inference. To prevent the variability of disclosure detection models from impacting our evaluation, models are provided user post and the gold standard disclosure and category labels (See Appendix ( for details about hyperparameter selections and for all the prompts used in BRANCH and the baseline models).
5 Experimental Results
5.1 Main Results
Table I presents the main results and shows that the evaluation metrics correlate strongly with each other. We display the percentage difference in performance on each metric with respect to GPT-4o with chain-of -fthuught prompting. Our main findings are summarized as follow:
BRANCH-o3-mini and BRANCH-GPT-40 outperforms all Shain-of-Thoughe baselines. On average, these models achieve a Spearman's p of 0.856, a log error of 2.08, and within an order of magnitude of the gold standard 69.79 69.79% of the time. Comparatively, the best-performing baseline model 03-mini CoT, which has 0.1 lower Spearman's correlation, 0.73 higher log error, and 10.66% lower range metric. We use the paired bootstrap test [4] for 10t iterations to compare

---

Documents
Reddit Ducuments
ShareGPT Documents
Method
Model
P
Log Rrr. Range
et
Log Frr.) Rangelir
pf
Log. Rm.. Rangelii
LLaMA InstA---ttttat-33...
[0.15]
7.14
31.74%
0.183
6.66
19.87%
0.146
8.79
15.32%
FEW-SHOT
LLaMMA-nstrages Jin
LLaMA-Instract- las (FT)
0.435
4.86
4,46
43.04%
30.00%
0.635
0:255
4.35
45.70%
30,46%
0.590
4.89
29.11%
37.97%
GPT-4o (022--0--66
0.565
3.94
42%17%
00.519
3.55
43.05%
0.008
4.69
40.51%
PROGRAM-OF- THOUGHT
LLaMA-Instractt3. Iua
4.03
37.83%
0.519
3.82
37.09%
39.24%
GPT-4o (2024-08-06)
0.673
3.33
54.78%
00.624
2.99
56.29%
0.678
3.97
51.90%
LLaMA-Instract--
0.589
5.88
46.09%
[0:429
3.90
43,71%
0,708
3.84
50.63%
GPT-4o-mini (2024-07-18)
37.83%
43.71%
002666
26.585
CHAUG-OF-TROUGHT
ol-preview (2024-19-12) 12)
CPT-4o (2024-08-06)
0.747
0.761
3.09
3,06
55 06%
55.22%
0.7727
0.730
2.82
2.72
56.95%
56.95%
0.750
0.779
3.79
3.52
53.16%
51.90%
DecSSckk (2025-01-20)
2.98
0.685
03-mini (2025-01-31)
0.744
2.81
59.13%
0.730
2.39
64.90%
0.722
3.60
48.10%
LLoMM--Inttrctt-.11a
0.699
3,47
51.74%
0.6466
3.44
48.34%
0.748
3.51
58, 23.23%
BRANCH (this work)
I.MMA-Instrass- sa (FT)
0.712
56.52%
0.670
3.03
0.346
49%
GPT-4o (2024-08-06 08-006
0.839
2.161
66.969
0.797
2.16
66.89%
0.871
2.181
67 09.0999
a3-mini (2025-01-31)
1.99
72.61
0.817
2.04
72.19%
73.42%
Table I: Performance of different methods on the privacy estimation task. t. 1. and represent statistical significance with p value of 0.05, 0.01, and 0.001, respectively, comparing the BRANCH model to their respective CoT baseline. Cells are colored by percentage A with respect to GPT-4o CoT performance:
15%
100%
55%
0%
+3%
+8%
+10%
o3-mini Chain -hhiu-t
03-mini BRANCH (This Work)
10a
10a
10:
10
10 Ros 20r
105
Ground Truth
Ground Truth
10:
10
20
10
10
10 Model Prediction
18
Model Prediction
Figure 3: Plots of the model prediction i compared to the ground truth k- in log scale. The dashed lines indicate the acceptable half-magnitude range surrounding the gold standard values. Incorrect predictions are shaded to indicate the level of magnitude of the residual errors.
BRANCH to GPT-4o, and LLaMA-3.1 1--B finetuned exhibit a statistically significant improvement in privacy risk estimation when using BRANCH over Chain-of-Thouu with Sfain-of-Thhoueee
baselines on the same model architecture. Table that o3-mini,
prompting. In comparison, when prompted the mathematical reasoning models R1, ol op-preveew, and 03-mini are notably worse than BRANCH at reasoning probabilistically over real-world scenarios in this task. We perform an in-depth analysis comparing error modes of Chain-of Chain-of-Thought and BRANCH in Section
Notably for BRANCH, finetuning L.LMA-3.. 3.1 Instrucct-8 for each individual module yields 4.78% higher range accuracy than the few-shot prompted baseline of BRANCH LLaMA-3. -Instrucct-70B. GPT-4o with reasoning prompts, R1, and o1 ol-preveew competitive with the finetuned BRANCH model and outperform BRANCH LLaMA-3.1 1.-3..IInsttee Instrrcct-708 by 5% in range accuracy, on average. These models outperform LLaMA-3.1 1.I.s-utttttttttt due their better cotruction-following capabilities, which is vital for our privacy risk estimation task. LLaMA-3.. IInstruct often ignores instructions, such providing percentage answer for a population query.
BRANCH safer than Chain-of-Thought prompting. BRANCH 03-mini and GPT-4o produce lower log errors than their Chain-of- Thouugh counterparts for 61.74% and 59.57% of posts, respectively. In Figure we plot the model predicted & with the gold standard k* values in log scale for o3- mini with BRANCH and Chain-of-
prompting. BRANCH 03-mini predictions are centered along the main line h = k", whereas 03-mini Chain-of- Chain-of-Thought contains more outliers, including conservative predictions that provide little utility to users as privacy estimation tools. o3-mini and GPT-4o baselines classify 28. 48% of k- v-values as high-estimate outliers- serious errors where the models significantly underestimate the privacy risk of a document. These mispredictions can mislead users about their online safety by portraying high-risk situations as low-risk. For example, in Figure
3 03-mini Chain-of -houugh significantly underestimates privacy risk to be f 2,500,000 despite
7

---

the true value being '- = 1. In comparison, BRANCH GPT-4o and o3-mini only exhibit 12.17% of cases where they underestimate privacy risk, with those errors being much smaller in magnitude.
BRANCH is generalizable to different domains and can account for highly sensitive personal information. Table [ presents the separate results of the Chain-of-Thouglh baselines and the BRANCH models on each domain, Reddit and ShareGPT. Notably, Stain-of-Thoughe prompting baselines struggle significantly more on User-LLM conversations than on Reddit posts, with o3-mini CoT producing 16.80% lower range accuracy. This is especially concerning given that User-LLM conversations frequently contain personally identifiable information (PII), making robust and accurate modeling crucial to capture serious privacy risks. In contrast, BRANCH not only outperforms the chain-of of-thought baselines in both domains, our method improves when evaluating posts with severe privacy risks, yielding better results on ULM conversations compared to user-written posts with 3.96% higher range accuracy and 0.085 higher Spearman's 's correlation, on average.
5.2 BRANCH Component Analysis
LLM predictions of demographic statisties. We investigate the strength of BRANCH for privacy risk estimation by individually evaluating the capabilities of LLMs for estimating individual statisties and modeling Bayesian networks in Figure[4 Using -writen single attribute probability queries (e.g.. "percentte of people in Townsbridge that are in Tech") with ground truths, caleulate the average percentage error of model probability estimates to the ground truth. Both o3-mini and GPT-4o demonstrate strong capability in estimating individual attributes, with average percentage errors of 16.60% and 17.45%, respectively-
than the average margin of error found in census data. BRANCH performance is similar across both domains, demonstrating that the models' capability in generating accurate demographic statistics translates to strong performance in privacy risk estimation.
Bayesian structure modeling. Prior work on Bayesian structure modeling uses ccore-based methods 722 to evaluate all polynomial orderings and conditional dependencies. Instead, we evaluate the structure of Bayesian networks in BRANCH with human-annotatee annotated Bayesian networks in Figure 4 and find that BRANCH has an average Kendall's t rank correlation of 0.59 and 0.39 for Reddit and ShareGPT, respectively, indicating moderate agreement. For modeling conditional dependencies, we find that disclosures are, on average, conditionally independent of 42.80% of prior variables in BRANCH and 45.81 % in human-annotated networks, suggesting that BRANCH approximates human reasoning in Bayesian network elicitation for modeling privacy risk.
BRANCH Estimation Component
BRANCH Ordering Component
35
BRANCH Model
708
Percentage Erro (96)
30-
30%
0.6
0.52
0.51
LLaMA 30
G77-40
0.22
03-min
25
24%
0.5
22%
21%
Kendall's
0.42
0.36
20-
37%
3656
17%
5.%%
18% 17%
0.3
0.31
25
10
0.1
AlI
Reddit
ShareGPT
All
Reddit
ShareGPT
Figure 4: Analysis of the individual components in BRANCH for query estimation and Bayesian model ordering. Query estimation is evaluated with percentage error against ground truth queries and answers, and model ordering is evaluated with Kendall's т against human-made Bayesian networks.
Comparing Chain-of-Thoughe and BRANCH
To better understand how BRANCH and Slain-of-Though prompting scale with input complexity, we compare GPT-4o and o3-mini performance across posts with varying numbers of personal disclosures. For posts with fewer than 4 disclosures, the range accuracy of both methods roughly similar, with BRANCH achieving 67.16% and Chain-of- Thouhhhtttt achieving 61.28% on average for both models. For posts with 4 or more disclosures, the performance gap between BRANCH and Chain-of-Thougeee prompting widens substantially. In BRANCH, 03-mini and GPT-4o increase their range accuracies to 74.22% and 69.5 69.53% respectively, while Chain-of-Thought prompting decreases 03-mini and GPT-40 range accuracy to 55.47% and 52.34%, respectively. For both models, BRANCH has higher accuracy
8

---

on complex posts than it does on the entire dataset of user posts, indicating that the Bayesian network modeling step in BRANCH enables models handle more personal attributes in a document.
Chain-of-Thought errors occur due to the lack of probabilistic modeling. Figure | presents example posts where BRANCH correctly and Chain-of-Thouul We analyze GPT-4o and 03-mini CoT errors and find that 25.64% of errors are due to the lack of dependencies modeled between disclosures. For instance, 03-mini uses 10% for the "prevalence of social anxiety", not accounting for being "on the spectrum" which increases social anxiety prevalence to 50%. 20.51% % of CoT errors occur due to the lack of subqueries that precisely estimate probability, and 15.38% of errors occur due to PII. like the unique business name "Townsbridge InfoTech" which results in k = 1. While BRANCH accurately captures this information in the Bayesian network, Chain Thouught underestimates the privacy risk & = 250 by assuming 1 in 10,000 people use this business name. In contrast, BRANCH errors arise from issues also common in Chain-of- Thought prompting, with 38.47% of CoT baseline errors and all BRANCH errors attributable to incorrect probability estimations or attribute selections. Thus, BRANCH outperforms Chain-of- Thought prompting on complex posts specifically due to its probabilistic methodology that can model conditional dependencies and personally identifiable attributes.
incorrectly estimates privacy risk.
Sampling and Uncertainty
In Section 5 we use sampling as the decoding method to generate point estimates for l values. To assess the variability of reasoning processes for the privacy risk estimate, we re-evaluate models times for privacy risk estimation construct mean point estimates. Specifically, with BRANCH and Chain-of- Thoohhh prompting times for privacy risk estimation to obtain mean estimates for all user posts from the test set in Reddit. We also use self-consistency to re- re-sample only the query estimation component in BRANCH and combine the expected answer pi for all queries in a post to construct k. We compare i with the gold standard k- using the evaluation metrics defined in Section 4.2 We report the variability of predictions using the coefficient of variation, calculated as the standard deviation of the k values normalized by their mean. For coffcsnsisten,y, we define the total variance as the sum of the individual variances across all queries within a post.
run GPT-4o
Model Ranges
Coeff Variation Spearman's 's øt Log Err.
Range %1
Rango Var. %6
RangCrighh Var % 1
CoT Re-Evaluation BRANCH Self-Consistency
1.072
0.691
3.17
49.67%
61.63%
33.85%
0.854
0.812
2.08
68. 21%
71 71.28%
63.16 %
BRANCH Re- Evaluation
1.013
0.838
1.91
68.21%
86.08 %
48.61 %
Table 2: GPT-4o Mean Estimate Results for Re-Evaluating the model and using Self-Consisteng
on the query estimation module in BRANCH. The average Coefficient of Variation (CV) is the normalized standard deviation of the predicted & values. Range accuracy is categorized as high or low variance depending on whether the predicted k's CV is above or below the mean.
BRANCH mean estimates are more accurate than individual k estimates. As seen in Table 2 BRANCH GPT-4o performance for the & estimates increases, resulting in 68.21% range accuracy. Re-running the entire BRANCH pipeline yields the best performance, achieving 0.25 lower log error than using one run of BRANCH GPT-4o. However, re-sampling the entire BRANCH model results in significantly higher variabiliit,, with a coefficient of variation of 1.013, compared to 0.854 achieved by applying Consisenstt only to the query estimation component. The low variance of the query estimation module in BRANCH under self-consistency, together with its comparable performance to a single BRANCH run, suggests that the pipeline is robust to fluctuations in I.L.-estimated demographic statistics, as the L.MMs used in BRANCH are internally consistent when estimating these quantities within our dataset domains. In comparison, re-evaluating much less consistent for privacy risk estimation, as the mean & estimate yields 5.55% lower range accuracy and 0.44 higher log error than one run of CoT prompting. highest coefficient of variation of 1.072, indicating that it is not only less reliable for accurate privacy risk estimation but also significantly more inconsistent across runs.
Thhough prompting is
Shain-of-Thougee also has the
The implicitly constructed Bayesian Networks in BRANCH are stable across repeated executions of the pipeline. The overall performance of the BRANCH model remained stable across 5 runs the entire pipeline, suggesting robustness in the Bayesian network elicitation process itself. To further quantify structural consistency, we compute the mean Structural Hamming distance (SHD) between the induced shared subgraphs across different BRANCH runs, which measures the number of edge

---

to transform one Bayesian network to another. The mean SHD across all user posts was 1.29, with a minimum of 0 and maximum of 10. By comparison, the mean maximum possible SHD (i.e., comparing a fully connected subgraph to an entirely disconnected one) 11.36, further affirming that the constructed Bayesian network structures in BRANCH are highly consistent across runs despite potentially noisy or missing priors. These findings collectively suggest that the BRANCH pipeline produces stable and consistent Bayesian network structures, even under variability in LLM outputs.
operations- pnerations-addiiions,, deletions, or
LLMs are well-calibrated for privacy risk estimation. Following insights from 84] we use model variance as a measurement of estimation uncertainty. To evaluate this, we stratify the range accuracy based on whether the coefficient of variation for each post is above or below the mean. We find that predictions of k with high variance are significantly more likely to be incorrect, with re-evaluating the entire BRANCH model resulting in 86.08% accuracy on low variance estimates, compared to 48.6 61 % for high variance privacy estimates. Chain-of-Thougee prompting exhibits similar trend, achieving 27.78% higher accuracy on posts with high certainty. This discrepancy is less pronounced when using self-consistency on the query estimation module, as BRANCH shows comparatively strong performance even on high-variance examples with 63. 16% accuracy. Nevertheless, model uncertainty is a useful indicator for identifying when privacy risk estimations are likely to be inaccurate.
8 Conclusion
We present probabilistic reasoning task for Large Language Models: quantifying the privacy risk of documents containing personal attributes. To make risk assessments interpretable, we develop a value for privacy risk, representing the number of individuals who share the specific combination of attributes mentioned in the text. We develop BRANCII, a new methodology that uses Bayesian Networks to model joint distribution over personal attributes. By decomposing the joint probability into a set of conditional probabilities, BRANCH enables more accurate and tractable estimation of the privacy risk. We construct a dataset annotated with gold standard values and find that BRANCH can accurately estimate the k value 72.61% % of the time. In our analyses, we demonstrate that BRANCH outperforms baseline models on complex inputs with several interconnected attributes due to its probabilistic decomposition and ability to accurately estimate statistics of individual attributes. Finally, we show that the variance in predicted k-values serves as a proxy for model uncertainty-wiiin where higher variance reliably signals less accurate i estimates. Our results show that BRANCH can be a practical tool to inform users about potential privacy risks when sharing information online.
Acknowledgements
We would like to thank Joseph Thomas for his help in constructing the k-anonymity dataset, refining synthetically generated posts, and annotating for the privacy risk value. This work is in part supported by the NSF CAREER Awards IIS-2144492 and IIS IIS-2052498. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. We would like to thank Microsoft's Azure Accelerate Foundation Models Research Program for providing computational resources to support this work.
References
[1] Lukas Aichberger, Kajetan Schweighofer, and Sepp Hochreiter. Rethinking uncertainty estima- tion in natural language generation, 2024.
[2] Chandan Akiti, Anna Squicciarini, and Sarah Rajtmajer. A semantics-based approach to disclosure classification in user-generated online content. In Trevor Cohn, Yulan He, and Yang Liu, editors, Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3490-3499, Online, November 2020. Association for Computational Linguistics.
[3] Nikolay Babakov, Ehud Reiter, and Alberto Bugarin. Scalability of bayesian network structure elicitation with large language models: a novel methodology and comparative analysis, 2024.
[4] Taylor Berg- Kang--Kikppttickk David Burkett, and Dan Klein. An empirical investigation of statistical significance in NLP. In ichi Tsujii, James Henderson, and Marius Paşca, editors, Proceed- ings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and
10

---

Computational Natural Language Learning, pages 995-1005, Jeju Island, Korea, July 2012.
Association for Computational Linguistics.
[5] Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models. Proceed- ings of the AAAI Conference on Artificial Intelligence, 38(16): 17682-1772-111 17690, March 2024.
[6] Michele Bezzi. An information theoretic approach for privacy metrics. Transactions on Data Privacy, 3:199-215, 12 2010.
[7] Taylor Blose, Prasanna Umar, Anna Cinzia Squicciarini, and Sarah Michele Rajtmajer. Privacy in crisis: study of self-disclosure during the coronavirus pandemic. CoRR, abs/2004.09717, 2020.
[8] Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Lierni Sestorain Saralegui, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, and Kellie Webster. Attributed question answering: Evaluation and modeling for attributed large language models, 2023.
[9] Glenn W. Brier. Verification of Forecasts Expressed in Terms of Probability. Monthly Weather Review, 78(1):1, January 1950.
[10] Matthew K. Buttice and Benjamin Highton. How does multilevel regression and poststratifica- tion perform with conventional national surveys? Political Analysis, 21(4):449 467, 2013.
[11] Yewon Byun, Shantanu Gupta, Zachary C. Lipton, Rachel Leah Childers, and Bryan Wilder. Using imperfect synthetic data in downstream inference tasks, 2025.
[12] Diego Calanzone, Stefano Teso, and Antonio Vergari. Towards logically consistent language models via probabilistic reasoning, 2024.
[13] Jianneng Cao and Panagiotis Karras. Publishing microdata with a robust privacy guarantee, 2012.
[14] Nicholas Carlini, Daphne Ippolito, Matthew Jagielski. Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models, 2023.
[15] Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks, 2023.
[16] Daniel Courgeau. From the Macro-Micro Opposition Multilevel Analysis in Demography, pages 43-91. Springer Netherlands, Dordrecht, 2003.
[17] AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning, 2025.
[18] Stefan Depeweg, José Miguel Heráández-Lobate Finale Doshi-Velez, and Steffen Udluft. Decomposition of uncertainty in bayesian deep learning for efficient and risk-sensitive learning, 2018.
[19] Yao Dou, Isadora Krsek, Tarek Naous, Anubha Kabra, Sauvik Das, Alan Ritter, and Wei Xu. Reducing privacy risks in online self-disclosures with language models. Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), pages 13732-13754 Bangkok, Thailand, August 2024. Association for Computational Linguistics.
[20] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, et al. The laaa 3 herd of models, 2024.
[21] Yu Feng, Ben Zhou, Weidong Lin, and Dan Roth. Bird: trustworthy bayesian inference framework for large language models, 2024.

---

[22] J.L. Fleiss. Statistical methods for rates and proportions Rates and proportions. Wiley, 1973.
[23] Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang, Zhu, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, and Saleema Amershi. Magentie-one: generalist multi-agent system for solving complex tasks, 2024.
[24] Yarin Gal and Zoubin Ghahramani. Dropout bayesian approximation: Representing model uncertainty in deep learning, 2016.
[25] Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models, 2023.
[26] James Gardner and Li Xiong. An integrated framework for de-identifying unstructured medical data. Data & Knowledge Engineering, 68( 08(12):1441-1451 2009. Including Special Section: 21st IEEE International Symposium on Computer-Ban -asee Medical Systems (IEEE CBMS 2008)
-Seven selected and extended papers on Biomedical Data Mining.
[27] Arijit Ghosh Chowdhury, Ramit Sawhney, Puneet Mathur, Debanjan Mahata, and Rajiv Ratn Shah. Speak up, fight back! detection of social media disclosures of sexual harass- ment. In Sudipta Kar, Farah Nadeem, Laura Burdick, Greg Durrett, and Na-Rae Han, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Student Research Workshop, pages 136-146, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
[28] Olga Gkountouna, Katerina Doka, Mingqiang Xue, Jianneng Cao, and Panagiotis Karras. One- off disclosure control by heterogeneous generalization. In 31 USENIX Security Symposium (USENIX Security 22), pages -3777, Boston, MA, August 2022. USENIX Association.
[29] Kunal Handa, Yarin Gal, Ellie Pavlick, Noah Goodman, Jacob Andreas, Alex Tamkin, and Belinda Z. Li. Bayesian preference elicitation with language models, 2024.
[30] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021.
[31] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021.
[32] Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. Are large pre-trained language models leaking your personal information?, 2022.
[33] Xun Jiang, Feng Li, Han Zhao, Jiaying Wang, Jun Shao, Shihao Xu, Shu Zhang, Weiling Chen, Xavier Tang, Yize Chen, Mengyue Wu, Weizhi Ma, Mengdi Wang, and Tianqiao Chen. Long term memory: The foundation of ai self-evolution, 2024.
[34] Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng Lyu, Kevin Blin, Fernando Gonzalez Adauto, Max Weiner, Mrinmaya Sachan, and Bernhard Schölkopf. Cladder: Assessing causal reasoning in language models, 2024.
[35] Curt Tigges Joseph Bloom and David Chanin. Saelens. https:://giihbb..cmm/bbbbeeeee col.oooo/s//mm SAELens 2024.
[36] Mintong Kang and Bo Li. rr-guard: Robust reasoning enabled 11m guardrail via knowledge- enhanced logical reasoning, 2024.
[37] Rajeeva L. Karandikar. Opinion polls and statistics. Calcutta Statistical Association Bulletin, 1231-2) 1999.
[38] Patrick Gage Kelley, Saranga Komanduri, Michelle L. Mazurek, Richard Shay, Timothy Vidas, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, and Julio Lopez. Guess again (and again and again): Measuring password strength by simulating password-cracking algorithms. In 2012 IEEE Symposium on Security and Privacy, pages 523-537, 2012.
12

---

[39] Ari Klein, Abeed Sarker, Masoud Rouhizadeh, Karen O' O'Connor, and Graciela Gonzalez. Detecting personal medication intake in Twitter: An annotated corpus and baseline classification system. In Kevin Bretonnel Cohen, Dina Demner-Fushman Sophia Ananiadou, and Junichi Tsujii, editors, BioNLP 2017, pages 136-142, Vancouver, Canada,, August 2017. Association for Computational Linguistics.
[40] Isadora Krsek, Anubha Kabra, Yao Dou, Tarek Naous, Laura A. Dabbish, Alan Ritter, Wei Xu, and Sauvik Das. Measuring. modeling, and helping people account for privacy risks in online self- ceff-disclosures with ai, 2024.
[41] Benjamin E. Lauderdale, Delia Bailey, Jack Blumenau, and Douglas Rivers. Model-based pre-election polling for national and sub-national outcomes in the us and uk. International Journal of Forecasting, 36(2):399-413, 2020.
[42] Jooyoung Lee, Sarah Rajtmajer, Eesha Srivatsavaya, and Shomir Wilson, Online self-disclosure. social support, and user engagement during the covid-19 pandemic. ACM Trans. Soc. Comput., 6(3-4):1-3) 2023.
[43] Ninghui Li, Tiancheng Li, and Suresh Venkatasubramannian t-closeness: Privacy beyond anonymity and I-diversity. In 2007 IEEE 23rd International Conference Data Engineering, pages 106-115, 2007.
[44] Yanjun Li, Qi Huang, Jin Jiang, Xusheng Du, Wenxin Xiang, Shiqi Zhang, Zean Pan, Liyuan Zhao, Yuyan Cui, Limei Ke, Bo Yin, Linfeng Liu, Guoqing Feng. Shouyi Yan, Liangeai Gao, Yang Liu, Yujuan Yuan, Yanying Guo, Yuqing Yang, and Qian Di. Large language model- based biological age prediction in large-scale populations. Nature Medicine, 31 29:7997-2 07 2025.
[45] Kun Liu and Evimaria Terzi A framework for computing the privacy scores of users in online social networks. In 2009 Ninth IEEE International Conference on Data Mining, pages 288-297, 2009.
[46] Xiao Liu, Zirui Wu, Xueqing Wu, Pan Lu, Kai-Wei Chang, and Yansong Feng. Are Ilms capable of data-based statistical and causal reasoning? benchmarking advanced quantitative reasoning with data, 2024.
[47] Yixin Liu, Avi Singh, C. Daniel Freeman, John D. Co-Reyes, and Peter J. Liu. Improving large language model fine-tuning for solving math problems, 2023.
[48] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019.
[49] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In The Twelfth International Conference on Learning Representatiinss 2024.
[50] Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella- Béguelin. Analyzing leakage of personally identifiable information in language models, 2023.
[51] Ashwin Machanavajjhala, Daniel Kifer, Johannes Gehrke, and Muthuramakrishnan Venkita- subramaniam. L-diversity: Privacy beyond k-anonymity. ACM Trans. Knowl. Discov. Data, 1(1):3ess, March 2007.
[52] Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction, 2021.
[53] Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. GAIA: benchmark for general ai assistants, 2023.
[54] Niloofar Mireshghallah, Maria Antoniak, Yash More, Yejin Choi, and Golnoosh Farnadi. Trust no bot: Discovering personal disclosures in human-LLM conversations in the wild. In First Conference Language Modeling, 2024.
[55] Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. Can lims keep a secret? testing privacy implications of language models via contextual integrity theory, 2024.
13

---

[56] John Morris, Justin Chiu, Ramin Zabih, and Alexander Rush. Unsupervised text deidentification. In Yoav Goldberg, Zoriitsa Kozareva, and Yue Zhang, editors, Findings of the Association for Computational Linguistics: EMNLP 2022, December 2022.
[57] John X. Morris, Thomas R. Campion, Sri Laasya Nutheti, Yifan Peng. Akhil Raj. Ramin Zabih, and Curtis L. Cole. Diri: Adversarial patient reidentification with large language models for evaluating clinical text anonymization, 2024.
[58] Aliakbar Nafar, Kristen Brent Venable, and Parisa Kordjamshidi. Probabilistic reasoning in generative large language models, 2024.
[59] OpenAl, ;, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, et al. Openai ol system card, 2024.
[60] OpenAl. 03-mini model, 2024. https://openal: com
[61] OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, al. Gpt-4 technical report, 2024.
[62] Batu Ozturkler, Nikolay Malkin, Zhen Wang, and Nebojsa Jojie. Thinksum: Probabilistic reasoning over sets using large language models, 2023.
[63] Leysia Palen and Paul Dourish. Unpacking" privacy" for a networked world. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 129-136, 2003.
[64] David K. Park, Andrew Gelman, and Joseph Bafumi. 11. State-Level Opinions from National Surveys: Poststratification Using Multilevel Logistic Regression, pages 209-228. Stanford University Press, Redwood City, 2006.
[65] Akshay Paruchuri, Jake Garrison, Shun Liao, John Hernandez, Jacob Sunshine, Tim Althoff, Xin Liu, and Daniel McDuff. What the odds? language models capable of probabilistic reasoning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11712-11733, 2024.
[66] Akshay Paruchuri, Jake Garrison, Shun Liao, John Hernandez, Jacob Sunshine, Tim Althoff, Xin Liu, and Daniel McDuff. What are the odds? language models are capable of probabilistic reasoning, 2024.
[67] John L. Pollock. Probability and Proportions, pages 29-66. Springer Netherlands, Dordrecht, 1988.
[68] Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin, Zheng Xu, Carson Denison, H. Brendan McMahan, Sergei Vassilvitskii, Steve Chien, and Abhradeep Guha Thakurta. How to dp-fy mi: practical guide to machine learning with differential privacy. Journal of Artificial Intelligence Research, 77:1113-1201, July 2023.
[69] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023.
[70] Krithika Ramesh, Nupoor Gandhi, Pulkit Madaan, Lisa Bauer, Charith Peris, and Anjalie Field. Evaluating differentially private synthetic data generation in high-stakes domains, 2024.
[71] Pierangela Samarati and Latanya Sweeney. Protecting privacy when disclosing information: k-anonymity and its enforcement through generalization and suppression. 1998.
[72] Mauro Scanagatta, Antonio Salmerón, and Fabio Stella. A survey on bayesian network structure learning from data. Prog. in Artif. Intell., 8(4):425 439, December 2019.
[73] George AF Seber. Statistical models for proportions and probabilities. Springer, 2013.
[74] Yijia Shao, Tianshi Li, Weiyan Shi, Yanchen I Liu, and Diyi Yang. Privacylens: Evaluating privacy norm awareness of language models in action, 2025.
14

---

[75] Weiyan Shi, Aiqi Cui, Evan Li, Ruoxi Jia, and Zhou Yu. Selective differential privacy for language modeling. 2022.
[76] Adam Shostack. Threat modeling: Designing for security. John Wiley & Sons, 2014.
[77] Robin Staab, Mark Vero, Mislav Balunovi',, c, and Martin T. Vechev. Beyond memorization: Violating privacy via inference with large language models. ArXiv, abs/2310.072992 2023.
[78] Albert Yu Sun. Eliott Zemour, Arushi Saxena, Udith Vaidyanathan, Eric Lin, Christian Lau, and Vaikkunth Mugunthan. Does fine-tuning gpt-3 with the openai api leak personally-ddeniffbble information?, 2024.
[79] Bolun Sun, Yifan Zhou, and Haiyun Jiang. Empowering users in digital privacy management through interactive Ilm-based agents, 2024.
[80] Latanya Sweeney. k-anonymity: a model for protecting privacy. Int. J. Uncertain. Fuzziness Knowl--Based Syst. 10(5):557-570, October 2002.
[81] Manuel Tomneau, Dhaval Adjodah, Joao Palotti, Nir Grinberg, and Samuel Fraiberger. Multilin- gual detection of personal employment status on Twitter. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Pupers), pages 6564 6587, Dublin, Ireland, May 2022. Association for Computational Linguistics.
[82] Aniket Vashishtha, Abbavaram Gowtham Reddy, Abhinav Kumar, Saketh Bachu, Vineeth N Balasubramanian, and Amit Sharma. Causal order: The key to leveraging imperfect experts in causal inference, 2025.
[83] Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning. arXiv preprint 1rXiv:2310.03731, 2023.
[84] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Seff-consistency improves chain of thought reasoning in language models, 2023.
[85] Yulong Wang, Tianhao Shen, Lifeng Liu, and Jian Xie. Sibyl: Simple yet effective agent framework for complex real-world reasoning, 2024.
[86] Zeyu Wang. CausalBench: A comprehensive benchmark for evaluating causal reasoning capabilities of large language models. In Kam-Fai Wong, Min Zhang, Ruifeng Xu, Jing Li, Zhongyu Wei, Lin Gui, Bin Liang, and Runcong Zhao, editors, Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10), pages 143-151, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
[87] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[88] Shepard Xia, Brian Lu, and Jason Eisner. Let's think vyr-by-var: Large language models enable ad hoc probabilistic reasoning. arXiv preprint arXiv:2412.02081/ 2024.
[89] Rui Xin, Niloofar Mireshghallah, Shuyue Stella Li, Michael Duan, Hyunwoo Kim, Yejin Choi, Yulia Tsvetkov, Sewoong Oh, and Pang Wei Koh. false sense of privacy: Evaluating textual data sanitization beyond surface-level privacy leakage, 2025.
[90] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models, 2023.
[91] Andrew Yates, Arman Cohan, and Nazli Goharian. Depression and self-harm risk assessment in online forums. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors, Proceedings of the 2017 Conference on Empirical Methods Natural Language Processing, pages 2968-299299 2978. Copenhagen, Denmark, September 2017. Association for Computational Linguistics.
15

---

[92] Zhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, Bingsheng Yao, Sauvik Das, Ada Lerner, Dakuo Wang, and Tianshi Li. "it's a fair game", or is it? examining how users navigate disclosure risks and benefits when using llm-based conversational agents. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI '24, New York, NY, USA, 2024. Association for Computing Machinery.
[93] Yizhang Zhu, Shiyin Du, Boyan Li, Yuyu Luo, and Nan Tang. Are large language models good statisticians?, 2024.
[94] Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. Language agents as optimizable graphs, 2024.
A Limitations
We construct comprehensive dataset capturing various topics and domains user posts with sensitive personal information. Due to the cost of annotating privacy risk in text, we intentionally focused on the natural distribution of atomic user posts on domains rich in personal attributes that map directly to structured and publicly available ground truth sources (e.g.. census data, university records), namely real- world user berr-written Reddit posts and user-LLM conversations. That said, we acknowledge that these domains may not capture the full diversity of real-world scenarios. Future work could look at developing user studies to evaluate BRANCH in more real-world contexts and exploring additional threat models and contexts (c.g.. an individual who also knows additional information not present in the post) to better understand BR BRANCH'S 's robustness acmoss use cases. Cross-domain generalization is also an important direction, including applications to Amazon reviews. job forums, and other online platforms where personal information is frequently shared. These efforts will help investigate how well BRANCH can generalize to diverse, real-world scenarios.
The distribution of disclosures in our dataset is usually between 3-6 per post and rarely extends past 8 disclosures, which closcly matches the natural distribution of personal information in these domains. While we find that BRANCH outperforms CoT-based methods in handling dense attribute interactions as the number of disclosures increases, we have not extensively tested posts with dozens of disclosures. Future work could estimate the privacy risk text that contains more than 10 personal disclosures, such as with the post history a Reddit user.
In our current setup, we are assessing how closely LLM-based methods approximate human experts. Human gold standard labels are validated based on doubly annotating a portion of our dataset and with the validation experiment that shows that for interest groups similar to those found in Reddit and ShareGPT, the human annotation methodology exhibits very low error when compared to the ground truths. However, ac acknowledge that finding a perfect ground truth unattainable for some cases, which an inherent limitation working with real data in this setting. Nevertheless, the authors of this paper manually review the annotation process to ensure the quality of the gold k value.
Our method BRANCH utilizes large language models to estimate privacy risk. This can pose a computational burden, particularly hecause our methodology requires multiple L.M inference steps to construct joint prohability, create queries, and estimate individual answers. The computational burden may also increase when utilizing model uncertainty as an indicator accuracy. reduce this burden. future work could look at:
1. LLM Output Caching:
By caching intermediate outputs, especially for frequently seen or structurally similar inputs, we can significantly reduce the computation time.
2. Lightweight Model Substitution: In many cases, such as simpler posts, Chain-of- Thhoughhtt outputs or lightweight, distilled models could be used to approximate certain stages of the BRANCH pipeline without full LLM calls.
3. Pipeline Optimization:
Batching and combining BRANCH steps reduce the total number of LLM calls required to estimate privacy risk.
In principle, with substantially larger of human-annotated data (albeit requiring significant upfront human annotation effort), it would also be possible to fine-tune an end-to-end model that climinates some intermediate components and significantly reduces inference-time computational cost.
To ground our methodology and establish an upper bound on potential privacy risk, we focus on a strong adversary with access to knowledge of all post-specific disclosure contexts. This setup enables a rigorous evaluation of the maximum possible privacy leakage under realistic but constrained assumptions. Future work could study user-personalized threat modeling, where privacy risk is estimated relative the knowledge scope of specific, personalized adversaries. In these cases, users can specify the types personal information that their threat model has access to (c.g., the knowledge that a coworker has about a user). We also plan on looking at general threat models with differing levels of personal access, such as quantifying privacy risk to stranger who does not have access to personal information outside of publicly available databases.
16

---

The primary purpose of our model is to provide users with supplementary tool to quantify the privacy risk associated with privacy-sensitive documents and online personal disclosures. While the failure cases of the best-perfnnnnng models tend towards overprotection as they estimate low k-values for post, there are some instances where models will misclassify a document to have a lower privacy risk than in reality. Prior user studies have been conducted in LLM usage for privacy protection, but we have not conducted a specific user study to test model for providing quantifiable privacy risk estimates. We plan to deploy user studies in the future to explore the utility of our model, the best ways to present this information users, and how users adjust behavior in response.
While we do not ohserve obvious biases against specific demographic groups in our dataset, we have not studied this extensively, as this requires significandly more data and human annotation resources than what is feasible within the scope and budget of this academic study. For evaluation, we compare the LLM- -LM-generated estimations to human-annotated ground truth in Section 5.2, ensuring that the model evaluation. Future rescarch could conduct comprehensive audit of all possible sources of hias (c.g.. gender, race, geography, education level, age, culture) and investigate whether there could he potential bias from the LLM--interna demographic priors utilized in our method, which could impact its use as tool for certain demographies. This analysis should also determine if failure cases unevenly affect certain groups, especially marginalized vulnerable populations, before these k-estimation models can be deployed to real users.
biases do not affect
B Impact Statement
This research and the data sharing plan were approved by the Institutional Review Board (IRB) at our institution. In our study, we sample from Reddit and ShareGPT following established data curation practices and treat LLM responses approximate statistical surrogates. We take several measures to safeguard the personal information in dataset. In accordance with IRB policy, anonymized all of the user posts and LLM conversations collected during the study by replacing
instances of personally identifiable information (PII), such as names and emails, with synthetic data. For these real documents and synthetically generated posts, we perform a rigorous manual inspection to ensure that the synthetically generated PII in our dataset does not correspond to any real individual. For instance, verify that no online search results can be found for a synthetically generated name. In our dataset, we explicitly collect portion of user posts with personal disclosures from marginalized vulnerable populations. We inspect our dataset to verify that the texts, titles, topics, and personal disclosures included in these posts contain no potential harm, such as hate speech or biases that may be perpetuated. that would impact these populations. All examples shown in the paper are either synthetically created by humans or by large language models and accurately reflect the real data. We hired in-house student annotators ($18 per hour) over crowd workers for annotation. Every annotator was informed that their annotations were used to create a privacy risk estimation dataset. For our synthetically generated posts, we take several precautions to ensure the fake user posts are natural while not posing any real harm. To prevent misuse by the aforementioned threat models, we will not release our dataset of real user posts and LLM conversations and finetuned models to the public. We will instead only share them upon request with researchers who agree to adhere to the following ethical guidelines:
Use the dataset is limited to research purposes only.
2. Redistribution of the models and dataset without the authors' permission is prohibited.
3. Compliance with Reddit's is official policies is mandatory.
Additionally, posts that have heen deleted by users will not be provided at the time of the data request. To request access these resources, please email the authors.
The primary purpose of our model is provide users with a supplementary tool to quantify the privacy risk associated with privacy-sensitive documents and online personal disclosures. In cases where the models incorrectly predict the privacy risk, these model failures do not pose an additional risk to user privacy. While the failure cases of the best-performing models tend towards overprotection as they estimate low k-values for a post, there are some instances where models will misclassify a document have lower privacy risk than in reality.
Overall, we believe there are many potential positive social benefits of our work. By developing tools to quantify the privacy risk of users for sensitive documents, we can inform users of potential cybersecurity risks and enable these users to make more informed decisions when sharing personal information online. As mentioned in the introduction, we envision this tool to serve as something akin to a password strength meter (38] for indicating the privacy risks of online disclosures. While no user studies have been conducted on this k-anonymity task, we plan investigate the utility of this tool for informing users about the risks of content sharing of personal information. Specifically, we envision this privacy risk quantification being utilized in a "Grammarly for Privacy" llike browser extension that helps users identify privacy-sensitive sensitive disclosures and quantify the overall risk of these disclosures, and that suggests user user- acceptabee rephrasings [19 40) that preserve the semantic meaning of the original disclosure but are less privacy-sensiee sensitive i... have higher predicted v-vauess
17

---

C Further Related Work
Demographic Estimation. There has been considerable work in political science and statistics on estimating the size of groups of people in population sharing certain characteristics. Prior work uses multilevel regression and post-stratification methods accurately model populations with overlapping characteristics like political
opinion and race national surveys and political polls [41 10 64 [6]. Other rescarch utilizes probability and
statistical theory to interpret the proportions as probabilities p for demographic subgroups [37] 22] 73 [7]
and
develop point estimates np for the number of people in subgroups using as the population size.
Privacy Protection in NLP. Research in NLP has found that LLMs engender several privacy risks by, for
example, generating personally identifiable information and email addresses unintentionally explicitly if 14
prompted [50] 78 32
55 74]
LLMs also capable of inferring private attributes from text. including Prior work has pre-traing trainrd LLMs with differential private data to develop privacy-
age, sex, and location
aware models [68] 75 70 and privacy enforcing algorithms have been developed in text or structured data to
uphold the privacy properties of k-anonymity 26 or B-likeness
(28) resisting machine learning-basee attacks.
D Synthetic Dataset
Metric
Train
Test
Dev
Number of Total Posts
100
106
14
Number of Orderings
216
230
35
Number of disclosures
905
889
135
Number of Domains
30
22
5
Average Length of Posts
795.41
727.48
905.36
Average Disclosures Ordering % of Posts in Shared Subreddits
4.19
3.87
3.86
49%
22%
0%
Table 3: Details about the Train, Test, and Development Splits of the privacy risk dataset.
The synthetic posts and the annotation process for obtaining the & value of the posts can be found here To construct the synthetic posts, we utilize the Reddit APT to obtain natural posts on the subreddits r/oakland, "TTooXChromoooees r/mentalhealthh r/cheyenne, r/gatech, and r/chapmauniiversityy Additionally, we used keyword searches to obtain mental health-related and women's 's health-related posts for more serious topics, such as "abortion". "birth control", "domestic violence". "depression". "suicide". and "bipolar disorder". 1000 posts were obtained from each subreddit, and the posts were manually filtered for posts that had at least 4 disclosures or other inferrable personal attributes.
We construct logically consistent user profiles by randomly sampling for personal attributes, such as age, gender, occupation, educational major, and personal experience. We use the natural posts as samples for user attribute distributions. For instance, an attribute distribution instance of age, gender, personal experiences, and mental health is created for natural post containing only mentions of those disclosures. User profile distributions are randomly sampled, and the personal attributes randomly modified. L.LM---1-IInstruccc a sampled user distribution and 3 natural posts as few-shot examples and instructed to create a post that consistent with the provided profile. The posts are immediately inspected for personally identifiable information (PII) and manually removed if existing PII in the synthetic post is related to a real person, community, or website.
is provided with
We finetune Ta-ga for informal style transfer for 10 epochs, 500 warmup steps, and a weight decay of 0.01. Due to the large size of Reddit posts, we use batch size of 1. Training takes at most 3 hours. For inference, we higher temperature of 0.9, as we find that this temperature results in typos and inconsistencies in the text similar to those found naturally on Reddit. This temperature also resulted in higher degree of informality that was natural on Reddit. In general, the percentage of words that were misspelled after using style transfer was 6%, which matched the spelling error rates on the subreddits that the synthetic posts were based on. The subreddits r/mental health and Chrooohhomosoeesss were very formal, having a spelling error rate of 2% compared to other Reddit communities with a misspelling rate of 21 %% these subreddits were often forums where people were secking serious advice or discussion. We evaluate the fidelity of these posts by providing a sample of 100 synthetic posts and 100 real posts to in-house annotators. For synthetic posts, the in-house annotators have a classification accuracy of 52%, and 50 mislabeled posts are included in our dataset for k-estimation.
To improve the fidelity of the synthetic posts further, in-house annotators were given synthetic posts and explicitly instructed to modify the posts seem more natural. Annotating a single post took 20 to 30 minutes. Reddit posts obtained from the respective subreddits were also provided as reference posts for tone, content, and writing. The in- house annotators were instructed to write down changes and comments that indicated if a post seemed unnatural. Only 2% of our synthetic Reddit posts require drastic tuning of the text, as most of the edits made were made regarding the style of the posts. In general, the common errors of LLaMA-3.1 1 -Inntrucct-8 8B are:
18

---

Too formulaic:
The post endings were composed in a very formulaic manner. In general. the posts were structured as personal story. before ending with 2-3 questions and expression of gratitude. Posts were manually edited to remove questions and develop different endings based on the references.
Posts often exhibited an unnatural amount repetition. For instance, post mentioned the phrase "I just feel like", or "I'm tired of" at least once, LLaMA-3. 1-n.--.----tttut------ would be inclined to repeat these phrases dozens of times in post. Posts were made to be more concise and natural.
Repetition:
Common Expressions: LLaMA-3.1 1-..--Inn Inntructt&BB often produced very common expressions from post to post. Examples include "Has anyone else experienced this?" and "I feel like I'm not really living" for depression issues are very common. Posts were edited to restructure these expressions.
Incongruity: Posts were sometimes generated with logical errors, such as time inconsistencics, where a post would mention "today" but describe something that happened last week. Other posts were generated with conflicting personal attributes, such as mentioning being first-year and a third-year in college. Some posts had sentences with tones inconsistent with the remainder of the text. Posts were reviewed to remove these logical and tonal inconsistencies.
Some sentences were vague, resulting in unnatural text. Examples include the phrases "I feel like I'm just stuck in this rut" and "I'm thrown into this hamster wheel going nowhere." These phrases are removed or modified to add additional details relevant to the user.
Vagueness:
Redundancy:
There are some sentences that would repeat the main idea of post, which was notably unnatural according to in-house annotators. For instance, for a post about someone being a domestic violence situation, feel like I'm trapped in this cycle of abuse and don't 't know how to escape" " redundant as their personal relationships were already mentioned. These sentences were removed.
First- Person Perspective: LLaMA-3. 3. ---nstrucct-- often had issues with perspective in generating questions. For instance, unnatural questions like "Domestic violence survivors, how did you get of it?" are generated in post about someone suffering from domestic violence. This is much more unnatural than asking "How do get out of this?" Questions are edited to be clearer and more grounded in the perspective of the user and the topic of the user post.
To evaluate the effectiveness of the synthetic posts, wc also evaluate a finetuned version of BRANCH without synthetic posts in the training set. We observe drop in range accuracy to below 40%. In contrast, models of the same size finetuned with the synthetic data show substantial performance gains over both prompt-based inference and training on real data alone, suggesting that this synthetic augmentation strategy supports generalization rather than hinders it.
E K-Estimation Dataset
Table TableB]shww shows the details of the train, test, and dev split of our k-cstimation dataset. Table Table 5 and Table provide the domain breakdown for the training set, development set, and test set, respectively. On average, annotating single user post for two or more orderings takes an hour to complete, and it took 30 minutes to review and revise the annotations. As each post annotated multiple times, the orderings result in different k: values, with differing high levels of certainty. house annotators are all college-educater and fluent English.
We follow prior work 19 and consider the following personal disclosure categories for privacy risk estimation: (1) location, (2) age. (3) relationship status, (4) gender, (5) pet, (6) appearance, (7) race/nationality, (8) sexual orientation, (9) health, (10) family, (1)) occupation, (12) mental health, (13) emotions, (14) reproductive health, (15) finance, (16) education, (17) crime, (18) events, (19) disclosure of other people, and (20) personally identifiable information.
Given a user post, annotators are instructed to identify and categorize all disclosures based on the augmented annotation schema. Annotators must generate a number & that matches the number of people sharing personal disclosures, including the subreddit name (e.g. ttwoxchromosomes for gender or r/cheyenne for location). Annotators model the task as a probability/proportion estimation task and utilize the probability chain rule to decompose joint probabilities into conditional probabilities. The annotators manually construct a Bayesian network from the disclosures by determining probability ordering for the self disclosures for broadest to most specific and by causality. To construct the Bayesian network structure, annotators determine the conditional dependencies of each disclosure with respect to the prior disclosures in the ordering. The conditional probability terms are then converted into textual queries and valid subqueries, and annotators find ground-truth answers to these queries based on publicly available statistics and census data. For subqueries, annotators additionally annotate for the existence of parentheses and the specific mathematical operation combine the subqueries together. Finally, annotators construct a math equation combining the answers and calculate the final v-vul for a post. To capture variations in probability orderings and variable dependencies, each post is annotated at least twice, ensuring multiple reasonable interpretations considered.
19

---

Domain
# of Posts
r/Salem
6
r/chicago
3
r/oxforduni
2
r'relationship_adviie
2
r/antiwork
Domain
# of Posts
t/mentalhealth
12
Total
14
VTToXXChromosomes
12
r/gatech
11
Table 5: Domain breakdown for the dev set.
s/chapmanuniversity
10
r/Cheyenne
8
Domain
# of Posts
r/oakland
6
r/manchester
4
Reddit
66
v'roswell
3
r/Cheyenne
5
r/eauclaire
3
r/gatech
5
r/oaklanduniversity
3
r/chicago
5
r/durham
2
r/ucla
5
rjacksonville
2
r/Salem
5
r/kansascity
2
r/BostonU
5
#/NYCApartmentss
2
r/hoodriver
5
r/duke
2
r/mit
5
r/smithcollege
2
r/woxchromosomes
5
r/udub
2
r/mentalhealth
5
r/uchicago
2
r/oakland
3
r/atlanta
r/oxforduni
2
r/southampton
1
r/venting
2
r/ossonnclleeee
r/MeetPeople
2
r/emersoncollege
1
r/antiwork
r/gvsu
1
/CPTSD
rfasktransgender
dllatingadviee
//braatffceding
r/Mounjaro ForType2
//antrepreneuRRide..e Ride
r'personalfinance
\/MeeteewPcopleHerr
d'SpicyAutise
r/motorcycle
r/SuicideWatch
/PhR4Friends
r/relationship advice
ShareGPT
40
Total
100
Total
106
Table 4: Domain breakdown for the train set.
Table 6: Domain breakdown for the test set.
Annotators further instructed to annotate for the reliability of the ground truth source. For instance, reliable ground truth sources come from educational institutions census datahases, while informal surveys labeled as unreliable. For each query, annotators also utilize ChatGPT [i] to obtain a prediction value for each query. Annotators record CharGPT's response and the reliability and annotator confidence of the prediction compared to the ground truth value. If ChatGPT does not return source. alternate Google source is used create prediction. Finally, disclosure queries labeled based the availability of ground truths. If there is either a reliable an unreliable ground truth for disclosure and associated query, the query labeled as feasible. there is no ground truth, but the disclosure vital to highlight or estimate (c.g., having two knee injurics) for privacy risk, the query is labeled as important. If there is no ground truth and the disclosure is relatively unimportant (e.g.. feeling sad). the query is labeled infeasible.
We conduct validation experiments to evaluate the human annotations of k and find that the average percentage of populations of interest is 22.24 22.. Notably, this percentage error calculated groups size 2 4, which reflects typical populations covered our dataset and in domains cover. While compounded errors of probability estimates are a theoretical concern, their practical impact remains low under our annotation protocol, especially for the domains (e.g., Reddit, ShareGPT) that utilize census information or public databases.
interest population
20

---

F BRANCH Details
The disclosures are ordered based on broadest to most specific and causality (ie., disclosures that are valid subgroups of other disclosures are ordered last). For instance, an age disclosure is ordered before health disclosure. as age is determining factor for several health conditions. Additionally, the dis- closures are ordered based on how much the conditional distribution changes from the joint probability. For instance, the probability P(malc, hip injury/ 24, Atlanta) is wildly different from the joint distribution P(24, male, hip injury Toowsbridge)) as it is unlikely for 24-ycar-old to have hip injury compared to older ages. Comparatively, P(24, hip injury/ Townsbridge, male) is much more similar to the joint distribution as the gender distribution of hip injuries is much more uniform, so gender occurs before age the probability ordering.
For BRANCH 03-mini, we make slight changes the pipeline. For the disclosure selection prompt, we find that 0i-m will often select of the personal disclosures due to its increased reasoning capabilities. We modify the prompt to emphasize the selection of disclosures that are feasibly estimated by humans with limited access to online databases. For the query generation module. m3-mini tends to generate hyperspecific queries that result in many overestimations of privacy risk with & 1. To mitigate the repetition of information, we provide the model with the history of queries generated for post and instruct the model to generate a new non-redundan query that does not repeat information generated in the existing queries. To further reduce the impact of hyperspecific queries, we utilize confidence thresholding method for query simplification. After the answer estimation module, 03-mini provides confidence score for its answer. If the confidence score is lower than a specified value, the generated query is specific and difficult estimate, so we instruct o3-mini to simplify and estimate the new query. We set the threshold as 0.55 and run the siresiilddiimllfficatiionn loop once for each query.
LLaMA-3.. 3.1 Instructt performs worse on BRANCH due to its instruction following capabilities. As such, we also modify the BRANCH pipeline accordingly:
The disclosure selection module output is checked to ensure that the included disclosures are only for disclosures provided in the prompt to prevent hallucinations.
The query generation module is repeatedly run until query is generated with the words "percent". "population", ", or "number".
The answer estimation module output is checked to ensure that the answers are at least I for population queries and less than for percentage queries.
Models are not prompted to generate subqueries for discrete cases involving addition, as LLaMA-3.1- Instruct-70B often generates subqueries that sum up be greater than 1.
For non-rcasoning models, we temperature of 0.7 for sample decoding, we find that running BRANCH with a temperature
decreases range metric performance by 5.41 %, on average. For finetuning BRANCH, we train Llama-3. 1- --Instruct-88 with LoRA on 4 A40 GPUs for 10 epochs, with a total batch size of 16, which takes at most hours per run. We use a learning rate of 3e-4 and Adam (48) as the optimizer with a weight decay of 0.01. We use cosine learning rate schedule with warmup ratio of 0.03, and for LoRA hyperparameters,
rank=8, alpha=32, and dropout=0.05. We update the embedding layer during finetuning as well. Overall, we finetune modules of BRANCH: selecting disclosures, ordering disclosures, determining conditional dependencies. query generation, and query estimation. The query generation method is finetuned to optionally produce subqueries. and we do not train a query simplification module for this finetuned variant. The query answers recomposed with Python interpreter.
G Metrics
We experiment with using percentage error to evaluate model predictions compared to gold standards and find that percentage error is much more susceptible to the influence of outliers compared to log error. For instance, the percentage error of k* = 2 and l = 100 is 4900%. We additionally find that the percentage error is inaccurate and does not agree with the other metrics for evaluating systems, as it evaluates GPT-4o Chain-of of Srain-of-Thoughh Thought to Chain-of of Thought
have an average percentage error of 1 13115%% By comparison, the LLaMA-3, I.Is-uut yields average percentage error of 26208%. 1e00ghy times better than GPT-4o. even though the model is worse in every other metrie. Additionally, percentage error is not as interpretable for k. especially for low k* values. For A = 2 and k = 100, the log error is 5.64, indicating that the model is 5.64 bits off from the gold standard k.. Privacy is evaluated by humans on logarithmic scale, so log error, which represents the average residuals between the model predicted & and the gold standard k", is more interpretable for humans. For the legend in Table I and Table 7 percentage error is computed by the difference between the inverse of the log errors (e.g., 5.64 ) to fit the differences in proportion for Spearman's p and Range accuracy. For selecting the hyperparameter a, we found that a = matched human assessments of privacy risk the best, following feedback from our in- house annotator. For instance, for privacy risk k 500, a prediction between [100, 2500] was found to be the most reasonable. We also report = 2 and a 10 in Figure
21

---

H
Experimental Details
We selected GPT-4o and o3-mini as models for BRANCH due their strong baseline performances. Hyperparam- eters were tuned using the development set of our dataset: a confidence threshold of 0.55 provided the highest range accuracy and prevented the model from over simplifili or under--inplifying queries; temperature values of 0 and were tested but underperformed compared to 0.7. Finally, used 5 reruns for uncertainty estimation as a practical compromise, balancing robustness with the computational cost of BRANCH's multi-step pipeline.
We also test linear regression models for predicting k. We obtain LLaMA-3.1-4 20 dense vector emheddings, which are then optionally emhedded with the sparse auto-encoder or sparse vector to predict & by fitting linear regression model.
-ee- res
R5
We use the dense
For ol -preview and DeepSeek R1, we experiment with regular few-shot prompting, zero-shot prompting, and program-of- thhougghhtttthh prompting on 30 samples. We find that all of these variants result in lower performance across all three metrics compared chain-of-thought prompting, with more than 10% lower accuracy. For the finetuned baseline models. LLaMA-3. 1-n..-Insttutt-811 Specifically, we use LoRA with 4 A40 GPUs for 5 epochs and a total batch size of 16. Finetuning takes at most 4 hours. We use learning rate of 3--4 and AdamW 48 as the optimizer with weight decay of 0.01. We use cosine learning rate schedule with warmup ratio of 0.03, and the LoRA hyperparameters the same as in finetuming BRANCH. The embedding layer is also updated. For finetuning the RoBERTa and LLaMA-3. 1..33.11IInnmmt Instrucc--88 8B baseline models, we train models to classify the base 10 bin values (10 (10(lo810(4)) as the k- &-number for given user post and disclosures. We also test regression models instead of classifying with base-10 10 bins and find that in practice, LLaMA 3.1--nstrutt-8E and NoBERTa-large perform much more poorly the regression setting than in the classification setting. We attribute this performance decrease to the limited amount of data that we have for training. Due to the large range of possible k-values, performance is much more difficult when requiring the model to predict millions of potential k-values compared to the closest base-10) 10 hin.
finetuned with the same parameters as BRANCH.
We test I.MMA-3.3-70B In- struct for all 3 prompting base-
lines and for prompting vari-
Model
2=2
a=
a 10
LLAMA-3.188B Dense Linear
13.04%
19.13%
20.87%
ant of BRANCH. We also
LLaMA-3.J Spase
11.30%
14.35%
17.83%
test an End-to-End reason- ing model of BRANCH, sim- ilar to the reasoning chains of ol-preview, RI, and o3- mini. Using the human pro- cess for annotating models. we finetune LLaMA-3.1-88 Instruct with LoRA to produce the individual chain-of chain-of-thought thought steps of selecting and ordering disclosures, determining the conditional independence as- sumptions, generating a textual query, estimating the answers to the queries, and recombin- ing the answers to produce fi- nal value all one inference step. This model setup mimics the thought process of humans, as well as the reasoning thread and final output produced by reasoning models.
lurge Finetuned
13.48%
30.00%
40.43%
1LaMA-3.1 l-nsstntt Few Shot
12.61%
21.74%
50,00%
LLaMA-3.1-Instratte (FT)
23.48%
43.04%
51.74%
ILaMA-3.1-lmsrrutto Fow Shot
12.17%
30,0055
40.00%
LLaMA-3.1 LLaMA-3.l-aasturhhe PoT
20.87%
37,83%%
$1.74%
LLaMA-3.1 L--M------ CoT
26.52%
16.09%
56.96%
LLMMA-3..llssttaeerre Shot
12.61%
29.13%
LaMAAAA--nnsttattte PoT
23.014%
42.17%
54, 35%
L..MA-3..-Instrnttt CoT
22.61%
45,22%%
56.57%
GPT-4o-mini CaT
22.61%
GPT-4o (D8-06) Shot
18.70%
42.17%
78%
GPT-4o (0S-06) PoT
31.74%
$4,7818
66.09%
GPT-4o (08- 06) CoT
32.17%
55,22%
62.61%
ol-preview CoT
33.18%
$5.66/2
63.91%
DeepSeck R. CoT
32.17%
56.96%
68.70%
03-mini CoT
33.91%
59.13%
66, 96%
BRANCH Finetuned End to End
27.39%
49.13%
BRANCH LLaMA-3.1 Imn Few Shot
29.57%
51.74%
58, 70%
BRANCH LLaMA-3.33 Few Shot
31. 33.5
53.48%
62.61%
BRANCH I.LaM-33 linnetuudd
32.17%
56.52%
67.39%
BRANCH GPT-4o (08-06) Few Shot
41.74%
66, 66.9
78.26/%
BRANCH o3 mini (01-31) Few Shot
40.43%
72.61%
$0.87%
Human Agreement
posts)
51.43%
78.79'
85.71%
Figure 5: Comparison of different a hyperparameter values for the range metric across all the models tested on our dataset.
I Further Results
We evaluate the doubly annotated user posts with the & metries as a thcoretical human upper bound of perfor- mance. We use one of the author's annotations as the gold standard and treat the other in-house annotator as the predictions. In doubly annotated data, we find that the variance mainly arises from the differing Bayesian networks that are constructed to solve for Human agreement achieves a Spearman's 's p of 0.916, log error of 1.57, and range metric of 78.79% indicating that annotators generally agree within the same order of magnitude and that k prediction models are still significantly short of human annotators.
Table 7 provides the results for the additional model setups on our & estimation task. The linear classifiers perform poorly with average Spearman's 's of -0.018, log error of 10.17, and range metric of 8.94%, indicating
22

---

Documents Log Eur.)
Reddit Documents Log Er.,
ShareGPT Documeuts
Model
PT
Range
pt
Rangef
P1
Log Err. Range12
L1MMA-311-8B Densel Linear
-0,031
10:56
19.135
-00002
11:26
9.23%
-0075
9.23
37.97%
LLaMA-3.1-SB Spurse Lincar
-0.149
0.025
9.07
11.71
25.32%
Ia-lare Pinctuned
0.339
5.17
30.001%
0.547
3,90
57.75%%
0.056
7.59
15.19%
LLaMA-3.1 3.1-snstraacn Few Shot
0.385
4.88
29.13%
0.227
1,66
34.11%
0,546
5.31
18.99%
LLaMA-3.3 Instractius PoT
LLaMA-3.--nnsractou
0.578
4.00
42.17%
0.411
3.97
41.72%
0.679
4.07
43.04%
LLaMA-3.2 Bantt---.-
0.595
0.374
4.12
44.37%
0.792
46.84%
BRANCH L.aMA--S (HT) End-to-Fad
0.625
3.38
49.13%
0.562
3.49
47.68%%
0.640
3.75
51 9099
BRANCH LLaMA-3. 70B Few Shor
0.754
3.171
53,48%
0.668
3.34
49,67%
0.827
2.85
60.7665
Table 7: Performance of additional methods on the k estimation task. t indicates a statisti- cal significance value of < 0.05 when comparing BRANCH to Chain-of- Seiinof--Thusee Thought using the same base model. Cells are colored by percentage A with respect to GPT-4o CoT performance:
15%
-10%
-5%
U%
+5%
+8%
+10%
GPT-4o Chain-ofThought
GPT-4o BRANCH (This Work)
191
10".
10a
Ground Truth
10h
Ground Truth
101
10r
10c
200
10
10:
10
10
19
10
16"
10t
10
123
10
18
10
Model Prediction
Model Predi ction
Figure 6: Plots of the GPT-4o prediction & compared to the ground truth k- in log scale. The dashed lines indicate the acceptable half- half-magnitude range surrounding the gold standard values. Incorrect predictions are shaded to indicate the level of magnitude of the residual errors.
that a linear relationship between disclosures and privacy risk does not exist. The finetuned RoBERTa-large is on par with L.L.MA33.1-Ims I-Instruct-70B Few-Shot prompting with a range accuracy of 30.00%. For LLaMA-3.3- find that model performance on par with
70B Instruct, variants, and the BRANCH model performs slightly worse than GPT-4o Chain-of- Chain-of-Thouuht. The comparable performance of LLaMA-3.3 with LLaMA-3.1 indicates that our k-estimation task is difficult across models of similar architectures, and new training data from recent text does not yield better probabilistic reasoning capabilities. BRANCH LLaMA-3.3 Instruttt results also statistically significant to LLaMA-3.3 I.nssruutt with Chain-of-Thouue prompting. We also evaluate the disclosure detection capabilities models by not providing GPT-4o with the gold standard labeled disclosures. GPT-4o Chain-of- Thouuht without the disclosure list drops to 3.27 log eтror and 49.67% range accuracy, indicating the importance of disclosure spans for k-estimation.
1..M--3.-IInstrucc.
for all the prompting
The BRANCH end-to-end finetuned model only yields a Spearman's 's of 0.562, log error of 3.49, and a range metric of 47.68% producing lower performance compared to BRANCH LLaMA-3.1 1-n---..--nsstutt-000 with few-shot prompting. We attribute this poor performance the complexity of the task. Given the number of intermediate steps in & estimation, LLLMA-3.. Instruct-SSB is incapable of producing the BRANCH process end- to soddto-edd, resulting in lower performance compared to BRANCH pipeline variants. Additionally, due to having only 216 training points, there is limited training data to learn the process end-to-end
o the dot plots for comparing the Chain-of of Thought prompting with BRANCH for GPT-4o. As seen with 03- mini, BRANCH GPT-4o is more centered around the main diagonal, predicting more values in the acceptable range. However, BRANCH GPT-4o also has some underestimation outliers similar GPT-4o CoT.
Figure F detailed breakdown of the errors of the models for & estimation. The log error magnitude and percentage of occurrence for all of the overestimation and underestimation errors is reported separately for each model. Importantly, BRANCH GPT-4o and 03-mini have some of the lowest rates of underestimation errors, with only ol -previiw being better. However, ol -previiw overestimates & much more, resulting in more limited utility in the model. Our BRANCH models are the best at underestimating privacy risk while maintaining utility by accurately predicting k. Figure F presents the range metric with the hyperparameters a and a 10. For a = 2, we find that the BRANCH oš-mini accuracy drops to 40% and increases to 81% for a = 10. The relative performance between models remains consistent regardless of the hyperparameter a.
J Ablation Studies
We further analyze the impact of certain modules utilized in BRANCH in Table 8 We utilize all of the unaffected intermediate output of the original BRANCH module. We analyze the impact of the conditional dependencies
23

---

Documents Log Err.
Reddit Documents
ShareGPT Documents
Model
Range%
Pt
Log Err.
Range%
Pf
Log Err.
Range%
GPT-4o Fully Disjoint Bayes Network
0.755
2.96
54.78%
0.661
3.33
48. 34%
0.848
2.26
67.09%
GPT-40 Fully Connected Bayes Network
0.797
2.40
62.61%
0.776
2.23
65. 56%
0.817
2.72
56.96%
GP No Sabquerries Module
0.837
2.30
63.91%
0.772
2.44
60.93%
0.860
2.02
69.62%
03-mini No Simplificatio Module
0.850
2.16
68.70%
0.813
2.19
68.21%
0.873
2.11
69.62%
Table 8: Model performances of differing BRANCH GPT-4o ablation studies. We utilize all of the inter-
mediate output in the original BRANCH model for better comparison. Cells are colored by percentage A with respect to the original BRANCH performance:
100%
.5%
0%
+5%
+8%
+10%
selection process in constructing an accurate Bayesian network for BRANCH-GPT-40. We construct fully disjoint Bayesian networks for cach user document, which assumes all personal disclosures to be independent of cach other, and fully connected Bayesian networks, where disclosures are dependent disclosures. We instruct LLMs generate new queries, estimate answers, and recombinc the probabilitics. BRANCH GPT-4o with fully independent disclosures results a range accuracy of 54.78%, and a log error of 2.96, and BRANCH GPT-4o with fully dependent disclosures results in a range accuracy of 62.61 %. and a log error of 2.40. Interestingly, we find that the fully disjoint Bayesian network performs poorly on Reddit and well on ShareGPT, achieving range accuracies of 48. 34% and 67.09%, respectively. On the other hand, the fully connected Bayesian network performs well on Reddit and poorly on ShareGPT, achieving range accuracies of 65.66% and 56.96% respectively. Thus, we find that the conditional dependence classification module in BRANCH is critical to high performance, as BRANCH outperforms hoth ablations hy producing queries that accurately represent the conditional probability terms while being easier to estimate for both the domains Reddit and SPareGPT.
all of the prior
We analyze the impact of the op- tional modules of BRANCH GPT- 4o. 12.5% of queries are broken down into subqueries, which pri- marily consist of age range queries, which estimate the percentage of people that are in a certain range and are divided by the size of the range. We use BRANCH without the subquery breakdown and find that performance drops, resulting in 0.837 Spearman's 's p. 2.30 log error, and 63.91% % range metric, indicating that the subquery does result in eas- ier query estimation. We note that BRANCH still outperforms all of the baseline models, even without this optional node. For the simplification module, find that only 6.56% of GPT-4o answers are classified as in- correct and are simplified. In com- parison, 35.86% of 63-mini answers are simplified due to low certainty. Removing the simplification mod- ule, we find that performance also drops slightly with a Spearman's 's of 1. log error of 2.16, and range
Model
% overestimale
underestimate
1.aMA-3.1 l Dense Linear
44.78% (1 (11.35)
36.09% (14.81)
LLaMA-3. Sparse Lincar
52.61% (10.71)
33.04% (12.88)
RoBERTa-lage (FT)
34.35% (6.44)
35.65% (7 34)
LLaMA-3. Instructes Few Shot
25.22% (7.98)
53,04% (9.76)
LLaMA-3.1 Instructes (FT)
12.17% (5.67)
44.78% (7.44)
LLaMA-3.1 I Instrrcctos Few Shot
49.57% (7.16)
20.43% (4.77)
LLaMA-3. Innstruccuss PoT
42.17% (6.10)
20.00% (5.45)
LLaMA-3. I-Instructos CoT
34.78% (6.82)
19.13% (5 84)
LLaMA-3.3-Instructt, Few Shot
46.09% (7.51)
24.78% (4.38)
LLLMA-3,3 Instructtn PoT
34.35% (6.89)
23.48% (5.32)
LLaMAA3.3-Imssrutto CoT
31.74% (6.33)
23.04% (5.70)
GPT-to-mini (08-06) CoT
10.87% (4.94)
51.30% (8.32)
GPT-4o (08-06) Few Shot
27.39% (5.79)
30.43% (6.21)
GPT-4o (08-06) PoT
14.78% (4.84)
30.43% (6.89)
GPT-40 (08-06) CoT
12.17% (4.20)
32.61% (6.
al-preview CaT
36.09% (5.76)
8.26% (6.11)
DeepSeek RI COT
26.96% (4.96)
16.09% (6.87)
03-mini CoT
16.52% (4.99)
24.35% (5.86)
BRANCH (FT) Eud-to-Eud
29.57% (6.62)
21.30% (5.41)
BRANCH LLaMA-3. Few Shot
25.22% (6.31)
23.04% (6.
BRANCH 3.aMA-333 Few Shot
25.65% (6.14)
20.87% (5.52)
BRANCH LLaMA-3. lan (FT)
16.52% (6.64)
26.96% (5.47)
BRANCH GPT-4o (08 (08-06) Few Shat
20.87% (4.56)
12.17% (5.62)
BRANCH o3 mini (01-31) Few Shot
15.22% (4.70)
12.17% (4.88)
Figure Detailed error breakdown of models on our dataset. The log error of overestimations and underestimations of privacy risks are shown in parentheses. accuracy of 68. 58.70%, indicating that utilizing verbalized certainty scores in 03- 03-mini can improve performance by simplifying specific queries.
K BRANCH Error Analysis
We perform an error analysis on BRANCH o3 mini and GPT-4o. For each incorrect privacy risk prediction, we manually identify and categorize the error reason. We find that all errors occur due to one of three reasons: (1) attribute selection, (2) query generation, or (3) probability estimation. Notably, these errors also occur frequently in Chain-of-Thought prompting errons, which occur 38.47% of the time. For BRANCH, 20.63% of 03-mini errors and 25.33% of GPT-4o errors occur due to disclosure selection, where BRANCH selects too few disclosures for estimation. 46.03% of errors in 03-mini and 37.33% of errors in GPT-4o emerge from the query generation component, as BRANCH may often generate too specific queries that are difficult to estimate accurately. These emrrs often further stem from overly strong conditional independence assumptions. 33.33% of
24

---

errors in 03-mini and 37.33% of enrrs in GPT-4o occur due to the probability estimation component, as models may underestimate simple yet specific query due to the lack of pre-training data, like the percentage of people who are social media managers a financial coach.
For underestimation errors, we also perform an analysis for the severity of this error. We define the error to be minor severity if the actual privacy risk It is greater than 50, moderate severity if the actual privacy risk is between 10 and 50, and major severity if the privacy risk is less than 10. We find that the majority of underestimation errors o3-mini are minor, with 53 33.85% for 03-mini and 50.00% for GPT-4o being on posts where the gold standard privacy risk is above 50. 15.38% and 14.29% of the errors of o3- mini and GPT-40 respectively are moderate severity, and the remaining 30.77% and 28.57% of 03-mini and GPT-4o underestimation errors are major. Given that BRANCH only underestimates error 12.17% of the time, our model has significant error rate of less than 4%, indicating the strength
Model
% Correct
GPT-4o CoT
52.34%
ol ol-preview CoT
57.81%
DeepSeek-RI CoT
57.81%
03-mini CoT
55.47%
BRANCH GPT-4o CoT BRANCH 03-mini CoT
69.53%
74.22%
Percentage of model predictions within the acceptable range of k= for posts with 4 or more personal disclosures.
Figure 8:
of our method. Moreover, we find that 83.33% of major severity underestimation errors are predictions with high variance, further allowing us to filter out these errors as uncertain model predictions.
L Comparing Chain-of- to BRANCH
Figure (urre/8ae Showws the performance of Chain-of-Thought prompting baselines and BRANCH GPT-4o and o3-mini on difficult posts with 4 more personal disclosures. We mainly utilize the range accuracy metric to determine when the model correct or not and compare the individual predictions between both methodologies. The best performing baseline model is ol oo-revveww and R1 with 57.81 % accuracy, while BRANCH o3-mini is the best overall model, correctly predicting 74.22% of posts with 4 or more personal disclosures, which is 1.61 % higher than its comprehensive range accuracy. BRANCH GPT-4o also outperforms the baseline models, successfully predicting the privacy risk for 69.53% of complex posts.
We further conduct a comparison between BRANCH and Chain-of- Prompting for 03-mini and GPT-4o, where the performance discrepancy is the largest. For posts where BRANCH correctly predicts is to he within the acceptable range but Chain-of- Thought prompting does not, the average number of disclosures in these posts is 4.44 and 4.26 for o3-mini and GPT-4o, respectively. 66.10% of these posts in BRANCH o3-mini and 64.9 %% of these posts in BRANCH GPT-4o have 4 more personal disclosures. By comparison, for user posts where the Chain-of- Chain-of-Thoughh Thought models correctly predict average number of disclosures in these posts is 3.68 and 3.6 for 03-mini and GPT-4o, respectively. Only 53.57% of these posts predicted by o3- m-mini and 50% of these posts predicted by GPT-44o have 4 or more disclosures.
be within the acceptable range but BRANCH does not, the
M Retrieval Augmented Generation
We also evaluate retrieval- augmented generation (RAG) systems by modeling query estimation as attributed question- answering task (8 with evidence. compare with BRANCH, we utilize the pre-existing output (e.g.. determining the conditional dependencies) of the GPT-4o pipeline. We create Thhievee-Thhn Read systems by using the Google Search API as a document retriever for each individual query. GPT-4o is instructed to select relevant information from the documents to estimate the answer or utilize its standalone knowledge to estimate the query if no information is found. We provide GPT-4o with 10, 20, 50, and 100 snippets of the relevant documents of a query, the entire HTML text of the top-10 documents, and 10 re-ranked snippets using BM25 on the 100 retrieved documents. We also use the commercial auureetl--uggmented AI search engine Perplexity-AN AI, namely sonar. sonar- pro, sonar-reason reaaorressnnnng.. sonar reasoning pro. and sonar deeedd research.. The results of the RAG systems reported in Figure
Retrieval Augmented Generation systems do not sufficiently outperform BRANCH. The best system, sonar- www.eeeeeeeareeeee only marginally improves performance for GPT-4o, achicving 0.839, log error of 2.16, and a range accuracy of 37.39% 64% of RAG systems perform worse than BRANCH-GPP
across all metrics, which contrasts with the strong performance of Al agents that use information retrieval to achieve state-of- the-art We altribute these results to the niche queries created in BR BRANCH, 71.30% of queries have document evidence in the first 10 search-retrieved documents. To remedy this, we simplify all the generated queries and run RAG with Google and Perplexity- -onar.. which still yields lower results than BRANCH with 3.48% lower range accuracy and 0,25 higher log error. RAG systems perform worse on UsM conversations than on User Posts, with 14.29% and 35.71 1% of systems exhibiting a log error of 0.32 higher on Reddit and ShareGPT documents, respectively.
results on benchmarks like GAIA 35 33 23).
25

---

Kargggg P8p Log Frrerr Spaammen
8.737
0,700
BRANCH
BRANCH
66.00
60.00
Fange
$8:00
60.00
63.00
$9.00
Soua
Rebd fo www.ouul---geeeedddd System
Retrienal-Aggnented System
Bett waall--ggmenteed Systum
Figure 9: Privacy risk estimation with retrieval-augmenned generation systems. BRANCH GPT-4o is highlighted in red.
Economics
total number UCLA
4: Parcentage of UCLA students that are studying business economics
number graduate
Retrieval Augmented Generation
year graduation
degrees awarded busileess
HUMAN
BRANCH
D
Answer Found
Figure 10: Example of query and the information retrieval process between RAG systems and humans. Retrieval Augmented Generation systems often fail to find evidence for generated query in BRANCH. Information retrieval is also difficult for humans, ground truths for probabilities are often found through multiple steps that break down disclosures into multiple search queries.
Overall, sonar- conar-deep-researe is the best RAG system, improving range accuracy and matching Spearman' 's P and log error compared BRANCH GPT-4o. The models exhibit high agreement with Spearman's 's 0.942, and the percentage of overlapping data points where both models are range k" 84.42%. saaaar-dee--reseeee
predicts larger i than BRANCH for 65.22% of posts and produces distribution than BRANCH GPT-4o by underestimating and overestimating privacy risk 15.65% 16.96% of the time, respectively.
On average, 67.11% of user posts with incorrect BRANCH predictions are also incorrect with RAG because BRANCH often produces niche queries. We prompt GPT-4o to determine if there is evidence for a query in the first 10 search-retrieved documents and find that 71.30% of queries result in document evidence. Figure T0 shows example failure case of Retrieval Augmented Generation. This information retrieval step hard for humans, requiring multiple information retrieval steps to adequately calculate ground truth for probability. Documents also negatively affect RAG performance; for example. "the percentage of college students that are second-years" (25%) is wrongly influenced by one snippet describing "90% first-year retention rate" at particular university. In another example, snippet had the statistic that 1.3% of women experienced domestic violence in the past year (based on a 2000 report), which resulted in the percentage women in the United States who have experienced domestic violence to be estimated as 1.3%. In reality, this metric is much higher, ranging between 25% and 30%. Retrieval Augmented Generation also results in incorrect query formats, as it will mix up percentage and population queries because the provided snippets may only provide available percentages for population query and only provide numbers for percentage query.
To improve retrieval accuracy, we create information retrieval baseline that assumes all disclosures be independent, resulting simple queries with answers that can be searched with retrieval. This baseline with 10 retrieved documents the worst- performing systems, resulting in 0.652 Spearman's 's p. 3.02 log error, and 54.78% range accuracy, demonstrating that simple retrieval systems cannot predict the & value of users, as the conditional dependencies of disclosures that result in niche queries vital for k-estimation performance.
26

---

Model Ranges
Precisiont
Recall
Macro-F1f
Interval Size
CoT Re- Se-Sampling
25.78
83.02
24.11
12.70
BRANCH Self-Consistency
33.49
45.53
25.60
3.80
BRANCH Re-Sampling
34.49
74.89
32.03
11.00
Table 9: GPT-4o Mean Estimate and Prediction Interval Results for Re-Sampling the model and
using Self-Consistency on the query estimation module BRANCH. The interval size is in log2-
N Prediction Intervals of Privacy Risk Estimation
To help provide a more informative range possible privacy risk values to users, we utilize L.LM re-sampling of Chain-of-
Ohain-of-Thought Thought and BRANCH GPT-4o to construct & prediction intervals using & and standard deviation os, defined as [max[] (1, k; 2. ha + 2ax] since k cannot be lower than 1. We also use self-consistens to construct prediction intervals from only sampling the query estimation module in BRANCH. Using the mean p: and variance o, of each probability query, create lower and upper bounds:
p: 2
if 2/0, > 0, otherwise.
P:
PPn
Pst min (ps + 1.
These probability terms are formulated to be within (0, 1J. as probability of O results k = 0. The answers to population queries are similarly constructed, and the lower and upper bounds of all the queries for post are combined to obtain the prediction interval [k,-, k.
We both methods 5 times all of the user posts from Reddit in the test set. Since each user post is manually annotated with j different orderings for kj, compare prediction intervals with gold standard post intervals kj ...
max ......., ,, kj We treat both intervals as integer sets classification Macro-F1, recall, and precision. True positives are integers in both the gold standard and prediction intervals: false positives are integers only in the prediction interval; and false negatives are integers only in the gold standard interval. We also report model variability with the log, size of prediction intervals. Table 9 provides the results these prediction intervals.
evaluate with
BRANCH prediction intervals result more accurate predictions Re-sampling BRANCH output yields the best prediction intervals, achieving a 32.03 Macro-F1 compared to 25.60 Macro-F1 for self-consistency prediction intervals. However, re-sampling BRANCH produces much larger variance, resulting in larger prediction intervals with average size of 11. 1..00 compared to 3.8 for self-consistency intervals.
Resampling Chain-of-Thoughe prompting is much less consistent for prediction intervals. Resampling also has the highest variance, with prediction interval size of 12.70. Due to the large prediction intervals, Chain-of- Thouught prompting has the highest Recall lowest Macro-Fl 83.02 and 24.11, 1, respectively. Comparatively, BRANCH consistently yields better performance in k-estimation across multiple runs and can be utilized to create accurate prediction intervals of privacy risk.
Computational Burden Analysis
We include a detailed analysis of inference time and memory usage (number of tokens and API cost) for BRANCH using GPT-4o in Table 10 which demonstrates strong performance across our tasks. For each row in the table below, use 5 Reddit posts containing between 2 and 10 disclosures. On average, the number of LLM calls, total inference time, and tokens generated scale approximately linearly with the number of disclosures a post.
Disclosures
of LLM Calls
Inference Time (s)
Tokens
Cost ($)
2
23
10,736
0.04
3
11
60
16,153
0.06
4
14
79
22,328
0,09
5
17
87
27,853
0.11
6
20
114
33,890
0.14
7
23
132
39,804
0.16
8
26
170
46, 108
0.18
9
29
213
52,538
0.21
10
32
233
58,938
0.24
Table 10: Inference time and memory usage of BPA-CH-GPT-40 across different disclosure counts.
27

---

Licenses
We list the details and licenses for the datasets and models used in this paper.
Datasets
ShareGPT:
sharegpt....
MIT License:
plob/main/Iicen
Models
laaaa3-8b-it-ee
Apache License v.2:
tree/main
RoBERTa-large
large
MIT License:
3LaMA-3.1-Instruee
Instruct
1-70B Iwwttnnntttiooooooo Iww.:nnnttt/llommmmmiiiiincc /tt.:/illloo/aannII//SS http:s//mngiiggaac.cc////se1ssss 1lammallammaaaa
Llama 3.1
Community License:
LLLMA-3.3- |ttpp/////aaggioeee///////a/aaaa
Llama 3.3
3-70B- 1000-mmmttrrttbbbbb//aaaa///1110 httpe://hagiunaaaae...///at1t111
License: Community
GPT-4o mini: htcerraghgr//oeeaaacccc//o/oloor
OpenAl Terms
GPT-4o:
http://////aocaaooaoaaaaooooo//a OpenAI Terms of Use:
Terms of
Use: heethpp.//paaaaa.ooo...aaaaaaaar
R1: httppe/www../rr////goooooo//ooee MIT License: https
03-mini:
httpa/////ap/aaat........./////r OpenAl Terms Use:
Prompts
We provide the few-shot prompts utilized BRANCH, the baseline models, the synthetic generation Reddit posts, and the rephrasing of Reddit text into LLaMA-3.1 1-1-3....ttttttt text for training data for T5-style transfer below. For the baseline models, we use the same prompt for Few-Shot, Program of Thought, and Chain-of of Thought prompting but vary the reasoning steps used in the provided examples.
28

---

Disclosure Selection Prompt
**Tas***; Determine the set of disclosures that are able to be estimated online.
**Personal Disclosure The act of intentionally sharing personal information about oneself with another person. This information something that others generally wouldn't know without being shared. Examples include static personal attributes such as name, age, and gender, or events an individual engages in over time, like health and education.
* "IIntruccioons*
Carefully review the post, the list disclosure spans their categories.
2. Refer to the subreddit name for potential information that can help search the query.
3. The subreddit name is also sometimes disclosure, may indicate location personal attribute, such gender. Include the subreddit
disclosure if is location subreddit and it is not mentioned **NNY** disclosure spans. Or. subreddit indicates a personal attribute and
contradict with the post, include this as valid, searchable disclosure span. A contradiction would be if the reddit user mentions explicitly another location or alternative personal attribute that goes against the norm of the subreddit.
Reflect deeply and thoroughly on *ALL* of the individual disclosure spans, and write down the specific/key IMPLICATIONS in format consistent with post.
Determine which disclosure spans are able to be estimated using trustworthy online statistics and data. Examples of trustworthy surveys surveys conducted by official government body, university student organization, university body. Other surveys that are trustworthy include disclosures about demographics. religion, relationships, sexual harassment, abortion, reproductive health, and mental health/ssychology.
6. Include disclosures that mention **OTHER PEOPLE* Include disclosures about locations that
users ** "FEEQUENT** such stores, coffee shops, etc. If the listed implications
estimated instead of the disclosure span itself, then include these disclosure spans. Include these disclosure spans the implications are these categories. 8. Present this list in <list></litDtagss. EACH **Personal Disclosure surrounded by <an- swer><<answerrtags and the corresponding **DICLLSURR CATEGORY** is surrounded by
<type></tpe>tags.
user planning doing something, consider the disclosure be valid the doing it. That is, include personal disclosure spans that have a degree of uncertainty. 10. =*DO * select duplicate disclosures. That is, two disclosure spans convey the same information but are written slightly differently. s*DO NOT** include both disclosures.
11. * D0 NOT** change the text of the disclosure spans. Return the disclosure spans **XXACTLY AS 18**
<examples>
* Reddit Post**; (reddit_post)
disclosure Span**; [self_disclel discclsurre_lsstt #ubreddi [subreddit]
** #n
Probability Ordering Prompt
**Task#*; Select orderings disclosures estimate sequentially, using the conditional rule probability.
**Personal Disclosure* The act of intentionally sharing personal information about oneself with another person. This information something that others generally wouldn't i know without being shared. Examples include static personal attributes such as name, age, gender, or events an individual engages in over time, like health and education.
** *#Istructioons*
Carefully review the Reddit post, the disclosure spans, their categories.
Refer to the subreddit name for potential information that can help search the query.
3. Reflect deeply and thoroughly on *ALI** of the individual disclosure spans, and write down the
potential ways to estimate each disclosure span using online data and statistics in list format.
Based on your listed implications, determine ordering estimate the potential queries as joint
29

---

probability. That is, given self disclosures as random variables (e.g.. A 24 years old). determine ordering A, B. C. D. such that the joint probability can easily be decomposed into conditional probabilities P(A) * P(B
P(C B. A) P(D | C, B, A)
Some guidelines for selecting the ordering are to select the disclosures that are the broadest/lestt specific first. For example, being certain race or being employed is broader than disclosures about your education level or working at specifie company.
Select *=OCCATION*** disclosures first, specifically disclosures that locations of residence first, including locations that people are moving. Locations include cities, states, countries, **universities**, or general institutions.
7. If the subreddit is an included location disclosure, select the disclosure that includes the location name *FISS*** over other location disclosures. If there select the subreddit name as the disclosure * ** disclosure has the name of the subreddit within the text, **DO NOT** add new disclosure span for subreddit name.
other mentions of location disclosures,
8. Select disclosures that are personal attributes revolving around **ONLY** the user **FIST*** such as **GENDER** age occupation. Do not order disclosures related to other people before these personal attributes, such as "relationship" or "family" disclosures.
relationship disclosures. order disclosure about romantic partner **FIRS*** (e.g.. wife, girlfriend, cx, partner) before ordering disclosures about other traits in the relationship (c.g., getting a divorce, being in a abusive houschold, etc.). 10. Select disclosures about onc's own education **EFFORE** the occupation.
11. Select disclosures based the of searching the queries. For example, is easier to search for P(male 24 years old) than P(24 years old male) since there are only two major categories to consider when estimating gender.
12. Order disclosures based on causality. For instance, for health disclosures, select the disclosure that may the health symptoms, such illness particular medication.
13. Present ordered disclosure spans **EXACTLY** as they are provided. #*DO NOT** break up a disclosure span into multiple spans.
14. Present this list in <list></ist>tags, with each query surrounded by <answer></answerrttgs. and each category surrounded by <type></tyne>tuss Make sure to include ALL queries.
<examples>
Please provide the implications with deep thinking, and determine the proper ordering of disclosures.
* *Reddit *ost**; (reddit_post)
disclosure Spans** (self disclosure_ #Sbbeeddi [subreddit]
#Asswr**:
Conditional Dependencies Prompt
**Task**; Determine if the of disclosures conditionally independent.
**Personal Disclosure**; The act of intentionally sharing personal information about oneself with another person. This information is something that others generally wouldn't 't know without it being shared. Examples include static personal attributes such as name, age, and gender, or events an individual engages in time, like health and education.
"Instructions
1. Carefully review the list of prior disclosure spans and their categories and the current disclosure span.
Refer to the Reddit post to understand the meaning of the disclosures in context.
For current disclosure span X, and prior disclosure spans A, B, ..., consider the disclosure pairs (X,
A), (X, B), etc.
3. Reflect deeply and thoroughly on *ALL* of the individual disclosure spans, and write down the specific/key (not general) IMPLICATIONS in a list format.
4. Based these implications, determine if the disclosure span X can be assumed to be conditionally independent to A without drastically affecting the distribution. That determine P(X A. [Z)) P(X
[Z]), where [Z] is a set of other random variables to condition on.
5. The prior disclosure span A is assumed to be conditionally independent if it does not drastically affect the distribution. Additionally, if removing the disclosure span as assumption results in probability that MUCH casier to search up with online data and statistics, the span A also assumed
30

---

to be conditionally independent.
On the other hand, disclosure span conditionally **DEPENDENT* if removing the conditional disclosure drastically changes the probability distribution. An example this include gender spans when conditioned on **OCCUPATION** For location spans, if there are other location spans that are near the current location span, then the spans are **CONDITIONALLY DEPENDENT** Account for these implications when deciding dependence.
If the disclosure span is conditionally independent, then it is dropped from the conditional probability. Present conditionally dependent disclosure spans #*EXACTLY** they are provided. Do break up a disclosure span into multiple spans. *DO NOT** include the current disclosure span in the list of prior disclosure spans.
Return the prior disclosureS SPANS that ARE NOT conditionally independent in <list></lisbtags. Enclose the spans in <answer></anwerrongs and the categories <type></tyne>tags.
<examples>
Please provide the implications with deep thinking, and determine the proper conditional independence assumptions of disclosures.
**eddii *o (reddit_post)
**Prior disclosure Spans**; * (prior disclosure_list) * disclosure Span*s; cecelfddscoouueeeecc span)
*n
Population Query Prompt
** Generate Google search query to determine population of people that share this specific disclosure span.
**Personal Disclosure**; The act of intentionally sharing personal information about oneself with another person. This information something that others generally wouldn't know without being shared. Examples include static personal attributes such as name, age, gender, or events an individual engages in over time, like health and education. Note: only information about the person posting (the "poster") is considered disclosure. Information about others is EXCLUDED.
Anonymnnmm Represents the number people who possess specific attribute or are in that specific situation. The generated search query should be aimed at understanding how many people have that particular characteristic, rather than a direct search of the span itself.
* Instruuctions*
Carefully review current disclosure Span(s).
2. Refer to the span category for clarity and additional information. For example, if the disclosure span
is "education", this indicates that the user is **SUUDENTs;
Refer to the subreddit name for potential information. If the subreddit is location, note this and in the query.
4. Consider the userbase of the particular subreddit. the subreddit used primarily by certain people,
like students, children, adults? If so, utilize this specific information in your query by focusing on these people.
Determine if the is new resident of location. If this number is smaller than the actual population of that location, so include this assumption as well.
Reflect deeply and thoroughly on *ALL* the information of the disclosure span implies for the poster, and write down the specific/key (not general) MPLICATIONS in list format.
Based your listed implications, formulate POPULATION or NUMBER of people that share the IMPLICATIONS ATTRIBUTES of the current disclosure span. Include the location information given. Present this query within <query><lquery>ttng. 8. If there is no location provided for certain population, note that the post is in English, so consider the population of interest either the United States the number of first-language English speakers. Be specific in your query.
accurate Google search query to yield the
<examples>
31

---

Please provide the implications with deep thinking, and formulate an accurate Google search query about the poster.
discolsure S; scef__disclosure__te *ubreddii [subreddit]
** #nswer**:
Probability Query Prompt
**
Generate Google search query to determine the k-Anonymity of specific disclosure span.
**Personal *; The act of intentionally sharing personal information about oneself with another person. This information something that others generally wouldn't know without being shared. Examples include static personal attributes such as name, age, and gender, or events an individual engages in over time, like health and education.
** "*k-Anonymity**: Represents the number of people who possess that specific attribute in that specific situation. The generated search query should be aimed at understanding how many people have that particular characteristic, rather than a direct search of the span itself.
** IInstructions
Carefully review the current disclosure Span(s) about the USER/POSTER. the current disclosure spans are separated by commas, then there are multiple current disclosure spans to estimate. Review prior disclosure Spans, and refer to the span category for clarity and additional contextual information. 2. Refer to the subreddit name for potential information. If the subreddit is location (i.c. city university). note this and use in the query population of interest.
For all other subreddits, assume that population interest the national population or population of English speakers as first language. If a prior disclosure span is a subreddit name that is not university or city, continue to use the United States as the population of interest.
4. Reflect deeply and thoroughly poster, and write down the specific/key (not general) IMPLICATIONS list format. Implications are user traits or concrete events that are happening, not **SU/JECTIVV** thoughts, concerns, or opinions on subject.
*ALL* the information of the disclosure span implies for the
5. Based on your listed implications, formulate accurate Google search query to yield the PERCENTAGE of PEOPLE sharing the prior disclosure spans THAT also share ALL the specific KEY implications of current disclosure span. Include the location information given. Present this query within <query></query>tag..
6. If current disclosure span consists of a comma separated list, estimate ALL the key implications of each disclosure span.
Note that the query includes the implications of the disclosure span rather than the exact text of the
disclosure span if the implications are **EASIER** to estimate than specific query itself.
8. Examples of these include financial situation, housing, parenthood, possession ownership, general education, health, employment status etc. These implications must be objective (i.e. something that has occurred user OR user's trait).
For medical disclosures, create queries about the percentage people that ** **AVE*** a symptom, condition, injury, etc. Do not create queries about the diagnoses.
10. The structure of the query must be *<<prior disclosures>THee <current disclosure implications>*** The disclosures that occur after the "THAT" clause are ONLY the current disclosure spans. Put the current disclosure spans after the "THAT" keyword.
11. *+DO NOT create queries in the following manner: Percentage of people <current disclosure implication(s)> AND** <prior disclosuress.
12. If there are no prior disclosures, then the query must be formulated as *ereeenttgee of **PEOPLE>* in the united states THAT <current disclosure implications >** Do not any other population group, you must use **PEOPLE**
refer to the general population.
13. Create queries about these disclosures objectively, as these disclosures are THINGS that happened to a user (c.g., going to an event) OR is user trait. *D NOT create subjective queries about the percentage of people **HAVING AN OPINION** or #*BEING CONCERNED** about something, unless ##xxpicitlyy** mentioned in the span.
<examples>
32

---

Please provide the implications with deep thinking, and formulate an accurate Google search query about the poster.
**Prior disclosure Spans**: [prior_disclosure__]]]
*Current disclosure Span**; {self_ celfffddsloosu__spae
Span Category**: [span_category * #Subreddd [subreddit]
*nnwwrr*:
Query Estimation Prompt
**Task**; Estimate the value for the specific search query. Present ONLY the numeric value within <answer></nswwertag...
#Instructions**:
1. Review the search query carefully understand what specific information is being sought.
2. Note that many queries conditional queries. These queries generally structured <population of interest>THAT <specific trait/activitys.. Estimate the percentage people population interest THAT share this specific trait.
3. For conditional queries, you are primarily estimating the disclosure span that occurs ** * the word "THAT". Estimate the percentage of people THAT share this trait. Otherwise, estimate the number people it population query.
4. DO NOT interpret the query in the reverse direction, such as estimating the number of people sharing the specific trait that are also the population of interest.
5. Estimate the answer based historic data to the query. Present the numeric value within
<answer></answerxtugs.
Provide numeric value no matter what,
7. If the number is a percentage, **REPRESENT THIS as decimal between and 1.
8. Please give confidence score a scale of O to for your prediction. The score must be in <scorc></score>ttng
<examples>
*+Search Query (search ]
Answer:
Generalization Subquery Prompt
**Task**: Break down queries about age disclosures into multiple queries for better estimation.
** **Personal * The of intentionally sharing personal information about oneself with another person. This information something that others generally wouldn's 't know without being shared. Examples include static personal attributes such as name, age, and gender, or events an individual engages in over time. like health and education.
**Instructions***
Review the following Google Search Query.
2. Reflect deeply and thoroughly on *ALL" of the aspocts of the query and the ability to search these aspects online, and write down the specific/key (not general) IMPLICATIONS in list format. Based your listed implications, determine if the following query be broken down into multiple subqueries based on online statistics and data.
A query can be SEPARATED ONLY IF the it can be estimated with a range or generalization. Census data reports ages in ranges (e.g.. 30-34), so this can be simplified by estimating the percentage of people within this age range. The second query should ask for the number of years in that age range. such that dividing range provides a uniform estimate for the percentage of people in single year.
percentage of people within this age range by the number years
age
5. Additionally, relationship status queries mention a gender of potential partner (e.g., wife, girlfriend, your fiancec), which can be broken down into **TWO** subqueries that divides the gender of the partner. This can be done by creating the first query for broad relationship family statistics.
33

---

For example, the disclosure "wife" can be generalized to the percentage of people that are married. Then, specify the number of bins divide this by. You estimate this with the expected number of genders. Dividing the query by this number will provide an estimate the percentage people that are married to a specific gender, i.c. wife.
If the query cannot be estimated by broken down into subqueries, **RETURN" the original query in <query></query>>agg
Otherwise, return the queries in <list></listtlags. Determine the mathematical operation needed be to combine subqueries into the original query, and return this <math><<malt>>tgs.
<examples>
Please provide the implications with deep thinking, and determine if the query needs hroken down into subqueries.
*+ [query]
** Answert#:
Diserete Subquery Prompt
**Task**; Determine if disclosures can be broken down into multiple queries for better estimation.
**Personal Disclosure* The act of intentionally sharing personal information about oneself with another person. This information something that others generally wouldn't know without being shared. Examples include static personal attributes such as name, age, gender, or events an individual engages in over time, like health and education.
** "Istrructions
Review following Google Search Query.
Reflect deeply and thoroughly on *ALL* of the aspects of the query and the ability to search these aspects online, and write down the specific/key (not general) IMPLICATIONS in list format.
3. Determine the following queries CONTAINS TWO SEPARATE. DISCRETE, disclosure spans in the CONDITIONAL. CLAUSE (e.g., "THAT CS Economics", etc.).
Discrete disclosure spans the conditional clause usually include the mention the word "OR". where one disclosure true or the other is true. If both or multiple are valid, then this is not a discrete disclosure span.
5. There is *CONDDTTOONLL * with the word "THAT". The presence of the word "OR" **UST BE INSIDE** the conditional clause. That discrete query that can be broken down will have the structure <... "THAT" OR ...>, Determine what the conditional clause is, and then detemrinc if the word "OR" is in the *CCONDITONNAL * If is, then the queries can he broken down. 6. If the word "or" comes ** the conditional clause, #*DO NOT** separate the query into multiple subqueries. The conditional clause **MUST CONTAIN the word **OR** s*DO NOT BREAKDOWN** disclosure spans that are just lists separated by commas. These do not indicate that there is a hidden "OR" in the span.
8. If there are two discrete spans in the conditional clause, break down the query into **TWO** subqueries that can be added up estimate the original query.
the query **DOES NOT** have discrete spans, then it cannot estimated by breaking down the query. *RETURN** the original query in <list></list>tags.
10. Otherwise, return the new queries in a list with <list></lstt-tass. Surround each query in <anserr<<aaswerpraags Determine the mathematical operation needed to be used to combine the subqueries into the original query, and return this in <math><<matthtaggs.
<cxamples>
Please provide the implications with deep thinking, and determine if the query needs to be broken down into suhqueries.
**uery**; [query)
** Answer**:
34

---

Evaluate Answer Prompt
**aak*** You are an expert on recognizing incorrect information. Determine if the answer to query is roughly correct.
**Personal D The act of intentionally sharing personal information about oneself with another person. This information is something that others generally wouldn't I know without it being shared. Examples include static personal attributes such as name, age, and gender, or events an individual engages in over time, like health and education.
*#Inttuuctoons*:
1. Review the Google Search Query and the answer.
2. This answer is produced by an arbitrary large language model, and it is not certain that the answer is accurate.
Using historic statisties and analogous data, determine the provided answer the query correct or not.
4. Provide your reasoning as to why the answer is likely to be correct or not.
5. Return your decision, "Yes" or "No" in <answer></answer>tgg based on if the answer trustworthy.
Only provide "No" if very likely that the estimate incorrect.
<examples>
Explain your thoughts.
**Query**; {query)
**Answer*;; (answer)
**Evaluation*:
Simplify Query Prompt
** Simplify the following Google Search query about disclosures.
**Personal Disclosure** The act of intentionally sharing personal information about oneself with another person. This information something that others generally wouldn't know without being shared. Examples include static personal attributes such as name, age, and gender, or events an individual engages in over time, like health and education.
* #tstruuctions
Review following Google Search Query.
Reflect deeply and thoroughly on *ALL* of the aspects of the query and the ability to search these aspects online, and write down the specific/key (not general) IMPLICATIONS in list format. Based your listed implications, determine what edits can be made that does not severely change the query but make casier estimate with online statistics and data. Examples of these changes to change location interest (e.g., from a city country) or to make generalization disclosure.
4. Return the simplified query in <query></query>tags.
<examples>
Please provide the implications with deep thinking, and formulate an simplified Google search query about the poster.
** **uurr***; {query)
Answertt:
Few-Shot; Chain-of- Thoughht; and Program Thought Prompts
**Task**; Estimate k-number corresponding a list of personal disclosures and Reddit post,
**Personal *isclosure**, The act of intentionally sharing personal information about oneself with another person. This information is something that others generally wouldn't 't know without it being shared. Examples include static personal attributes such name, age, and gender, events an
35

---

individual engages in over time, like health and education. Note: only information about the person posting (the "poster") is considered disclosure. Information about others is EXCLUDED.
"*/-Anonymity"" ;; Represents the number people who possess that specific attribute or are in that specific situation. The generated search query should be aimed at understanding how many people have that particular characteristie, rather than a direct search of the span itself.
##Inttuctionns*
1. Carefully review the Reddit post and disclosure Span(s).
2. Refer to the span category for clarity and additional information. For example, if the disclosure span
is "education", this indicates that the user is **STUENT***
Refer the subreddit name for potential information. If the subreddit is location, this and in query.
4. Reflect deeply and thoroughly on *ALII* the information of the disclosure span implies for the poster, and write down the specific/key (not general) IMPLICATIONS in list format. Based your listed implications, determine which disclosures be estimated with online statisties or trustworthy surveys.
Based on the estimatable disclosures, estimate the k- A-Anonymity value, which is a number that represents all the people that share the specific disclosure spans out of everyone in the world.
7. If there is location mentioned in the post, that the initial starting population. Otherwise, estimate the population the number of native English speakers (around 500 million) people. Present this k-value as exact integer number within <answer></anwwerttgg.
<examples>
**Reddit Post**; (reddit_ post]
Example disclosure Span**; [self_ taff_ddscloou__iiss *#xample Subreddi***; (subreddit]
**xample Answerts:
Synthetic Generation Prompt
You are a (age) (gender) person at (location). You are (race). You are a (occupation). You are lincome_level]. You are (orientation] and currently (relationship). You currently feel femotional condition]. You have [events] soon.
You will be given example Reddit posts from subreddit as reference. You are tasked with creating a new Reddit post that is *IMILARR**** to the provided posts in writing style and content.
Your response should reflect your profile, should specific, and include least "*4** of your personal **distinet** elements the response. Do not vague or generic.
The personal elements must occur **naturally** in the posts similar to the example posts provided. Additionally, once using personal element, **DO NOT** repeat the information again. For relationship elements, do not center the post the paragraph on the relationship.
Write your answer like Reddit comment or post. DO NOT MENTION the words "Reddit", "fellow", "Redditor" in your response. Return your post in canswer></anwwerttgg...
Write your responses with similar formulations and use language as assistant above examples. Notice that capitalization, punctuation, and correct grammar are often neglected, reflecting a relaxed, colloquial style.
Provided Examples: <examples>
Subreddit: [subreddit)
New Post:
36

---

LLaMA-3.1 Rephrase Prompt
You will be given a Reddit sentence from the subreddit (subreddit). Please rephrase this Reddit sentence using your own language. DO NOT omit any information. Return the post in <answer></answeerttgg.
Here are some example reddit sentences and example paraphrases.
Example Posts: <examples>
Real Post: {post)
Checklist
Claims: Do main claims made the abstract and introduction accurately reflect the paper's contributions and scope? Claims in the paper should match theoretical and experimental results in terms of how much the results can be expected to generalize. The paper's contributions should be elearly stated in the abstract and introduction, along with any important assumptions and limitations. It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. [Yes], see Section []
The paper's contributions should be clearly stated in the abstract and introduction, along with any important assumptions and limitations. is fine to include aspirational goals as motivation long as is clear that these goals not attained by the paper. [Yes], Section [
2. Limitations: The authors are encouraged to create a separate "Limitations" section in their paper. The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model well-speeification, asymptotic approximations only holding locally). (Yes), see Scction [or discussion on the Limitations.
3. Theory, Assumptions and Proofs: If you are including theoretical results, did you state the full set of assumptions of all theoretical results, and did you include complete proof's of all theoretical results? All assumptions should be clearly stated or referenced in the statement of any theorems. The proofs can either appear in the main paper the supplemental material, but if they appear the supplemental material, authors are encouraged to provide a short proof sketch to provide intuition. [N/A]
Did you state the full set of assumptions of all theoretical results? All assumptions should be clearly stated or referenced in the statement of any theorems. [N/A]
Did you include complete proofs of all thcoretical results? The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, authors are encouraged to provide short proof sketch to provide intuition. You are encouraged to discuss the relationship between your results and related results in the literature. [N/A]
4. Experimental Result Reproducibility: If the contribution is dataset or model, what steps did you take to make your results reproducible or verifiable? Depending on the contribution, reproducibility can be accomplished in various ways. [Yes), see Section 3 for the methodology. Section g for the prompts, Scction 4. 1 on how to construct the dataset, Section B on how to access the datasct, and Scctions H and F for the model details.
5. Open Access to Data and Code: If you experiments. did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? While we encourage release of code and data, we understand that this might not be possible, so no is an acceptable answer. [Yes], see Section f the prompts, Sections [H [add for the model details. We provide the synthetic posts of our dataset in the supplementary materials. Given the private nature of the user posts, we do not provide the real posts.
6. Experimental Setting/ Details: If you ran experiments, did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? The full details can be provided with the code, but the important details should be in the main paper, and information about how hyperparameters were selected should appear either in the paper supplementary materials. [Yes], see Section 4.3 and Section for
for model experimental details, and see Section 4. details the dataset.
7. Experiment Statistical Significance: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments? [Yes), see Scction 5 for statistical significance f standard deviation experiments.
8. Experiments Compute Resource: For each experiment, does the paper provide sufticient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? [Yes], see Section H and 7
37

---

9. Code Of Ethics: Have you read the NeurIPS Code of Ethics and ensured that your research conforms to it? [Yes]
10. Broader Impacts: If appropriate for the scope and focus of your paper, did you discuss potential negative societal impacts of your work? [Yes], see Section B for discussion of the Broader Impacts of our research.
11. Safeguards: Do you have safeguards in place for responsible release of models with a high risk for misuse (c.g., pretrained language models)? Released models that have a high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model. [Yes], see Section [ for a discussion the safeguards we put place. We apply several safeguards for the responsible release and utilization of our privacy risk estimation dataset due to its sensitive personal nature, such as requiring emails to access the dataset and redacting all instances of personally identifiable information in the final release of our dataset.
12. Licenses: If you are using existing assets (e.g.. code, data, models), did you cite the creators and respect the license and terms of use? Cite the original paper that produced the code package dataset. If possible, include URL. Be sure check the original license and respect its conditions. [Yes), see Scction P
13. Assets: If you are relcasing new assets, did you document them and provide these details alongside the assets? Researchers should communicate the details of the datasct the model part of their submissions via structured templates. [Yes], Section 4.1 for details about the dataset and Section A for limitations about the dataset.
14. Crowdsoureing and Research with Human Subjects: If you used crowdsourcing conducted research with human subjects, did you include the full text of instructions given participants and screenshots, applicable, as well as details about compensation (if any)? Including this information the supplemental material is fine, but if the main contribution of your paper involves human subjects, then we strongly encourage you to include as much detail as possible in the main paper. According to the NeurIPS Code of Ethics, you must pay workers involved in data collection, curation, or other labor at least the minimum wage in your country. [N/A]. We utilize in- in-ouse annotators for the creation our dataset and human annotations. All in-house annotators are paid $18 an hour, well above the minimum wage in our country.
15. IRB Approvals: Did you descrihe any potential participant risks and obtain Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your institution). applicable? Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. For initial submissions, do not include any information that would break anonymity, such the institution conducting the review. [Yes), see Section 4.1 and B for the inclusion of our Institutional Review Board approval.
16. Declaration of LLM usage: Does the paper describe the usage of LLMs if it is important, original, non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required. [N/A]
38


Reviews:
[{"review_id": "NJWxt9Ow2E", "reviewer": "Reviewer_TWWc", "summary": "The paper proposes a novel method, BRANCH, to use LLMs for probabilistic reasoning to estimate the privacy risk of user-provided documents. The authors create a dataset using Reddit posts and ShareGPT conversations, along with a synthetic set generated using LLaMA-3.1-Instruct-8B and T5-large models to make the data publicly shareable. The proposed method decomposes the joint probability of personal attributes using a Bayesian network, with disclosure categories labeled automatically by a model or manually by human annotators. They evaluate the method using Spearman's rank correlation between the predicted and gold-standard k-values, and with a log error metric over the k-values, motivated by the wide variance in population sizes.", "questions": "1. Do you observe any systematic bias across demographic categories? How do you ensure that LLM-internal demographic priors do not skew both training and evaluation? \n2.  Which stages (e.g., dependency modeling, subquery breakdown) are most critical to performance? Could a simplified variant retain similar accuracy? BRANCH includes many modular components, so I was curious to see if we can simplify them for cheaper cost to implement the method. \n3. The paper reports a high Spearman correlation (p=0.916) between annotators, which suggests stable relative ranking. Could you provide additional insight into the actual variance or range of K values across different disclosure subsets for the same post? e.g., how much does k fluctuate depending on disclosure inclusion?", "limitations": "W5 is not a weakness the authors are required to address within this paper, but rather limitations that are inherent to this class of approaches. Unless the authors choose to adopt a theoretically grounded framework, these limitations may not be easily resolved.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## **Strengths**\n1. Potential to be interpretable with bayesian factorization; conditioning on plausible priors (e.g., percentage of women in tech) sounds valid intuition than less tractable queries. Those query decomposition would contribute to capture more localized awareness of uncertainty for low-confidence answers.\n2. Original problem framing; the work provides a unique data curator view of privacy into a data contributor view \" how unique am I given what I said online?\". This enables bayesian approaches with conditional probabilities of each subquery factors.\n3. Practical settings; the authors create the synthetic dataset with Reddit to address the privacy issues in realistic settings. IRB approval obtained and the sensitive data is handled via T5 model generation and PII redaction.\n4. Strong experimental results\n## **Weaknesses**\n1. Engineering heavy; the proposed method relies on a large number of heuristic components from disclosure selection to subquery decomposition to answer estimation. While effective, the final pipeline seems to be more of a procedural engineering than a coherent algorithmic framewrk.\n2. Heavy reliance on LLMs; \n- Both the estimations of conditional probabilities and the creation of benchmark dataset to evaluate the method are based on LLM responses. This makes the method critically exposed to LLM bias, even though the authors attempt to mitigate this by involving human annotations.\n- LLM guesses are treated as statistical inference. The estimation of conditional probabilities via LLM generations lacks rigorous grounding. Such an approach falls into an assumption that the the LLM internal demographic believes as a valid values to model probabilistic estimates for k-values.\n3. Some of those human involvement is also heuristic. Esp. the selection of disclosure appears heuristic, and despite their trying to estimate the variance of the disclosures, it still remains vulnerable to cherry-picked selections.\n4. The prompt-based estimation is not quite innovative -- e.g., asking LLMs to output statistical answer or LLM-as-query-answerer is pretty common in current research trends. \n5. No theoretical contribution; this limits its potential to generate deeper insights for future works from a core ML standpoint.", "was_revised": true, "final_justification": "Responses the authors addressed are acknowledged. They are helpful to understand paper structures and how the responses are located within the paper, but it does not mean that the major weaknesses are addressed or mitigated. Hence, I keep my original positive score.", "rebuttal": ">Both the estimations of conditional probabilities and the creation of the benchmark dataset rely on LLM responses. This makes the method critically exposed to LLM bias, even though the authors attempt to mitigate this through human annotations. LLM guesses are treated as statistical inference. The estimation of conditional probabilities via LLM generations lacks rigorous grounding and risks assuming that the LLM's internal demographic beliefs are valid for modeling probabilistic estimates.\n\nWe thank the reviewers for raising this important concern. We would like to clarify that the benchmark dataset used to evaluate our method is constructed through careful human annotation, not generated by LLMs. As stated in lines 160–163, human annotators created the reference Bayesian networks and derived ground truth probabilities, drawing from real-world sources such as census data and public university statistics. We will revise the introduction to make this distinction more explicit.\n\n>Do you observe any systematic bias across demographic categories? How do you ensure that LLM-internal demographic priors do not skew both training and evaluation?\n\nThis is a very interesting point. We discussed this in the Limitations section (Appendix A) and Impact Statement (Appendix B) in our paper. We do not observe obvious biases against specific demographic groups in our dataset; however, there could be some potential bias and potential LLM-internal demographic priors that come with the base LLMs we used in our method. This challenge is not unique to our approach, but shared by many if not all LLM-based inference methods, including chain-of-thought prompting baselines discussed in our paper. In our study, we sample from Reddit and ShareGPT following established data curation practices and treat LLM responses as approximate statistical surrogates. For evaluation, we compare the LLM-generated estimations to human-annotated ground truth in Section 5.2, ensuring that LLM-internal biases do not affect the model evaluation. A comprehensive audit of all possible sources of bias (e.g., gender, race, geography, education level, age, culture) would require significantly more data and human annotation resources than what is feasible within the scope and budget of this academic study. We acknowledge this as an important direction for future work. Our research is exploratory and academic in nature, aimed at advancing privacy-preserving methods for protecting user data. It is not intended for direct real-world deployment without further safeguards and a deeper understanding and mitigation of potential biases. \n\n>Some of those human involvement is also heuristic. Esp. the selection of disclosure appears heuristic, and despite their trying to estimate the variance of the disclosures, it still remains vulnerable to cherry-picked selections.\n\nWe thank the reviewer for their comments and concerns. We would like to clarify that our human annotation process, including the selection of disclosures, is guided by prior work [1], which defines personal disclosures and provides a structured label schema, as mentioned in line 160 and Appendix C. We carefully adhere to this schema and consider a general threat model that accounts for all utterances of personal information within a post, so annotators do not filter out a subset of personal information in the text based on a heuristic process. Additionally, we doubly annotate a portion of our data to assess human consistency and annotator variance. We apologize for the confusion and will provide the full annotation guidelines in the Appendix. \n\nThe human involvement for selecting disclosures is only for filtering redundant information (e.g., multiple mentions of the same occupation in a post) that do not add statistical value to the privacy risk estimate. Regarding the variance during human annotation, this largely arises from the process of Bayesian network structure elicitation, not from the disclosure content itself. This includes variation in edge directionality and topological ordering of the variables, which may result in different conditional probability tables used to estimate k. To further assess model robustness, future work should look at user studies and explicitly examine performance on edge cases, including adversarial scenarios.\n\n[1] Dou, Y., Krsek, I., Naous, T., Kabra, A., Das, S., Ritter, A., & Xu, W. (2023). Reducing Privacy Risks in Online Self-Disclosures with Language Models. ArXiv, abs/2311.09538.\n\n>The prompt-based estimation is not quite innovative -- e.g., asking LLMs to output statistical answer or LLM-as-query-answerer is pretty common in current research trends.\n\nWe thank the reviewer for this observation and acknowledge the existing research that also utilizes LLMs as direct query-answering tools. We would like to clarify that our contribution goes beyond statistical query estimation with LLMs, as we introduce a probabilistic framework that decomposes joint probabilities into a structured Bayesian network and estimates the occurrence rates of personal information using conditional probability distributions. Large Language Models perform structured probabilistic reasoning, which is less explored in the literature, especially in practical domains with grounded human-annotated evaluations like user-level privacy risk estimation. We will clarify this distinction more explicitly in the paper.\n\n>No theoretical contribution; this limits its potential to generate deeper insights for future works from a core ML standpoint.\n\nWe acknowledge this point mentioned by the reviewer and agree that our paper does not contain contributions to ML theory. Instead, our work is primarily empirical, similar to a lot of recent work on LLMs and deep learning. Our task is application-driven, aiming to address a critical gap in user-side privacy support by using the probabilistic reasoning capabilities of LLMs, which we believe opens up useful directions for applied research in other tasks. \n\n>Which stages (e.g., dependency modeling, subquery breakdown) are most critical to performance? Could a simplified variant retain similar accuracy? BRANCH includes many modular components, so I was curious to see if we can simplify them for a cheaper cost to implement the method.\n\nWe thank the reviewer for this question! We run ablation studies (included in Appendix J) and find that the dependency modeling is the most critical component for performance, largely due to its key importance in constructing a Bayesian Network to estimate the privacy risk. The heuristic modules marginally improve performance, so a simplified variant, such as removing the components of subquery breakdown would result in a cheaper inference cost but comparable performance, especially for the simpler cases with less number of disclosures. In principle, with a substantially larger set of human-annotated data (albeit requiring significant upfront human annotation effort), it would be possible to fine-tune an end-to-end model that eliminates some intermediate components and significantly reduces inference-time computational cost.\n\n>The paper reports a high Spearman correlation (p=0.916) between annotators, which suggests stable relative ranking. Could you provide additional insight into the actual variance or range of K values across different disclosure subsets for the same post? e.g., how much does k fluctuate depending on disclosure inclusion?\n\nIn our doubly annotated data, the variance mainly arises from the differing Bayesian networks that are constructed to solve for k, rather than from differing disclosure subsets. On average, we see an average log difference between the human gold k predictions of 1.56. This variance indicates that annotators generally agree within the same order of magnitude, supporting the observed high Spearman correlation and low variance."}, {"review_id": "xlfqPN4mvy", "reviewer": "Reviewer_9q4K", "summary": "This paper introduces BRANCH (Bayesian Network Reasoning for k-ANonymity using Conditional Hierarchies), a novel Large Language Model (LLM) methodology to estimate the k-privacy value of user-generated text. The k-privacy value represents the size of the population matching disclosed personal information, aiming to provide end-users with an interpretable privacy risk estimate without requiring access to comprehensive databases. The authors also present a new dataset for privacy risk estimation, consisting of user-LLM conversations and Reddit posts.\n\nSummary:\n\nThe core problem addressed is the difficulty for end-users to understand the privacy implications of disclosing personal information online. The paper proposes BRANCH, which tackles this by:\n\n- Self-Disclosure Identification: Identifying personal disclosures in user documents.\n\n- Bayesian Network Structure Elicitation: Implicitly constructing a Bayesian network to capture interdependencies between attributes by ordering disclosures and determining conditional dependencies.\n\n- Query Generation and Subquery Decomposition: Converting conditional probabilities into search queries, which can be decomposed into subqueries for better estimation.\n\n- Query Estimation and Probability Recombination: Using LLMs' inherent knowledge (potentially augmented with Retrieval-Augmented Generation) to estimate query answers and then recombining these probabilities using a Python interpreter to calculate the final k-value.\n\n- Experiments show BRANCH estimates the k-value correctly 73% of the time, a 13% improvement over 03-mini with Chain-of-Thought reasoning. The paper also highlights that LLM uncertainty correlates with accuracy.", "questions": "- Are there any error modes where BRANCH performs worse than CoT? Some examples of when BRANCH fails would be valuable to understand its limitations.\n\n- Why not use ground-truth census APIs or structured data sources to improve query answers? The model currently relies on LLMs’ internal knowledge, which may drift. Could retrieval-augmented models improve robustness?\n\n- How was the “gold” k value validated in cases where no ground truth exists?\n\n- For many Reddit or ShareGPT examples, ground truth is unclear. What confidence do we have in those values?\n\n- What criteria were used to select the “ideal” ordering of attributes in the Bayesian network? The paper says LLMs are instructed to find a “feasible ordering” — but is there a heuristic or constraint guiding that process?", "limitations": "- The attacker model assumes someone familiar with the document context and matching on all attributes, but ignores auxiliary knowledge, linkage attacks, or adversaries with machine learning tools. The assumption that k provides meaningful protection without database access is not well-defended against modern adversaries.\n\n- The framework assumes accurate estimates from LLMs for subqueries, but no adversarial robustness or sensitivity analysis is performed (e.g., effect of minor wording changes or noisy queries).\n\n- As mentioned earlier, a significant portion of the training data (50/220) is synthetic, generated by LLaMA. There’s no ablation to test the effect of synthetic data on generalization, which could affect conclusions.\n\n- The BRANCH framework depends on manually annotated disclosures, Bayesian network structures, and conditional dependencies. This raises scalability and reproducibility concerns — it’s unclear how well the pipeline performs in the absence of such annotations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n- Novel Problem Framing: The paper shifts the focus of privacy risk assessment from dataset owners to end-users, providing an interpretable k value. This user-centric approach is highly relevant and practical for online privacy. \n\n\n\n- Innovative Methodology (BRANCH): Decomposing the complex problem of probabilistic reasoning into manageable steps (Bayesian network construction, query generation, subquery decomposition, and probability recombination) is a strong technical contribution. The use of LLMs to implicitly construct Bayesian networks and estimate conditional probabilities is a creative application of their reasoning capabilities. \n\n\n\n- New Dataset and Evaluation: The creation of a new, manually annotated dataset of Reddit posts and user-LLM conversations specifically for privacy risk estimation is valuable. The rigorous annotation process, including double annotation and ground truth validation against census records, indicates a high-quality dataset. \n\n\n\n\n\n- Interpretability: The k-anonymity metric provides an intuitive and interpretable value for users, which is crucial for practical privacy tools. \n\n\n- Thorough Error Analysis: The detailed analysis of Chain-of-Thought errors (conditional dependency, lack of subqueries, PII) and BRANCH errors provides valuable insights into the challenges of probabilistic reasoning for LLMs in this domain. \n\n\nWeaknesses:\n\n- Computational Burden: The paper acknowledges that BRANCH can pose a computational burden due to multiple LLM inference steps. While caching LLM outputs is suggested for future work, this is a significant practical limitation for real-time applications. \n\n- Generalizability of Threat Model: The current threat model assumes an adversary with knowledge of post-specific disclosure contexts. While practical, exploring more generalized threat models where adversaries have varying levels of access to information (e.g., publicly available databases) would strengthen the work. \n\n- Lack of User Study: The paper states that no specific user study has been conducted to test the model for providing quantifiable privacy risk estimates, despite prior HCI studies motivating the work. This is a missed opportunity to validate the real-world utility and user perception of the tool. There’s no user study or even simulated deployment experiment to validate how users interpret or benefit from k.\n\n- Potential Biases: The paper identifies that it has not analyzed whether the model itself may have biases that impact its use for certain demographics. This is a critical ethical consideration, especially for marginalized or vulnerable populations, and should be addressed before deployment. \n\n\n- Dataset Size and Scope: While the dataset is well-constructed, 220 documents (180 Reddit, 40 ShareGPT) might be considered modest for training robust LLM-based reasoning systems, especially given the complexity of probabilistic reasoning. The maximum of 10 disclosures per post is also a limitation for more extensive personal profiles. 50/220 documents are synthetic, and LLaMA-generated data may carry stylistic or topical biases. While synthetic data is common for privacy work, no ablation is presented on model performance with vs. without these documents. This raises concerns about data contamination or optimism bias.\nAnother thing I am a little worried about is that while BRANCH outperforms CoT in range accuracy, many documents only require reasoning over 2–3 attributes. For these simpler cases, CoT performs comparably, and the performance gap emerges only for 4+ attributes (Figure 4).\nThis limits the claim that BRANCH is broadly superior; it is specifically better for complex inputs, not necessarily the dominant approach in all privacy scenarios.\n\n- The “privacy strength meter” analogy is appealing, but there's no evidence this would aid users in practice.\n\n- The methodology relies heavily on manual annotation of disclosures, Bayesian structures, and equations, which limits scalability and reproducibility. While inter-annotator agreement is reported (ρ = 0.916), constructing Bayesian networks with conditional dependencies is subjective and nontrivial.", "was_revised": true, "final_justification": "The authors have answered all my concerns in a very detailed manner and their justification makes sense to me.", "rebuttal": ">Computational Burden: The paper acknowledges that BRANCH can pose a computational burden due to multiple LLM inference steps. \n\nWe thank the reviewer for their comments regarding the computational burden of BRANCH. Similar to other advanced reasoning models such as o3, BRANCH incurs higher latency with the trade-off of having better accuracy. The focus of this work is to develop a framework that can utilize LLMs’ probabilistic reasoning capabilities to estimate privacy risk as accurately as possible. As such, BRANCH is composed of multiple LLM inference steps that prioritize interpretability and robustness over inference time speed. \n\nHowever, we acknowledge that the multi-step inference pipeline of BRANCH poses a practical limitation. We will provide more concrete discussions of future research directions to mitigate this limitation, including:\n\n- LLM Output Caching.\n- Lightweight Model Substitution. In simpler posts, lightweight, distilled models can approximate BRANCH without full LLM calls.\n- Pipeline Optimization to combine BRANCH steps.\n\n>Generalizability of Threat Model: The current threat model assumes an adversary with knowledge of post-specific disclosure contexts. The attacker model ignores auxiliary knowledge, linkage attacks, or adversaries with machine learning tools.\n\nWe thank the reviewer for highlighting the point regarding the generalizability of the threat model and agree that this is a promising research direction for the future. In this work, we primarily introduce an unexplored framework for estimating privacy risk using probabilistic reasoning with LLMs, focusing on helping users determine if an individual person could identify them from their text. To ground our methodology and establish an upper bound on potential risk, we focus on a strong adversary with access to knowledge of all post-specific disclosure contexts. This setup enables a rigorous evaluation of the maximum possible privacy leakage under realistic but constrained assumptions. \n\nWe fully agree that extending BRANCH to accommodate more personalized threat models will further enhance its applicability. Modeling such adversaries, however, introduces new challenges, as it requires inferring both user preferences and adversary-specific background knowledge and uncertainty. Future work could study user-personalized threat modeling, where privacy risk is estimated relative to the knowledge scope of specific, personalized adversaries. Our current work defines privacy risk depending on the specific threat model. As such, our privacy risk value can change and be used to measure privacy risk against modern adversaries. We will update the limitations section to integrate this discussion for personalized privacy assessments.\n>Lack of User Study: The paper states that no specific user study has been conducted to test the model for providing quantifiable privacy risk estimates. There’s no user study or even simulated deployment experiment to validate how users interpret or benefit from k.\n\nWe acknowledge the reviewer’s concern regarding the lack of a user study in the current version of the work. Our primary focus has been to introduce a new framework and evaluation dataset for estimating the privacy risk of sensitive posts on online forums using probabilistic reasoning with LLMs. Our contributions center on technical feasibility and the design of interpretable, post-specific privacy risk signals.\n\nWe agree that understanding how users interpret and benefit from BRANCH’s privacy risk value is essential for practical deployment. While we are in the process of conducting such a study, user studies require careful design and extended timelines, and these studies are outside the scope of the current paper. Our goal in this work is to establish a robust, generalizable framework for privacy risk estimation, independent of application-specific deployments. We believe that a dedicated HCI-focused follow-up would be better suited to fully explore user-facing aspects, including presentation modes (e.g., privacy risk meter) and how users adjust behavior in response.\n>Potential Biases: The paper identifies that it has not analyzed whether the model itself may have biases that impact its use for certain demographics. \n\nWe appreciate the reviewer for raising this important point. Our work presents a methodological contribution for evaluating user-side privacy using LLMs, not a deployable system intended for real-world use at this stage. In our Limitations section (Appendix A) and Impact Statement (Appendix B), we discuss the potential concern about bias (not unique to our approach but inherent in nearly all existing LLMs) and affirm that we did not comprehensively analyze potential demographic biases inherent in the data. Our evaluation dataset is constructed by randomly sampling a comprehensive set of Reddit posts and ShareGPT conversations, which ensures that our method is representative of the data distribution but does not introduce or remove any particular biases. We will add further discussions in the paper to more explicitly state that any future deployment of this method should include thorough analysis and mitigation of potential demographic bias, especially when applied to data from vulnerable groups. \n\n>Dataset Size and Scope: 220 documents might be considered modest for training robust LLM-based reasoning systems. The maximum of 10 disclosures per post is also a limitation for more extensive personal profiles. LLaMA-generated data may carry stylistic or topical biases. \n\nWe thank the reviewer for their concerns. Due to the cost of annotation, we intentionally focused on atomic user posts. This required significant manual effort to ensure reliable gold-standard privacy risk estimates. Notably, the distribution of disclosures in our dataset is usually between 3–6 disclosures per post, closely matching the natural distribution of personal information across various subreddits, including from longer Reddit posts. We do not anticipate any fundamental limitations preventing BRANCH from generalizing to texts with more than 10 disclosures, though we have not extensively tested this case, as it is rare in naturally occurring data. We will clarify the scope of our dataset in the paper.\n\nWe include a portion of synthetic data in our paper, solely for the purpose of responsible research, such that we can publicly share this annotated data. We use 50 synthetic documents solely for training; all evaluations are conducted exclusively on human-written posts.  LLaMA-generated data may carry distributional biases, so annotators extensively refined the synthetic posts to better match the content and tone of real disclosures, using human-written posts as references. When finetuned on the training set data without synthetic posts, we observe a drop in range accuracy to below 40%. In contrast, models of the same size finetuned with the synthetic data show substantial performance gains over both prompt-based inference and training on real data alone, suggesting that this augmentation strategy supports generalization rather than hinders it.\n\n>The methodology relies heavily on manual annotation of disclosures, Bayesian structures, and equations, which limits scalability and reproducibility. While inter-annotator agreement is reported (ρ = 0.916), constructing Bayesian networks with conditional dependencies is subjective and nontrivial.\n\nWe agree with the reviewer on raising this point about scalability. Due to the need to construct a high-quality benchmark dataset, manual annotation is necessary to ensure reliable ground-truth labels for evaluation. Importantly, however, our proposed LLM-based method does *not* rely on manual annotations for inference. \nWhile constructing Bayesian networks involves some subjectivity, multiple valid network configurations yield similar estimations of the final k-values, as annotators’ predicted k-values were generally within an order of magnitude of each other. This is why we observe high inter-annotator agreement (ρ = 0.916), despite the flexibility in network construction. This consistency suggests that our method is robust to variations in network construction.\n>How was the “gold” k value validated in cases where no ground truth exists? For many Reddit or ShareGPT examples, ground truth is unclear. What confidence do we have in those values?\n\nIn our current setup, we are assessing how closely LLM-based methods approximate human experts. Human gold standard labels are validated using double annotation and with the validation experiment in Section 4.1 (lines 184-194), which shows that for interest groups similar to those found in Reddit and ShareGPT, the human annotation methodology exhibits very low error when compared to the ground truths. While a perfect ground truth is still unattainable for some cases, we have made every effort to construct the most accurate evaluation data feasible. Authors of the paper also manually review the annotation process to ensure the quality of the gold k value. We will add to the paper to acknowledge this inherent limitation of working with real-world data in this challenging setting. \n>Why not use ground-truth census APIs or structured data sources to improve query answers? Could retrieval-augmented models improve robustness?\n\nThis is a great point! We test RAG models in Appendix M (lines 999 onwards) and find that RAG systems only marginally improve performance, indicating that LLM internal knowledge is effective. We acknowledge that temporal drift is a concern to consider for possible deployment, but this temporal drift of demographic data would occur over a period of several years. \n>Are there any error modes where BRANCH performs worse than CoT? \n\nWe did not identify specific error modes where BRANCH consistently performs worse than CoT. We do an error analysis in the paper (included in Appendix K) and find that the errors caused by BRANCH are similar to error types commonly found in CoT."}, {"review_id": "95bWHz3rJo", "reviewer": "Reviewer_Pbz9", "summary": "This paper introduces BRANCH, a novel framework that enables large language models to estimate privacy risk in user-generated text by modeling the joint distribution of self-disclosed attributes via Bayesian networks. It achieves significantly better performance than existing reasoning baselines and provides interpretable k-anonymity scores along with uncertainty estimates.", "questions": "Please refer to Weaknesses.", "limitations": "Yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n• Novel LLM task and framework: The paper introduces a new task of text-level privacy risk estimation using LLMs and proposes BRANCH. This framework models the joint distribution of personal attributes via Bayesian networks. The formulation is original and practically relevant.\n\n• Well-structured and interpretable methodology: BRANCH decomposes complex probabilistic reasoning into conditional probability estimation and recombination. The framework is logically rigorous, clearly structured, and leverages LLMs’ natural language querying capabilities to produce interpretable outputs.\n\n• Comprehensive experiments and superior performance: The authors conduct thorough evaluations across multiple competitive baselines. Results consistently demonstrate the validity and superiority of BRANCH. \n\nWeaknesses\n\n• The paper does not analyze how consistent the LLM-generated Bayesian network structures are across multiple runs. It is unclear whether the inferred dependency orderings and structures are consistent, especially when statistical priors are noisy or unavailable.\n\n•  BRANCH assumes certain variables to be conditionally independent (e.g., gender and housing status) to simplify modeling. However, the paper does not quantify the impact of incorrect independence assumptions on the final k-value estimation. \n\n• The approach relies heavily on LLM-estimated demographic statistics (e.g., tech employment rates, birth rates), but the paper does not include a formal sensitivity analysis to assess how deviations in these estimates affect the final privacy risk predictions.\n\n• While the model is evaluated on Reddit and ShareGPT posts, a substantial portion of the training data is synthetic. It is uncertain how well BRANCH generalizes to diverse, real-world scenarios, especially outside the synthetic training distribution.", "was_revised": false, "final_justification": "", "rebuttal": ">The paper does not analyze how consistent the LLM-generated Bayesian network structures are across multiple runs. It is unclear whether the inferred dependency orderings and structures are consistent, especially when statistical priors are noisy or unavailable.\n\nWe thank the reviewer for pointing out the importance of evaluating the consistency of LLM-generated Bayesian network structures. In our paper, we conducted 5 runs (see lines 333-334) of the full BRANCH pipeline and found that the overall performance of the BRANCH model remained stable, suggesting robustness in Bayesian network elicitation. \n\nTo further quantify structural consistency, we analyzed the topological orderings of the Bayesian network using Kendall’s Tau across the 5 runs and obtained a score of 0.682, indicating a strong correlation in variable orderings across runs. Additionally, we computed the mean structural Hamming distance (SHD) between the induced shared subgraphs across different BRANCH runs. The mean SHD was 1.29, with a minimum of 0 and a maximum of 10, showing that only ~1.3 edge operations (additions, deletions, or reversals) are typically needed to transform one network into another across the 5 BRANCH runs. For context, the mean maximum possible SHD (i.e., comparing a fully connected subgraph to an entirely disconnected one) is 11.36, further affirming that the Bayesian network structures are highly consistent across runs despite noisy or missing priors. These findings collectively suggest that the BRANCH pipeline produces stable and consistent Bayesian network structures, even under variability in LLM outputs.\n\n>BRANCH assumes certain variables to be conditionally independent (e.g., gender and housing status) to simplify modeling. However, the paper does not quantify the impact of incorrect independence assumptions on the final k-value estimation.\n\nWe appreciate the reviewer’s point about the impact of independence assumptions. We conducted ablation studies (included in Appendix J) to compare different conditional independence assumptions (i.e., completely dependent structure or completely independent structure). We find that BRANCH with our selected conditional dependence assumptions consistently achieves the best performance, indicating that these modeling choices are empirically justified. We will add this discussion to the main paper.\n\nTo further assess the impact of incorrect independence assumptions, we conducted a detailed error analysis of the best-performing BRANCH system (included in Appendix K). We find that approximately 40% of the observed errors are due to issues in query generation, many of which can be traced to overly strong conditional independence assumptions. This suggests that while BRANCH is robust overall, a portion of its errors stem from independence assumptions, motivating future work on more accurate data-driven modeling of these dependencies.\n\n>The approach relies heavily on LLM-estimated demographic statistics (e.g., tech employment rates, birth rates), but the paper does not include a formal sensitivity analysis to assess how deviations in these estimates affect the final privacy risk predictions.\n\nWe thank the reviewer for this point. While we do not conduct a full formal sensitivity analysis, we explore the variability of the final k-values under self-consistency sampling for individual LLM-estimated demographic statistics (see Section 7). Specifically, we obtain multiple answers for the same queries and find that the resulting k-values constructed from these individual query answers exhibit low variance across samples, suggesting that the pipeline is robust to fluctuations in LLM-estimated demographic statistics. This indicates that the LLM is internally consistent when estimating these quantities in the domains in our dataset and that such variability does not significantly affect the final privacy risk predictions. We also empirically validate their reliability by comparing LLM-based estimates of individual queries to human-annotated ground truth statistical estimates in Section 5.2, observing that LLMs are quite accurate at producing statistical distributions in these domains. We will further clarify this discussion in the paper.\n\n>While the model is evaluated on Reddit and ShareGPT posts, a substantial portion of the training data is synthetic. It is uncertain how well BRANCH generalizes to diverse, real-world scenarios, especially outside the synthetic training distribution.\n\nWe appreciate the reviewer’s concern regarding generalization beyond our current domains. In this work, we primarily introduce an unexplored framework for estimating privacy risk using probabilistic reasoning with LLMs, focusing on helping users determine if an individual person could identify them from their text. BRANCH is evaluated solely on real-world user-written Reddit and ShareGPT posts, both of which are common platforms for sharing personal disclosures. That said, we acknowledge that these domains may not capture the full diversity of real-world scenarios.\n\nFuture work could look at developing user studies to evaluate BRANCH in more real-world contexts and exploring additional threat models and contexts (e.g., an individual who also knows additional information not present in the post) to better understand BRANCH’s robustness across use cases. Cross-domain generalization is also an important direction, including applications to Amazon reviews, job forums, and other online platforms where personal information is frequently shared. These efforts will help investigate how well BRANCH can generalize to diverse, real-world scenarios."}, {"review_id": "epKpw12iqn", "reviewer": "Reviewer_nErv", "summary": "This paper introduces BRANCH (Bayesian Network Reasoning for k-ANonymity using Conditional Hierarchies), a novel methodology for estimating privacy risk in user-generated content. The work addresses the challenge of quantifying how many people worldwide share the personal attributes disclosed in a text, providing users with an interpretable k-value representing their privacy risk. BRANCH factorises joint probability distributions of personal information by constructing Bayesian networks, estimating individual attribute probabilities separately, and combining them to compute the final k-value. The methodology achieves 72.61% accuracy in k-value estimation, representing a 13% improvement over o3-mini with chain-of-thought reasoning. The authors also demonstrate that LLM uncertainty correlates with prediction accuracy, with high-variance predictions being 37.47% less accurate on average.", "questions": "1. How does the computational cost of BRANCH scale with the number of disclosures? Could you provide a detailed analysis of inference time and memory requirements? What strategies could be employed to improve efficiency whilst maintaining accuracy?\n2. Threat Model Generalisation: Could the methodology be extended to handle partial knowledge threat models where adversaries only have access to subsets of disclosed information? How would this affect the Bayesian network structure and probability estimation?\n3. Model Selection and Hyperparameter Justification: The paper would benefit from clearer justification for specific model choices (e.g., why GPT-4o and o3-mini specifically) and hyperparameter selections (e.g., confidence threshold of 0.55, temperature of 0.7, 5 reruns for uncertainty estimation).\n4. Generalisation Beyond Current Dataset: How would BRANCH perform on documents with significantly more disclosures (>10) or in domains beyond Reddit and ShareGPT? What adaptations would be necessary for other types of sensitive content or different cultural contexts?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- Quality: The paper presents a methodologically sound approach to privacy risk assessment. The Bayesian network framework is well-motivated and appropriately designed for decomposing complex joint probabilities. The experimental design is comprehensive, including proper statistical significance testing, ablation studies, and error analysis. The evaluation covers multiple domains (Reddit posts and ShareGPT conversations) and includes uncertainty quantification.\n- Clarity: The paper is generally well-written with clear exposition of the methodology. Figure 2 effectively illustrates the BRANCH framework, and the examples in Figure 1 clearly demonstrate the limitations of chain-of-thought approaches. The mathematical formulation is precise, and the experimental setup is described in sufficient detail for reproducibility.\n- Significance: This work addresses an important practical problem in privacy protection and introduces a novel application of probabilistic reasoning to LLMs. The k-anonymity framework provides users with interpretable risk estimates, which could have real-world applications in privacy-aware systems. The methodology demonstrates meaningful improvements over existing approaches and could inform future research in privacy-preserving technologies.\n- Originality: The application of Bayesian networks to privacy risk estimation with LLMs is novel. The decomposition of joint probabilities into conditional dependencies represents a creative approach to handling complex attribute interactions. The integration of uncertainty quantification and the development of a comprehensive dataset for this task contribute to the work's originality.\n## Weaknesses\n- Scalability and Computational Burden: The methodology requires multiple LLM inference steps for Bayesian network construction, query generation, and individual probability estimation. This creates significant computational overhead, particularly when incorporating uncertainty quantification. The scalability to documents with more than 10 disclosures remains unexplored.\n- Limited Threat Model: The current approach assumes adversaries have complete knowledge of disclosed attributes, which may not reflect realistic threat scenarios. The lack of specialised threat models where users can specify what information adversaries possess limits practical applicability.\n- Dataset Limitations: The maximum number of disclosures per document is around 10, and the dataset is primarily focused on Reddit and ShareGPT conversations. The generalisability to other domains and longer documents with more complex attribute interactions is unclear.\n- Ground Truth Validation: Whilst the authors report 22.24% average percentage error compared to census data, there are concerns about compounding errors when combining multiple probability estimates. The reliance on human annotators for ground truth introduces potential biases.", "was_revised": true, "final_justification": "The author strongly addressed my concerns in the paper, and this would be a good addition to the conference proceedings.", "rebuttal": ">Scalability and Computational Burden: The methodology requires multiple LLM inference steps for Bayesian network construction, query generation, and individual probability estimation.\n\nWe thank the reviewer for their comments regarding the computational burden of BRANCH. Similar to other advanced reasoning models such as o3, BRANCH incurs higher latency with the trade-off of having better accuracy. The focus of this work is to develop a framework that can utilize LLM’s probabilistic reasoning capabilities to estimate privacy risk as accurately as possible. As such, BRANCH is an advanced reasoning model with multiple LLM inference steps that prioritizes interpretability and robustness over inference time speed. \n\nHowever, we acknowledge that the multi-step inference pipeline of BRANCH poses a practical limitation. We will clarify this in the revised paper and provide more concrete discussion of future research directions to mitigate this limitation, including:\n\n- *LLM Output Caching*: By caching intermediate outputs, especially for frequently seen or structurally similar inputs, we can significantly reduce the computation time.\n- *Lightweight Model Substitution*: In many cases, such as simpler posts, we can utilize Chain-of-Thought outputs or lightweight, distilled models to approximate certain stages of the BRANCH pipeline without full LLM calls.\n- *Pipeline Optimization*: We are investigating batching and combining BRANCH steps to reduce the total number of LLM calls required to estimate privacy risk.\n\n>Dataset Limitations: The maximum number of disclosures per document is around 10, and the dataset is primarily focused on Reddit and ShareGPT conversations. The generalisability to other domains and longer documents with more complex attribute interactions is unclear. What adaptations would be necessary for other types of sensitive content or different cultural contexts?\n\nDue to the cost of annotating privacy risk in text, we intentionally focused on the natural distribution of atomic user posts on domains rich in personal attributes that map directly to structured and publicly available ground truth sources (e.g., census data, university records). This required significant manual effort to ensure reliable gold-standard privacy risk estimates. Notably, the distribution of disclosures in our dataset is usually between 3–6 per post and rarely extends past 8 disclosures, which closely matches the natural distribution of personal information across various subreddits, including longer Reddit posts on communities like r/AITA. In these domains, we demonstrate that BRANCH is well-equipped to predict the privacy risk of sensitive documents, which serves as a strong foundation for initial development. We do not anticipate any fundamental limitations that would prevent BRANCH from generalizing to texts with more than 10 disclosures, especially since our analysis indicates that BRANCH outperforms CoT-based methods in handling dense attribute interactions as the number of disclosures increases. However, we have not extensively tested this case, as it is rare in naturally occurring data. We plan to explore such settings in future work, including domains with more complex attribute dependencies, especially where adversarial knowledge varies in granularity or source.\n\nIn culturally or contextually specific settings (e.g., financial communities with company-specific norms), additional adaptations will be necessary. If domain-specific ground truth data is available, we can incorporate retrieval-augmented generation (RAG) modules to contextualize inference. In lower-resource settings, fine-tuning expert estimators or adapting BRANCH with weak supervision from auxiliary statistics may improve reliability.\n\n>Limited Threat Model: The current approach assumes adversaries have complete knowledge of disclosed attributes, which may not reflect realistic threat scenarios. The lack of specialised threat models where users can specify what information adversaries possess limits practical applicability.\n\nWe thank the reviewer for highlighting the point regarding the generalizability of the threat model and agree that this is a promising research direction for the future. In this work, we primarily introduce an unexplored framework for estimating privacy risk using probabilistic reasoning with LLMs, focusing on helping users determine if an individual person could identify them from their text. To ground our methodology and establish an upper bound on potential risk, we focus on a strong adversary with access to knowledge of all post-specific disclosure contexts. This setup enables a rigorous evaluation of the maximum possible privacy leakage under realistic but constrained assumptions. \n\nWe fully agree that extending BRANCH to accommodate more personalized threat models will further enhance its applicability. Modeling such adversaries, however, introduces new challenges, as it requires inferring both user preferences and adversary-specific background knowledge and uncertainty. Future work could study user-personalized threat modeling, where privacy risk is estimated relative to the knowledge scope of specific, personalized adversaries. This would include adversaries with access to a user’s post history, scenarios where attacker knowledge is drawn from only public data, and threat models that have access to machine learning tools, similar to existing work in authorship attribution [1]. To accommodate these threat models, BRANCH would need to condition the Bayesian network on only the known subset, which may alter its structure, conditional dependencies, and downstream probability estimates. We will update the limitations section of our paper to integrate this discussion for more nuanced and personalized privacy assessments in future research.\n\n[1] Huang, B., Chen, C., & Shu, K. (2024). Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges. ACM SIGKDD Explorations Newsletter, 26, 21 - 43.\n\n>Ground Truth Validation: Whilst the authors report 22.24% average percentage error compared to census data, there are concerns about compounding errors when combining multiple probability estimates. The reliance on human annotators for ground truth introduces potential biases.\n\nWe reduce the risk of annotator bias through verifying our dataset with double annotation (see section 4.1) and achieving a high Spearman correlation (ρ = 0.916), which indicates strong consistency among annotators with relevant expertise. Additionally, the reported 22.24% average percentage error is calculated over population groups of size ≥4, which reflects the typical populations covered in our dataset. Thus, while compounded errors are a theoretical concern, their practical impact remains low under our annotation protocol, especially for the domains (e.g., Reddit, ShareGPT) that utilize census information or public databases. We will update this discussion in the paper.\n\n>How does the computational cost of BRANCH scale with the number of disclosures? Could you provide a detailed analysis of inference time and memory requirements? What strategies could be employed to improve efficiency whilst maintaining accuracy?\n\nWe thank the reviewer for raising this important question. We include a detailed analysis of inference time and memory usage for BRANCH using GPT-4o, which demonstrates strong performance across our tasks. For each row in the table below, we use 5 Reddit posts containing between 2 and 10 disclosures. On average, the number of LLM calls, total inference time, and tokens generated scale approximately linearly with the number of disclosures in a post.\n\nTo improve efficiency while maintaining accuracy, we are actively exploring several strategies:\n- *LLM Output Caching*: Caching intermediate outputs, particularly for frequently occurring or structurally similar disclosures, can substantially reduce redundant computation.\n- *Lightweight Model Substitution*: For simpler posts or lower-risk scenarios, we can substitute specific stages of BRANCH with Chain-of-Thought prompting or distilled lightweight models, reducing the number of full LLM calls required.\n- *Pipeline Optimization*: We are optimizing the pipeline through batching and combining steps where possible, further decreasing the total number of model queries needed for inference.\n\n\nWe will include the analysis table and a discussion of these strategies in the final version of the paper to better contextualize the computational tradeoffs of BRANCH.\n| # of Disclosures | # of LLM Calls | Inference Time (s) | # of Tokens | API Cost ($) |\n|--|--|--|--|--|\n| 2 | 8  | 23  | 10,736  | 0.04  |\n| 3 | 11  | 60  | 16,153  | 0.06  |\n| 4 | 14  | 79  | 22,328  | 0.09  |\n| 5 | 17  | 87  | 27,853  | 0.11  |\n| 6 | 20 | 114 | 33,890 | 0.14 |\n| 7 | 23 | 132 | 39,804 | 0.16 |\n| 8 | 26 | 170 | 46,108 | 0.18 |\n| 9 | 29 | 213 | 52,538 | 0.21 |\n| 10   | 32 | 233 | 58,938 | 0.24 |\n\n>Model Selection and Hyperparameter Justification: The paper would benefit from clearer justification for specific model choices (e.g., why GPT-4o and o3-mini specifically) and hyperparameter selections (e.g., confidence threshold of 0.55, temperature of 0.7, 5 reruns for uncertainty estimation).\n\nWe thank the reviewer for their suggestions. We selected GPT-4o and o3-mini as models for BRANCH due to their strong baseline performances, and hyperparameters were tuned using the development set of our dataset: a confidence threshold of 0.55 provided the highest range accuracy and prevented the model from over-simplifying or under-simplifying queries; temperature values of 0 and 1 were tested but underperformed compared to 0.7. Finally, we used 5 reruns for uncertainty estimation as a practical compromise, balancing robustness with the computational cost of BRANCH’s multi-step pipeline. We will clarify all of these hyperparameter choices in the paper."}]

Author rebuttal:

Meta-review: The paper introduces BRANCH, a new method that uses LLMs to estimate the privacy risk of a text. The main idea is to estimate a k-value, which represents the number of people in the world who share the personal attributes mentioned in a document.
The BRANCH method breaks down this complex problem into smaller, manageable steps. It treats the personal information as random variables in a Bayesian network to account for dependencies between different attributes (e.g., age, location, health conditions). It then uses an LLM to individually estimate the probability of each attribute. Finally, it combines these individual probabilities based on the structure of the Bayesian network to calculate the final k-value.

This approach is shown to be more accurate than traditional methods, particularly for documents with multiple personal attributes. The paper also highlights the importance of quantifying LLM uncertainty, as predictions with high variance were found to be less accurate.

The reviewers recommend accepting the paper given its originality and novelty, strong methodology, high-quality dataset and evaluation, and practical significance.

The reviewers also raised several weaknesses, including the computational cost, the reliance on LLMs, and limitations of the dataset and threat model. However, the authors' rebuttals were considered comprehensive and satisfactory, addressing most of these concerns.

In light of the above, I support accepting the paper.
Decision: Accept (poster)