URANIA: Differentially Private Insights into AI Use
Authors: Daogao Liu
Edith Cohen
Badih Ghazi
Peter Kairouz
Pritish Kamath
Alexander Knop
Ravi Kumar
Pasin Manurangsi
Adam Sealfon
Da Yu
Chiyuan Zhang
Venue: COLM 2025 (2025)
Keywords: Differential Privacy
Clustering
Summarization
TL;DR: We introduce a novel framework primarily designed for generating insights about LLM chatbot interactions while providing differential privacy guarantees, while also being applicable to other text corpora.

Abstract: We introduce _Urania_, a novel framework for generating insights about LLM chatbot interactions with rigorous differential privacy (DP) guarantees. The framework employs a private clustering mechanism and innovative keyword extraction methods, including frequency-based, TF-IDF-based, and LLM-guided approaches. By leveraging DP tools such as clustering, partition selection, and histogram-based summarization, _Urania_ provides end-to-end privacy protection.
Our evaluation assesses lexical and semantic content preservation, pair similarity, and LLM-based metrics, benchmarking against a non-private method inspired by _Clio_  (Tamkin et al. 2024). Moreover, we develop a simple empirical privacy evaluation that demonstrates the enhanced robustness of our DP pipeline. The results show the framework's ability to extract meaningful conversational insights while maintaining stringent user privacy, effectively balancing data utility with privacy preservation.

Published as a conference paper at COLM 2025
URANIA: Differentially Private Insights into AI Use
Daogao Liu
Edith Cohen edi wwwtthhhewwannncoom com
Badih Ghazi
Peter Kairouz
Liudaogao@gmail.com com
badiiggazii@gail.. com
kairouzegoogle.co com
Pritish Kamath pri gritishkégoogle.co com
Alexander Knop
Ravi Kumar
Pasin Manurangsi
aaknopégmail.com com
ravi. Bovi.k53@gmil.coom
pasinegoogle.com
Adam Sealfon adamseal fonegoggelo
Da Yu
Chiyuan Zhang
dayuwork@google. com
chiyuan@google.com
Google Research
Abstract
We introduce URANIA, a novel framework for generating insights about LLM chatbot interactions with rigorous differential privacy (DP') guarantees. The framework employs a private clustering mechanism and innovative keyword extraction methods, including rrqueencybassed TF-IDF-based, and LLM-guided approaches. By leveraging DP tools such as clustering, partition selection, and histogram-based summarization, URANIA provides end-to-end privacy protection. Our evaluation assesses lexical and semantic content preservation, pair similarity, and LLM-based metrics, benchmark- ing against a non-private method inspired by CLIO (Tamkin et al., 2024). Moreover, we develop simple empirical privacy evaluation that demon- strates the enhanced robustness of our DP pipeline. The results show the framework's ability to extract meaningful conversational insights while maintaining stringent user privacy, effectively balancing data utility with privacy preservation.
1 Introduction
Large language models (LLMs) have become ubiquitous tools used by hundreds of millions of users daily through chatbots such as ChatGPT, Gemini, Claudé.ai, and DeepSeek. It is valuable for both LLM platform providers and the general public to understand the high-level use cases for which these chatbots are employed. For example, such information could help platform providers detect if the chatbots are used for purposes that violate safety policies. However, while LLM platform providers have access to data of user queries, there are serious privacy concerns about revealing information about user queries.
Recently, Tamkin et al. (2024) proposed CLIO, a system that uses LLMs to aggregate insights while preserving user privacy, building Zheng al. (2023); Zhao et al. (2024) (see SE for more details). CLIO provides insights about how LLM chatbots (Claude.ai in their case) are used in the real world and visualizes these patterns in a graphical interface. Subsequently, Handa et al. (2025) applied CLIO to more than four million Claude. ai conversations, to provide insight into which economic tasks are performed by or with chatbot help.
CLIO is based on heuristic privacy protections. It starts by asking an LLM to produce individual summaries and to strip them out of private information from original user queries. The queries are clustered based on their individual summaries, and only large clusters are finally released, along with a joint summary of all the queries within each cluster. Furthermore, these clusters are organized into hierarchy.
In other words, the privacy protection that CLIO provides is loosely based on the notion of k-anonymity (Sweeney, 2002). However, the exact privacy guarantee of CLIO is hard to formalize, as it depends on properties the LLM used in the method. For example, it relies on prompts such as "When answering, do not include any personally identifiable information (PII), like names, locations, phone numbers, email addresses, and so on. When answering, do not include any proper nouns". While such approaches work reasonably well in practice, they rely

---

these to compare the performance of Published as a conference paper at COLM 2025
DP Clustering
DP Histogram Over Keywords
LLM Summarization from Keywords
Records
Summaries
heatt,
wellnese tion
Figure 1: Illustration of our proposed URANIA method.
heuristic properties of the LLMs, and hence may may not work well with future versions of the LLMs, thereby making such a system hard to maintain. To counter this, CLIO also includes a privacy auditor, also based on an LLM via a prompt that starts as "You are tasked with assessing whether some content
on a scale of to 5. Here's what the scale means: ... while acknowledging that such system can be perfect. In fact, Tamkin et al. difficult to apply
(2024) explicitly note that "since Clio produces rich textual descriptions, formal guarantees such differential privacy and k-anonynityy. Thus, natural question arises:
Is it possible to obtain acceptable utility for the task of summarizing user queries, with formal end-lo-end DP guarantees?
Differential privacy (DP) (Dwork al., 2006) is powerful mathematical notion that is considered the gold standard for privacy protection in data analytics and machine learning, and had been widely applied in practice (see, e.g., Desfontaines, 2021).
Our Contributions. We propose URANIA, a novel framework for summarizing user queries to an LLM chatbot with formal end-to-end DP guarantee, that does not rely on any heuristic properties of the underlying L.M used (64); our approach uses L.LMs for keyword extraction, and utilizes existing tools including DP clustering, partition selection and histogram release (see $2 for descriptions of these tools). En route,
In $2: we introduce the formal problem statement of text summarization of user queries and discuss the DP tools used in the work,
In $3: propose simplified version of CLIO that refer to SIMPLE-CLE (similar to CLIO, this does not satisfy any formal privacy guarantees). We consider this simplified version since the implementation of CLIO is not publicly available, and also, CLIO has additional aspects where it obtains many "facets" for each query, e.g., "concerning content" score, language of query, etc., which we ignore for simplicity. In 55: we propose a family of evaluation metrics to compare the quality of any two query summarization methods, and SIMPLE-CLIO and URANIA, and finally,
In SF: we propose a simple empirical privacy evaluation method using membership inference-style attack to conceptually demonstrate that indeed URANIA more robust than SIMPLE-CLIO.
Although we focus here on the application summarizing user queries an LLM chatbot, our techniques could be more generally applicable to summarizing arbitrary text corpora.'
This might require change the specific prompts that use.
2

---

Published as a conference paper at COLM 2025
We view the formalization of the text summarization problem and the design and evaluation of the first DP mechanism for it as our main contributions. We believe there are several avenues for improving such system, either by better prompt engineering by using better DP subroutines (e.g, for private clustering). Thus, we view our work as the first step in this promising and important research direction.
2 Formal Problem and Preliminaries
We formalize the "text summarization" problem by describing the (i) inpul to the problem, (ii) desired output, and (ii) approaches that we use for measuring ufility.
Input. The input is dataset (x Xn) of conversations, where each element x; X represents a conversation between a user and an LLM.
Output. The desired output for our problem is s {s ,k),,.,,,,, consisting of summaries 9j E S. The set S is potentially infinite (e.g., the set of all strings in our case). The DP guarantee applies only to the release of this output set s of summaries.
Utility. Evaluating the utility of privacy-preserving summaries presents unique challenges due to the unstructured nature of text data. For utility measurement purposes only, we produce a mapping Φ : x + s so that p(x;) is the summary with which the record x is associated. Importantly, releasing this mapping would violate DP and hence is never retured-itt serves solely as an internal mechanism for evaluation.
Rather than defining explicit utility function, we adopt comparative evaluation frame- work that assesses utility across multiple dimensions: (i) Lexical and semantic content preservation, (ii) Topic coverage and representation, (iii) Embedding bineedding-bbsed semantic sim- ilarity, (iv) LLM-based assessment. This multifaceted approach allows us to quantify the privacy-utility trade-off across different privacy parameter settings and implementation configurations. We defer the detailed discussion of these evaluations and their results to $5.
2.1
Differential Privacy
All the datasets in this paper collections of records. A randomized function that maps input datasets to an output space is referred to as a mechanism. For mechanism M and dataset x, we use (xx to denote the random variable returned over the output space of M. Two datasets x and x' are said to be adjacent, denoted x ~ x', if, one can be obtained by adding or removing one record from the other; this is referred to the "add-remove" adjacency. We consider the following notion of (e, -ddiffernnet privacy (DP).
Definition 2.1. Let > 0 and d E [0, 1]. A mechanism M satisfies (E, ( ddiffeeernttial privacy ((E, 5)DD for short) if for all adjacent datasets x x', and for any (measurable) event it holds that Pr[M E E] < et Pr(M(xx) E E + 5
We also say that mechanism satisfies eDDP it satisfies (e, O)-DP. To prove privacy guaran- tees, we will use basic properties of DP stated in Proposition A.l (in ŠA).
DP Clustering. Clustering is a central primitive in unsupervised machine learning (see, e.g., Ikotun et al., 2023). Abstractly speaking, a (Euclidean) clustering algorithm takes as input a dataset of vectors in IRd and returns a set C C Rd of k "cluster centers" (for specified parameter k), that are representative of the dataset; formalized for example, via the k-means objective (Lloyd, 1982). Clustering algorithms have been well-studied in the literature for several decades, and more recently with DP guarantees as well. We discuss more related work in SE. While our framework is compatible with any DP clustering algorithm, specifically, we use the implementation of DP clustering the open-source Google DP library?, as described in Chang & Kamath (2021); we denote the (e, instantiation of any such algorithm producing (up to) k clusters DP- DP-K-Meansk..-
3

---

Published as a conference paper at COLM 2025
DP Histogram Release. In the private histogram release problem over set B of bins, the input is a multi-set (2,,,,,... where each It; C B with |4j S k. The desired output is histogram that must be close to L h; possible, where abuse notation to interpret h; as an indicator vector in {0,1)
There are multiple algorithms for this problem (e.g,, the Laplace mechanism, Gaussian mechanism, etc.); in this paper, we use the discrete Laplace mechanism introduced by Ghosh et al. (2012), where PHR is the instantiation of this algorithm that is e-DP and tolerates removal of up elements. See ŠA for formal details.
Theorem 2.2 (Ghosh et al. (2012)). For all integers k > O and > 0: PHRi, satisfies e-D..
DP Partition Selection. In the private partition selection problem over (potentially infinite) set B, the input is a multi-set (h1, n2..... where each h; C B with [hj] I k. The desired output is subset S C B such that S C U; h; and is as large possible. We use the partition selection algorithm introduced by Desfontaines et al. (2022); P is the instantiation of this algorithm that is (e, See SA for formal details.
Theorem 2.3 (Desfontaines et al. (2022)). For all integers k 0, 0 0and 8 E [0, 1]: PPS satisfies (e,
3 SIMPLE-CLIO
We present a non-DP (public) framework for hierarchical text summarization, which is simplified version of CLIO (Tamkin et al., 2024). Our approach transforms a dataset x = (x), Xn) X" of conversations into high-level summaries ($1,..., E S through three stages. The pseudocode can be found in Algorithm 4 (in SB) with the implementation details as discussed below.
1. [Embeddiig Generation] To use clustering algorithm, we convert conversations to numerical vectors. A typical way to do this is to use pre-trained embedding models; specifically, use all all-mpnet-base-w following Tamkin et al. (2024), we first map the conversation to a summary text before applying the embedding model; this ensures that conversations that are similar in content but structured differently get mapped to embeddings that are closer. We use an LLM to map each conversation the summary text, using a prompt as described in SG.1.
(Reimers & Gurevych, 2019; 2022). However,
2. [Clustering] We apply the standard k-means clustering (Lloyd, 1982) to group similar conversations based on their embeddings. We select k to achieve an average cluster size of approximately 150 conversations. We assume KMFANS;()) returns the centers of clusters and the assignments (points in each cluster).
[Summary Generation] For each cluster, we generate a representative summary by sampling both random conversations from the cluster and contrastive conversations near the cluster center. We then use an LLM to produce summary that captures the main theme of the cluster. The prompt used for this step is provided in SG.2.
This framework provides an approach for organizing and summarizing large collections of conversations, enabling the identification of key topics without relying on manual annota- tion or predefined categories. However, as mentioned earlier, it does not satisfy any formal privacy guarantee. We next proceed to describing URANIA, our DP framework.
4 URANIA: A DP Framework for Text Summarization
We present URANIA, framework for hierarchical text summarization (with Hierarchy and Simple Visualization presented in HH) that builds on the public framework while providing formal DP guarantees. Unlike SIMPLE-CLIO that is non-private, URANIA incorporates privacy-preserving mechanisms at critical steps to ensure that the resulting summaries do not leak sensitive information about individual conversations.
Our approach takes a dataset x = Xx) E X of conversations, a predefined set IC of keywords, number k IN of clusters, number of keywords used for summary, and
4

---

Published as a conference paper at COLM 2025
Algorithm 1 JRAAIAA/Aeofaeeeseaeee Differentially Private Text Corpora Summarization
Parameters:
Number of clusters k, number i of keywords used for summaries, and cluster size threshold
Parameters: Privacy parameters Ecr Ehstr Esize 0, [0, 1].
Input: Input dataset x = (x), Xn) E A", keyword set (g)) C Sk
Output: High Level Summaries
// Step 1: Extract Embeddings from Conversations
e - |ExtractEmbeddings(x)))) : [n]]
// Step 2 (a): Cluster embeddings and assign records to cluster centers
C DP-KMEANSAAAA(e)
Let Ci...., C2 S (,), such that E Cj iff arg miny lley -e:ll e:ll
1/ Step 3: Extract keywords
Let K; Ø for ;
Initialize empty keyword sets // Estimate sizes clusters.
Let C(size) + PHRieem ((IG))e)))
for to k such that cisite) 2 do
if C 0 then
K f (andomkeywwrls(((.)))
// Random keywords for empty clusters
else
Step (b): Keyward generation for qualifying clusters
samples; (andomSample(C,,m))
// Sample up to conversations from cluster
Let relevannt.kyworrs be empty collection
for samples; do Append LLMSeeccKeewwodd(x..5. 5) to relevant kelworss
// Select to relevant keywords
Let FiA be the number of times keyword & present in relevant.keywords private hist, - PHR (frj))ex)>
// Apply DP histogram am mechanism Select top keywords
Ki prriiaa hiit,,
// Step 4: Generate summaries
(LLMSummarize(k)) [k]]
return s
privacy parameters Ec, Ehist Esize > 0 & E [0. The choice of the set K. of keywords is critical to the quality of the generated summaries; discuss multiple strategies for this in $4.1. Our method produces high-level topical summaries s (sp...., E S through the following privacy-preserving pipeline:
1. [Embedding Generation] This step is the same as SIMPLE-CLIO.
2. [Clustering] Non-private clustering algorithms usually output both cluster centers and cluster assignments. However, cluster assignments are non-private for add-remove adjacency; hence, DP clustering algorithms output only cluster centers.
(a) First, we generate cluster centers using an (Ec, 6e) 6 -DP k-means algorithm. Although we the algorithm described in Chang & Kamath (2021), any DP clustering algorithm can be used here and the same privacy guarantees would follow.
(b) Each conversation is assigned to its nearest cluster center.
3. [K IKeyword Extraction] For each cluster, we:
Apply DP to the cluster size with privacy parameter ésize the noised size is smaller than threshold r, we skip the cluster to avoid generating summaries that will not be meaningful.
Sample several conversations from the cluster. For empty clusters that pass the threshold due to noise, we randomly select keywords from the predefined set. Use LLM to extract relevant keywords from the predefined set K. for each sampled conversation. For the purpose of DP, we restrict the number of keywords chosen per conversation to at most 5. See prompts in SG.3.
Apply a DP histogram mechanism with privacy parameter éhist to identify the most frequent keywords.

---

Published as a conference paper at COLM 2025
Test various settings of Esize and to evaluate the trade-off between privacy protection and utility.
4.
[Summary Generation] For each cluster, generate a high-level summary using only the selected keywords. See prompts in $G.4. We also discuss creating a hierarchy of summaries in SH.
Theorem 4.1 (Privacy Guarantee). Let K, E IN, Ec, Ehist Esize > 0, and de E [0, Then URANIAK,RRCRNeeAeaa satisfies (Ec + Ehist + Esize, OcD
Proof. The algorithm can be split into four parts: cluster center generation from the original dataset (steps 1-2(a)), estimating cluster sizes, algorithm generating key words for given list of cluster centers passing size test and the original dataset (step 3 (b)), and summary generation from given set of keywords (step 4). We use DP properties (Proposition A.1).
The first part satisfies (ec, 6DP since it consists of operations applied per record and subsequent application of the (ec, instantiation of the DP-KMEANS algorithm. The second part is -Di DPP provided that the set of cluster centers is fixed since it again applies per-record operations and an - histogram release algorithm. The third part D provided that the set of cluster centers is fixed since it again applies per-record operations and an hhstt histogram release algorithm, by parallel composition. Altogether, this means that the algorithm producing Ky,..,, Kg is (Ec + Eist + Esize, bc) -DP due to basic composition.
Finally, note that the third part is processing the output of the first two parts, so due to the post-processing property of DP, the entire algorithm satisfies (Ec + Ehsst + Esizer 6cD
Remark 4.2. The process of choosing the set of keywords K. could itself consume some privacy budget, which needs to be accounted for separately. We discuss these methods next.
4.1 Choosing the Set of Keywords IC
The effectiveness of URANIA crucially depends on the quality of the keyword set IC used during cluster summarization. We explore four approaches for constructing keyword sets K, with varying privacy guarantees: namely, KwSet-TFIDF and KwSet-LLM satisfy DP, KwSet-Public uses a public dataset to obtain keywords, and KwSet-Hybrid uses a hybrid approach. We describe these methods informally below and formally in SC. We discuss some related work in SE.
TF-IDF-Based Selection with Partition Selection (KwSet-TFIDF), We compute the doc- ument frequency (DF) of all terms with additive Laplace noise to satisfy DP. Using these noisy DF values with token frequency (TF), we select 3-5 keywords for each conversation. We then apply partition selection (Theorem 2.3) to privately identify significant keywords, followed by LLM refinement to obtain 200 final keywords (see SG.5 for prompt used for refining). This approach is better suited when direct LLM access to conversations raises additional privacy concerns.
LLM-Based Selection with Partition Selection (KwSet-LLM), We first use an LLM select 3-5 keywords for each conversation independently. This produces large candidate set of keywords across all conversations. We then apply partition selection to privately identify the most representative keywords while satisfying (e, dD. This yields several thousand keywords, which we refine using the LLM to produce a final set of ~00 keywords. See detailed prompts in SG.5.
Iterative NLP-based Refinement (KwSet-Public), As a non-private baseline, we first use multiple NLP techniques (Named Entity Recognition, noun chunk extraction, and RAKE keyword extraction) to extract an initial set of keywords from a standalone (but related) public dataset. We then sequentially update this set using an LLM. In each iteration, we present the LLM with the current set of keywords and each new conversation, prompting it to output words to remove and words to add. This sequential refinement continues until the keyword set stabilizes reaches desired size; see detailed prompt in SG.6.

---

Published as a conference paper at COLM 2025
Combined Public-Private Selection (KwSet-Hybrid), In this hybrid approach, we leverage KwSet-Public while maintaining privacy guarantees. We provide the LLM with KwSet- Public along with several private conversations, asking it to output relevant keywords only when necessary. We then apply partition selection over these selected keywords to ensure DP. We propose this approach to address potential distribution shifts between the public conversations used in constructing KwSet-Public and the private conversations we aim to analyze. This hybrid method allows for adaptation to new topics terms in the private dataset while still benefiting from the high-quality foundation of the public keyword set. See detailed prompt in $G.7.
5 Evaluations
Our evaluation strategy adopts multifaceted approach to assess the privacy-utility tradeoff in our DP pipeline. We compare summaries generated by our private URANIA pipeline against those from the (non-private) SIMPLE-CLIO on identical conversation sets, using the latter as a reasonable proxy for ground truth.
We employ both non-LLM methods (comparing key phrases, n-grams, topics, and em- bedding similarities) and LLM-based evaluations (comparative quality assessments and independent scoring) to comprehensively measure how well our privacy-enhanced sum- maries preserve the essential information captured in the public summaries. This approach allows to quantify both the utility preservation and information retention of our DP pipeline while maintaining stronger formal privacy guarantees than the original framework. Additionally, we evaluate privacy protection through AUC score comparisons between the public and private pipelines, providing a more complete picture of the privacy-utility tradeoff in our system, which will be presented in the later sections (See SF).
5.1 Evaluation Methodology
Our evaluation methodology involves running both the public and private pipelines on the same conversation dataset, generating two sets of summaries: public (non-DP) and private (DP). We then compare these summaries using both automated metrics and LLM-based evaluations.
Automated Evaluation. We employ three automated approaches to measure the similarity between private and public summaries:
1. Lexical Content Analysis:
We extract key phrases, noun chunks, and TF-IDF keywords from both sets of summaries and compute similarity metrics with Jaccard similarity between these feature sets.
N-gram and Topic Analysis: We analyze tokens and 2-grams extracted from both sets of summaries to assess content preservation at different granularities. Additionally, use BERTopic (Grootendorst, 2022) to extract topics from both sets of summaries and measure their overlap.
3. Embedding Space Proximity: For each private summary, we compute its distance to the nearest public summary in the embedding space using the with the all-mpnet-base- embedding. This measures how well the private summaries preserve the semantic content of their public counterparts.
model
LLM-based Evaluation. We complement automated metrics with LLM-based evaluations that assess summary quality. In the prompt, we randomly chose the order of public and private summaries to avoid positional bias.
4. Comparative Ranking: We randomly sample conversations along with their correspond- ing private and public summaries. An LLM evaluator rates which summary is better a scale from 1-5, where indicates the private summary is clearly better and 5 indicates the public summary is clearly better.
7

---

Published as a conference paper at COLM 2025
5. Binary Preference: For sampled conversations, an LLM evaluator makes a binary choice between private and public summaries, selecting which one better summarizes the original conversation.
Tables 1 and 2 present the results of our comprehensive evaluation, comparing the private and public summaries across various metrics and configurations. The results of lexical content, n-gram and topic analysis are presented in Table 1, and the embedding space proximity and LLM-based evaluation results can be found in Table 2.
Finally, propose simple empirical privacy evaluation method using membership inference-style attack to conceptually demonstrate that URANIA is indeed more robust than SIMPLE-CLIO; details in SF.
Experimental Setup & Results
Table 1: Lexical content, n-gram, and topic similarity Between Private and Public Summaries.
Configuration
Key Phrases
Noun Chunks
TF-IDF
Tokens
2-Grams
Topics
Very Low Privacy (ec=10.0, hhstt
0.028
0.026
0.96
0.133
0.100
0.723
Low Privacy (8,=8.0, &hist 4.00)
0.026
0.025
0.96
0.130
0.096
0.461
Medium Privacy (4.=4.0, Ehist 2.00)
0.027
0.023
0.96
0.113
0.077
0.271
High Privacy 22,=,.0, Phist 1.00
0.035
0.021
0.94
0,063
0.033
0.078
KwSet-LLM (Low Privacy)
0.032
0.031
0.98
0.090
0.065
0.589
KwSet-Public (Low Privacy)
0.035
0.021
0.98
0.131
0.083
0.514
KwSet-Hybrid (Low Privacy)
0.030
0.022
0.96
0.145
0.092
0.446
No Privacy (e = oo)
0.024
0.027
0.98
0.139
0.112
0.907
Table 2: Embedding space proximity between and comparative LLM evaluation of private and public summaries.
Configuration
Mean Cosine
Median
Comparative Ranking (1-5, lower is better)
Binary
Preference (% DP preferred)
Similarity
Very Low Privacy (e.=10.0,, 0hist55.0
0.750
0.745
2.56
62
Low Privacy (8,=8.0, 8hist =4.0)
0.749
0.744
2.40
65
Medium Privacy (f. 4.0, #hist
0.748
0.742
2.39
69
Privacy (c. =20,, =10)
0.774
0.773
2.50
64
KwSet-LLM (Low Privacy)
0.747
0.744
2.37
68
KwSet-Public (Low Privacy)
0.735
0.728
2.64
69
KwSet-Hybrid (Low Privacy)
0.739
0.732
2.55
70
No Privacy (e = oo)
0.759
0.754
We analyze the conversations from the popular public datasets LMSYS-1M-Chat (Zheng et al., 2023). Specifically we use gemini-2. 0-flash-001 language model. In constructing KwSet- Public, we use the public WildChat dataset (Zhao et al., 2024), In particular, we set the cluster size privacy parameter ésize 1 for cluster size verification and implement 1-DP keyword set generation.
We evaluate our approach across different privacy parameter settings and keyword set configurations. We consider varying values of & and &istr while keeping the keyword set to be generated using KwSet-TFIDF. We also fix & = 8.0 and éhist 4.0 and evaluate with different choices of keyword sets. Note that URANIA with is still distinct from
8

---

Published as a conference paper at COLM 2025
SIMPLE-CLIO because the former generates cluster summaries via keywords, whereas the latter does so directly from individual summaries.
5.3 Discussion
The evaluation results reveal several key insights about the performance of our framework.
Privacy-Utility Trade-off. As expected, we observe a trade-off between privacy protection and summary quality. Higher privacy guarantees (lower values) generally result in lower similarity to public summaries, particularly evident in the declining topic similarity scores in Table 1, where coverage drops from 0.723 at very low privacy to just 0.078 at high privacy, However, the degradation is not uniform across all metrics, suggesting that some aspects of summary quality are more robust to privacy noise than others.
Impact of Keyword Set Selection. The choice of keyword set significantly influences the quality of private summaries. Table 1 demonstrates that different keyword sets yield varying performance despite identical privacy parameters. For example, KwSet-LLM has better topic coverage compared to KwSet-TFIDF at the same level of privacy. This suggests that carefully curated keyword sets can partially mitigate the utility loss from DP.
Topic Coverage and Preservation. Following CLIO's evaluation approach, we treat public summaries ground truth and measure the percentage of topics successfully captured by our DP method. Our private approach demonstrates reasonable topic coverage at lower privacy levels (0.723 at 10.0), but this metric proves highly sensitive to increased privacy protection, precipitously declining to 0.078 at high privacy (e. = 2.0).
Examining the transition from very low privacy (e. 10.0) to low privacy (Ec 8.0) in Table 1, while the number of private summaries decreases modestly from approximately 3,700 to 3,300 (an 11% reduction), topic coverage drops more dramatically from 0.723 to 0.461 (a 36% reduction). This disproportionate decline suggests that the DP clustering algorithm may systematically exclude certain conversation types even at this early privacy transition, though our current evaluation cannot determine which specific topics are affected. This sharp threshold effect raises important research questions about how different topic types respond to privacy constraints and whether alternative DP approaches might exhibit more gradual degradation.
A primary contributing factor to this performance degradation is our DP-KMEANS imple- mentation. As the privacy budget tightens, the algorithm generates fewer viable cluster centers, with many centers positioned suboptimally relative to actual data points. Conse- quently, some centers fail to attract any conversation assignments, further compromising topic coverage at higher privacy levels.
The clustering performance illustrates this progressive decline quantitatively: the number of final clusters decreases from approximately 3,700 at very low privacy to just 300 at high privacy configurations. While this trend underscores the cascading effect of DP on clustering quality, it also raises an intriguing research question: Can alternative DP-clustering methods or novel implementations mitigate these limitations? Although such performance degrada- tion appears somewhat inherent to DP techniques, which fundamentally trade granularity for privacy protection, exploring alternative algorithmic approaches may reveal strategies to better preserve minority representations while maintaining robust privacy guarantees.
Semantic Preservation. The embedding-based evaluation in Table 2 shows consistently high cosine similarity scores across all configurations 03777, indicating that private summaries generally maintain the semantic essence of their public counterparts, even at higher privacy levels. This is particularly notable because it suggests that overall meaning is preserved even when specific lexical content differs.
LLM Evaluation Insights. Table shows that LLM evaluators sometimes prefer private summaries over public ones, with comparative ranking scores generally below 3 (where 3 would indicate no preference). The binary preference results further confirm this trend, with 62-70% of evaluations favoring the -eggenerteed summaries. This suggests that the

---

Published as a conference paper at COLM 2025
constraints imposed by our DP approach (limiting to predetermined keywords, focusing most frequent themes) can occasionally produce more concise and focused summaries than the unconstrained public approach.
Independent Scoring Results. We have an LLM evaluator independently score private and public summaries
scale from 1-5 (where 1 means very poor and 5 means excellent) based on how well they summarize the original conversations. The private summaries are slightly better than public summaries, but both achieve very low scores on average (<11.4). This suggests that there might be significant room for improvement in the overall quality of conversation summarization, regardless of privacy considerations, and highlights the challenging nature of the summarization task itself.
These findings highlight both the capabilities and limitations of DP text summarization. Our framework demonstrates that it is possible to generate meaningful summaries while provid- ing formal privacy guarantees, but practitioners should carefully consider the privacy-utility trade-off when configuring the system for real-world applications. However, qualitative analysis of specific examples (see ŞD) reveals important limitations in our private summaries that may not be captured by automated evaluation metrics.
Discussion and Future Work
Our work demonstrates that meaningful privacy guarantees can be achieved while maintain- ing useful conversation summarization capabilities. We identify several future directions:
Stronger Privacy Attacks. While we formalized specific empirical privacy evaluation, developing stronger attacks against non-DP summarization systems remains an important direction. The privacy vulnerability of clustering-based algorithms is relatively unexplored compared to other machine learning paradigms. More sophisticated attacks would provide valuable insights into the precise privacy risks of non-DP systems and better quantify the benefits of DP approaches.
Online Learning and Adaptation. An important extension would be adapting our frame- work to online settings where new conversations continuously arise. This presents chal- lenges maintaining privacy guarantees while incorporating new data, evolving keyword sets to capture emerging topics, and efficiently updating cluster structures. This remains an open problem for real-world applications of private conversation summarization.
Utility Improvements. Further improvements could narrow the quality gap between private and non-private summarization approaches. These include exploring alternate privacy mechanisms with better utility-privacy trade-offs, developing more sophisticated keyword selection methods, and refining summarization prompts to generate more informative cluster summaries.
User-level DP. Our current pipeline exclusively addresses DP at the record-level, where privacy guarantees are provided when only single conversation is modified. However, in practical scenarios, individual users typically contribute multiple conversations with large language models (LLMs). This motivates an important open research problem: ex- tending the privacy notion to user-level DP, wherein the privacy guarantees hold when all conversations from single user removed.
Broader Privacy Landscape. While our work focuses on formal DP guarantees for released summaries, we acknowledge that comprehensive privacy protection requires addressing multiple risk vectors. Our approach provides one layer of protection against information leakage through published analyses, but does not address other important concerns such as centralized data collection risks, third-party model processing vulnerabilities, or inference attacks on the underlying conversation data. Future work should explore how formal DP guarantees can be integrated with other privacy-preseringg techniques (such as federated learning, secure multi vartt-patty computation, local DP) to create more comprehensive privacy frameworks for conversation analysis systems.
10

---

Published as a conference paper at COLM 2025
References
Maria-Florina Balcan, Travis Dick, Yingyu Liang, Wenlong Mou, and Hongyang Zhang. Differentially private clustering high-dimensional Euclidean spaces. In ICML, pp. 322-331, 2017.
Avrim Blum, Cynthia Dwork, Frank McSherry, and Kobbi Nissi.. Practical privacy: the sulq framework. In PODS, pp. 128-138, 2005.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In S & pp. 1897-1914, 2022.
Alisa Chang and Pritish Kamath.
Practical differentially private clustering. wtwgl//bboog/raccccall.oo p tober 2021. Google Research Blog, Published October 21, 2021.
https:/ https://research:
Oc-
Alisa Chang, Badih Ghazi, Ravi Kumar, and Pasin Manurangsi. Locally private k-means in
one round. In ICML, pp. 1441-1451, 2021.
ChatGPT. Open AI. URL https::/ccattppt.com
Claude,ai. Anthropic. URL https://claude..ai/
DeepSeek. DeepSeek. URL https::/ww.ddeepsekk.
Damien Desfontaines. A list of real-world uses of differential privacy. https: https://desfontain.
dirbloo/reall-wolldddiiifoe ifr oet. html, 10 2021. Ted is writing things (per- sonal blog).
Damien Desfontaines, James Voss, Bryant Gipson, and Chinmoy Mandayam. Differentially private partition selection. PoPETS, 2022(():339-352, 2022.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam D. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, pp. 265-284, 2006.
Gemini. Google. URL
com/.
Badih Ghazi, Ravi Kumar, and Pasin Manurangsi. Differentially private clustering: Tight approximation ratios. In NeurIPS, 2020.
Arpita Ghosh, Tim Roughgarden, and Mukund Sundararajan. Universally utility-
maximizing privacy mechanisms. SICOMP, 41(6):1673-1693, 2012.
Maarten Grootendorst. KeyBERT: Minimal keyword extraction with BERT,, 2020. URL
https://doi.org// 2448// 4461265.
Maarten Grootendorst. BERTopic: neural topic modeling with class-based TF-IDF proce- dure. arXiv, 2203.05794, 2022.
Anupam Gupta, Katrina Ligett, Frank McSherry, Aaron Roth, and Kunal Talwar. Differen- tially private combinatorial optimization. In SODA, pp. 1106-1125, 2010.
Kunal Handa, Alex Tamkin, Miles McCain, Saffron Huang, Esin Durmus, Sarah Heck, Jared Mueller, Jerry Hong, Stuart Ritchie, Tim Belonax, Kevin K. Troy, Dario Amodei, Jared Kaplan, Jack Clark, and Deep Ganguli. Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations, arXiv, 2503.04761, 2025.
Abiodun M. Ikotun, Absalom E. Ezugwu, Laith Abualigah, Belal Abuhaija, and Heming Jia. K-means clustering algorithms: A comprehensive review, variants analysis, and advances in the era of big data. Inf. Sci., 622:178-210, 2023.
Stuart Lloyd. Least squares quantization in PCM. TOIT, 28(2):129-137, 1982.
11

---

Published as a conference paper at COLM 2025
Reza Yousefi Maragheh, Chenhao Fang, Charan Chand Irugu, Parth Parikh, Jason Cho, Jianpeng Xu, Saranyan Sukumar, Malay Patel, Evren Korpeoglu, Sushant Kumar, et al. LLM-TAKE: theme-aware keyword extraction using large language models. In BigData, 2023.
Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Smooth sensitivity and sampling in private data analysis. In STOC, pp. 75-84, 2007.
Nils Reimers and Iryna Gurevych. Sentence-BERG Sentence embeddings using Siamese BERT-networks. arxiv, 1908. 10084, 2019.
Nils Reimers and Iryna Gurevych. all-mpnet-base-v2:
MPNet-based sentence embedding model, 2022. URL https://huggiingface consennence conteeeeeessssteeeeeeaaeorrreeee
all all-mpnet-base-v2
Alexandre Sablayrolles, Matthijs Douze, Yann Ollivier, Cordelia Schmid, and Hervé Jégou. White-box vs black-box: Bayes optimal strategies for membership inference. In ICML, 2019.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In S & P, 2017.
Latanya Sweeney. k-anonymity: A model for protecting privacy. Int. J. Uncertain. Fuzziness Knowl. Based Syst., 10(5):557-570, 2002.
Alex Tamkin, Miles McCain, Kunal Handa, Esin Durmus, Liane Lovitt, Ankur Rathi, Saffron Huang, Alfred Mountfield, Jerry Hong, Stuart Ritchie, Michael Stern, Brian Clarke, Landon Goldberg, Theodore R. Sumers, Jared Mueller, William McEachen, Wes Mitchell, Shan Carter, Jack Clark, Jared Kaplan, and Deep Ganguli. Clio: Privacy-Preserving Insights into Real-World AT Use. Xin, 2412.13678, 2024.
Eliad Tsfadia, Edith Cohen, Haim Kaplan, Yishay Mansour, and Uri Stemmer. FriendlyCore: Practical differentially private aggregation. In ICML, pp. 21828-21863, 2022.
Salil P. Vadhan. The complexity of differential privacy. In Tutorials on the Foundations of Cryptography, pp. 347-450. Springer International Publishing, 2017.
Yang Wang, Zheyi Sha, Kunhai Lin, Chaobing Feng, Kunhong Zhu, Lipeng Wang, Xuewu Jiao, Fei Huang, Chao Ye, Dengwu He, et al. One-step reach: LLM-based keyword generation for sponsored search advertising. The Web Conference (Companion), 2024.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In CSF, 2018.
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wild- chat: Im chatgpt interaction logs in the wild. ar 2405.01470, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging LLM-as-a-judde with MT-bench and chatbot arena. In NeurIPS, 2023.
12

---

Published as a conference paper at COLM 2025
A Formal Details of DP Tools
In this section we recall basic properties of DP as well as formally describe the tools of private partition and private histogram release as alluded in $2.
Proposition A.1 (Properties of DP).
[Parallel Composition] If M : D - R. satisfies (E, (),P), then for any disjoint collection of subsets O O, C 2 given a dataset D C Q the mechanism that returns (M(DD also satisfies (e,
[Basic Composition] If mechanism M1 with output space Y satisfies (E,, 01) D and the mechanism M2y satisfies (E2, DD) for all E Y. Then the mechanism M that maps x to (y1 Mi(x), ~ M2y (x)) satisfies (E1 + E2 + 62)
[Postprocessing]1 If M with output space satisfies (e, ODDP, then for any function f Z, the mechanism M that maps x + f(M(x)) satisfies (E.
The (truncated) discrete Laplace distribution (E) is the distribution on Z such that
{ Bet --(x for 2 6 [-T, 0
Pr X = x] = X-DLap,
=10
where Per = 11e 20 2141)
1-e
otherwise
The case of r = co is simply the discrete Laplace distribution DLap(e).
A.1 Private Histogram Release
The mechanism PHR (h),.... hn) simply returns a noisy histogram obtained by adding discrete Laplace noise DLap(e/k) to each value of the true histogram.
Algorithm 2 Private Histogram Release PHR (XI,..,,
Parameters: 8 > 0 and N.
Input: / ,y S B with |h1 S k. Output: A histogram h z
ht* + h;;;; for b E B do
h; interpreted as indicator vector in {
Let Co ~ DLap(e/k)
hn + h + Tb
return
A.2 Private Partition Selection
The mechanism (hik,e,shh,..... hn) returns a subset elements for which the noisy count, after adding truncated discrete Laplace noise DLap, (e/k) is more than т. The result for k = 1 was provided by Desfontaines et al. (2022) and the result for general k can be derived using the group privacy property of DP (see Lemma 2.2 Vadhan (2017)).
13

---

Published as a conference paper at COLM 2025
Algorithm 3 Private Partition Selection P (X1,,.., (*..
Parameters: k E N, and d E [0,1]. Input: i,, My C Ω with |4j & k. Output: A set R Ug
é - 8. 8,0 dlk
e-1 d' +28'-11 )
[ log
R 2
for w E U31 do
Cu + (i : h4 w)]
La (E)
+ > 7 then R RU [w}
return
B Description of SIMPLE-CLIO
We formally describe the SIMPLE-CLIO algorithm in Algorithm complementing the de- scription in $3.
Algorithm 4 SIMPLE-CLIO; Non-Differentially Private Text Corpora Summarization
Parameters: k; parameter for KMeans clustering
Input: Input dataset x (x1, ,,, Xn) E X
Output: High Level Summaries
($1,-..) (s)) sk)
S
// Step 1:
Extract Embeddings from Conversations e <- (ExtractEmbedddings(())) i [n]]
// Step 2: Clustering (K-means)
C.C + KMEANS;(e))
centers, C assignments
// Step 3: Summarize each Cluster
< Ø
// Initialize summaries Draw representative samples based on
for j 4 to do
samples, - SampleConversations(;;, C,,);
the center and cluster
$j f LLM-Sunmariz((ampples))
// Generate summary using LLM
$ + s {s
return
The sub-routines used in the above algorithm operate as follows:
ExtractEmbeddings(t) uses LLM to create "summary" for (see prompt in SG.1), and thereafter uses an embedding model that maps the summary to real-valued vector. Specifically, we use all-mpnet -bssevv2 2022).
1all-mpet-bbae--22 embedding model (Reimers Gurevych, 2019;
KMEANS (e) applies the k-means algorithm that partitions the vectors e into k clusters that minimizes the k-means objective.
SampleConversations draws small (e.g., 10) number of random conversations from the specified cluster.
LLM-Summarize uses an LLM to generate a summary associated to the cluster using the sampled conversations from the cluster (see prompt in SG.2).
14

---

Published as a conference paper at COLM 2025
C Keyword Selection Algorithms
We formally describe the methods for extract keywords from documents within each cluster, complementing the description in $4.1. In these methods, we apply the extraction on the concatenation of the various facets of the inputs x,'s. However, for simplicity, we write the pseudocode assuming they are applied on the raw inputs.
TF-IDF-Based Selection with Partition Selection (KwSet-TFIDF), This method proceeds by constructing token frequency matrix TF - where TF; is the number of times the token appears in input xj. We normalize the token frequency matrix such that each column has { norm at most Wmx-
Additionally, we construct a document frequency vector df E ZiV,, where df, is the number of times the token j appears across documents. This privatized to obtain df by adding discrete Laplace noise. And similarly, the number of documents is estimated as A by adding discrete Laplace noise to n.
The inverse document frequency idf is set as log(fi dd). And finally, the TFIDF matrix is constructed by multiplying each column of the TF matrix with the idf vector.
Algorithm Differentially Private TF-IDF Keyword Extraction (KwSet-TFIDF)
Parameters: Maximum keywords per conversation k, maximum document weight Wmax, maximum total keywords K.
Parameters: Privacy parameters fuel 0, sel C [0, 1]. Input: Input dataset x (x1, Xy, structured facets
Output: Differentially private keyword set K.
// Step Build TF matrix and apply L1 clipping
TF, V + CountVectorizer(x))
// Build term frequency matrix and token list
for < to x do
11,1 > Wmax then Wmas
TF, - TF. |[TF
// Clip document weights
// Step 3: Compute noisy document frequencies
df E TF, for j [V];
// Compute document frequencies Noise scale
Finl
Wmas +1
df + max DLa)lace(d).1);
// Add DP noise Noi sy document count
+ DLaplace(r);
// Step 4: Compute DP TF-IDF and extract keywords
idF + log(A/df);
// Compute DP IDF // Compute DP TF-IDF // Extract top k keywords per conversation // Select final keywords with DP
TFIDF TF idFDP,
top tokens from TF₁ for each [n]
// Step 5: Apply DP selection mechanism
X. + PPSKAmA ha).
// Step 6: Limit to maximum keywords (optional)
if [/C] K then
+ (XC, by original counts)
// Step 7: LLM refinement
CC LLMRefineKeywords(C..K
// See prompt in $6.5
return
15

---

Published as a conference paper at COLM 2025
Algorithm 6 DP Selection from Facet Keywords with LLM Refinement (KwSet-LLM)
Parameters: Maximum keywords per conversation m, maximum total keywords Privacy parameters Exe > 0, geel (0,1). Input: Input dataset x = (x1, 22,,,,,, E X
Parameters:
Output: Differentially private keyword set Keefineed
// Step 1 Extract set of at most keywords h;; for each
for = I,..., do
GetKeywords(x:))
// See prompt template 56.5,
Step 2: Apply DP selection mechanism
X - Paacc (k))
// Step 3: LLM refinement
while |C] K do
refined +- LLLMQuery/efineProo(((()))
// refinement (see $G.5)
return raffmne
Algorithm/ 7 LLM-guided Sequential Keyword Refinement (KwSet-Public)
Initial batch size no, sequential batch size b, maximum initial keywords Kg- Input: Input dataset = (x1, x2...., Xn) structured facets
Parameters:
Output:
Refined keyword ICPu
// Step 1: Extract text and split dataset
Xinit f- x(1 nt], Xseq <- x(1ig + :
Split into initial and sequential sets
// Step 2: Extract and refine initial keywords
Krm UxEx (NER(x) U NounChuuns(()) U RAAE(xx))
// Multi-method extraction // Remove redundancy, merge concepts (see $G.
Krub + LLMRefine( Kymw);
K <- RandomSample(AKru min(//Cuu), Ko));
// Limit initial set
// Step 3: Sequential updating with LLM feedback
<- to [|xsey| [b]] do
batch_text + (Xseq 1)b :bbb);
// Merge batch conversations // Get update suggestions (see $G. // Apply updates
aemmove,appedd <- LLMUpdate(ACWW) (eathethtttt)))) batch ICPud + (ACPuò remove) U append;
return ACPub
16

---

Published as a conference paper at COLM 2025
Algorithm 8 Combined Public-Private DP Keyword Selection (KwSet-Hybrid)
Parameters: Maximum number of keywords k per conversation.
Parameters: Privacy parameters Ebyb >> 0, Ohyb E (0,
Input: Public keyword set Kmmb, input dataset X2,,X, Xn)
Output: Hybrid DP keyword set Khhy
// Step 1: LLM-guided candidate extraction in batches
for = 1 to n do
+ G x
Using prompt; see $6.7
// Step 2: Partition selection to select keywords corresponding to private conversations pprivate (h).,n())....
return Kprionte U Kppb
17

---

Published as a conference paper at COLM 2025
D Summary Comparison Examples
To better understand the trade-offs between public and private summaries,, we present several illustrative examples that highlight the key differences in specificity and contextual relevance.
Table Comparison of Public vs Private Summary Examples
CONVERSATION:
The user is asking how to extract the last character file in BASH and convert it to hexadecimal.
SUMMARIES:
Public: Scripting and Command-Line Task Automation Requests
Private: Software Development, Version Control, and Educational Resources
CONVERSATION:
The assistant explains the key differences between reinforcement learning and unsuper- vised learning, focusing on feedback and goals.
SUMMARIES:
Public: Deep Learning, Training, and Applications Explanation
Private:
Al Model Training, Deployment, and Management Considerations
CONVERSATION:
The user is asking if the assistant speaks Russian and the assistant confirms and offers help.
SUMMARIES:
Public: Russian Language Support and Communication
Private:
Al-Powered Applications and Open Source Technology
As these examples demonstrate, public summaries tend to be more specific and directly relevant to the actual conversation content, while private summaries are often broader and sometimes miss key contextual details. This limitation stems from the privacy constraints that prevent fine-grained keyword extraction- for instance, in the language example, spe- cific terms like "Russian" " may not appear in our predetermined keyword sets, leading to generic categorizations that fail to capture the conversation's s essence.
These patterns suggest that while LLM evaluators may prefer the broader categorizations produced by private pipeline for their apparent comprehensiveness, human evaluators might better discern the loss of specificity and contextual relevance. The trade-offs observed in these examples highlight the necessity of incorporating human-based evaluation in future work to provide a more nuanced assessment summary quality, as human judgment may be more sensitive to the subtle but important differences in semantic accuracy and practical utility that automated metrics might overlook.
18

---

Published as a conference paper at COLM 2025
E Additional Related Work
Analysis of LLM conversations Our work builds on real-world conversations with LLM chatbots from recent open-source datasets (Zheng al., 2023; Zhao et al., 2024). Prior work has analyzed such conversations to extract structure and insight. For instance, Zhao et al. (2024) define a set of common conversation categories and prompt LLMs to classify conversations accordingly. In contrast, Zheng et al. (2023) and Tamkin et al. (2024) adopt more flexible approaches that do not rely on predefined categories. Zheng et al. (2023) cluster conversations in the embedding space and then use LLMs to summarize each cluster's main theme. Tamkin et al. (2024) build on this by first summarizing individual conversations into concise summaries and then applying hierarchical clustering. While these works provide valuable frameworks for understanding LLM behavior, none offer formal privacy guarantees. Our work addresses this gap by introducing the first DP conversation summarization pipeline.
Clustering. Both Tamkin et al. (2024) and our work employ clustering step to group conversations with similar embeddings together. We use k-means, which is one of the most popular formulations of clustering; see e.g. Ikotun et al. (2023) for comprehensive survey. For non-DP k-means, we use the classic algorithm of Lloyd (1982). For DP k-means, several algorithms have been proposed, starting with Blum et al. (2005) who devised DP version of Lloyd's algorithm. This was followed up by series of works improving different algorithmic guarantees (e.g. Nissim et al. (2007); Gupta al. (2010); Balcan al. (2017); Ghazi et al. (2020)), culminating in the algorithm of Chang et al. (2021) which not only gives a nearly optimal (theoretical) approximation ratio but also practical. We ended up using the open-source library Chang & Kamnth (2021) based on this paper. We remark that there are also more recent works which provide practical DP clustering algorithms, e.g., Tsfadia et al. (2022); nevertheless, we are unaware of any open-source library based on these papers.
Membership inference attacks (MIA) Our empirical privacy evaluation ($F) is in spirit similar to MIA, which aims to infer whether specific data points were included in the training data of a given model or algorithm, based on its outputs. While various MIA techniques have been proposed, they primarily target machine learning models (Shokri et al., 2017; Yeom et al., 2018; Sablayrolies et al., 2019; Carlini et al., 2022). In this work, we design an empirical privacy evaluation method tailored to attack the LLM conversation summarization pipeline.
Keyword generation with LLMs Our keyword set generation method ($4.1) is related to recent work on extracting keywords from text corpora using LLMs (Grootendorst, 2020; Maragheh al., 2023; Wang al., 2024). These approaches typically fine-tune or prompt an LLM to extract keywords. However, such pipelines are not applicable in a DP setting, as documents may contain unique keywords that could leak sensitive information. In this work, we propose keyword generation methods that only output keywords shared across multiple conversations, allowing us provide formal DP guarantees.
19

---

Published as a conference paper at COLM 2025
F Empirical Privacy Leakage Evaluation
To empirically evaluate privacy protection, we conducted simple experiment to compare the privacy leakage of our private and the public pipelines. Specifically, we created synthetic dataset of 100 conversations: 1 sensitive conversation on health/ medical topics and 99 non-sensitive conversations on general topics (food, travel, homework help and health). We ran both pipelines on this dataset, then quantify the privacy leakage by measuring the maximum embedding similarity between the sensitive conversation and the generated summaries. We consider simple thresholding-based detector of the sensitive conversation, and measure the AUC under different thresholds under 200 runs where half of the runs include the sensitive conversation.
Public Pipeline
Trua Pulbbe 2ate
1oC tams $.50
Paic Poditivo Rato
Fabm Rana
(a) Empirical Privacy Leakage Measurement against DP pipeline 21, AUC 0.53)
(b) Empirical Privacy Leakage Measurement against public pipeline (AUC 0.58)
Figure 2: ROC curves for Empirical Privacy Leakage. The DP pipeline (a) shows perfor- mance equivalent to random guessing (AUC 0.53), while the public pipeline (b) is more vulnerable with an AUC of 0.58.
Figure shows the results. Our DP approach achieved an AUC of only 0.53, effectively equivalent to random guessing (0.5), demonstrating strong privacy protection. In contrast, the non-private pipeline showed greater vulnerability with an AUC of 0.58, indicating that embedding similarity can leak information of the sensitive conversations when privacy mechanisms are absent.
It is worth noting that our experiment used casually generated synthetic conversations rather than carefully crafted adversarial examples. more sophisticated experiment might potentially achieve better performance against either pipeline. However, even with this simple approach, we can observe a clear difference in privacy protection between the DP and non-DP methods.
These results provide empirical evidence that our DP approach offers practical protection against such privacy leakage while maintaining useful summarization capabilities, even when using a relatively high privacy budget (E = 21) on relatively small dataset (of no more than 100 conversations).
20

---

Published as a conference paper at COLM 2025
G
LLM Prompts
We provide some example prompts we used in the pipeline.
G.1 Extracting Embeddings from Records
Below, we provide the prompt used in the ExtractEmbeddings method in Algorithm 4 and Algorithm 1. The input record is substituted in place of (x) in the prompt. Note that apply the embedding function specifically to the 'Summary' portion of the LLM response obtained using this prompt, rather than to the entire response.
Prompt for ExtractEmbeddings((1
You advanced assistant that processes conversations and extracts meaningful facets for downstream embedding clustering tasks. For the conversation provided, return a structured JSON object with the following:
Topics: The overarching topics discussed in the conversation.
Subtopics: Specific details points under each topic.
Intent: The main intent of the conversation (e.g., seeking help, sharing
opinion).
Entities: Important names, organizations, or proper nouns discussed.
Keywords: The most important words/phrases that could help differentiate the
conversation.
Summary: concise, one-sentence description summarizing the conversation, up 20 words.
Ignore the conversation if it is not in English. Output the result valid JSON object with extra text.
### Example:
User: I'm having a hard time preparing for job interviews. How do I answer behavioral questions? I also get nervous. Any tips how stay calm during the process?
Expected JSON Output:
{
'Topics': ['Job Interviews', 'Behavioral Questions', 'Stress Management 'Subtopics': ['Handling difficult questions', 'Managing anxiety during interviews', 'Tips for preparation'], 'Intent': 'Seeking advice',
'Entities': [ ['job interview', "behavioral questions'],
'Keywords': job preparation', 'nerves', 'behavioral interview tips'], 'Summary': 'The user seeking advice on preparing for behavioral questions managing stress during job interviews.',
### Now, analyze the following conversation and generate the output in the format:
User: (x)
Return the structured output in JSON format.
21

---

Published as a conference paper at COLM 2025
G.2 LLM-Summarize in SIMPLE-CLIO (Algorithm 4)
We provide the prompt used for LLM-Summarize method in SIMPLE-CLIO. In both CLIO and our proposed pipeline implementations, select contrastive summaries from the nearest points located outside the designated cluster. For the sake of algorithmic clarity, we omitted this nuanced selection strategy from Algorithm maintain simplified representation of the core approach.
LLM-Summarize
You are tasked with summarizing a group of related statements into a short, precise, and accurate description and name. Your goal is to create concise summary that captures the essence of these statements and distinguishes them from other similar groups of statements.
Summarize all the statements into clear, precise, two-sentence description in the past tense. Your summary should be specific to this group and distinguish it from of the other groups. After creating the summary, generate the contrastive group statements. This name should at most words long short name (perhaps less) and be specific but also reflective of most of the statements (rather than reflecting only one or two).
Present your output the following format: <summary> [Insert your two-sentence summary here] </summary> <name> [Insert your generated short name here] </name>
Below the related statements:
<answers>
(summary bathinncluster_sumariiie
(summary 2 in aithin.cluster
wtwpinccssteessummrriiie.. wummaries.. i
</answers>
For context, here are statements from nearby groups that are NOT part of the group you're re summarizing:
<contrastive.answers>
(summary contrastive.summariees
(summary 2 contrastive cerettivvesummaiiie i
</contrastive answers>
not elaborate beyond what you say the tags. Remember analyze both the statements and the contrastive statements carefully to ensure your summary and name accurately represent the specific group while distinguishing it from others.
22

---

Published as a conference paper at COLM 2025
G.3
LLMselectKeywonds
The following prompts are used to extract keywords given the conversation and the KwSet.
LLMSelectKeywords
Below is summary of conversation. Based on this summary, select the most relevant {m) keywords from the given set of keywords.
Summary: {summary)
Available Keywords: (keyword, ), {keyword2).
Select up to [m} most relevant keywords from the set return them as a Python list (e.g., [''keword1', 'keyword2', ....]
If there is no relevant keyword, return the Python list ['NA']
G.4 LLM-Summarize in Algorithm
We modified the prompt for Algorithm 4, and got the following prompt for LLM-Summarize in Algorithm 1.
LLM--Smmarize
You are a language model tasked with summarizing group of related keywords example texts into a concise topic description.
Your goal is to generate: *xconcise topic keywords and examples.
(:10 words) that best describes the theme of these
2. A **brief, 2-sentence description** explaining what this cluster is about.
Please return your response in the following format:
<topic> [Insert topic name here] </topic> <description> [Insert brief summary here] </description>
Below the keywords associated with the cluster: <keywords>
[keyword1], [keyword2), </keywords>
Ensure the topic is **specifick*, descriptive, and meaningful.
23

---

Published as a conference paper at COLM 2025
G.5 Prompts for KwSet-LLM and KwSet-TFIDF
We used LLM choose keywords from conversations in the process of generating KwSet. We provide the prompt for generating KwSet-LLM for example.
KwSet-LLM Generation (Before Refinement)
You are knowledgeable assistant tasked with generating set of comprehensive keywords that cover the topics, intents, and entities present in a dataset of human conversations.
For the sunmary provided, generate list **3-5** keywords that reflect:
core topic(s) the conversation
Key entities or names mentioned
Keywords useful for identifying conversation categories or topics
Keep keywords concise and relevant.
Summary: dioneessttio_suumas
Use this JSON schema:
Response (("keywords": list[str]]]
Return only the JSON object without any explanations, decorations, or code blocks. Only the JSON should be returned.
As keyword set of smaller size leads to smaller utility loss due to privacy, we use an LLM to refine the keyword set (for both KwSet-TFIDF and KwSet-LLM) using the following prompt template.
Refine KwSet
You are an expert in topic summarization. We have extracted keywords from a large collection of conversations. These keywords used for summarizing the main topics the conversations based sentence statistics. Your task refine this list and select the most relevant (num} keywords that best capture the core topics.
Here is the extracted keyword list:
{json.dumps (keywords, indent=2)}
Please return exact [num} of the most relevant keywords in JSON format using the following schema:
"'json
{(
"keywords": ["keyword1", "keyword2", numm)"
})
Do not exceed {num} keywords. If necessary, prioritize the most commonly occurring, diverse, and representative words.
24

---

Published as a conference paper at COLM 2025
G.6 Prompt for KwSet-Public
Refine KwSet-Puublc
You are refining of extracted key phrases for topic summarization.
Below is a list of candidate keywords extracted from conversations: [' j-joonalll.cndddall keywordss
Your task:
**Remove meaningless words** such as random numbers, dates, single letters, generic words (e.g., "thing", "something", "stuff"). **Delete overly specific entity names** (e.g., "John Doe", "xyz123", "April 5, 2023", "ID number").
**Remove redundant words** that mean the same thing.
**Merge similar concepts** (e.g., "AI fairness" and "bias in AI" should be unified). **Keep only the most informative, meaningful terms** that are useful for summar izing topics.
**Avoid vague or overly broad words** like "technology", "question", "people".
**Ensure the final keywords are suitable for clustering and summarizationte.
Return only the sxrefined keyword set** comma-separated list.
Update KwSet-Public by adding and removing keywords
You expert in topic summarization and keyword analysis. The current public keyword set contains [Len(existinngkkewwrrss)) words: [(', jooin(existing..eeorrs])))
Here is batch of private conversations: "(batch.text]"
Your task:
Compare the batch of private conversations with the current keyword set. Identify keywords in the current set that redundant irrelevant. Identify new, meaningful keywords present in the private conversations that not in the current set.
Output ONLY complete, valid JSON object (with no extra text) with the following structure:
{{
'words.to_remove" [list of words to remove], 'words.to_append" [list of words to add]
Ensure your output starts with 'f' and ends with 4 (i.e., the JSON object must be complete, including the closing bracket). Keep your answer as concise as possible.
25

---

Published as a conference paper at COLM 2025
G.7 Prompt for KwSet-Hybrid
Refine KwSet-Hybrid
You are an expert topic analysis and keyword extraction.
The current public keyword set contains (len(i keywords)) words: [(', ".join(public keywords))]
Here is batch of private conversations: '/batch.text/'
Your task:
Analyze the above conversations identify any new, meaningful keywords that are not already present in the public set.
Omit any non-English conversations keywords.
Return ONLY concise, comma-separated 1ist of candidate keywords with additiona explanation of at most words.
26

---

Published as a conference paper at COLM 2025
G.8 Prompt for Evaluation
As discussed before, we do some LLM-based evaluations. Now we provide the prompts for the binary preference.
Prompt for Binary Preference
You expert evaluator of text summarization systems. You will be given original text and two different summaries of that text. Your task is to evaluate which summary better captures the key themes content of the original text.
Original Text: {text)
Summary A:
{summary a}
Summary B:
(summary.J) .b}
Please evaluate which summary better captures the key themes and content of the original text. Consider factors such Accuracy of the information
Coverage important topics
Clarity and coherence
Relevance to the original text
Provide your reasoning, then end with a clear choice the format: <choice><</Choice> where is either "A" for the first summary or "B" for the second summary.
27

---

Published as a conference paper at COLM 2025
H
Hierarchy and Visualization
H.1 Hierarchical Organization
Table 4: Examples of Summaries and Cluster Levels
SUMMARY:
The conversation starts with greetings in Portuguese, inquiring about the other person's well-being.
CLUSTERING:
Base:
Al Travel Assistant for Multilingual and Localized Recommendations
Top: Al Assistant Capabilitiies, Performance, and Applications
SUMMARY:
The user requested Python program to create SQLite database table named 'legacy' with specified fields.
CLUSTERING:
Base: Python Data Manipulation with Pandas and SQL
Top: Diverse Al Applications, Analysis, and Performance
SUMMARY:
The Al introduces itself, clarifies its capabilities, and offers assistance to the user.
CLUSTERING:
Base: Al Assistant Initial Interaction and Prompting
Top: Diverse AI Applications, Analysis, and Performance
SUMMARY:
The assistant provides detailed introduction of Tilley Chemical Co., Inc., including its history, services, and sustainability efforts.
CLUSTERING:
Base: Chemical Company Profile: Manufacturing, Performance, and Optimization
Top: Broad AI Applications, Ethics, and General Knowledge
Following CLIO'S approach (Tamkin et al., 2024), we implement hierarchical organization of summaries to improve navigation and comprehension of large conversation datasets. Our process creates two-level hierarchy with high-level topic clusters containing related lower-level summaries:
1. Low-level Summary Generation: We first generate approximately 4,000 low-level sum- maries using our DP pipeline.
2. Embedding and Clustering: These summaries are converted to embeddings and clus- tered using k-means with ~ 70 to identify broader thematic groups.
3. High-level Naming: For each of the 70 high-level clusters, we prompt Gemini suggest descriptive names based on the contained summaries.
Deduplication and Refinement: We use Gemini to deduplicate and refine these sug- gested names, eliminating overlaps and ensuring distinctivened Low-level Assignment: We represent each high-level name as a center in the embedding space and assign each low-level cluster to its nearest high-level name.
Final Renaming: Based on the final assignment of low-level clusters, we rename each high-level cluster to better represent its contents.
This hierarchical organization allows users to navigate from broad topics to specific conver- sation summaries, making the system more useful for exploring large conversation datasets
28

---

Published as a conference paper at COLM 2025
while maintaining DP guarantees. Some examples of the summary facet, base, and top cluster are demonstrated in Table
29


Reviews:
[{"review_id": "AVGzBAZ45f", "reviewer": "Reviewer_yuSm", "summary": "This paper introduces URANIA, a novel framework for generating insights about LLM chatbot interactions while providing formal differential privacy (DP) guarantees. The authors present a well-structured approach that builds upon previous work (CLIO by Tamkin et al., 2024) but addresses a critical limitation by incorporating rigorous privacy protections.\n\nThe quality of the technical work is good. The authors formally define the text summarization problem within a DP framework, develop appropriate algorithms that leverage existing DP tools (clustering, partition selection, and histogram release), and provide mathematical proofs for their privacy guarantees. The experimental evaluation is comprehensive, comparing their approach against a simplified version of CLIO across multiple metrics.\n\nThe paper is generally well-written and clear. The problem definition, technical approach, and experimental results are presented in a logical flow. The figures, especially Figure 1, effectively illustrate the proposed method's components. However, some sections could benefit from additional examples to clarify complex concepts.\n\nIn terms of originality, URANIA represents an advancement over existing approaches to LLM usage analysis. While the individual DP components used are not novel, their combination and application to text summarization in this context is innovative. The different keyword extraction methods (KwSet-TFIDF, KwSet-LLM, etc.) show thoughtful consideration of various privacy-utility tradeoffs.\n\nAs LLMs become increasingly prevalent in daily life, understanding how they are used while protecting user privacy is crucial. The empirical privacy evaluation demonstrating URANIA's robustness compared to non-private alternatives provides compelling evidence of its practical value.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. **Novel Privacy-Preserving Framework**: URANIA introduces the a framework for LLM conversation summarization with formal end-to-end differential privacy guarantees. This addresses a significant gap in existing approaches like CLIO.\n\n2. **Theoretical Foundations**: The paper provides solid theoretical foundations, including formal problem statements, privacy proofs, and a principled evaluation methodology that advances the field.\n\n3. **Practical Utility**: Despite the inherent tradeoffs between privacy and utility, the authors demonstrate that URANIA can maintain reasonable utility across various privacy budget configurations, making it practical for real-world applications.\n\n4. **Comprehensive Evaluation**: The evaluation methodology is thorough, using both automated metrics and LLM-based assessments to provide a multifaceted view of the system's performance. The empirical privacy evaluation effectively demonstrates the benefits of DP protections.\n\n5. **Important Application Domain**: As LLMs become ubiquitous tools used by millions of people, understanding usage patterns while protecting privacy is increasingly important. This work addresses a meaningful problem with significant societal implications.", "reasons_to_reject": "1. **Simplified CLIO Implementation**: Since the authors didn't have access to the original CLIO implementation, they created a simplified version for comparison. This raises questions about how URANIA would compare against the full CLIO system with all its features. A more direct comparison with the original system would strengthen the paper.\n\n2. **Parameter Sensitivity**: While the paper evaluates URANIA across different privacy parameter settings, a more comprehensive analysis of how specific parameters affect various aspects of performance would be valuable. For instance, how do different cluster size thresholds ($\\tau$) impact topic coverage and privacy?\n\n3. **Keyword Set Quality**: The keyword set selection appears crucial to URANIA's performance, yet the paper doesn't provide detailed examples of the generated keyword sets or thoroughly analyze how they differ. More examples and analysis here would help readers understand the tradeoffs between different keyword generation approaches.\n\n4. **Scalability and Computational Efficiency**: The paper doesn't adequately address the computational efficiency of URANIA compared to non-private alternatives. As the dataset size grows, how does the performance scale? This is particularly relevant for the DP-K-Means algorithm, which is known to be more computationally intensive.\n\n5. **User-Level DP**: While the authors acknowledge this as future work, the current record-level DP guarantees may be insufficient in practice since individual users typically contribute multiple conversations. A more detailed discussion of the challenges in extending to user-level DP would strengthen the paper.", "questions_to_authors": "1. It would be helpful to include concrete examples of the generated summaries from both URANIA and SIMPLE-CLIO for the same input conversations. This would give readers a better intuition about the quality difference.\n\n2. The authors might consider discussing how their approach could be adapted to handle multimodal conversations that include images, audio, or other media types.\n\n3. The empirical privacy evaluation in Section E is interesting but somewhat limited. Expanding this with more sophisticated attacks would strengthen the privacy analysis.\n\n4. Table 2 mentioned in the text appears to be missing from the main paper (it's referred to on page 7 but appears in Appendix C). Important tables should be included in the main text.\n\n5. The discussion about the degradation in topic coverage at higher privacy levels (lines 309-330) is insightful. The authors might consider expanding on potential solutions to this problem, perhaps by exploring alternative DP clustering approaches.", "extra_scores": {}}, {"review_id": "YdfwkrBN1a", "reviewer": "Reviewer_qwBB", "summary": "This paper presents URANIA, a framework for generating insights about LLM chatbot conversations with formal differential privacy (DP) guarantees. This work is closely related to Clio, a recent system from Anthropic aiming to generate insights into in-the-wild LLM use while preserving user privacy. Clio doesn't offer formal privacy guarantees and hence this work attempts to bridge this gap. URANIA uses a pipeline that chains together multiple DP algorithms, starting with DP clustering, followed by DP keyword extraction, and finally synthesize summaries of the identified keywords. The evaluation compares URANIA with a simplified version of a Clio-inspired pipeline. Privacy-utility trade-offs were observed, especially in topic coverage.\n\nI overall enjoyed reading this paper and consider it to make a novel and timely contribution. The quality of the work is solid. My main reservation is regarding whether a formal guarantee is really a useful goal to pursue for this problem, as I haven't seen a compelling threat that requires it to be addressed. That said, I think it still deserves further research, and this paper offers a viable solution and valuable insights. Hence, I recommend acceptance.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- It's exploring an intriguing problem of AI for privacy, namely designing an LLM-powered method to gain insights about AI use.\n- Offering formal privacy gurantee is a significant improvement compared to the prior work (Clio)\n- The paper is well-written. The framework is clearly explained and holds the promises of formal privacy guarantees. The utility evaluation show promising results.", "reasons_to_reject": "- In the discussion, the drastically reduced topic coverage from 0.723 at low privacy to 0.078 at high privacy was interpreted as \"the rare or niche topics are progressively marginalized.\" However, there is no evidence provided to support that the reduced coverage is related to rare or niche topics rather than missing major topics. Further evidence and justification are needed, or the conclusion should be adjusted.\n- This paper seems to focus solely on the privacy leakage risks caused by releasing the clustering results, which is a fine goal to pursue for a paper. However, at a high level, I'm not convinced that a formal privacy guarantee is addressing the most important privacy risks (e.g., a centralized dataset is still required, and it uses a Gemini model to process data, which involves raw data leakage). The empirical privacy evaluation in Appendix E feels somewhat contrived and doesn't demonstrate a significant privacy issue that can be addressed with the private pipeline versus the \"public\" one.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vvCAAXK2nK", "reviewer": "Reviewer_hkoW", "summary": "This paper presents URANIA, a framework for extracting insights from LLM chatbot interactions with formal differential privacy (DP) guarantees. It uses DP clustering, keyword extraction (frequency, TF-IDF, LLM-guided), and other DP tools for end-to-end privacy. URANIA is evaluated against a non-private baseline, to assess utility and privacy robustness. The paper addresses an important problem with a sound DP-based framework. The end-to-end DP application to this specific problem, including DP keyword set generation, is novel.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper employs formal DP for a deep, insightful, and impactful solution, and offers offers rigorous privacy, unlike heuristic methods. \n\n* It addresses the critical and growing need for privacy-preserving analytics in AI.\n\n* It also provides a reasonable empirical study of the privacy-utility trade-off, despite known challenges in this area.", "reasons_to_reject": "* The multi-stage pipeline and keyword dependency could be challenging to optimize in practice. \n\n* The proposed approach offers per-conversation privacy, while user-level DP is also important. I'd appreciate more upfront discussion on this and the underlying assumption (correlation between conversations, etc)\n\n* Human evaluation, even small scale, on the summarization results could supplement the LLM-based evaluation.\n\n* The writing could be improved a bit, e.g., I'd try to combine Table 1 and Table 2 results, and link the metrics with the descriptions in line 264-274 better.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers are agreed that this paper is ready for publication, addresses an important and timely topic, and makes a useful contribution. I also agree that this paper addresses a very important topic (how to analyze chatbot conversations while preserving privacy) and provides practical methods. The reviewers provide some concrete suggestions for improvement to the camera-ready version of the paper, including additional examples (e.g. keywords) and discussion/framing, that I hope the authors will incorporate.
Decision: Accept