Inspecting the Representation Manifold of Differentially-Private Text (2025)

Abstract: Differential Privacy (DP) for text has recently taken the form of text paraphrasing using language models and temperature sampling to better balance privacy and utility. However, the geometric distortion of DP regarding the structure and complexity in the representation space remains unexplored. By estimating the intrinsic dimension of paraphrased text across varying privacy budgets, we find that word-level methods severely raise the representation manifold, while sentence-level methods produce paraphrases whose manifolds are topologically more consistent with human-written paraphrases. Among sentence-level methods, masked paraphrasing, compared to causal paraphrasing, demonstrates superior preservation of structural complexity, suggesting that autoregressive generation propagates distortions from unnatural word choices that cascade and inflate the representation space.

## introduction language models (lms) @xcite are trained on extensive corpora of text containing sensitive information. several studies demonstrated that sensitive information can be extracted from lms @xcite @xcite , raising significant privacy concerns and prompting the integration of privacy mechanisms. to protect against unintended disclosure of information, differential privacy (dp) @xcite has been tailored to raw text @xcite . through a randomized mechanism, dp formalizes privacy through a notion of indistinguishability, ensuring that texts remain statistically unaffected by the addition or removal of individual samples in the text corpus. while early randomized mechanisms exploit the distances between words in the embedding space @xcite to replace words with a noisy approximation of their nearest neighbor, grammatical constraints associated with word-level privatization @xcite has led to a shift towards paraphrasing text at sentence-level by leveraging lms @xcite @xcite . contribution. we inspect the representation geometry of text paraphrased under the privacy constraints of dp, accounting for different levels of privacy. @xcite discovered that highdimensional signals reside on low-dimensional manifolds, a property that holds across neural representations @xcite . building on intrinsic dimensionality (id), we estimate the id of texts and interpret id shifts as a proxy for distortions on their structure and complexity. specifically, we compare differentially-private transformations operating on word-level and sentence-level. we find that word-level dp deviates the most from human-authored paraphrases, significantly altering the underlying representation space. concerning sentence-level dp, we argue that bidirectional paraphrasing based on masked substitution mitigates cascading errors that arise in sequential generation. ## background we briefly provide the necessary foundations for differential privacy and intrinsic dimensionality. ## methodology we aim to investigate how privacy-preserving transformations alter the geometry of paraphrases relative to those generated without privacy guarantees. for our experiments, we utilize mrpc @xcite , a dataset containing sentence pairs labeled for semantic equivalence. we selected sentence pairs that provide a reference and paraphrase to ensure a controlled basis for assessing geometric distortions in representation subspaces. ## findings figure 1 presents the deviation in the number of id as a function of the privacy budget. to establish a lower bound for id shifts, we measure the id difference between reference sentences and their humanauthored paraphrases from mrpc. this yields an id shift of approximately 0.12, indicating that naturally occurring paraphrasing introduces only minimal geometric distortions in the representation space. any privacy-preserving transformation that deviates strongly from this baseline alters the structure and complexity of text representations beyond natural variation, potentially affecting readability. word-level perturbation. since madlib is applied at word-level, its randomized mechanism perturbs words independently, disregarding sentence structure and grammatical coherence. this results in fragmented and disorganized text, a phe- mrpc showing a sentence and its human-authored paraphrase. note that differentially-private paraphrases at word-level are obtained using a privacy budget of ε = 25, whereas differentially-private paraphrases at sentence-level are obtained using a privacy budget of ε = 100. ## conclusion we analyze the transformative effects of applying dp to text, focusing on how privacy constraints induce geometric distortions in the representation space. by leveraging the id as a measure of structural complexity, we assess the extent to which prominent dp mechanisms alter latent subspaces and reshape linguistic representations. our find-ings reveal that word-level dp introduces severe id shifts, leading to drastically inflated representation manifolds. for sentence-level dp, we observe distinct differences between their representation geometry, depending on how words are substituted and whether errors from suboptimal word choices accumulate and propagate throughout a sentence. limitations. a limitation of our inspection is that id estimation, while a powerful tool for inspecting representation geometry of text, does not directly capture linguistic quality. although id shifts provide evidence of geometric distortions, connecting these distortions to measures of fluency @xcite and adequacy @xcite would complement our understanding of alterations induced by dp rewriting.