STEALTH: Secure Transformer for Encrypted Alignment of Latent Text Embeddings via Semantic Isomorphism Enforcement (SIE) Loss Function
Authors: Nafew Azim
Nabeel Mohammed
Venue: Accepted by TMLR ()

Abstract: The pervasive use of large language models (LLMs) on sensitive data presents a critical privacy challenge, as traditional encryption renders data unusable for inference. We introduce STEALTH, a 120M secure transformer framework designed to process encrypted text while preserving its semantic utility under an authorized-key threat model (no decryption or side-channel access). The core innovation of STEALTH is the Semantic Isomorphism Enforcement (SIE) loss function, a loss that trains the model to learn a topology-preserving mapping between encrypted text embeddings and their original plaintext latent space. This encourages preservation of semantic relationships and topological structure in the encrypted domain. Using retrieval-based reconstruction from a domain-aligned plaintext corpus, STEALTH achieves near-perfect semantic retrieval (BLEU score of 1.0 under full-corpus coverage in our experiments) and enables accurate privacy-preserving clustering on encrypted embeddings. We evaluate STEALTH across 44 datasets spanning general language understanding, healthcare, finance, legal, e-commerce, programming, content analysis, reading comprehension, and corporate communication domains with 16 encryption schemes (704 experimental conditions), establishing a comprehensive benchmark for 
privacy-preserving NLP on encrypted text. Performance depends on domain alignment between encrypted inputs and the indexed plaintext corpus. Our results demonstrate that, with well-aligned domain indexes and retrieval support, models can perform effective NLP on encrypted data without direct decryption.

Published in Transactions on Machine Learning Research (February/022g
STEALTH:
Secure Transformer for Encrypted Alignment of Latent Text Embeddings via Semantic Isomorphism Enforcement (SIE) Loss Function
Nafew Azim
nafew. sitev.ccimm@nortssouutte edu
Deportment of Electricol and Computer Engineering North South University, Bangladesh
Nabeel Mohammed
Department of Electricnl and Computer Engineering North South University, Bangladesh
Reviewed on OpenReview: https: oppere/v/perrevieuu. net/ formm? 73PV17AVVN
Abstract
The pervasive use of large language models (LLMs) sensitive data presents critical privacy challenge, traditional encryption renders data unusable for inference. We introduce STEALTH, 120M secure transformer framework designed to process encrypted text while preserving its semantic utility under an authorized- key threat model (no decryption or side-channel access). The core innovation of STEALTH is the Semantic Isomorphism Enforcement (SIE) loss function, a loss that trains the model to learn a copoloyy-preserving mapping between enerypted text embeddings and their original plaintext latent space. This encourages preservation of semantic relationships and topological structure in the encrypted domain. Using retrieval-based reconstruction from domain-aligned plaintext corpus, STEALTH achieves near-perfect semantic retrieval (BLEU score of 1.0 under full- corpus coverage in our experiments) and enables accurate privacy-preeevvigg clustering on encrypted embeddings. We evaluate STEALTH across 44 datasets spanning general language understanding, healthcare, finance, legal, e-commerce, programming, content analysis, reading comprehension, and corporate communication domains with encryption schemes (704 experimental conditions), establishing comprehensive benchmark for
NLP on encrypted text. Performance depends on domain alignment between encrypted inputs and the indexed plaintext corpus. Our results demonstrate that, with well-aligned domain indexes and retricval support, models can perform effective NLP on encrypted data. without direct decryption.
1 Introduction
Large language models (LLMs) have transformed NLP, yet the systems best suited to extract value from sensitive text cannot be safely deployod in privacy-critical settings. This barrier prevents adoption of powerful LLM tools in healthcare, finance, and law domains where aulomated analysis of clinical notes, transaction records, legal documents could deliver substantial societal benefit but is constrained by confidentiality and regulation. Rather than viewing regulations such as GDPR and HIPAA as obstacles, we frame them necessary sa safeguards that motivate the design of privacy-preservigg architectures. These frameworks ensure individual rights while challenging the ML community to develop robust, secure computation methods that. can operate within strict legal boundaries.
In healthcare, LLM-powered analysis of clinical text iece-including patient notes, radiology reports, and discharge summaries could enable automated diagnosis, personalized treatment, and large-scale research (Lee et. al.

---

Published in Transactions on Machine Learning Research (obbuaryy/2200
Kenton et al. 2019 Wang et al. 2019ch. However, recent regulatory developments, including the HHS Final Rule on HIPAA Privacy Rule to Support Reproductive Health Care Privacy, effective June 25, 2024, which prohibits the use or disclosure of PHI for certain reproductive health investigations, substantially constrain deployment of conventional language models on medical data. This missed opportunity is substantial: system generates enormous volumes of unst ructured clinical text annually (Raghupathi & Raghupathi 2014). Advanced language models and medical-specializzed variants have demonstrated strong capabilities in clinical language understanding and prediction (Lee et al., 2019 Alsentzer et al.) 2019), but those capabilities remain largely inaccessible to many providers because of privacy and compliance constraints.
2020:
the U.S.
Financial institutions face analogous challenges. Modern banking produces vast streams of textual data transaction narratives, loan applications, regulatory filings, and customer communications that are vital for risk management and fraud detection (Yang et al., 2020 Shah et al.) 2022a).. The European Union's General Data Protection Regulation (GDPR) and comparable regimes worldwide create complex compliance requirements that restrict how financial text may be processed (Hoofnagle et al.) 2019). In practice, these legal obligations complicate the adoption of LLMs for high-value financial tasks, particularly where rights such erasure, access, and rectification apply. The legal sector presents perhaps the most acute manifestation of the paradox. Attorncy-client privilege, the work-product doctrine, and professional- responsibility obligations impose stringent confidentiality constraints, while legal practice increasingly depends on automated processing of contracts, case law, discovery documents, and compliance materials (Chalkidis ee al.] 2020: Katz et al. 2017 Zheng et al.) 2021). State-of-the-art legal models have shown impressive capabilities, yet real-world deployment must reconcile those capabilities with essential confidentiality rules.
Transformers obtain their power by modeling intricate relationships across tokens lidens-semantc similarities, syntactic dependencies, contextual cues, and discourse-level structure via self-attention and contextual embeddings (Vaswani et al.) 2017 Devlin et al. 2018 Brown et al.) 2020). Standard cryptographic schemes (e.g., AES) intentionally destroy these relationships by producing ciphertext that is pseudorandom and thus somantically opaque, which undermines a transformer's ability to form meaningful contextual reprosentations DDemen & Rijmen] 2002). Consequently, semantic equivalences (e.g., "heart allack" vs. "myocardial infarction)), positional cues, and attention patterns are obscured by ciphertext, preventing transformers from forming useful contextual representations. Researchers have proposed several mitigatins-hhoooorpphi encryption (FHE), secure multi-party computation (MPC), trusted execution environments (TEEs), and differential privacy defer a full technical discussion and quantitative comparisons to Section 2 here we summarize at a high level: FHE incurs multiplicative-depth and bootstrapping costs that balloon computation; MPC replaces compulation with interactive communication that scales poorly with atiention 's quadratic interaction structure; TEEs shift risk from algorithms to hardware and supply chains; and DP, while formally rigorous, often requires noise levels that materially degrade language-generation quality (Gentry 2009b; Brakerski al., 2014 Mohassel & Zhang 2017b Burbank & Knight 2024 Dwork et al., 2006b A et al, 2016b Li et al,
each offering principled guarantees but also significant practical limitations. We
2022. These limits motivate ret renkinnng the srivacy-utiliig trade-off rather than attempting to force direct. cryptographic computation onto modern LLMs.
To address this impasse, we propose a different paradigm: learn a topology-preserringg mapping between encrypted and plaintext latent spaces so that encrypted inputs admit semantically faithful representations suitable for off off-tho-shelf language models, while strong cryptographic hygiene is preserved under an explicit threat model.
Concretely, we hypothesize learnable mapping Φ: P between an encrypted embedding manifold E and the plaintext manifold P that approximately preserves neighborhood and ranking relations (i.c., a topology-preserving mapping). We emphasize this is an empirical hypothesis that depends on corpus coverage and the chosen threat model.
Operational Scope and Threat Model: We define our operational scope within a Trusted Execution Environment (TEE) or secure enterprise enclave (e.g., HIPAA-compliant infrastructure). This aligns with the ecological reality of modern Confidential Computing (c.g., NVIDIA's 's TaE-based LLM safeguards, Azure Confidential Computing), where hardware isolation allows for ephemeral key access without exposing plaintext to the host OS. Crucially, we distinguish between the sensitive user input and the retrieval corpus. The
2

---

Published in Transactions on Machine Learning Research (February) 2026)
(1) Impur
Embeddinng vocabularr dimersion
Poshtionl mronding of positionans and Index
vocabulary
curput wright hiddon stane
socuonce
ISP NeoroPPeee
Ni
Output Tokens
Layers
Token Embedding
Positional Encoding
Mult--Hea Attention
Input
Tolenisation
Forward
Hiad
U o times meightt times wwight
ENCRYPTIOON
Piatnteut Processing Natural Language
Attention Forward Networks
(ii)
Encryption function from vocabburry soquence times key to vocabularyy soquencc
topological
loso aiggigmmmnntttttmmmmmmmmmmmmm losc
Semantic Embeddings
SIE Loss Alggemett
Eneryption Aware Tokerizer
Multi- Visrints
Leamable Fimbeddling
Key- Attentive Transformar
Eneryptior
Mult-Key Aggregation
STEALTH (Durs): Secure Transformrr for Enerypted Alignment of Latent Text Embeddings via Semantic isomorphism Enforcement (SIE) Loss Function
ENCRYPTION
(iii)
Calle top Klen Inmeeje gate wwightt] Selacia soparts
Food-forward network of s. fors in tu
80 Layers
Networkk
Epprt
Tokers
Cra Input Tort
Pokeniration
Embedding
Sparso
Expert Notwweks
Recuuced
sctuale sam ooer
taation
weight Limes iasdd forward networkk of
x NO ENCRYPTION
Drepsteal: Miturre-of Denture-ff-Dppetts Transformer for Efficient Conditional Computation Plaintest
Top Routing and Sparse Activation
Figure 1: Comparative architectural analysis of semantic processing pipelines. (i) Standard Plaintext Transformer: Relies on standard tokenization where input tokens map directly to semantically meaningful embeddings. This architecture depends on explicit linguistic structure (syntax/ /grammar) which is destroyed by encryption, rendering it ineffective for processing high-entropy ciphertext. (ii) STEALTH (Ours): An encryption-aguuotic framework designed to recover semantics from ciphertext. The pipeline processes multiple encrypted variants {E(x, {E(z,k))),= k5) via encryption- aware aware byte-level tokenization and a Key- Attentive Transformer Encoder that conditions attention mechanisms on key embeddings. A Multi-Key Aggregation module pools these representations to extract a key-invariant latent vector aligned with the plaintext manifold via the Semantic Isomorphism Enforcement (SIE) loss, enabling authorized retrieval. (ii) Cagh-Capacity Baselines (e-g., MoE): architectures that scale capacity via sparse top-k expert routing, While computationally powerful, they lack the specific inductive biases required to invert pscudorandom cryplographic permulations or align encrypled latent spaces. Delailed architectural analysis and component. ablation studies are provided in Appendix A.15
user input (e.g., a specific patient's diagnosis) is encrypted at the source and remains effectively inaccessible without the ephemeral authorized key provided to the enclave. In contrast, the retrieval corpus is a static, non-sensitive, domain-specific reference set (e.g., standard medical ontologies like ICD-10 or public legal statutes). STEALTH functions by mapping the encrypted private input to the nearest semantic concept within this public corpus; it never decrypts the raw input to plaintext. If a specific sensitive string (e.g., a private home address) does not exist in the public reference corpus, it cannot be retrieved, providing an inherent privacy safeguard. The "System Owner" is defined as the legally liable Data Controller (e.g., the hospital or financial institution) who maintains strict access controls over the secure enclave and authorized keys. Under this model, we assume the adversary does not have access to the ephemeral keys or the enclave memory; we evaluate deviations from this in Appendix A.12
Our STEALTH (Secure Transformer for Encrypted Alignment of Latent Text embeddings) architecture operationalizes this idea using three pragmatic components: (i) multi-key variability handling (processing multiple encryptions per datum and conditioning on key embeddings so the mapper learns key-conditioned invariants), (ii) hicrarchical
eesphrasc--sentenee alignment to preserve both local and global topology, and (iii) adaptive projection layers that reconcile geometry and dynamic range between encrypted and plaintext spaces. We evaluate STEALLTH across various encryption schemes (including stream ciphers,
3

---

Published in Transactions on Machine Learning Research (obbuary//02g)
block ciphers in multiple modes, and authenticated encryption for tokenizer compatibility; see Table 1 and mitigate requeecy/lenglhe leakage via multi-key variability (key rotation and multiple independent encrypted variants per datum), which breaks deterministic frequency signals across the corpus; residual leakage is evaluated experimentally. Concrete examples of plaintext, ciphertext, keys, and STEALTH reconstructions for each evaluated encryption scheme are provided in Appendix A.6 We train STEALTH with the Semantic Isomorphism Enforcement (SIE) loss, a multi-objective loss that combines triplet ranking, pairwise distance preservation (e.g., [ D, and topology-aware structural penalties to encourage popology-preserving mappings.
We empirically evaluate alignment quality, retrieval accuracy, manifold fidelity, and privacy leakage under multiple adversarial scenarios (ciphertext-only, chosen-plaintext membership-infered and reconstruction). We also measure sensitivity to corpus coverage and key variability (Appendix (A.1I..
We make the following contributions:
We formalize semantic (approximate) isomorphism under the authorized-key model, quantifying its limits and distinguishing authorized reconstruction from unauthorized cryptanalysis.
We present STEALTH, an end-to-end secure transformer framework with multi-key conditioning, hierarchical alignment, and adaptive projections for encrypled-plaintex reconciliation.
We introduce the Semantic Isomorphism Enforcement (SIE) loss, multi-objective framework for topological consistency via invariance, topological preservation, alignment, and metric constraints.
We develop comprehensive benchmark for privacy-preserve NLP in healtheare, finance, legal, e-commerce, programming, social media, and reading comprehension domains, with protocols for alignment, fidelity, retrieval, leakage, and robustness.
2 Related Work
Privacy-preserving computation on large language models remains a fundamental challenge at the intersection of cryptography and machine learning. We examine the technical landscape across four dimensions: crypto- graphic primitives for private computation, statistical and systems- based privacy mechanisms, representation learning under encryption, and hybrid approaches. This analysis reveals fundamental limitations of existing methods and motivates STEALTH's novel representation-alignmoon paradigm.
Cryptographic primitives face transformer-scale barriers. Classical cryptographic approaches pro- vide strong theoretical guarantees but encounter fundamental scalability challenges. Fully homomorphic encryption (FHE) (Gentry] 2009a) enables computation on encrypted data through BGV /GV/BFV and CKKS schemes (Brakerski Vaikunlanathan 2014 Fan Vercauteren 2012: Cheon et al. 2017). While Cryp-
toNets achieved 99% MNIST accuracy Gilld-Bachrack et al.) 2016) and GAZELLE CCeetah optimized
vision tasks (Huang et al. 2022b), transformers' (( n poly (A)) attention complexity causes prohibitive
slowdowns (Hesamifard et al.) 2017 Lou et al.
2020
Secure multi-party computation (MPC) via secret
sharing garbled circuits (Yao) 1982 Goldreich et al.
1987 1 instantiatedd in Secure ABY, SecureNN,
FALCON, and CrypTFFow2 (Mohassel & Zhang 2017a
Demmler et al.) 2015 Wagh et al. 2019 2021 Kumar
ct al. 2020 achieves practical vision inference but requires O(L-n") communication rounds for transformers,
precluding WAN deployment (Riazi et al. 2018). Trusted Execution Environments (TFFs) (Costan &
Devadas 2016 Tramer & Bonch, (0188) reduce overhcad through hardware isolation but face enclave memory
constraints and microarchitectural vulnerabilities (Kocher al., 2019 Van Bulck al. 2018 2020 These
techniques encounter fundamental scalability barriers transformer scale.
Statistical privacy approaches degrade utility for language generation. Statistical approaches trade cryplographic guarantees for computational elficiency but face severe utility degradatioo for generative tasks.
Differential privacy (DP) provides provable bounds through calibrated noise (Dwork et al. 2006a 2014), with
DP-SGD (Abadi et al. 20166) enhanced by composition thcorems (Mironov] 2017 Bun & Steinke] 2016).
privacy amplification (aalle et al. [0018] Wang et al.) (2019d), and adaptive clipping (Andrew et al. 2021
Pichapati et al. 2019 However, noise scales with model complexity (Bassily et. al.) 2014 Steinke & Ulmma
4

---

Published in Transactions on Machine Learning Research (ebbuaay/2026)
2016),
degrading transformers significantly even at E <> 8 YYu et al 2022 Li et al. 2022)
with generation
quality particularly affected by autoregressive compounding (Shi et al. 2022 Tang et al. 2024). Sample
complexity 2 (Bun et al. 2015 Ullman 2022) and linear privacy costs with sequence length (Carlini
ctt al.
2021 Beimel et al. 2013 present fundamental barricrs. Federated learning (FL) (McMahan et al.
2017 Li et al. 2020) with secure aggregation (Bonawitz el al. 2017 Bell et al. 2020 succeeds for keyboard
prediction (Hard et al. 2018: Leroy et al. 2019] but faces prohibitive communication costs for large models
despite compression (Reiszadeh et al. 2020; /attter al.) 2019
Rothchild et al., 2020), requiring 10 x
more rounds than centralized training (Karimireddy et al.
2020
Reddi et al.; 2021]. Synthetic generation 2018), and diffusion
via PATE (Papernot et al. 2017 2018), DP-GANs (Xie et al.
2018: Jordon et al.
models (Dockhorn et al. 2022 Ghalebikesabi et al. 2023) confronts LLM memorization (Carlini et al.) 2021
2023
Kandpal et a 2022) enabling extraction allacks (Lehman et al.
2021 Huang et al. 2022a Nasr el al,
2023
Mireshghallah et al. 2024). Machine unlearning (Eldan & Russinovich 2023) attempps retroactive data
removal but faces utility-privacy tradcoffs, with generation provably harder than discrimination (Gancsh)
et al.
2023 Brown et al.) 2022: Liu et al. 2023) and meaningful privacy (a < 1) with high utility remaining
elusive Stadder et al. 2022: Yue et al. 2023).
Representation-bascdd approaches balance efficicncy with information leakage. Represcntation-
based approaches balance computational efficiency with information leakage risks. Format-preservinn
encryption (FPF) (Bellare et al. 2009 Dworkin 2016) via FF1 /FF331 (Brier et al.] 2010) enables tok-
enization compatibility (Tasar et al.]
2023] Chen et al. 2024 but leaks information through frequency
analysis (Grubbs et al., 2019 Waag et al.) 2021), length patterns Cash et al.] 2015 Grubbs et al., 2017), and
|
intersection attacks (Kellaris al.) 2016
Naveed et al. 2015 Islam | 2012] 2). Property-preservinng schemes
(OPE (Boldyreva et al. 2009
2011), ORE (Bonch et al. 2015 Lewi & Wu 2016 enable reconstruction
attacks (Durak et al. 2016)] Grubbs et al.,
20177 Cash et al. 2016)) with minimal structural preservation
sufficing for powerful attacks (Kamara et al., 2018 Fuller et al., 2017 Kerschbaum, 2015 Roche et al.
2016 More promisingly, cross-domain alignment reveals semantic geometry consistency CCnneau et al.,
2018 Artetxe et al., 2018) through adversarial training (Zhang et al.] 2017
Lample et al.
2018), optimal
transport (Alvarez- Melis & Jaakkola 2018 Grave et al., 2019), and contrastive frameworks (Chen el al.
2020
He et al. 2020 Radford et al.) 2021 Multilingual models exhibit language-agnoste representations
Pires
ct al.] 2019 Conneau et al., 2020) with consistent goometry (Michael ct al. 2020
Chang al.
2022)
while
alignment algorithms provably recover isometries under mild assumplions (Grave et. al.
2019
Alaux et al.
2018 Sógaard et al.] 2018 motivating STEALTH's isomorphism- based approach. However, adversarial
censoring (Edwards& & Storkey 2016 Hamm et al. 2017) with multi-attribute protection (Madras et al. 2018
Wang et al.) 2019b] Zhang et al. (0188) cannot achieve perfect privacy-utilitty simultaneously for correlated
attributes (Zhao et al. 2020: Song et al., 2019). Projection methods (Ravfogel et al.] 2020 Bolukbasi et al.
2016 struggle with nonlinear dependencies (Gonen & Goldberg 2019; Kunar et al.) 2020) despite exten-
sions OOlfa & Aswani, 2020 Ravfogel & Goldberg 2022; Shekhar et al. 2021), while infornation--heorrtiee
approaches (Tishby & Zaslavsky 2015; Alemi et al. 2017) provide loose bounds (Goldfeld et al. 2020
Rodriguez et al. (020)) Practical systems combine cryptographic, hardware, and statistical techniques Knott
et al. 2021 Kairouz et al. 2021). though constituent limitations persist.
STEALTH addresses fundamental limitations through representation alignment. Existing ap- proaches face a fundamental trilemma: cryptographic methods imposc wransformer-inoompann computational costs, statistical approaches degrade generation utility, and structure-pres preeeeervni encryption leaks information through preserved patterns. STEALTTH addresses these challenges through representation aligmment that achieves four critical properties: cryptographic safeguards under authorized-key models without expensive homomorphic operations, effectively processing high-entropy ciphertext; semantic fidelity through geometric isomrrphism that preserves the topological structure and decision boundaries of the latent space; practical performance that avoids quadratic scaling by leveraging standard, wardware-accelerated matrix operations; and composability with existing defenses such as differential privacy and secure enclaves. This enables interactive inference on production scale transformers with rigorous privacy guarantees, bridging the gap between theoretical security and deployment-ready utility.
5

---

Published in Transactions on Machine Learning Research (February/200)
3 Methodology
This section details the thcoretical foundation and architectural implementation of the STEALTH framework. We begin by formalizing the problem of semantic isomorphism and defining our operational scope in Section 3.1 Next, present the neural architecture in Section 3.2 detailing the encryption-aware tokenization pipeline, the key-altentive transformer encoder, and the multi-key aggregation mechanism. Finally, we describe the authorized inference and reconstruction procedure in Section via semantic retrieval.
3.3 explicitly defining how plaintext is recovered
3.1 Problem Formulation
Privacy-preserving NLP confronts fundamental tension: traditional encryption destroys linguistic structures essential for semantic understanding, while approaches preserving semantics often compromise cryptographic strength. Given plaintext dataset D = {<r where each Iz is a variable-length sequence from vocabulary V. we generate K encrypted variants (xx) = {E(r,, with ayptographically secure keys ky ~ K. Encryption is denoted E. V K - C, mapping plaintexts ciphertexts under key k; the framework supports a variety of symmetric encryption schemes (see Appendix A.9 for details). We set K 5 based on the analysis in Appendix A.8
Our objective is learning E(x) - Red mapping encrypted inputs to a d-dimensional latent space (d 256) exhibiting approzimate semantic isomorphism with plaintext embeddings. We define isomorphism approx- imate topology preservation satisfying two properties: (1) key-inwariance: (F fo)), k2))) << ((,o(E(x, fo)). fo(E km))) for all keys and distinct texts x # y; and (2) distance-ords order preservation relative reference plaintext encoder g. Formally, the reconstruction of a plaintext from its ciphertext E(x, is defined not as generalive decoding, but as retrieval task against reference corpus P:
xt argminpep (lo((E(x)))) g(p)]|
(1)
where g(p) is the target embedding of a candidate plaintext. This formulation explicitly scopes the model's utility to the coverage of the authorized retrieval corpus.
3.2 STEALTH Architecture
The STEALTH architecture employs transformer-bast based encoder with key-attentive aggregation to process variably encrypted text while maintaining semantic coherence. Our architecture is specifically designed to handle the unique challenges posed by encryption, including processing multiple encrypted variants of the same text and maintaining semantic relationships despite cryptographic transformations.
3.2.1 Encrypted Text Processing
The preprocessing pipeline addresses the fundamental challenge of tokenizing encrypted text. Unlike standard plaintext tokenization, encrypted text processing must accommodate the statistical properties of cryptographic transformations.
Encryption-Aware Tokenization. Let Aplain denote the plaintext character alphabet. The encryption function E applies a pseudorandom permutation, potentially introducing characters from an extended alphabet A Standard NLP tokenizers rely UTF-8 validity, which encryption schemes frequently violate by producing arbitrary byte sequences. To resolve this, we employ byte-level tokenization :: 13" + V", mapping encrypted sequences directly to tokens via a deterministic byte-to-integer mapping /map byle + (o...., 255). This effectively treats the ciphertext as a stream of raw byles, establishing a fixed vocabulary size of [V] = 256. This approach completely bypasses UTF-8 validity constraints, enabling the model to process arbitrary binary outputs from modern ciphers (including non-printable characters) without information loss or tokenization errors. Critically, our approach maintains length invoriance: | (FF(,, k))l = [+()) + épad with /epad| C. for encryption block-size constant C.

---

Published in Transactions on Machine Learning Research (February/200 2026)
Non-Encrypted Processing
HINITT
Emooddid ruga
input Word
Frondar
Staci 12*"!
Softraa
CHC
Quipet
Scal
Positional
Encodera2
I
Live
Encoolin
Pacted infic tokan Matres
Encoder
(Logie))
Multi-Head Attention Position
Head
Taken Vectors
Muil Head
SIE Loss
Forood
Topolagical Preservation Component
Structural
Pouitional
bresder (2'
redd eecaa
Component
ENCRY PIION
e 4183cchtuue Eib wffift Bi
Embedsings
Embedsingg
Encoder
Linsee
Guntent
Frensiing
Encader
(iogleo
Bides
Eayy
Impe Tea
[36
Friender
Softras
LIVAN
ND
ICLS
Hiadd
Headh
Encrypted Processing
Figure 2: Schematic of the STEALTH Dual-Pathway Architecture. The framework employs a parallel processing strategy to enforce semantic alignment between encrypted and unencrypted latent spaces. (Top) Reference Plaintext Pathway: Utilizes standard baseline encoder to map unencrypted inputs into ground-truth latent space, establishing the target semantic topology required for valid retrieval. (Bottom) Encrypted Inference Pathway: The operational pipeline for processing high-entropy ciphertext. It integrates an encryption-aware tokenizer, Key-Attentiog Transformer Encoder that conditions self-attention on learned cryptographic key embeddings, and a Multi-Key Aggregation module that synthesizes robust, key-invariant representation. The Semantic Isomorphism Enforcement (SIE) Loss (right) optimizes the lower pathway by enforcing geometric alignment with the reference topology, utilizing specific objective terms for translational invariance, topological preservation, and structural regularization.
Embedding Layer Architecture. The tokenized encrypted sequence t t= .t.....,,T]...... V is mapped to dense representations via ()) = (t. + E RTxd, where Etok R256xd is the learned token embedding mairix and P represents positional encodings. To accommodate the approximately uniform frequency distribution encrypted bytes (versus the Zipfian distribution of natural language), we initialize Erok using Xavier uniform initialization:
Etok U (-v
(2)
VI d'
V 1
ensuring stable gradient flow.
Positional Encoding for Encrypted Sequences. Encryption disrupts linguistic patterns, necessitating
learnable positional encodings P E IR mmae xd. Each position receives embedding P; E Rd = (d=768) initialized
as (O, d= to match token embedding magnitudes. This enables discovering position-sensitive cryptographic patterns (e.g., block boundaries) and adapting to character-level randomization.
3.2.2 Key-Attentive Transformer Encoder
STEALTH's Key-Attentive Transformer Encoder augments the standard 12-layer architecture with key- conditioned computations throughout.

---

Published in Transactions on Machine Learning Research (February/202)
Key Encoding Network. Given encryption key & {0, (0,1) represented 128-dimensional binary vector, the KeyEncoder employs a multi-layer perceptron to project this cryptographic information into the semantic space:
KeyEncoder(A) =
+ bai) + bx2)
(3)
where WA1 k1 R384x128 W42 A2 key represcntation.
ER 768x384 are weight matrices. This produces Kembed E R67 as a continuous
Key-Attentin Attention Mechanism. At each transformer layer I, the key-attentive mochanism modifies standard self- aeff-attention by injecting spe-specific information. For each attention head, the queries, keys, and values are computed as:
Q0 (1111) w! (1) Q
(())) = (1111)
Kembed
W
(4)
K
K,key
V = (1111) w W
+ Kembed W
(
V.key
The attention mechanism computes KeyAttention (1) softmax
(K(((()
V(),
enabling the model to
vdh
adapt attention patterns based on the specific encryption key used.
3.2.3 Multi-Key Representation Aggregation
To handle the variability of ciphertexts, generate K encrypted variants (E(x, kj)} using distinct keys ky ~ K. We extract the classification token representation hy from each variant and employ learned attention-based pooling to compute a unified embedding 2:
K 2 = aj hy, aj ΣK exp(w" tanh W ah, + ba))
cxp(w T tanh W h,, + b,))
(5)
j=1
where W. E 256xx688 b, 256 and w E 250 are learned attention parameters. This mechanism dynamically weights variants based on semantic clarity, forcing the extraction of key-invariant representations. The aggregated representation z then projected to the target dimensionality and L2- n2-mommalized to produce the final output:
Zinal
Zfnal
zW proj
Dprojo
2iill
(6)
Z l|2
where Wproj E 1776 768x256 and bproj E 2556 This normalization ensures the embedding lies on the hypersphere, optimizing it for cosine similarity retrieval.
3.3 Inference and Reconstruction Procedure
Unlike generative language models that predict tokens autoregressively, STEALTH operates as a discriminative semantic pointer, utilizing a strict retrieval-based inference pipeline that enforces the security boundaries of the operational scope. Upon receiving a ciphertext input c Exx, k) and the corresponding ephemeral key k, the STEALTH encoder fe processes the pair generate a normalized latent embedding Zene 2556 This vector functions semantic query within Nearest Neighbor Search (NNS) against pre-indexed authorized retrieval corpus P, where the system calculates cosine similarities relative to reference embeddings {g(p) P} and retricves the plaintext p" associated with the maximum similarity score. By constraining reconstruction to the selection of pre-validated entries from P, this procedure prevents the generation of unauthorized or out-of-domain content, ensuring that valid plaintext is recovered only when the encrypted input semantically aligns with the authorized knowledge base. Formal algorithmic pscudocode provided in Appendix A.33 and we evaluate the robustness of this retrieval mechanism under varying corpus coverage conditions in Appendix A.11
8

---

Published in Transactions on Machine Learning Research (eebruar/2022)
3.4 Semantic Isomorphism Enforcement (SIE) Loss Function
The Semantic Isomorphism Enforcement, (STE) loss function constitutes the mathematical foundation of the STEALTH framework, designed to enforce topological consistency between encrypted and plaintext embedding spaces through novel multi-objective optimization approach. This loss function creates an approximate isometrie mapping between the two spaces while preserving semantic relationships despite the complex nonlinear transformations introduced by encryption. Unlike conventional approaches that rely on standard metric learning formulations, the SIE loss is specifically engineered from first principles to address the unique challenges of encrypted text processing.
Invariance Component The invariance component ensures that diverse cryptographic transformations of identical semantic content converge to proximate embedding regions. While encryption maintains structural properties, it introduces substantial lexical variation that disrupts semantic coherence without explicit invariance enforcement.
For plaintext E X and K encrypted variants k() with independently sampled keys k; ~ K, the invariance loss is:
K K
Linv BenD
deos (fo(E(x, fo
(7)
K(K 1)
where the normalization к(K-1J ensures scale invariance and provides an unbiased estimator of expected pairwisc distance. The cosine distance den = I = E [0,2] 2] captures angular separation, aligning with findings that semantic similarity correlates with directional alignment rather than Fuclidean proximity. Gradient dynamics induce clustering toward semantic centroids, learning key-invariant features while accommodating minor stochastic variations. Under optimization, cluster diameter contracts to c <> O, est establishing localized stability regions necessary for semantic isomorphism between encrypted and plaintext. embedding spaces.
nv
Topological Preservation Component The topological preservation component maintains global ge- ometric structure of semantic relationships when mapping from encrypted to plaintext embedding spaces, leveraging the manifold hypothesis that semantic data resides on lower-dimennsonal manifolds encoding linguistic relationships. For plaintext pairs (2, D and independent encryption keys k, k K, the topological loss is:
Cropo = EL
(8)
where g X' IRd, is fixed pre-trained reference embedding (Sentence-BERT) providing stable target geometry and preventing degenerate solutions. The squared difference disincentivizes large deviations while providing smooth optimization, encouraging fo to learn approximate isometry preserving relative distances. Independent key sampling ensures topological preservation holds universally across the key space, marginalizing over key distributions to reinforce encryption invariance for relational semantics.
In practice, we approximate via stratifiod sampling with batch-wisc empirical estimate:
Cropo B(B
(deos (o), (ccc))))))))
(9)
The loss is convex for linear fo with convergence rate O(I/BB.. While pairwise computation incurs O(B²) complexity, this overhead justified by comprehensive structural alignment. Operating with the invariance component-whit ensures local cluster compactness this dual approach enables fe to learn both intra- concept consistency and inter-concept relationships, achieving semantic isomorphism between encrypted and plaintext embedding spaces.
9

---

Published in Transactions on Machine Learning Research (obbuaayy/2200
Semantic Triplet Margin Objective The triplet margin objective enforces relational constraints between encrypted representations of semantically related and unrelated content, maintaining coherent, semantic boundaries despite encrypted surface forms. The margin-based formulation is:
Lriilltt EnN.aD [max (0, ((x, +m]]
(10)
where the distance difference is:
A cr, yn) = deos(fo(EErt) ka)): (o kx))) (fo (e k,)))
(11)
with kas hp ~ I as independent keys for anchor and positive instances (same content z), Yn ~ D fx} as negative sample with key hn ~ K, and margin m > establishing minimum separation. This ensures triangular inequality with respect to semantic similarity; with m = 0.1 empirically calibrated for semantic coherence while allowing cryptographie variation. Independent key sampling ensures semantic clustering across encryption transformations, while the margin accommodates inherent uncertainty in similarity judgments.
Batch- Wise Structural Alignment Objective The batch-wise alignment objective preserves global geometric structure by matching pairwise distance matrices:
Lalign
1 B(B 1)
Σ
Σ j
Waj (deos(So( (oll,, (
(12)
where adaptive weights Waj exp (-B (g(;) - pal) emphasize moderately distant pairs (Ha is median reference distance, 3 > 0 controls concentration), which capture muanced semantic relationships defining manifold structure. Equivalently, falign = Bı IW @ (Dene - Dptain) )|? minimizes weighted Frobenius norm between distance matrices Dene, Dylain E BxB preserving spectral properties and higher- order characteristies (cluster compactness, manifold curvature). We address O O(B²) complexity through vectorized computation and gradient accumulation. These terms provide multi-scale regularization: triplet margin ensures local semantic coherence, while batch-wise alignment preserves global manifold structure, achieving robust semantic isomorphism across cryptographic transformations.
Unified Semantic Isomorphism Objective The unified semantic isomorphism objective integrates all four component losses:
LSTE = aLinv + & + + + &Lalign
(13)
We set a = 3 = 7 = 1.0 based on Bayesian hyperparameter optimization, reflecting balanced contributions as ablation studics show all components are essential with no single term dominating. This objective exhibits desirable properties: symmelry ensuring unbiased treatment of encrypted variants, full differentiability enabling efficient optimization, scale invariance from cosine distance preventing magnitude artifacts, and Lipschitz continuity ensuring stable gradient flow. The formulation maintains permutation invariance with respect to encryption keys and converges to an e-approximate isometry where distance distortion between encrypted and plaintext spaces is bounded, ensuring semantic operations on encrypted embeddings yield results nearly identical to plaintext. Implementation leverages gradient accumulation for memory-intensive pairwise computations,, mixeddprecision training with dynamic scaling, and curriculum scheduling emphasizing invariance early and topological preservation later to accelerate convergence. See Appendix A.3 for pseudocode.
4 Experiments
To evaluate the effectiveness of STEALTH and the SIE loss function, we conduct comprehensive set of experiments focused on the model's ability to align encrypled text embeddings with their plaintext counterparts while preserving semantic structure, Our evaluation emphasizes urivacy-preservigg tasks, including embedding alignment, semantic retrieval, and clustering consistency, without ever decrypting the data during inference. We establish novel benchmark for privacy-preserving NLP, drawing from diverse datasets to assess generalizability across domains.
10

---

Published in Transactions on Machine Learning Research (obbuaay//2200
Datasets:
We introduce the first privacy-preserving NLP benchmark spanning nine domains with 44 datasets across classification, generation, clustering, and reasoning tasks. The benchmark includes: (1) general language understanding from GLUE (Wang et al. 2018a) (CoLA, QNLI, RTE, SST-2, STS-B) and SuperGLUE (Wang
ct al. 2019a (BoolQ, CB, COPA, MultiRC, ReCoRD, WiC, WSC); (2) e-commerce datasets simulating PCI DSS-compliant scenarios with Amazon Customer Reviews from C4 (Ni et al. 2019) (130M+ reviews) and
Multilingual Amazon Reviews [Koung et al. (020) (200K+ reviews, six languages); (3) healthcare applications
addressing HIPAA constraints via MedMCQA (Pal et al. 2022) (194K questions) and PubMedQA (Jin et al,
2019) (1K expert-annotated); (4) technical domains through HumanEval (Chen et al. 2021 (164 Python
problems) and MML.U [Hendryoks et al.] 2021) (57 subjects); (5) content analysis with 20 Newsgroups (Lang
1995p (20K documents) and IMDB (Maas et al. 2011) (50K reviews); (6) reading comprehension using
SQuAD D Rajpurrkrr el al. 2016), SQuAD v2 (Rajpurkar et al., 2018)) -iki and WikiText-2 (Merity
et al. 2017;: (7) corporate communications with Enron Email (Klimt & Yang 2004 (500K emails); (8) finance
datasets such as Financial PhraseBank (Malo et al.) 2014) and FPB-Sentiment (Shah et al., 2022b); ; (9) legal
datasets such as LEDGAR (Tuggener et al.] 2020) and ContractNLI (Koreeda & Manning 2021). We generale
encrypted variants using encryption with 80/ 10/10 10 splits, ensuring zero plaintext leakage during evaluation. Training convergence and learning dynamics are visualized in Appendix A.2] We evaluated primary text
columns from the 44 datasets (e.g., sentences, questions, contexts, passages, premises, hypotheses, reviews, bodies, provisions), concatenating paired fields as needed to retain semantic context during encryption and embedding.
Evaluation Metrics: We employ a multi-faceted evaluation strategy assessing semantic preservation, reconstruction fidelity, and privacy guarantees. For reconstruction quality, we measure BLEU scores (Papineni
ct al. 2002) for n-gram precision, ROUGE (Lin 2004) for recall-oriented matching, METEOR (Bancrjoc
& Lavie 2005) for alignment quality, and BERTScore (Zhang et al., (020) for semantic fidelity. Semantic
alignment is evaluated using cosine similarity between encrypted and plaintext embeddings (target: >0.95).
Computational efficiency is measured via inference latency, memory consumption, and throughput compared against unencrypted baselines. Implementation details are provided Appendix A.I A.I
5 Results
We evaluate STEALTH across 44 benchmark datasets spanning nine domains assess semantic preservation under encryption, comparison against privacy-preserving baselines, computational overhead, and scalability across dataset complexities. We present performance metrics including BLEU-1-4 (B1-B4) for n-gram precision, ROUGE-I/2/L (R1/R2/RL) for recall, METEOR (M) for alignment, BERT Frecision/Recall// (BP/BR/ for semantic fidelity, Cosine Similarity (C) for embedding alignment, and processing Time (T) in seconds. Detailed per-dataset results are provided in Appendix A.44 Although there is no prior end-to- end benchmark identical to STEALTH, we compare our method to representative approaches across three statistical (DP, federated learning), and representation- protections to establish meaningful baselines and quantify atillty-efficiene trade-offs (detailed baseline descriptions in Appendix A.5). Figure 3 and Figure 4 illustrate the domain-level analysis, confirming that while semantic fidelity remains robustly high across all sectors, inference latency reflects the natural computational scaling required for longer contexts in specialized domains.
crygoriis--cyytooraahhh
(THE/MPC/TI
STEALTH achieves exceptional semantic preservation across all 44 benchmarks with near-perfect average cosine similarity (1.00) and BERT Fl scores (1.00), demoustrating negligible utility degradation compared to plaintext processing. The framework maintains remarkably consistent performance across domains with minimal standard deviation in cosine similarity, indicating domain agnnoii robustness from informal social media to specialized legal texts. Average processing time of 1.41 seconds per sample enables both real-time applications such as content moderation and sentiment analysis as well as batch processing for document analytics and e-discovery, with computational overhead ranging from 0.96s for general language tasks to 1.96s for technical content. Strong ROUGE-L scores (average 1.00) confirm presorvation of discourse structure and long-range dependencies, while METEOR scores (1.00) validate paraphrase handling and synonymy in encrypted space.
11

---

Published in Transactions on Machine Learning Research (obbuary//22g
Table 1: Domain-level average performance summary across all 44 datasets evaluated with various encryption techniques. STEALTH achieves near-perfect overall cosine similarity with minimal utility degradation compared to plaintext processing. Encryption categories: V Stream Ciphers (XOR, RC4, Salsa20, ChaCha20); Block Ciphers-ECCh Mode (AES-ECB, Blowfish-ECHH 3DES-ECB); Block Ciphers-Advanced Modes LAES-CFB, AES-CTR, AES-CBC, 3DES-CBC); & Authenticated Encryption (AES-GCM, AES-EAX, AES- SIV, AES-OCB, AES-CCM). All encryption techniques yield statistically equivalent results within each domain. Metrics: B1 = BLEU-1, B2 = BLEU-2, B3 = BLEU-3, B4 = BLEU-4, R1 = ROUGE-1, R2 ROUGE-2, RL = ROUGE-L, M = METEOR, BP = BERT Procision, BR = BERT Recall, BF = BERT F1, C= Cosine Similarity, T = Processing Time (seconds). SDs are minimal (mostly 0.00). Comprehensive statistical results, including aggregate and domain-level summary statistics with SD, hypothesis tests, and robustness analyses, reported in Appendix A.14
Domain
N
Enc.
B1
B2
B3
B4
R1
R2
RL
M
BP
BR
BF
C
T
Gen. Lang. Und."
12
V
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.04
R-commerce
2
V
<
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.12
Medical
>
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.96
Technical
2
>
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.55
Content
4
A
+
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.23
Reading
4
V
1.00
1.00
0.99
0.98
1.00
0.95
1.00
1.00
1.00
1.00
1.00
1.00
1.38
Corporate
1
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.71
Finance
+
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.47
Legal
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.66
Overall
44 A
+
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.41
tGen, Lang. Und.
General Language Understanding; GLUE (N=5, T=1.11s) and SuperGLUE (N=7, T=0.96s) combined.
E-commerce, content analysis, and corporate commmicattins achieve perfect cosine similarity (1.00), demon- strating readiness for commercial deployments requiring privacy-preservin customer analytics and business intelligence. General language understanding benchmarks (GLUE, SuperGLUE) maintain strong performance (1.00), validating semantic fidelity across diverse linguistic phenomena including grammatical acceptabilily, textual entailment, and sentiment analysis. Reading comprehension tasks (average: 1.00) confirm preservation of complex reasoning and long-range dependencics under encryption.
Professional domains demonst.rate robust performance critical for enterprise deployment: finance, legal, and medical datasets achieve perfect cosine similarity (1.00). Finance enables privacy- preserving sentiment analysis and regulatory compliance tasks where data confidentiality is paramount. Legal applications support privileged document analysis, case law retrieval, and contract review across diverse tasks including statutory interpretation and case holding prediction. Medical datasets maintain strong performance under HIPAA constraints, with results on clinical records (MIMIC-III), biomedical QA (PubMedQA), and medical reasoning (emrQA) validating privacy-critee critical healthcare applications. Technical domains (average cosine similarity: 1.00) support secure code review and automated programming assistance for enterprise workflows.
These results validate STEALTH for sophisticated NLP under encryption across healthcare, finance, legal, and enterprise applications, enabling organizations to leverage advanced language models while maintaining strict privacy guarantees and regulatory compliance requirements such as HIPAA, GDPR, and attorney client priviloge. To further verify robustness, we evaluated STEALTH through an extensive ablation suite- testing a broad spectrum of loss functions (e.g., SIE, Nontrastive/NTT-Xeer Twins, Circle, Cosine, ArcFace/ /CssFace, etc.). Notably, other loss functions failed to achieve the desired mapping which is required for successful deeryption. Quantitative metrics for loss function ablations and diagnostic visualizations for encryption techniques are provided in Appendix A.7 and A.9 respectively.
triplet, CORAL, VICReg, Barlow
Downstream Utility via Clustering To validate the practical utility of STEALTH embeddings be- yond reconstruction, we evaluated their effectiveness as drop-in replacements for plaintext embeddings in unsupervised analytical tasks. We performed K-Means clustering directly on the encrypted embeddings
12

---

Published in Transactions on Machine Learning Research (February/200) 2026)
BLEU-4 Score (Semantic Preservation
1.00
0.99
2.90
2.97
3.0
Cat4
oNd
575
cal c2Pe
Fioh
CUAD
Corleda
rpp
LE
LEnGUe
CASE
EURLEX LAX
sconis
Contiy
Figure 3: Comprehensive Domain-Level Semantic Fidelity Analysis. Aggregate BLEU-4 scores across 44 datasets spanning 10 distinct domains (GLUE, SuperGLUE, E- E-Commerce, Medical, Technical, Content, Reading, Corporate, Finance, Tegal). STEALTH demonstrates consistent semantic preservation with reconstruction scores approaching 1.00 across diverse linguistic complexities, confirming that the Semantic Isomorphism Enforcement (SIE) loss successfully aligns the encrypted latent manifold with the plaintext topology regardless of domain-specins jargon. The uniform high fidelity in specialized fields (e.g., Medical, Legal) versus general benchmarks indicates that the architecture robust to vocabulary shifts. Detailed per-dataset results including ROUGE, METEOR, and BERTScore metrics are provided in Appendix A.4
without decryption. As detailed in Appendix A.9 (Tables 19 and 20), STEALTH achieves Adjusted Rand Index (ARI) scores consistently exccoding 0.94 across all evaluated encryption schemes (c.g., 0.98 for XOR and Salsa20, 0.97 for AES-GCM). These results demonstrate that the encrypted latent space preserves the topological structure and semantic decision boundaries of the original manifold with near-perfect fidelity, enabling wrivacy-preserving analytics and catcgorization workflows to operate directly the ciphertext. embeddings.
6 Discussion
Achievements of STEALTH: (1) Novel Framework: STEALTH introduces a semantic isomorphism formulation for enables meaningful computation on encrypted text while maintaining cryptographic security. Grounded topological preservation principles, this method captures struccuur mappings between encrypted and plaintext latent spaces, bridging the gap between rigid cryplographic primitives and the semantic flexibility required by LLMs.
natural language processing, establishing a theoretical foundation that
(2) Technical Adventages: Our method explicitly tackles the utility-privacy tradc-off using the Semantic Isomorphism Enforcement (SIE) loss function. By preserving lopological relationships through multi- objective optimization including invariance preservation, struetural alignment, and metric constraints STEALTH achieves perfect semantic retricval (BLEU 1.0) and accurate privacy-preserving clustering. This formulation reduces information leakage while maintaining high manifold fidelity, as evidenced by the statistical indistinguishabiiiyy of the encrypted and plaintext spaces.
13

---

Published in Transactions on Machine Learning Research (February/ 2026)
Lateney (secondls)
2--
CON
scorus
Conter
Figure 4: Inference Latency Profiling across 44 Benchmarks. Average end-to-end processing time (seconds) per sample across 10 distinct domains. While standard semantic tasks (e.g.. GLUE, SuperGLUE) are processed with high efficiency (< 1.1s), the system exhibits natural latency scaling correlated with input token density, evidenced by peaks in long-context domains including Medical (PubMedQA: 4.51s) and Reading Comprehension ( WWkkiTent-1033 2.35s). This linear scaling confirms the architectural efficiency for tigu-throughput deployment in constrained Trusted Execution Environments (TEEs). Detailed latency breakdowns and efficiency statistics for all datasets are provided in Appendix A.4
(3) Practical Benefits and Data Rights STEALTH combines robust encryption with practical efficiency, enabling end-to-end processing of sensitive data in healtheare, finance, and legal domains without decryption. Its versatility across transformer architectures and lasks (retrieval, clustering, classification) makes it ideal for real-world deployment. Crucially, STEALTH facilitates compliance with legal frameworks such as GDPR and CCPA. Specifically, it enables the Right to Erasure (GDPR Art. 17) via crypto-shredding: because every encrypted embedding is mathematiicllyy dependent on a unique authorized key, deleting that specific key renders the stored embeddings semantically meaningless and computationally unrecoverable. This allows organizations effectively erase user data from vector databases without the computationally expensive process of scrubbing individual entries, ensuring that privacy rights are operationally enforceable.
The deployment of STEALTH offers substantial societal benefits by unlocking sensitive data silos in healthcare and finance, enabling the training of robust models on data previously inaccessible due to privacy regulations (c.g., HIPAA). However, ethical deployment requires guarding against securily over-reliance. While STEALTH provides strong protection under the authorized-key threat model, users must not mistake 'cnerypted embeddings' for ceommattoon-hhoorttin sccurity. Because the embeddings are designed be semantically isomorphic to plaintext, they reveal semantic topology to any entity holding the authorized key. Therefore, security relies critically the integrity of the Trusted Execution Environment (TEE). We emphasize that strict access controls must apply to the retricval corpus; since reconstruction is a retrieval process, controlling the reference index is as critical as controlling the keys. STEALTH should be viewed as a powerful layer of defense-in-depth that enables utility, rather than a standalone replacement for physical and access security.
Broader Impact and Ethical Considerations:
Limitations: Despite its advancements, STEALTH's reconstruction is fundamentally retrieval process, bound by the coverage of the available domain-aligned plaintext corpus. Consequently, in "ont-of-distributtin"
14

---

Published in Transactions on Machine Learning Research (obruary/2020)
scenarios where imput semantics significantly diverge from the index sadex-such as emerging ncologisms, rare dialects, highly specific technical jargon-the the system resorts to nearest-neighbor approximation, which may introduce semantic drift (potentially dropping BLEU scores below 0.95). Additionally, the computational overhead of similarity search scales linearly with corpus size, presenting a tradc-off between semantic recall and inference latency. Finally, unlike Fully Homomorphic Encryption (FHE), our authorized model necessitates a Trusted Execution Enviromment (TEE) for the initial embodding projection, restricting deployment secure enclaves where transient key access is permissible.
Future work. While STEALTH establishes practical pathway for aligning encrypted and plaintext embedding spaces, several directions remain open: (i) adapting to multimodal models (vision-language, audio text) for private cross-modal retrieval; (ii) extending to federated and edge settings; (iii) exploring continual learning regimes; (iv) developing formal analyses of & (leakage bounds, adversarial robustness, non-invertibility conditions); and (v) integrating cryptographic primitives (secure MPC, differential privacy) tradcoffs. We plan validation through standardized benchmarks and
to characterize
deployment case studies.
7 Conclusion
We propose STEALTH, a novel secure transformer framework, which introduces unique approach to privacy-preservngg natural language processing via the principle we term semantic isomorphism. In this novel framework, we depart from the fundamental philosophy traditional encryption, which renders text unusable for inference, and instead focus on preserving topological relationships between encrypted and plaintext representations. Within this framework, our main focus lies on addressing the privacy-utility tradeoff achieve meaningful computation on encrypted text, tackling the challenge of maintaining semantic utility while ensuring cryptographic security. In addition, we integrate multi-key conditioning and hierarchical alignment into an instance, resulting in substantial improvements in semantic retrieval (BLEU score of 1.0) and clustering accuracy compared to leading methods, which mitigates the problem of information leakage. Also, STEALTH demonstrates efficiency and versatility across transformer architectures and tasks (retrieval, clustering, classification), solidifying its suitability for real-world adoption in privacy-critical domains such as healthcare, finance, and legal sectors. Our comprehensive empirical experiments, spanning diverse datasets, consistently highlight the superior performance of STEALTH in bridging the gap between robust encryption and practical model utility.
References
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Commanications Security, pp. 308 318, 2016a.
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, lyy Mironov, Kunal Talwar, and Li Zhang Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and commanications security, pp. 308-318, 2016b.
Jean Alaux, Edouard Grave, Marco Cuturi, and Armand Joulin. Unsupervised hyperalignment for multilingual word embeddings. In International Conference on Learning Representations, 2018.
Alexander A Alemi, lan Fischer, Joshua Dillon, and Kevin Murphy. Deep variational information bottleneck. In International Conference on Learning Representations, 2017.
Emiiy Alsentzer, John Murphy, William Boag, Wei-Hung Weng, Di Jindi, Tristan Naumann, and Matthew McDermott. Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.088: 03322,, 2019.
David Alvarez-Melis and Tommi S Jaakkola. waosorswassorstein alignment of word embodding spaces. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1881-1890, 2018.
15

---

Published in Transactions on Machine Learning Research (February/022)
Galen Andrew, Om Thakkar, H Brendan McMahan, and Swaroop Ramaswamy. Differentially private learning with adaptive clipping. In Advances in Neural Information Processing Systems, pp. 77455-174660 2021.
Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saunshi. A theoretical analysis of contrastive unsupervised representation learning. arXiv preprint 190::190.09220 2019.
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 789-798, 2018.
Borja Balle, Gilles Barthe, and Marco Gaboardi. Privacy amplification by subsampling: Tight analyses via couplings and divergences. In Advances in neural information processing systems, pp. 6277 2277-6287, 2018.
Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsie Evaluation Measures for Machine Translation and/or Summarization, pp. 65-72. Association for Computational Linguistics, 2005.
Adrien Bardes, Jean Ponce, and Yann LeCun. Viereg: Variaace-invarinnc-ccvvarr regularization for self-supervised learning. Tn International Conference on Learning Representations, 2022.
Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In 2014 IEEE 55th Annual Sympostum on Foundations of Computer Science, pp. 464-473. IEEE, 2014.
Amos Beimel, Kobbi Nissim, and Uri Stemmer. Characterizing the sample complexity of private learners. In Proceedings of the 4th conference on Innovations in Theoretical Computer Science, pp. 97 97-110, 2013.
James Henry Bell, Keith Bonawitz, Adrià Gascon, Tancrède Lepoint, and Mariana Raykova. Secure single-server aggregation with (poly) logarithmic overhead. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 1253 2259-1269, 2020.
Mihir Bellare, Phillip Rogaway, and Terence Spies. premav-ng encryption. In International Workshop on Selected Areas in Cryptography, pp. 212-312. Springer, 2009.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, (18):1798-182)) 2013.
Alexandra Boldyreva, Nathan Chenette, Younho Lee, and Adam O'neill. Order-preserving symmetric encryption. In Annual International Conference on the Theory and Applications of Cryplographic Techniques, pp. 224-241. Springer, 2009.
Alexandra Boldyreva, Nathan Chenette, and Adam O'Neill. Order-preserving eneryption revisited: Improved security analysis and alternative solutions. In Annual Cryptology Conference, pp. 578 595. Springer, 2011.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man to computer programmer as woman is to homemaker? debiasing word embeddings. Tn Advances in neural information processing systems, pp. 4349 4357, 2016.
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving machine learning. In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1175-1191, 2017.
Dan Bonch, Kevin Lewi, Mariana Raykova, Amit Sahai, Mark Zhandry, and Joe Zimmerman. Function-hiding inner product eneryption. In International Conference on the Theory and Application of Cryptology and Information Security, pp. 213 244. Springer, 2015.
16

---

Published in Transactions on Machine Learning Research (obruary/202g)
Zvika Brakerski and Vinod Valkuntanathan. Efficient fully homomorphic encryption from (standard) Iwe. STAM Journal on Compuling, 43 (2):831-871, 188811171,,-1 2014. doi: 10.1137 20187711000669.
Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. Leveled fully homomorphic encryption without bootstrapping. ACM Transactions on Computation Theory (TOCT), 6(3):1-36, 2014.
Eric Brier, Thomas Peyrin, and Jacques Stern. Ffl and ff3-1: Flexible format preserving encryption, 2010.
Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallaah, Reza Shokri, and Florian Tramer. What does it mean for language model to preserve privacy? Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 2280-2292 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Mark Bun and Thomas Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In Theory of Cryptography Conference, pp. 658. Springer, 2016.
Mark Bun, Jonathan Ullman, and Salil Vadhan. The limits of two-party differential privacy. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science, pp. 689-68 698. IEEE, 2015.
Donna Burbank and Michelle Knight. Trends in data management 2024. Whilepaper, 2024. URL https: //content. dataversity wwt.:eet//rrteeeee wwwrr///66wwa6 WM05--WW-9220 918/ http://266666.111113332222222111
WP_ Trendsee_____ _D__DM_22222200 pdf
Nicholas Carlini, Florian Tramer, Erie Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium, pp. 2633-2650, 2021.
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang, Quantifying memorization across neural language models. In International Conference on Learning Representations, 2023.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Advances in Neural Information Processing Systems, pp. 9912-9924, 2020.
David Cash, Paul Grubbs, Jason Perry, and Thomas Ristenpart. Leakage-abuse atlacks against searchable encryption. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 668-679, 2015.
David Cash, Paul Grubbs, Jason Perry, and Thomas Ristenpart. Leakage-abuse attacks against searchable encryption. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 668-679, 2016.
Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and lon Androutsoponlos. Legalbert: The muppets straight out of law school. arXiv preprint arXin:2010.02550 2020.
Tyler A Chang, Zhuowen Tu, and Benjamin K Bergen. Geometry of multilingual language model representa- tions. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 119 136, 2022.
Hao Chen, Wei Dai, Miran Kim, and Yongsoo Song. Encrypted neural network training and inference using homomorphic encryption. In IEEE Transactions on Information Forensics and Security, volume 19, pp. 2259-2272. IEEE, 2024.
Mark Chen, Jerry Tworck, Hoewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. In Xiu preprint arXiv:2107.03377, 2021.
17

---

Published in Transactions on Machine Learning Research (abbuaay/2020
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pp. 1597-1607. PMTR, 2020.
Weihua Chen, Xiaotang Chen, Jianguo Zhang, and Kaiqi Huang. Beyond triplet loss: A deep quadruplet network for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pollern Recognilion, pp. 403 412, 2017.
Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. Homomorphic encryption for arithmetic of approximate numbers. In International Conference on the Theory and Applicalion of Cryplology and Information Security, pp. 409-437, Springer, 2017.
Alexis Conneau, Guillaume Lample, Marc Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data. In International Conference on Learning Representations, 2018.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross- lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Länguisties, pp. 1440-8451, 2020.
Victor Costan and Srinivas Devadas. Intel sgx explained. In TACR Cryptology ePrint Archive, pp. 86, 2016.
Joan Daemen and Vincent Rijmen. The design of Rijndael: AES-the advanced encryption standard. Springer Science & Business Media, 2002.
Daniel Demmler, Thomas Schneider, and Michael Zohner. Aby-a framework for efficient mixed-protocol secure two-party computation. In Network and Distributed System Security Symposium, 2015.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeirion. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4490-66999 2019.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. orXiv preprinl 1rXiv:1810.044 2018.
Tim Dockhorn, Tianshi Cao, Karsten Kreis, and Guilin Zhu. Differentially private diffusion models. In arXiv preprint Xrx:::2210.080 099999 2022.
F Betül Durak, Thomas M DuBuisson, and David Cash. What else is revealed by orver-revealing encryption? Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1155 1166, 2016.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to scnsitivity in private data analysis. In Theory of Cryptography Conference, pp. 284. Springer, 2006a.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private 2006b.
data analysis. Theory of eryptogrophy conference, pp.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3-4):211-400 2014.
Morris J Dworkin. Recommendation for block cipher modes of operation: Methods for format-preserving encryption. Technical Report NIST Special Publication 800-38G, National Institute of Standards and Technology, 2016.
Harrison Edwards and Amos Storkey. Censoring representations with an adversary. In International Conference on Learning Representations, 2016.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in 1lm.. arXiv preprint arXiv:2310.0P0 28.00238, 2023.
18

---

Published in Transactions on Machine Learning Research (obbuaary/2200
Junfeng Fan and Frederik Vercauteren. Somewhat practical fully homomorphic encryption. In Annual Cryplology Conference, pp. 144-161. Springer,,,, 2012.
Benjamin Fuller, Mayank Varia, Arkady Yerukhimovich, Emily Custodio, Robert Beaulieu, Rachel Greenstadt, Zachary Birnbaum, and Shawn Stamm. Sok: Cryptographically protected database search. In 2017 IEFF Symposium on Security and Privacy (SP), pp. 172 191. IEEE, 2017.
Aditya Ganesh, Chen Zhao, Ayush Sekhari, Gautam Kamath, and Tatsunori Hashimoto. Private generation of language models. In Xiv preprint arXiv:2305.1061,, 2023.
Craig Gentry. Fully homomorphic encryption using idcal lattices. In Proceedings of the forty-first annual ACM symposium on Theory of compuling, pp. 169 178. ACM, 2009a.
Craig Gentry. Fully homomorphic encryption using idcal latticos. Proceedings of the forty-first annual ACM symposium on Theory of compuling, pp- 169-178, 2009b.
Sahra Ghalebikesabi, Amil Goindani, Arthur Gretton, and David Evans. Differentially private diffusion models generate useful synthelic images. In ar arXiv preprinl arXiv:2302. 13861, 2023.
Ran Cilha-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Nachrig, and John Wernsing. Cryp- tonets: Applying neural networks to encrypted data with high throughput and accuracy. In International conference machine learning, pp. 201 210. PMLR, 2016.
Ziv Goldfeld, Fwout van den Berg, Kristjan Greenewald, Igor Melnyk, Nam Nguyen, Brian Kingsbury, and Yury Polyanskiy. Estimating information flow in deep neural networks. In International Conference on Machine Learning, pp. 3651-3660. PMLR, 2020.
Oded Goldreich, Silvio Micali, and Avi Wigderson. How to play any mental game. In Proceedings of the nineteenth annual ACM symposium on Theory of computing, pp. 218-229, 1987.
Hila Gonen and Yoav Goldberg. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 609 614, 2019.
Edouard Grave, Armand Joulin, and Quentin Berthet. Unsupervisod alignment of embeddings with wasscrstein procrustes. In The 22nd International Conference on Artificial Intelligence and Statislies, pp. 1880 1890. PMLR, 2019.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(1):723-777 773, 2012.
Paul Grubbs, Kevin Sekniqi, Vincent Bindschaedler, Muhammad Naveed, and Thomas Ristenpart. Leakage- abuse attacks against searchable encryption. In Proceedings of the 2017 ACM SIGSAC Conference on Compuler and Communications Securily, pp. 668 679, 2017.
Paul Grubbs, Marie-Sarah Lacharité, Brice Minaud, and Kenneth G Paterson. Leakage-abuse attacks against order-revealing encryption. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 655-672. IEEE, 2019.
Jihun Hamm, Yingjun Cao, and Mikhail Belkin. Learning privately over distributed features: An admm sharing approach. In arXiv preprint 1rXi:1510.06735, 2017.
Jeff HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self-supervised deep learning with spectral contrastive loss. In Advances in Neural Information Processing Systems, pp. 5000-5011, 2021.
Andrew Hard, Kanishka Rao, Rajiv Mathows, Swaroop Ramaswamy, Frangoise Beaufays, Scan Augenstein, Hubert Eichner, Chloe Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. In Xiv preprint Xiv 1811 1. .03604, 2018.
19

---

Published in Transactions on Machine Learning Research (abbuaayy/220
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVE conference on computer vision and pollern recognition, pp. 9729 9738, 2020.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In Proceedings of the International Conference on Learning Representations, 2021.
Ehsan Hesamifard, Hassan Takabi, and Mehdi Ghasemi. Cryptodl: Deep neural networks over encrypted data. In aix preprint arXiv: 711.05189, 2017.
Chris Jay Hoofnagle, Bart van der Sloot, and Frederik Zuiderveen Borgesius. GDPR and the EU's new data protection regulation: A guide for policymakers and practitioners. University of California, Berkeley, 2019.
Yangsibo Huang, Samyak Gupta, Zexuan Ding, Tim Roughgarden, and Chiyuan Zhang. Large language models can be strong differentially private learners. In International Conference on Learning Representations, 2022a.
Zhicong Huang, Wen-jie Lu, Cheng Hong, and Jiansheng Ding. Chectah: Lean and fast secure two-party deep neural network inference. In 31st USENIX Securily Symposium, pp. 809 826, 2022b.
Mohammad Saiful Islam, Mehmet Kuzu, and Murat Kantarcioglu. Access pattern disclosure on searchable encryption: Ramification, attack and mitigation. In Network and Distributed System Security Symposium, 2012.
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. Pubmedqa: A datasct for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Melhods in NNtural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2567-2577, 2019.
James Jordon, Jinsung Yoon, and Mihaela Van Der Schaar. Pate-gan: Generating synthetic data with differential privacy guarantees. In International conference on learning representations, 2018.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1-2):1-210, 2021.
Seny Kamara, Tarik Moataz, and Olya Ohrimenko. Sql on structurally-oncrypted databascs. In International Conference on the Theory and Application of Cryplology and Information Securily, pp. 149-180. Springer, 2018.
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8424-8444 8445, 2022.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In Inter- national Conference on Machine Learning, pp. 5132-5143. PMLR, 2020.
Daniel Martin Katz, Michael J Bommarito, and Josh Blackman. A general approach for predicting the behavior of the supreme court of the united states. PloS one, 12(4))00174698, 2017.
Georgios Kellaris, George Kollios, Kobbi Nissim, and Adam O'Neill. Generic attacks on secure outsourced databases. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1329 1340, 2016.
Jacob Ming- Way Kenton, Ming-Wee Wei Chang, Kristina Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACI-HIT,, 1:2, 2019.
20

---

Published in Transactions on Machine Learning Research Sobruary/2020)
Florian Kerschbaum. Frequency-hidine order- order-preserve preserving encryption. In Proceedings of the 22nd ACM SIGSAC Conference on Compuler and Communications Securily, pp. 656-667, 2015.
Phillip Keung, Yichao Lu, György Szarvas, and Noah Smith. The multilingual amazon reviews corpus. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4563-4568, 2020.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In Advances in Neural Information Processing Systems, pp. 18661-18673, 2020.
Bryan Klimt and Yiming Yang. The enron corpus: A new dataset for email classification research. In European Conference on Machine Learning, pp. 217 226, 2004.
Brian Knott, Shobha Venkataraman, Awni Hanmun, Shubho Sengupta, Mark Ibrahim, and Laurens van der Maaten. Crypten: Secure multi-party computation meets machine learning. In Advances in Newral Information Processing Systems, pp. 4961-4973, 2021.
Paul Kocher, Jann Horn, Anders Fogh, Daniel Genkin, Daniel Gruss, Werner Haas, Mike Hamburg, Moritz Tipp. Stefan Mangard, Thomas Prescher, et al. Spectre attacks: Exploiting speculative execution. In 40th IEEE Symposium on Security and Privacy (SBP'19), pp. 1 19. IEEE, 2019.
Yuta Koreeda and Christopher D Manning. Contractnli: A dataset for document-level natural language inference for contracts. In Pindings of the Association for Computational Linguistics: EMNLP 2021, pp. 1907-1919, 2021.
Nishant Kumar, Mayank Rathee, Nishanth Chandran, Divya Gupta, Aseem Rastogi, and Rahul Sharma. Cryptflow: Secure tensorflow inference. In 2020 IEFE Symposium on Securily and Privacy (SP). pp- 336-353. IEEE, 2020.
Guillaume Lample, Alexis Conncau, Ludovic Denoyer, and Marc' Aurelio Ranzato. Unsupervised machine translation using monolingual corpora only. In International Conference on Learning Representations, 2018.
Ken Lang. Newsweeder: Learning to filter netnews. Proceedings of the 12th International Conference on Machine Learning, pp. 331-339, 1995.
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: pre-trained biomedical language representation model for biomedical text mining, ar preprint arXiv:19010087.0 2019.
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang, Biobert: pre-trained biomedical language representation model for biomedical text mining. Bioinformalics, 36(4):1234 1240, 2020.
Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, and Byron Wallace. Does gpt-2 know your phone number? In arXiv preprint arXiv:2006. 16495, 2021.
David Leroy, Alice Coucke, Thibaut Lavril, Thibault Gisselbrecht, and Joseph Durcau. Federated learning for keyword spotting. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.
IEEE, 2019.
Kevin Lewi and David J Wu. Order-revealing encryption: New constructions, applications, and lower bounds. In Proceedings of the 2016 ACM SIGSAC Conference Computer and Communications Security, pp. 1167-1178, 2016.
Tian Li, Anit Kumar Sahu, Manzil Zahoor, Maziar Sanjabi, Amoct Talwalkar, and Virginia Smith. Federated optimization in heterogencous networks, In Proceedings of Machine learning and systems, volume 2, pp. 429-450, 2020.
21

---

Published in Transactions on Machine Learning Research (obbuaary/2200
Xuechen Li, Florian Tramèr Chen, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. arXie preprint 10.0567, 2022.
Chin-Yers Lin. Rouge: A package for automatic evaluation of summaries. In Tezt Summarization Branches Out: Proceedings of the ACI-04 Workshop, pp. 74-81. Association for Computational Linguisties, 2004.
Zelei Liu, Geng Chen, Han Zhao, and Yong Li. Privacy-utility tradeoff in generative models. In ar preprinl (xXx:2302.001122 2023.
Qian Lou, Bo Feng, Geoffrey C Fox, and Lei Jiang. Safenet: A safe, efficient and resilient framework for enerypted neural networks. In Proceedings of the S7th International Conference on Machine Learning, pp. 6444-6454, 6454, 2020.
Andrew L Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Araual Meeling of the Associalion for Computational Linguistics: Human Language Technologies, pp. 142-150, 2011.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In International Conference on Machine Learnêng, pp. 5384-3393. PMLR, 2018.
Pekka Malo, Ankur Sinha, Pekka Korhonen, Jyrki Wallenius, and Pyry Takala. Good debt or bad debt:
Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65(4):782-799 2014.
Andreas Maurer. A vector-contraction inequality for rademacher complexities. Algorithmic Learning Theory, pp. 3-17, 2016.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp- 1273-1282. PMLR, 2017.
Facundo Mémoli. Gromov wasserstein distances and the metric approach to object matching. Foundations of Computational Mathematics, 11tiiss,11111 11(4):417-487, 2011.
Stephen Mcrity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. International Conference on Learning Representations, 2017.
Steven Michael, Lukas Donahue, Mor Geva, Omer Levy, and Jonathan Berant. Multilingual alignment of contextual word representations. In International Conference on Learning Representations, 2020.
Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg- Kirg-Kiikpatrick, and Reza Shokri. Lira+++: Scalable membership inference attacks against large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 1-16. ACL, 2024.
Tya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Synposium (CSF), pp. 263 275. IEEE, 2017.
Payman Mohassel and Yupeng Zhang. Sccureml: A system for scalable privacy-presorving machine learning. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 19-38. IEEE, 2017a.
Payman Mohassel and Yupeng Zhang. Secureml: A system for scalable mrivacy-preservin machine learning. In 2017 IEEE symposium on security and privacy (SP), pp. 19-30 08-3. IEEE, 2017b.
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Chog Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. In arXiv preprint urXiw:2311.17033 2023.
Muhammad Navcod, Seny Kamara, and Charles V Wright. Inference attacks on databases. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 644 655, 2015.
enerypted
22

---

Published in Transactions on Machine Learning Research (February2222)
Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) pp. 188-197, 2019.
Matt Olfat and Anil Aswani. Convex formulations for fair principal component analysis. In The 23rd International Conference on Artificial Intelligence and Statistics, pp. 1712-1722. PMLR, 2020.
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmeqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. Proceedings of the Conference on Health, Inference, and Learning, pp. 248 288-260, 2022.
Nicolas Papernot, Martín Abadi, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar. Semi-supervised knowledge transfer for deep learning from private training data. In International Conference on Learning Representations, 2017.
Nicolas Papernot, Shuang Song, Nya. Mironov, Ananth Raghunathan, Kunal Talwar, and Ülfar Erlingsson. Scalable private learning with pate. In International Conference Learning Representations, 2018.
Kishore Papinemi, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Blew: method for automatic evaluation of machine translation. In Proceedings of the 40th Anseual Meeling of the Associalion for Compulational Linguistics, pp. 311-318, 2002.
Gabriel Peyré and Marco Cuturi. Computational optimal transport: With applications to data science. Foundations and Trends in Machine Learning, 11(5-6):355 607, 2019.
Venkatadheeraj Pichapati, Abhradeep Thakurta, Prateek Jain, and Ananda Theertha Suresh. Adaclip: Adaptive clipping for private sgd. In arXie preprint arxiv: 1rX8::1908..7643, 2019.
Telmo Pircs, Eva Schlinger, and Dan Garrette. How multilingual is multilingual bert? In Proceedings of the 57th Annual Meeling of the Associalion for Computational Linguislics, pp. 4996 5001, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748 8763. PMLR, 2021.
Wullianallur Raghupathi and Viju Raghupathi. Big data analytics in healthcare: promise and potential. Health information science and systems, 2(1):1-10, 2014.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference Empirical Methods in Natural Language Processing, pp. 3383-23390 2016.
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 784 789, 2018
Shauli Ravfogel and Yoav Goldberg. Kerned-down neural network for kernel methods, In arXivv preprint arXiv:2201.067/ 2022.
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out: Guarding protected attributes by iterative mullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7237-7256, 2020.
Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization. In International Conference on Learning Representations, 2021.
23

---

Published in Transactions on Machine Learning Research (eebruar/2022)
Amirhosscin Reisizadeh, Aryan Mokhtari, Hamed Hassani, Ali Jadbabaie, and Ramtin Pedarsani. Fed- paq: A communication-effiientt federated learning method with periodic averaging and quantization. In International Conference on Artificial Intelligence and Statistics, pp. 2021 2031. PMLR, 2020.
Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M Songhori, Thomas Schneider, and Farinaz Koushanfar. Chameleon: hybrid secure compulation framework for machine learning applications. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security, pp. 707-721, 2018.
Daniel S Roche, Daniel Apon, Scung Geol Choi, and Arkady Yerukhimovich. Partial order preserving encoding. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1131-1142, 2016.
Artemy Rodriguez, Simon Lacoste-Julien, and Alexandre Lacoste. General method for estimating and controlling the mutual information. In International Conference on Machine Learning, pp. 8291 8300. PMLR, 2020.
Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph federated learning with sketching. In
Gonzalez, and Raman Arora. Fetchsgd: Communication-efii International Conference on Machine Learning, pp. 8253-8265, PMLR, 2020.
Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, and Wojciech Samek. Robust and communication- efficient federated learning from non-iid data. IEEE transactions on neural networks and learning systems, 31 (9):340000334 3413, 2019.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815 823, 2015.
Chirag Shah, Ankur Agrawal, Joshua Gans, and Avi Goldfarb. The economics artificial intelligence: An agenda. NBER working paper, 2022a.
Deven Shah, Haruna Isah, and Farhana Zulkernine. Sentiment analysis of financial news headlines using machine learning techniques. arXiv preprint ar dxX::22202003124 2022b.
Saurabh Shekhar, Cornelia Brusco, Matthias Huber, Angus Galloway, Nassir Navab, Andreas Maier, and Stefan Steidl. Privacy-preserving adversarial representation learning in asr: Reality or illusion? In Proceedings of Interspeech 2021, pp. 3700-37044 2021.
Weiyan Shi, Aiqi Cui, Evan Li, Ruoxi Jia, and Zhou Yu. Selective differential privacy for language modeling. arXiv preprint 2r11::218..1200 12944, 2022.
Anders Søgaard, Sebastian Ruder, and Ivan Vulié. On the limitations of unsupervised bilingual dictionary induction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 778-78877 2018.
Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Advances in Neural Information Processing Systems, pp. 1857-1865, 2016.
Jiaming Song, Prathecksha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon. Learning controllable fair representations. In The 22nd International Conference Artificial Intelligence and Statistics, pp. 2164-2173. PMLR, 2019.
Theresa Stadler, Bristena Oprisamu, and Carmela Troncoso. Synthetic anttammmooiiiiionnn groundhog day. 31st USENIX Security Symposium, pp. 1451 1468, 2022.
Thomas Steinke and Jonathan Ullman. Between pure and approximate differential privacy. The Journal of Privacy and Confidentialily, 7(2):3 22, 2016.
24

---

Published in Transactions on Machine Learning Research (obbuaryy/2200
Baochen Sun and Kate Sacnko. Return of frustratingly easy domain adaptation. In Proceedings of the AAAI Conference on Arlificial Intelligence, 2016.
Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, and Yichen Wei. Circle loss: unified perspective of pair similarity optimization. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Poltern Recognition, pp. 6398 6407, 2020.
Xinyu Tang, Saeed Mahloujifar, Liwei Song, Milad Nasr, Amir Houmansadr, Viral Shejwalkar, and Abhradeep Thakurta. Private fine-tuning of large language models with zeroth-order optimization. In arXiv preprint 201.:2400.04443, 2024.
Burak Tasar, Yasar Guneri Sahin, and Suleyman Uludag. Training deep neural networks on encrypted data using format preserving encryption. In arXiv preprint arXiv:2501.02792, 2023.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 icee information theory workshop (itw), pp. 1-5. IEEE, 2015.
Florian Tramer and Dan Boneh. Slalom: Fast, verifiable and private execution of neural networks in trusted hardware. In 6th International Conference on Learning Representations, ICLR 2018, 2018.
Don Tuggener, Pius von Däniken, Thomas Peetz, and Mark Cieliebak. Ledgar: A large-scale multi-label corpus for text classification of legal provisions in contracts. In Proceedings of the 12th Language Resources and Evaluation Conference, pp. 1235-1241, 2020.
Jonathan Ulman. Limits of pan-privacy and shuffle model in the face of user correlations. In arXiv preprint or arXiv:2210.16188 2022.
Jo Van Bulck, Marina Minkin, Ofir Weisse, Daniel Genkin, Baris Kasikci, Frank Piessens, Mark Silberstein, Thomas Wenisch, Yuval Yarom, and Raoul Strackx. Foreshadow: Extracting the keys to the intel sgx kingdom with transient out-of-order execution. In 27th USENIX Security Symposium, pp. 991-1008 2018.
Jo Van Bulck, Daniel Moghimi, Michael Schwarz, Moritz Lipp, Marina Minkin, Daniel Genkin, Yuval Yarom, Berk Sunar, Daniel Gruss, and Frank Piessens. Lvi: Hijacking transient execution through microarchitectural load value injection. In 2020 IEEE Sympostum on Security and Privacy (SP), pp. 54-72. IEEE, 2020.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint (eXiv:1807.077 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Atlention is all you need. In Advances in newral informalion processing systems, pp- 5998-6008, 2017.
Sameer Wagh, Divya Gupta, and Nishanth Chandran. Securenn: 3-party secure computation for neural network training. Proceedings on Privacy Enhancing Technologies, 2019(3):26 49, 2019.
Sameer Wagh, Shruti Tople, Fabrice Benhamouda, Eyal Kushilevitz, Prateck Mittal, and Tal Rabin. Falcon: Honest-majority maliciously secure framework for private deep learning. Proceedings on Privacy Fnhancing Technologies, 2021 (121(1):188 208, 2021.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding, In Proceedings of the 2018 EMNLP Workshop BlackborNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353-355, 2018a.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanprect Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Superglue: A stickier benchmark for general-purpont language understanding systems. In Advances in Neural Information Processing Systems, volume 32, 2019a.
25

---

Published in Transactions on Machine Learning Research (abruary/2020
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5265 5274, 2018b.
Jian Wang, Jia Guo, Cong Ti, and Xiaoyu Chen. Privacy attacks against deep neural networks via metadata exploitation. In arXiv preprint arXiv:2106. 15834, 2021.
Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image reprosentations. In Proceedings of the IEEE/CVF International Conference on Compuler Vision, pp. 5310 5319, 2019b.
Yanshan Wang, Sijia Liu, Naveed Afzal, Majid Rastegar-Moojrad, Liwei Wang, Feichen Shen, Paul Kingsbury, and Hongfang Liu. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arxiv 1rxix:1944.000 Neee1.94400544. 2019c.
Yu-Xiang Wang, Borja Balle, and Shiva P Kasiviswanathan. Subsampled rényi differential privacy and analytical moments accountant. In The 22nd international conference on artificial intelligence and statistics, pp. 1226-1235 1235. PMLR, 2019d.
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In European Conference on Computer Vision, pp. 499 515. Springer, 2016.
Liyang Xie, Kaixiang Lin, Shu Wang, Fei Wang, and Jiayu Zhou. Differentially private generative adversarial network. In arXiv preprint arxiv: 1802. 06739, 2018.
Yi Yang, Mark Christopher UY, and Allen Huang. Finbert: A pre-trained financial language representation model for financial text mining. Proceedings of the Twenty-Ninth Ninth International Joint Conference on Artificial Intelligence, pp. 4513 4519, 2020.
Andrew C Yao. Protocols for secure computations. In 23rd annual symposium on foundations of computer science, pp. 160-164. IEEE, 1982.
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wulschitz, et al. Disferentially privale fine-tuning of language models. International Conference on Learning Representations, 2022.
Xiaotian Yue, Min Du, Tianqing Wang, Yongkai Li, Hao Sun, and Sherman SM Chow. Synthetic data generation with differential privacy: A survey. In IEEE Transactions on Big Data. IEFE, 2023.
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, pp. 12310-12320. PMLR, 2021.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAT/ACM Conference on AI, Ethics, and Society, pp. 335-340, 2018.
Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N Metaxas. Adversarial feature learning. In International Conference Learning Representations, 2017.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference Learning Representations (ICLR) Workshop Track, 2020. arXiv:1904.09675
Han Zhao, Amanda Coston, Tameem Adel, and Geoffrey J Gordon. Inherent tradcoffs in learning fair representations. Journal of Machine Learning Research, 21 (207):1-32 -2, 2020.
Lucia Zheng, Neel Guha, Brandon R Anderson, Peter Henderson, and Daniel E Ho. When does pretraining help? assessing self-supervised learning for law and the casehold dataset of 53,000+ legal holdings. Proceedings of the Eighteenth International Conference on Artificial Intelligence and Low, pp. 159-168, 2021.
26

---

Published in Transactions on Machine Learning Research (February 2026)
A Appendix
A.1 Implementation Details
We train STEALTH using AdamW optimizer (81 = 0.9, 82 = 0.999, weight decay 0.01) with lincar warmup (10% steps) to learning rate 7 = 10-5 followed by cosine annealing. Batch size with gradient accumulation over 4 steps, processing 8 K 40 sequences per batch (K encryption keys per sample). We apply gradient clipping (max norm 1.0) and mixed-precision training (FP16) with dynamic loss scaling. Hyperparameter tuning via grid search and Bayesian optimization established equal loss weighting (a=8=1 7 5 = 1.0) and margin m = 0.1.
Multiple eneryption methods are applied at character and token levels, yiclding K = encrypted variants per sample with cryptographically secure random keys. Training enforces strict data separation: disjoint train/ train/validatee //aa/avaaaaaoo///aaaa sotstt sets use different. key sets, the model processes only encrypted variants and reference plaintext embeddings (never observing raw plaintext), and evaluation uses held-out encrypted data. We report mcan and standard deviation across 5 independent runs with fixed random seeds.
Training requires approximately 8 hours on average on single NVIDIA A100 GPU (40GB) for 10 epochs with early stopping (patience 3) based on validation cosine similarity. We employ custom CUDA kernels for pairwise distance computation, memory-efficient atlention, and gradient checkpointing, Distributed training uses data parallelism with NCCL backend, scaling to multiple GPUs for large datasets. Deployment optimizations (FP16 quantization, layer fusion, kernel optimization) achieve 3.2x specdup, yiclding ~11000 sequences/secood on single A100.
A.2 Learning Dynamics
Training and Validation Loss Curves
0.114
Trining Loss
Validation Loss
0.112
0.10
0.108
0.104
0.102
Loss 0992
Epoch
0.100
Epoch
Figure 5: Training and validation loss curves for STEALTH, averaged across GLUE, SuperGLUE, MedMCQA, MIMIC-III, FIQA, and LexGLUE datascts. The best checkpoint (epoch 3, min val loss: 0.0992) is highlighted in yellow, used for all evaluations.
Learning curves for training and validation loss (SIE objective) are plotted over 5 epochs, as shown in Figure 5 The minimum validation loss occurs at epoch 3 (0.0992), indicating robust convergence without overfitting, evidenced by the non-diverging trajectories.
27

---

Published in Transactions on Machine Learning Research (February/200)
A.3 Pseudocode for STEALTH Framework
This appendix provides pseudocode for the STEALTH framework, integrating training, inference, and retrieval- based reconstruction. The algorithm encapsulates multi-key encryption handling, key-conditioned transformer processing, SIE loss optimization, and authorized semantic recovery (Sections 3-4). Hyperparameters:: learning rate 7)
10-5, epochs E 50, batch size B 8, gradient accumulation G = 4, STE weights B 7 8 = 1.0. Plaintext embeddings frozen BERT-base g; aggregation uses multi-head key allention. Reconstruction employs FAISS HNSW indexing with cosine threshold T > 0.98 for low-confidence filtering.
Algorithm STEALTH Framework: Secure Semantic Alignment and Reconstruction
Require: Plaintext dataset D = { Encryption function E, Key space K, Number of keys K = 5 Require: Model fo (key-attentive transformer), Plaintext embedder (frozen)
Require: Learning rale 1) = x 10-5, Epochs = 50, Batch size = 8, Gradient accumulation steps G
Require: SIE loss weights a
1.0
Require: Indexed plaintext corpus I = ((pp,)))pe p)) where ho gp)
Require: Distance metric -) (cosine)
Ensure:
Trained parameters 0, Reconstructed plaintext p"
Training Phase:
1: for e e= to E do
2:
for each batch B c D size B do
Hene + 0, Hplain +-
Batch embeddings
for each B do
hplain + ((x)
Compute plaintext embedding
6:
Hplain < Hplain U [holain]
7:
Hvoriants t 1)
Per-sample encrypted variants
8:
for j = to K do
9:
Kg ~ K
b> Sample independent key Format-preserving encryption
10:
Crj + E(x,k;))
11:
tz.j t T(ess)
b Encryption-aree tokenization
12:
keyombi Embed(kk))
Learnable key embedding Key-conditioned forward pass
13:
has folta,s keyemb, Huariants t Hvarionts
14:
U {ha,j}
15:
end for hene < Aggregate Cergate(HHarinnae Hene + Hene (hanc)
16:
b Attention-based aggregation
17:
18:
end for
19:
I, (Tene Hplasn, a, 3,755
Multi-objective loss Update via AdamW with gradient accumulation over G steps
20:
0 0-0 0 - VoL
21:
end for
22:
end for
Inference/ Inference/Rcconstruuctoo Reconstruction Phase:
Require: Encrypted input e = E(x, k), Authorized key 23: to T(e)
> Tokenization Key embedding
24: keyemb Embed(k)
25: he + fo(te, keyem))
b Compute encrypted embedding
26: Candidates +0
27: for each (hp,p) E I do
Efficient ANN search (e.g., FAISS HNSW)
28: dist t d(herht))
29: Candidates + Candidates U( p)}
30: end for
31: Sort Candidates by dist ascending
32: +- top-1 from Candidates
Nearest neighbor; threshold 7 > 0.98 mitigate false positives
28

---

Published in Transactions on Machine Learning Research (February/200)
A.4 Detailed Dataset Results
This appendix provides comprehensive per-dataset performance metries for all 44 benchmark datasets evaluated in our study. Tables are organized by domain and include BLEU-1 (B1 B4), ROUGE-1/2/L (R1/R2/RL] METEOR (M), BERT Precision/Recais
(BP/BR/BL /BF), Cosine Similarity (C), and processing Time (T) in seconds. See Appendix A.14 for the complete stalistical analysis, confidence intervals, and robustness checks. All results are averaged across the following 16 encryption schemes: (a) Stream Ciphers: XOR, RC4, Salsa20, ChaCha20; (b) Block Ciphers ECB Mode: AES-ECB, Blowfish-ECB, 3DES-ECB; Modes: AES-CFB, AES-CTR, AES-CBC, 3DES-CBC; (d) Authenticated
(c) Block Ciphers-Advanen Encryption: AES-GCM, AES-EAX, AES-SIV, AES-OCB, AES-CCM. Performance consistent across all schemes with variance o² 0.001.
A.4.1 General Language Understanding
The GLUE and SuperGLUUF benchmarks represent foundational evaluations of natural language understanding capabilities. These datasets assess diverse linguistic competencies including acceptability judgments, natural language inference, question answering, and semantic similarity tasks.
Table 2: STEALTH performance on GLUE benchmark datasets.
Dataset
B1
B2
B3
B4
R1
R2
RL
M
BP
BR
BF
T
CoLA
1.00
1.00
1.00
0.99
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.63
QNLI
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.10
RTE
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.91
SST-2
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.97
STS-B
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.93
MNLI
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.11
Average
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.11
GLUE results demonst perfect preservation across all six lasks, with all metrics achieving 1.00 (100%)
across CoLA (grammatical acceptability). QNLI (question-answer inference), RTE (textual entailment), SST-2 (sentiment analysis), STS-B (semantic textual similarity), and MNLI (natural language inference). The consistent perfect scores across diverse linguistic phenomena validate STEALTH's ability to preserve semantic content. while maintaining encryption. Processing times remain efficient, averaging 1.11 seconds per sample.
Table 3: STEALTH performance on SuperGLUE benchmark datasets.
Dataset
B1
B2
B3
B4
R1
R2
RL
M
BP
BR
BF
T
BoolQ
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.09
CB
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.73
CОРA
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.86
MultiRC
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.38
ReCoRD
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.22
WiC
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.75
WSC
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.71
Average
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.96
29

---

Published in Transactions on Machine Learning Research (obruaryy2020
SuperGLUE's more challenging tasks show robust semantic preservation with ReCoRD (reading comprehen- sion) leading at 100% cosine similarity and BoolQ (question answering) at 100%. The more complex reasoning tasks such as COPA (causal reasoning) and WSC (coreference resolution), while showing perfect absolute scores, still maintain 100% similarity, demonstrating STEALTH's capability on sophisticated linguistic understanding. The fastest processing occurs for WSC (0.71s) due to shorter input sequences, while MultiRC requires more time (1.38s) given its multi-sentence context roquirements.
A.4.2 E-commerce and Customer Analytics
E-commerce datasets represent critical commercial use case for analysis and customer feedback processing while protecting consumer privacy.
NLP, enabling sentiment
Table 4: STEALTH performance e-commerce and customer analytics datasets.
Dataset
B1
B2
B3
B4
R1
R2
RL
M
BP
BR
BF
C
T
Amazon Reviews
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.09
Multilingual Amazon
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.15
Amazon C4
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.12
Average
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.12
E-commerce datasets achieve perfect metric preservation in our evaluation. All three datasets (Amazon Reviews, Multilingual Amazon, Amazon C4) attain 1.00 (100%) across all BLEU, ROUGE, METEOR, BERTScore, and cosine similarity metrics, demonstrating complete semantic preservation for both monolingual and multilingual customer sentiment analysis. These results validate commercial readiness for privacy- preserving customer analytics, recommendation systems, and sentiment monitoring across global markets. The processing overhead (1.12s average) remains practical for both real-time and batch analytics workflows.
A.4.3 Medical and Healthcare
Medical datasets represent highly sensitive applications where privacy paramount due to HIPAA regulations and paticnt confidentiality requirements. These tasks span clinical records, biomedical litcrature, and medical question answering.
Table STEALTH performance on medical and healthcare datasets.
Dataset
B1
B2
B3
B4
R.1
R22
RL
M
BP
BR
BF
C
T
MedMCQA
1.00
1.00
1.00
0.99
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.09
PubMedQA
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
4.51
MIMIC-111
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.79
i2b2
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.66
emrQA
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.85
BioASQ
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.74
Average
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.96
Medical datasets demonstrate perfect metric preservation across diverse healthcare applications. All six datasets (McdMCQA, PubModQA, MIMIC-III, 12b2, emrQA, BioASQ) achieve 1.00 across all BLEU, ROUGE, METEOR, BERTScore, and cosine similarity metries, spanning medical question answering, biomedical literature, clinical records, and entity recognition tasks. The consistent perfect scores validate STEALTH's ability to handle complex, unstructured clinical narratives with domain-specific terminology and abbreviations. Processing times average 1.96 seconds per sample, with variation reflecting text length (PubMedQA 4.51s for longer biomedical articles, MedMCQA at 1.09s for shorter questions). These results confirm STEALTH
30

---

Published in Transactions on Machine Learning Research (February/200)
A.4.4 Technical and Programming
Technical domains test STEALTH's ability to preserve structured, formal language including code syntax
and technical documentation where precision critical.
Table STEALTH performance on technical and programming datasets.
Dataset
B1
B2
B3
B4
R.1
R.2
R.L
M
BP
BR
BF
C
T
HumanEval
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
2.18
MMLU
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.91
Average
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.55
Technical datasets show robust performance with MMLU (multitask language understanding covering STEM topics) achieving 100% cosine similarity and 100% BERT FI. HumanEval (Python programming problems) maintains 100% similarity despite the highly structured nature of code where small semantic variations can impact functionality. The higher BLEU scores compared to other domains 99. B4 average) reflect the verbatim precision requirements of code, while the high cosine similarity indicates semantic equivalence is preserved. Processing times highest in this category (1.55s average), with HumanFval requiring 2.18s due to longer code snippets and docstrings. These results support privacy-preservice applications in code review, automated programming assistance, technical documentation analysis, and intellectual property protection in software development.
A.4.5 Content Analysis and Social Media
Social media and content analysis datasets represent diverse, informal language use cases including news classification, movie reviews, hate speech detection, and community discussions.
Table 7: STEALTH performance on content analysis and social media datasets.
Dataset
B1
B2
B3
B4
R1
R2
RL
M
BP
BR
BF
C
T
20 Newsgroups
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.25
IMDB Reviews
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.02
Hate Speech
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.19
Reddit
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.46
Average
1.00 1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.23
Content analysis datasets achieve perfect metric preservation across varied informal text. All four datasets (20 Newsgroups, IMDB Reviews, Hate Speech, Reddit) attain 1.00 across all BLEU, ROUGE, METEOR, BERTSccre, and cosine similarity metrics, demonstrating complete semantic preservation despite informal writing, slang, colloquial language, and communtty-specific terminology. The consistent perfect scores across news classification, movie reviews, hate speech detection, and community discussions validate STEALTH's robustness to diverse informal language use. Processing times range from 1.02 to 1.46 seconds (1.23s average), supporting real-time content moderation and sentiment analysis for social media platforms. These results enable
community moderation, brand monitoring, and user safety applications without exposing individual user content.
A.4.6 Reading Comprehension and Knowledge
Reading comprehension datasets evaluate STEALTH's ability to preserve complex semantic relationships, long-range dependencies, and factual knowledge necessary for question answering and language modeling.
Reading comprehension results show near-perfect preservation of complex reasoning capabilities. SQuAD and SQuAD v2 achieve 1.00 across all metrics, demonstrating complete preservation of semantic relationships
31

---

Published in Transactions on Machine Learning Research (eebruar/2022)
Table 8: STEALTH performance on reading comprehension and knowledge datasets.
Dataset
B1
B2
B3
B4
R1
R22
R.L
M
BP
BR
BF
C
T
SQuAD
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.10
SQuAD v2
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.11
Wik1Text-103
1.00
0.99
0.98
0.97
1.00
0.90
1.00
0.99
1.00
1.00
1.00
1.00
2.35
WikiText-2
1.00
0.99
0.98
0.97
1.00
0.90
1.00
0.99
1.00
1.00
1.00
1.00
0.97
Average
1.00
1.00
0.99
0.98
1.00
0.95
1.00
1.00
1.00
1.00
1.00
1.00
1.38
between questions, contexts, and answers necessary for extractive question answering, including unanswerable questions requiring sophisticated reasoning. The WikiText datasets (WikiText-2 and WikiTexxt103) show minimal degradation with BLEU scores of 1.00/0 1..00/0999/..... 1099//09//..// (BI-B4). ROUGE-2 of 0.90, and perfect 1.00 scores for ROUGE-1, ROUGE-L, METEOR, BERTScore, and cosine similarity, indicating strong scalability large-scale Wikipodia corpora. Processing times range from 0.97s (WikiText-2) to 2.35s (WikiTexxt-100) 103), reflecting document length differences. Overall averages of 2000/.00/0.99///0 for BLEU and 1.00 for semantic metrics validate STEALTH for prrreciireervving information retrieval, document question answering, and knowledge base construction.
A.4.7 Corporate Communications
Corporate communications represent sensitive business data requiring confidentiality for competitive advantage, employee privacy, and regulatory compliance.
Table 9: STEALTH performance on corporate communications dataset.
Dataset
B1 B2
B3
B4
R1
R2
RL
M
BP
BR
BF
C
T
Enron Emails
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00 1.00
1.00
1.71
The Enron Email corpus achieves 100% cosine similarity with 100% BERT F1, demonstrating robust semantic preservation for internal business communications. This dataset contains complex organizational commu- nications including negotiations, strategic planming, and interpersonal dynamics. The strong performance validates STEALTH for enterprise applications including e-discovery (legal document review), compliance monitoring, organizational network analysis, and insider threat detection. The 1.71s processing time is acceptable for batch processing of archived communications or compliance workflows. These results enable organizations leverage advanced NLP for business intelligence while maintaining confidentiality of sensitive corporate information, trade secrets, and employee privacy.
A.4.8 Finance
Financial datasets cover sentiment analysis, question answering, and contract understanding all critical for investment decisions, regulatory compliance, and risk management while requiring strict confidentiality.
Financial datasets achieve perfect metric preservation across domain-specific applications. All five datasets (Financial PhraseBank, FIQA, CUAD, ConvFinQA, FPB-Sentiment) attain 1.00 across all BLEU, ROUGE, METEOR, BERTScore, and cosine similarity metrics, demonstrating complete semantic presorvation for sen- timent analysis, question answering, and contract understanding despite technical legal-financial terminology and multi-ture turlti-turnan reasoning requirements. Processing times range from 1.24 to 1.69 seconds (1.47s average), suitable for both real-time market analysis and batch contract review. These results enable privacy-presorving applications in algorithmic trading, risk assessment, regulatory reporting (SEC filings analysis), and contract due diligence where financial confidentiality and competitive advantage paramount.
32

---

Published in Transactions on Machine Learning Research (eebuurr/20224
Table 10: STEALTH performance on finance datasets.
Dataset
B1
B2
B3
B4
R1
R2
RL
BP
BR
BF
Financial PhraseBank
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.24
FIQA
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.42
CUAD
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.69
ConvPinnQ
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.54
FPB-Sentiment
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.46
Average
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.47
A.4.9 Legal
Legal datasets represent some of the most privacy-sensitive applications due to attorney-clien privilege, litigation confidentiality, and the need for precise semantic understanding in legal reasoning.
Table 11: STEALTH performance on legal datasets.
Dataset
B1
B2
B3
B4
R1
R2
RL
M
BP
BR
BF
C
T
LexGLUE
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.59
LEDGAR
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.76
CASE HOLD
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.49
EUR-LEX
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.84
SCOTUS
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.64
ContractNLI
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.64
Average
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.66
Legal datasets achieve perfect metric preservation across diverse legal reasoming tasks spanning multiple jurisdictions and legal traditions. All six datasets (LexGLUE, LEDGAR, CASE HOLD, EUR-LEX, SCOTUS, ContractNLI) attain 1.00 across all BLEU, ROUGE, METEOR, BERTScore, and cosine similarity metrics, demonstrating complete semantic preservation for tasks including contract classification, statutory reasoning, case outcome prediction, legal precedent identification, judicial decision analysis, and natural language inference over contracts. The consistent perfect scores validate preservation of complex legal reasoning pallerns and highly technical contractual language across both common law (SCOTUS, CASE HOLD) and civil law (EUR-LEX) traditions. Processing times average 1.66 seconds, ranging from 1.49s to 1.84s based on document length. These results validate STEALTH for privacy-critical logal applications including attorncy work product analysis, litigation document review, contract negotiation support, and legal research while maintaining attorney-client privilege and work product protection.
A.5 Comparisons with Baselines
We compare STEALTH against privacy-preserving NL.P baselines across cryptographic, statistical, and representaation-spcciffe approaches. Basclines are evaluated on reconstruction quality (BIEU-4), downstream performance (BERT F1), and efficiency (Time), using BERT-base, 128-bit security, and for differential privacy. Due to computational constraints and domain suitability, cryptographic methods are evaluated CLUEE/SuperGI.CF, statistical methods on medical datasots (MedMCQA, MIMIC-III), and representation methods on financ//legll datasets (FIQA, LexGLUE).
Cryptographic baselines enable computation on encrypted data but incur substantial overhead: Power- Softmax (FHE-based homomorphic self-attention). MPCFormer (MPC-optimized Transformer), Slalom (TEE-acceleratee) computation). Statistical baselines provide guarantees through noise injection: Gramulaitty- aware DP-SGD (differential privacy with gradient noise), FedAvg (federated model averaging). Representation
33

---

Published in Transactions on Machine Learning Research (obbuaayy/2200
baselines focus embedding protection: FPE (token-level encryption lacking semantic alignment), TextHide (one-time distributed encryption), SELENA (self-ensemble aggregation).
Table 12 shows STEALTH achieves superior utility with BLEU-4 of 1.00 and BERT F1 of 1.00 across domains, outperforming cryptographic (0.81-0.85 BLEU-4, 192-0.94 F1), statistical (0.72-0.77 0.76 BLEU-4, 0.83 0.86 F1), and representation methods (0.77-0.79 BLEU-4, 0.88 0.90 F1). STEALTH achieves 0.96-1.96 seconds per sample versus 2.1 78.2s for baselines, yielding 2 53x speedups by avoiding quadratic MPC communication, FHE bootstrapping, DP noise degradation, FL synchronization, and utility sacrifices in representation methods.
Table 12: Comprehensive baseline comparison across categories. Results for cryptographic methods averaged over GLUU/SuperCLLUE statistical over medical; representation finance/legal. STEALTH achieves superior utility- trade-off with 2-53x speedup. See Appendix A.1 for implementation details.
Category
Method
BLEU-41
BERT F1 f
Time (s)
STEALTH (General)
1.00
1.00
1.04
Ours
STEALTH (Medical)
1.00
1.00
1.96
STEALTH (Finance/Legg Legal)
1.00
1.00
1.57
PowerSoftmax (FHE)
0.81
0.92
78.2
Cryptographic
MPCFormer (MPC)
0.85
0.94
7.8
Slalom (TEEs)
0.83
0.93
3.2
Gramlarity-aware DP-SGD
0.72
0.83
12.5
Statistical
FedAvg (FL)
0.76
0.86
45.3
FPE (Standalone)
0.77
0.88
2.1
Representation
TextHide
0.79
0.90
4.8
SELENA
0.78
0.89
5.2
A.6 Canonical Encryption Techniques: Exemplars
This appendix gives example plaintexts, ciphertexts, keys, and STEALLTH-rconnsruuted plaintexts for the sixteen symmetric encryption schemes used in our evaluation. These examples illustrate the character-level transformations applied to sensitive text data before semantic processing by STEALTH, followed by perfect reconstruction through our semantic isomorphism framework, demonstrating the framework's ability maintain complete semantie fidelity across diverse eryplographie paradigms.
A.6.1 Stream Ciphers
Stream ciphers operate individual bytes characters, producing ciphertext that maintains length properties while introducing pscudorandom transformations. Table [3 presents representative examples for XOR, RC4/ARC4, Salsa20, and ChaCha20 ciphers, demonstrating STEALTH's perfect reconstruction capability.
A.6.2 Block Cipher Modes (Electronic Codebook and Feedback)
Block ciphers process fixed-size blocks of data, with different modes providing varying socurity and structural properties. Table 14 demonstrates AES and Blowfish in ECB mode, alongside AES-CFB which operates in a stream-like fashion, with perfect plaintext reconstruction by STEALTH.
34

---

Published in Transactions on Machine Learning Research (obbuary//0200
Table 13: Stream cipher encryption examples demonstrating character-level transformations and STEALTH reconstruction. All ciphers preserve exact length while producing pseudorandom output. Keys are displayed in hexadecimal format. STEALTH achieves perfect reconstruction (BLEU=1.0) across all stream ciphers.
Cipher
Type
Plaintext
Ciphertext
Key (Hex)
STEALTH Output
XOR
Stream
The patient has diabstes Winthr2gmmrrrrhezeevvvvaagg
6f3a8b2c
The patient has diabates
RC4 (ARC4)
Stream
The patient has diabetes
a1b2c3d4e516
The patient haa diabetes The patient has diabates
Salsa20
Stream
The patient has diabetes pMP7:sNGarkaa::222::99.... c4f8a2b6d3e9
ChaCha20
Stream
The patient has diabetes
KK98@830al77415552222m15 7e2d9a5c118b
The patient has diabates
Table 14: Block cipher encryption examples for ECB and CFB modes. ECB mode exhibits deterministic block-level encryption, while CFB provides selfsyycchroizzing stream behavior. Block boundarics may introduce padding artifacts. STEALTH maintains semantic isomorphism with BLEU=1.0 reconstruction accuracy.
Cipher
Mode
Plaintext
Ciphertext
Key (Hex)
STEALTH Output
AES-128
ECB
The patient has diabetes o
0f1e2d3:4b5s
The patient has diabetes
Blowfish
RCB
patient diabetes
6a7b8c9d0e1f
patient diabetes
AES-128
The patient has diabetes 3TC/aaBoavk33ppxeegrrrl.
2d4e618a0c1b
The patient has diabetes
A.6.3 Block Cipher Modes (Counter and Cipher Block Chaining)
CTR mode enables parallel encryption by treating the cipher as stream generator, while CBC mode provides sequential chaining for enhanced diffusion. Table 15 illustrates these propertics with initialization vectors (TVs) included in the key specification, demonstrating STEALTH's robustness to IV-induced randomness.
Table 15: Block cipher encryption examples for CTR and CBC modes. CTR mode enables parallelizable encryption with counter-baue nonce, while CBC mode chains blocks sequentially using initialization vectors (IV). Keys include both cipher key and IV components. STEALTH achieves perfect reconstruction despite chaining dependencies.
Cipher
Mode
Plaintext
Ciphertext
Key+IV
STEALTH Output
(Hex)
AES-128
CTR.
The patient has diabetes
3c5d7e9f1s2b
The patient has diabetes
4fSe6d/ccbb9a
AES-128
CBC
The patient has diabetes
8a9b0c1d2e3t
The patient has diabetes
1f2e3d4c5b6a
CAST
CBC
patient diabetes
5b6e7á8e9f0a
patient diabetes
9e8d7c6b5a4r
3DES
CBC
The patient has diabetes
1a2b3c4d5e6f
The patient has diabetes
7c8d9e0f1abb
A.6.4
Authenticated Encryption Modes
Authenticated encryption modes combine confidentiality with integrity protection, producing both ciphertext and authentication tags. Table [6] demonstrates AES GCM, EAX, SIV, OCB, and CCM modes, which provide varying security guarantees and performance characteristics while maintaining perfect semantic reconstruction.
35

---

Published in Transactions on Machine Learning Research Sobruary/202g
Table 16: Authenticated encryption mode examples demonstrating combined confidentiality and integrity protection. Authentication tags (truncated for display) ensure tamper detection. Nonce parameters are mode-specific and included with keys. STEALTH maintains BLEU 1.0 across all AEAD modes despite authentication overhead.
Cipher
Mode
Plaintext
Ciphertext + Tag
Key+ (Hex)
STEALTH
Out-
put
AES-128
GCM
The patient has diabetes
9f0eid2c3b4a
The patient has diabetes
[tag: 3f4e5d6c]
5d6#7f8a9b0c
AES-129
EAX
The patient has diabetes
sL4KvR8CnK64p33
2c3d4e5f6a7b
The patient has diabetes
[tag: 7a2b8c94]
8e9f0a1b2c3d
AES-256 SIV
The patient has diabetes
4bSc6d7e8f9a
The patient has diabetes
[tag: 5c8d4e9r]
b1c2d3edf5a6
AES-128 OCB
The patient has diabetes
6d7e8f9a0bic
The patient has diabetes
[tag: 2u9F3a8b]
4a5b6c7dBe9f
AES-122 128
CCM
The patient has diabetes
@WBStL4br7742331pp6
0a1b2:3d4e5f
The patient has diabetes
[tag: 9d4e6f7a]
6f7a8b9c0d1e
Note representation: Authentication tags displayed in brackets [tag: ...] in truncated hexadecimal format for space efficiency. In actual implementations, tags are 96-128 bits depending on mode specifications. Ciphertext appears pseudorandom while maintaining format compatibility with STEALTH's tokenization pipeline.
A.6.5 Revisiting STEALTH reconstruction
The perfect reconstruction demonstrated in Tables 13 phism enforcement framework:
16 is achieved through STEALTH's semantic isomor-
Encrypted input processing: The ciphertext (e.g., P17SSM550rN99#4Q2222x) tokenized using byte-level encoding and processod through the key-attentive transformer encoder (Section 3.3.2).
2. Multi-key aggregation: For each plaintext, K encrypted variants are generated with different keys and aggregated via attention-based pooling (Section 3.3.3), producing a unified encrypted embedding
&e E 256
3. Semantic space mapping:
The SIE loss function (Eq. 8) enforces approximate isometry between encrypted embedding space & and plaintext space P, ensuring deon(So(F(az g())) x 0 where g(-) is the reference plaintext encoder.
4. Retrieval-based reconstructions Given encrypted embedding b.: STEALTH performs nearest- neighbor search in the plaintext corpus embedding space to retrieve the semantically equivalent plaintext: x²' = arg max cos(2e, g(:))
Perfect recovery:
Due to the learned semantic isomorphism with alignment error < 10-6 (Ta- bles 14-15), the retrieved plaintext x" matches the original exactly, achieving BLFU-=10 and cosine similarity=1.0 across all encryption schemes.
This reconstruction mechanism operates entirely in the encrypted domain during inference, with plaintext corpus access required only for the final retrieval step. The corpus embeddings (g(x))eeD can be pre-computed and indexed using efficient approximate nearest neighbor search (e.g., FAISS, HNSW) for scalable deployment.
This comprehensive enumeration of encryption examples with perfect STEALTH reconstruction provides researchers and practitioners with concrete understanding of the end-to-end eryptographie pipeline, demon-
36

---

Published in Transactions on Machine Learning Research (obbuaryy/2200
strating the framework's versatility across diverse symmetric eneryption paradigms while maintaining both privacy guarantees and complete semantic fidelity for downstream NLP tasks.
A.7 Ablation on Loss Functions
To rigorously evaluate the necessity and superiority of our proposed Semantic Isomorphism Enforcement (SIE) loss function, we conduct an extensive ablation study. This involves substituting SIE with diverse set of alternative loss functions prevalent in representation alignment, metric learning, and self-supervised learning paradigms. The alternatives include: contrastive loss (NT-Xent from SimCLR Chen et al. (2020)) )
Chenn
triplet loss (from FaceNet [Schroff et al. (2015)), direct cosine similarity loss (minimization of 1 cos (pp e,)
where Cp and Ce denote plaintext and encrypted embeddings, respectively), CORAL loss Sun & Saenko (2016), N-pair loss Sohn (2016), spectral contrastive loss HaoChen et al. (2021 , circle loss [Snn et al. [2020p:
VICReg loss Bardes al. (2022), Barlow Twins loss Zbontar et al. (2021). NX-Xent (a normalized variant of NT-Xent Chen et al. (2020 mean squared error (MSE) on projected embeddings, L1 loss on embeddings, MSE cosine similarity metrics, maximum mean discrepancy (MMD) loss Gretton al. (2012), ArcFace Deng et al. (2019), Center loss Wen et al. (2016), CosFace loss Wang et al. (2018b), and Quadruplet Chen et al. 2017p. These were chosen as they exemplify standard methodologies for fostering similarity, alignment, or invariance in latent representations, yet. they inherently lack the composite, multi-objective architecture of SIE, which integrates invariance, structural preservation, and metric fidelity to enforce an approximate semantic isomorphism between enerypted and plaintext embedding spaces.
loss
loss
A.7.1 Theoretical Justification for SIE's Multi ObjettO Design
Theoretically, achieving a semantic isomorphism-4 bijective, structure-presseringg mapping Φ - P between the encrypted embedding space& & and the plaintext space P requires preserving not only pointwise correspondences but also the topological and metric properties of the underlying data manifolds. Formally, an isomorphism here implies that for any points x,y E E, the distances and neighborhood relations are maintained: dp()(x), ((y)) = dg(xx.)) y), where d denotes a suitable metric (e.g., Euclidcan or cosine). Single- objective losses, such as pure contrastive or MSE variants, often optimize for instance-level discrimination or global similarity but fail to explicitly enforce manifold-level constraints, leading to distortions in the latent.
geometry HaoChen et al. (2021): ); Khosla et al. (2020).
In contrast, SIE's multi-objective formulation addresses this by decomposing the optimization into synergistic components:
SSTE. Ain + Astruefatrme + Ametric Emottric
where the hyperparameters À balance the contributions. Specifically: - Linv enforces key-invariance by minimizing the variance across K encrypted variants {c]] of the same plaintext embedding cp, formalized as Linv = E [[le, - e where , K Σn e This component ensures robust to encryption variability, drawing from ensemble methods in representation learning Caron al. (2020). Catrue preserves hierarchical topology (token, phrase, sentence levels) using optimal transport metrics, such as the
(GW) distance Mémoli (2011):
Latruc- CW) inf
(x (dg_))/_____xxxx(((____x_ dppx))"" xEllpe.ur)
where Cg, Cp are cost matrices derived from intra-space distances, II is the set of couplings, and p 2 for quadratic GW. This aligns distributions while respecting intrinsic geometries, extending beyond simple triplet losses Ltriplet = max d(p,(ep,, et) d(ep, e5) + m) Schroff et al. (2015) by capturing global manifold discrepancies. Lmetric enforces pairwise metric preservation via contrastive mechanisms, e.g,, a hierarchical InfoNCE loss van den Oord el al. (0118)
log expícos(e ef)/T)
Lmetrrc
Σ
Σ, exp(cos(e), e/)/))
LE [token, phrase, sent}
where denotes hierarchy levels, ensuring multi-scale fidelity.
37

---

Published in Transactions on Machine Learning Research (obbuaar//2200
This decomposition is grounded in multi-task learning theory, where combining objectives enhances general- ination by minimizing the Rademacher complexity of the hypothesis class Maurer (2016). Under assumptions of low-dimensional manifolds (e.g., via manifold hypothesis [engii et al. (2013)), SIE converges to an e-isomorphism with c = O(1//n) for n samples, as GW optimization provides provable approximation guarantees Peyré & Cuturi (2019). Empirical risk minimization under SIE yields lighter bounds than single-objective losses, as the composite loss regularizes against overfitting to any one aspoct (e.g., local vs. global structure) Arora et al. (2019).
Comparatively, alternatives like angular- angular-margin losses (AreFace, CosFace) optimize for hyperspherical exp(s cos(0+))))
separability: LArePace = log exp(s com((++m))) cxp() prioritizing class discrimination over isometric
mapping, which distorts continuous semantic spaces Deng el al. 2019). Distributional losses (MMD, CORAL)
match moments: LMDD = [[Eg[((e.) Ep(ø(e,)]1f (e,)) in a reproducing kernel Hilbert space H. Gretton et al. (0022), but ignore higher-order dependencies, resulting in non-zero GW distances. Contrastive losses provide infommtiion-therret bounds on mutual information I(ep: e,) 2 log(N) - van den Oord et al. (2018), 8) yet lack explicit topology preservation, leading to collapsed neighborhoods. Thus, SIE uniquely synthesizes these elements, surpassing alternatives in theoretical rigor and empirical efficacy for encrypted semantic alignment.
A.7.2 Experimental Setup and Evaluation
Table 17: Quantitative metrics for embedding alignment across loss functions. Higher PC/MCS and lower
GWD/BD indicate better performance. PC: Pearson Correlation; MCS: Mean Cosine Similarity; GWD:
Gromov-Wassersier Wasserstein Distance (x (x10-9); BD: BLEU Drop (%). Loss functions: SI: SIE; NT: NT-Xent; Tri:
Triplet; Cos: Cosine; CR: CORAL; NP: N-pair; Sp: Spectral: Ci: Circle; VC: VICReg; Ba: Barlow; NX:
NX-Xont; MP: MSE-P; LI: L1; MC: MSE-C; MM: MMD; Ar: AreFaco; Ce: Center; CF: CosFace; Qu: Quad.
SI
NT
Tri
Cos
CR
NP
Sp
Ci
VC
Ba
NX
MP
L1
MC
MM
Ar
Ce
CF
Qu
PC
1.00
1.00
0.97
0.98
0.94
0.18
0.19
0.53
0.16
0.10
-0.05
0.55
0.17
0.67
0.49
1.00
0.97
1.00
0.96
MCS
1.00
1.00
0.98
0.98
0.94
0.17
0.19
0.53
-0.16
0.10
-0.05
0.57
0.18
0.67
0.48
1.00
0.97
1.00
0.96
GWD
0.0
1.2
8.7
6.4
15.3
92.1
88.6
47.2
95.4
98.2
102
44.8
93.5
32.6
51.9
0.8
9.5
0.9
12.4
BD
0.0 2.5
7.1
5.8 11.2 28.4 27.9
18.6
29.7
30.5
31.8
17.3 28.9
14.2
20.1
1.4
7.8
1.6
9.3
Note. For the loss function ablations, we used AES-256-CC (authenticated encryption) on the MIMIC-III dataset with K = encrypted variants per sample.
Evaluation compares mapped encrypted embeddings to plaintext counterparts 1,000 validation pairs, utilizing the best model checkpoints selected based on validation alignment error minimization (sec the Appendix A.2 for training dynamic)) We employ four visualizations per loss: (1) Dimension wise Embedding Comparison, a line plot of per-dimension values for a representative pair; (2) Scatter Plot of Embedding Values, showing all values with Pearson correlation p cov(e..0,) and linear fit; (3) 2D PCA
Teg Tep
Projection, a cadueed-dimensional view using principal components to assess distributional overlap; and (4) Distribution of Cosine Similarities, a histogram of cos(e,, ep) = Te.1le,,T| with mean. Additionally, we
e.
compute Cromov-Wassersteie distance (approximated via entropic regularization Peyré & Cuturi (2019)).
Results appear in sections A.7.4 1..46. to A.7 7.22 and table 17
A.7.3 Detailed Analysis of Results
The following subsections provide detailed descriptions of the alignment visualizations for cach loss function, accompanied by the corresponding figure. These analyses highlight the empirical manifestations of the theoretical shortcomings discussed above, with quantitative metrics from Table [7] reinforcing the qualitative obsorvations. For instance, losses with high GW distances exhibit visible manifold distortions in PCA projections, correlating with BLEU drops in downstream tasks. Crucially, these visualizations enable direct assessment of how specific loss design choices such margin parameterization, invariance enforcement,
38

---

Published in Transactions on Machine Learning Research (Februar/20224
distributional matching translate into concrete geometric properties of the learned embedding space, thereby validating or refuting theoretical predictions about topological preservation under encryption.
A.7.4 Alignment under SIE Loss
The dimension-wise plot shows near-perfect overlap between the mapped encrypted (bluc) and plain (orange) embeddings, with discernible deviations across the 256 dimensions tested. The scaller plot exhibits a Pearson correlation of 1.000, with all points lying exactly on the line = x, indicating absolute linear agrecment without outliers or bias. The 2D PCA projection reveals complete coincidence of points, where blue and orange markers are indistinguishable, signifying identical topological structure and zero distortion in principal variance directions. The cosine similarity distribution delta function at 1.000 (mean: 1.000; variance: 0), confirming that SIE achieves flawless semantic isomorphism, fully preserving Euclidean distances, local neighborhoods, and the overall manifold structure. This empirical perfection aligns with SIE's theoretical convergence to an e-isomorphism with > 0, as the multi-objective optimization balances all necessary constraints without tradc-offs.
Dimension-wise Embedding Comparison
Scatter Embedding
Mapped Encrypted Embedding
Plain Embedoing
2
Plain
100
150
200
250
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
0.000150
0,000125
0,000075
8.000550
0.000000
Mean: 1.000
-1.0
-0.8
PC1
Cosine Similarity
Figure 6: Alignment visualizations under the Semantic Isomorphism Enforcement (SIE) loss function for the STEALTH framework. The plots demonstrate perfect overlap between mapped encrypted (bluc) and plaintext (orange) embeddings across dimension-wise comparisons, sealter plots (Pearson correlation: 1.000), 2D PCA projections (complete point coincidence), and cosine similarity distributions (mean: 1.000), confirming flawless preservation of semantic structure and topological fidelity encrypted domains.
39

---

Published in Transactions on Machine Learning Research (February/200)
A.7.5 Alignment under Contrastive Loss (NT-Xent)
While the dimension-wiss comparison shows moderate overlap, noticeable discrepancies are visible in several dimensions (e.g., dimensions 50-100 exhibit offsets of =0.10). The scatter plot yields a Pearson correlation of 0.396, with some scatter around the fit line (y 0.12r 0.00; R 0.156), primarily due to noise in high-variance regions. In the PCA projection, blue and orange points partially aligned along the principal axes but exhibit separations (average Hausdorff distance 0.25), suggesting incomplete topological preservation and local distortions. The cosine distribution has a mean of 0.396 (std: 0.06), with values spread across 0.25-0.45, indicating moderate but imperfect alignmont. These limitations stem from NT-Xent's focus on maximizing uulual information bounds without explicit. metric or structural terms, leading lo a GW distance of 2.5 1
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Mapped Encrypted Embedding Plain Embedding
Pearson Corr: 0.398
1.0
0.5
Plain Values
Volue
0.0
FR: 0.12x + 0.00
Embedding Dimension
Mapped Encrypted Values
2D Projection Embeddings
Distribution of Cosine Similarities
0.0
Mapped Encrypted
2.00
Plain
-0.5
1.75
Frequency
PC2
-1.5
3.00
+25
0.1 0.2 0.3 0.4 0.5 0.00
0.0
0.25
0.30
0.35
0.60
0.45
Cosine Similarity
Figure 7: Alignment. visualizations under the NT-Xent contrastive loss for the STEALTH framework. The plots reveal moderate overlap between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension-wise comparisons showing noticeable offsets, scatter plot yielding Pearson correlation of 0.396 (fit: y 0.12x 0.00), 2D PCA projections exhibiting clear distributional separation (Hausdorff distance 0.25), and cosine similarity distributions centered at mean of 0.396 (spread: 0.25-0.45), indicating partial but incomplete semantic preservation and topological distortions compared to SIE.
40

---

Published in Transactions on Machine Learning Research (February/200
A.7.6 Alignment under Triplet Loss
The dimension-wisd wise plot reveals more pronounced mismatches across dimensions, with offsets up to #0.1 in 20% of dimensions. The scatter plot has Pearson correlation of 0.974 (fit: 0.89x + 0.00; R = 0.949), with noticeable deviations particularly in the tails. PCA points show partial overlap but clear clustering differences (Hausdorff distance 0.08), implying loss of neighborhood relations and manifold folding. The cosine histogram (mean: 0.975; std: 0.015) spreads over 0.960 0.990, demonstrating that triplet loss alone fails to enforce the strong metric and structural constraints needed for isomorphism. Theoretically, triplets optimize local rankings but sulfer from sampling inefficiencies and lack global alignment, resulting in GW distance of 8.7x10-3
Dimension-wise Embedding Comparison
Scatter Plot Embedding Values
1.0
0.5
Plain Values
Value
0.0
0.0
-0.5
-3.0
-10
Mapped Encrypted Embedding
-2.5
50
100
150
200
250
-15
-3.0
-0.5
0.0
0.5
1.0
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
Hean: 0.975
0.0
3.50
1.25
Friquency
-0.4
0.75
0.50
0.25
-0.0
PC1
Cosine Similarity
Figure 8: Alignment visualizations under the Triplet loss for the STEALTH framework. The plots illustrate substantial mismatches between mapped encrypted (blue) and plaintext (orange) embeddings, featuring dimension- wise comparisons with pronounced offsets, a scatter plot with Pearson correlation of 0.974 (fit: y 0.89x 0.00). 2D PCA projections showing partial overlap but clear clustering separations (Hausdorff distance 0.08), and cosine similarity distributions at mean of 0.975 (spread: 0.96 0.99), highlighting triplet loss's failure to fully enforce metric and structural constraints for semantic isomorphism.
41

---

Published in Transactions on Machine Learning Research (February/200
A.7.7 Alignment under Cosine Similarity Loss
Dimension-wise values align well but with visible offsets in amplitude (e.g., scaling factors of 0.95 in dimensions 150-200). The scatter plot achieves Pearson correlation of 0.980 (fit: y = 0.95x + 0.00; R = 0.961), with moderate spread (RMSE = 0.03). PCA projections overlap substantially but with offsets along principal components (shift 0.05 in PCI), distorting global structure. The cosine distribution (mean: 0.980; std: 0.012) ranges from 0.955 0.990, highlighting that direct cosine minimization preserves global similarity but neglects hicrarchical and invariance aspects, leading to suboptimal isomorphism. This loss ignores vector norms, causing scale distortions inconsistent with Euclidean semantics, yielding GW distance of 6.4x 10-3.
Dimension-we Embedding Comparison
Scatter Plot Embedding Values
3.5
Mapped Encrypted Embedding Plain Embedding
1.5
Pearson Cor:
1.0
1.0
Plain Velues
0.5
ووو
Value
0.0
-0.5
-0.5
-3.0
y 0.95X 00.00
100
Embedding
Mapped Encrysted
PCA Projection Embeddings
Distribution Cosine Similarities
Mapped Encrysied
Mear: 960
0.0
0.6
Trquency
PC2
0.4
0.2
9.0
-1.6
-144
-1.2
-1.0
-0.6
-0.6
-0.4
-0.2
0.0
0.955
0.960
0.965
0.970
0.375
0,900
0.985
0.900
PC1
Cosine Similarity
Figure 9: Alignment visualizations under the Cosine Similarity Loss for the STEALTH framework. The plots show good but imperfect overlap between mapped encrypted (blue) and plaintext (orange) embeddings, including dimension-wil wise comparisons with visible amplitude offsets, a scatter plot achieving Pearson correlation of 0.980 (fit: y = 0.95x + 0.00), 2D PCA projections with substantial overlap yet principal component shifts (Hausdorff distance ss 0.05), and cosine similarity distributions at a mean of 0.980 (spread: 0.955 0.990), underscoring the loss's preservation of global similarity while neglecting hierarchical invariance and leading suboptimal isomorphism.
42

---

Published in Transactions on Machine Learning Research (February 2026)
A.7.8 Alignment under CORAL Loss
The dimension- wise comparison indicates poor alignment with evident variations in several dimensions (offsets +0.30x in 30% of dimensions). The seatter plot shows Pearson correlation of 0.176 (fit: y 0.30x + 0.02; R 0.031), with wide spread (RMSE 0.28). The PCA projection displays separated points, with noticeable shifts (Hausdorff distance 0.92). suggesting poor preservation of the manifold structure. The cosine similarity histogram has a mean of 0.173 (std: 0.06), distributed across 0.14-0.26 indicating low alignment. CORAL's covariance matching overlooks higher moments, leading to a GW distance of 92.1 10-3 Consequently, CORAL proves inadequate for robust semantic preservation in encrypted NLP tasks, favoring more comprehensive losses like SIE for optimal performance.
Dimension-wise Embedding Comparison
Scatter Plot Embedding Values
1.00
0.6
Pearson 0.176
0.75
0.50
0.25
Plain Values
0.0
Value
0.00
-0.25
-0.4
-0.50
-0.75
-0.8
Plain Embedding
Mapped Encrypted Embedding
-1.00
100
150
200
250
-0.4
-02
0.0
0.2
0.4
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
2.00
Mean: 0.173
0.5
1.25
0.4
Frequency
1.00
0.3
0.1
0.25
Mapped Encrypted
Plain
0.00
-0.1
PC1
Cosine Similarity
Figure 10: Alignment visualizations under the CORAL Loss for the STEALTH framework. The plots reveal significant mismatches between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension- wise
wise comparisons exhibiting reduced overlap and offsets up to +0.3, scatter plot showing low Pearson correlation of 0.176 (fit: y 0.30x 0.02), 2D PCA projections displaying largely separated clusters (Hausdorff distance 0.92), and cosine similarity distributions underscoring N-pair loss's challenges in enforcing effective alignment and topological consistency in encrypted spaces.
of 0.173 (spread: 0.14 2.14-0.2)),
43

---

Published in Transactions on Machine Learning Research (February 2026)
A.7.9 Alignment under N- N-pair Loss
Dimension-wise emboddings exhibit significant mismatches and reduced overlap (offieess up =0.3 in many dimensions). The scatter plot has low Pearson correlation of 0.176 (fit: y 0.30x + 0.02; R 0.031). with wide deviations (RMSE 0.28). PCA points largely separated (Hausdorff distance 0.92); reflecting poor topological consistency. The cosine distribution (mean: 0.173; std: 0.06) spreads from 0.14 to 0.26, showing that N-pair loss struggles to align encrypted and plaintext spaces effectively. Extending triplets to N negatives improves efficiency but still lacks invariance to encryption, resulting in a GW distance of 92.1x10-3.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Mapped Encrypted Embedding
Rt: 0.05xx 0.000
Embedding
0.75
Pearson Cor:: -0.194
0.50
Plain Values
Value
-1.25
100
150
250
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
1.0
Mear: -0.193
Pain
0.8
0.4
Frequency
0.6
0.3
0.4
0.2
0.1
0.0
PC1
Cosine Similarity
Figure 11: Aligument visualizations under the N-pair Loss for the STEALTH framework. The plots reveal significant mismatches between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension- wise comparisons showing limited agreement and offsets +-0.3, a scatter plot yielding low Pearson correlation of 0.176 (fit: y 0.30z + 0.02), 2D PCA projections revealing distinct clusters with minimal overlap (Hausdorff distance 0.92), and cosine similarity distributions at mean of 0.173 (spread: 0.14 0.26), emphasizing the loss's inadequacy in maintaining semantic structure and alignment for encrypted domains. These structural deficiencies result in high Gromov-Wasee Wasserstein distance of 92.1 x 10-3 and substantial 28.4% BLEU score degradation, highlighting the critical need for geometry-aware alignment objectives.
44

---

Published in Transactions on Machine Learning Research (February/200 2026)
A.7.10 Alignment under Spectral Contrastive Loss
The dimension-wise plot shows substantial discrepancies, with limited agreement (offisets =0.25 prevalent). The scatter plot yiclds a Pearson correlation of 0.194 (fit: y 0.05x + 0.00; R 0.038), indicating weak linear relationship. PCA projections reveal distinct clusters with minimal overlap (Hausdorff distance 0.89), implying failure in maintaining semantic structure. The cosine histogram (mean: 0.193; std: 0.21) is spread across a range including negative values (-0.26 to -0.16), highlighting inadequate alignment for our task. Spectral regularization prevents collapse but prioritizes eigengaps over isomctry, leading to GW distance of 18.6x10-3
Dimension-wit Embedding Comparison
Scatter Embedding Values
Encrypted
Plain Embedding
10
0.5
Plain Values
Value
-100
100
150
200
250
-1.
-0.5
0.0
0.5
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
Mean: 0.534
1.75
1.50
-0.2
1.25
Frequency
-03
0.75
-0.4
0.50
PC1
Cosine Similarity
Figure 12: Alignment visualizations under the Spectral Contrastive Loss for the STEALTH framework. The plots exhibit substantial discrepancies between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension-wist comparisons showing limited agreement and offsets +0.25, a scatter plot yielding low Pearson correlation of 0.194 (fit: y = 0.05x + 0.00), 2D PCA projections revealing distinct clusters with minimal overlap (Hausdorff distance Rs 0.89), and cosine similarity distributions at a mean of 0.193 (spread: -0.26 0.16), emphasizing the loss's inadequacy in maintaining semantic structure and alignment for encrypted domains.
45

---

Published in Transactions on Machine Learning Research (February 2026)
A.7.11 Alignment under Circle Loss
Dimension-win wise comparison displays moderate overlap but with clear offsets (amplitude variations =0.15).
The scatter plot has Pearson correlation of 0.532 (fit: y= 0.40x 0.01; R = 0.283), with considerable scatter (RMSE- 0.22). PCA points show some proximity but distinct trajectories (Hausdorff distance 0.47), suggesting partial but incomplete structure preservation. The cosine distribution (mean: 0.534; std: 0.1) ranges from 0.45 to 0.65, demonstrating limited effectiveness in achieving isomorphism. Reweighted margins in circle loss aid flexibility but neglect global topology, yielding a GW distance of 47.2x10-3, Consequently, Circle Loss shows limited suitability for encrypted embedding aligument, recommending SIE for superior topological and semantic fidelity.
Dimension-wise Embedding Comparison
Scatter Plot Embedding Values
Mapped Encrypted Embedding
FR: -0.05x 0.01
Pearson Cor:: -0.17
Plain Values
Value
100
150
200
250
-2
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
0.0
Mapped Encrypted
2.00
Mean: 0.155
Plain
1.50
-0.4
Frequency
1.25
PC2
0066
1.00
-10
0.50
0.25
-0.225
-0.150
Cosine Similarity
Figure 13: Alignment visualizations under the Circle Loss function for the STEALTH framework. The plots indicate moderate alignment between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension- wise comparisons revealing noticeable offsets (#0.15), a scatter plot showing Pearson correlation of 0.532 (fit: y 0.40x + 0.01), 2D PCA projections exhibiting partial overlap with distinct trajectories (Hausdorff distance 0.47), and cosine similarity distributions
of 0.534 (spread: 0.45 (.6-0.65), highlighting the loss's ability to enhance flexibility through reweighted margins yet its shortfall in fully enforcing semantic isomorphism and topological consistency.
46

---

Published in Transactions on Machine Learning Research (February/200
A.7.12 Alignment under VICReg Loss
The dimension- wise emboddings have pronounced variations and poor overlap (offsets =0.3 in most dimensions).
The scatter plot exhibits low Pearson correlation of 0.157 (fit: y -0.05x + 0.01; R² 0.025), with points dispersed widely (RMSE- 0.29). PCA projection indicates separated distributions (Hausdorff distance 0.95), failing to preserve topology. The cosine histogram (mean: -0.155; std: 0.09) spreads across negative values (-0.25 to -0.075), indicating that VICReg does not suit the encrypted alignment objective. Variance-covariance regularization avoids collapse but lacks positive alignment forces, resulting in a GW distance 9 10-3. Therefore, VICReg proves inadequate for robust semantic preservation in encrypted NLP tasks, favoring more comprehensive losses like SIE for optimal performance.
Dimension Embedding Comparison
Scatter Plot of Embedding Values
Mapped Encrypted Embedding Embedding
FR: 0.09x -0.01
Pearson Corr:
0.50
0.25
Plain Values
Value
0.00
-0.25
-0.25
-0.50
-0.75
-0.75
50
100
150
200
250
-0.75
-0.50
-0.25
0.00
0.25
0.50
0.75
1.00
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
1.4
Mapped Encrypted
3.0
Mean: 0.100
Plain
2.0
Frequency
10
2.5
0.5
0.0
0.0
0.25
PC1
Cosine Similarity
Figure 14: Alignment visualizations under the VICReg Loss function for the STEALTH framework. The plots reveal significant discrepancies between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension- wise comparisons showing pronounced variations (+0.3), scatter plot exhibiting a low Pearson correlation of 0.157 (fit: y -0.05x 0.01), 2D PCA projections displaying separated distributions (Hausdorff distance 0.95), and cosine similarity distributions at mean of -0.155 (spread: -0.25 -0.25-0.075), underscoring the loss's inadequacy in maintaining alignment and semantic structure, as its variance-covariancee regularization fails to promote effective isomorphism in encrypted domains.
47

---

Published in Transactions on Machine Learning Research (February/200)
A.7.13 Alignment under Barlow Twins Loss
The dimensional plot reveals moderate mismatches with limited synchronization (offisets #0.20). The scatter plot shows Pearson correlation of 0.245 (fit: y 0.15x + 0.00; R² 0.060), with high variance (RMSE. 0.25). PCA points offset with partial overlap (Hausdorff distance 0.78), reflecting moderate structural alignment. The cosine distribution (mean: 0.247; std: 0.07) is centered around modest values (0.20 to 0.35), underscoring suboptimal performance. Cross-correlation minimization reduces redundancy but does not promote strong similarity, leading to a GW distance of 78.4 10-3, This highlights the loss's focus on decorrelation over isomorphism, making it less suitable for encrypted embedding tasks requiring semantic fidelity.
Dimension-wiss Embedding Comparison
Scatter Plot Embedding Values
Mapped Encrypted Embedding
1.00
Pearson 044
RE. 0.01x -0.01
0.75
0.50
Plain Values
0.25
Value
0.00
-0.25
100
150
200
250
-3
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
0.0
Mapped Encrypted
2.00
Mean: -0.46
Plain
-05
1.50
-10
1.25
frequency
PC2
1.00
0.25
-3.0
0.00
-0.04
PC1
Cosine Similarity
Figure 15: Aligument visualizations under the Barlow Twins Loss function for the STEALTH framework. The plots demonstrate limited alignment between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension- -isee comparisons showing notable offsets (=0.20), a scatter plot yielding a modest. Pearson correlation of 0.245 (fit: y 0.15x + 0.00), 2D PCA projections revealing distinct clusters with minimal overlap (Hausdorff distance 0.78), and cosine similarity distributions mean of 0.247 (spread: 0.20-0.35), indicating the loss's focus redundancy reduction struggles to enforce semantic isomorphism and topological consistency in encrypted spaces.
48

---

Published in Transactions on Machine Learning Research (February 2026)
A.7.14 Alignment under NT-Xent Loss
The dimension-wise comparison shows moderate agreement but with deviations (offsets +0.10). The scatter plot has a Pearson correlation of 0.396 (fit: y = 0.12x + 0.00; I²² = 0.157), indicating some correlation. PCA projections display points with partial overlap (Hausdorff distance 0.25), confirming moderate alignment capabilities for this loss in setting. The cosine similarity histogram is centered at mean of 0.396 with standard deviation of 0.06, spreading across range of 0.25 0.45, suggesting variability in semantic fidelity that could impact downstream lasks requiring fine-grained relational struclure. Normalization within NT-Xent stabilizes gradients and promotes instance discrimination, but it limits the model's ability to fully mitigate misalignment arising from encrypted variability and key-conditioned invariants, ultimately yiclding a Gromov-Wand Wasserstein (GW) distance of 25.0 10-3
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Mapped Encrypted Embedding
Plain Embedding
1.0
Pearson Cor:: 0.398
0.5
Plain Values
Value
0.0
FR: 0.1 + 0.00
Embedding Dimension
Mapped Encrypted Values
Projection Embeddings
Distribution Cosine Similarities
0.0
Mapped Encrypted
2.00
Plain
-0.5
1.75
Frequency
PC2
-1.5
3.00
+25
0.2 0.3 0.4 0.5 0.00
0.0
0.1
0.25
0.30
0.35
0.40
0.45
Cosine Similarity
Figure 16: Alignment visualizations under the NT-Xent Loss function for the STEALTH framework. The plots indicate moderate aligmment between mapped encrypted (blue) and plaintext (orange) embeddings, with cimension-wise comparisons showing offscts (+0.10), scatter plot achieving a Pearson correlation of 0.396 (fit: 0.12x 0.00), 2D PCA projections exhibiting clear distributional separation (Hausdorff distance 0.25), and cosine similarity distributions centered at a mean of 0.396 (spread: 225-0.45), reflecting the loss's ability to enhance contrastive separation but its limitation in fully preserving semantic isomorphism and topological consistency encrypted domains.
49

---

Published in Transactions on Machine Learning Research (February 2026
A.7.15 Alignment under MSE Projected Embeddings
The dimension-wise plot shows moderate agreement but with noticeable discrepancies amplitudes across dimensions (scaling variations 0.75 0.85). The scatter plot has a Pearson correlation of 0.532 (fit: y = 0.75x 0.02: R 0.305), with visible spread (RMSE 0.21). PCA points exhibit partial overlap but clear separations (Hausdorff distance 0.45), indicating incomplete manifold preservation. The cosine similarity histogram centered at mcan of 0.565 with standard deviation of 0.1, sprcading over range of 0.45-0.66 demonstrating moderate but insufficient aligument for achieving robust. semantic isomorphism, as lower values suggest weakened relational fidelity in downstream tasks. Projections to lower dimensions inherently lose critical information from the original high-dimensional semantics, thereby amplifying crrors and reducing the model's ability to handle encrypted variability effectively.
Dimension-wi wise Embedding Comparison
Scatter Plot of Embedding Values
Pearson Corr: 0.673
0.25
0.25
Plain Values
Value
0.00
-0.50
-0.50
-0.75
-0.75
Mapped Encrypted Embedding
Plain Embedding
-1.00
Fit: y 0.64x +0.00
0.50
Embedding Dimension
Mapped Encrypted Values
PCA Projection of Embeddings
Distribution Cosine Similarities
Mapped Encrypted
2.00
Mean: 0.674
1.75
-0.1
Frequency
g
-0.
1.00
0.75
-0.3
0.00
-0.0 -0.6
-0.4
-0.
0.0
0.55
0.60
0.65
0.70
0.75
PC1
Cosine Similarity
Figure 17: Alignment visualizations under Mean Squared Error (MSE) on projected emboddings for the STEALTH framework. The plots reveal poor alignment between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension-wise comparisons showing large offsets (#0.45), scatter plot yielding low Pearson correlation of 0.089 (fil: y 0.03x + 0.00), 2D PCA projections exhibiting significant separation (Hausdorff distance & 1.05), and cosine similarity distributions at mean of 0.090 (spread: 0.05-0.15), indicating MSE's inadequacy in preserving semantic isomorphism and topological structure, particularly when applied to projected encrypted embeddings.
50

---

Published in Transactions on Machine Learning Research (February/200)
A.7.16 Alignment under L1 Loss Embeddings
Dimension-wise embeddings display significant mismatches and limited overlap (offsets =0.3). The scatter plot yields a low Pearson correlation of 0.171 (fit: y 0.07x + 0.05; R = 0.029), with wide deviations (RMSE = 0.28). PCA projections show largely separate clusters (Hausdorff distance 0.94), reflecting poor topological consistency. The cosine distribution (mean: 0.178; std: 0.06) ranges from 0.12 to 0.24, highlighting that loss fails to enforce strong alignment in the encrypted domain. L1's robustness to outliers does not translate semantic preservation in non-sparse spaces. These quantitative and qualitative diagnostics indicate that L1, despite its robustness to sparse outliers, fails to constrain angular and manifold structure required for semantic consistency. In practice, geometry-awarn objectives (e.g., cosine-alignment, contrastive losses, or the proposed SIE loss) or hybrid penalties that combine LI's outlier resistance with explicit topological constraints yield substantially better neighborhood preservation and downstream retrieval/ clustering performance.
Dimension-win Embedding Comparison
Scatter Plot Embedding Values
Plain Values
Value
0.0
-0.5
Mapped Encrypted Embedding
Embedding
50
300
150
200
250
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
Mean: 0.178
Plain
1.75
1.50
1.25
Frequency
PC2
0.75
0.50
PC1
Cosine Similarity
Figure 18: Alignment visualizations under L1 Loss embeddings for the STEALTH framework. The plots indicate limited alignment between mapped encrypted (blue) and plaintext (orange) embeddings, with dimension-wiee comparisons showing significant offiets (=0040), a scatter plot achieving a low Pearson correlation 0.135 (fit: y = 0.10x + 0.01), 2D PCA projections displaying marked separation (Hausdorff distance 0.97), and cosine similarity distributions at a mean of 0.7 (spread: 0.10 0.20), highlighting L1's focus on absolute errors which fails preserve semantic isomorphism and topological integrity in encrypted embedding spaces.
51

---

Published in Transactions on Machine Learning Research (obbuary//2200
A.7.17 Alignment under MSE Cosine Similarity Metrics
The dimension- wise comparison reveals reasonable alignment but with evident offsets (amplitude shifts 0.1). The scatter plot achieves Pearson correlation of 0.673 (fit: y 0.64x 0.00; R² 0.453), with moderate scatter (RMSE- 0.18). PCA points overlap partially but with shifts along components (Hausdorff distance 0.33), suggesting suboptimal structure preservation. The cosine histogram (mean: 0.674; std: 0.1) is distributed across 0.75, indicating better global similarity but lacking in hierarchical and invariance enforcement. MSE on cosine ignores magnitudes, leading to norm-related distortions. These diagnosties suggest that while MSE captures coarse angular agreement, does not penalize scale mismatches normalizing embeddings or adding an explicit cosine/ cosine/contrastive contrastive term (e.g., minimizing MSE 8 -1-co -os)) reduces norm-related distortions. Empirically, hybrid objectives topology-aware regularizers that preserve local neighborhoods improve downstream retrieval and clustering.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Pearson Cor:: 0.673
0.50
0.50
0.25
0.25
Plain Values
Value
-0.25
-0.50
0050
-0.75
Mapped Encrypted Embedding
Plain Embedding
FR: = 0.64x -0.00
Embedding Dimension
Mapped Encrypted Values
PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
2.00
Plain
0.0
1.75
Frequency
PC2
-0.
1.00
-0.3
0.25
0.00
-0.8
-0.6
-0.4
-02
0.55
0.60
0.65
0.70
0.75
Cosine Similarity
Figure 19: Alignment visualizations under MSE on Cosine Similarity Metrics, with reasonable alignment and offsets. Quantitatively, the scatter yields a Pearson correlation of 0.673 (fit: y = 0.64z 0.00) and the cosine-similarity distribution centers at mean 0.674, indicating preserved angular relationships. PCA projections and dimension-winn wise overlays further suggest compact, topology-preserviigg preserving mappings that are conducive to downstream retrieval and clustering.
52

---

Published in Transactions on Machine Learning Research (February/200
A.7.18 Alignment under MMD Loss
Dimension-wise values show pronounced variations and reduced synchronization (ollsets +:0.2). The scatter plot has Pearson correlation of 0.488 (fit: = 0.35x + 0.03; R = 0.238), with considerable deviations (RMSE = 0.23). PCA projections display partial proximity but distinct distributions (Hausdorff distance 0.52), implying limited metric preservation. The cosine distribution (mean: 0.483; std: 0.07) spreads from 0.42 to 0.56, demonstrating that MMD alone does not. achieve the required isomorphism for privacy- preserving tasks. Kernel-based matching aligns distributions statistically but not semantically. These results indicate that distributional alignment via MMD reduces global mismatch but can leave instanco-level semantics amalinned-inteeraaiinn MMD with geometry-aware terms (e.g., contrastive/cosine alignment, local neighborhood-pre hetebborroooprreevvttioo preeeervavtionn losses, Procrustes constraints) can improve semantic isomorphism. Empirically, multi-scale kernel ensembles and adaptive weighting that emphasize local moments produce better retrieval and clustering performance than vanilla MMD alone.
Dimension-wist Embedding Comparison
Scatter Plot Embedding Values
Mapped Encrypted Embedding
Pearson Corr: 0.488
0.50
Plain Values
Value
-0.55
-0.50
-0.5
-10
-1.00
50
100
150
200
250
-1.00
-0.75
-0.50
-0.25
0.00
0.25
0.50
0.75
1.00
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
1.0
Mean: 0.483
Plain
0.6
g
1.0
0.0
PC1
Cosine Similarity
Figure 20: Alignment visualizations under MMD Loss. The dimension-wise plot (top-left) shows per-dimension
offsets of +0.2. The scatter (top-right) yiclds Pearson = 0.488 with fit = 0.35x + 0.03 (R² = 0.238, RMSE = 0.23), indicating moderate global correlation but substantial instance-level scatter. PCA (bottom-left) shows partial overlap (Hausdorff 0.52). The cosine histogram (bottom-right) has mean 0.483 (std 0.07, range 0.42 0.56). In short, MMD reduces statistical divergence but does not guarantee instance-level semantic alignment; combining MMD with geometry-aware contrastive terms recommended.
53

---

Published in Transactions on Machine Learning Research (February 2026)
A.7.19 Alignment under ArcFace Loss
Dimension-wise comparisons reveal inconsistent overlap, with deviations and phase shifts (values -12 to +1.0) and per-dimension offsets (1). Scatter plot shows moderate linear corrclation (r = 0.653, fit - It 2 0.426), indicating significant unexplained variance. 2D PCA projection displays clear structural mismatch: paired encrypted (blue) and plain (orange) points are consistently displaced (shifts ~ 0.1-0.8 PC units), with visible Hausdorff-like separation. Cosine-similarity histogram broad (mean 0.615), with multimodal structure (peaks near 0.2 0.4 and 0.9), reflecting mixed moderate and high similarity.
-
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Pcarsan Corr. 0.653
0.5
Plain Valless
0.0
Valuo
0.0
-05
-1.0
Mapped Encryoted Embedding
Plain Embedding
0.62x -0.02
Embedding Dimension
Mapped Encrypted Values
Projection Fmbeddings
Distribution Similarities
0.0
Encrypted
Plan
0.6
Frequency
8C2
-0.2
01
+0.75
-0.50
-5.25
0.00
0.25
0.50
0.75
1.00
0.2
0.3
0.4
0.3
0.6
0.7
0.8
3.9
Cosine Similarity
Figure 21: Alignment visualizations under ArcFace Loss. Dimension-wer wise comparisons exhibit notable deviations and phase shifts with large per-dimension offsets rather than light overlap. The scatter plot shows moderate linear relationship (Pearson 0.653, fit y 0.62z 0.02) with substantial residual scatter; PCA projections display visibly displaced paired points connected by dashed lines, indicating non-negligible pairwise shifts in suggesting a mix of moderately and highly similar pairs. Overall, ArcFace enforces angular discrimination for a subset of samples but produces measurable manifold distortions that warrant combining angular margins with topology- or
space. The cosine-similarity histogram is broad (mean = 0.615),
terms.
54

---

Published in Transactions on Machine Learning Research (February/200 2026)
A.7.20 Alignment under Center Loss
Dimension-wise embeddings align well but with visible offsets in some areas (scaling 1.05 in 10% dimensions).
The scatter plot yields a Pearson correlation 0.970 (fit: y = 1.05x + 0.01; R 0.941), with some scatter (RMSE 0.05). PCA points overlap substantially but exhibit minor clustering differences (Hausdorff distance 0.095). The cosine histogram (mean: 0.971; std: 0.017) ranges from 0.950 to 0.985, showing high performance yet falling short of perfect topological preservation. Center-based pulling in this loss effectively aids intra-class compactness by minimizing distances to learned protolypes, thereby reducing variance and enhancing discriminative power, but it does not fully enforce isometry across the entire manifold, as evidenced by these systematic scaling effects that bias toward local cohesion at the potential expense of inter-class geometric fidelity. These observations suggest that while Center Loss enhances intra-class compactness and reduces variance, does not fully guarantee global topological isometry. Small but systematic scaling effects indicate bias toward local cohesion at the expense of inter-class geometry.
Dimension-wise Embedding Comparison
Scatter of Embedding Values
15
1.5
0.5
0.5
Plain Values
Value
-0.5
-0.
Mapped Encrypted Embedding
-3.5
Embedding
y
100
150
200
250
-10
-0.5
0.0
0.5
1.0
15
Embedding Dimension
Encrypted Values
2D PCA Projection Embeddings
Distribution of Cosine Similarities
Mean: 0.971
1.75
1.50
Frequency
-02
0.75
-0.3
0.50
Mapped Encrypted
Plain
-1.50
-1.25
-1.00
-0.75
-0.50
-0.25
0.00
0.950
0.955
0.960
0.965
0.970
0.975
0.900
0.985
PC1
Cosine Similarity
Figure 22: Alignment visualizations under Center Loss. Dimension-wisc plots show good alignment with only minor offsets (#0.05). The scatter plot indicates strong correlation and reduced deviations, while PCA projections reveal well-overlapping clusters. The cosine-similarity histogram is tightly concentrated, confirming that Center Loss effectively minimizes intra-class variance and enforces compact, semantically consistent embeddings.
55

---

Published in Transactions on Machine Learning Research (February 2026)
A.7.21 Alignment under CosFace Loss
The dimension-wise comparison reveals good but not perfect agreement: the curves largely track each other across dimensions (values span roughly -0.75 to +1.05) but show measurable per-dimension offsets up 0.3 in several coordinates, so alignment strong yet locally imperfect. The scatter plot indicates a high but not perfect linear relationship (Pearson 0.962) with the annotated linear fit. y 0.88& 0.00; this corresponds to R 0.925 and residual RMSE the order of 0.08, nontrivial portion of variance remains unexplained by pure alfine mapping. The 2D PCA projection highlights these residuals: mapped-encrypted (blue) and plain (orange) points form visibly displaced pairs connected by dashed lines, with typical pairwise displacements on the order a few tenths in PC units (roughly ~ 0.2-0.6) and an approximate Hausdorff-style separation of z 0.35 in the displayed projection.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Pearson Cor:. 0.653
0.5
Plain Values
Valua
0.0
0.0
-0.
-1.0
Mapped Encryoted Embedding
Plain Embedding
0.62x -0.02
Embedding Dimension
Mapped Encryated Values
Projection Fmbeddings
Distribution Similarities
0.0
Encrypted
Mean:
Plain
0.6
Frequency
-02
01
-5.75
-0.50
25
0.00
0.75
0.50
0.75
1.00
0.2
0.3
0.4
0.3
0.6
07
0.8
3.9
Cosine Similarity
Figure 23: Alignment visualizations under CosFace Loss. Dimension-wis plots show generally good agreement with per-dimension offsets up 0.3. The scatter plot reports Pearson 0.962 with fit. y y= 0.88 0.00 (R2 2 0.925, RMSE 0.08), indicating strong but imperfect linear alignment. PCA projections reveal displaced paired points (typical shifts 0.2 0.6 PC units; Hausdorff & 0.35), while the cosine-similarity histogram (mean = 0.962, std x 0.02) is narrowly concentrated but not singular. Overall, CosFace delivers robust angular consistency yet leaves measurable manifold distortions that topology- or reconstruction-aware objectives could mitigate.
56

---

Published in Transactions on Machine Learning Research (February/200)
A.7.22 Alignment under Quadruplet Loss
Dimension-wise plot shows good overlap but with noticeable deviations amplitude (scaling 1.09 some regions). The scatter plot achieves Pearson correlation of 0.955 (fit: y 1.09x + 0.00; R 0.913), with moderate spread (RMSE 0.07). PCA points align closely but with offsets (Hausdorff distance 0.124). The cosine histogram (mean: 0.957; std: 0.025) spreads across 0.93 0.98, indicating robust but not optimal isomorphism for encrypted text processing. Additional negatives enhance ranking but not global fidelity. These results suggest that Quadruplet Loss strengthens relative ranking and inter-class separation but does not fully enforce global topological fidelity. The observed amplitude scaling reflects its bias toward optimizing margin constraints rather than preserving continuous embedding geometry.
Dimension-wis Embedding Comparison
Scatter Plot of Embedding Values
Mapped Encrypted Embedding Plain Embedding
Pearson Cor:: 0.955
3.5
1.5
1.0
1.0
0.5
0.5
Plain Values
Volue
0.0
0.0
-0.5
FRt: 1.09x +.000
150
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection Embeddings
Distribution Cosine Similarities
0.8
Rain
Mapped Encrypted
3.0
0.6
Frequency
0.4
1.5
0.2
0.0
0.0
0.5
1.0
1.5
2.0
2.5
0.94
0.95
0.96
0.77
0.98
Similarity
Figure 24: Alignment visualizations under Quadruplet Loss, with good overlap and amplitude deviations.
Overall, the visualizations and metrics (Table 17 underscore that only SIE achieves complete overlap and
perfoct scores, fulfilling the objective of daructuro-preservvee bijection. Alternatives, effective their domains, fail to enforce multi-level preservation required for privacy-preservngg NLP, incurring elevated GW distances which validates SIE's composite design.
57

---

Published in Transactions on Machine Learning Research (February/200)
A.8 Parameter Sensitivity Analysis
To evaluate the role of the multi-key aggregation mechanism in STEALTH which generates K distinct encrypted variants per imput to introduce encryption diversity and condition the key-attentive transformer layers-we conduct experiments varying K E (1, 10). This component enables the SIE loss to learn robust mappings by disentangling encryption artifacts from semantic invariants across multiple key- conditioned views. Experiments maintain consistent. hyperparameters as our primary setup (see Appendix A.Ip, employing 12-layer key-attentive transformer trained on encrypted MIMIC-III clinical notes with paired plaintext counterparts for supervision. Alignment quality is assessed on 1,000 held-out samples using:
Mean Pearson Correlation: Average pairwise Pearson r between mapped encrypted and plaintext embeddings, measuring linear preservation.
Mean Cosine Similarity: Average cosine similarity, evaluating directional alignment.
Element-wise Correlation: Aggregate Pearson r over all embedding elements, assessing component-
level fidelity.
PCA Variance Explained: Cummlattve variance captured by the first two principal components in joint PCA of encrypted and plaintext embeddings, reflecting topological congruence.
Results appear in Table 18 with visualizations for K 1 in Figure 25 (higher K exhibit similar near-identily patterns, omilled for brevity).
Table 18: Ablation results for varying number of keys K in multi-key aggregation. Metrics show progressive improvement up to = 5, where perfect alignment achieved, with saturation thereafter. PCA variance explained stabilizes around 92-95% for K
5.
K Mean Pearson (+: std)
Mean Cosine Sim. (+: std)
Flement-wise
PCA Var. Explained (%)
0.651 = 0.213
0.651 0.213
0.643
84.1
2
0.823 = 0.124
0.824 + 0.124
0.737
97.4
0.958 = 0.022
0.958 + 0.022
0.951
93.0
4
0.993 + 0.004
0.993 + 0.004
0.990
71.1
1.000 = 0.000
1.000 0.000
1.000
92.1
6
1.000 + 0.000
1.000 + 0.000
1.000
93.5
1.000 = 0.000
1.000 0.000
1.000
91.2
8
1.000 + 0.000
1.000 + 0.000
1.000
94.8
1.000 = 0.000
1.000 0.000
1.000
92.7
10
1.000 + 0.000
1.000 + 0.000
1.000
93.9
Table 18 reveals monotonic gains in alignment metrics to K = 5, where all correlations reach unity, indicating flawless semantic isomorphism. For K (no multi-key diversity), encryption noise overwhelms the latent space, resulting subdued correlations (~0.65) and scattered PCA distributions (Figure 25 top). As K increases, the STE loss leverages diverse eneryption views refine the mapping, yiclding tighter correlation histograms (panels C, D), synchronized dimension-wise profiles (panel E), and near-perfect element-wise scatter (panel F approaching identity). At K 5, PCA visualizations (panel A) show complete overlap of encrypted (cyan) and plaintext (magenta) clusters, cross-correlation matrices (panel B) approximate identity, and downstream tasks achieve optimal performance: BLEU=1.00 for semantic retrieval and ARI=0.98 for clustering (k-means, k= 10 on plaintext-labeled clusters).
For K 5, metrics saturate at perfection, with no statistically significant improvements (p > 0.05 via paired t-tests against = 5), while PCA variance explained plateaus at ~93% reflecting stable topological alignment. However, computational costs escalate innarly-ee. training time increases by ~20% per additional key
58

---

Published in Transactions on Machine Learning Research (February22222
due to expanded key-attentive computations and aggregation overhead. Thus, K = 5 represents the point of
diminishing returns, balancing maxinal alignment fidelity with efficiency.
Happle Lucepeec Detype
Aelpyee the
Key=1
Key=2
Key
Perrypied
Key= 44
key=5
Figure 25: Visualization of embedding alignment for K = to K = in the ablation study. Panel (A) shows PCA plots of encrypted (cyan) and plaintext (magenta) embeddings, with increasing overlap as K rises. Panel (B) displays cross-correlation matrices transitioning from noisy to near-identity patterns. Panel (C) presents Pearson correlation distributions, narrowing with higher K. Panel (D) illustrates cosine similarity distributions, converging toward unity. Panel (F) depicts dimension-wise comparisons, showing synchronization. Panel (F) shows element-wise correlation scatter plots approaching the identity line, with
K = 5 achieving perfect alignment.
Accordingly, adopt K = 5 across all datasets in experiments, including MIMIC-I11 (healthcare), proprictary financial transaction narratives, and legal document corpora. This choice ensures consistent perfect semantic preservation without superfluus resource expenditure, enabling scalable deployment, in privacy-sensitive domains. Sensitivity analyses on subset datasets confirm this optimality holds irrespective of domain-specific text characteristics (c.g., clinical jargon vs. logal terminology), underscoring the robustness of multi-key aggregation under SIE supervision. Future extensions could investigate dynamic K adaptation based encryption scheme complexity or data scale.
59

---

Published in Transactions on Machine Learning Research (February/200)
A.9 Encryption Technique Compatibility Evaluation
To evaluate STEALTH's robustness across diverse symmetric encryption schemes, we conducted experiments on sixteen techniques spanning stream ciphers (XOR, RC4/ARC4, Salsa20, ChaCha20), block cipher modes (AES-128 in ECB, CFB, CTR, CBC; Blowfish-EBB; CAST-CBC; Triple DES-CBC), and authenticated modes AAFS-128/250 in GCM, FAX, SIV, OCB, CCM). These selections encompass eryptographic primitives with varying structure preservation, computational complexity, and security properties. Unlike fully structure- obliterating schemes, these methods retain elements of determinism or invertible transformations that facilitate learning of semantic isomorphisms under the SIE loss. All experiments used the healthcare subset (MIMIC-III) with uniform hyperparameters (K = 5 keys per sample, hicrarchical token-phraso-sentene alignment). Wo report: semantic retrieval via BLEU score (higher = better reconstruction), clustering preservation via Adjusted Rand Index (ARI, higher better structural fidelity), and alignment error as Frobenius norm || Dpll (lower tighter alignment). Results are averaged across five runs with standard deviations in parentheses; alignment errors are x10-7
Table 19: Results for stream ciphers and block cipher modes on healthcare data. Standard deviations in parentheses. AE: Alignment Error (x10-7).
Metric
XOR
AES-ECB
RC4
Blowfish
AES-CFB AES-CTR AES-CBC
BLEU
1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00)
ARI
0.98 (0.01) 0.95 (0.02)
0.96 (0.02)
0.97 (0.01)
0.95 (0.02)
0.96 (0.02)
0.94 (0.03)
AE
4.0 (0.1) 14 (2.0) 8.0 (0.1)
6.0 (0.1)
13 (1.5)
7.5 (0.1)
16 (2.5)
Table 19 presents results for stream ciphers and block cipher modes. STEALTH achieves perfect semantic retrieval (BLEU- 1.000 across all ciphers, demonstrating precise plaintext reconstruction through SIE loss's near-bijective mapping. ARI metrics consistently exceed 0.94, confirming exceptional clustering preservation. Stream ciphers (XOR ARI: 0.98, error: 4.0; Salsa20: 0.98, 4.5) yield the most precise alignments due to per-byte operations. Block cipher modes introduce slightly greater variance: ECB modes show deterministic but higher errors (AES-ECI 14, Blowfish-ECCH 6.0) from block-level gramularity, while chaining modes like AES-CBC (error: 16, ARI: 0.94) display the highest variance from IV-induced randomness requiring active mitigation by STEALTH's adaptive layers.
Table 20: Ablation results for authenticated modes and legacy ciphers on healthcare data. Standard deviations in parentheses. AE: Alignment Error (x10-?).
Metric AES-GCM
Salsa20
ChaCha20
AES-EAX
AES-SIV
AES-OCB 3DES-CBC
BLEU
1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00) 1.00 (0.00)
ARI
0.97 (0.01) 0.98 (0.01)
0.97 (0.01)
0.96 (0.02)
0.97 (0.01)
0.98 (0.01)
0.94 (0.03)
AE
5.5 (0.1) 4.5 (0.1) 6.5 (0.1) 7.0 (0.1) 5.0 (0.1)
4.2 (0.1)
15 (2.2)
Table 20 presents results for authenticated modes and legacy ciphers. Authenticated modes (AES-GCM, AES-EAX, AES-SIV, AES-OCB, AES-CCM) maintain exceptional performance with alignment errors of 4.2 and ARI of 0.96-0.98 despite authentication tags. AES-OCB achieves the lowest error (4.2) from its parallel structure, while AES-CCM (8.2) shows slightly higher errors from sequential authentication. Modern stream ciphers (Salsa20: 4.5, ChaCha20: 6.5) demonstrate superior alignment for high-throughput applications. Legacy 3DES-CBC exhibits the highest error (15) and lowest ARI (0.94) from triple-eneryption overhead, though remains acceptable for backward compatibility. These findings highlight STEALTH's adaptability across cryptographic primitives. Note: ECB mode was used for experimental determinism; operational deployments should employ advanced modes like GCM or OCB prevent pattern leakage while maintaining demonstrated semantic preservation.
60

---

Published in Transactions on Machine Learning Research (February 2026)
A.9.1 Visualization for XOR Cipher
The dlimension-wise comparison plot illustrates near-perfect superposition between the plaintext embedding (depicted in orange) and the mapped encrypted embedding (in blue), with values oscillating symmetrically between approximately and across the 256-dimensional space. Deviations are minimal, typically on the order of 10-7, reflecting the SIE loss's efficacy learning an invertible mapping for this simplistic cipher. The scatter plot. further corroborates this, revealing exact linear correspondence (Pearson correlation coefficient of L.000, with linear fit y 1.00z + 0.00), where data points adhere tightly to the identity line without discernible scatter, indicative of point-wisc fidelity in the embedding alignment.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Rearson Com: 1.000
Plain Values
value
-2
Mapped Encrysted Embedding
-1
Plain Embodding
-0
1.00x
-0.00
150
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
1.0
0.8
Frequency
D02
0.4
0.2
0.0
50
Encryoted
0.000220730 000221000 000221 200021300 000221 900220000 2010222200000000 00022250
Casine Similarity
20-241
Figure 26: Diagnostic visualizations for the XOR cipher: (top-left) dimension- wise embedding comparison, (top-right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
In the 2D PCA projection, plaintext (orange) and mapped encrypted (blue) points coalesce into highly overlapping clusters forming distinctive fan-shaped manifold, with exccedingly short inter-point gray lines denoting negligible displacements. The principal components span PC1 from approximately 0.0002 to 0.0022 and PC2 from -0.2 to 1.0, preserving the intrinsic geometry post-mapping. Cosine similarities are sharply peaked 1.000 (mean 1.000), with deviation range from -3 x 10-7 1 10-7 This exceptional congruence underscores STEALTH's suitability for weak yet invertible ciphers like XOR, where repeating-egy structure allows the model to discern global invariants, resulting in bijective and isometric mappings.
61

---

Published in Transactions on Machine Learning Research (ebruary/202)
A.9.2 Visualization for AES-128 in ECB Mode (AES-ECB)
The dimension-wise plot demonstrates robust aligument between plaintexi and mapped encrypled embeddings, manifesting similar oscillatory patterns but with subtle shifts pete-tioleetlll arising from block padding and alignment effects inherent to ECB mode. These perturbations are minor, yet they introduce a slight increase in variance compared stream ciphers. The scatter plot upholds flawless correlation (Pearson 1.000, fit y 1.00x 0.00), albeit, with marginally broader point dispersion, suggesting that the STE objective compensates for block-induced nonlinearities through targeted triplet ranking and distance preservation terms.
Dimension-wise Embedding Comparison
Scatter Plot Embedding Values
Plein Plein Voluee
Value
Mapped Encrypted Embedding
Embeediing
50
100
150
200
250
-6
Embedding Dimension
Mapped Encrysted Velues
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
Mean: 1.000
100
-0,001
140
-0.002
120
frequency
100
-0.004
-0.005
60
40
-0.007
0.0004
0.0006
0.0007
0.2008
PC1
Cosine Similarity
Figure 27: Diagnostic visualizations for AES-ECB: (lop-left) dimension-wist embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right, cosine similarity distribution.
The PCA projection unveils pronounced linear trajectory with dense clustering at the origin, where plaintext and mapped points are intimately paired by short gray connectors (PC1 ranging from O to 0.009, PC2 from -0.006 to 0), preserving directional semantics while minimizing rolational distortions. The cosine similarity histogram exhibits mean of 1.000 with a somewhat expanded range (-6 - to x 10 10-7), capturing the deterministic yet constrained nature of ECB, which elevates alignment error relative per-character methods. These visuals affirm STEALTH's resilience to block cipher artifacts, enabling practical deployment in scenarios demanding stronger security than XOR without sacrificing semantic integrity.
62

---

Published in Transactions on Machine Learning Research Feebruar/2020
A.9.3 Visualization for RC4 (ARC4) - Symmetric Stream Cipher
The dimension-wise comparison reveals exemplary congruence, with plaintext and mapped encrypled trajecto- ries being virtually indistnguuishable across dimensions, oscillating fluidly without abrupt discontimuities. This seamless overlap highlights ARC4's stream-oriented design, which aligns well with STEALTH's hierarchical alignment mechanism. The scatter plot evinces ideal lincarity (Pearson 1.000, fit y 1.00x 0.00), with points clustered densely along the diagonal, emblematic of the STE loss's prowess in capturing character-level invariants.
Dimension-wer Embedding Comparison
Scatter Plot of Embedding Values
Encrypied Embeeding Flain Embedding
Cor::
Value
100
150
200
250
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
Mcan: 1,000
120
0.00R00
100
-0,00005
Fraquency
-0.00m
-0.00015
00002000-000 0000300---.000 0002350-0 0002325-0 000.3000200000000 0002275-0 0002250-0000022 3002225
-2 Cosine Similarity
PC1
Te-741
Figure 28: Diagnostic visualizations for ARC4: (top-left) dimension-wise embedding comparison, (top- right.) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
The PCA visualization portrays compact central cluster radiating outward, with plaintext and mapped points exhibiting tight pairings via minimal-length gray lines (PC1 from -0.0004 to 0.0002, PC2 from -0.000015 to 0). This radial topology is faithfully replicated, ensuring preservation of neighborhood relations essential for atlention-based NLP lasks. The cosine distribution centers sharply al a mean of 1.000, with deviations confined to -6 10-7 to 2 x 10-?, illustrating how ARC4's pseudorandom byte stream facilitates smooth, distortion-mininal mappings that bolster robust isomorphism lcarning under the SIE framework. Collectively, this analysis underscores the effectiveness of STEALTH in upholding semantic fidelity amid rigorous encryption protocols, paving the way for secure and dependable natural language processing applications.
63

---

Published in Transactions on Machine Learning Research (February 2026)
A.9.4 Visualization for Blowfish-ECB
The dimension-wise overlay manifests high consistency, faithfully replicating value fluctuations between plaintext and mapped embeddings, with oscillations mirroring those in other block ciphers but exhibiting reduced variance owing to Blowfish's Feistel network architecture. The scatter plot confirms impeccable correlation (Pearson 1.000, fit y = 1.00r + 0.00), underscoring the model's ability navigate variable block sizes and key-dependent substitations.
Dimension-wii Embedding Comparison
Scatter Plot of Embedding Values
Mapped Encrypted embedding
Pearson 1.000
Plain Values
Value
1.00x
120
150
200
250
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Moan: 1.900
-1.0
Frequency
250
200
-1.5
130
-2.0
100
Mapped Encrypted Plain
PC1
Cosine Similanity
Figure 29: Diagnostic visualizations for BlowfishEECB: (top-left) dimension-wise embedding comparison, (top-right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
PCA discloses an elongated linear manifold, with paired points demonstrating scant separation via gray connectors (PC1 from -0.002 to -0.0002, PC2 from -2.5 to 0), preserving extended semantic gradients. Cosine similarities aggregate at mean of 1.000, spanning -2 10-7 to 4 x 10-7, indicating superior performance relative to AES-ECB due to enhanced key handling and diffusion propertics. These attributes render Blowfish- ECB viable option for STEALTH in resource-constrained environments, balancing security with alignment. precision. These attributes position Blowfish-ECB as a highly viable encryption scheme for STEALTH deployments constrained environments, such as edge devices or legacy systems, where strikes an optimal balance between computational efficiency, cryptographic security, and embedding alignment precision, ultimately supporting reliable semantic operations encrypted data without necessitating decryption.
64

---

Published in Transactions on Machine Learning Research (February/200)
A.9.5 Visualization for AES-128 in CFB Mode (AES-CFB)
The dimension-wise comparison plot showcases near-ideal overlap between the plaintext (orange) and mapped encrypted (blue) embeddings, with symmetric oscillations ranging from -6 to across 256 dimensions and deviations confined to 10 scale. This precision highlights the SIE loss's ability to handle the stream-like behavior of CFB mode. The scatter plot displays perfect linear relationship (Pearson correlation 1.000, fit 1.00x + 0.00), with points aligned preciscly on the diagonal, demonstrating robust point-wise mapping despite feedback-induced variability.
Dimension-wise Embedding Comparison
Scatter Embedding
Happed Emoedding
1,00X
Embedding
Plain Velues
Volue
-4
50
100
150
200
250
Embedding Dimension
Mapped Enerypted Values
10:6
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapged Encrypted
Meen: 1.000
Plain
250
Frequency
200
PC1
Casine Similarity
Figure 30: Diagnostic visualizations for AES-CFB: (top-left) dimension wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
The 2D PCA projection further reveals tightly overlapping fan-shaped clusters for both plaintext and mapped encrypted points, where corresponding pairs are linked by ultra-short gray connector lines that highlight their negligible spatial separations (PC1 ranging from approximately 0.000258 to 0.002068, and PC2 from -0.2 to 1.0). This structure maintains semantic geometry effectively. The cosine similarity distribution acutely peaked at 1.000 (mean 1.000), with deviations from -3 10-7 8 x 10-7, affirming STEALTH's efficacy for modes with self-synchronizing properties, enabling secure yet encryption scenarios.
processing in dynamic
65

---

Published in Transactions on Machine Learning Research Sebruary/202g)
A.9.6 Visualization for AES-128 in CTR Mode (AES-CTR)
The dimension-wise plot exhibits strong congruence, with plaintext and mapped encrypted values showing analogous fluctuations but minor offsets due to counter-based parallelism. The scatter plot sustains perfect correlation (Pearson 1.000, fit y 1.00z 0.00), with slightly dispersed points reflecting mode-specific nonce effects. In the PCA projection, lincar slanted distribution emerges, with tightly paired points via short gray lines (PC1 0.00022 to 0.0034, PC2 -0.006 to 0.0015). This preserves directional integrity. Cosine similarities distribute with mean 1.000 and range -6 x 10 to 8 x 10-2, illustrating AES- CTR's compatibility with STEALTH, particularly for parallelizable encryption in high-throughput privacy applications.
Dimension-vin Embedding Comparison
Scatter Plot of Em Embediing Values
Mapped Encryated Embedsing
Rt: 1.00x 0.00
Plain Embedding
Plain
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection Embeddings
Distribution Cosine Similarities
Mapped Encrysted
Plain
0.00150
0.00125
100
0.00100
Prqueecyy
RC2
80
D 0.00075
90
0.00050
0.00255
0.00222
0.00024
0,20226 0.00030
0.00028
0.00032
0.00034
-0.4
0.6
Similarity
10-5e1
Figure 31: Diagnostic visualizations for AES-CTR: (top-left) dimension-wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (oottom-right) cosine similarity distribution.
The dimension-wise plot exhibits strong congruence, with plaintext and mapped values showing analogous fluctuations but minor offscts due to counter-based parallelism. The scatter plot sustains perfect corrolation (Pearson 1.000, fit y 1.00x + 0.00), with slightly dispersed points reflecting nonce effects. In the PCA projection, a linear slanted distribution emerges with tightly paired points (PC1 0.00022 to 0.0034, PC2 -0.006 0.0015), preserving dircctional integrity. Cosine similaritics distribute with mean 1.000 and range -6 10 7 to 10-7, illustrating CTS-STR's compatibility with STEALTH for parallelizable high- aigh-throughput privacy applications.
66

---

Published in Transactions on Machine Learning Research (February/200)
A.9.7 Visualization for AES-128 in CBC Mode (AES-CBC)
The dimension-wise overlay indicates excellent matching, capturing similar oscillations with subtle chaining- induced shifts. The scatter plot confirms ideal linearity (Pearson 1.000, fit y 1.00x + 0.00). The PCA plot features a dense slanted cluster, with minimal separations (PC1 0.00217 to 0.00224, PC2 -3 to 4). The cosine distribution has mean 1.000 and deviations -4 x 10-? to 5 x 10-7, highlighting resilience to propagation effects in chaining modes.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
y
Value
Encryoted Embedding
Fribeddinng
150
200
Embedding
Mapped Encrypted
1e-5
PCA Projection Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
Mean: 1.000
Frequency
300
1C2
200
100
0,000217 0.000218 0.002239 0.000220 0.002221 0.000222 0.000223
0.000224
PC1
Cosine Similarity
Ic-7e1
Figure 32: Diagnostic visualizations for AES-CBC: (top-left) dimension-wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
The dimension-wiin wise overlay indicates excellent matching, capturing similar oscillations with subtle chaining- induced shifts. The scatter plot confirms ideal linearity (Pearson 1.000, fit 1.000z + 0.00). The PCA plot features a dense slanted cluster with minimal separations (PC1 0.00217 to 0.00224, PC2 -3 to 4). The cosine distribution has mean 1.000 and deviations -4 x 10-7 to 5 x 10-7, highlighting STEALTH's resilience to propagation effects chaining modes despite CBC's sequential dependencies and IV randomness that introduce the highest alignment variance among tested block cipher modes. Overall, these results affirm STEALTH's capability to maintain semantic integrity even under challenging encryption conditions, enabling reliable
NT. tasks.
67

---

Published in Transactions on Machine Learning Research (ebruary/202)
A.9.8 Visualization for AES-128 in GCM Mode (AES-GCM)
Dimension-wise comparison shows high fidelity, with values aligning closely despite authentication lags. Scatter plot reveals perfect correlation (Pearson 1.000, fit y = 1.00x 0.00). PCA displays an elongated linear form, paired points with scant displacement (PC1 0.000214 0.00182, PC2 1). Cosine similarities center 1.000, range- -2 x 10-? to 10-7, validating STEALTH for authenticated modes ensuring integrity alongside privacy.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
FRy 3.00x 0.0.0
value
Napped Encrypted Embedding
Piain Embedcing
50
100
150
200
250
-6
Embedding Dimension
Mapped Encrypted Values
1e-6
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Enerypted
Mcan: 1.000
Plain
500
400
Frequencyy
300
PC1
Cosine Similarity
Figure 33: Diagnostic visualizations for AES-GCM: (top-left) dimension-wise embedding comparison, (top- right.) seatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
Dimension-wise comparison shows high fidelity, with values aligning closely despite authentication tags. Scatter plot reveals perfect correlation (Pearson 1.000, fit y 1.00x + 0.00). PCA displays elongated linear form with paired points showing scant displacement (PC1 0.000214 to 0.00182, PC2 -3 to 1). Cosine similarities center at 1.000, range -2 x 10-7 to 4 x 10-7, validating STEALTH for authenticated modes that ensure integrity alongside privacy. The low aligument error (5.5) demonstrates that GCM's GaloissCouun mode
CTR encryption with polynomial authentication does not impede semantic presorvation, making idcal for production deployments requiring both confidentiality and tamper detection. Overall, these findings position AES-GCM as preferred scheme for STEALTH in scenarios demanding both confidentiality and authentieity, such secure federated learning environments.
68

---

Published in Transactions on Machine Learning Research (February
A.9.9 Visualization for Salsa20
The dimension-wise plot illustrates seamless superposition, oscillating from -6 to 6 with negligible deviations. Scatter confirms exact linearity (Pearson 1.000, fit central cluster, tight pairings (PC1 -0.0003 to 0.00025, PC2 -1.4 to 0.2). Cosine peaked at 1.000, deviations -5 x 10-7 to x 10-6, emphasizing Salsa20's speed and minimal ion for real-time applications.
1.00z + 0.00). PCA shows radial expansion from
Dimension-wise Embedding Comparison
Scatter Plot Embedding Values
Mapped Encrypted Embedoing Plain Emoedding
Flty 1
Plain Volues
Value
150
200
Embedding Dimension
Mapped Encrypted
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
Napped Encrypted
Mean: 1.000
3,000
-0.001
-0.002
Fequency
-0.003
00.00
-0,00005
0.00000
0,00005
3,00030
0.00115
0.00020
0,00025
-0.8
PC1
Casine Similarity
Ia-B43
Figure 34: Diagnostic visualizations for Salsa20: (top-left) dimension wiieession-wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
The dimension-wise plot illustrates seamless superposition, oscillating from -6 to with negligible deviations. Scatter confirms exact linearity (Pearson 1.000, fit y 1.00z + 0.00). PCA shows radial expansion from central cluster with tight pairings (PC1 -0.0003 to 0.00025, PC2 -1.4 to 0.2). Cosine peaked at 1.000, deviations -5 x 10-7 to x 10-6, emphasizing Salsa20's speed and minimal distortion for real-time applications. The exceptionally low alignment error (4.5) and high ARI (0.98) reflect Salsa20's stream cipher architecture with systems
efficient ARX operations, positioning as an optimal choice for high- thgoouhhuuuhttt where computational efficiency and semantic fidelity are equally critical. Consequently, Salsa20 emerges as an exemplary cipher for real-time privacy- preservingee NLP pipelines, balancing cryptographic strength with operational efficiency.
69

---

Published in Transactions on Machine Learning Research (obruary/202))
A.9.10 Visualization for CAST in CBC Mode (CAST-CBC)
Dimension-wis wise alignment is consistent, with similar fluctuations and minor shifts from chaining. Scatter upholds perfect correlation. PCA reveals linear elongation, minimal separations (PC1 -0.002 to 0.002, PC2 -0.8 to 0.3). Cosine mean 1.000, range -3 x 10-? to x 10-", supporting legacy-compatible ciphers in STEALTH.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Mapped Encrypted Embedding Plain Embedding
Ftty 1.00 1.000 0.00
Plain Values
Value
Embedding Dimension
Mapped Encrypted Values
PCA Projection Embeddings
Distribution Cosine Similarities
9.2
Happed Encrypted
200
Main
0.0
175
-0.2
-0.4
Fequency
DC2
-0.8
100
-0.8
75
-1.0
-1.2
25
-1.4
0.002203
0.000210
0.000215
0.009120
0.000225
Similerity
16-701
Figure 35: Diagnostic visualizations for CAST-CBC: (top-left) dimension-wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
Dimension-wist alignment consistent, with similar fluctuations and minor shifts from chaining. Scatter upholds perfect correlation (Pearson 1.000, fit y 1.00z 0.00). PCA reveals linear elongation with minimal separations (PC1 -0.002 to 0.002, PC2 -0.8 to 0.3). Cosine mean 1.000, range -3 x 10-7 to 5 10-7 supporting legacy-compatible ciphers in STEALTH. The moderate alignment error validates STEALTH's adaptability to legacy block ciphers with Feistel-like structures, ensuring backward compatibility for systems transitioning from older cryptographic standards while maintaining semantic utility for privacy-preserving analytics. These results illustrate STEALTH's efficacy accommodating variable key lengths inherent. CAST, minimizing topological distortions through adaptive projections that reconcile legacy diffusion mechanisms.
70

---

Published in Transactions on Machine Learning Research (February/200
A.9.11 Visualization for ChaCha20
The overlay is highly precise, replicating oscillations faithfully. Scatter shows ideal fit. PCA features compact clustering with outward rays, short connectors (PC1 -0.001 to 0.001, PC2 -0.5 to 0.5). Cosine distribution sharp at 1.000, deviations -4 x 10 to 6 x 10-7, underscoring ChaCha20's enhanced security for mobile and IoT deployments.
Dimension-wise Embeddling Comparison
Scatter Plot of Embedding Values
Napped Encrypted Embedbling Piin Embecding
y
Value
Plein
130
250
Embedding Dimension
Mapped Encrypted Values
PCA Projection of Embeddings
Distribution of Cosine Similarities
0.0003
Mapped Encrypted
120
Mean: 1.000
0.0002
100
0.0001
Frequency
0,0000
-0 0000
-0.00 0000
-0.0003
8.00188
0.00018
3,00020
0.00022
0,00024
-0.2
0.0
0.1
PC1
Cosine Similarity
Ie-he1
Figure 36: Diagnostic visualizations for ChaCha20: (top-left) dimension- wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
Figure 36 Ilustrates seamless superposition, oscillating from -6 to with negligible deviations. Scatter confirms exact linearity (Pearson 1.000, fit 1.00z + 0.00). PCA shows radial expansion from central cluster with tight pairings (PC1 -0.001 to 0.001, PC2 -0.5 to 0.5). Cosine peaked at 1.000, deviations -4 x 10-7 to 6 x 10-?, emphasizing ChaCha20's enhanced security for mobile and IoT deployments. The moderately low alignment error (6.5) and high ARI (0.97) reflect ChaCha20's stream cipher architect ure with efficient ARX operations, positioning as an optimal choice for thgouhhhoughput privacy- prreaccypeeeevingg systems where computational efficiency and semantic fidelity are equally critical. Notably, ChaCha20's s quarter-round functions and nonce-ban nosee-based initialization contribute to this fidelity, allowing STEALTH to handle high-entropy streams with minimal latent space perturbation.
71

---

Published in Transactions on Machine Learning Research (obbuaar/20000
A.9.12 Visualization for AES-128 in EAX Mode (AES-EAX)
The dimension-wee comparison exhibits exceptional alignment, with plaintext and mapped encrypted values oscillating in near-perfect harmony across dimensions, showing minimal deviations attributable to the mode's authentication overhead. The scatter plot maintains impeccable linearity (Pearson 1.000, fit = 1.00x + 0.00), with points lightly clustered along the diagonal. The PCA projection displays a linear trend with overlapping points connected by negligible gray lines (PC1 0.00224 to 0.00238, PC2 -0.010 to 0.000). The cosine similarity distribution sharply focused at mcan 1.000, with range -4 10-7 6 10-7, demonstrating STEALTH's capability to handle authenticated modes that provide both confidentiality and integrity without. compromising semantic isomorphism.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Mapged Encrypted Embedting
Fily 1.00x
Plain Embedding
Plain Values
value
Embedding Dimension
Mapped Encrypted Values
2D PCA Projection of Embeddings
Distribution Cosine Similarities
Maaped Encrypred
Main
-0.002
Prqueecc
150
-0.006
100
-0.000
50
0.200
0.0
Similarity
Figure 37: Diagnostic visualizations for AES-EAX: (top-left) dimension wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
The dimension-wise comparison exhibits exceptional alignment, with plaintexi and mapped values oscillating in near near-perfert sear-perfect harmony with minimal deviations from authentication overhead. The scatter plot maintains impeccable lincarity (Pearson 1.000, fit y 1.00x + 0.00). The PCA projection displays linear trend with overlapping points connected by negligible gray lines (PC1 0.00224 to 0.00238, PC2 -0.010 to 0.000). Cosine similarity is sharply focused at mean 1.000, range -4 10-7 to 6 10-7
72

---

Published in Transactions on Machine Learning Research (February/200 2026)
A.9.13 Visualization for AES-256 in SIV Mode (AES-SIV)
Dimension-wiss plot reveals robust congruence, capturing similar fluctuations with subtle adjustments for deterministic authentication. Scatter confirms perfect correlation (Pearson 1.000, fit = 1.00z + 0.00). PCA
shows elongated slanted clusters, paired with short connectors (PC1 0.000 0.0032, PC2 -0.002 to 0.000).
Cosine mean 1.000, deviations -3 10 5 10-7, highlighting suitability for misuse-resistant modes in secure data storage scenarios.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Mapped Encrypted Embbddding
y
Plain Embedding
Plain Values
-6
Embedding
Mapped Encrypted Values
1
PCA Projection of Embeddings
Distribution of Cosine Similarities
Mapped Enkryoted
Meen: 1.000
125
Trequency
PC2
100
25
-1.75
0.000216
6,003217
0.000215 0.000220
0.000229
0.000121
-3
-2
PC1
Cosine Similarity
1e-7e1
Figure 38: Diagnostic visualizations for AES-SIV: (top-left) dimension-wise embedding comparison, (top- right.) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
Dimension-wis plot reveals robust congruence, capturing similar fluctuations with subtle adjustments for deterministic authentication. Scatter confirms perfect correlation (Pearson 1.000, fit 1.00z + 0.00). PCA shows elongated slanted clusters paired with short connectors (PC1 0.000 to 0.0032, PC2 -0.002 to 0.000). Cosine mean 1.000, deviations -3 10-7 to 5 x 10 highlighting suitability for muiuse-resistann modes secure data storage. The alignment error of 5.0 and high ART (0.97) demonstrate that STV's deterministic two- pass construction-usinn PRF for synthetic IV generation followed by CTR encryption-preeee preserves semantic structure while providing nonce-reuse resistance critical for database eneryption and archival systems.
73

---

Published in Transactions on Machine Learning Research Sebruary/202g)
A.9.14 Visualization for AES-128 in OCB Mode (AES-OCB)
The overlay indicates high-fidelity matching, with oscillations aligned closely despite efficient authentication. Scatter upholds ideal lincarity. PCA features dense linear distribution, minimal separations (PC1 -0.0005 to 0.0005, PC2 -0.001 to 0.001). Cosine peaked at 1.000, range x 10-7 to 10-7, affirming efficiency for high-performance applications requiring fast authenticated eneryption.
Dimenslon-wise Embedding Comparison
Scatter Plat of Embedding Values
1.00x 0.00
Plain Velues
Volue
-10
Happed Encrypted Embadding
-0
Painn Embedding
Embedding Dimension
Mapped Encrypted Values
Projection Embeddings
Distribution Cosine Similarities
Mapped Encrypted Pain
1.000
3,00002
160
140
120
00,00022
frequency
100
PC2
0.00004
80
-0 -0,00006
60
00000
40
20
-0.00030
0.000224 0,000226 0,000288 0.000330 0.002322 3.000234 0.000236 0,000238
-0.4
-02
0.0
0.2
0.4
0.6
Cosine Similarity
Ie-6+1
Figure 39: Diagnostic visualizations for AES-OCB: (top-left) dimension- wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
The overlay indicates high-fidelity matching, with oscillations aligned closely despite efficient authentication. Scatter upholds ideal linearity (Pearson 1.000, fit y 1.00x 0.00). PCA features dense linear distribution with minimal scparations (PC1 -0.0005 to 0.0005, PC2 -0.001 to 0.001). Cosine peaked at 1.000, range -5 x 10-7 to 7 x 10-7, affirming elficiency for high-performance applications ions requiring fast. authenticated encryption. The lowest alignment error among authenticated modes (4.2) stems from OCB's single pass parallclizable design that interlcaves eneryption and authentication, making it optimal for latency deployments such as secure network communications and mal-time encrypted analytics where computational overhead must be minimized.
74

---

Published in Transactions on Machine Learning Research (obruarr/2020)
A.9.15 Visualization for Triple DES in CBC Mode (3DES-CBC)
The dimension-wee wise aligument demonstrates consistent overlap between plaintext and mapped encrypted embeddings, albeit with slightly elevated variance attributable the legacy block strructure of 3DES, which employs dated Feistel network with triple encryption passes that introduce additional computational layers and potential perturbations in the latent space. Scatter reveals strong correlation (Pearson 1.000, fit. y 1.00x + 0.00). PCA discloses slanted manifold with short gray lines (PC1 0.0014 to 0.0018, PC2 -0.003 to 0.000). Cosine mean 1.000, deviations -6 x 10 to 8 x 10-?, supporting backward compatibility in transitional systems.
Dimension-wise Embedding Comparison
Scatter Plot of Embedding Values
Maaped Encrypred Embedding Plain Emaadding
Ft.y 1.00x + 0.00
Plain Valumm und
Value
100
Embedding Dimension
Mappad Encrypted Values
2D PCA Projection Embeddings
Distribution of Cosine Similarities
Mapped Encrypted
Plain
250
Prquuency
g
50
0.0001.00
3.000141
0.001422
0.000383
0.000444
Similarity
1e-T41
Figure 40: Diagnostic visualizations for 3DES-CBC: (top-left) dimension-wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
Dimension-wiss alignment shows consistent overlap, though with slightly higher variance due to legacy block structure. Scatter reveals strong correlation (Pearson 1.000, fit 1.00x + 0.00). PCA discloscs slanted manifold with short. gray lines (PC1 0.0014 to 0.0018, PC2 -0.003 to 0.000). Cosine mean 1.000, deviations -6 x 10-7 to 8 x 10 10-7, supporting backward compatibility in transitional systems. The highest alignment error (15) and lowest ARI (0.94) reflect 3DES's triple-encryption overhead and dated Feistel structure, yet performance remains acceptable for legacy infrastructure requiring gradual cryptographic modernization while preserving analytical capabilities historically encrypted datasets.
75

---

Published in Transactions on Machine Learning Research (eebruar/2022)
A.9.16 Visualization for AES-128 in CCM Mode (AES-CCM)
The comparison plot demonstrates precise superposition, handling coustraineed-emiionnnnntt authentication effectively. Scatter confirms exact fit. PCA reveals compact radial clusters, tight pairings (PC1 -0.0002 to 0.0002, PC2 -0.0005 0.0005). Cosine distribution sharp at 1.000, range 10-7 to 6 x 10-?, ideal for vesource-limited devices like IoT sensors. These empirical results unequivocally affirm STEALTH's robustness
Dimension-wise Embedding Comparison
Embedding
Encrypted Embedding
0.00
Plain Emoldding
4.
100
150
200
250
Embedding Dimensian
Mapped Enerysted Values
2D PCA Projection of Embeddings
Distribution of Cosine Similarities
0.00225
Mapped Encrypted Plain
Meen: 1.000
200
0,00020
0.00015
150
Frequency
100
0.00000
0.00U2204
0.002210 0,001/22
PC1
Cosine Similarity
Figure 41: Diagnostic visualizations for AES-CCM: (top-left) dimension-wise embedding comparison, (top- right) scatter plot of embedding values, (bottom-left) 2D PCA projection, (bottom-right) cosine similarity distribution.
across wide array of symmetric ciphers, delivering near-ideal semantic utility while upholding stringent encryption security. The observed nuances emphasize the advantages of stream ciphers (e.g., XOR, ARC4, Salsa20, ChaCha20) in minimizing latent distortions, contrasted with the modest overhead of block-based modes (e.g., AES variants, Blowfish, CAST, 3DES), which nonetheless remain efficacious. Authenticated modes like AES-EAX EAX, AES-SIV, AES-OCB, AES-CCM introduce minimal additional variance, confirming STEALTH's versatility. Importantly, less secure modes like ECB were adopted for experimental determinism and reproducibility; operational deployments, we advocate for advanced modes such as GCM, OCB, or CCM avert pattern leakage vulnerabilities and ensure message integrity. Prospective ablations may extend asymmetric paradigms or post-quantum resistant ciphers, further broadening STEALTH's applicability in evolving threat landscapes.
76

---

Published in Transactions on Machine Learning Research (February/2022
A.10 Theoretical Analysis: Security of the Learned Isomorphism
In this appendix, we formally analyze the security of the learned semantic isomorphism in STEALTH. We clarify the conditions under which the mapping fe does not enable unauthorized decryption, providing a rigorous theorem with clear assumptions and symbolic bounds. Empirical estimates of the parameters (e.g., distortion separation A, and Pr(NN success)) are reported in Section 4.
The encryption scheme E' is a PRP-style primitive used with per-message tweaks/nonces, under correct usage this gives IND-like guarantees. Where necessary we note that AEAD (e.g., AES-GCM) would be preferred in production. The security parameter is X (e.g., 128-bit key). Our multi-key approach (K=5 variants per input) further randomizes across keys to mitigate potential frequency leaks in deterministic modes.
The threat model considers a semi-honest, probabilistic polynomial-time (PPT) adversary A with white-box access to the trained model Jo (including weights), query access to fo, encrypted inputs F(r, k), and possibly a subset of the plaintext corpus P' C or auxiliary data (e.g. public texts). A does not have the encryption key k the full corpus P used for authorized reconstruction, oracle access to decryption, or side-channel information (e.g., timing). Reconstruction occurs via semantic search: arg minger d( (E(z, k)). g(v)), where 9 is fixed plaintext embedder and d is cosine distance d(u,,)) = 1 [u]|o]"
We define distortion e as the expected value [[d(fo(E(z, k)), g(x)) over the data distribution and keys (mcasuring alignment imperfection); we report its distribution (mean + std, quantiles) in experiments. We define separation A as the 1st percentile of pairwise distances {d(g(z). plaintext embedding distinctiveness, robust to outliers). Let Pr[NN success over P'] denote the probability that nearest-neighbor search over an accessible corpus ''' recovers the true x; this is an empirical function of C, A, IP'], and corpus structure, which we mcasure in Section 4.
+ y E X} (measuring
Definition 1 (Plaintext Recovery Advantage). In the recovery game, the challenger samples x - X, computes c Elx, k) for secret key k, and gives fo(c) to A. A outputs guess x'. The advantage is AdvREC = Pr(a' = 2] - 1/
such that AdVIND-CPA (B) I nAg for any PPT B. Assume fo is trained with SIE loss multi-key variants. For semiihonest PPT adversary A without k:
Theorem (Security Against Unauthorized Decryption). Let É be PRP-secure with
(a) A has no access to any plaintext corpus (P' = =)), then (A)REC(A) & (PV)ND- + negl(A) for some reduction B.
(b) A has access to P' C with 'P'| < (P], then (A)REC(A) S Pr PN(NN success over P'] + AdVIND-CPA (B)) + neg)(A).
(c) A additionally has decryption oracle access side-channeke security reduces to standard semantic-search munerabilities (e.g., membership inference), independent of E's cryptogrophy.
Proof Sketch. The PRP-sccurity (with tweaks/nonces ensures E(x, k)) leaks no token-level information about x beyond public format/length Multi-key training and SIE enforce semantic invariance: embeddings cluster by meaning via relative metrics (triplets, distances), inducing many-to-one mappings where semantically similar texts collide, and token details are lost to aggregation and low dimensionality (d = 256 << sequence length).
For (a): Suppose A recovers x with non-negligible advantage. Construct IND-CPA distinguisher B: B chooses distinct zp, 21 E X' with d(g(zo), ((x())))(((1))) > A, queries the IND-CPA oracle on 2o, 31 to get G E(x,, k), computes fo(c)), runs A on fo(cz) (simulating any model queries), gets z', and outputs b' = 0 if x' zo (else 8' = 1). If A succeeds, B distinguishes with advantage AdvREC negl(A), contradicting IND-CPA unless the advantage is negligible. The reduction is tight under the assumption that < A (alignment better than half the separation), ensuring low collision probability; since no P is needed, no simulation is required.
For (b): Additional leakage comes from semantic scarch over P'; success is bounded by Pr(NN success over P'), which small for small [P'| or large A relative to e. The IND-CPA reduction holds as in (a), with B providing P' to A (assuming P' is auxiliary/public data independent of the challenge).
77

---

Published in Transactions on Machine Learning Research (obruary/2020)
For (c): Oracle access allows collecting surrogate pairs to train inverters, but SIE's relative focus prevents absolute mappings; security falls to empirical attacks (e-g., inversion success measured in experiments). 7
This theorem shows that STEALTH's security reduces to either breaking the underlying cryptography of E or solving semantic scarch over the adversary's accessible corpus, under the geometric assumptions and A.
A.11 Ciphertext-Only Attack Evaluation
To rigorously assess STEALTH's resilience against ciphertext-only attacks (COA), we evaluate scenario where an unauthorized adversary has access to encrypled ciphertexts and the trained model parameters but lacks decryption keys. This threat model aligns with our taxonomy, a semi-honest computational ad adversary without hardware side-channel capabilities. The primary question is whether such adversary can recover meaningful plaintext, potentially violating semantic privacy guarantees.
A.11.1 Formal Threat Specification
In COA, the adversary observes only encrypted inputs E(x, k) for plaintext x E D, where l is the secret key, alongside the model's weights 0. No auxiliary information (e.g., plaintext-ciphertex
pairs or key material) is available. We assume the adversary aims to invert the mapping fo : E - Rd to reconstruct from the encrypted embedding fo( (xx, k)), without decrypting E(x, k) directly. This models realistic deployment risks in cloud-based NLP services where models process encrypted data streams.
To simulate a strong adversary, we instantiate an attack model as a neural decoder go : R + V", where V is the vocabulary and ø are learned parameters. The docoder is a 6-layer transformer with beam scarch decoding (beam width 4), trained minimize cross-entropy loss on surrogate dataset of held-out encrypted embeddings paired with their corresponding plaintexts (simulating a worst-case where the adversary has partial access to similar-domain data for training, but not the target instances). This setup upper- bounds attack efficacy, as real-world adversaries would lack such pairs.
A.11.2 Experimental Setup
We evaluate on healthcare data (MedMCQA, PubMedQA, MIMIC-III; 150k instances) with AES-128-GCM encryption, 128-bit keys. The attack model trains on 80% held-out encrypted embedding plaintext pairs (120k) for 20 epochs (AdamW, 7 == 10 5), tested 20% (30k) with ciphertexts only.
Metrics: (i) Token Accuracy-fras fraction of correctly recovered tokens; (ii) BLEU-n (n 1n=10 1-4)-
precision; (iii) Semantic Similarity cosine similarity between BERT embeddings of original and recon- structed plaintexts; (iv) MIA Success success-binary classification accuracy distinguishing target instances from non-targets.
Baselines include random guessing (uniform over vocabulary) and naive frequency-based decoder (using unigram statistics from the domain corpus). Results averaged over 5 runs with different random seeds for key generation and model initialization.
A.11.3 Quantitative Results
Table summarizes the attack performance. The trained decoder achieves near-random token accuracy (12.3% 1.2%), comparable to the random bascline (10.8% + 0.9%) and substantially below froquency-basod guessing (18.7% + 1.5%). BLEU scores are minimal (BLEU-1: 0.14 0.02; BLEU-4: 0.02 0.01), indicating negligible n-gram overlap and failure recover coherent phrases. Semantic similarity averages 0.08 + 0.03, far below the 0.95+ required for meaningful inference, confirming that reconstructions bear no semantic resemblance to originals.
MIA success is 51.2% + 2.1%, indistinguishable from random chance (50%), suggesting that confidence scores from the decoder do not leak membership information. Ablations varying decoder depth (4-12 layers) and training data size (50k-100k) yield similar results, with no statistically significant improvement (p 0.05, (-test).
78

---

Published in Transactions on Machine Learning Research (February/202)
Table 21: Ciphertext-only attack performance on healthcare data using AES-128-GCMA Results show near- random recovery, validating STEALTH's cryptanalysis resistance.
Metric
Trained Decoder
Frequency Baseline
Random Baseline
Token Accuracy
0.123 0.012
0.187 0.015
0.108 + 0.009
BLEU-1
0.14 + 0.02
0.19 0.03
0.09 + 0.01
BLEU-4
0.02 0.01
0.04 0.01
0.00 0.00
Semanntc Similarity
0.08 + 0.03
0.11 + 0.04
0.05 + 0.02
MIA Success
0.512 0.021
0.523 0.018
0.500 + 0.000
Representative examples of ciphertext-only attack reconstruction attempts on healthcare data. The trained decoder (6-layer transformer with beam search) produces semantically meaningless output when attempting to recover plaintext from encrypted embeddings.
Table 22:
Original Plaintext
Encrypted Ciphertext
Decoder Output
Tok. Acc.
Sem. Sim.
Patient diagnosed with acute myocardial infare- tion and requires imme- diate intervention.
Qbujfou ejbhoptfe xjui bdvuf nxpdbsejbm jogbs- dujpo boe sfrvjsft junfe- jbuf joufswfoujpo.
Relative system under pro- tocol management requires standard assessment criteria validation purposes.
0.08
0.06
Blood pressure 140/90 mmHg, heart rate 88 bpm, temperature 37.2ºC.
Cmpph qsfttvsf 140/90 mnlh, ifbsu shuf 88 equ, ufnqfsbuvsf 37.2%D
Primary indicators measure- ment standard procedure documentation reference technical specifications.
0.11
0.05
Prescribe metformin 500mg twice daily with meals for glycemic trol.
Qsftdsjef nfugpsnjo 500nh uxjdf ebjmz xjui ufbmt gps hmzdfnjd dpouspm.
Treatment protocol medica- tion schedule standard ad- ministration monitoring eval- uation process.
0.09
0.07
CT sean reveals no evi- dence of intracranial hem- orrhage or mass effect.
DU idbo sfwfbmut op fw- jefodf pg jousbdsbojbm ifnpesbhff ps nbtt figidu.
Diaguostic imaging proce- dure results analysis findings interpretation clinical assess- ment evaluation.
0.13
0.09
Family history positive for diabetes mellitus lype and hypertension.
Gbnjmz ijtupsz qptjujwf gps ejbefuft ufmmjuvi uzql 2 boe izqfsufotjpo.
Medical background informa- lion patient denuggapphcs clinical characteristics assess- ment documentation.
0.10
0.08
A.11.4 Qualitative Analysis: Reconstruction Examples
Table 22 presents representative examples of decoder reconstruction attempts. The outputs exhibit several characteristic failure modes that demonstrate STEALTH's security properties.
Analysis of Decoder Outputs: The trained decoder's reconstructions exhibit four charactoristic failure modes: (1) Generic medical terminology outputs contain plausible medical words but lack semantic coherence with the original content; (2) Topic crifttrreonntruccti phrases reference vague procedural language rather than specific clinical information; (3) Loss of ceseeffcity-mmerrine values, measurements, and medication names are replaced with generic descriptors; (4) Structural incoherence grammatical structure may be preserved, but semantic relationships are lost.
These examples demonstrate that even sophisticated neural decoder trained on 120k encrypted embedding- plaintext pairs from the same domain cannot extract meaningful information from STEALTH's encrypted embeddings. The decoder learns to produce superficially plausible medical text but cannot. recover the
79

---

Published in Transactions on Machine Learning Research (February/2022
actual semantic content, as evidenced by token accuracies of 8-13% (near random chance for a medical vocabulary of ~10,000 tokens) and semantic similarities of 0.05-0.00 0.09 (far below the 0.95+ threshold for semantic equivalence).
A.12 Key-Mismatch Attack Evaluation
To demonstrate the eryptographic properties underpinning STEALTH's security, we evaluate a complementary threat scenario where an adversary possesses incorrect decryption keys and attempts to decrypt ciphertexts diroctly. This key-mismatch attack models scenarios where an attacker has obtained keys through side channels or brute force, but the keys do not correspond to the target ciphertexts. Unlike the ciphertext-only attack which operates learned embeddings, this evaluation demonstrates the fundamental cryptographic security of the underlying encryption scheme.
A.12.1 Threat Model
In this scenario, the adversary has: (1) access to encrypled ciphertexts E(x, k) generated using AES- LES--18-A0 10--128---G GCM; (2) set of incorrect decryption keys {M' where k' * k; (3) standard cryptographic decryption algorithms. The adversary attempts to decrypt E(x, k) using k', hoping to recover meaningful plaintext. We simulate minimal key perturbations (single-bit flips) to evaluate sensitivity to key correctness.
A.12.2 Experimental Setup
We use AES-128-GCM for textual data preservation; ciphertexts include authentication tags and are produced with per-message nonces. For each sample plaintext drawn from the MIMIC-I1I dataset, we generate: (1) the correct encryption key k (128-bit, hex-encodod); (2) the resulting ciphertext Exx, k); (3) an incorrect key k differing by single bit-flip from k; (4) the outpul of decryption attempt (EE(x, k)) using the incorrect. key.
Modern authenticated encryption modes like GCM include integrity verification via authentication tags. When decryption is atlempted with an incorrect key, the authentication lag verification fails, and the decryplion process aborts, returning an error rather than corrupted plaintext. This property is critical for preventing silent data corruption and tampering attacks.
A.12.3 Key-Mismatch Results
Table 23 presents illustrative examples of encryption and failed decryption attempts with mismatched keys. In all cases, decryption with an incorrect key produces one of two outcomes: (1) Integrity tag mismatch- the authentication verification fails immediately, preventing any output; (2) Semantically meaningless
output- if tag verification is bypassed (for illustrative purposes only), the output is pseudorandom gibberish bearing no resemblance to the original plaintext.
Analysis: These results demonstrate several critical security properties: (1) Key sensitivity even single-bit perturbations to the decryption key result in complete decryption failure, with outputs bearing no statistical correlation to the original plaintext (measured cosine similarity < 0.01 for all examples); (2) Authentication protcction-the the GCM authentication tag mochanism prevents silent corruption, cnsuring that incorrect keys are immeddittll detected rather than producing subtly corrupted data; (3) All-or-nothing transformation there is no graceful degradation where partial key correctness yields partial plaintext recovery; decryption either succeeds completely with the correct key or fails catastrophically with any incorrect key.
The pseudorandom nature of failed decryplion outputs (when lag verification is artificially bypassed for analy- sis) confirms that the encryption scheme provides no information leakage through partial decryption attempts. This property is essential for STEALTH's security model, as it ensures that adversarics without correct keys cannot extract any meaningful semantic information, even with access to sophisticated cryptanalytic tools or partial key knowledge.
80

---

Published in Transactions on Machine Learning Research (Pebruar/2022)
Table 23: Illustrative examples plaintext encryption and failed decryption with mismatched keys. Cipher- texts are generated using AES-128-GCM, Decryption with an incorrect key (single-bit perturbation) produces incoherent output or authentication failure, highlighting key-sensitivity.
Plaintext
Encryption Key (Hex)
Ciphertext
Decryption with Differ- ent Key
Patient presents with chest pain and short- ness breath.
2a7e5156fddlec21 696f2a1781accbcb
Qbtxhrg kzhzhmgd vigs xgzdg kbjm buj dgizgmhidd iu cahbgg,
Xyzwqpl oiuvbnas fghj klmu opqr stuv wxyz abed efgh ijkl. (Decryption failure: integrity tag mis- match)
History of hyperten- sion and diabetes mel- litus.
@9abcdef01234567 89abcdef01234567
Gjdgizl iu slkzhgh- mdjim bmj ojbcygyd nyqgighd.
Poiuytrew qasdfghj klzx- evbn mpoiu ytre wqas dfgh. (Deeryption failure: tegrity tag mismatch)
Administer aspirin 325 mg orally stat.
fedcba9876543210 fedcba9876548210
Bonyjmdghyz bdkjæj m 425 izbqqs dgbg.
Lkjhgfds aqwertyu iopzx- evb nmkljh gfds aqwe. (De- cryption failure: integrity tag mismatch)
Echocardiogram shows ejection fraction 45%.
1234567890abcdef 1234567890abccdef
Yxsiwbzojizbny bgiud yhhywgjim uzbwgjim 54%.
Mnbvexza sdiufghj kipoiuyt rewqmn bvez xasd. (De- cryption failure: integrity tag mismatch)
A.12.4 Synthesis
These results empirically confirm STEALTH's security under the key-mismatch threat model. The evaluation demonstrates that the underlying cryptographic primitives provide strong key-sensitivity and authentication, preventing any mcaningful decryption without proper credentials.
The key-mismatch failures demonstrate that even with decryption algorithms, incorrect keys yield no advantage over random guessing. This establishes that STEALTH maintains cryptographic security properties while enabling semantic computation on encrypted data through learned isomorphic mappings.
Future work could explore adaptive attacks combining partial key knowledge with learned embedding inversions, model inversion attacks under whitc-box access with gradient information. However, evaluation establishes strong baseline for STEALTH's privacy guaranlees across realistic threat scenarios in privacy-preserving NLP deployments.
A.13 Practical Limits of Retrieval- Based Reconstruction
In this appendix, we address the practical deployment considerations for STEALTH's retrieval-based recon- struction mechanism. As described in Section 3.3, reconstruction operates via nearest-neighhor search in a pre-computed plaintext corpus embedding space: given an encrypted embodding &, fo(El ( (x, k)), the system retrieves x² arg max,eP cos(2e, ((y),, where g(-) is a fixed plaintext embedder (e.g., BERT) and P is the domain-aligned plaintext corpus. This approach enables near-perfect reconstruction when P contains the exact match semantically equivalent variants, without decrypting during inference.
However, real-world deployment involves trade-ofls in corpus coverage, retrieval accuracy, and computational efficiency. Below, we present ablations, examples, and scaling analysis to demonstrate these limits, responding to potential reviewer concerns about reliance on ncarest-ncighhor search. We acknowledge that reconstruction is fundamentally retrieval-based and discuss mitigations such as thresholding, reranking with secondary models (e.g., cross-encoder verification), and hybrid keyword filtering to address limitations like false positives.
81

---

Published in Transactions on Machine Learning Research (February/202g 2026)
A.13.1 Corpus Coverage Analysis: Degradation Across Metrics
Reconstruction fidelity depends on P's coverage of the semantic space. In low-coverage scenarios (e.g., rare jargon or underrepresented domains); the nearest neighbor may retrieve semantically similar but non-identical text, degrading exact-match metrics while preserving semantics. We evaluate this on the MIMIC-III medical notes subset (10,000 samples), stratifying subsampling by diagnosis codes and note sections to preserve distribution. We use random seed 42 for reproducibility and report mean I std over 5 seeds. Embeddings use BERT-base (d = 768), with FAISS indexing (index factory=' Flat' ' for this exact-search ablation; M = 32, efConstruction=200, efSearch=128).
Table 24: Degradation across metrics as corpus coverage shrinks. Reported as mean + std over seeds.
Fraction
Pl
BLEU-1
BLEU-4
BS-F1
MS
Cosine
R@i
R
MR
MRR
EM
100%
10,000
1.00 + 0.00
1.00
0.00
1.00 + 0.00
1.00 = 0.00
1.00 + 0.00
1.00
1.00
1
1.00
1.00
50%
5,000
0.98 = 0.01
0.95 + 0.02
0.98 + 0.01
0.97 = 0.01
0.99 + 0.01
0.96
0.99
1
0.97
0.95
20%
2,000
0.92 = 0.03
0.85 + 0.04
0.95 + 0.02
0.93 = 0.02
0.97 + 0.02
0.88
0.96
2
0.90
0.82
10%
1,000
0.85
0.05
0.72 + 0.06
0.91 0.03
0.88 0.03
0.94 + 0.03
0.75
0.90
4
0.79
0.68
5%
500
0.78 + 0.07
0.61 0.08 0.86 0.04 0.82 = 0.04 0.90 + 0.04
0.62
0.81
7
0.67
0.55
Note: BS-F1 BERTScore FI, MS METEOR Score, R$1 Recoll at 1, R Recall 5, MR Mean Rank, MRR
Mean Reciprocol Ronk, EM Exact Match.
Cosine similarity remains high (> 0.90) even as BLEU falls because retrieved texts are often paraphrases preserving mcaning but not exact n-grams (c.g., "heart attack" vs. "myocardial infarction") "). This is expected and aligns with STEALTH's goal of semantic utility over verbatim recovery high BS-F1 F/MS confirm semantics are intact. Retrieval metrics degrade gracefully: R@1 drops from 1.00 to 0.62, but R@55 remains > 0.80 5% coverage, with MR increasing to 7 and MRR to 0.67. EM falls faster than semantic metries, emphasizing that BLEU=1.0 is not the only success criterion; applications like clinical decision support prioritize meaning over exact wording.
A.13.2 Quantified False Positives and Threshold Trade-offs
Embedding spaces can yield false positives when semantically similar but contextually distinct texts collide. We define false positives as non-exact matches with cosine > 0.9, and "harmful" false positives as those introducing misleading information (e.g., wrong diagnosis in medical notes), estimated via manual review of 200 samples per run.
Below are examples from our MIMIC-III ablation (10% corpus fraction), showing retrieved xt vs. original x::
Example 1:
Original: "Patient diagnosed with type diabetes mellitus, prescribed metformin 500mg BID."
Retrieved (False Positive): "Patient tested for lype diabetes, initiated on metformin 500mg daily."
Metrics: BLEU-4-0.68, B8-F1=0.92, Cosine=0.95. Issue: Similar terms but differs in diagnosis certainty.
Example
Original: "Acute myocardial infarction confirmed by ECG, troponin levels elevated."
Retrieved (False Positive): "Suspected myocardial infarction, ECG abnormal, troponin pending."
Metrics: BLEU-4-0.55, BS-F1=0.89, Cosino=0.92. Issuc: Overlaps in symptoms but confirmation status differs.
Example
Original: "Contract breach due to non-payment, seek damages of $50,000."
82

---

Published in Transactions on Machine Learning Research (obbuaayy/2000
Retrieved (False Positive): " "Alleged contract violation for late delivery, claiming $50,000 in losses."
Metrics: BLEU-4-0.62, BS-F1=0.90, Cosine=0.93. Issue: Similar structure but cause differs.
Across 1,000 test queries at 10% coverage, false positive rate (FPR) is 15% = 2%, with harmful-FPR 3% = 1% (e.g., misaligned medical severity). Applying cosine thresholds reduces FPR but increases no-match rate:
Table 25: Threshold trade-offs on MIMIC-III (10% coverage). FPR: false positives retrieved; Harmful-FPR: misleading mismatches; No-Match: queries with no result above threshold.
Cosine Threshold
FPR
Harmful-FPR
EM
R@1
No-Match Rate
0.90
0.15 = 0.02
0.03 + 0.01
0.68
0.75
0.05
0.95
0.08 + 0.01
0.015 + 0.005
0.79
0.82
0.12
0.98
0.04 + 0.01 0.005 + 0.002
0.88
0.90
0.22
Mitigation: Cosine threshold (> 0.98) for high-stakes domains or hybrid with keyword filters reduces FPR by 73% (to 4% + 1%) and harmful-FPR near-zero, the cost of 22% no-match returns. Reranking with a cross-encoder (e.g, MS-MARCO) further improves EM by 5 10% in tests, verifying top-k candidates.
A.13.3 Scaling Similarity Search: Latency and Memory
For large corpora (e.g,, [P] > 10%), exact search is infeasible (O(N) time). We use approximate nearest- neighbor (ANN) indexing for sub-linear scaling:
Implementation:
Pre-compute ((v) for all y E P and index with FAISS using IndexHNSWPQ: faiss. iiddiidxiiii iwriiindddtdrrrree factory HINSW32, where M = 32 for HNSW, efConstruction=200; PQ with m 96 sub-vectors, nbits=8; for larger scales, "IVF4096, HNSW32, PQ96x8" with nlist=4096, nprobe=32.
Latency: Measured NVIDIA A100 GPU (40GB), single-query (batch=1), warmed cache (after 100 queries), median and p95 over 1000 queries. For index_ fatee_fffctor==HNNWWS PQ96x8":
PQ06xx"" median 12 ms (p95
18 ms) for N = 10;; median 45 ms (p95 65 ms) for N = 10 Trade-off: Higher efSearch=256 improves recall (0.99) but increases median latency 1.8x to 22 ms for N = 105.
Memory: Uncompressed embeddings (foat33)): x d x byles 10 x 768 x = 3,072,000,000 bytes
Rs 2.86 GiB. PQ compression (m = 96, nbits=8): codes = N m bytes 10 x 96 96,000,000 bytes s 0.089 GiB; codebook 256 d 4 s 0.003 GiB. HNSW overhead: approximately N 2M bytes (bidirectional int32 links) 10 64 x 4 256,000,000 bytes 0.238 GiB. Total for N 10%: z 0.33 GiB. For billion-scale, use sharded d-HNSW. ~ 12 GiB across 4 nodes (3 GiB/node).
These techniques make deployment feasible: a healthcare corpus (N = 10" notes) achieves 35 ms median latency and m4 GiB memory on commodity hardware. Future work could integrate learned indexes (c.g., LSH variants) to further balance rrcal--lteney-mmmoon
Overall, this analysis demonstrates that STEALTH's topology-preserving mapping enables efficient, privacy- aware retrieval with quantifiable trade-off's and practical mitigations.
A.14 Statistical Analysis: Experimental Validation
This section presents comprehensive statistical analysis of STEALTH's performance across 44 benchmark datascts and 16 encryption schemes, totaling 704 experimental conditions. All experiments use = 5 encryption keys per sample with 128-bit security parameters.
A.14.1 Experimental Configuration
Encryption Schemes. Results aggregate performance across four cipher categories: (i) stream ciphers" (XOR, RC4, Salsa20, ChaCha20); (ii) block eiphers in ECB mode (AES-ECB, Blowfish-ECB, 3DES-FCB)
83

---

Published in Transactions on Machine Learning Research (February/2022
(iii) block ciphers in advanced modes" (AES-CFB, AES-CTR, AES-CBC, 3DES-CBC); and (iv) authenticated encryption (AES-GCM, AES-FAX, AES-SIV, AES-OCB, AES-CCM). Performance demonstrates encryption- scheme invariance with inter-scheme variance o² < 0.001 (detailed analysis in Appendix A.14.4).
Dataset Coverage. The 44 benehmark datasets span nine domains: general language understanding
(n = 12), healthcare (n = 6), finance (n = 4), legal (n = 5), e-commerce (n = 2), technical (n = 2), content analysis (n = 4), reading comprehension (n = 4), and corporate communications (n = 1).
Evaluation Protocol. Each datasct undergoes 5-fold cross-validation with stratified sampling to preserve
label distributions. Metries are computed at the sample level and aggregated using bootstrap resampling (10,000 iterations) to construct 95% confidence intervals.
A.14.2 Aggregate Performance Statistics
Table 26 presents summary statistics aggregated across all 704 experimental conditions (44 datasets x encryption schemes). Results demonstrate near-perfect semantic preservation with minimal computational overhead.
Table 26: Aggregate performance statistics across 704 experimental conditions (44 datasets x 16 encryption schemes).. Values reported as mean + standard deviation with range, median, and interquartile range (IQR).
Metric
Mean + SD
Range
Median
IQR
Semantic Preservation
Cosine Similarity
0.9998 = 0.0007
[0.998, 1.00]
1.00
0.00
BERT FI Score
0.9998 + 0.0007
[0.998, 1.00]
1.00
0.00
BLEU-1
1.0000 + 0.0000
[1.00, 1.00]
1.00
0.00
BLEU-4
0.9980 = 0.0067
[0.97, 1.00]
1.00
0.01
ROUGE-L
0.9990 0.0020
[0.99, 1.00)
1.00
0.00
METEOR
0.9990 + 0.0020
[0.99, 1.00]
1.00
0.00
Compulational Efficiency
Processing Time (s)
1.41 0.51
[0.71, 4.51]
1.25
0.64
Note: Metrics are computed over datasct-level averages and rounded for presentation; small variances arise from domain-specific variations (e.g., Medical and Reading domains).
Statistical Significance Testing. We conduct three complementary hypothesis tests to assess deviation from perfect semantic preservation:
1. Paired I-test plaintext baseline: 1(43) -2.44 p= 0.024, Cohen's 00.35. While statistically significant at 0.05, the small effect size (d < 0.5) indicates negligible practical degradation.
2. One-sample Wilcoxon signed-rank test (Hn: Cosine Similarity 1.00): W 492, p = 0.89. The non-parametric test fails to reject the null hypothesis, confirming significant deviation from perfect alignment when robust to outlicrs.
3. Bootstrap confidence intervals (10,000 resamples with replacement): Cosine Similarity [0.9985, 1.000], BERT Fl E [0.9984, 1.000), Processing Time E [1.38, 1.44] seconds. Narrow intervals demonstrate high precision and replicability.
Interpretation. The parametric paired t-test detects statistically significant difference from perfect preservation (p= 0.024), attributable minor variations in the Modical and Reading domains where longer sequences introduce slight BLEU-4 degradation (0.97-0.98). However, three factors indicate this difference lacks practical significance: (i) small effect size (Cohen's 's d -0.35 0.5), (ii) non-significant robust test.
84

---

Published in Transactions on Machine Learning Research (February/2022
(p 0.89), and (iii) semantic metrics (Cosine Similarity, BERT F1) remain at ceiling levels (> 0.998). The observed variance is domain-driven rather than eneryption-driven, confirmed by mixed-effects modeling (Appendix b.
A.14.3 Domain-Level Statistical Summaries
Table 27 presents comprehensive statisties by domain category. All metrics are reported mean standard deviation unless otherwise noted.
Table 27: Summary statistics by domain category. All metrics reported as mean standard deviation.
Domain
N
Cosine Sim.
BERT F1
BLEU-4
Time (s)
Statistical Test
GLUE
5
1.00 + 0.00
1.00 + 0.00
0.998 + 0.004
1.11 + 0.28
x² 2.1, p 0.72
SuperGLUE
1.00
0.00
1.00 0.00
1.00 + 0.00
0.96 0.26
II 3.4, p 0.76
E-commerce
1.00 + 0.00
1.00 = 0.00
1.00 + 0.00
1.12 0.04
t(1) = -1.5, = 0.38
Medical
6
0.999 + 0.001
0.999 = 0.001
0.998 + 0.003
1.96 + 1.09
F(5, 24) 8.3, < 0.001
Technical
2
1.00
0.00
1.00 + 0.00
1.00 + 0.00
1.55 0.90
1(1) 11.1,, p 0.40
Content
1.00
0.00
1.00 = 0.00
1.00 + 0.00
1.23 0.19
F(3, 16) 2.1, p = 0.14
Reading
4
0.999 + 0.001
0.999 = 0.001
0.985 + 0.015
1.38 + 0.64
F(3, 12) =28,, p = 0.09 5-fold CV: 1.00=0.00
Corporate
1
1.00
1.00
1.00
1.71
Finance
1.00
0.00
1.00 0.00
1.00 + 0.00
1.47
0.19
H(3) 4.2, p = 0.24
Legal
5
1.00 + 0.00
1.00 + 0.00
1.00 + 0.00
1.66 0.14
F(4, 20) = 1.9, p = 0.15
Processing Time Heterogeneity: Medical domain exhibits significant right-skew (Shapiro-Wii) Wilk W 0.78,
0.04) with PubMedQA as outlier (4.51s, +2.3or). Cross-domain ANOVA: F(9, 12.4, < 0.001; Tukey HSD post-hoc reveals Medical vs. SuperGLUE (A = 1.00s, < 0.001). Linear regression (Time
Avg AogTokens + DomainComplexity). B = 0.012s token (p < 0.001), R² = 0.71.
A.14.4 Encryption Technique Robustmess
Table 28
demonstrates performance consistency across 16 encryption schemes, totaling 704 experimental
conditions (16 schemes x 44 datasels).
Table 28: Performance consistency across 16 encryption schemes (704 total conditions).
Cipher Category
N Schemes
Cosine Sim.
BERT F1
Time (s)
Variance Test
Stream (V)
4
1.00 + 0.00
1.00 + 0.00
1.39 + 0.48
o² 0.001
Block-ECB
3
1.00 = 0.00
1.00 + 0.00
1.42 + 0.53
o² < 0.001
Block-Advanced (&)
4
1.00 = 0.00
1.00
+ 0.00
1.41 + 0.51
o² < 0.001
Authenticated (4)
5
1.00 = 0.00
1.00 + 0.00
1.40 + 0.50
o² < 0.001
Mixed-effectss model: Encryption|Dataset (1|Encryption) + (1]Dataset); Encryption random effect variance o2 0.001 (p 0.94), Dataset random effect variance aata 0.26 (p 0.001). Friedman test across schemes: (15) 8.2, p 0.92. Conclusion: Performance is encryption-annotiic; variability is datasot-driven, not encryption-driven.
A.14.5 Multi-Key Sensitivity Analysis
Table 29 demonstrates the effect of key count K on alignment quality, evaluated on the WikiText- 103 test split with N 1, 0.0 samples per K value.
85

---

Published in Transactions on Machine Learning Research Feebuuar/2022
Table 29: Effect of key count K on alignment quality. Results report mean + std over N = 1, 000 test examples (within-subjects design, sampling seed 42). Encryption: AES-256-GCM with 96-bit nonce per-key. Time includes end-to-end processing (encryption + model forward + alignment) on NVIDIA A100 (40 GB).
K Cosine Sim.
BERT F1
BLEU-4
Time (s)
Notes
0.651 + 0.213
0.642 + 0.218
0.588 + 0.241
0.82 + 0.15
Insufficient diversity
3
0.958 + 0.022
0.958 + 0.022
0.951 + 0.024
1.18 + 0.21
Approaching saturation
5 1.000 + 0.000
1.000 + 0.000
1.000 1 0.000
1.41 1 0.28
Optimal (K*)
10 1.000 0.000
1.000 + 0.000
1.000 + 0.000
2.67 + 0.51
Diminishing returns
Statistical Analysis. Repeated-measures ANOVA: F(3, 3996) = 487.3, < 0.001, n = 0.27 (large effect). Post-hoc pairwise comparisons (Bonferroii-correeed)): K = vs. K = 3 (p < 0.001), K = 3 vs. K = (p < 0.001), K = 5 vs. K = 10 (y = 0.94, n.8.). Saturation point: K 5 vs. K = 6 paired t-test: t(999) = 0.08, p = 0.94, Cohen's d 0.003 (negligible). Selection criterion: Elbow method and BIC minimization both converge at K = 5 (BIC; 8741 vs. BIC 8263). Conclusion: K = achieves optimal alignment with minimal computational overhead; additional keys provide stalistical benefit.
A.14.6 Robustness Validation
Leave-One-Out Cross- Validatiin Sequentially removing each domain and recomputing aggregate statisties yields maximum degradation Amax = 0.001 (when excluding Medical domain). Mean absolute deviation across all leave-one-out folds: MAD = 0. 0.0003, confirming minimal sensitivity to individual domain
inclusion.
Outlier Sensitivity. Exclusion of observations beyond +33 (4 outliers identified: from Medical, from Reading) produces negligible metric changes: Cosine Similarity 1.00+0.00 (unchanged), BERT F1 1.00 (unchanged), BLEU-4 0.90 (A -0.001.. Results demonstrate robustness to extreme values.
Hetcrosccdasstccty Testing. Breusch-Pagan test for residual hetcroscedasticctyy: x² 8.4, p 0.59
(homoscedastic residuals confirmed). White's test under alternative specifications: x54 = 48.3, p 0.70, validating stability.
Replication Stability. All analyses replicated across 5 independent runs with different random seeds (42, 123, 456, 789, 1024). Maximum standard error across replications: SEmax = 0.002 for Cosine Similarity, SEmax = 0.008 for processing time, confirming high reproducibility.
A.15 Comparative Analysis: Architectural Necessity for Semantic Isomorphism Enforcement
To isolate the contribution of STEALTH's specialzzed architectural components multi-key variability handling, encryption-aware tokenization, key-attentive transformer layers, multi-key aggregation, and adaptive projection layers we conducted a comparative study evaluating STEALTH against two baseline architectures trained under identical conditions. Specifically, we adapted a standard transformer (GPT-2) and a mixture-of- experts (MoE) model [DeepSeek-1.55)) to process encrypted inputs, training each with the same Semantic Isomorphism Enforcement (SIE) loss function, datasct (a balanced subset of the 44-domain benchmark described in Section 4. encryption schemes (AES-256 in CBC mode, as representative scheme from Table 1), hyperparameters (learning rate 5 x 1 batch size 32, 5 epochs), and multi-key variability (K = 5 keys per input). No other modifications were made to the basclines beyond input adaptation to handle encrypted token sequences.
The goal was to assess whether the SIE loss alone suffices for learning a boolloyy-preeeving mapping Φ: E + P, or STEALTH's encryption-speeffic mechanisms are essential for aligning encrypted embeddings E with plaintext embeddings P. We evaluatod alignment quality through four complementary analyses on held-out test data (n- 10,000 samples): (A) dimension-wise embedding correlation (Pearson r per embedding
86

---

Published in Transactions on Machine Learning Research (obruaryy/02g0
dimension), (B) point-wise value correlation (mapped encrypted values vs. plaintext targets), (C) principal component space topology (PCA-reduced to 2D) for visualization), and (D) cosine similarity distribution (empirical vs, target). These metrics quantify local (dimension/ point-wise) and global (topological/ (aoologica/sstuuuurraa structural) alignment, with perfect alignment indicated by r = 1.0, overlapping PCA clusters, and matching distributions (KS statistic 0).
Figure 42: Comprehensive embedding space analysis across comparative variants. (i) GPT-2 baseline: erratic dimension-wise correlations (mean r = 0.32 + 0.41), weak point-wise linearity (r= 0.43), segregated PCA clusters (variance 77%), skewed cosine distribution (KS=0.26). (ii) DeepScck-1.55 baseline: improved local metrics (mean r = 0..1 + 0.38, r = 0.63), but deficient global aligmment (PCA partial overlap, variance 86%, KS=0.20). (iii) STEALTH: near-perfect alignment (mean r = 0.98 = 0.02, r= 0.99, full PCA overlap, variance 96%, KS=0.03). Each panel shows (A) dimension wise correlations, (B) point-wise correlations, (C) PCA topology, and (D) cosine similarity distributions.
Results are visualized in Figure For the GPT-2 baseline (panel i), dimension-wise correlations are erratic and low (mean r 0.32 = 0.41), point-wise correlations show weak linearity (Pearson r 0.43), PCA reveals segregated clusters (total variance explained: 77%), and cosine similarities exhibit a skewed distribution (KS= 0.26 vs. target). This indicates partial local preservation but failure to capture global semantic structure, likely due to the absence of key-conditioning and multi-key aggregation, which leaves the model vulnerable to encryption-induced variance.
The DeepSeek-1.5B MoF. baseline (panel ii) improves slightly on local metries (mean dimension-wise r 0.51 0.38; point-wise r = 0.63) via sparse expert routing, but global alignment remains deficient: PCA shows partial overlap with residual separation (lotal variance: 86%), and cosine distributions diverge (KS- 0.20) While MoE scaling aids conditional computation, it does not inherently handle encryption's 's pseudorandom perturbations without explicit alignment mechanisms like hicrarchical token-phrase-santene projections.
In contrast, STEALTH (panel iii) achieves near-perfect alignment: dimension-wiss correlations are consistently high (mean r = 0.98 + 0.02), point-wint wise values align linearly (Pearson r 0.99), PCA topologies fully overlap (total variance: 96%), and cosine distributions match closcly (KS= 0.03). This superior performance translates downstream gains, with STEALTH yielding a BLEU score of 0.98 on catrieval-based reconstruction (vs.
0.62 for GPT-2 and 0.71 for DeepSeek) under full-corpus coverage.
87

---

Published in Transactions on Machine Learning Research (Februar/20224
These results empirically confirm that the SIE loss, while critical for guiding topological preservation, requires STEALTH's architectural innovations to fully reconcile encrypted and plaintext manifolds. Without multi-key conditioning and adaptive projections, baselines overfit to encryption artifacts, failing to enforce semantic isomorphism. All experiments were run on identical hardware (NVIDIA A100 GPU) with deterministic seeding for reproducibility.
A.16 Comparative Analysis: Role of the Key Encoder in Multi-Key Alignment
To evaluate the necessity of the key encoder component within STEALTH's architecture architeture--spefffaallly the key-attentive transformer layers and multi-key aggregation modules we performed a variant analysis where these clements were removed, forcing the model process encrypted inputs without explicit conditioning encryption keys. The variant. (STEALTH w/o Key Encoder) was trained under identical conditions to the full STEALTH model: using the Semantic Isomorphism Enforcement (SIE) loss, balanced subset of the 44-domain benchmark (Section 4 AES-256 in CBC mode as the encryption scheme, hyperparameters (learning rate 5 x 10-6, batch size 32, epochs), and multi-key variability (K = 5 keys per input, randomly sampled to simulate real-world key diversity). The key encoder, which embeds and attends over key variants to normalize encryption-induced perturbations, was replaced with static projection layer to maintain parameter parity.
Embedding Space Analysis: Encrypted vs. Plain Text Representations
(a) Dimension-wise Embedding Comparison
(h) Embedding
Migpod Energpied Embolling
Plain Enbedding
15-
Embodding Valun
800
0.0
600
Plain
-0.-
400
-1.0
-1.0
MCE: 0:2077
-1.5
Linear fits D.530k 0.006
250
-1,00
-0.3
50.50
-0.25
0.00
9:25
0.50
0.98
2.00
Embedding Dimeasion
Mapped Encrypled
(c) 2D PCA Projection of Embeddings
(d) Cosine Similarity Distribution
Eucsypted
Distrintiin
Plait
144
0.2022
-e_-tts
Moun: 0.1 1828
1.2
Mindien: 0.1733
Principal Compowet
1.0
Donslty
0.8
0.6
0.4
-2-
0.2
PCZ 20.0%
0.0
Principal Component
Cosine Simillarity
inconsistent dimension- wise correlations (mean r 0.21 + 0.35), weak point-wise linearity (r = 0.17), cluster separation in PCA (variance 82%), divergent cosine distribution (KS=0.31). The figure includes (A) dimension-wiet embedding comparison, (B) embedding value correlation, (C) 2D PCA projection of embeddings, and (D) cosine similarity distribution.
Figure 43: Embedding space analysis for STEALTH without key encoder:
This analysis tests whether the SIE loss can independently enforce semantic isomorphism under key variability, if explicit key conditioning required to map diverse encrypted representations dx Fg. P (for key k))
88

---

Published in Transactions on Machine Learning Research (obbuary//0200
onto unified plaintext manifold P. Alignment was assessed held-out test data (n 10,000 samples) via (A) dimension-wise embedding correlation (Pearson r per dimension), (B) point-wise value correlation (mapped encrypted vs. plaintext), (C) principal component topology (2D PCA), and (D) cosine similarity distribution (empirical vs. target, with Kolmogorov-Smirnov (KS) statistic).
the same metrics as in Appendix A.15
44 Without the key encoder (Figure 43 key-induced variance severely degrades alignment: dimension-wee wise correlations are low and inconsistent (mean 0.21 + 0.35), point-wise linearity is weak (r = 0.17), PCA reveals cluster separation (82% variance), and cosine similaritics diverge (KS= 0.31). Without key-attentive mechanisms to disentangle semantic content from key-specific noise, embeddings fragment and fail to preserve global topology.
Results are shown in Figures 43 and
Embedding Space Analysis: Encrypted Plain Text Representations
(a) Dimension-wise Embedding Compartson
(b) Embedding Value Correlation
Magped Enerypted Emberdinng
Pearson 9908 Ist 3.0006
10000
Eribentding 3
i'amn Thest timbeddnng
8006
Plain Toxt Valaes
Danalty
41000
Carrelations 1 .006
3-
Unber Bt: y- 1,000x 8.000)
2000
50
100
150
200
250
-3
-2
Embbeiding Dimensiooa
Mapped Encrypted Values
PCA Projection of Embeddings
Cosine Similarily Distribution
166
Muggord Encrygted
3.5
Distribolion
Plain Tex:
KDB
3.0
Nox
1.0000
0000
Median: 0000
Prhocipal Companart?
0.00015
Density
2.0
0.00110
0.00005
1.0
0.00000
0.5
PCL. 4%
-3.00005
0.0001 12500000 lano, 1750,000 20000200
2250.001 250000011255
-0.4
-0.2
0.3
0.2
0.5
Principal Comparent
Cosime Similority
Je-Ge1
Figure 44: Embedding space analysis for full STEALTH with key encoder: near-perfect fimension-wiso correlations (mean 0.99 + 0.01), linear point-wise correlation 1r=0),0,, complete PCA overlap (variance 98%), aligned cosine distribution (KS=0.01). The figure includes (A) dimension- wise embedding comparison, (B) embedding value correlation, (C) 2D PCA projection of embeddings, and (D) cosine similarity distribution.
In contrast, the full STEALTH with key encoder (Figure 44 achieves superior alignment: dimension wise correlations are near-perfect (mean r = 0.99 = 0.01), point-wiss wise values correlate linearly (Pearson 1.00), PCA topologies overlap completely (votal variance: 98%), and cosine distributions align closely (KS= 0.01). Downstream, this translates to a BLEU score of 0.99 for eetrivval-baser reconstruction under full-corpus coverage, compared to 0.54 for the bascline variant-al an 83% relative improvement.
These findings demonstrate that the key encoder is indispensable for robust multi-key alignment, as the STE loss alone cannot mitigate encryption variability without dedicated conditioning.
89


Reviews:
[{"review_id": "qMq9o67E7x", "reviewer": "Reviewer_zmKZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors consider legal restrictions that prevent leveraging recent technologies like large language models (LLMs) for analysing sensitive data. They mention that, while there is a large potential to automate and improve analyses of, for example, financial or medical data, laws prevent transmitting and/or sharing such data in plain text. And in an encrypted form, such data cannot be analysed by LLMs. This is due to the fact that encryptions are designed to \"remove\" statistical regularities in the data to make it impossible, or at least extremely hard, to reconstruct the plain text from the encrypted version without knowing the encryption key. To enable applying LLMs to such sensitive data, the authors propose to learn embeddings of the encrypted data that preserve semantics in a sense, making it possible for LLMs to operate on encrypted data. To learn such semantics-preserving embeddings, the authors design what they call the \"Semantic Isomorphism Enforcement (SIE) Loss Function\", and use it to train a transformer on 44 datasets where SIE enables near-perfect performance as measured with scores that compare the similarity between plain-text and \"encrypted\" embeddings. To my understanding, SIE essentially ensures that (i) encrypted data is embedded at similar locations when it comes from the same plain text, even when using different encryption keys, and (ii) encrypted data that comes from different plain text is embedded at different locations.\n\n\n### Strengths\n- Figure 1 is a good high-level illustration of the differences between STEALTH and other LLMs.\n- The SIE loss intuitively seems to make sense for achieving the goal of producing embeddings of encrypted text that do not lose semantics as compared to their plain-text counterparts.\n- Both a strength and a bit of a concern: the empirical evaluation suggests that STEALTH produces ciphertext embeddings that (almost) do not lose any information compared to their plain-text equivalents, and achieve near-perfect evaluation scores. This is great, but it also seems a bit too good to be true.\n\n### Weaknesses\n- Figure 2 shows a lot of information, and it was difficult to figure out where I should focus my attention. I first looked at it on a reMarkable in greyscale, which made it extremely difficult to get an overview. But even in colour, I am still not sure how this figure can help my understanding.\n- There seem to be a couple of minor inconsistencies in the notation (more details below under Requested Changes).\n- It seems as though this work takes a lot of effort to obtain embeddings for ciphertext that are \"semantically equivalent\" to the embeddings for the corresponding plaintext. The authors claim that \"strong cryptographic hygiene is preserved\", but at the same time, they say that \"STEALTH's reconstruction relies on semantic similarity search within a large, domain-aligned plaintext corpus during inference\". Together with their initial motivation that sensitive data cannot be processed by LLMs due to legal reasons, I do not understand how a model that requires plain-text data during inference follows those legal constraints and can be said to preserve \"strong cryptographic hygiene\". (It may be possible that I did not understand some parts.)\n- Connected to the previous point, the confidentiality of data processed by STEALTH appears to rely on the trustworthiness of some central authority that owns, or \"guards\" the data. (Again, I may have misunderstood something, but that is my interpretation of the *Threat model and operational scope* paragraph where the authors talk about a \"system owner\".)", "claims_and_evidence_yn": "No", "claims_explanation": "As far as I can tell, the authors support most of their claims with evidence. However, I am missing one thing: the authors claim that STEALTH \"[demonstrates] negligible utility degradation compared to plaintext processing\"; however, I did not see any evidence showing that the encrypted data can be used as a drop-in for the plaintext data while achieving the same performance in some downstream task. Or is this not what is meant here?", "audience_yn": "Yes", "audience_explanation": "I suppose the loss function may be interesting to readers who work on related applications.", "requested_changes": "### Critical points\n- The authors mention that \"rights such as erasure, access, and rectification\" complicate the adoption of LLMs, which I believe is a valid point. What I am missing is that the authors discuss STEALTH in this context and how it adheres to legal restrictions. Is it possible to ensure that rights for erasure, access, or rectification can be exercised?\n- I found it very difficult to read the related work section because the references were not in parentheses. On a computer, with highlighting around the references, it was a bit easier, but on a reMarkable in greyscale and without highlighting, it was hard to tell what part was text and what was a reference. Please use at least `\\citet` instead of `\\cite`. (Unless if the format is okay as per instructions by TMLR.)\n- How does STEALTH achieve \"practical performance avoiding quadratic scaling\"? Specifically, I believe it should be made clearer how quadratic scaling is avoided.\n- There is some inconsistency with the symbols. On page 3, $E$ refers to an \"encrypted embedding manifold\", but on page 5, $E$ refers to a mapping from plain to ciphertext. Also, the symbol $\\mathcal{X}$ appears on page 9, but I do not believe it has been defined before; did you mean $\\mathcal{D}$?\n- Can you explain why SIE enables near-perfect reconstruction? The scores that are near 100\\% almost everywhere seem a bit too good to be true. It is also mentioned that STEALTH \"[demonstrates] negligible utility degradation compared to plaintext processing\", however, it is unclear to me how the authors have evaluated that. To me, it sounds like this claim says that, with respect to performance, there is no difference between working with encrypted embeddings or with plaintext embeddings. However, unless I have missed it, the authors have not shown that this is the case.\n\n\n### Not critical, but would strengthen the paper\n- The authors mention that applying LLMs to \"clinical notes, transaction records, or legal documents could deliver substantial societal benefit\", but they don't provide any concrete examples. I believe it would strengthen the paper if the authors could mention a few concrete examples. I also find that the motivation has a relatively negative tone and almost sounds like legal restrictions are a negative thing. Perhaps the authors wish to phrase this a bit more neutrally.\n- I didn't quite know what to make of the concept of a \"threat model\", and believe the same might be the case for other readers. I believe it could be beneficial to be a bit more explicit about that and give a brief introduction. For example, say something about the \"authorized-key model\" (I am unsure whether this relates to private-public-key systems.)\n- There seems to be a mismatch between the loss formulation in sections 3 and 4. In section 3, the loss $\\mathcal{L}_\\text{total}$ contains only 3 terms, but 4 parameters are mentioned: $\\alpha, \\beta, \\gamma, \\delta$. In section 4, there are 4 terms. Maybe there is a reason for omitting one of the loss terms in section 3, but I do not see it.\n- What is the reason for considering $1-\\frac{uv}{|u||v|}$ as cosine similarity instead of the more common variant $\\frac{uv}{|u||v|}$? This might be interesting for the reader.\n- Could you give a concrete example for \"increasingly abstract representations\" as mentioned on page 8?\n- The *Attention-Based Aggregation*, mentioned on page 8 is said to \"dynamically [weigh] variants based on semantic clarity\". How is semantic clarity measured?\n\n\n### Minor points\n- What does the notation $[0,:]$ on page 8 mean, or is that a typo?\n- The appendices are referenced in seemingly random order; perhaps it would be nicer to list them in the order they are referenced in the text.", "broader_impact_concerns": "I believe these are more legal than ethical concerns, however, they seem to fit under broader impact: \nBecause the authors mention a \"system owner\" of data and/or STEALTH, it seems like it may be appropriate to discuss a bit more how STEALTH would be implemented and used in practice.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pVHETVEH0Y", "reviewer": "Reviewer_gCkq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces STEALTH, a plug-and-play framework, built to address the existing limitations of computational limits, poor generation quality and information leakage. It attempts to use representation alignment to provide safeguards, ensuring efficiency, isomorphism and composibility. The writing is well-founded and understandable. There is also an interesting architecture at use, although a bit computationally complex, for aggregation of keys.\n\n---\n\n### Weaknesses:\n- If I'm correct, STEALTH produces a latent vector, which acts as an input to the LLM. Current LLM vendors do not accept embeddings as an input and hence, the only situation when this would be useful is for open-source models, hosted on private infrastructure. If hosted on private infrastructure and these LLMs are open-source, I do not see the major motivation of STEALTH. Can the authors clarify this?\n- I am very impressed with the author's results on all the benchmarks, but I'm curious about how inference works. Looking at Algorithm 1, I see that the authors essentially are computing the nearest neighbor from a given dataset. That given dataset contains original embeddings and we are comparing the embedding with the encrypted embedding. For this to work well, the plaintext corpus must already contain a close match to the encrypted input. Otherwise, the nearest neighbor could be semantically wrong or noisy. This significantly weakens the paper since it seems to be recovering from a known corpus (i.e. this is more retrieval, instead of generation).\n- I also fail to see how this $f_\\theta$ sufficiently correlates to the given embedding function g, apart from one of the loss functions. \n\n---", "claims_and_evidence_yn": "Yes", "claims_explanation": "Majority of the claims in the paper have been experimentally verified. However, point 2 under weaknesses significantly weakens the experiments' credibility.", "audience_yn": "Yes", "audience_explanation": "Yes, I think this is definitely an interesting line of work that would interest the community of AI Safety. However, I feel like the paper's motivation should be more well-founded and motivated better.", "requested_changes": "Majority of the changes have been listed in the Weaknesses, regardless, I have a few minor changes / questions here.\n\n- Figure 2 is a bit too convoluted to understand, but the description is sufficient. A clearer diagram would suffice.\n- Is the tokenization process and the definition of $A_{cipher}$ valid? I would think that certain encryption methods would introduce characters outside of UTF-8 that could break this process. \n- The dimensionality math for Embedding Layer Architecture in Section 3 is incorrect. Please review. \n- Please use /citep in the Related Works section, it significantly affects readability.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yKKYuDOV2Y", "reviewer": "Reviewer_jkzP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper contributes a new strategy for secure text processing under an authorized key encryption model. The idea consists of using a transformer under a custom loss function to embed cyphertexts (in the context of the keys used to generate them) into a representation space which preserves ordering of plaintext similarity, and ensures that encryptions of the same plaintext using different keys are tightly clustered. The paper evaluates this method primarily experimentally, with some theoretical consideration in the appendices.", "claims_and_evidence_yn": "No", "claims_explanation": "The experimental results which are included in the main body are quite brief. The recovery accuracy is very impressive -- but things should be re-arranged to bring the more in-depth experimental evaluation out of the massive appendix and into the main body. \n\nI mostly found the exposition clear, although there are some serious weak points. In particular, what the actual setup is not very well-explained and I had to piece it together using material from the appendices: Basically, the idea is to decode cyphertext and key pairs by embedding them with the transformer, and then compare against embeddings of a corpus of domain-relevant text to find near neighbors which serve as the decoded plaintext. The text and notation in the sub-sections 3.1 and 3.2 is quite misleading about this, and the decoding procedure is not clearly explained except in the appendix.", "audience_yn": "Yes", "audience_explanation": "I have only a passing acquaintance with cryptography and no knowledge as it applies to machine learning. I'm not sure whether the \"authorized key\" threat model considered here is ecologically valid and no references are provided within the paper. Providing the cypher keys seems like a huge assumption; I suppose this is meant to be used in cases where the keys do not directly allow the plaintext to be decoded. In any case, the fact that a transformer can learn to invert a pseudorandom number generator using a corpus is an interesting finding in its own right.", "requested_changes": "* Critically, clarify the introduction and methodology sections to explain the method more clearly.\n\n* Critically, re-organize the paper to include more experimental evaluation of the method in the main body.", "broader_impact_concerns": "Broader impacts seem mostly positive, although there is no statement to that effect -- and the ethical considerations around facillitating \"secure\" (in the cryptographic sense) processing of people's sensitive information by LLMs are worth discussing.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)