Unextractable Protocol Models: Collaborative Training and Inference without Weight Materialization
Authors: Alexander Long
Chamin P Hewa Koneputugodage
Thalaiyasingam Ajanthan
Yan Zuo
Gil Avraham
Violetta Shevchenko
Hadi Mohaghegh Dolatabadi
Sameera Ramasinghe
Venue: NeurIPS 2025 poster ()
Keywords: Decentralized training
LLMs
Distributed training
Open source
Weight secrecy
Primary area: infrastructure
TL;DR: We provide a decentralized framework for collaborators to run large model computation (training/inference) without any collaborator getting full access to the model.

Abstract: We consider a decentralized setup in which the participants collaboratively train and serve a large neural network, and where each participant only processes a subset of the model. 
In this setup, we explore the possibility of unmaterializable weights, where a full weight set is never available to any one participant.
We introduce Unextractable Protocol Models (UPMs): a training and inference framework that leverages the sharded model setup to ensure model shards (i.e.,, subsets) held by participants are incompatible at different time steps. UPMs periodically inject time-varying, random, invertible transforms at participant boundaries; preserving the overall network function yet rendering cross-time assemblies incoherent.
On Qwen-2.5-0.5B and Llama-3.2-1B, 10 000 transforms leave FP32 perplexity unchanged ($\Delta$PPL$< 0.01$; Jensen–Shannon drift $<4 \times 10^{-5}$), and we show how to control growth for lower precision datatypes. Applying a transform every 30s adds 3% latency, 0.1% bandwidth, and 10% GPU-memory overhead at inference, while training overhead falls to 1.6% time and < 1% memory. We consider several attacks, showing that the requirements of direct attacks are impractical and easy to defend against, and that gradient-based fine-tuning of stitched partitions consumes $\geq 60\%$ of the tokens required to train from scratch. By enabling models to be collaboratively trained yet not extracted, UPMs make it practical to embed programmatic incentive mechanisms in community-driven decentralized training.

Unextractable Protocol Models: Collaborative Training and Inference without Weight Materialization
Alexander Long"
Chamin Hewa Koneputugodage"
Thalaiyasingam Ajanthan
Yan Zuo
Gil Avraham Violetta Shevchenko
Hadi Mohaghegh Dolatabadi
Sameera Ramasinghe
Pluralis Research
Abstract
We consider a decentralized setup in which the participants collaboratively train and large neural network, and where each participant only processes a subset of the model. In this setup, we explore the possibility of unmaterializable weights, where full weight set never available to any one participant. We introduce Unextractable Protocol Models (UPMs): a training and inference framework that leverages the sharded model setup ensure model shards (i.e., subsets) held by participants are incompatible at different time steps. UPMs periodically inject time- varying, random, invertible transforms at participant boundaries; preserving the overall network function yet rendering cross time assemblies incoherent. On Qwen- 2.5-0.5B and Llama-3.2-1B, 10 000 transforms leave FP32 perplexity unchanged (APPL< 0.01; -Shannn drift < 4 10-"), and we show how to control growth for lower precision datatypes. Applying a transform every 30s adds 3% latency, 0.1% bandwidth, and 10% GPU-memory overhead at inference, while training overhead falls to 1.6% time and < 1% memory. We consider several attacks, showing that the requirements of direct attacks are impractical and easy defend against, and that gradient-based fine-tuning of stitched partitions consumes 2 60% of the tokens required to train from scratch. By enabling models be collaboratively trained yet not extracted, UPMs make it practical to embed programmatic incentive mechanisms in community-drrver decentralized training.
Introduction
Foundation models are trained massively distributed infrastructures [3 35
and there and 47). Even though decentralized systems may facilitate volunteer -aauneer--bsed, multi partippatty training: without way for contributors to recoup the hundreds of millions training costs 23 4T, such collaborations remain economically infeasible at large scales 57]. A viable solution is to design incentivization strategies to allow value flow from model usage to the training contributors, however, this is not possible the model is freely accessible to participants (as with a full weight set participants can independently capture model value).
growing interest in scaling beyond centralized computing clusters via communication-effiae
fault-tolerant decentralized systems [1]
28
51
50
To resolve this seeming contradiction, how can a model be collaboratively trained and hosted, while the weight set remains private, we introduce Unextractable Protocol Models (UPMs). UPMs utilize structural properties of the Model Parallel (MP) 17 54 training setup, where the neural network is sharded into small subsets and split over devices (in our case, participants), and no one device accesses the full model at any time. This is novel setting, where the model architecture, hyperparameters. and datasets can be open-sourced, yet full weight set never exists in any location at any one point.
Equual contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In the decentralized case, a motivated attacker may attempt to periodically rejoin training in different stages to obtain a full copy the model. UPMs address this by making the weights of the model shards time-dependent and incompatible across time-steps. Periodically, time-varying random invertible transforms are morphed into the model weights at the stage boundaries. These transforms cancel out at the same time step, preserving the end-to-end network function, however, they are incompatible across time steps, ensuring that stitching shards from different times results in an incoherent model.
We show that our morphing approach is compatible with various neural network architectures, including transformer layers [60], and discuss the efficacy of different types of transformations, showing virtually no performance degradation and negligible communication and memory overhead for inference. Furthermore, we analyze strong learning based attacks to stitch shards from different time steps, and show that recovering the full model functionality requires training costs in the same order as training the full model from scratch. Finally, we discuss how our framework affects optimization, and show that training is unaffected for compatible optimizer.
We make the following contributions:
We introduce Unextractable Protocol Models (UPMs), a framework enabling collaborative training and inference without allowing any participant to extract the full model weights.
We provide thorough analysis of UPMs, identifying which architectures and layer types they support, the key requirements for the transforms, how they affect optimization, and potential attack scenarios along with practical defenses.
We empirically demonstrate on billion- billion-parameter parameter decoder- based language models that UPMs do not alter the model performance even with up to 10k morphing steps, and the communication and memory overhead is negligible compared to inference/training cost.
2 Setting: Decentralized Models and the Weight Materialization Threat
We first introduce the decentralized model setup (i.e., the protocol), where participants collaboratively train serve a model. To ensure economic viability, we argue that the full model weights cannot be allowed to materialize, motivating a design where each participant processes only a model subset. We then examine potential extraction attempts, where adversaries try to reconstruct the full weight set across time steps. The next section introduces UPMs, which prevent this by ensuring that weights across time steps form an incoherent model, while preserving exact functionality within each step.
2.1 Decentralized Protocols for Large Models
We consider decentralized protocol that enables collaborative training and inference of large models by pooling participants' compute resources. This level of compute not within the reach of a small group of individuals and therefore requires large group of participants. The setting is inherently trustless, we can assume an honest majority but must assume there are malicious actors.
Economic feasibility. For such protocol to be economically feasible, it must be able to derive value from the trained model. The key to this, explained in Section A is to ensure that the trained model's utility is excludable: accessible only through the protocol. As long as excludability holds, access can be priced and revenue shared among participants.
To achieve excludability, we propose guaranteeing unextractability; preventing participants from being able to extract the full weight set. This means the full model only exists within the protocol, preventing weight materialization for any participant. We show that unextractability can be ensured (with high probability) with a lightweight construction that uses the structure of the model setup, as opposed to using expensive cryptographic primitives. Notably, code and data remain open, enabling open-source development of large models while preserving economic fcasibility.
We also need to consider partial excludability, where an attacker can produce a model M' with equivalent performance to the protocol model M using information gained during participation (see Section A We quantify excludability by defining the excludability ratio E, C(M') where C(M)
C(M)
is the (extremely large) compute required to train M in the protocol, and C(M') is compute required to produce M (both participating in the protocol and extra training). If E is very low, so C(M') is within the reach of a small group of individuals, the protocol is not economically viable.

---

R Replicas of each Pipeline
C' Compute Nodes
Stage
Stage
Stage
A Attacker Nodes
Time Steps
Figure 1: pipeline parallel setup with R pipelines and S stages, where each stage is occupied by one of C' compute nodes. Participants can operate multiple nodes within stage (color indicates participant identity). Nodes in red indicate an attacker, who hides their identity try and get access to different stages of the model over time.
Model setup. We require each participant to only hold and processes a subset of the model. This be done through Model Parallelism (MP) [14 [54]. which divides the model into shards distributed across devices. We consider special case of MP, Pipeline Parallelism (PP) [4] [7]. where each shard contains consecutive block of layers called pipeline stage. Participants only communicate with adjacent stages, avoiding the costly all-to-all communication common in other MP strategies.
Assume the model F: X + y is divided into S sequential pipeline stages. The model computes
F(Xo) = fs o fs-1 .... o fi())),
X; ((x--100)) for i S,
(1)
where X; are the intermediate model outputs (i.e., activations) with Xo the input to the model, and stage Sa has parameters 0,. The protocol maintains replicas each pipeline, both allowing to scale for more participants and compute (data parallelism) and ensuring fault tolerance (stages are not lost a participant drops out).
Suppose a compute pool of > RS nodes from all participants. The protocol randomly allocates nodes to each of the RS slots, each specific replica of stage, without revealing the stage identity. The protocol can implement stricter measures, such as ensuring that a participant can only hold slots from a single stage. We assume that the total required compute for the protocol is much larger than any participant can provide, any attacker (or attack coalition) has total compute A where < RS < C. Thus, an attacker cannot have access to stages at any given time. However, they can hide their identity and access different stages over multiple time steps, which we discuss below. Our overall setup illustrated in Figure]
2.2 Threat of Weight Materialization
Let us consider how an attacker (or attack coalition) can attempt to extract the full model weights over T time steps. Note that the attacker controls small fraction p < 1 of all nodes, so the probability of the protocol allocating a particular slot to the attacker is approximately p.
For the attacker to be successful, the attacker must access at least one slot from each of the S stages. an attacker has multiple nodes, they can masquerade as multiple participants simultaneously to try and access multiple stages. Since p < 1, it is highly unlikely that the attacker will be assigned to every stage, fact it likely that < S. However, by repeatedly leaving the protocol and rejoining new participant, the attacker may gain access all stages over many time steps. The probability of this happening within time steps is approximately (pRT)S (derived in Section C). Note that training and inference require extremely large number of time steps, even small p can ensure a successful attack once T pR
Such piecewise Sybil attacks pose a threat of weight materialization. One can try to deter this via centralized authentication systems or cryptographic solutions (3] but they require additional constraints on the protocol and incur prohibitive computation and/or communication overhead at scale. Instead, we leverage the sharded model structure itself to enforce weight unmaterializability.

---

(A)
(B)
(C)
Introduce ideatity transform between neighbsess
Initial model F
Trandormer Layer
X
Vi
Uni
9(X)
Nosmi (2),,
Norm Uv
x(x) xT, 10:7m
Splii transform I inlo transforms and 7.-
Break up into subfonctions
Norma (2), Us
V
T.
T.
U1i
x Norm(X)TT,a Ue:
Fold transforms into neighbours
Group into pipeline stages fi and simplify
s(x) Q.77
Wiali
VIT
T U4-1
X22)
Figure 2: Our framework ensures stage weights vary time by folding in transforms, yet the end-to-end function is preserved. (A) Our transform inducing process. (B) Our framework reasons about subfunctions 9;. which are grouped into pipeline stages. (C) A transformer layer as a stage with transforms applied. Note that every weight is morphed: Time must align with T;-1 and T; due to the skip connection, meaning compatibility at stage boundary also depends on the other boundary.
3 Unextractable Protocol Models
To prevent an attacker from stitching together weights collected at different times, we make the weights time-dependent while ensuring the end-to-end network function is unchanged at any point in time. We achieve this by periodically morphing the weights with transformations that cancel out, thus weights collected at different times are incoherent as their integrated transforms do not cancel out. Note that the transforms are repeatedly applied to weights without removing old transforms, and knowledge of the transform is discarded after the morphing.
Every morphing step, where transforms folded into the model weights, defines a new transform time step in the protocol. Consider two neighboring stages fa and fi+11 with weights 0,(() and (1) at transform time step We morph weights that 0,(4) and 0.+1 (1) remain compatible, but (t) and (t') for t * t' are incompatible. As shown in Figure 2 (A), this is done by introducing an identity function between the neighbors, decomposing it into a random transform and its inverse, and then folding them into the weights of the neighbors. This operation leaves the end-to-end function F' unchanged, though at each time step the intermediate activations between stages lie in different space. Consequently, (t) and (t') are incompatible, as (t+) is only compatible with activations produced by 0: (t'). We now formalize this process.
3.1 Approach
The key to our approach is to identify which weight matrices need transforms applied to them and how to apply those transforms. This involves decomposing the model into smaller subfunctions that allow transforms to be applied between them.
Valid subfunctions.
We first define a valid subfunction as a subfunction of the form
gi (X) O; V;
where U, and Vú are weight matrices, which we call the entry and exit matrices of the subfunction, and o; is any (possibly nonlinear) function. Let d denote the output dimension of 9:-
We now show how to apply transforms between valid subfunctions so they become incompatible
across time steps. Consider two consecutive valid subfunctions gi & (XU.) V; and gi+1 Di+1 (XU++1) Vi+1, composed as 9c+1 gs. As shown in (see Figure (A)), this done by
1. inserting identity transforma between them: 96+1 o 9i 9/+1 o gs. I(X) Xla.
2. splitting it into a random transform and its inverse: T(X) = X-T- GL R and
3. folding the transforms into the weights: V; + V.T. U41 T-
2hhis time step may be different from the time step of inference calls or training iterations.
Group of d x d invertible matrices over JR

---

This procedure is called morphing step and advances the protocol a transform time step. We index components by time step, e.g., (t( = (t( (t). In practice, the morphing step between 1 and consists of: 9i generates the random transform pair (1)) and T(1) ((t)-1 sends it to Ji+1. both fold them into their weights and then discard them. Since the transforms are discarded, we consider them to be ephemeral, they are baked into the weights and there is no other knowledge of them.
To see why this causes cross-time incompatibility, assume we have access to gi time and 9++1 at time 1' with & < t'. Then we have access to Vi(t), [t)), (t+) and Ti()). However, Vi(t) and (t') cannot be used together: if (X) = Z before any transforms are applied, then using (t(t) within 9: yields ZT; (t) but (t') expects
Thus, (t) and 1 are inconsistent, their composition no longer preserves g++1 o gs. Aligning the weights requires the bridge matrix T = (t + 1). Ti..)((("), but we only know the last term (t'',, so we only know the bridge matrix for t' = + 1. For t' > ++1 1, aligning the weights requires guessing random invertible matrix. Finally, to prevent attackers from rejoining the protocol within consecutive time steps, the protocol enforces two-step joining delay.
Allowing identity entry/exit matrices. As $ can be anything, the main constraint in Equation 2 is the need for there to exist entry and exit weight matrices. We now relax this constraint so that at a boundary gi 1i+1, only one of Vi, Ul+1 need to be weight matrix; the other we can form as an explicit identity matrix that gets morphed. For example, if our subfunction does not have a weight matrix Vi, then we replace it with Q, where Qo = La- (,(t) and (t') will remain incompatible and require a bridge matrix. However, we cannot have both Vú and U/+1 as identity matrices; although (t) and Q++1(t)) stay incompatible, they contain no information, allowing an attacker to remove them without altering the function.
Many neural network components can be expressed as compositions of these subfunctions. For example, MLPs can be written as compositions (X) (XU,) (XU;)Q with element-wise activation function a, or equivalently (X) = a = oQ.
Extending to complex components. We extend valid subfunctions to multiple inputs and outputs by generating a transform at each exit matrix, reusing transforms if the exit matrices map to the same input (or have the same dimension). Examples include RNN blocks H = (H,HHU +XUUn22) V Xi+1 = & (H, + X2Ua2)Vaa or self attention SA(X) (XUUq. XUK. XUv) (here & is multi-headed attention). Subfunctions can also be combined: scaling in depth by composing multiple subfunctions, scaling in breadth by applying in parallel like skip connections: (+XU,)V,, XQ.). or multiplying them for gating: Σe GA &
Generalizing subfunctions. Ultimately, what matters is embedding the inverse transform into the next subfunction's 's weights while preserving the overall composition, leading to subfunctions of the form gi (X) = & (Ω (Q(X)U)) V; where (x = T; (())) V; for some morphing function U, + U; that involves T-1 and preserves the shape of U.. For example, convolution layers can be expressed as 9r (x() = & (im2col(X)U:) where X E IR. xd; im2col(X) E R R.d....... U, e (Pdd and P is the convolution patch size. Thus for - E Rd xdi-1, Xm2cco((TT)] ),, = im2col(X)(IP T T',, so the inverse should be folded into U/ as (Ip 2 T
Normalization Layers. Normalization layers pose a challenge, as they alter subfunctions to be of the form & (Norm(X)U.(1) V,, The issue is that this is a non-linear transformation of X before the matrix multiplication with U,, so our previous strategy does not work. However, we propose the following workaround for RMSNorm layers, which have been shown to match or exceed LayerNorm and BatchNorm performance across many tasks while also being more efficient [67]. Given X E bbx with rows/instances 2ii), RMSNorm is of the form
norm(X (X) = diag ((a-)1ad0)))) -1/
RMSNorm(X) = norm(X) (X) Xdiag(w),
(3)
where norm(-) applies row-wise RMS normalization and w Rd are the feature scaling parameters.
We introduce a transform accumulation matrix into the normalization: xXrm(X) - norm ( with (0) initialized as an orthogonal matrix. Then transforms are also folded into (1), hiding them and Q(0), while preserving the norm due to the orthogonality of Q(0). We also absorb diag ww) into U, so that transforms can be folded into the latter, (.) diag(w) U, (during training this is equivalent to removing w). We now have the desired functional equivalence (see Section D for
5

---

details), after morphing step Q: + +T Qi and Uj -
Ji (Xi-IT) & (norm (X;- ( V & (RMSNorm(XX-.Xxx-11U)) V,
(4)
(5)
Full model morphing. Having established how to apply transforms between neural network components, we now consider the overall model. As shown in 2 we decompose our model into such subfunctions, and group contiguous subfunction blocks into pipeline stages. Within each stage, redundant transforms are removed.
Extraction probabilities. Our construction makes certain stage matrices or their generalized grmms-dependen on transforms at the stage boundaries. When these weights depend on transforms from only one boundary, we call it partial incompatibility. For UPMs with partially incompatible stages, the probability of an attacker accessing all stages within T time steps ((T 2) x (3)R) 8-1 (derived Section Although this offers stronger protection than having unextractability, it remains relatively high practical settings.
the entry/exit
Ideally, we would have full incompatibilityy, where least one weight in each stage depends on the transforms from both boundaries. Skip connections inherently achieve this; thus transformer layers naturally satisfy full incompatibility (see Figure 2 (C), where both transforms are applied Q at each skip connection). For UPMs with fully incompatible stages, the approximate probability is now (T' (3p3R) (see Section C). which is very low for realistic settings (provided R remains small).
3.2 Transforms
Transforms must be sufficiently random to resist brute-force or bias-exploiting attacks, strongly alter weights to prevent easy inversion, and remain numerically stable with minimal floating-point ppint error. We find that these properties largely depend on the transforms' singular values (detailed in Appendix(E) In particular, controlling the condition number is erucial, since numerical error increases exponentially with Thus, we select two transform classes:
Haar orthogonal matrices. Orthogonal matrices (QQT= = 7) have unit singular values, minimal numerical error, determinant =1, and easily computable inverse (T-1 T Uniform sampling from the Haar distribution ensures no exploitable bias.
Low-condition number matrices. To introduce higher frequencies while maintaining numerical stability, we use matrices of the form UDVT, where U, V are Haar orthogonal matrices, and D diag( ")) u, u[- Singular values now lie within "ee=",see] bounding condition number by e2 ~ 11 2s. We cycle scale factor to prevent cumulative scale drift.
3.3
Training
While forward-past equivalence under transforms is straightforward, ensuring correctness in the backward pass requires extra care. Transforms affect weights and gradients differently: weights transform W(ry WT then its corresponding gradients satisfy G(r) GT-T, For optimizers using only the first gradient moment M(), we change the update step from ((++1) W(1)
(f(((()) (() to w W - nf (G (+T--))) Furthermore, for some optimizers this is equivalent to update steps with the untransformed model when using orthogonal transforms:
w(+++) (W()) nf ))) (T) We implement training with Muon which satisfies
(T)
both properties. Further details and derivations appear in Section
3.4 Inference
Although morphing steps leave the forward function unchanged mathematically. repeated transform folding accumulates numerical precision error. During training, gradient descent corrects such errors, but at inference there is no correction mechanism. We mitigate this by storing gigh-precision weights on disk and keeping a low-precision copy in GPU VRAM for inference. apply the transform to the high-precision weights on disk, cast them to low precision, and reload them into GPU VRAM. Since transforms occur infrequently, tisk-to-VRAN transfer overhead is negligible. Floating-pont -pinnt error analysis in Section E.2 confirms this effectively limits error accumulation.
each transform step, we

---

10
Jensen-Shanmer Distance
10-4
Perplexity Increase
Gwen BF16
Gwen FP16
Qwen FP32
BF16-HP
10a
Qwen FP15.HP
Qwen FP32 HP
10
10-7-
Lima BF16
10-9
10:4
Lama BF16 HP
Liama FP16-HP
10"
10'
107
10a
10a
10a
107
10
10'
10*
FP32
Morph Steps
Morph Steps
Figure 3: Functional Equivalence during Inference. We evaluate the cumulative effect of transforms by measuring logit drift (via Jensen-Shannon distance) and perplexity increase on the WikiText test split across different numbers of orthogonal morphing steps. Results are shown for Qwen 2.5 0.5B and Llama 3.2 IB across multiple precisions. Solid lines denote our high-precision (HP) workaround described in Section 3.4 Without the workaround, low precision leads to logit drift and rising perplexity, whereas with it, both remain stable over 10k steps.
4 Related Work
4.1 Decentralized and Federated Learning
Collaborative training primarily uses Data Parallelism (DP), where nodes store full model replicas,
limiting scalability and exposing models to extraction |||
Altemattve methods like
SWARM [5]] and Tasklets [66] shard models across nodes, and Hivemind (28) and Petals [6] futher
improve decentralized inference with throughput reaching tokens/sec for 70B model. However, these methods have no mechanism to prevent model extraction. Federated Learning (FL) [30] 36 20
keeps data decentralized, emphasizing privacy, and has been extended both large models
and incentives [21 43.. Unlike FL., our work targets weight secrecy and robustness against untrusted participants, addressing fundamentally different challenges.
4.2 Communication Efficient Decentralized Training
Since decentralized training operates low-bandwidth and high-latency networks, communication efficient methods that preserve training performance are essential. Many methods focus on the data parallel (DP) setting, where weight gradients need to be synchronized between devices 761 5 M
68). However, they are insufficient for the pipeline parallel setup we consider, where activations and their gradients are also communicated between devices. Recently, several approaches have emerged to address this gap. Ajanthan et al. [2] further improves approaches that use pipeline schedule strategies to mask communication overhead by making optimization asynchronous. Ramasinghe et [47]
losslessly compress activations and their gradients, and has been deployed in a public decentralized run As a result, pipeline parallel decentralized training is now well established.
4.3 Model Extraction Attacks
Black-box extraction attacks aim replicate model capabilities without direct access. Tramèr et al. (59 demonstrated feasibility through prediction APIs, and Krishna et al. [2 extended this to
neural language models via transfer learning and targeted queries. Wallace et al. [62 later extracted
translation models via imitation learning, and Lee et al. 229 recently showed projection layers could be stolen with limited API queries. In contrast, address extraction threats in collaborative training, where attackers participate directly rather than through restricted black-box access.
5 Results
We empirically validate our approach as follows. First, we verify that inference-time morphing preserves model behavior (Section [5T). and quantify its practical overhead (Section 5.2). Next, we demonstrate that training under our framework matches unconstrained baselines for orthogonal transforms, consistent with theory (Section 5.3). Finally, we evaluate model-stitching and learning- based attacks (Section 5.4
7

---

GPT-2 (355M)
GPT-2 (1.56)
GPT-2 (124M)
20
Marph
Training Loss
K:055
Training Loss
CH0.01
Training Loss
Cw0.01
Orthoggnal
Ofthoggnal
Transforme
No Thansforms
Thandorms
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
6.0
0.5
1.0
2.0
Tokens (B)
Tokeens (B)
Tokens (B)
GPT-2 (124M)
GPT-2 Medium (355M)
GPT-2 (1.5B)
109
10r
102
Relative Loss Gap
Loss Gap
20-2
10
10-2
Relotive Loss
10
10
10
10
20
20
2012
Orthiggonal
Orthogora
10"h
0.0
0.5
1.5
1.5 2.0
10"3
0.0
0.5
1.0
2.0
10
0.0
0.5 1.0
1.5
2.0
Takens (B)
Tokens (0)
Tokens (B)
Figure 4: UPM training, Training on 1.9B Wib tokens with Muon across three GPT-2 scales,
comparing our UPM framework (with various transforms, morphing occurs every 0.5M tokens) to
no-transform baseline. Top: training loss. Bottom: relative loss gap vs. the baseline. As per the analysis in Section 3.3 orthogonal transforms are behaviorally indistinguishable from the baseline (overlapping curves, constant and near-zero relative difference). A very small e (e = 0.01) also tracks the baseline, e 0.1 drifts slowly and 0.5 drifts slowly for at least 100 morphing steps before it becomes unstable and fails.
5.1 Functional Equivalence
While UPMs preserve the model's end-to-end function mathematically, during inference the precision error builds as discussed in Section 3.4 To quantify this, we start with two pretrained open weight LLMs, Qwen 2.5-0.5B 45 and Llama 3.2-1B [3], and iteratively apply our morphing steps. After each morphing step, we measure deviation from the original model in two areas: output logits distribution using Censen-Shannon distance, and language modeling performance, using validation perplexity. Evaluation performed on the WikiText (v2-raw) validation set [37]. We test three floating-point precisions: FP32 (single), FP16 (half), and BF16. We also apply the high-precision accumulation workaround from Section 3.4 where transforms are folded into high-precision weights on disk before downcasting to the GPU.
As shown in Figure 3 without the workaround (dashed lines) logit drift and perplexity grow with the number of morphing steps: FP32 changes remain small even after 10k steps, but FP16 and BF16 drift substantially, eventually rendering the model unusable. With the workaround (solid lines), growth is negligible across all precisions, consistent with our floating-poin error analysis in Section
5.2 Inference Overhead
We consider the case of Llama 3.2 1B (which has dimension D = 2048), with a batch size B = 4, sequence length = 1024, and morphing every 30s. The latter implies that new participants need to wait minute before getting assigned a stage so that participants cannot leave and join in adjacent time steps (this time would required for authentication checks anyway).
Time: On an NVIDIA A100 GPU (FP32), morphing step requires approximately 0.05s, 95% of which is orthogonal matrix generation in FP64, yielding an amortized latency overhead of about 3%. This overhead can be further reduced by overlapping local computations with communication.
Communication overhead: Inference steps require sending activations of shape (B,S. D) while morphing steps require sending transforms of shape (D, D), resulting in an amortized bandwidth overhead of x 0.1%.
Memory overhead: Each layer stores four additional D matrices, increasing storage by roughly 20% per layer, or 10% extra GPU memory overall (around 1GB for typical inference workloads).
8

---

GPT-2
GPT-2 Med
GPT-2 XL
Original
3.28
3.26
3.16
Orthogonal
3.27
3.26
3.17
0.01
3.28
3.26
3.16
= 0.1
3.66
3.74
4.48
= 0.5
5.09
5.01
5.13
Table 1: UPM Training. Final validation loss for the experiment in Figure demonstrating the same
trend: orthogonal transforms have identical behaviour to the baseline, and drift increase with e.
5.3 Training
As explained in Section 3.3 we use the Muon optimizer for our training experiments. Muon is notable for having powered numerous speedrun records on CIFAR-I0 and NanoGPT [8]] To demonstrate
that UPMs support effective training, we build on a NanoGPT speedrun record [48] whose setup
most closely mirrors modern GPT architectures: GPT-2 model (124 M parameters) equipped
with RMS-NoMN RoPE, and ReLU² activations. The goal of the NanoGPT speedrun is to reach a validation loss of 3.28 Web in as few tokens as possible. The particular record we use achieves this target in just 1.9 billion tokens. We also demonstrate this with larger GPT-2 sizes.
We show training loss curves for both the original and with UPMs in Figure experimenting with Consistent with the theory in
different transforms, and report the final validation loss in Table [ Section 3.3 orthogonal transforms yield identical loss curves to the baseline, while non-orthogonal transforms (s > 0) introduce gradual drift that accumulates with ε. Applying a transform every 100 steps adds only 1.6% time overhead and <1 % memory overhead. These overheads are far smaller than at inference since the backward pass dominates both time and memory costs.
5.4 Attacks
In Section 3.1
explained that weights accessed across stages at different transform time steps inconsistent, and that the bridge matrix can only be determined if the time steps are adjacent (which prevent by adding waiting window rejoining). We now examine strategies for recovering consistent weights from inconsistent snapshots, and show they are impractical.
5.4.1 Model stitching attacks
esides transforms and weights, participants can observe intermediate activations X,. Suppose an attacker obtains stage i and i + 1 at time f and t' > t + 1, they have (tt,,, T.(,), X,(t) and Tt',", X, (L"), (t'). To make the weights compatible, they need the bridge matrix T (: However, if the same input was provided to the network in both time steps, then (t) = (t),T, so I can expressed as a solution to a matrix system (see Section G for a detailed explanation). Furthermore, if (t) is full rank, then / can be solved for by inverting (t).). and if not, T might be able to be determined using least squares. Such attack would reduce the excludability ratio of the model close to zero. Although the compute requirement of reaching each stage the protocol may be costly, it is extremely small compared to training from scratch.
However, requiring identical inputs across the time steps the weights were collected at is impractical and easy to defend against. Since the attack hinges on there being equal activations X, at both times steps. Section G both explains how an attacker could try to force this, and why those attacks still remain impractical and easily mitigated.
5.4.2 Learning based attacks
Beyond attacks that directly try to align incompatible weights that have been stitched together, adversaries might use cerning-based attacks, fine-tuning the stitched weights to reconstruct the original model. If fine-tuning requires little compute, the excludability ratio approaches zero, undermining the model's economic security. We evaluate such attacks on pretrained Llama 3.2- model using the Feb dataset [2). We use both the base Llama model trained by Meta (on wide variety of data) and a model we trained from scratch on Web. We then apply transforms to these model (treating each transformer layer stage) and stitch together weights from different

---

Stitched (Liama)
(FineWeb)
Tbkens
100
A 2.22%%%
Required
80
435.9%
Tokens
Morph Type Orthegonw
1.0
60
1% of Scratch
emb A1
0.0
40
r-0.5
r=0.1
Average
0.6
Scratch
0.4
FineWeeb
0.2
25
Perplexity
Perplexity
Tokens required, as a percentage of training from scratch to perplexity of 20, to reach various perplexities on Fine Web with a stitched Llama 3.2 IB model. We evaluate this with two sets of pretrained model weights that are morphed and then stitched: the base Llama model weights released by Meta (which is trained on wide variety of data) and our model weights that we train from scratch on Web. Transforms are applied such that the boundary between transformer layers are inconsistent. In all cases, finetuning the stitched model to perplexity of 20 required at least 60% of the compute required to train the model from scratch to the same perplexity. Right: Average extra tokens in billions to reach a desired perplexity as increases.
Figure 5: Learning-based Attacks. Left::
transforms. From this, we measure the compute required to reach various perplexity thresholds when finetuning (using FineWeb) compared to training from scratch. At the start of training we observe the perplexity of the stitched models are >60 000, comparable to randomly initialized model. As shown in F finetuning a stitched model that used orthogonal transforms still required roughly 60% of the compute of training from scratch. Small transforms yield similar costs and larger values slightly increase these costs, which we also show in terms of tokens.
Thus, UPMs are able to not only prevent direct weight set extraction, but also prevent information leakage that can allow an equivalent model to be produced with reasonable compute. Even when all weights are collected (though all are incompatible), the excludability ratio does not fall below 0.6. Such compute not within the means of even medium sized attack coalitions.
6 Conclusion
We introduce Unextractable Protocol Models (UPMs), which apply random invertible transforms at each pipeline boundary prevent piecewise Sybil attacks. In experiments with Qwen 2.5-0.5B and Llama 3.2-IB, during inference UPMs incur negligible logit and perplexity drift after 10 000 morphs, and add only 0.1% bandwidth and 3% latency overhead (amortized). We also demonstrated that training dynamics are unaffected for orthogonal transforms with the Muon optimizer, consistent with theory. Finally, we also show that stitching and learning-basee attacks are computationally expensive and impractical. By binding model value to the protocol rather than static weights, UPMs enable compute providers to jointly serve and monetize large models without risking weight leakage. Our preliminary training results suggest path toward fully trustless, unextractable training, positioning UPMs as a key technical pillar for open, decentralized AI.
Limitations Our security relies on standard cryptographic assumptions, notably that most par- ticipants are honest; thus, security weakens if a large portion is dishonest (see Section C.. Our experimental evaluation is limited single-machine simulations, excluding real-world decentralized implementations and side-channel attacks (e.g., timing, cache). Consequently, actual latency and bandwidth overhead depend on real deployment factors like batch sizes and network conditions.
Broader Impact This work develops Unextractable Protocol Models (UPMs) allow for economic sustainability and open participation in large-scale decentralized AI However, UPMs prevent the ability to inspect or disable model weights. This prevents safety evaluations and content-moode mooeerttioor controls, such as determining if activations have unwanted correlations, and could empower actors who wish to host unregulated or harmful AI services without fear of takedown or forensic analysis. To mitigate these, we suggest ensuring that diverse stakeholder participants jointly monitor model behavior and enforce usage policies, and continuously have transparent metrics evaluated and logged, enabling external safety validation.
10

---

References
[1] Nur Ahmed and Muntasir Wahed. The de-democratization of ai: Deep learning and the compute divide in artificial intelligence research. arXiv preprint 2XXi:20010.15888 2020.
[2] Thalaiyasingam Ajanthan, Sameera Ramasinghe, Yan Zuo, Gil Avraham, and Alexander Long. Nesterov method for asynchronous pipeline parallel optimization. ICML, 2025.
[3] Kenneth Arrow. Economic welfare and the allocation of resources for invention. In Richard Nelson, editor, The Rate and Direction of Inventive Activity: Economic and Social Factors, pages 609-626. Princeton University Press, Princeton, NJ, 1962. doi: 10.1515/978| 10.15155/978840077777622222 400879762-024, URI wttph:tps///wwtt nber..berr htt.o/r..wr.aattt//e/e/////aaata c2144/22144.44110 pdf
[4] Gil Avraham, Yan Zuo, Violetta Shevchenko, Hadi Mohaghegh Dolatabadi, Thalaiyasingam Ajanthan, Sameera Ramasinghe, Chamin Hewa Koneputugodage, and Alexander Long. NodeO: Model parallel training over the internet with protocol models, 2025. URL htLhhttp://gghmmm
com/Plute sResearch/noded
[5] Michael Blot, David Picard, Matthieu Cord, and Nicolas Thome. Gossip training for deep learning. arXiv preprint arXiv:1611.0990 2016.
[6] Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Max Ryabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, and Colin Raffel. Petals: Collaborative inference and fine-tuning of large models. arXiv preprint arXiv:220900118.. 2022.
[7] Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah. Randomized gossip algorithms. IEEE transactions on information theory, (266):500-2553 2006.
[8] Ben Cottier, Robi Rahman, Loredana Fattorini, Nestor Maslej, and David Owen. The rising costs of training frontier ai models, 2024.
[9] Michael Diskin, Alexey Bukhtiyarov, Max Ryabinin, Lucile Saulnier, Anton Sinitsin, Dmitry Popov, Dmitry Pyrkin, Maxim Kashirin, Alexander Borzunov, Albert Villanova del Moral, et al. Distributed deep learning in open collaborations. Advances in Neural Information Processing Systems, 34:7879-7888 2021.
[10] Michael Diskin, Alexey Bukhtiyarov, Max Ryabinin, Lucile Saulnier, Anton Sinitsin, Dmitry Popov, Dmitry Pyrkin, Maxim Kashirin, Alexander Borzunov, Albert Villanova del Moral, et al. Distributed deep learning in open collaborations. Advances in Neural Information Processing Systems, 34:7879-7897, 2021.
[11] Arthur Douillard, Qixuan Feng, Andrei A. Rusu, Rachita Chhaparia, Yani Donchev, Adhiguna Kuncoro, Mare Aurelio Ranzato, Arthur Szlam, and Jiajun Shen. Diloco: Distributed low- communication training of language models, 2024. URL
08105
[12] Craig Gentry. Fully homomorphic encryption using ideal lattices. In Proceedings of the forty-first annual ACM symposium on Theory of computing, pages 169-178, 2009.
[13] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al. The Ilama 3 herd of models, 2024. URL /tttps://arxiv org/ abs 2407 a22/ 21783
[14] Aaron Harlap, Deepak Narayanan, Amar Phanishayee, Vivek Seshadri, Nikhil Devanur, Greg Ganger, and Phil Gibbons. Pipedream: Fast and efficient pipeline parallel dnn training, arXiv preprint arXiv: 1806. 03377, 2018.
[15] Nicholas J. Higham. Accuracy and Stability of Numerical Algorithms. Society for Industrial and Applied Mathematics, second edition, 2002. doi: 10.1 10.1137/1..7880988711 97808997180277.
[16] R.A. Horn and C.R. Johnson. Matrix Analysis. Cambridge University Press, 2012. ISBN 9781 978118 139788885.

---

[17] Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, Hy- oukJoong Lee, Jiquan Ngiam, Quoc V Le. Yonghui Wu, al. Gpipe: Efficient training of giant neural networks using pipeline parallelism. Advances in neural information processing systems, 32, 2019.
[18] Keller Jordan, Jeremy Bernstein, Brendan Rappazzo, @femnbearbskkysociill, Boza Vlado, You Jiacheng, Franz Cesista, Braden Koszarsky, and @ @Grad62304977. modded-nanogpt: Speedrunning the nanogpt baseline, 2024. URL
modded-nanogpt.
[19] Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein, Muon: An optimizer for hidden layers in neural networks, 2024. URL
https://kellerjordde
[20] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Ar- jun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, al. Advances and open problems in federated learning. Foundations and Trends@ in Machine Learning, 14(1-2): 210, 2021.
[21] Jiawen Kang, Zehui Xiong, Dusit Niyato, Han Yu, Ying-Chang Liang, and Dong In Kim. Incentive design for efficient federated learning in mobile networks: A contract theory approach. In 2019 IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS), pages 1-5. IEEE, 2019.
[22] Amy Kapezynski and Talha Syed. The continuum of excludability and the limits of patents. The Yale Law Journal, 12207):1900-163, 2013. http://ppppp////.aalllaaaa/aaaaa jurnall org/ essay/the- cottinucntti of www.:eaoooooeecuuceaaaa...eeeeee hatbb--oootoouuooooooccoooaaaaaa
-atentss
[23] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint orXiv:2001.8866/ 2020.
[24] Andrej Karpathy and contributors. nanogpt: The simplest, fastest repository for train- http:s////gittub..ooo/araapppppe
ing/finetuning medium-sized gpts., 2023.
[25] Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, and Sebastian Stich. A unified theory of decentralized sgd with changing topology and local updates. In International Conference on Machine Learning, pages 5381-5393. PMLR, 2020.
[26] Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, and Sebastian Stich. A unified theory decentralized sgd with changing topology and local updates. In International Conference on Machine Learning, pages 5381-5393. PMLR, 2020.
[27] Kalpesh Krishna, Gaurav Singh Tomar, Ankur P Parikh, Nicolas Papernot, and Mohit Iyyer. Thieves on sesame street! model extraction of bert-based apis. In International Conference on Learning Representationss, 2020.
[28]
Seaming@home-Team. Hivemind: a Library for Decentralized Deep Learning. https Crrithub.com/eearrii at- home/nivemind] 2020.
[29] James Lee, Sarah Chen, and David Wang. Stealing part of a production language model. arXiv preprint frXiv:2401.XXXXx 2024.
[30] Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, and Bingsheng He. survey on federated learning systems: Vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering, 35(4):3347-3366
2021.
[31] Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. Advances in neural information processing systems, 30, 2017.
[32] Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. Advances in neural information processing systems, 30, 2017.
12

---

[33] Xiangru Lian, Wei Zhang, Ce Zhang, and Ji Liu. Asynchronous decentralized parallel stochastic gradient descent. In International Conference on Machine Learning, pages -0522 PMLR, 2018.
[34] Wanchao Liang, Tianyu Liu, Less Wright, Will Constable, Andrew Gu, Chien-Chin Huang, Iris Zhang, Wei Feng, Howard Huang, Junjie Wang, Sanket Purandare, Gokul Nadathur, and Stratos Idreos. Torchtitan: One-stop pytorch native solution for production ready LLM pretraining. In The Thirteenth International Conference on Learning Representations, 2025.
URL https://opeereviee: http:///wwnrreeeeeeeeeoooraaaaas
[35] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprini arXiv:2412. 19437, 2024.
[36] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficcient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1282. PMLR, 2017.
[37] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture 2016.
models. arXiv preprint
[38] Francesco Mezzadri. How to generate random matrices from the classical compact groups, 2007. httpa///////////aa.oooo////aaata
[39] Elinor Ostrom. Understanding Institutional Diversity. Princeton University Press, Princeton,
NJ, 2005. ISBN 978-0-691-12238-4
[40] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc., 2019.
[41] David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104. 10350, 2021.
[42] Guilherme Penedo, Hynek Kydlícek, Loubna Ben Allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data scale. ArXiv, fos/2406.115571 2024.
[43] Zhe Peng, Jianliang Xu, Xiaowen Chu, Shang Gao, Yuan Yao, Rong Gu, and Yuzhe Tang. Vfchain: Enabling verifiable and auditable federated learning via blockchain systems. IEEE Trunsuctions on Network Science and Engineering, 9()): 2021.
[44] Gary P. Pisano and David J. Teece. How to capture value from innovation: Shaping intellectual property and industry architecture. California Management Review, 50(1):278-296, 2007. doi: 10.2307/41166428. URL https://:://do orgg 10. 2007 01100/4116648
[45] Owen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang. Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang. Yu Wan. Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https:tt:: atxpv. org/ abs/ 221.
[46] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5301-
5310. PMLR, 09-15 Jun 2019.
13

---

[47] Sameera Ramasinghe, Thalaiyasingam Ajanthan, Gil Avraham, Yan Zuo, and Alexander Long. Protocol models: Scaling decentralized training with communication--fiieee model parallelism. 2025. URL http::ttt///aat org/ werggabs//a0 waw./ 2000./0060000
2024. [48] Tyler Romero. Nanogpt speedrun worklog.
http::/www..yyleerreerro.oc
lanoggt-speedrun-wokkll
[49] Max Ryabinin and Anton Gusev. Towards crowdsourced training of large neural networks using decentralized mixture-of-experts Advances in Neural Information Processing Systems, 33: 3659-3672, 2020.
[50] Max Ryabinin, Eduard Gorbunov, Vsevolod Plokhotnyuk, and Gennady Pekhimenko. Moshpit sgd: Communication-effiicn decentralized training on heterogencous unreliable devices. Advances in Neural Information Processing Systems, 34:18195-18211 2021.
[51] Max Ryabinin, Tim Dettmers, Michael Diskin, and Alexander Borzunov. lelism: Training large models can be surprisingly communicationn-eficccent arXiv preprint arXiv:2301. 11913, 2023.
Swarm paral-
[52] Paul A. Samuelson. The pure theory of public expenditure. The Review of Economics and Statistics, 36(4):387-389 1954. doi: 10.2307/1925895. URLphttps h/wp::///wwww jstor org/ stable/1925895
[53] Adi Shamir. How to share a secret. Communications the ACM, 22(11)6122611, 1979.
the
[54] Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-Im: Training multi-billion parameter language models using model parallelism. arXiv preprint wwXhv:190900805: 2019.
[55] Zhenheng Tang, Shaohuai Shi, and Xiaowen Chu. Communication-eficienn decentralized learning with sparsification and adaptive peer selection. In 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS), pages 1207-1208. IEEE, 2020.
[56] David J. Teece. Profiting from technological innovation: Implications for integration, collaboration, licensing and public policy. Research Policy, 15(6):285-305, 1986. doi: 00.001600088-73338819900777
[57] Neil C Thompson, Kristjan Greenewald, Keeheon Lee, and Gabriel F Manso. Deep learning's diminishing returns: The cost of improvement becoming unsustainable. leee Spectrum, 58 (10):50-55, 2021.
[58] torchtune maintainers and contributors. torchtune: Pytorch's finetuning library, April 2024.
https////iittbbb.o http://wwww...oooooooo//o/cotott
[59] Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. In USENIX Security Symposium, pages 601-618, 2016.
[60] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[61] Thijs Vogels, Sai Praneeth Karimireddy, and Martin Jaggi. Powersgd: Practical low-rank gradient compression for distributed optimization. Advances in Neurul Information Processing Systems, 32, 2019.
[62] Eric Wallace, Mitchell Mitchell, and Carla Finn. Imitation attacks and defenses for black-box machine translation systems. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5531-5544 2020.
[63] Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yan Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. ArXiv, bbs//901.00623.. 2019.
14

---

[64] Zhi- Zhi John Xu. Yaoyu Zhang, and Yanyang Xiao. Training behavior of deep neural network in frequency domain. In Neural Information Processing: 26th International Conference, ICONIP 2019, Sydney, NSW, Australia, December 12-15, 2019, Proceedings, Part page 264-274. Opringer-Verlag Verlag, 2019. doi: 10.1007/978-33 1-19978883--0000 --0-77778---00--770--------
[65] Sixing Yu, J Pablo Muñoz, and Ali Jannesari. Federated foundation models: Privacy-preserving and collaborative learning for large models. arXiv preprint arXiv:2305.11414, 2023.
[66] Binhang Yuan, Yongjun He, Jared Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Re, and Ce Zhang. Decentralized training of foundation models in heterogeneous environments. Advances in Neural Information Processing Systems, 35:25464-25477, 2022.
[67] Biao Zhang and Rico Sennrich. Root Mean Square Layer Normalization. In Advances in Neural Information Processing Systems 32, Vancouver, Canada, 2019. URL https::/opppeeeeeeee het/reeferrecee//pdf///4dSS..f.e
[68] Tongtian Zhu, Tianyu Zhang, Mingze Wang. Zhanpeng Zhou, and Can Wang. single merging suffices: Recovering server-based learning performance in decentralized learning, 2025. URL
attpe://arrxi..org/as/////
06542
[69] Weiming Zhuang, Chen Chen, and Lingjuan Lyu. When foundation model meets federated learning: Motivations, challenges, and future directions. arXiv preprint arXiv:2306. /5546, 2023.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer:
[Yes]
Justification: We validate our theroetical claims in Section 3.1 and our experimental claims in Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See Section 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
We reason about the main theoretical result in Section 3.1 Any justifications
only briefly given in the main paper (e.g. Sections 3.3 and 5.4) are properly justified in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We give full details on how implement our framework and on the transforms we used. All of our experiments are done using openly available models. We provide additional details (e.g. hyper-parameters) in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
17

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification: Our framework is simple to implement, with all details needed to implement given. We use openly available neural network frameworks (discussed in the appendix) and use openly available data (explained in the main paper).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.co tttpcc//pbbbc muublc//guidee wttggdedee/ooeeuubmmmmmeeee www.///oddesissiiiooao wammisssioooiccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Most of these are given in the main paper, hyperparameterss given in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Experiments are too computationally expensive to report error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We give few of these in the main paper (memory and time overheads), and give the rest (compute resource details) the appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http://ppss//paapp..cc///////sas
Answer: [Yes]
Justification:
We do not breach any item.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section
6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See Section 5
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwwppppappphhaoocc thcode. com com/datasets
20

---

has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
21

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: For word-level editing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
22

---

A The economics of decentralized models
While the decentralization of large models allows the huge computation cost of training and serving inference at scale to be spread across a large number of participants, by nature such setting is trustless. We can assume an honest majority of participants, but we must also assume that there are bad actors within the protocol. As result, any information that participants receive unprotected access to, be it code. data, or weights, can be utilized outside the protocol. We now explain what must be protected in order for the economics of decentralized protocols for large models to be rational, and why unextractability is sufficient to guarantee this.
A.1 Appropriability and Excludability
The key issue here is called the appropriabilityy problem in economics: the difficulty for producers to appropriate (capture) value from their product 56 44). Not only do they need to recoup the cost of production, but there needs to be surplus value as incentive for the production. In the context of decentralized protocol models, the producers are the participants of the protocol and the product is the ability to utilize the model via the protocol (query the trained model). The value of this model utility (the product) is largely governed by the capability of the model.
Increasing the appropriability of the product is often discussed in terms of the related economic notion of excludability, the degree to which non-paying consumers can be denied access to the product
52 33 [2]] By only allowing access to query the model via the protocol, the protocol can then
monetize serving inference for general consumers and distribute returns to participants who supplied compute. This gives participants the incentive to supply compute in the first place. However, if one of the participants can gain access to the full model, then there is the ability to query the model outside the protocol and the excludability of the protocol disappears.
This is the motivation for unextractable protocol models, ensuring that no participant can recreate the full weight set while participating in the protocol, and thus guaranteeing the excludability of the protocol.
A.2 Partial Excludability
Furthermore, we can consider partial excludability: if an attacker can produce an equivalent model using whatever knowledge they gained from participating in the protocol and additionally some amount of compute that is substantial (which includes the compute required to participate in the protocol), then the excludability is fractional and depends on how large that required compute cost is relative to the overall compute power needed to train the model from scratch.
Let formalize partial excludability for our situation. Assume that a performance measure P(M) has been defined for any model M along with a tolerance value e, for example the validation perplexity of the model the dataset it is being trained with tolerance of 0.5 perplexity, or perhaps the validation perplexity on an independent dataset. Let us also consider a compute measure C(M) for obtaining model M, for example the amount of flops, GPU hours, cost of compute.
Consider the case of an original model trained from scratch in the protocol M with performance measure P (M), and a model M' that an attacker has derived from participating the protocol with performance measure ((M'), where IP(M) - P(M') S ε. Then to quantify how excludable M and assuming this attack is the best possible attack, define its excludability ratio the ratio between the compute C(M') required to derive M', and the compute C(M) required to train M from scratch
E, C(M')
C(M)
B Possible Extraction Scenarios and their Excludability Ratio
Let us consider possible extraction scenarios and their excludability ratio, noting that we want to ensure that all feasible attacks have high excludability ratio. In fact, an excludability ratio of more
23

---

than 0.1 is very likely to still be out of the reach of individual attackers / attack coalitions, as this is still an enormous amount of compute.
Since we can consider the case when the code and data are completely open (as it is economically feasible with framework), with enough compute anyone can train their own model. However, this requires the same amount of compute as the decentralized protocol, C, and so the excludability ratio one. On the other hand, if an attacker can somehow compromise the security of the protocol and directly gain access to all trained weights, then their required compute is zero and the excludability ratio is also zero. This extremely unlikely as we assume that standard security measures have been put in place.
Now let consider the situation where the attacker participates in the protocol. If the attacker manages to collect weights from each stage over multiple time steps and there are no unextractabilityy measures put in place, then the required compute is whatever was needed to participate in the protocol until the attacker collected all weights. While this compute might be substantial for an individual (as participating in the protocol requires compute), it is dwarfed by the enormous amount of compute needed for training the large model, so the excludability ratio is essentially zero.
On the other hand, let us say that there are still no unextractabiliity measures in place, and the attacker manages to gain access to a large subset of the entire weight set. By keeping those weights fixed, and training all the other weights in the model, the attacker may be able to gain access to model with equivalent capability. If the compute required for training is some fraction a of C, then the excludability ratio is a. Similarly, when there are unextractability measures in place, the attacker might be able to collect a full set of incompatible weights, and finetune from these weights in order to align them and thus gain compatible model with some fraction of C. In both these cases, if is small enough that an adversarial entity has access to aC compute but not C compute (so cannot train the model themselves but could carry out one of these attacks), then the appropriability of the model is severely reduced.
Now let us consider attacks with our unextractability framework in place. It is still possible for an attacker to participate in the protocol and manage to collect weights from each stage over multiple time steps well the transforms needed to make the weights compatible. However, shown in Section C the probability of this happening can be made vanishingly small, especially for the case of transformers.
Another possible attack is to use intermediate results gained while in the protocol to solve for the required bridge matrices. We discuss such attacks in Sections 5.4.1 and G including easy ways to prevent them.
Finally, an attacker might use whatever set of weights they obtained while participating in the protocol, which will be incompatible with each other, to train an equivalent model with less compute than training from scratch. This is the learning based attack discussed in Section 5.4.2] We demonstrated that even with access full set of (incompatible) weights, the excludability ratio remains quite high.
C Decentralized Model Pipeline Setup and Extraction Probabilities
Let us first summarize our setting from Section
We consider the situation of pipeline with S stages and R replicas of each stage, and each of the RSS slots (individual replicas of a stage) require a compute node.
The combined compute pool from all participants is C' > RS nodes, nodes are allocated slots at random (hiding the stage identity)
The protocol might implement stricter measures like the nodes belonging single participant
can only be allocated to slots in a single stage
We assume that the total nodes attacker controls is A < RS < C. This is reasonable since are considering à large-scale model that requires collaboration from multiple participants, meaning no one participant has the computational capability to hold all the nodes in the protocol. Thus the fraction that the attacker controls is A << 1.
For further clarity, we assume the following standard requirements of the protocol:
24

---

In order to participate. participant has to prove they have at least one compute node (i.e., they have the GPU compute at least run single stage)
Secure, authenticated communication is established between nodes.
Decentralized work verification ensures that all participants contribute valid activations.
Finally, in order to compute the probability of an attacker being able to extract the entire model using a piecewise Sybil attack in various scenarios, let us assume that RS is small compared to C. This not unreasonable since the monetary worth of large models attracts participants (c.f. blockchain). The benefit of this is that choosing compute to fill RS slots from nodes is well approximated by sampling with replacement.
Thus the probability of a particular slot being controlled by an attacker is then p. the expected number of attacker-occupied slots per timestep is pftS, and the expected number across ' timesteps is pRST. Furthermore the probability of the attacker getting access to a particular stage i (in any replica) in a time step is
Pr = 1 - p)
We now calculate the probability of an attacker being successful under different scenarios. In many cases modification of the binomial approximation: 1 (1 =²) or for |a]] < 1, laz| < 1.
Note that this is an over-approximation for x > 0 (which the region we use).
Default setup (no unextractability measures). In the default case, for the attacker to be successful the attacker must have access to all S stages, each of which can be from any replica and from any time step. Let us assume that the attacker is in the protocol for T time steps, then they form complete weight set by getting access to the stages over the T time steps.
The probability of ever capturing stage within 7 time steps is P = 1 - (1 (1- = 11 11-(1 (p)pp) R).T.
This means that the probability of getting access to all S stages over the T time steps is
[1 11- p) | (pRT);.
Note that p. R. S are fixed with p being very small (10-5 - 10 and R and S being small integers (R is probably 3-100, S is around 10-100). However T can be very large (millions or billions of steps) so the probability is almost certain for the attacker (in practice there would be some mechanisms to prevent easy switching of stages between timesteps, slightly decreasing the chance).
C.1 With Unextractability
Now we consider the case when we make the stages time- varying so that different stages are not compatible across time steps. Note however, that the bridge matrix between neighboring stages is available for subsequent time steps (see Section
Thus we consider two cases
The weights in any stage i can be partitioned into two sets, 0,,1 and 0,,2, where the transforms between stage i and i are only applied to weights in fi,1, and the transforms between stage i and + are only applied to weights in 0;22- We call this partial incompatibility.
The weights in any stage cannot be partitioned in the above manner, thus there is at least one weight the stage that depends on both the transform between stage i- - and and also depends on the transform between stage i and i 1. We call this full incompatibility.
Full Incompatibility. Let us first consider full incompatibility. Then if an attacker gets access to stage in time step t, in order for it to be compatible the weights of stage - and stage + 1 need to be also gained at time t. Thus between any set of three consecutive stages, the weight of all three stages have to be at the same transform time step. Since participants only have the bridge matrix (to convert weights from one time-step to another) between subsequent time steps, this can be done stage and + are one time step away from stage i. However since this needs to hold between any set of three consecutive stages, this can only work if all stages have a way to convert their weights to single time step.
Thus the attacker needs all stages within three consecutive time steps, as then any stage's weights can be converted to the middle time step. Then the probability of specific stage being covered
25

---

within a specific three time steps is P1.3 (1 p) and the probability of all S stages covered
in the three time steps is pi = (1-18 (1 p) p) In T'' time steps there are T - 2 sets of three consecutive time steps, so the probability of successful attack over T time stages
P p = (1 (1- (1-(11 p) (1- = (T - 2)
(9)
Thus as long as 3pft is reasonably small, e.g. less than 0.2, the probability of success small even for very large T.
Partial Incompatibility. Now let us consider partial incompatiiility. Then each pair of 0,,2, 0 between stages and i + must at the same time-step. though any other pair can be at different time steps. Furthermore, since participants have bridge matrices between subsequent time steps, these can be transformed to neighboring time steps only.
Thus the attacker needs each pair of stages and + within three consecutive time steps, as then the boundary weights of those two stages can be converted the middle time step, and all such pairing can be stitched together. Thus the probability of getting two specific stages within three time steps is p getting this within the T - opportunities is
1 - (1 (T
(10)
and finally doing this for all 5 1 pairs of stage boundaries is
(1 - (1 - p²,)) - = - ()
(11)
Note that T is now scaled by S, unlike in the full incompatibility case. While this is much less probable than the default setup, with reasonable p there is a high probability for large T. However, unlike the default setup, the requirements here are much easier to defend against, as shown in the next section.
Achieving Full Incompatibility. An casy way to achieve full incompatibility is to have skip connections in the stage. Then the stage is of the form (XU:) X QQ and transforms get applied (XT:-- + 1;T- thus making (i( (t) 1)T,(t) be dependent both neighbors.
C.2 Mechanisms
A key issue is therefore to reduce the chance that an attacker can control two neighboring stages within three consecutive time steps. We can therefore take some effective measure
Once compute has been assigned to a stage, have the compute keep that stage (note that there are other replicas for redundancy and for checking malicious activity)
To stop attacker compute from switching (in the hopes of using the same compute to get stage they need) we require newly joined nodes to wait two time steps before participating.
Have a preference for assigning nodes that have been waiting for longer join the pipeline.
Thus, an attacker can no longer switch compute within the required three consecutive time steps. Instead, assuming they have A > 2 nodes, they need to switch all A nodes at the same time and hope that at least two of the nodes get allocated to the required two neighboring stages within three time steps. However, assuming node dropout is not very often, this is highly unlikely: if there is a lot of nodes waiting most likely other nodes will get assigned, and if there is not lot nodes waiting then the attacker nodes (even when masquerading as new identities) are likely be assigned back their original slots.
Finally, note that inference is the hardest case, as weights never go "stale", whereas during the early part of training, weights will drastically change over time and thus go stale. Thus, there is an extra time requirement on the attacker during (early stage) training.
26

---

D Applying UPMs to different components and architectures
D.I MLPs
Let us consider an n-layer MLP In o 9n-1 o ...91 where gi(X) (XU;) Vi, the U; are weight matrices, V; I, and a is an element-wise activation function. Let the output after the 5th layer be X,, i.e X, (g: o gi-1 o ...91) ) (Xo).
When we apply transforms, we have that
Ui(t) (t)
TT Vi(t) Vi(t (T))))))
(12)
(13)
X,(1) = Xt - 1)T((() - -.Z(())....(())
(14)
Note that we only have partial incompatibility (no matrices in stage depend on both boundaries).
Using weights at different time steps Let us assume an attacker tries to steal consistent set of weights by having different blocks different time steps. In particular, us assume they have fę at time & and fi+1 at time t2 > f Then they have
(1(+2) = T= - 1) = (4)
(15)
(1((1) = Vi(11 - ()7;(t)) = (T((1)..T:((4)
(16)
X,(t1) Xi(ti - (t)) = (I).(t)..
(17)
U1+1 (t2) (te)(0))4411tz -1) T, (t))...T
(18)
V/+1(12 = Vi+1 11) = (T)
(19)
X (12) Xi+1 (122+1((2) (I)
(20)
as well as (ttt), (T(t), Ti+t),
In order to get consistent weights, they need to have (i()), (14 for the same time step t. Thus they need one the following configurations
Vi(0) = I, (0) = U1+1 = Vi(t))(UHz1(t))
Vi(tt), = - (t2)
VV((t) = Vi(tt)f, (t2)
where π Ti4, + 1)...T((I2 ()T((t()) is the bridge matrix, and anything the attacker has is bolded. Note that the only thing missing in three configurations is exactly T, which the attacker only knows 122 = t + Otherwise we need to factorize (12) into (t,) -1 where only V=(4))-1 is known and we know that (1(4)) and Í are square and invertible. However there are infinitely many possibilities, as TV = (TA) v- (VA VA--v-lu = Tv for any invertible matrix A.
D.2 RMSNorm
As explained in the main paper, we want RMSNorn applied layers follow
(RMSNorm
(XT) UATINNomMXXTU:))
= (RMMSNor =
(21)
and in order to do that we (1) introduce a transform accumulation matrix into the normalization: norm(X) + norm(XQ(t)) where Q(0) is initialized to an orthogonal matrix, and (2) move the feature scaling weights diag(w) into the next layer weights (1) diag(w)U; (which for training equivalent to removing them). Then our layer is now the form
f: (X,-1) = (
(22)
27

---

and so after a morphing step Qi + 1 Qi we have the desired functional equivalence
fi (X) = & norm X, (X.,
(23)
2 (norm (x)
(24)
+ (RMSNoo((X..)TT...
(25)
where the RMSNorm in the last line is the original version with diag(w) set to 7 (since we have already moved diag(w) w) into (t()).
D.3 Transformer
Let us consider a modern transformer layer with RMSNorm prenormalization. After applying the above change for RMSNorm layers, we have that
RMSNorm (X) norm (XQ)
(26)
where Q is fixed (random) orthogonal matrix and the weights have been subsumed into the next layer. Thus the transformer layer is given by
f((X;) = (MLP o (X;)
(27)
ATT(X) = SA(RMSNOrm (X)) + X
(28)
MLP(X) (RMSNNNM) (X) X
(29)
where
SA(X) = Au(XUx,, XU,)
(30)
RMSNorm, (X) norm (XQn1)X
(31)
RMSNorma (X) = norm (XQn2)X.
(32)
Applying transforms (see Figure (C)) we get
ATT(X) SA(RMSNomm, (X))
(33)
MLP(X) = a (RMSNNom_(() (Q2(())
(34)
SA(X) (U,,
(35)
RMSNorm (X) = norm norm(XQ.att)) (t)
(36)
RMSNorma (X) norm (()
(37)
where if the incoming transform is T and the outgoing transform is Tl:
(t) = T - 1)
(38)
(x(t) T (t)Un( -0) 1)
(39)
U2(1) = (1)
(40)
U,(t) = TT -1)
(41)
Vi(() = Vo(t - (1)
(42)
Qi(t) = (t 1)tine(t) -
(43)
Q2²(t) (t) - 1)
(44)
(1) (I) 1)
(45)
Vw2(t) = Vwz(t - (w)(t=1)Titt)
(46)
Q2(t) = (1)(()
(47)
Here we have added an intermediate transform Time, which is not redundant due to Q and Q2-
Note that we have full incompatibility due to the skip connections: Q1 depends on T and Tint and Q2 depends on Time and T; so there is no way to partition the matrices in way that one partition can only depend on T. without T and vice-versa for the other.
28

---

For SwiGLU MLPs, we have that
MLP = (SiLU(RMSNorm) (XX)m (RMSNorm)(X)Uua)) (7 Vw +
(48)
(49)
so we have the additional transformed weight
U (n) = Tind (1)Umal -
(50)
E
Transforms
We first analyze the key properties want in transforms, then describe the specific families use.
E.1 Transform Properties
There multiple factors that would like in order for the weights be sufficiently scrambled:
the scrambling is done sufficiently randomly with respect to an infinite class of transforms, so
is not possible brute force all possible transforms exploit there being a bias towards certain transforms within the class
the new weights are far distance away from the old weights (so the effect the network of single transform without its inverse is not small) the transforms introduce high frequencies, which makes finetuning transformed parameters difficult.
On the other hand, we want to control the transforms such that
large the result does not overflow or underflow.
the floating point error
turns out that we can link a lot of these properties the singular values of our transforms,
which we will now define notation for. For matrix M E Raxd we will denote its singular values by {M:(M) where the indexing orders them from largest to smallest. Some important properties are: the condition number is given by x(M) = aa(M))
ai(M) the determinant is given by
det(M) I (M), and the Frobenius norm is given by || 1
Floating point error The floating point error (relative forward error) of computing W' ....... by post-multiplying the transforms T; one at time is (see Section B.2 for proof)
||
W'
Jakw Tyaak
x(n"-1) x-1
1#1
(51)
||
k=1
where and the T: are x d, W has a condition number of sw and the T; have condition number du of k, and Ya = I-du where u is the unit roundoff (half machine epsilon). Thus if k & 1 then the error grows exponentially in the number of transforms, which severely limits the number of transforms that can be applied before the error is too large. Thus the singular values of the transforms should be as close to each other as possible.
Scale Control When applying morphing steps V + VT, TT the resulting size of the weights as measured by the Frobenius norm is governed by the spectra of T. The von-Neumann trace-inequality [16] gives
[VT <
Σ (In
ITU P ((a(T))(())))3)
(52)
Thus if the singular values of T are all large (small), then the Frobenius norm of V greatly increases (decreases) and Frobenius norm U greatly decreases (increases). On the other hand, if 7 has both large and small singular values, then the effect on U and V is unpredictable, and likely to make both
29

---

grow (note that just one singular value needs to blow up). However from the previous paragraph we want keep the condition number small, so we anyway keep the singular values of I close each other. Thus to control the scale of the weights, we either have the singular values close to one, or cycle between large and small singular values each time step so that the scaling cancels out on every second time step.
Introducing high frequencies. It has been observed that neural networks trained with SGD are biased towards low frequency functions, in particular the low frequencies of the target function are learned before the high frequencies [64 63 46). Thus we want to add transforms into weights such that just applying the transform without its inverse introduces high frequencies, meaning to undo the transform with gradient descent requires learning to cancel out the introduced high frequencies (a high frequency target). To inject high freeurreyenccy behavior into the network's 's input-output map, our transforms should amplify the directions weight-space that correspond to seghffreuueccy components of the learned function without amplifying other directions, i.e. have larger singular values for high-frequency directions and smaller ones for the low-frequency directions. With our random transforms, simple approach is to sample random orthonormal basis (singular vectors) and assign a spectrum of singular values that is skewed toward larger values, on average this will inject more high-frequency content than flat spectrum would.
Scrambling distance. We need to generate transforms from a large class of transforms randomly, and ideally uniform randomly that there is no bias towards specific transforms in that class. However most classes of transforms will contain transforms that do not do a good job at scrambling: e.g. the class of orthogonal matrices contains rotation matrices, and if T corresponds to the rotation matrix of a small angle then I[ - TWI|2 ||2 might be small. Furthermore, folding in multiple rotation matrices might make them cancel out. In order for attacker to not be able align weights from different time steps, we need to ensure that transforms always drastically change the weight matrix and never cycle back close to the original weights. Thus we need ensure that our class of transforms is distributed enough that low scrambling matrices have an extremely low probability.
E.2 Controlling Floating Point Error Drift
As discussed in Section 3.4 when folding in transforms a large number of times the floating point error builds. We now analyze this formally.
E.2.1 FP analysis
We use the standard error bound for matrix multiplication from Higham [15. Given two d x d matrices and B in precision P their error is bounded by
[|]((A)) - ABI|2 S [a||-A|2||B]]
(53)
where fl(-) is floating point multiplication in precision P. 7a = dup zo dup and up is the unit roundoff in precision P.
I-dup
Given some computation C' whose result in floating point multiplication in precision P is denoted ((P), we want to compute relative errors
[c(P)) - C
(54)
||
with respect to the condition number of the matrices.
Some important results are: ||AB||2 I ||||| B1|2. ||A||2 = m ||AB| 2 Omin aA) (B),
<(A) (A) <(AB) - x(A)x(B) (3) and ||AB]|2 2 a (A)
(an(A)
Single Product: The relative enror in terms of the condition number of the matrices for single matrix multiply is:
ABBl2 [[[(( ||AB||2
max (A)o (B)
< 7
(55)
mmink (A mA) (B)
= =akk (A))(A)
(56)
30

---

Chain of Products: Now let us consider doing chain of n 1 matrices, e.g., transforms T; applied an initial weight matrix W: C, = W.....Tn. Let us denote the floating point representation by (CP) fl(...WT))..II.).
Now the error in the kth multiplication is bounded by
[[ - Calla S -
(57)
We will bound the total error in the chain the of the errors in each multiplication times the norm of the remaining computation done to that multiplication result (first order approximation neglecting quadratic terms):
|| - 2
Σ [l (T) T..
(58)
7a
(59)
so the total relative error is approximately bounded by
||
Cnll2
CK- 1|2 TelTTal22//T
I 9d
(60)
|2,12
11Cr112
1C [|||||1111 T2
S 78
(61)
Omin (kk 1 kk) 1...
S 788
x(CR-1)x(Ib)
(62)
Σ k k=101
S Tah(( (W)
(TT)).
(63)
If each T. have the same condition number <(Ti) then this is geometric given by
[(C/P) C2
Vakow m(m"-1) myakk K ==1
x-1 K
* 1
(64)
1|C.112
E.2.2 Comparison for different precisions
Let us consider two d x weight matrices, W1 and W2, which are stored on GPU in low low-precision datatype LP, e.g. FP32 or FP16. Their matrix multiplication in precision LP is (.(L") (LP).
Note that when we generate orthogonal transform 7 (t), usually do this in high precision HP anyway (Haar orthogonal is very sensitive to precision), specifically FP64. Thus, since the weight matrices on the GPU are in low precision LP, we have three options
(A) Cast down transforms to LP and then fold them into the weights
(B) Whenever a transform needs be folded in, cast the weights to HP. fold in the transforms, and then cast the result back to LP
(C) Keep a copy of the weights in HP, and always fold transforms into this copy. Whenever the model needs to be queried, update the weight matrix by casting this copy LP.
Let us now analyze the floating point error behavior of these three methods for a chain WT,... where the 7; are orthogonal (so <(T - 1). From above, we have that the relative error is bounded by ndupkw.
Now in practice for large models, d is one of 1024, 2048, 4096 or 8192, so we will use d = 10
The condition number of weight matrix kw is usually around 102 109 we will use kw 10
For FP64, uFp64 R 10-16 and for lower precisions we have upp32 10-8 upp16 a 10 and WBF16 Rs 10-3 Then the floating point error single multiply is given by 10-10 for FP64, 10-2 for FP32, 102 for FP16 and 10 for BF16
31

---

Now let us consider our three options from before with n 10 transforms applied and low precision set as FP32. Then the relative error
(A): ndu 10"10010~$10 102
(B): a single matrix multiplication becomes ())))(A))(() + uLP, however this then accumulates in subsequent matrix multiplications so we can think of the bound as essentially something like ndw where up is somewhere between unp and uLP. Thus overall still bounded by nduLp*(W) (W)) = 10*10010-30 10000 = = 10
(C): this is the error in high precision plus single low precision transfer, so ndu
ULP = 10 1001000001106 1600 + 10 8 = 10
E.3 Transform Classes
Haar distributed orthogonal matrices. The first class is orthogonal matrices, which by definition satisfy QQ" = Q" = I. Their singular values are all one, which leads to nice properties for our use case: they have a determinant of 1 or -1 (so they keep the relative scale of the weights) and they have condition number of one (so they do not introduce too much floating error). Furthermore T-1 = ' so the inverse is both easy to compute and does not require additional storage. We can the of orthogonal matrices,
also generate them uniform randomly from the Haar distribution which is important so that there is no bias in the generation of the matrices that an attacker can use. The Haar distribution is uniform measure over orthogonal matrices, with the property that if Q is an orthogonal matrix sampled from the Haar distribution and U and V are any orthogonal matrices, then UQV is also sample from the Haar distribution (88 Note that this property of orthogonal matrices from the Haar distribution implies the following useful property: given a transformed weight W TW, then all candidates for T are equally likely, so there no way to distinguish between this decomposition of W' and any decomposition of the form W = (TV) orthogonal matrix. Thus, while the singular values and the right singular vectors are unchanged, the left singular vectors are sufficiently scrambled giving no useful knowledge of W.
where V is any
A standard way to generate a matrix from this distribution is to use the QR decomposition matrices with elements from the standard normal distribution
38
Q, R = torch. linalg. qr (torch. randn d))
(65) (66)
T Q torch, torch.diag() (R) sign
where we use PyTorch [40 code.
We also experimentally confirm that these transforms scramble with high probability. We generate orthogonal matrices T; from the Haar distribution, and multiply them with random normal generated weights: W = WT....f, where we vary n. We find that the distribution of the W' are such that its relative Frobenius norm
| - W ||
(67)
is tightly distributed around /2, and the cosine similarity of the flattened weights (where flat(.) is the flattening operation)
(lat(W) (,), flat(W")) (atW"))
(68)
fflat( [ (W]lll(faa(((""))
is tightly distributed around O. This indicates that in practice the transforms are unlikely to align with given weight matrix, instead transforming it to an almost orthogonal direction.
Low condition number matrices In order to generate matrices with higher frequencies yet small condition number, we generate matrices of the form U DVT where U, V are Haar orthogonal matrices and D diag(d) is such that d; se",, U, 21-c, Thus the singular values, which the entries of as this is in SVD form, are within [se- se and the condition number is bounded by 22 ( + 2s). As the determinant, which ad scales the weights, we force the weight scale to cycle by cycling
32

---

Optimization
Let denote transformation of components of the network from its original space to the one determined by a transform T applied to weights by the subscript (T)- Then we note that the transformation of the weights W and the gradients of the weights G are not the same:
Weight Space Transformation: W(T) WT
Gradient Space Transformation: G(T) GT
To see this note G L. G(T) W(T L, so
G = V (V (V () (VwWT) G(T)
(69)
which implies G(r)- GT
Note that we do not need to compute the transformed gradient G(z) explicitly, we use the transformed weight W(zy in the forward pass, in the backward pass automatic differentiation will
automatically calculate G(T)-
F11
Stochastic gradient descent variants
Most variants of stochastic gradient descent (SGD) use an update of the form
w(+1) = w(() ( Guid)
(70)
where G is the current gradient and Gold saved information from previous gradients (e.g. nh moments of the past gradients). Thus there are two alternatives we can have which may not necessarily be the same, either we can ensure the parameter updates are equivalent to parameter updates in the original space
I(++)= = (w(( nf((())) Gola)
(71)
(T)
-(w() nf((()) , T
(72)
or we can ensure that every part of the computation uses the transformed result
w(++1) (wi") (T) nf ((at)) (T) G(T)
(73)
-WOT nf (G(IT-T .GT))
(74)
Furthermore, might not be trivial to compute Gold from Gold, The most common saved gradient information is estimate of the first moment of the gradients, E[g"), for which ((T) cannot be computed
E[g), for which mąr) mI!'T.
However Adam/Adami also the second moment from just v. The issue here is that sign information is lost, so we cannot apply the transformation. As result, stick optimizers that only use m, e.g. SGD + Momentum.
In our implementation we use Muon, which also only uses m. Furthermore we use Equation (7)) which is easier to implement, rather than Equation (7I). However, for Muon (and many other optimizers), we use orthogonal transforms T then Equation (7I) and Equation 73) are equivalent, with the key point being that 7-T = T for orthogonal matrices.
F22 Pure SGD
In pure SGD, take steps of the form
w(+1) w(() -g(()
(75)
If we transform our weights before the forward pass: our weights become (W()) = W(1T, and our gradient becomes G(T)) = GT-T, so our step is
((++1) = (w (mn - (T( )(T)
(76)
= =W(T- - TTT
(77)
33

---

Note this is valid step, we are just applying SGD to a different network.
What would the step be if we did the step in original space and then transformed to the weight space:
w
(wlo) nTG(1) (T)
(78)
-(wl0) - n7G(()) T
(79)
-WOOT' n
(80)
What correction term would make it equivalent: we would need transform our gradients by T
w =(w()) -(((()))()) =W - -1:00110100 --
(81)
(82)
-WOTT
(83)
Issue with this correction: Note however this is one transform correction. If we are just now applying the nh transform, normal gradient descent would be
( W(OT())...
(1).
(84)
and to make the step equivalent to doing the step in the original space we would need to store Q(n) (()) and then apply
((++) (()..)). (V). Qm)))))
(85)
However storing (n) means we have an exact way to undo the transforms on our weights: W(T)
(()) (n).
F33 SGD Momentum
In SGD + momentum, we take steps of the form
((++1)= W( 17 ((m + )
(86)
If we transform our weights before the forward pass: our weights become (W()) and our gradient becomes G(T) GT-T,, step
W
W(++1) = (Tr) 1 (am( (()
(87)
= -WOT - 17 (t- BG(UT-T))
(88)
which is not valid step as the previous momentum term not the right space.
Fixing the momentum to be the right space: we need to apply the gradient space transform 7-T the momentum too
(t-1) ((++1) =(W()) (mm(t))) + (BC((()))))
(89)
-w0T-1 (am + 3G(0T-7).
(90)
Correcting this to be equivalent: we would need apply T the momentum (not doing the above correction) and T to the gradient:
((++)) (W(1)) - (am (n
((G(())) (1)))))
(91)
(am ( 3
(92)
Note however that correction steps are needed if T is orthogonal, then T TT
34

---

F4 Muon
In Muon, we take steps of the form
w = w(0 - n11 (amlt-1) + 3G
(93)
where O(M) an efficient approximate orthogonalization propeed [9] orthogonalization procedure is the mapping USVT + UVT
The non-approximate
If transform our weights before the forward pass, and apply our momentum correction term before the optimization step:
w(++))= = -(W") ((n)) 110 (am (am B(G()))))
(94)
=WOT - 110 (amllt-) (G00T-T)
(95)
If T is orthogonal: Then 7-T = T, and since SVD(M) = USVT implies that SVD(MT) USTVTT), then this is equivalent to original updates in the space.
=
G Matrix System attack
Applying transforms to the weights means that the intermediate activations (!) have been trans- formed. Thus one way to determine a bridge matrix is to use the intermediate activations.
Let us assume that the attacker has access to stage i and i at time & and t' > t, so they have VT(t), (t),, (t) from stage i and (!')), X,()), U (f') from stage + Note that
x(((') = (
(96)
so we have that
x,(t) X(()f
(97)
where î T,(t 1))..T((t') is the bridge matrix between times and t' for the boundary between stage and stage 1. This means that T the solution of matrix system.
However in reality the Xt) change over time steps for reasons other than the transforms (t)... This is due (1) different initial inputs (t) being used at different time t, and (2) when training, over different time steps the weights change.
We now investigate these two issues: the requirements the attacker needs to have this matrix system be defined, and whether the attacker can determine the bridge matrix from the matrix system.
Requirements for the matrix system. The assumption that the same initial data was inputted to the network at both time steps is impractical: we are serving inference or training from a very large dataset where data is randomly sent to different pipelines, the chances of the same initial data being inputted to the network are extremely small.
However the important requirement in the attack was that the intermediate activations to stage i were the same at both time steps, which an attacker can force. In order to do this, the attacker requires an additional stage j < i at both and L'. Then at both time steps, the attacker can output equivalent false activations:
time step they can output (V, and time step they can output FV) where F is a fixed full rank matrix. As long X, (t) ends up being full column rank, then this can work.
However this is also easy to defend against: we can check if the activations outputted from node follow similar distribution activations at previous time steps, and also check against the distribution of activations of the same stage in other replicas. The attacker is likely to be caught and expelled between time step and (as we have put in place measures to ensure >> t).
Determining I' from the matrix system. We consider the matrix system in Equation 97 Note that Xt(t) and (t't') are of the same shape b x d where b is the batch size and d is the embedding dimension, while I is d x d.
35

---

If (t) has full column rank then T can be uniquely determined by (X1(t))* Xt(t' where indicates the Moore-Penrose pseudo-inverse.
(t) does not have full column rank, then there are infinitely many solutions. To bias towards a specific solution we can use least squares, or if we know that ' is orthogonal Procrustes. This is likely to only give a good solution if (t) is close to full column rank.
orthogonal
H Further Experimental Details
We use PyTorch 40 for our implementations. Since our framework is simple, we implement
within three frameworks, torchtune [58], torchtitan [3 and NanoGPT [24]. We use torchtune for our inference time experiments, torchtitan for our learning attack experiments, and NanoGPT (and its derivatives, discussed below) for the training experiments.
When generating and folding in our transforms, we do this float64, which one the reasons that the morphing step is expensive. This turns matrices (doing the QR decomposition at a lower precision causes the error in Q² Q to be large). Furthermore reduce floating point error, when folding we first cast the weights to FP64, multiply in our transforms, and then cast back down to the original precision of the weights.
be very important for generating orthogonal
For the learning attack experiments, use the default hyperparameters in torchtitan. We grid search on the learning rate between torchtune's ' learning rate for finetuning Llama, 2e-5, to torchtitan's learning rate for from scratch training Llama, 3e-4. We find that for a few inconsistent boundaries, the optimal learning rate is closer to finetuning, while for many inconsistent boundaries (which we show results on) the optimal learning rate is closer from scratch training.
For the training experiments we use the Muon speed-runs of NanoGPT [8] However since the architecture has diverged away from the standard transformer architecture over time, we use re- implementation specific speed-run milestone where the architecture is the closest to modern transformers 48).
Stitched Model (Llama)
Stitched Model (FineWeb)
(% Compute Optimal)
100
Tokens Required
80
Morph Type Orthogonal
60
s=0.01
8=0.1
==0,5
Scratch
20
Base Llama
01
35
25
35
25
Perplexity
Perplexity
Figure Tokens required, as percentage of compute optimal (20B tokens), to reach various perplexities on Win with stitched Llama 3.2 1B model. We evaluate this with two sets of pretrained model weights that are morphed and then stitched: the base Llama model weights released by Meta (which is trained on wide variety of data) and our model weights that we train from scratch on FineWeb. Transforms are applied such that the boundary between transformer layers are inconsistent.
I Further Results
In Section (.ctioon554.mm m.4ees were trained to reach perplexity of 20, which is similar to the perplexity of the original base Llama model released by Meta (roughly 19). Here we continue running these models (learning based attacks and their baselines) to perplexity of 17, which is enough for the baseline to reach compute optimal (20B tokens for our 1B parameter Llama model), updating Figure to Figure 6 We represent the tokens required percentage of compute optimal.
36

---

Here, training with orthogonal required at least 80% of compute optimal (roughly 80% of baseline compute) reach perplexity of 17, and ε transforms required roughly the same compute as the baseline.
J Feasibility of Decentralized Training
We briefly review the literature in communication efficient training, noting that several works have made progress towards large-scale model training in the decentralized setting and that progress here may be more advanced than many researchers are awarc.
Communication efficient DDP is effectively solved, as it is side effect of much of the federated learning literature. Gossip protocols can be used in place of the synchronous all reduce 5 or simply many inner steps can be taken for each outer step [II] Convergence guarantees can be obtained this setting [31 33
even with the communication graph altering during training [55]
Moshpit-SGD [50] introduces a notable approach that is both communication efficient and scales well with heterogeneous compute and communication bandwidth. Diskin et [IO] perform real run, over 200 MB/s interconnects, using devices with a range of capabilities on a dynamic swarm to train 72.5M parameter ALBERT-xlarge variant. Here community of volunteers completed training run that would have taken over 3 months using standard 8x ExV100 100 setup, within 22 days on consumer hardware. Nodes were verified with an allow-listing approach, all gradient computations are assumed to be correct and each node computes full model gradients and hence has a copy of model weights.
Large models also possible, with SWARM parallelism (51) showing pipeline parallel training becomes less communication intensive relative to compute as models grow larger; hence improving utilization. This work practically demonstrated training of IB parameter LLM on T4 GPU's with 500 MB/s interconnects, achieving roughly 20% throughput overhead to centralized training. maintaining very high accelerator utilization, and also possessing basic fault tolerance. This achieved with redundancy within each pipeline stage, and dynamic and clastic routing between each stage, and the assumption of good actors. Learning @ Home 49 propose the Decentralized MoE architecture and an asynchronous training scheme in order to achieve communication efficiency over heterogeneous nodes. Such an approach can theoretically scale to very large parameter sizes but has not been practically demonstrated beyond 257M, and while node failures are handled, byzantine nodes are not.
Recently, Avraham et al. performed a public, large-scale, pipeline-parallel. decentralized training run the 7.5-billion-paramee scale. This run was based on the SWARM [5I) framework and used the technique from Ramasinghe et al. [47] to losslessly compress activations and their gradients.
37


Reviews:
[{"review_id": "LGPlnnDs6V", "reviewer": "Reviewer_mGZF", "summary": "This paper address the important problem of collaborative machine learning by proposing a framework that enables both training and inference without allowing any single participant to access or extract the full model weights. From the reviewer's perspective, this approach to creating \"unmaterializable\" models is novel and addresses a significant challenge in collaborative machine learning.", "questions": "Regarding the experiments, have the authors noticed any effect on the experimental results when varying the number of attackers?", "limitations": "The authors do a good job of outlining the limitations of their work.\n- They point out that while the morphing steps don’t mathematically change the forward function (or the inference time behavior), precision errors build up computationally when transforms are applied many times.  \n- They’ve analyzed the overheads for communication, inference, and memory.\n- They also admit that their experiments are limited to single-machine simulations, leaving out real-world decentralized setups and side-channel attacks (like timing or cache-based ones).", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n- **Motivation:** The motivation is strong, diving into a critical, new, and underexplored area: unmaterializable weights, where no participant can obtain the full weight set.\n    \n- **Method:** The core idea of introducing an identity function between neighbors, and then decomposing it into a random transform and its inverse, is both simple and elegant. \n    \n- **Performance:** The experimental results are impressive. It is a strong practical result that the proposed protocol adds only minimal overhead—approximately 0.1% in bandwidth and 3% in amortized latency—while maintaining its security promises.\n    \n- **Transparency about Limitations:** The reviewer appreciate that the authors have a dedicated section clearly discussing the limitations of their work. They transparently acknowledge issues such as the fact that their experiments are limited to single-machine simulations. \n\n**Weaknesses:**\n- **Orthogonality under Nonlinear Self-Attention**: In Figure 3.1(c), the self-attention operation is nonlinear. How is the orthogonality of \\( T_{i-1} \\) preserved in this case? Since the transformation is not simply \\( T_{i-1}T_{i-1}^{-1} \\), but involves a nonlinear self-attention step, it is unclear how orthogonality is maintained. Could the authors provide a more detailed explanation of this point?\n- **Clarity of Attack Descriptions:** The discussion of various attacks in Section 5.3 could be improved for clarity. To enhance readability, it would be helpful if the authors provided more background details on the specific attack methodologies they evaluated against. This could perhaps be included in an appendix to make the paper more self-contained for readers who may not be experts in this specific area.", "was_revised": true, "final_justification": "The authors' response addresses my primary concerns. I raise my score accordingly to reflect the paper's contribution in posing an important scientific question and its elegant method.", "rebuttal": "We sincerely thank the reviewer for praising our method and acknowledging that it \"addresses a significant challenge in collaborative machine learning\", characterizing this as a \"critical, new, and underexplored area.\"\n\n### **W1: Orthogonality under Nonlinear Self-Attention**\nWe do not require the self-attention layer to be linear, we only need it to satisfy the definition of a valid subfunction (Equation 2) or its generalization (lines 150–175), in which _the function can be highly non-linear but the inputs interact linearly with the weights so that the transforms cancel out_.\nWe explain explicitly on line 161 why self-attention qualifies as a valid sub-function.\nHowever, the normalization layer that precedes the self-attention function poses a challenge, as discussed on lines 176–191.\nA key contribution of our work is to show that a standard Transformer block with RMSNorm layers is a valid generalized subfunction under this definition when _orthogonal transforms are used for $T$_, and can therefore be morphed by our approach without changing the network's forward function.\nConcretely, we are able to pass the transform through the input norm, so $Norm(XT^{-1})TU=Norm(X)T^{-1}TU=Norm(X)U$, because its orthogonality ensures that the the normalization factor of RMSNorm does not change.\n\n### **W2: Clarity of Attack Descriptions**\nThank you for the suggestion. \nWe already provide detailed discussions of each attack in the appendices: general **stitching attacks** with probability analysis in Appendix A, **matrix system attacks** with more realistic requirements in Appendix E, and **learning-based attacks** with more results in Appendix G.2.\nFollowing your suggestion, we will add a separate section in the appendix concisely summarizing these attacks.\n\n### **Q1: Varying the number of attackers**\nFor **stitching attacks**, Appendix A quantifies how varying the number of attackers affects the probability of successful extraction.\nFor example, with 24 stages and 10 replicas, if attackers control 1% of the compute (e.g., 100 compromised nodes out of a total of 10,000 nodes) the probability of extraction within 10,000 steps is $2\\times 10^{-7}$ %.\nFor **matrix system attacks**, Appendix E details the requirements of a more realistic variant, which becomes easier to mount as the number of attackers increases (however the countermeasures are irrespective of the number of attackers).\nFor **learning-based attacks**, the decisive factor is the attackers' aggregate compute, not their count. Figure 4 demonstrates experimentally how much compute they require relative to the cost of fully training the model (measured in training tokens)."}, {"review_id": "DtuUimqeFS", "reviewer": "Reviewer_kk7D", "summary": "The authors propose a new collaborative training framework, entitled Unextractable Protocol Models (UPMs). The framework ensures that adversarial participants cannot extract the full model weights by joining the protocol in consecutive stages, under different aliases, in order to stitch portions of the model together to recover the entire model. Analysis and experimental validation of UPMs showcase that they retain accurate outputs, even in the face of precision errors, while introducing time-varying random invertible transforms into the model weights at stage boundaries. This protocol will allow safer and more secure collaborative training and makes strides within the trustless,  decentralized setting.", "questions": "1. I was a little confused by Figure 4. How is it possible that the transformations actually reduce the number of tokens required to reach a perplexity of 20. There's no free lunch, especially when the transformations are in some sense a barrier to true performance.\n2. How many morph steps in total are used during training? If each iteration is one morphing step (from what I've gathered from Line 139), aren't tens of thousands of iterations common in training an LLM from scratch?\n3. Relating to Q2, if the number of morph steps is quite large, there does seem to be a sizable decrease in model performance for certain LLMs. First, how much degradation is there for a normal training or fine-tuning session? Second, would UPMs no longer be viable for large-scale training at smaller precision? Thus, UPMs likely cannot be utilized in low-memory settings.\n4. Is there still any valuable information that an adversarial agent can glean from picking up different portions of the model, even if they have different transformations? For example, would size, magnitude, or distribution of the values throughout the weights be sensitive information?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "# Strengths\n\n1. Trustless, decentralized learning is an extremely realistic and useful area of research. The proposed protocol is an exciting idea that propels collaborative training a step forward.\n2. The underlying idea, and proof-of-concept, is novel and well-worked.\n3. The paper is clear, clean, and well-organized.\n4. Experiments are detailed and well-done (outside of the weaknesses listed below).\n\n# Weaknesses\n\n1. Model performance does seem to degrade somewhat substantially when utilizing UPMs (Figure 3) at usual training precisions.\n2. In the distributed setting, devices usually have low memory capacity and utilize low-precision training. As a result, UPM may be quite noisy when being utilized in resource-constrained settings (which are common to distributed settings).", "was_revised": true, "final_justification": "Very original and interesting paper with great results. It’s an accept from me! Authors did a wonderful job in their rebuttals.", "rebuttal": "We deeply appreciate the reviewer's endorsement of our setting as \"extremely realistic and useful\", their description of the method as \"an exciting idea that propels collaborative training a step forward\", and their assessment of the experiments as \"detailed and well-done\".\n\n### **W1: Performance degradation due to precision**\nUPMs **do not degrade model performance during training or inference in practice** (we will clarify this). We provide the experiment in Figure 3, which highlights the drift caused by finite precision, to **point out an important consideration when implementing UPMs**.\n\nDuring **training**, gradient descent naturally corrects any floating-point drift introduced by numerical imprecision from the morphing, as demonstrated by the results in Table 1 and Figure 6.\n\nDuring **inference**, we have two options. \nThe first option is to run inference in high precision (FP32 or FP64) as Figure 3 shows that performance degrades extremely slowly for FP32, with perplexity increasing by less than $1\\times 10^{-4}$ over 10,000 transform steps.\nThe second option is to keep the final weights in high precision on disk while maintaining a low-precision copy in GPU VRAM for inference.\nThen, at each transform step we apply the high-precision transform to the high-precision weights on disk, which are cast down and loaded into GPU VRAM to replace the copy used for inference.\nBecause transforms are applied infrequently, disk-to-VRAM communication is negligible, and the on-device memory holds only the low-precision weights.\nNote that our experiment in Figure 3 shows that such a workaround is important for low-precision inference as naive FP16/BF16 inference would slowly cause model weights to drift due to accumulating floating point error.\n\n### **W2: Performance in resource-constrained settings**\nAs discussed in Section 5.2, UPMs add minimal overhead for both communication and memory (given the inference workaround above). Only the transform must be communicated every few steps, which is smaller than an activation gradient, and once folded into a layer it can be discarded introducing minimal memory overhead.\nFurthermore, because UPMs operate in a pipeline-parallel setting where the model is split stage-wise across devices, even GPUs with limited VRAM (e.g., T4s) are already supported, as they need to hold only a single stage.\nFor devices that are even more resource-constrained, stages can be further split and still be supported by our framework. For example, splitting a Transformer layer between the self-attention and MLP blocks would still work with our transforms, though we leave concrete experiments to future work. \n\n### **Q1: Understanding Figure 4**\nThe experiments in Figure 4 are **demonstrating learning-based attacks**, in which attackers have stitched together trained but incompatible weights (each with different transforms applied because they are from different time steps), rather than results from training models from scratch.\nAlthough incompatible weights cannot be combined to recover the original function, they still contain useful information.\nConsequently, a plausible attack is to fine-tune starting from these weights to determine whether a useful model can be obtained with less computation than training the entire model from scratch.\nFigure 4 shows that the computation required, measured as the number of tokens used during training, is less than training from scratch, with at most a 36% reduction. \nExtended results in Figure 5 in the appendix indicate that the reduction is at most 20% for performance matching compute optimal training.\n\n**For results on training from scratch** with and without UPMs, see Section 5.4 and Table 1.\nAs expected, the number of tokens that UPMs require to match the perplexity of the baseline is the same or more (not less), and it specifically the same for orthogonal transforms.\nAdditional plots are provided in Figure 6 in the appendices.\n\n### **Q2: Morphing frequency during training**\nWe apply transforms frequently during training (and inference) to prevent an attacker from leaving the protocol and rejoining within a single transform window.\nThus, as mentioned on line 270, our standard is to morph every 30, which during training corresponds to a morph every 10-20 forward-backward iterations. \nIf we want to ensure that no participant can exit and re-enter within those 30s, we can require each new participant to wait 30s before joining the protocol.\n\n### **Q3: Performance degradation during training**\nAs mentioned in the response to W1, during training gradient descent corrects for small precision errors leading to **no performance degradation**. \nWe experimentally demonstrate this Table 1, training with orthogonal transforms yields no degradation in validation loss after a fixed number of iterations, whereas other transforms do as those transforms alter the loss landscape. \nWe further show in Figure 6 in the appendix that the error in training loss (training drift) is small and constant throughout training with orthogonal transforms. \n\n### **Q4: Information that can be gleaned by participants**\nOur method causes not only the weights but also the intermediate activations and gradients to change continually.\nBecause we often use orthogonal transforms, which preserve weight and gradient norms, an adversary could still observe consistent statistics for those norms.\nHowever, there is little an adversary can do with that information."}, {"review_id": "rxZgc9D1ai", "reviewer": "Reviewer_vesi", "summary": "This paper proposes Un-extractable Protocol Models (UPM), a model parallel based training setup under decentralized setting to provide a solution for full model attack under the distributed setting. And authors provide experiment evaluation on GPU devices to verify the proposed method on Llama 3.2 1B on Functional Equivalence, inference overhead, and training quality. \n\nContribution:\n- introduce Unextractable Protocol Models (UPMs), a framework enabling collaborative training and inference without allowing any participant to extract the full model weights.\n- Use experiment to verify the inference overhead, and training quality.", "questions": "- The setting, is it a new one? \n- Why the setting is a reasonable setting? \n- Especially why this kind of attacker is reasonable here?\n- How about other solutions for the current setting? Why the proposed one?\n- how to synthesize the distributed setting here, and why this kind of experiments can really evaluate the latency in real distributed setting\n- Since evaluated on GPU environment, does the setting of hyper parameters (for example, batch size) would impact the overall latency? Does these conclusion changes when batch size changes?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- __Strengths__\n  - Clarity: the paper is cleared written, especially providing throughout description on the proposed model architecture\n  - Quality: meaningful solution and design.\n  - Originality: Good. the paper gets insights from model parallel to propose such framework for distributed setting. \n\n- __Weakness__\n  - Limited discussion on why previous work is not sufficient for targeted setting\n  - Unclear about the real-world applicable setting for mentioned attacker. Why it is a readable one?\n  - Limited description for detailed experiment setting, say, how to synthesize the distributed setting here, and why this kind of experiments can really evaluate the latency in real distributed setting", "was_revised": true, "final_justification": "The authors have addressed most of my concerns and I will maintain my support to the paper.", "rebuttal": "We are grateful to the reviewer for praising the paper as \"clearly written\" and recognizing it is a meaningful solution.\n\n### **W1,W2,Q1-Q4: Understanding the setting, previous work, and other solutions**\nWe wish to clarify that this is a new setting (see line 32), which reviewer kk7D praised as \"extremely realistic and useful\".\nDecentralized training is an early but rapidly growing sub-field (see Appendix H for a full discussion of emerging work).\nThe problem our work addresses is that the real world economics of large scale decentralised training are not practical, as there is no incentive for participants to collectively donate the millions of dollars worth of compute required to train a very large and expensive model **if one of them can easily leak the trained weights** (explained on lines 22-27). The solution is for **a scheme that allows for such a model to be collaboratively trained and hosted while no participant can gain access to the entire weight set** (lines 28-29). In short, unmaterializability lets models be created, hosted, and used solely within the protocol, enabling incentive mechanisms such as revenue-sharing among participants. If any participant can obtain the full model, these incentives become impossible. We will make the setting and its novelty more clear in the improved version.\n\nAs this is a novel setting there is no directly comparable prior work. As discussed in Section 4, the closest related work is black-box extraction attacks, which aim to gain a model with similar capabilities using only API queries. Other potential solutions for our setting, discussed on line 97, either rely on centralization or on expensive cryptographic primitives such as secret sharing or homomorphic encryption. Our method exploits the inherent structure of distributed pipeline parallel training, making it much more efficient. We also discuss the differences and similarities between our setting, Federated Learning and Decentralized Federated Learning in the related work (Section 4).\n\n### **W3,Q5,Q6: Evaluating latency**\nThere has been several recent works on feasibility of decentralized training [8,36,37] (see Appendix H for a full discussion) which focus on latency and throughput numbers that can be achieved in this setting. \nOur evaluation in Section 5.2 shows that the overhead of UPMs in this setting is quite low during inference (e.g., 3% impact to overall latency and 0.1% impact to total communication volume), the phase where morphing's impact is most pronounced. During training, the relative impact on all metrics decreases because memory use, communication, and computation grow with the model’s operations.\n\nThese figures depend on both **the morphing frequency and the communication efficiency of the distributed implementation**. Although the frequency can be reduced, the system must prevent participants from rejoining within a morphing interval. We did our communication overhead calculations relative to our centralized implementation, so the overhead of our method in practice could be much lower. Other hyper-parameters may influence communication efficiency, but to a much lesser extent."}, {"review_id": "P3EVNaBWQM", "reviewer": "Reviewer_Lkyy", "summary": "The paper proposes addresses the issue of model stealing in decentralized collaborative training setup where each party contributes training of a part of a large model. An attacker may join such collaborative training at different times and accrue the entire model parameters. The paper proposes to morph invertible transforms in consecutive model stages such that in a given time-step these transforms cancel each other (keeping model outputs the same as without transforms) but in different time steps they do not (changing the model outputs). This way an adversary cannot just join the collaborative training at different times to steal the model. They evaluate their protocol on models with ~1B params, discuss the computation and latency overhead, and potential attacks against their protocol.", "questions": "- Do you need to apply transforms for each stage for this defense to work? Can you omit a few complex stages and still get the same protection from model stealing?\n- Is it necessary to apply complex matrix based transforms or simple linear transforms can suffice? For instance, what if each stage add a random time-dependent noise to its output and subtract the same from input of the next stage? This might also reduce the complexity in backward pass as all noise is just a constant.", "limitations": "- Perplexity increase is not trivial for fp16/bf16, which can be concerning.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- Idea of morphing time-dependent transforms is very simple and seems effective in most cases. Given the growing sizes of modern ML models, this can be a simple and effective method to thwart model stealing.\n- The paper does a nice job of explaining the idea. It is well-written and easy to understand.\n- Evaluations show that morphing does not impact functionality of the model in most cases and the latency overhead at inference time is minimal.\n- The paper provides a comprehensive set of subfunctions to be used for various common functions involved in modern LLMs.\n\nWeaknesses\n- The motivation of the attacker is not clear to me. If the attacker can host an entire model and  that the final trained model will anyways be available for use by everyone, can the attacker not just steal the final model? Why does the attacker want the intermediate models?\n- Perplexity increase is not trivial for fp16/bf16, which can be concerning.\n\nMinor:\n- Line 130: should be figure 2A.", "was_revised": true, "final_justification": "The rebuttal address my concerns and I would like to maintain my rating of Accept for this paper.", "rebuttal": "We greatly appreciate the reviewer's characterization of our method as \"simple and effective,\" and their remark that the manuscript is \"well-written and easy to understand\".\n\n### **W1: Attacker Motivation**\nThe final trained model is not intended to be accessible by everyone, **even at inference**, only participants that hold part of the model can see their portion. \nAs a result, after training the model exists, and yields value, only if enough participants continue hosting model portions to support inference.\nThe model's value would be lost if the model was released after training (like current open weight models), or leaked by attackers. Note that **even partially trained versions of the model can still have decent performance**, and thus value, so it is important to also protect against weight extraction during training.\n\n### **W2: Perplexity Increase**\nThese experiments are provided to point out **an important consideration when implementing UPMs**. During training, gradient descent naturally corrects any floating-point drift introduced by numerical imprecision from the morphing, as demonstrated by the results in Table 1 and Figure 6. For low-precision inference, a workaround would be to actually morph high-precision weights (FP32 or FP64) stored on disk, which are then cast down for a low-precision copy in GPU VRAM used for inference. Note that our results in Figure 3 show that perplexity increase is extremely slow for FP32. Because transforms are applied infrequently, disk-to-VRAM communication is negligible, and the on-device memory holds only the low-precision weights.\n\n### **W3: Typos**\nThanks, we will fix this in the improved version.\n\n### **Q1: Transforming All Stages vs Some Stages**\nYes, it can work for only transforming a few stages, however as the weights are now less \"scrambled\", this becomes a trade-off with how much compute an attacker needs to pull off a learning attack. It is quite likely that a better trade-off can be gained by transforming more complex stages as you suggest. We leave investigating this trade-off to future work.\n\n### **Q2: Possibility of Simpler Transforms**\nAdding random time-dependent noise to a layer's output does not change the weights, it only changes the activations. Thus weights are not inconsistent over time and can easily be pieced together to get the full model.  \nWe specifically targeted complex and random matrix based transforms over simple linear transforms in order to make learning based attacks difficult. It is conceivable that transformations such as simple scaling of the entire weight matrix would be easy to undo by gradient descent."}, {"review_id": "PswTFm7HJg", "reviewer": "Reviewer_Ed6f", "summary": "This paper introduces Unextractable Protocol Models, a framework for collaborative training and inference of large neural networks where participants each hold only a subset of model parameters, preventing any single entity from extracting the complete model weights. \nThe approach periodically applies time-varying, random, invertible transforms at participant boundaries that preserve network functionality but render cross-time parameter assemblies incoherent and mitigate attacks that attempt to collect model shards across different time steps. Experiments demonstrate that the method maintains model performance while adding minimal computational overhead and significantly increases the cost for attackers to reconstruct full models.", "questions": "- Why is it that you need double precision to compute the matrices $Q$? Numercial stability? If so, where does this come from? Can you please elaborate on these dynamics?", "limitations": "Sufficiently discussed.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**:\n- The paper addresses an interesting problem where, instead of keeping data private, the model has to be kept private and inaccessible by participants. This could be relevant in cases where a model learns patterns and behavior that must not be made public.\n- The contributions are well described and nicely distinguished from related work, espeically federated learning\n\n\n**Weaknesses**\n- While I appreciate the approach and think that model security is an important topic, the abstract appears overly specific to me. I think, the abstract would benefit more from a discussion _why_ model secrecy is key in future decentralized applications. \n- Similarly, the introduction does not discuss _why_ model secrecy is practically relevant. I was also wondering what the incentive for a participant would be to train a model they do not benefit from? In Federated Learning, clients at least get to benefit from the global model and their peers' data. \n- To me it is not fully clear how the level of model secrecy interacts with the number of transforms, i.e., what level is considered as \"secret\" or how do you compute likelihood for a successful model extraction? This question is most fundamental in my review.\n- When doing a speed run comparison, showing the loss or perplexity trajectory over time is usually beneficial for the reader to understand whether the proposed approach is viable for their individual use cases.", "was_revised": true, "final_justification": "The authors have addressed my concerns.", "rebuttal": "We thank the reviewer for their acknowledgement of the clarity of the setting and the differentiation to related work \n\n### **W1,W2: Understanding Model Secrecy**\nModel secrecy is key in decentralized models as it is **the only way to make the economics of large scale decentralised training practical**. The fundamental problem we address is that today, there is no economic incentive for participants to collectively donate the millions of dollars worth of compute required to train a very large and expensive model if one of them can easily leak the trained weights (explained on lines 22-27). The solution is for a scheme that allows for such a model to be collaboratively trained and hosted while no participant can gain access to the entire weight set (lines 28-29). This allows schemes for programmatic value flow (for example, assigning revenue to participants) to be implemented. We note that reviewer kk7D praises this setting as \"extremely realistic and useful\". We will make the setting, its importance, and differences to existing settings such as federated learning more pronounced in the improved version. \n\n### **W3: Model Secrecy and Number of Transforms** \nIf a simple static model is hosted, which anyone can join and contribute to hosting, an attacker can gain access to the network and collect weights from different stages over time, eventually assembling the full model. At a high level, our approach prevents this by making the model weights incompatible with each other at different time steps. Thus **model security is not tied to the number of transforms**, but to ensuring that an attacker cannot gain a set of weights that use the same transforms, which requires **ensuring that a new transform is applied between when an attacker exits the protocol and enters again**. Note that this can be ensured by making participants have to wait for a transform step before entering, so there is a trade-off between the join rate and the overhead of the transforms (as more frequent transforms require more computation). As mentioned on line 270, our practical implementation has a transform step every 30s, meaning that anyone joining the protocol has to wait at least 30s, and has an overhead of 3% to total latency.   \nWe provide a much more detailed write up of the setup and the probability of extraction for the attacker (both with and without UPMs) in Appendix A. \n\n### **W4: Loss Curves** \nAs mentioned in lines 323-326, **for orthogonal transforms the loss curve is exactly the same as the original loss curve**. We demonstrate this in Figure 6 in the appendix, where we plot the difference between the original loss curve and our approach over time, showing that for orthogonal transforms with on GPT2 124M, 355M, and 1.5B, the difference between the loss curves at each step is roughly 1e-2 (the final loss is 3.28) over 1.75B tokens. This highlights the importance of our decision to use orthogonal transforms during training; orthogonal transforms make the transformation on gradient space also orthogonal, so the exact same steps are taken (which is mentioned in Section 3.3, proven in Appendix D, and experimentally shown in Figure 6).  To highlight the extent to which our approach does not alter training, we include the loss curve as a table below (data from the same source used to plot Figure 6.).\n\n| Method         |   1    |  500   |  1000  |  1500  |  2000  |  2500  |  3000  |  3500  |  3611  |  \n|---------------|--------|--------|--------|--------|--------|--------|--------|--------|--------|  \n| NoTransforms   | 11.013 | 3.986  | 3.528  | 3.408  | 3.407  | 3.444  | 3.241  | 3.145  | 3.273  |  \n| Our Approach     | 10.979 | 3.993  | 3.524  | 3.420  | 3.408  | 3.445  | 3.241  | 3.139  | 3.268  |\n\n### **Q1: Double Precision for Computing Transforms** \nDouble precision is needed to properly generate and represent the transforms, as otherwise there is large floating point representation error. Generating random orthogonal matrices as per Eq. 52-53 in Appendix C2 with single precision (compared to double precision) drastically reduces the orthogonality of the resulting transforms (how large $||Q^TQ-I||_2$ is). After generating transforms in double precision, the transforms can be combined with single precision weights and the output stored in single precision (that is, the precision of the actual weight matrices is never altered), and this is sufficient to avoid numerical precision errors as shown in Figure 3."}]

Author rebuttal:

Meta-review: This paper introduces Unextractable Protocol Models (UPMs), a framework for collaborative training and inference in which model shards remain incompatible across time, preventing any participant from reconstructing the full weight set. The method leverages pipeline parallelism and periodically applies invertible transforms at stage boundaries. Experiments on billion-parameter models show that UPMs preserve model accuracy, incur minimal training and inference overhead, and significantly raise the cost of model extraction attacks.

Reviewers found the paper novel, elegant, and clearly written. They highlighted the strength of the idea, the thorough analysis of attacks, the practical overhead results, and the relevance of the problem to decentralized machine learning. At the same time, they pointed out limitations: the approach applies only to pipeline parallel training, its guarantees are operational rather than cryptographic, the evaluation was carried out in single-machine simulations rather than real decentralized deployments, and inference drift occurs under low precision. The authors provided clarifications and mitigation strategies during the rebuttal, and reviewers were satisfied, with all ultimately recommending acceptance.

Beyond the reviewer discussion, I remain somewhat cautious about two aspects. First, the approach is essentially security by obfuscation, relying on repeated morphing to frustrate current extraction techniques rather than offering strong cryptographic guarantees. While this appears effective against the attacks considered, it may be fragile in the longer term. Second, the scope is narrower than the title suggests: the method is tied to pipeline-parallel setups and was evaluated only in simulated single-machine environments, not in a truly decentralized system. These caveats should be reflected more explicitly in the presentation.

Overall, despite these limitations, the work provides a timely and practical mechanism for collaborative training that has generated strong support from the reviewers.
Decision: Accept (poster)