Black-Box Privacy Attacks on Shared Representations in Multitask Learning
Authors: John Abascal
Nicolás Berrios
Alina Oprea
Jonathan Ullman
Adam Smith
Matthew Jagielski
Venue: ICLR 2026 Poster (2026)
Keywords: multitask learning
privacy
attacks
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We propose a black-box task-inference threat model, where the goal is to determine if an entire distribution was used to train a multitask model. By leveraging task structure, we construct high-power attacks without reference models for calibration.

Abstract: The proliferation of diverse data across users and organizations has driven the development of machine learning methods that enable multiple entities to jointly train models while minimizing data sharing. Among these, *multitask learning* (MTL) is a powerful paradigm that leverages similarities among multiple tasks, each with insufficient samples to train a standalone model, to solve them simultaneously. MTL accomplishes this by learning a *shared representation* that captures common structure between tasks and generalizes well across them all. Despite being designed to be the smallest unit of shared information necessary to effectively learn patterns across multiple tasks, these shared representations can inadvertently leak sensitive information about the particular tasks they were trained~on.

In this work, we investigate privacy leakage in shared representations through the lens of inference attacks. Towards this, we propose a novel, *black-box task-inference* threat model where the adversary, given the embedding vectors produced by querying the shared representation on samples from a particular task, aims to determine whether the task was present in the multitask training dataset. Motivated by analysis of tracing attacks on mean estimation over mixtures of Gaussian distributions, we develop efficient, purely black-box attacks on machine learning models that exploit the dependencies between embeddings from the same task without requiring shadow models or labeled reference data. We evaluate our attacks across vision and language domains when MTL is used for personalization and for solving multiple distinct learning problems, and demonstrate that even with access only to fresh task samples rather than training data, a black-box adversary can successfully infer a task's inclusion in training.

Published as a conference paper at ICLR 2026
BLACK-BOX PRIVACY ATTACKS ON SHARED REPRE- SENTATIONS IN MULTITASK LEARNING
John Abascal", Alina Oprea & Jonathan Ullman
Northeastern University
(abascal.j, a.oprea, J.ullman)eeorttee j.ullmannehorrthasttem...m
Nicolás Berrios!
Adam Smith
Georgia Institute of Technology
Boston University & Google Deepmind ads22 (uu.. edu
berrios@gatech. edu
Matthew Jagielski
Google Deepmind
jagielsk, www.@aiatttiii.......oo com
"Corresponding author.
"Work completed while visiting Northeastern University.
ABSTRACT
The proliferation of diverse data across users and organizations has driven the development of machine learning methods that enable multiple entities jointly train models while minimizing data sharing. Among these, multitask learning (MTL) is a powerful paradigm that leverages similarities among multiple tasks, each with insufficient samples to train standalone model, to solve them simulta- neously. MTL accomplishes this by learning shared representation that captures common structure between tasks and generalizes well across them all. Despite being designed to be the smallest unit of shared information necessary to effectively learn patterns across multiple tasks, these shared representations can inadvertently leak sensitive information about the particular tasks they were trained on.
this work, we investigate privacy leakage shared representations through the lens of inference attacks. Towards this, propose novel, black-box task- inference threat model where the adversary, given the embedding vectors produced by querying the shared representation on samples from particular task, aims determine whether the task was present in the multitask training dataset. Motivated by analysis of tracing attacks on mean estimation over mixtures of Gaussian distributions, we develop efficient, purely black-box attacks on machine learning models that exploit the dependencies between embeddings from the same task without requiring shadow models or labeled reference data. We evaluate our attacks across vision and language domains when MTL used for personalization and for solving multiple distinct learning problems, and demonstrate that even with access only to fresh task samples rather than training data, a black-box adversary can successfully infer task's inclusion in training.
INTRODUCTION
Multitask learning (MTL) has emerged powerful paradigm that leverages similarities among multiple learning tasks, each with insufficient samples to train a standalone model, to solve them simultaneously while minimizing data sharing across multiple entities, such as users and organizations. MTI. accomplishes this goal by learning a shared representation that captures common structure between the tasks. Concretely, a shared representation could be neural network that learns a mapping of the data from all tasks into a shared feature space, where similar data points across tasks cluster together, and task-specific output layers (or heads) operate on these embeddings-tiie is, the outputs
"oode available httptttpss //tp::::////eee http:///////h//./o/////// attc:p///...aa...ccoo/ooeeaaaaae attacks

---

Published as a conference paper at ICLR 2026
of the shared representation- to make predictions. MTL methods have shown remarkable success in various domains, including computer vision (Girshick 2015). natural language processing (Collobert
& Weston 2008). federated learning (Smith 20.1 Hanzely & Richtarik 2021] Mansour et al.
2020|GGoshh Ghosh et al. 2020), drug discovery (Ramsundar et al. 2015). and financial forecasting (Ghosn & Bengio (996). For example, MTL can be used personalize image classification models by
learning a shared representation across many users and locally adapting to each user's small, on-device
photo library.
Despite being designed to capture only generic patterns that can be applied to several downstream tasks, these shared representations can inadvertently leak sensitive information about specific tasks, or underlying data distributions, that they were trained on. The privacy risks that arise are of particular concern when data from several sensitive entities, such as individual users, private organizations, or data silos, are jointly used for training across multiple tasks or non-uniform data distributions. Here, shared representations are often the minimum piece of information that each entity must contribute in order to achieve strong generalization while only contributing limited number of samples. This point is highlighted in prior work on privacy-preee ppree-ppres collaborative learning (Shen et al. (023)) where sharing only the representation, not task task-specific layers, demonstrably improves convergence rates and model performance when learning over multiple parties.
In our work, we study privacy attacks at the level of entire tasks and broadly investigate the privacy risks of jointly learning over multiple tasks by mounting these attacks on the smallest unit of information required jointly train model, the shared representation. In particular, we attempt to determine whether, and to what extent, an adversary can infer a given task's ' inclusion in training given black-box query access to (i.e. the ability to sample embedding vectors from) the shared representation. While there is prior work that explores privacy attacks on MTL, this work is limited to sample-lever membership inference and model extraction (Yan et al., 2024) where the adversary can make queries to the task-specific heads of the multitask model and train reference models.
Towards this goal, we propose a new threat model called mask-inference and develop black-box attacks with minimal adversarial knowledge to infer the inclusion of a task in MTL. Unlike membership- inference (Homer et al., 2008: Sannararaman et al. 2009 Dwork et al. 2015 Shokri et al. (0166). task-inference generalizes individual privacy attacks from the sample level to the task level, where the adversary aims to determine the presence of an entire target task, rather than a particular sample, in the training set. Critically, our threat model only assumes that the adversary has access to samples from the target task's 's distribution. We identify two variants of our threat model, one with a strong adversary who has the specific samples used train the shared representation and another with a weak adversary who only has independent samples from the target task's s distribution. Our experiments demonstrate notable separation between these two threat models in terms of their attack capabilities, and we provide analysis of a simplified learning setting to help explain our empirical findings in machine learning. We observe that while both adversaries can mount successful attacks, having access to training samples provides a sizeable advantage. Furthermore, in contrast to prior work (Shokri et al. 2016 Carlini et al. 2022 Liu al. 202 Chaudhari al.) 2022 Abascal et al.. 2023) Hartmann et al.) [0223] Yan et al. 2024), neither of the task-inference adversaries require auxiliary datasets to train reference, or shadow, models to calibrate their attack. Instead, our attack exploits the key observation that embeddings from the same task are codependent, which allows an adversary with multiple samples to amplify the membership signal and construct a powerful statistical test without reference models.
We comprehensively evaluate our attacks across vision (CelebA (Liu et al. 2015) and language (Stack Overflow (Annamoradnejad et al.) 2022) datasets for two representative MTL use cases. The first is personalization, where each user is a task with few samples for a single, shared learning objective (e.g. personalized spam detector). The second is solving multiple learning problems simultaneously. where each task is a distinct classification problem with insufficient data to train standalone model, but whose shared underlying structure allows them to be solved jointly (c.g.. detecting different post topics). The privacy risks of task-inference are therefore contextual: in personalization, the attack leaks an individual's participation in training, while for multiple learning problems, it reveals the inclusion of an entire subpopulation. We also study the factors that lead to musk-inference leakage by observing how attack success varies in the model's 's generalization gaps. Our findings demonstrate that a purely black-box adversary can successfully infer inclusion of tasks in MTL using only query access to the shared representation, even with weak access to training data samples.
2

---

Published as a conference paper at ICLR 2026
2 BACKGROUND AND RELATED WORK
Multitask Learning. The goal of MTL (see (Caruana 1997) for an early survey) is to learn jointly over several related tasks, often with sparse data, by exploiting shared features between them. A task T is defined as distribution over samples in X y (e.g. d-dimensional real vectors that correspond to binary labels). In this work, we assume that the tasks for some learning problem are drawn from a task distribution, 2. To leverage common features between tasks, we can learn a shared representation h X' + Z, that maps samples in A' to lower dimensional space, Z, where the representation vectors, called embeddings, capture the shared structure of the tasks. Mapping these samples to a lower dimensional space simplifies the learning problem, allowing us to use simple, linear classifiers 9i : 2 + {0, 1}. Given data samples from T tasks F = (T)....; -T) and loss function C. learning the shared representation h can be written as the following optimization problem:
Σ C(gs: o ho, T²)
min he M
min 921 ST
The shared representation / often takes the form of a neural network that maps an example x E X to an embedding vector 2 Z with task-specific classifiers {g1..... 9r being linear classifiers applied to the embedding vector. Thus, the goal is to learn an h such that it maps data samples to embedding vectors that are linearly separable.
In this work, we consider two MTL settings that are representative of common scenarios: (1) personalization, where each user or person represents task with a personalized objective (e.g.. multilabel face detection, recommendation, etc.) learned from sparse data by leveraging the shared representation trained over all users; and (2) multiple learning problems, where the tasks correspond to distinct classification problems (c.g., detection of facial attributes, binary topic classification, etc.) that share similar underlying structure. We choose these two settings because they highlight how our proposed threat model (Section 3) captures and abstracts the privacy threats posed by existing attacks at the individual level (Shokri al., 2016), user level (Kandpal et al. 2023), and property level (Ateniese et al. 2015).
MTL has deep connections to federated learning (FL) (McMahan et al. 2017), a machine learning framework where a central, trusted server coordinates model training across several parties, such as users or silos (Huang et al.] [0022) Several works study the connections between FL and MTI. (Smith)
et al. 2017: Yu et al.) [022]] Tan et al. 2203] Hu et al. 2023 Fallah et al.) 2020 Mansour et al. 2020), many of which focus on techniques to learn personalized models by locally adapting a
shared representation on-device. The widespread use of FL (Lu & Wang. 2022) serves additional
motivation for studying privacy leakage from the shared components of collaborative learning.
Privacy Attacks on ML Models. While overparameterized models are known to memorize training data (Zhang et al. (017)), they often do without compromising generalization (Belkin et al. 2019 Feldman| (020), making privacy attacks a significant threat (Shokri et al.) (016)) These risks are typically studied through membership-inferees
attacks (or MIAs), which represent fundamental form of privacy leakage. The goal of an MIA adversary is to determine whether specific record was used in training (Homer et al. 2008). We generalize this notion of leakage by defining a novel threat model for MTL that restricts the adversary's 's access to the shared representation, as it constitutes the minimal information that must be exchanged for jointly training over tasks with few samples. One work (aan et al.) (024) adapts sample-leve MIAs to the MTL setting but assumes query access to the task task-specific classification heads; this assumption would render our goal of task-level inference trivial.
Our proposed threat model is distinct in that generalizes and interpolates several existing privacy attacks that operate at coarser granularity than the sample-level depending on the definition of tasks within MTL. Namely, property (Ateniese et al., 2015 Chaudhari et al. 2022, Hartmann et al., 2023) and dataset (Maini et al. 2024) inference aim to determine the frequency and inclusion of entire subpopulations in the training data of a machine learning model, respectively. Similarly, the goal of a user-inference (Kandpal et al. 2023) adversary is to infer whether a user's 's entire data contribution was present in training. In contrast to these works, we target the intermediate embedding directly, rather than the final model, and assume minimal access side information like reference data.
3

---

Published as a conference paper at ICLR 2026
Multiple works have also studied attacks on representations under different assumptions. Existing attacks on representations (Song & Raghunatham, 2020, Liu et al. 2021a) target models that are trained with contrastive loss, which explicitly maximizes the separation between embeddings of different samples. Our work differs from these approaches by investigating leakage of sensitive information from representations which are learned implicitly generalize across diverse set of supervised, downstream tasks.
THREAT MODEL
We investigate whether, and to what degree, shared representations leak information about the specific tasks they were trained through the lens of inference attacks (Homer al. (008). In this setting, there is an MTL model that is trained on several tasks simultaneously to learn a shared representation, encoder, and individual task layers. Black-box queries can then be made the encoder receive representation vectors, or embeddings, for any given input. A challenge task drawn from the same distribution the tasks used train the MTL model. Our adversary uses their query access, along with data drawn from the challenge task, to infer whether the challenge task was included while training the MTL model. In this work, we study this leakage from purely black-box perspective, assuming no knowledge of the underlying data distributions to train shadow models, as is common in prior works (Carlimi et al., (022). We can describe our threat model using the following security game between challenger and an adversary:
Mask-Inference Security Game
1. The challenger receives T tasks F (Ti...., ... TT), drawn from a task distribution Q. For each of the tasks, T;, the challenger is given a batch of samples X; and concatenates the T batches into a dataset D = (X,.....,...!
2. The challenger trains shared representation he + TMTI(D) by simultaneously learning task-specific models that share h.
3. The challenger randomly selects b E {0, If O, the challenger samples challenge task -- from Q uniformly at random, such that F" & Otherwise, the challenger samples
7" from F uniformly random.
4. The challenger sends a batch of samples, X", drawn from the challenge task, T"
5. The adversary, using the batch of samples and black-box access to ha, guesses a bit t- A(ho (X,-)).
6. The adversary wins if b = b and loses otherwise.
A key aspect of this security game is that the adversary only requires samples drawn from the challenge task, rather than the specific training samples from D. We claim that, even with fresh, unseen data from a challenge task, task-inference adversary can successfully infer task's inclusion in multitask training. We use the following terms to define this distinction: A task-inference adversary is strong if they have access training samples when b 1 weak if they do not have access to training samples when b =
Our proposed threat model is general and interpolates several existing attack types depending on the contextual definition of tasks. When tasks correspond to unique users in the dataset, task-inference leaks the same information as a user-inference (Kandpal al. 2023) attack. If further specify that each user has at most one training example, our threat model reduces to membership-inferee When tasks are defined by labeling or learning problem, task-inference mirrors property or dataset-
inference (Chaudhari et al. 2022. Hartmann et al., 2023 Maini et al. 2024).
We note that our instantiation of the task-inference adversary operates in a purely black-box manner. In contrast to most existing work on membership and property-inference (Shokri et al., 2016 Carlini ct al. 2022 Mahloujifar et al. 2022 Liu et al., 2021bb Abascal et al., 2023 Chen et al. 2023), our adversary does not require metaclassifien or shadow models to mount their attack. The power of our approach comes from receiving a set of correlated samples from the challenge task, rather than just single sample. By composing test statistic multiple embeddings, the membership signal is amplified, allowing the adversary to distinguish between IN and OUT tasks without additional information.
4

---

Published as a conference paper at ICLR 2026
4 ANALYZING MISK-INFEEENCE
To motivate further exploration of attacks on deep learning models, we present simplified analog of multitask learning via mean estimation. A more detailed treatment of the problem and proofs of our theorems in this section can be found in Appendix B.2
Let M be the set of task means {#1..... pT) which are sampled i..d. from ( where is the "true" mean that we would like to estimate. For each "task" 14, we can sample N < 7 vectors i.i.d. from (M, Ta and store them in the set X; ( ). Since no individual dataset X, would yield an accurate estimate of jí we can instead compute the sample "multitask" mean using all of the X,'s
We note that ji is Gaussian with expectation E [+] f and covariance Cov (Ø) T N-T p.X
7
Ei=
1== Xi.)
La
We construct an adversary. based on prior work (Dwork et al. 2015), with similar variants to the strong and weak adversaries detailed in Section We consider challenger C. who releases the statistic á and an adversary AA who wants to learn whether given task 14, was included in the dataset that was used to compute jr.
C flips coin {0, 1} and sends A a challenge set of k N samples from training task μIN E M (if b = 1) or a new, unseen task POUT ~ ( Ia )) (if b = 0)
2. A takes the sample mean of their samples pB = ExeB
3. A computes the test statistic (p fi PB p.
4. A picks a threshold y and returns 6 1,27
This attack adapts test statistic originally intended for membership-inffrencc which measures the correlation between the released statistic and the adversary's 's samples. Unlike the membership- inference setting, the adversary is attempting to detect traces of the task, or data's 's distribution, Hi+ rather than any particular sample itself. the following theorems, we find that the inclusion of task, [4g can be inferred by both the strong and weak adversaries, and the strong adversary gets slight advantage from having access to samples in X4,
Theorem 4.1 (Strong Adversary). Let T be the index of the target task and suppose that the challenger sends the adversary set of l samples such that, when the task is IN, the k samples are drawn uniformly at random from X,. Then, when μt is OUT and IN, respectively, we have
d T
o²
E [ouul] = and P. x
[an]] P.X
+
N
Theorem 4.1 shows that the strong adversary's 's test statistic, 2, grows in the dimension of the data divided by the total number of tasks used learn the multitask mean, jr. This result demonstrates that detecting traces of tasks can be seen number of tasks = 1, sampling N datapoints, X, only from the single task to estimate the task mean, and allowing the adversary to use k = sample reduces 2IN to the membership cembbrrshp-inffeennce test
|sooss
generalization of membership inference, as setting the
statistic, which has expectation e d. N In our setting, the test statistic 2 of an adversary who wants to trace a single example would be O NT).
Theorem 4.2 (Weak Adversary). Let T be the index of the target task and suppose the challenger sends the adversary set of le samples such that, when the task is IN, the k samples are drawn i.i.d. from the same distribution as X,,NN,, NN,,, III).. Then the expected value of the statistic, 2, when pr is OUT is O. When μt is IN: EE [=IN] = 402
X.
Informally, Theorem 4.2 shows that the expectation of the weak adversary's test statistic depends only the number of tasks, and does not benefit from smaller number of total training samples.
Theorem 4.3 (Variance of 2 (Informal)). The variance of 2 when μ4 is IN or OUT is
Var (ZIN) R Var (ZoUT) = d _ 0t 8.X
kk k N
(0202)
a.X
T
KN
KN
5

---

Published as a conference paper at ICLR 2026
We note that the strong adversary always achieves greater attack success rates than the weak adversary because the distance between the expected value of the test statistic, 2, when the task is IN or OUT is strictly larger than that of the weak adversary at similar level of variance. Furthermore, this attack on mean estimation shows that the adversary's success is dependent on two distinct parts: (1) the knowledge they receive from having member data (i.e. the TN have about the distributions, or tasks, that compose the dataset (i.e. the 40² term).
d o² term) and (2) the knowledge they
5 METHODOLOGY
In this section, we realize the threat model in Section 3 for machine learning models by introducing two efficient, purely black box attacks on shared representations.
5.1 OUR
ATTACKS
Our two simple and efficient black-bos box attacks are inspired by prior work on membershiip-inferenee, which shows that embeddings of training samples are more robust to augmentations (e.g., random rotations) than non-members (Liu et al. 2021a). Our key insight is to generalize this principle by treating distinct samples from the same task as "natural" augmentations one another. We therefore hypothesize that their embeddings will exhibit strong codependencies. We argue this is caused by distribution-lewe memorization, where the shared representation "overfits" implicitly learned properties of entire task distributions, irrespective of the training data's labeling in MTI,, in manner analogous sample-level memorization.
Coordinate- Wise Variance Attack (Algorithm 1 To mount the coordinate-wise variance attack, batch of le data samples from given task,
the adversary first queries the shared representation then aggregates the embeddings into a set E = (11())), ...................... fig (xx)))) Then, the adversary computes the empirical covariance matrix of E and takes the trace divided by the dimension of the embedding vectors to be the task-inferee ies--inference statistic, 2. This is equivalent to computing the sum of coordinate-wise variances of the embeddings. Lastly, the adversary sets threshold y such that any task with 7 is labeled as /N, and any task with < is labeled as OUT. The full attack algorithm can be found in Appendix A.]
again use the fact that the shared representation produces close embeddings for data from the same task, but we measure similarity of the entire embedding vectors rather their individual coordinates by taking their inner products (or cosine similarities). As in the previous attack, the adversary first queries the shared representation and aggregates the embeddings E. Then, for each unique pair of data samples (2i, Tty) + j. the adversary computes the absolute value of the inner product of their embeddings, z, and stores the value in a set S. The adversary then takes the mean of the z's (5) and applies threshold y as in the variance attack. The full version of Algorithm can be found in Appendix A.
Pairwise Inner Product Attack (Algorithm 2 For second attack,
2016 Carlini et al. 2022) and property inference (Chaudhari et al. 2022 Mahloujilar et al.) 2022) atlacks that train shadow models, our task-inference adversary requires only query access to the shared representation, eliminating the need for shadow, or reference, models to calibrate the attack. While slightly stronger adversary could construct labeled, auxiliary dataset of known OUT tasks to calibrate their attack and perform a more powerful exact one-sided test, as in (Carlini et al. (022), we find that simple thresholding is sufficient to achieve high success rates while maintaining the purely black-box aspect of the threat model. Thus, using the adversary's sampling access to task data and query access to the shared representation, we attempt to reduce the noise in embedding vectors by applying a whitening transformation (A.7.I.
Normalizing Embeddings Unlike prior work on membership-infereen (Shokri
EVALUATION
We present a thorough evaluation of our efficient, black-box task- inferrnneee attacks on MTL models across the vision and language domains. We study the performance for both strong and weak

---

Published as a conference paper at ICLR 2026
adversaries, and we investigate the sources of fask-inference leakage by analyzing how attack success correlates with the model's 's generalization. We note the practicality of our attacks, requiring as few as four samples and taking roughly 0.1 seconds per black-box query and task-inference prediction single RTX 4090 GPU. Additional details on these experiments are available in Appendix [
6.1 MTL TRAINING
In this study, we consider the original instantiation of MTL training which is described in the seminal work on the topic of MTL (Caruana 1997). We highlight that this MTL algorithm a slight variation of centralized FedSGD (McMahan et al. (017), one of the baseline algorithms in the original paper on collaborative learning. For all of our models, we allow all tasks to share all but the task-specific classification heads. In our experiments. the shared representation takes the form of neural network, and the classification heads are linear. During MTL training. we perform a forward pass and route each embedding vector to its corresponding task-specific layer. Thus, in the backward pass, the loss from all tasks is used to train the shared encoder, while only the loss from each particular task is used to train each linear layer. A detailed description of the multitask training setup used in our evaluation, along with the corresponding algorithm (A (Algorithm 3) can be found in Appendix A.3
6.2 MODELS, DATASETS, AND METRICS
In our vision experiments, we use ResNet models (He et al., 2015) of differing sizes as the shared representation for MTL. We evaluate our attacks on these vision models using the CelebA (Liu et al. 2015) and Federated EMNIST (FEMNIST) (Caldas et al. 2019) datasets. A detailed description of our FEMNIST experiments can be found in Appendix A.6| For our language experiments, we use downsized variants of the BERT (Devlin et al. 2018 Ture al. 2019 Bhargava et al. 2021 architecture, which are pretrained on MNLI (Williams et al. additionally evaluate our attack Gemma 3 270M (Team al. 2025) fine-tuned for personalized
20118, for downstream classification. We
next-token prediction on the Reddit dataset (Völske et al. (017).
To measure the performance of task-inference attacks, we use metrics that commonly found in the inference attack literature (Carlini et al. 2222]; Zarifzadeh et al.) 2024 Ye et al. 2022 Kandpal al., (2023.. We report the ROC curves of our attack, along with true positive rates (TPR) at fixed low false positive rates (FPR) and area under the curve (AUC). To highlight the black-box nature of our attack, report the (TPR, FPR) pairs when blindly thresholding our test statisties the 50th, 75th, and 90th percentiles. More details on the models, datasets, and metrics used in this evaluation can be found in Appendices A.2 and and A.4
6.3 PERSONALIZATION
Here, we present the results of our experiments in the vision and language settings where MTL is used for personalization. Across all experiments, the datasets are organized by individual users, with each user contributing multiple samples. In the MTL framework, the multiple tasks are the users, and each user's task-specific layer, which is linear in this case, receives updates from their data. The shared representation is trained on all of the tasks. When the adversary infers a task's 's inclusion in multitask training, they are leaking whether or not an individual's data was present at all in updates to the shared representation. Like in -infbrrship-iffeeneeee the strong adversary will have a batch of real training samples at their disposal to perform this test, but the weak adversary receives fresh samples that helong to the user and were never seen during training. Additional details for all of our experiments, along with additional results for the FEMNIST dataset, can be found in Appendix A.5.1
CelebA First, we report the results for our attacks in the vision setting. We train ResNet-34 (He al. 2015)) model, pre-trained ImageNet (Deng et al. 2009), to perform binary facial attribute detection on the CelebA dataset. In this MTL setup, each task corresponds to a unique person, and we train separate linear layer for each of the 512 total tasks (256 IN, 256 OUT).
The results of this experiment are shown in Figure la and Table I We find that while both the strong and weak adversaries can achieve noivtrivial success rates in determining task's inclusion in training, the true positive rate of the weak adversary is bounded above by the true positive rate of the strong adversary for any fixed FPR. Moreover, we that our variance attack nets better TPR in the
7

---

Published as a conference paper at ICLR 2026
42-
Groop AMC- 822 Wesiene dooa
the enrerttiiaa (aicnn Brse Fiiklt ozorgi pl as53
loaer Product De AAREO
ALC-2244
92
8.
0.2
24
04
C.6
(a) CelebA
(b) FEMNIST
42
Walin ErSigi KGL LH44
rster ceertodac1RIne HNC- 0334
03.0
02
(c) Stack Overflow (Classification)
(d) Reddit (Generation)
Figure 1: ROC curves of our Task-Inference Attacks (MTL for Personalization)
low FPR regime, but the inner product attack sees better FPR at higher FPR. At fixed FPR of 1%, the TPR of our variance attack on CelebA achieves a TPR of 61.2% and 2.9% for the strong and weak adversaries, respectively.
Stack Overflow Next, we present the results for our language experiments on the StackOverflow dataset. We train BERT Small (Tuee al. 2019) model perform topic classification, where the tasks are users who contributed posts to Stack Overflow, with 256 tasks total (128 IN, 128 OUT).
Figure Icland and Table show the results for this attack. We find that the strong adversary achieves nearly perfect AUC for both attacks and an empirically-observed FPR of 0% at the 75th and 90th percentile thresholds over all runs of the attack. The weak adversary again sees nontrivial AUC for both attacks, and we see tradeoff between the TPR of the variance and inner product attacks at particularly high FPR. At fixed FPR of 1% the inner product adversary with weak access to the data achieves TPR of 8.2%, while the strong adversary achieves TPR of 98.5%. By analyzing the topic frequencies in the dataset, we empirically find that individual users tend to only write about a small subset of topics. Across all posts, the median posts about 31 (or 12. 1%) of the 256 total topics, which could lead to high distinguishability of users in representation space.
Reddit Lastly, we present the results for our language experiments on the Reddit dataset (Völske et al. 2017). We fine-tune a pretrained Gemma (Team et al. (2025) model for next-token prediction, where the 256 IN tasks are Reddit users with 64 to 512 posts each, and we evaluate our attack over 512 total tasks (256 IN, 256 OUT). To personalize the model, we train a task- LoRA Hu et al. (2022) adapter on the language modeling head for each user, while all tasks jointly update the rest of the parameters.
Figure and and Table I show the results for this experiment. We find that both the strong and weak adversaries achieve high attack AUC with both the inner product and variance attacks. At fixed FPR of 1%, the variance attack achieves a TPR of 19.7% and 19.0% for the strong and weak adversaries, respectively. We observe that the coordinate- wise variance attack consistently outperforms the inner product attack in this setting because (1) the low-rank adapters used for personalization induce a spread in the embedding space for trained users and (2) the learned embeddings not intended for classification, thus they do not exhibit the same clustering properties we observed in our experiments on discriminative models.
8

---

Published as a conference paper at ICLR 2026
6.4 MULTIPLE LEARNING PROBLEMS
We present the results of our experiments models trained using MTL to solve multiple related learning problems. contrast to the datasets used to train MTL models in the experiments presented from Section 6.3 the datasets are split by learnable classes in the dataset rather than by person. For example, Stack Overflow can be split into its constituent topics, and we can have an MTTL model with tailored head for detecting each individual topic's presence. We draw attention the fact that in this setting, the labels for the learning problem are directly tied to the task. In other words, if a task is included, there are no positive labels corresponding that task in the training dataset. Additional details for the experiments in this section can be found in Appendix A.5.2
CelebA We the ResNet50 (He al., 2013) architecture the backbone for our MTL model trained on CelebA as there is ample data per task. Each of the 20 IN tasks (40 total) has corre- sponding linear head for binary attribute prediction, and no two tasks share the same labeling. For example, the task head dedicated to hair color does not make predictions for the task head dedicated to detecting glasses.
In Table 2 and Figure 2 we see that the strong adversary is able to achieve an AUC of 0.745 using our coordinate-wise variance attack and a TPR 22.8% at FPR of 0% over all runs when using the 90th percentile threshold. The weak sees nontrivial success rates, with TPR roughly 2x larger than the FPR when using the 90th percentile threshold.
Stack Overflow In this experiment, each of the 140 total tasks corresponds to the inclusion of a positively labeled topic in the dataset. For example, if the Stack Overflow topic "Python" is IN, there is a task head dedicated detecting whether post includes the topic "Python". Using a BERT Medium Ture et al. 2019) (41M parameters) model as the shared representation, we use MTL to learn linear task heads that specialized to detecting the presence of single topic within post.
02-
624
frier Prbkiliiing 8
medeen (iring (4C--ar1
02
24
Do
24 64
CA
(a) CelebA
(b) Stack Overflow
Figure 2: ROC of fask-Inference Attacks (MTL for Multiple Learning Problems)
Figure 2b shows that in the setting where tasks correspond directly to the training data labeling in MTL, the gap in attack performance between the strong and weak adversaries becomes small. This can be attributed to the fact that the model necessarily has to achieve high utility on the training tasks in order to solve the learning problems. This contrasts our findings in Section [6.3] where the task heads in MTL are not solving distinct learning problems, and the separation between both adversaries notably larger. In Table 2 we see that the (TPR, FPR) pairs for both adversaries are nearly equal, and the weak adversary is able to achieve 19.6% TPR at 0.2% FPR.
6.5 INVESTIGATING SOURCES OF TASK- INFERNNFEE LEAKAGE
We investigate the factors that lead to task- leakage by studying how attack success varies with the model's generalization gap. Theorem 4.1 shows an advantage for the strong adversary, and we observe the gap between strong and weak adversaries is large in our MTL experiments, especially with sparse tasks like in CelebA. Similar to memmership-inferemn is closely tied to the model's generalization gap. To explore this, measured the inner product attack's 's AUC on Stack Overflow models (trained for personalization as in Section 6.3 at different stages of training.
lusk-inference success

---

Published as a conference paper at ICLR 2026
290
(a) Strong and Weak (Personalization)
(b) Weak (Multiple Learning Problems)
Figure 3: Relationship between Generalization Gaps and Inner Product Attack AUC Stack Overflow
As shown in Figure 3a the strong adversary's s AUC rapidly increases as the generalization gap, or difference between training loss and a validation loss on OUT task data, grows. This finding where leakage is linked to overfitting particular training
mirrors traditional bemmershhp-inference examples. In contrast, the weak adversary's AUC grows with different quantity: the gap between the loss unseen samples from IN lasks and the validation loss OUT samples. This suggests the shared representation is not only memorizing training points but also generalizing more effectively on the training tasks. The observation that models memorize information about tasks as a whole is consistent with findings from prior work on user-inference attacks on LLMs (Kandpal et al. 2023
7 CONCLUSION AND DISCUSSION
In this work, we study privacy leakage when learning models over mixture of tasks in multitask learning. In particular, we focus on the setting where MTL is used to jointly train model for many tasks at once by learning a shared representation that captures common features between tasks. We propose a novel, purely black-box task- inss-inference threat model, where the adversary's 's goal is to infer the inclusion of target task in training given only query access to the smallest shared component when learning jointly over many tasks, the shared representation. By analyzing our task-inference threat model in the context of tracing attacks Gaussian mean estimation, between strong adversaries with access to training samples and weak adversaries with access to fresh samples from the target task's distribution that did not appear the training dataset. To verify our analysis, we propose purely black-box attacks on machine learning models trained with MTL and we conduct extensive experimentation in the vision and language domains for multiple use cases of MTL. We find that our attacks can consistently achieve non-trivial success rates in terms of AUC and TPR, even when the adversary has no reference knowledge and chooses thresholds based on percentiles. Additionally, we attempt to understand the factors that lead to task-inference leakage by measuring how attack AUC varies for both the strong and weak adversaries as a function of the model's 's generalization gap.
find separation
8 ACKNOWLEDGMENTS
We are grateful the ICLR reviewers for useful feedback and comments. Adam Smith was supported at BU by US NSF award CNS-2232694. Part of this work was done while he was a visiting scientist Google Deepmind. John Abascal and Alina Oprea were supported Northeastern by US NSF award NNS-2244488 and by grant from Coefficient Giving. Jonathan Ullman was supported at Northeastern by US NSF awards CNS-2232694 and CNS-2247484.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep leaming with differential privacy. In Proceedings of the 2016 ACM SIGSAC
10

---

Published as a conference paper at ICLR 2026
Conference Computer and Communications Security, ACM, oct 2016. doi: 10.1145/2976749. 2978318. URL https: http://i..........//o 2190o.0 2997831
John Abascal, Stanley Wu, Alina Oprea, and Jonathan Ullman. Tmi! finetuned models leak private information from their pretraining data. Proc. Priv. Enhancing Technol., 2024:202-231 2023.
URL http::: //app:::///.... sewant htt.://a......ooooooooa/0222211.
Sushant Agarwal, Gautam Kamath, Mahbod Majid, Argyris Mouzakis, Rose Silver, and Jonathan Ull- man. Private Mean Estimation with Person-Level Differential Privacy, pp. 2819-2880. 2025. doi: 01137/1.978166 1978322.92. URL BRLhhttss://eubba siam. wwt.:oogg//oooooooo wttp:///aass/....1110.. 97816119 029661198822.1 92
Issa Annamoradnejad, Jafar Habibi, and Mohammadamin Fazli. Multi-view approach to suggest moderation actions community question answering sites. Information Sciences, 100:144- 154, 2022. ISSN 0020-0255. doi: http://///.aaa.......///////////
URL https
//w..sciennedirectt wwm.coo//c/////oe
Giuseppe Ateniese, Luigi Mancini, Angelo Spognardi, Antonio Villani, Domenico Vitali, and Giovanni Felici. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. Int. J. Secur. Nenw, 10(3):37--150 sep 2015. ISSN 1747-8405, doi: HTt.:/ISSN..2.11.. 071829
00.1504/USN.2055.071222
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine- learning practice and the classical biri-variance trade-off. Proceedings of the National Academy of Sciences, 156932) jul 2019. doi: htt033/paa.19037011110 URL https
/ 190.00000
Prajjwal Bhargava, Aleksandr Drozd, and Anna Rogers. Generalization in nli: Ways (not) to go beyond simple heuristics, 2021.
Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečný, H. Brendan McMahan, Virginia Smith, and Ameet Talwalkar, Leaf: benchmark for federated settings, 2019.
URL
org/ abs /1812 01097
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. Mem- bership inference attacks from first principles. 43rd IEEE Symposium on Security and Privacy, SP 2022, San Francisco, CA, USA, May 22-26, 2022, pp. 1897-1914. IEER, 2022, doi: 10. 1109/ 2022. 9833649
01442114.0022.8833644 URL htttpsttss: //ttttts://oo..eeo .
Rich Caruana. Multitask learning. Machine Learning, 28(1): 2 1997. ISSN 0885-6125. doi: 10.1023/a:1007:3796600 URL /tttt://áx..o wwtp:xxxxooo.... www.ooo......o.. http:://cc....ooooooooo/...../11 /1tp://....000011111177771717171
Zachary Charles, Arun Ganesh, Ryan McKenna, H. Brendan McMahan, Nicole Mitchell, Krishna Pillutla, and Keith Rush. Fine-tuning large language models with user-level differential privacy, 2024. wttlttts:////
Harsh Chaudhari, John Abascal, Alina Oprea, Matthew Jagielski, Florian Tramèr, and Jonathan Ullman. Snap: Efficient extraction of private properties with poisoning. 2022.
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, and Yang Zhang. Face-auditor: Data diting in facial recognition systems, 2023. URL
Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks, 2018. URL https
02257
Lynn Chua, Badih Ghazi, Yangsibo Huang, Pritish Kamath, Ravi Kumar, Daogao Liu, Pasin Ma- nurangsi, Amer Sinha, and Chiyuan Zhang. Mind the privacy unit! user-level differential pri- vacy for language model tmn-tuunig. In First Conference on Language Modeling, 2024. URL net f
htttps://ppentt.

---

Published as a conference paper at ICLR 2026
Ronan Collobert and Jason Weston. A unified architecture for natural language processing: deep neural networks with multitask learning. In Proceedings of the 25th International Conference Machine Learning. ICML '08, pp. 160-167, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 9781 9781605582054 doi: 10. 10.1145/13901 1390177 URL https
7/doi. 1111/11911111111 1391177
Rachel Cummings, Vitaly Feldman, Audra McMillan, and Kunal Talwar. Mean estimation with user-level privacy under data heterogeneity. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 978171 1781713871088.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference Computer Vision and Pattern Recognition, pp. 248-255, 2009. doi: 10. 200.//CPR..009.522222
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lec, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Shai Halevi and Tal Rabin (eds.), Theory of Cryptography, pp. 265-284, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-32732-5.
Cynthia Dwork, Adam Smith, Thomas Steinke, Jonathan Ullman, and Salil Vadhan. Robust trace- ability from trace amounts. 2015 IEEE 56th Annual Symposium on Foundations of Computer Science, pp. 650-669, 2015. doi: Bt/.//////S/.......
Alireza Fallah, Aryan Mokhtari. and Asuman Ozdaglar. Personalized federated learning: A meta- learning approach, 2020. URL
Vitaly Feldman. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, STOC 2020, pp. 954-959, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450369794, doi: 10.1 10.1145/355713..384400
URL httttttp: :/tp::::///do.o. http://ooo.or/....////
3384290
Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 19586-19597, Cur- ran Associates, Inc., 2020. URL /tL//tttss//pppceeeiinnn... www.:iiiinnnnnuuuouppeee cc www.ppercccpppppppppppppppp paper files/ pdf
paper fi / fa2er7/000001112/220000000111310
Joumana Ghosn and Yoshua Bengio. Multi-task learning for stock selection. In M.C. Mozer, M. Jordan, and T. Petsche (eds.), Advances in Neural Information Processing Systems, volume MIT Press, 1996. URL BtL.Atppp://proeeeeeesss.seee neurips
Sacaapr__filee files/paper/ pdf
Ross Girshick. Fast r-enn. In 2015 JEEE International Conference on Computer Vision (ICCV), pp. 1440-1448, 2015. doi: 10.1 10.1109/CC2222055..66
Filip Hanzely and Peter Richtarik. Federated learning of a mixture of global and local models, 2021. URL hnttpss//opeereett
forum d J jctMh
Valentin Hartmann, Leo Meynent, Maxime Peyrard, Dimitrios Dimitriadis, Shruti Tople, and Robert West. Distribution Inference Risks: Identifying and Mitigating Sources Leakage In 2023 Secure and Trustworthy Machine Learning (SaTML), pp. 136-149, Los Alamitos, CA, USA, February 2023. IEBE Computer Society, doi: 10.1009/SATML555555 00018. URL /ttps:://ci..eeee /ttp:////.Ss.ieeeeeeeteeeeeeeees 2023. 00018
IEEE Conference
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
12

---

Published as a conference paper at ICLR 2026
Nils Homer, Szaboles Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling, John Pearson, Dietrich A Stephan, Stanley F Nelson, and David Craig. Resolving individuals contributing trace amounts of DNA to highly complex mixtures using high-density SNP genotyping microarrays. PLoS Genet., 1(8):e000117, August 2008.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. and Weizhu Chen. LoRA: Low-rank adaptation of large language models. International Conference on Learning Representations, 2022. URL httpt::://// http/opnnrvee..neeetee
id=nZeVKeeFyff
Shengyuan Hu, Steven Wu, and Virginia Smith. Private multi-task learning: Formulation and applications to federated learning. Transactions Machine Learning Research, 2023, ISSN
2835-8856. URL htt.aa/s/ppeeeeeeeeeeeeeereeeaeo
Chao Huang, Jianwei Huang, and Xin Liu. Cross-silo federated learning: Challenges and op-
portunities. ArXiv, abs/2206.11944/ 2022. URL
Nikhil Kandpal, Krishna Pillutla, Alina Oprea, Peter Kairouz, Christopher Choquette-Chop and Zheng Xu. User inference attacks on large language models. In International Workshop on Federated Learming in the Age of Foundation Models in Conjunction with NeurlPS 2023, 2023.
URL https: Tttttps://ppereevviii
net
Daniel Levy, Ziteng Sun, Kareem Amin, Satyen Kale, Alex Kulesza, Mehryar Mohri, and Ananda Theertha Suresh. Learning with user-level privacy. In Proceedings of the 35th In- ternational Conference on Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2021. Curran Associates Inc. ISBN 9781713845393.
Hongbin Liu, Jinyuan Jia, Wenjie Qu, and Neil Zhenqiang Gong. Encodermi: Membership inference against pre-trained encoders in contrastive learning. In Proceedings of the 202/ ACM SIGSAC New York, doi: 10.1145/
Conference on Computer and Communications Security, CCS '21, pp.
NY. USA. 2021a. Association for Computing Machinery. ISBN 978 4500844444
14.0449.3484749, URL http::ttt::///t //doi. org/ 11451 /465//44602200 110220.344477 3484749
Hongbin Liu, Jinyuan Jia, Wenjie Qu, and Neil Zhenqiang Gong. Encodermi: Membership inference against pre-trained encoders in contrastive learning- In Proceedings the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 2081-2092 202
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, In International Confer- on Learning Representations, 2017.
Wang Lu and Jindong Wang. Personalizedfl: Personalized federated learning toolkit. 2022.
https
Saeed Mahloujifar, Esha Ghosh, and Melissa Chase. Property inference from poisoning. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1120-1 137, 2022. doi: 10. 109//////44,.4
2022.9833623.
Pratyush Maini, Hengrui Jia, Nicolas Papernot, and Adam Dziedzic. Lim dataset inference: Did you train on my dataset?, 06 2024.
Yishay Mansour, Mehryar Mohri, Jae Ro, and Ananda Theertha Suresh. Three approaches for personalization with applications federated learning, 2020. URL htttps://arxxv..or abs/ abs/2002. 10619
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication Efficcientfiiittttte Learning of Deep Networks from Decentralized Data. In Aarti Singh and Jerry Zhu (eds.), Proceedings of the 20th International Conference Artificial Intelligence and Staristics, volume 54 of Proceedings of Machine Learning Research, pp. 1273-1288 PMLR, 20-22 Apr 2017. WRI.tttss:t... http:ss///rccceeeeee ngs m www.mmlrrr.ss....mmmmmmmmmmmmmle www.s/rsss...o.... v54 http///maaaaannnammcmmmaammaa html
13

---

Published as a conference paper at ICLR 2026
John Nguyen, Jianyu Wang, Kshitiz Malik, Maziar Sanjabi, and Michael Rabbat. Where to begin? on the impact of pre-training and initialization in federated learning, 2023. URL htt.:hhtts//aatt //rxivv org/ 2206.0662222020222222220 15387
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-perfooe perforrmace deep learning library, 2019. URL htt.tttp:////. arxiv. org/ abs 1912 01703
Bharath Ramsundar, Steven M. Kearnes, Patrick F. Riley, Dale Webster, David Konerding, and Vijay S. Pande. Massively multitask networks for drug discovery. ArXiv, 10000002.02072 2015. URL stgp:///coosssmmmccccccooooooaa httpp//chh...cccaoooooooooppaaaa
Sriram Sankararaman, Guillaume Obozinski, Michael Jordan, and Eran Halperin. Genomic privacy and limits of individual detection in a pool. Nature genetics, (1(9))665-90) 9677766677777 2009.
Zebang Shen. Jiayuan Ye, Anmin Kang, Hamed Hassani, and Reza Shokri. Share your representation only: Guaranteed improvement of the privacy-utiligg tradeoff in federated learning. In The Eleventh
International Conference on Learning Representations, 2023. URL httpss:///p:eeeeeett
net/ www.oooommoooooeeeeeeoooo
Reza Shokri, Marco Stronati, and Vitaly Shmatikov. Membership inference attacks machine learning models. CoRR, abs/1610.05820, 2016. URL http:://aa....o///aaaa.. 05820
Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. Federated multi-task learning. In I. Guyon, U. Von Luxburg. S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ings www.reuurii... http:pp.ccccpppe cc/paper. fi files/papes
ciates, Inc., 2017. URL
Congzheng Song and Ananth Raghunathan. Information leakage in embedding models. In Pro- ceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, CCS '20, pp. 377-390, New York, NY. USA, 2020. Association for Computing Machinery, ISBN 970445038889, doi: 10.1145/3372297.3412222 /tttttss:://aoi. /00p://///000..00011111115 0852299.3317221
Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, 34(12):9587-9603 2023. doi: 10.1 2.0.TN 60699.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot. Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, Iain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan- Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky. Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, Christopher A. Choquette-Choo CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimczak- Pluciíska, Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadic, Jetha Chan, Joe Stanton, John Wieting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho,
14

---

Published as a conference paper at ICLR 2026
Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan, Noveen Sachdeva, Oskar Bunyan. Pankil Botarda, Paul Caron, Paul Kishan Rubenstein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Pöder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger, Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberry, Vlad Feinberg. Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean- -antttet Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma 3 technical report, 2025.
wttp://rg/abb/......323 19786
Tulia Ture, Ming-Was Chang, Kenton Lee, and Kristina Toutanova. Well-read students learn better: On the importance of pre-training compact models, 2019.
/bs/
1908. 08962
Michael Völske, Martin Potthast, Shahbaz Syed, and Benno Stein. TL;DR: Mining Reddit to learn
automatic summarization. In Proceedings of the Workshop on New Frontiers in Summarization, pp. 59-63, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10. 10.18653/1//117-55000 URLhttts: www. ac weeb...e.... org/ httpa/aatoogg//---550500
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technolo- gies, Volume (Long Papers), pp. 1112-1122, Association for Computational Linguistics, 2018. URL http: /ttp:://ccweess www.awwwebb... org/ http://ahhoooooooooooo
H. Yan, A. Yan, L. Hu, J. Liang, and H. Hu. Mtl-leak: Privacy risk assessment in multi-tasi learning. IEEE Transactions Dependable and Secure Computing, 21(01):204-21) jan 2024. ISSN 1941-0018. doi: 10. T0.100//DDSC22223333277835
Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, and Reza Shokri. En- hanced membership inference attacks against machine learning models. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, CCS '22, pp. 3093-3106 New York, NY, USA, 2022. Association for Computing Machinery. ISBN 978 1781445394 doi: 10.1 00.144//5548663356000
htt.:/i.orr/..1111111111 1145 30055/5546605 114 3560675
Tao Yu, Eugene Bagdasaryan, and Vitaly Shmatikov. Salvaging federated learning by local adaptation, 2022.
04758
Sajjad Zarifzadeh, Philippe Liu, and Reza Shokri. Low-cost high- power membership inference attacks, 2024. URL httph::tsssssiiiieeee wttp://wrggaaab/....ooom
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understand- ing deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017.
A ADDITIONAL RESULTS AND DETAILS FROM METHODOLOGY AND EVALUATION
In this section, we provide additional details, figures, and results for Sections 5 and 6
15

---

Published as a conference paper at ICLR 2026
A.1 ATTACK ALGORIHTM
We provide full versions of the attack algorithms described in Section 5.1
Algorithm 1 Coordinate- Wise Variance Attack
1: Input: A shared representation hag a challenge set X,
2: E = 0
3: for I; E X do
4: 6 f- (z())) and E E fe.)
Query rep. on challenge set
5: Compute Q, the empirical covariance matrix of E
6: return tr(Q)
Return avg. coordinate-wise variance
Algorithm 2 Pairwise Inner Product Attack
Input: A shared representation hg, challenge set X
2: == f}
3: for I; X, do
4:
ej f- ha(x;)
Query rep. on challenge set D Apply whitening Store normalized embedding
5:
eq f W(e - 8)
6:
E+ Eu,fee
7: S =
8: for all unique pairs (ei, ej); i + j; ei, do
9:
if uuss_cosine_sin aarityy then
10:
(e), |el Te.
11: = ey)l and + s (2}
12:
return 5
Return avg. pairwise inner product
A.2
MODELS AND DATASETS
A.2.1 VISTON MODELS
In our vision experiments, we ResNet models He al. (2015) of differing sizes the shared representation for MTL. The ResNet architecture has gained widespread adoption across computer vision applications due its computational efficiency and high performance on variety of datasets. This model architecture makes use of residual connections, which stabilize training and convergence to help maintain high utility when trained on large image datasets. When available, such in our CelebA experiments, we use larger ResNet models which are pretrained on the ImageNer Deng et al. (2009) dataset from the PyTorch Paszke et al. (2019) library.
A.2.2 LANGUAGE MODELS
For our language experiments, we use the BERT Devlin et al. (2018) architecture as the shared representation. In particular, we use the downsized variants of BERTTURE et al. (2019); [Bhargava et al. (202T) which are pretrained on MNLI Williams et al. (2018) for downstream sequence classification.
A.2.3 VISION DATASETS
We evaluate our rask-inference attacks on both the CelebA Liu al. (0155) faces dataset and the Federated EMNIST (FEMNIST) Caldas et al. (2019) handwritten character and images dataset. CelebA contains high-resolution images celebrities, each with a unique identifier, and 40 facial attributes with binary labels. The FEMNIST dataset contains 28x28 grayscale images of 62 different types of handwritten characters. In our experiments on CelebA, we split the dataset into tasks by person and by facial attribute for each of the two MTL cases consider. We split FEMNIST by writer and train an MTL model for character recognition, personalized to each writer's hand-drawn images.
16

---

Published as a conference paper at ICLR 2026
A.2.4 LANGUAGE DATASETS
To evaluate our attack in the language setting, we use the Stack Overflow Annamoradnejad et al. (2022) dataset, which consists of posts from the Stack Overflow website with corresponding ratings, user ID, and topic tags for each post. In our evaluation, we split the posts into tasks by user ID and by topic for each of the MTL use cases.
A.3 MULTITASK TRAINING
Algorithm| 3 shows the general training loop we use for MTL. In practice, we use Adam W Loshchiloy & Hutter (017) for our Update step, and perform shuffling tasks and batching for efficiency. Additionally, when there sufficient data available per task, we randomly subsample data from each task for each training epoch.
Algorithm 3 Multitask Learning Training Loop
1: Input: A shared representation hg. T task-specific layers fga,..... gär J. a training set split by
task D {(X1, (Xr, 3r)], number of training rounds t, loss function L
2: for E [t] do
3:
EMTTI <-
Initialize multitask loss
4: for j E [T] do
5:
Pj - ga, hog Xj )}
> Get predictions for task
6:
EMTI. t LMTT. + L(pp, Vz)
7:
for E [T] do
Update each task specific layer
8:
3j 4 Update (ßj, Vg,
9: + Update (9, Vol)
b Update shared representation
return ho
Since the number of samples per task can be smaller than the embedding dimension, we use several regularization techniques keep the task-specific linear layers from overfitting to noisy embeddings early during MTL training. To summarize, we use large values for weight decay, or Le regularization, learn a bottleneck on the shared representation to decrease the embedding dimension, normalize gradients Chhiients/Chhnnnaee et al. (2018) across tasks during model updates, apply clipping to gradients, and, in our vision experiments, apply standard augmentations to training images.
In our vision experiments, the CelebA dataset has very few samples per task-specific layer; 22 per celebrity after holding out samples for the weak adversary. Thus, we do two iterations of "warm start" training Nguyen al. (2023). where we freeze the shared representation and optimize the task-specific layers for 40 epochs with with low learning rate (c.g. 77 = 10 4). In our language experiments where linear projection on the typical, high-dimensional penultimate layer of our BERT and ResNet models to 16 or 32 dimensions in order reduce the chance of task task-speeific layers overfitting. Each training step in MTL is performed over a minibatch of tasks, rather than the entire accumulated gradient of the dataset. We additionally use gradient clipping (C 1), gradient normalization across tasks Chen et al. 2018), and smaller weight decay parameter (e.g. À = 10 than the task-specific layers (e.g. 10 "" regularize the shared layers. In total, we train for 200-300 epochs, or communication rounds, for all of our models, passing through all of the tasks in the dataset each time.
BERT models, do 20 epochs of warm start training. We also apply
A.4 METRICS
Throughout our evaluation, we analyze the ROC curves of our attack, which measure the relationship between true positive rate (TPR) and false positive rate (FPR). As summary statistics, we report the area under the ROC curve, or AUC, as well as the TPR at fixed low FPR (e.g. 1%). Our evaluation distinguishes between two adversarial settings: strong adversary with access to training samples and a weak adversary limited to auxiliary data from the challenge task that was never seen during training. Thus, present ROC curves for each adversary. Additionally, can be performed in a purely black-bar box setting. This results the adversary not necessarily having sufficient knowledge select the threshold that yields an optimal tradeoff between TPR and FPR.
highlight that our attacks
17

---

Published as a conference paper at ICLR 2026
So, also report the (TPR, FPR) pairs when thresholding test statistics at the 50th, 75th, and 90th percentiles.
A.5 EXPERIMENTAL SETUP
Here, we provide details for the experimental setup for both of the MTL use cases we study, In each of our experiments, run. In each trial, we subsample the task-specific data such that the adversary has a fresh combination of few samples from each of the target tasks.
aggregate results over several training runs and many attack trials per training
A.5.1 PERSONALIZATION
CelebA We filter CelebA such that each unique individual has roughly 30 images containing their face, and we hold out 8 samples per individual to mount an attack with the weak adversary. In our experiments, we jointly train the MTL model on 256 total tasks with 22 samples per task, or roughly 5600 total samples, and thus use several regularization techniques to ensure that the ResNet layers learn performant embeddings. During each training step, we randomly sample batch of tasks and update their corresponding task heads, while aggregating these tasks' gradients to update the shared representation. Once the model trained, we run 128 trials of each attack on all 5 512 tasks (256 IN and 256 OUT), using 8 samples per task for the strong adversary and 4 of the held out samples per task for the weak adversary.
Stack Overflow We filter the dataset such that each individual has at least 48 total posts; 32 posts for the model's s training set and 16 to be used by the weak adversary. Then, we split the dataset into 128 IN tasks and 128 OUT tasks, yielding two datasets of roughly 10k posts, with each user contributing about 140 posts on average. We train the shared representation and task-specific linear layers jointly the IN dataset. The MTL model is trained detect the presence of 256 unique topics in posts, and each post can have multiple corresponding topics. After training is complete, we run each attack on all 256 tasks for 128 trials, using 8 samples and 4 samples per trial for the strong and weak adversary, respectively.
Reddit We filter the Reddit dataset such that each unique user (or task) has at least 64 posts and at most 512 posts. We set aside 16 posts per user for the weak adversary. Then, we split the users into 256 IN tasks and 256 OUT tasks. We start with a pretrained Gemma 270M model and jointly train shared representation and task-specific LoRA adapters (with 16 and 32) for next-token prediction. After each training run is complete, we run each attack on all 512 tasks for 256 trials, subsampling 16 samples and 8 samples per trial for the strong and weak adversary, respectively.
A.5.2 MULTIPLE LEARNING PROBLEMS
CelebA For our evaluation on CelebA when MTL is used solve multiple learning problems simultaneously, we split the dataset into potentially overlapping tasks by facial attribute, 20 IN and 20 OUT, with at least 1024 samples with corresponding positive and negative labelings for each task. Because some tasks have very low positive label frequencies (e.g. < 1 1%), this minimum sample size ensures sufficient positive examples for the task head learn from. We average the results of our experiments over 8 MTL runs, and run the attacks on the 40 total tasks 128 times each, using 16 samples and 8 samples for the strong and weak adversary, respectively.
Stack Overflow In these experiments on Stack Overflow, in contract to the experiments in Sec- tion 6.3] the posts are not split by user. Rather, each of the 140 total tasks corresponds to the inclusion of positively labeled topic in the dataset. If particular topic is not included, there is no task head that can learn positively labeled "Python" samples. Additionally, because the posts in the Stack Overflow dataset can contain multiple topics, the training data is not disjoint between tasks; the labeling of each task dataset is unique. We randomly split the data into 70 IN tasks and 70 OUT tasks, with each task specifcc dataset containing 1024 posts. We run both of our attacks the BERT model 128 times, using 32 samples and 16 samples each trial for the strong and weak adversaries, respectively.
18

---

Published as a conference paper at ICLR 2026
Table (TPR, FPR) and Balanced Accuracy for Black-Box Percentile Thresholds for Inner Product Attack at 50th 75th, and 90th Percentile Thresholds (Personalization)
5pth Percentle
75th Perventile
90 Pencentile
Dotoset
Access
(IPR, HPK)
Acc.
(TPR, FPK)
Acc.
(IPR, FPK)
Acc.
CelebA
Sirong
Weak
(62.40% 47. (7%)
(805. 20%5
50%
(16. T5, 3,355
24.696
52.55%
71.7%
(19.5%, 0.5%%
(11.0%% 8.0%%
61.00%
50.5%
FEMNIST (Fig
Weak
(61.15,
(68.2%,. 48.8%)
61.1%
58.2%
(33.55
22.854)
52.22%
58.7%
(14.5%, 5.
(11.2%%
56.95
61.2%
Stack Overllow Fi
10
Strong
Weak
(98. 78. 1.2%)
41.77)
58.299
7%
(34.156, 15.830)
(50 0%, 0%)
75.07%
50.196
(16.3%, 3.6%))
110.9%% 095)
56.35
60.0%
Reddit (Pig id
Strong
Weak
(68.7%. 31 5%
(62.636.372 37.4%)
62.67%
68 7%
(30. 15, 10.9%8]
(35.636, 14.076)
60.5%
64.1%
(14.8%, 6 6.25%
(14.1%, 6.9%)
54.8%
54.156
Table 2: (TPR, FPR) and Balanced Accuracy for Black-Box Percentile Thresholds for Inner Product Attack at 50th h5th and 90th Percentile Thresholds (Multiple Learning Problems)
50th Percentlle (IFR. FPR)
75th Percentile
goye Persentile
Dataset
Access
Acc.
(TPR, FFR)
Acc.
CIPR, FPR)
Ace.
CelebA (F
Strong
Weak
47.55)
(68. 995, 31.15)
47.556
68 47%
52.5%
(42 6%% 7.4%) 05, 21.05)
67.6%
545
(13.5%,6.6%
[22.895, (95)
50.5%
61.45
Stuck Overflow (Fig
Strong
Weak
(85.95,,14.155
(875,, 13.9%)
87%
85.9%
(47. 2.4,6%
(48. 3%, 1.758
72.6%
73.5%
(19.6%,0:25%
(19.9%% 0.19))
50.7%
50.0%6
A.6 FEMNIST RESULTS
We report additional results in the vision setting for Federated EMNIST Caldas et al. (2019). We train a MTL model with ResNet8 He et al. (2015) as the shared representation and personalize each spec-specific layer to unique writer in the dataset. Each writer in the FEMNIST has significantly more data than each individual in CelebA. Thus, we train the MTL model on 128 tasks with 128 samples per task and hold out 16 samples to mount our attack with the weak adversary. For each of the 128 runs of the attack, the strong adversary receives 16 training samples, and the weak adversary receives 8 of the held out samples. Figure Tbb and Table T show the ROC curves and (TPR, FPR) pairs of our task-inference attacks on Federated EMNIST. We observe that both the strong and weak adversaries are able to achieve nontrivial success rates, and the strong adversary sees TPR of 3% at fixed 1 % FPR. When the weak adversary picks the 90th percentile threshold, the inner product attack yields a 2.5x higher TPR than FPR.
A.7 TABLES FOR PERCENTILE-BASE THRESHOLDS
Here, we present the tables which contain the results of our experiments where we choose the decision threshold for our test statistic using the 50th, t5th and 90th percentile values.
A.7.1 WHITENING TRANSFORMATION
Applying a whitening transformation to the embeddings transforms them into random vectors with covariance equal to the identity matrix. By doing this, we attempt to contract the axes in the the representation space that dominate our task-inference statistics. We find that whitened embeddings often provide better signal to the adversary for our inner product attack (Algorithm 2 than raw embeddings. To compute the transformation, we first estimate the covariance matrix of the embedding space by pooling all of the embeddings available to the adversary, regardless of task or inclusion in the model's training set, and computing the regularized covariance matrix Ereg = (Q + X xx(() Ia) where Q is the sample covariance matrix, d is the embedding dimension, and À is a small constant. Once well-conditioned Ereg has been estimated, a common choice for the whitening transformation
would be W = Dn To ensure that the covariance estimate is not dependent task's data, we compute whitening transformations for each of the 27 tasks that we input to our attack, leaving out the data from one task each time. When applying W to the embeddings, additionally center the embeddings by computing ē, the sample mean of all embedding vectors. Thus, normalize given embedding by computing ((e- e).
a challenge
19

---

Published as a conference paper at ICLR 2026
A.7.2 VARIANCE ATTACK TEST STATISTIC
Across all of our experiments in both MTL settings, the statistic produced by our inner product attack (Algorithm{2) is consistent with respect to the ordering of IN and OUT distributions. In contrast, when studying leakage in MTL for personalization, we observe a discrepancy the test statistic distributions for vision and language datasets. Figure 4 shows that the coordinate-wise variance statistic (Algorithm is larger for IN tasks than OUT tasks.
[Streng!
14 (Strogg)
OUT
OUT
130 300
600
5:0
8.0
5.0
7.5
100
125
15.0
175
(a)
(b)
Figure 4: Distribution of Coordinate-Wist Wise Variance Statistic; (a) CelebA; (b) Stack Overflow
To investigate this discrepancy in the ordering of coordinate-wise variance test statistic, compute the pairwise inner product, or similarity, of the weights in the task-specific layers for our CelebA, FEMNIST, and Stack Overflow models after MTL training. Figure the distributions of the task head inner products, where we see that the vision models produce embeddings that lead spec-fie layers to be correlated on average (i.e. inner product not centered around 0). Because the task heads are correlated, the primary signal for our attack is the shared representation's 's "overconfidence" certain directions of the embedding space, which yields higher coordinate-wist wise variance. We find that the task heads of our FEMNIST models have the highest correlation, followed by our CelebA models, then Stack Overflow, which maps to the ordering of the AUC scores we see in our experiments on MTL for personalization.
ColebA
Stack Overtoow
Product Task Heads Afber MTL Training
Figure 5: Task-Specific Layer Similarity (Personalization)
A.8 ABLATIONS ON SYNTHETIC DATA
We attempt to understand mask-infeeence leakage by rumning our attacks on synthetic dataset. The data generation process for this dataset is identical to the mean estimation example in Section 4 but we adapt it for machine learning. To create the synthetic dataset, we start by sampling 27 i.i.d. tasks, piii....22 from a d-dimensional Gaussian distribution. For each of the tasks, Ps, we sample a dataset of N d-dimensional vectors from the corresponding task distribution. To label the data for a learning problem, we first generate a random projection matrix E Rxd, where d is the data dimension and & is the embedding dimension. Then, we randomly sample the "target" task-specific layers .I.... 9T E & and get the label for each sample, z. coming from task 7 e [T] to be the sign of the inner product between 9r and the embedding vector H (that is, for any sample X
20

---

Published as a conference paper at ICLR 2026
Sriing adertary
0.00
8.50
8.00
380
(a) Embedding Dimension
(b) Number of Samples Per Task
(c) Number of Tasks
Figure 6: Ablations on Synthetic Data
yky sign((g. HX )). This particular dataset is well suited for MTL as all tasks have unique labeling functions, but a common projection into the embedding space.
In our experiments on synthetic data, we use simple neural network with 512 hidden units and linear projection layer into the embedding space to approximate H. We vary the embedding dimension, number of samples per task, and number of tasks in the dataset, then measure our coordinate-wise wise variance attack's s AUC over training runs, with 64 trials of each attack on 8 random samples from each task. The results of this experiment are shown in Figure 6 We observe that increasing the number of total samples in the dataset, whether by increasing the number of samples per task or the total number of tasks, has sharp impact on the strong adversary's 's AUC. As the model has more samples learn from, the gap between the strong and weak adversary's 's AUC scores shrinks. We also find that, in this suite of experiments, the embedding dimension has little impact on task-inference AUC.
B ANALYZING TASK-INFEEENCE VIA TRACING ATTACKS
To motivate further exploration of attacks on deep learning models, we present simplified analog of multitask learning via mean estimation. Proofs of our theorems this Section can be found in Appendix B.2.
Suppose that there exists set of task means M (. WT} which are sampled i.i.d. from N I'))) where z is the "true" mean that we would like to estimate, For each "task" 14i we can sample N (where N is relatively small) points i.i.d. from W(p, L2) and store them in the set X; = f........ Ig,N instead compute the sample "multitask" mean using all of the X,' K,'s
J. Since no individual dataset X, would yield an accurate estimate of fi we can
N
7= T
XIJ
This "multitask" mean be viewed analogous to the weights of the shared representation in multitask learning, because we average over data, X;, sampled from a mixture of Gaussian distributions, (N,,, T2,,), which share features, produce accurate estimate of the underlying common parameter, p. We note that f is Gaussian with expectation and covariance
( +
) Ild
[ = P a.X
E.
Cov (ft)
T
N N.T
B. TRACING ATTACK FOR TASK INFERENCE
Here, we construct an adversary, based on prior work on tracing Dwork et al. (2013), with similar variants to the strong and weak adversaries detailed in Section We consider challenger who releases the statistic j and an adversary who wants to learn whether a given task μ4, was included in the dataset that was used compute jr.
21

---

Published as a conference paper at ICLR 2026
Weal
wndoe 28
T-128 Gleang AMC-2.343
Mhxtt A-10-8 1ng
(Mrvng datin 782
WPark d-cie 778
Fasa Fustive
(b) Distribution of Test Statistic 2
(a) ROC Curves for Strong and Weak Tracing Adversaries
Figure 7: Results of Our Tracing Attack on Multitask Mean Estimation (T = 256; d = 256; N = 8; k 4)
One possible attack would be the following:
1. The adversary receives a challenge set, or batch of data, B, from the challenger where |B] i S N, such that the samples come from a task that was used compute P. JIN = U4, or a task that was not used to compute ji but comes from the same underlying task distribution, POUT (i, r2
2. First, the adversary computes the sample mean of the batch
1 HB
Σ
K
ZEB
3. Then, the adversary computes the test statistic
z ==
(ft
MB (b)
MB
4. Lastly, the adversary performs thresholded classification, returning > y for some threshold
Our adversary adapts test statistic originally intended for membership-inferent Dwork et al. (2015) that measures the correlation between the released statistic and the samples available to the adversary. However, unlike the membership-intee infferenceee setting, the adversary is attempting detect traces of the task, or data's 's distribution, (4i, rather than any particular sample from one the datasets X; In the following theorems, find that the inclusion of task, μıs be inferred by both the strong and weak adversaries, and the strong adversary gets a slight advantage from having access to samples in X,.
Theorem B.1 (Strong Adversary: Theorem 4.1 in Main Body). Let be the index of the target task and suppose that the challenger sends the adversary a challenge set of k samples such that, when the task is IN, the samples are drawn uniformly at random from X,. Then the expected value of the statistic, z, when Pr is OUT is
E
[
and when μy is IN, the expected value of z is
d T
σ² N
E M.X
[iv]
In Theorem 4.1 we see that the expectation of the strong adversary's test statistic, 2, primarily grows in the dimension the data divided by the total number of tasks used to learn the multitask mean, ü. Through this result, we demonstrate that detecting traces of tasks can be seen of membership inference, as setting the number of tasks = 1, sampling N datapoints, X, only from the single task to estimate the task mean, and allowing the adversary to use = 1 sample reduces N to the membership-nnfeee inffrencce test statistic, which has expectation e ($) In our setting, single sample in the multitask mean, the expectation of their statistic
generalization
if the adversary wants
22

---

Published as a conference paper at ICLR 2026
would be ④ (NT) Because the total dataset size for mean estimation is T N with N < T, tracing tasks is an easier objective for the adversary than tracing individual samples, as there larger separation between the distributions of 2nN and 2OUT. Moreover, because the dataset is composed of multiple task distributions, a weak task-inference adversary can mount attacks by using data that comes from one of the included tasks but
used in the estimation of j..
Theorem B.2 (Weak Adversary: Theorem 4.2 in Main Body). Let be the index of the target task and suppose the challenger sends the adversary a challenge set of k samples such that, when the task is IN, the k samples are drawn ii.d. from the same distribution X,, N(,,, oa). Then the expected value of the statistic, 2, when ue, is OUT is When μ4, is IN, the expected value of 2 is
d T
E A.X
FIN
Informally, Theorem shows that the expectation of the weak adversary's test statistic depends only on the number of tasks, and does not benefit from smaller number of total training samples. Similar to the strong case, when analyzing the weak case, we split the statistically dependent and independent components of the sum, as the batch is drawn from the underlying task distribution for some fixed task which was included in the estimate. In contrast with the strong case, there is no overlap between the adversary's challenge set and the training data. Thus, we need not account for overlapping X ss. Rather, we only need to consider that the challenge set samples come from the same distribution, or task, as one of the tasks in the dataset used to compute j.
Theorem B.3 (Variance of 2; Theorem 4.3 in Main Body). The variance of when ps is OUT is
(k ( N
Var (ZOUT) R.X
0a
+
+
(0202)
T
KN
KN
When pl, is IN.
Var (EIv) S 3ar (Zour) X,X
M,X
We note that the strong adversary always achieves greater attack success rates than the weak adversary because the distance between the expected value of the test statistic, 2, when the task is IN or OUT is strictly larger than that of the weak adversary attack on mean estimation shows that the adversary's 's success is dependent on two distinct parts: the knowledge they receive from having member data (i.e. the _ term) and the knowledge they have about the distributions, or tasks, that compose the dataset (i.e. the & o term). To empirically verify results, simulated the attack with parameters T 256, d = 256, N = 8, and & 4. The ROC curves and distribution of z for both the strong and weak adversaries are shown in Figure
similar level of variance. Furthermore, this
B.2 PROOFS FOR SECTION
In this section, we provide the proofs for our theorems in Section Because the random variables in our estimation and attack are nested, we use the subscript X to denote taking probability over sampling the data and subscript yu to denote taking probability over sampling tasks.
Theorem 4.1 (Strong Adversary). Let be the index of the target task and suppose that the challenger sends the adversary a challenge set of l samples B such that, when the task is IN, the k samples are drawn uniformly at random from X,. Then the expected value of the statistic, when ptr is OUT is
E [FOUT] 0 n.
and when (4 is IN, the expected value of z is
σ² N
E
[iN]]
0² +
X.X
T
Proof. In the strong case, for tasks that were included, the adversary has access to samples which were used to compute the mean, P.. Suppose that the strong adversary computes 2, then, when μrr is OUT,
23

---

Published as a conference paper at ICLR 2026
E [zour]
E [i
μB pi)l
a.X
M.X
= E li+ T2]
E [AB - [4]
MiX
0
In contrast, when the batch of points is IN
[iv]]
Ki F PB - -1i)
H,X
n.
d
=
Σ
E
l(At pa)s (uB - p)d
M.N
For succinctness, we drop the summation over d dimensions as they are i.i.d.
E ((A x.
p))
(BB
[i]]
E n.X
[âtu - PHB - pa + إi²]
EE [ApB]
ft E [UB] - x. +
n.X
n.X
E. [iuul] - nT2
M.X
Now, expanding ja and μB
)
)
E. P.X
[ppp]
E.
1
1
Σ
X
112
H.X
T
N
j=1
1=1
We assume that k S N. Separating the correlated and uncorrelated tasks, have
24

---

Published as a conference paper at ICLR 2026
N Σ
| h
1
1
E
Σ N
Xi.
k
P.X
T 1
j=1
!=ر
1 N Σ
|
+ E P.X
XTs
Xrs
12
TN |=1
N Σ
k
X.
E
- N
T
1
Xi.i
x,X
E
h
_
1
Σ
XTs
j=l
N Σ
+ E A.X
1
XTi
1
Σ
Xrss
x²
TN
To
j=1
|=1
トΣ Xms )]
T-1
11
+
B
1
Σ
XT.
|
1
12?
T
niX
TN
h
j=1
k
1
Σ
E
[X
+
Σ
E
[Xrs X - NkR²
TNK
nxX
j#t
Now, taking the expectation over sampling the data then taking the expectation over sampling tasks,
E +
+02
Eu( NNEE))
TNK
j=1
(k(p² + 02 + +0²) + (Nk - (x²2 + +02) - -Nkp²)
TNK
TNk (ko² + koo² + (Nk- - k))²²)
(Nko² + +ko
TNk
π² +
o²
T
NT
Summing over the d i.i.d. dimensions, we get
= _ 2 + σ²
d
[aN
x
T
N
25

---

Published as a conference paper at ICLR 2026
(Weak Adversary). Let r be the index of the target task and suppose the challenger sends the adversary a challenge set of samples such that, when the task is IN, the samples are drawn ii.d. from the same distribution as (V,, lg). Then the expected value of the statistic, 2, when 14, is OUT is O. When μr is IN, the expected value of 2
Theorem 4.2
d T
E n,X
[aNN]
0²
Proof. The proof for the OUT case is identical to the proof for Theorem 4.1 When the weak adversary's challenge batch is IN
[ZIN]
E Ki0e
F4, PB -m]]
H,X
H.N
E 1(A p)) (MB p){]
M.x
For succinctness, we drop the summation over d dimensions as they are i.i.d.
[A M,X
(1)
(BB - [i]]
IE [upB - PuB - pp + p² H.X
TE 1 E. [uB] - jr E
[App]]
lil D?2
n.X
24x
E. [App]
1I22
n.X
Expanding fi and jA yields
1 k
)
E X
Jipn]
n1
E
Xis
X
112
x,
T
N
To
iml
We assume that k < N. Once again separating the correlated and uncorrelated tasks, we have
N
k
E
Σ
X
E
XT.J
T
N
a.x
k
j=1
j=l
N
+ E M.X
Xr.
Σ
Xrs
12
TN
ho
j=1
j=1
)
T
+
X
1
1
Xms
112
H.X
TN
jml
Now, note that the challenge batch samples are fresh, i.i.d samples from the task distribution N (H ILL)). Thus, the challenge batch and f are independent over sampling the data, but correlated over tasks. Taking the expectation over both data and tasks,
26

---

Published as a conference paper at ICLR 2026
1 &
T 1
ス
-1² n²
E
Td
T
T --
- - n1
T
T
T-
1
T p
(1² + 02) - 7.2
T
711² + - 7
1
σ²
T
Summing over the d i..dd dimensions, we get that under the weak adversary
d
E
[=i]]
2
T
Theorem 4.3 \Variance). The variance of a when pr is OUT is
Var (ZOUT) 4.X
d o'
+ 04
+
k + N K:
(0'03)
T
KN
When Je, is IN,
Var (ZIN) S 3ar (ZOUT) X.X
P.X
Proof. First, we define the following random variables:
a a= jt - No (1,
2
0²
T
NT ll
B = 15 B ~ N (0,
(6(202 +
0²
Ml
h
Then, the variance in the OUT case is
Var (ZOUT) = Var ((a: B))
R.X
n.X
d
Var (ac: 3.)
X.X
For succinctness, we drop the summation with index over the i.idd dimensions of the random variables. Then, we have
27

---

Published as a conference paper at ICLR 2026
Var (a . B) :3== = E [a²0²]
L [a9]
n.X
M.X
n.
E [a²8²]
M.X
E. [a2]
E [62]
N.X
-Var Var (3)
n.X
niX
[22
0²
0² 0²)
+
T
NT
1
0
(k <<N (0)02)
k
N
T
Summing over the d dimensions, we have
Var (ZOUT)
d
+ at
kN +
(*tr))
KN [0P07)
M.X T
Now, we can bound the variance of 2IN = (ax, B}
Var (2IN) = Var (1a, {))
A.
M.X
d
Σ
Var (as (8)) (Ba) MiX
E [[a: -b)7]]
- I
[a= BB1²
a.X
E [(a: (8)3]]
Coy (at, B,)
MiN
a.X
E I(a: - 3.0 - 22 (ar)Var
Var (8c) n,X n.X
where p is the correlation coefficient between o; and 3, Now, using the Schuway inequality and the fact that p² 2 0
[a IE
1891 axx
- -aa (a.) Var (B:) M.X
n.X
n.X
//aar (as) P.N
3aa (01)
- p² Var (as) Var (3x)
n.X
n,X
Σ Var (a;) (8x) - -aV (ax) Var (3:) n.X
M,X n,X
5 Σ 3ar (as) Var (8)) H.X n.X
3Va (ZOUT) M.X
28

---

Published as a conference paper at ICLR 2026
We note that in this proof, the variance of 2n is bounded by setting the squared correlation equal to O. in the weak
In practice, p² since we know from Theorems 4.1 and 4.2 that E
[a,
3.]
a²
case and a² o² in the strong case. T +
NT
C ADDITIONAL DISCUSSION
We provide additional discussion points to supplement Section 7
C.I POTENTIAL DEFENSES
Throughout our evaluation, we use gradient normalization and clipping, along with weight decay (or L2 regularization) to improve the stability MTL with few samples per task. In contrast to observations for empirical defenses made in the membershiipinferrncce literature Carlini et al. (2022), our task-inference attack succeeds even when the target MTL. model is trained using gradient clipping. We find that these heuristic methods to mitigate overfitting not sufficient to defend against task-inference.
Thus, we evaluate differential privacy (Dwork et al. 2006 Abadi et al. 2016) with task-level guarantees as certified defense. This corresponds to user-level (or group-level) differential privacy, where neighboring datasets are defined by the inclusion omission of user's entire contribution to the dataset, rather than an individual sample. While there are several works studying user-level differential privacy when training machine learning models Levy al. (2021): Chua et al. (2024); Charles et al.] (2024) and estimating high dimensional means Cummings et al. (2022); Agarwal et al. (2025), only one work develops algorithms with client-level privacy guarantees Hu al. (0023 when MTL is applied to collaborative (or federated) learning.
In this of experiments, we instantiate simple version of task-level DP training by clipping the aggregate task gradients, which bounds any given task's ' influence, before adding Gaussian noise the average update of the shared parameters. The task heads are not trained with DP they are only trained on their corresponding task's 's data. We focus on repeating the vision experiments on ResNet-8 and FEMNIST discussed in Section A.6 but now with formal privacy guarantees. We choose FEMNIST (and the small ResNet-8 model) in particular because the dataset provides sufficient number samples per task to maintain non-trivial model utility under the constraints of DP. Table T shows that while task-level DP mitigates task-inference risk, it incurs prohibitive cost to model utility. In particular, even at a relatively high privacy budget e = 16, the small amount of added noise degrades the quality of the shared representation, causing the model's top-1 accuracy to drop to 29%. These results reinforce our findings in Section 6.5) which show that the task-inference success heavily depends on the utility of the learned representation.
Table 3: Task-Inference Success (AUC) and Model Utility (Top-1 Acc.) with Task-Level Differential Privacy FEMNIST.
Variance Attack AUC
Dataset
Privacy Budget (s)
Top-1 Acc.
Strong
Weak
No Noise (E 00)
71%
0.69
0.57
FEMNIST
16.0
29%
0.49
0.48
8.0
26%
0.49
0.49
C.2 LACK OF SUITABLE BASELINES
Prior (sample-level) inference attacks on encoder models Liu et al. (2021b) exploit the tendency of well- trained encoders to produce similar embeddings for augmentations of the same training image, and rely on shadow models trained on labeled reference data to calibrate their attack. contrast, our adversary has strictly weaker access: given an input sample, they receive only the corresponding
29

---

Published as a conference paper at ICLR 2026
embedding vector output by the shared representation. Neither of the adversaries consider has knowledge of the model's weights, task task-specific classification heads, nor the ability to compute per- sample loss confidence scores. This renders baselines, such aggregated membership -ineeiieeeeeeeeeeeee ill-suited for direct comparison since even purely black-box membership inference attacks rely on statistics derived from full classifier outputs. The task-inference adversary in threat model queries the shared representation once per sample as a black-box function that maps inputs to embeddings. Our attacks leverage the implicitly learned dependencies and correlations between embedding vectors from the same task.
D USE OF LARGE LANGUAGE MODELS
Large language models were used to polish writing by rephrasing sentences, helping condense long paragraphs, and checking for typos and grammatical errors.
30


Reviews:
[{"review_id": "A4Dx89y0Om", "reviewer": "Reviewer_LGXe", "summary": "The paper introduces a new privacy attack in the multi-task learning (MTL) framework, where a shared representation is learned from data across multiple tasks. The goal of the attack is to infer whether a given task was included in the model’s training phase. Unlike traditional membership inference attacks at the sample level, this work operates at the task level, making it a higher-level privacy threat. The proposed attack is black-box and notably requires no auxiliary data or shadow models. \nTwo adversary settings are considered: \nStrong adversary, which has access to some task data or samples drawn from the task’s distribution.\nWeak adversary, which lacks access to such samples.", "questions": "- Can applying DP-SGD or other differential privacy methods mitigate the attack’s success rate? \n- Is the proposed attack scalable to larger and more complex models such as transformers or LLM-scale encoders?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper introduces a new threat model that examines how shared representations in multi-task learning can lead to privacy leakage.\n- It presents a black-box attack formulated under both weak and strong adversary assumptions, which makes the study applicable to real-world conditions.\n- A theoretical discussion is provided to explain the connection between representation sharing and privacy risks, highlighting the main factors that affect the effectiveness of the attack.", "weaknesses": "- There is a noticeable difference in performance between the weak and strong adversary scenarios.\n- The paper does not explore or evaluate any potential defense strategies against the proposed attack.\n- The method used to determine the attack’s decision threshold seems ad hoc, with limited justification or analysis of how it might perform in more realistic environments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oUC22HSGP3", "reviewer": "Reviewer_2x6h", "summary": "This paper introduces a novel privacy threat called task-inference in Multitask Learning (MTL). The authors investigate whether a shared representation, the minimal component required for joint training in MTL, leaks information about the specific tasks used to train it. They propose a purely black-box threat model where an adversary, with only query access to the shared representation, aims to determine if a given task was included in the training set. The paper develops two efficient attacks, a Coordinate-Wise Variance Attack and a Pairwise Inner Product Attack, which exploit the codependencies of embeddings from the same task without requiring shadow models or auxiliary reference data. Experiments across vision (CelebA, FEMNIST) and language (Stack Overflow) datasets demonstrate that these attacks are effective, even for a \"weak\" adversary who only has access to fresh samples from a task, not the original training data. The paper links this leakage to the model's generalization gap, suggesting that models memorize properties of entire task distributions.", "questions": "- Could the authors clarify how many independent MTL training runs were used for the main experiments in Figures 1 and 2? Are the reported ROC curves and AUC scores averaged across these runs?\n- How critical is the whitening transformation to the \"Pairwise Inner Product Attack\"? How many tasks must an adversary pool embed to compute a stable covariance matrix, and how does the attack perform without this step?\n- Have the authors considered comparing their attacks to a baseline where a standard, sample-level membership inference attack's predictions are aggregated (e.g., by averaging confidence scores or a majority vote) to produce a task-level inference?\n- For the \"multiple learning problems\" case on Stack Overflow, the performance gap between strong and weak adversaries nearly disappears. Does this imply that any high-utility MTL model where tasks are defined by distinct labels will be inherently vulnerable to task inference, even from adversaries with no access to training data?\n- L2​ decay or dropout on the shared representation be effective mitigations, or are formal defenses like user-level DP essential?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This paper presents a novel and highly relevant threat model, \"task-inference,\" which generalizes sample-level membership inference to the task level. This is a more practical and realistic threat for collaborative learning paradigms like MTL and federated learning, where the unit of privacy is often an entire user or data silo.\n- The proposed attacks are efficient and operate under minimal adversarial assumptions. A significant strength is that they are purely black-box and do not require training costly shadow models or having access to large, labeled auxiliary datasets, which are common barriers for other inference attacks.\n- The paper provides strong theoretical motivation for the attacks by analyzing an analogous tracing attack in a simplified Gaussian mean estimation problem. This analysis formally explains the observed performance gap between \"strong\" and \"weak\" adversaries.\n- The experimental evaluation is comprehensive, validating the attacks across both vision and language domains (ResNet on CelebA/FEMNIST and BERT on Stack Overflow).\n- The authors test the attacks in two distinct and practical MTL scenarios: personalization (where a task is a user) and solving multiple learning problems (where a task is a distinct classification objective), demonstrating the broad applicability of the threat.", "weaknesses": "- The paper does not clearly state whether the primary experimental results (e.g., in Figure 1 and 2) are averaged over multiple independent MTL training runs. While ablations mention multiple runs, the stability of the main attack results against different model initializations is not explicitly confirmed.\n- The practical utility of the \"blind\" percentile-based thresholds is questionable. As shown in Table 1, these thresholds can result in very high false positive rates (e.g., 47.6% or 41.7%), which would make the attack unreliable for an adversary who truly has no reference data to set a better threshold.\n- The \"Pairwise Inner Product Attack\" uses a whitening transformation that is computed by pooling all available embeddings. This seems to imply the adversary needs access to a broad set of embeddings from many different tasks, which somewhat weakens the \"minimal knowledge\" claim. The paper does not ablate the impact of this step.\n- The paper’s discussion of defenses is very limited. It mentions user-level differential privacy (DP) but does not evaluate it or any other potential mitigation, such as representation compression or other regularization techniques beyond the gradient clipping that was already used during training.\n- There is no comparison against adapted baselines. For instance, it's unclear how these novel attacks would perform against a simpler approach, like running a standard sample-level membership inference attack on all of the adversary's samples and aggregating the results (e.g., via majority vote) to get a task-level prediction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PLpbwMn5Z3", "reviewer": "Reviewer_cjJ7", "summary": "The paper investigates privacy leakage in the shared representations from Multitask learning. To do this, the paper presents a black box attach which tries to infer the inclusion of a task during MTL. This is done by assuming an adversary has access to samples from the target task's distribution (either from the training set, a strong adversary, or independently from the target task distribution; a week adversary). and queries the shared representation of the model. Experiments demonstrate the success of the attack across vision and language settings.", "questions": "* How is the relationship between the attack simplicity and its attack power in the context of existing methods from prior work?\n\n* Can the paper comment on how some existing defenses mitigate the task-inference risk?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The paper instroduces a novel task-inference threat model that generalizes the sample-level membership inference to the task level. \n\n* The attachs presented a black box and is not limited by the requirement of shadow models nor labeled reference data.\n\n* The empirical evaluations span both vision and language areas and use multiple datasets (CelebA, FEMNIST, StackOverflow) and MTL two common MTL cases (personalization and multi-problem learning).\n\n* The attack algorithms are clearly described, and the results highlight that the non-trivial success of the attack.", "weaknesses": "* It is not clear how this novel attack compares to existing relevant baselines in these inference attachks (such as potentially those requiring stronger assumptions for the adversary).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KrHgleVuoT", "reviewer": "Reviewer_Yemp", "summary": "This paper studies a new class of black-box privacy attacks on federated learning (FL) that operate by inverting shared representations rather than raw gradients. The proposed approach relies solely on the exchanged embeddings or logits accessible under practical FL protocols. The key idea is to train a small decoder network that learns to reconstruct private client data from aggregated or intermediate representations by leveraging auxiliary generative priors and feature alignment loss. The attack is evaluated under various FL architectures and datasets. The authors demonstrate that with limited black-box access, attackers can recover coarse visual features and demographic attributes, revealing new privacy risks.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- **Realistic and underexplored attack setting.** The black-box assumption makes the proposed attack more realistic than previous white-box gradient inversion works. This aligns well with real-world FL deployments where model internals are hidden.\n\n- **Clear motivation and threat model.** The paper clearly defines the attacker’s capabilities and constraints. The scenario of accessing shared embeddings (but not gradients) accurately reflects common FL architectures such as SplitFed or representation-sharing systems.", "weaknesses": "- **No formal privacy quantification.** The paper demonstrates qualitative image reconstructions but lacks rigorous privacy leakage measurement, e.g., mutual information, feature attribution metrics, or membership inference accuracy. Without quantitative metrics, it is difficult to assess the real severity of the attack.\n\n- **Dependence on auxiliary data.** The decoder is trained using an auxiliary dataset sampled from a similar distribution as the target data. This assumption is strong and limits the generality of the attack, particularly for domains like healthcare or finance where public data is unavailable.\n\n- **Weak comparison baseline design.** Some baselines (e.g., DLG, MI-FL) are disadvantaged because they rely on gradient access. The paper could include more relevant black-box baselines such as feature-space inversion or model inversion methods.\n\n- **Limited scalability and realism.** The attack is evaluated only on relatively small models and datasets. In practice, large models (e.g., ViT-L or ResNet-152) and high-dimensional embeddings could make black-box inversion considerably harder, but this is not tested.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)