# Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixing

Ousmane Touat, Jezekael Brunon, Yacine Belal, Julien Nicolas, César Sabater, Mohamed Maouche, Sonia Ben Mokhtar

► To cite this version:

Ousmane Touat, Jezekael Brunon, Yacine Belal, Julien Nicolas, César Sabater, et al. Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixing. MIDDLEWARE '25: Proceedings of the 26th International Middleware Conference, Dec 2025, Nashville, TN, United States. pp.180-194, 10.1145/3721462.3770770. hal-04933985v2

HAL Id: hal-04933985

https://hal.science/hal-04933985v2

Submitted on 13 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixing

Ousmane Touat

ousmane.touat@insa-lyon.fr

CNRS, INSA Lyon - LIRIS

Villeurbanne, France

Jezekael Brunon

jezekael.brunon@insa-lyon.fr

INSA Lyon - LIRIS

Villeurbanne, France

Yacine Belal

yacine.belal@insa-lyon.fr

INSA Lyon - LIRIS

Villeurbanne, France

Julien Nicolas

julien.nicolas@insa-lyon.fr

CNRS, INSA Lyon - LIRIS, McGill -

ILLS

Villeurbanne, France

César Sabater

cesar.sabater@insa-lyon.fr

CNRS, INSA Lyon - LIRIS

Villeurbanne, France

Mohamed Maouche

mohamed.maouche@inria.fr

Inria, INSA Lyon - CITI

Villeurbanne, France

Sonia Ben Mokhtar

sonia.ben-mokhtar@cnrs.fr

CNRS, INSA Lyon - LIRIS

Villeurbanne, France

Abstract

The primary promise of decentralized learning is to allow users to engage in the training of machine learning models in a collaborative manner while keeping their data on their premises and without relying on any central entity. However, this paradigm necessitates the exchange of model parameters or gradients between peers. Such exchanges can be exploited to infer sensitive information about training data, which is achieved through privacy attacks (e.g., Membership Inference Attacks – MIA). In order to devise effective defense mechanisms, it is important to understand the factors that increase/reduce the vulnerability of a given decentralized learning architecture to MIA. In this study, we extensively explore the vulnerability to MIA of various decentralized learning architectures by varying the graph structure (e.g., number of neighbors), the graph dynamics, and the aggregation strategy, across diverse datasets and data distributions. Our key finding, which to the best of our knowledge we are the first to report, is that the vulnerability to MIA is heavily correlated to (i) the local model mixing strategy performed by each node upon reception of models from neighboring nodes and (ii) the global mixing properties of the communication graph. We illustrate these results experimentally using four datasets and by theoretically analyzing the mixing properties of various decentralized architectures. We also empirically show that enhancing mixing properties is highly beneficial when combined with other privacy-preserving techniques such as Differential Privacy. Our paper draws a set of lessons learned for devising decentralized learning systems that reduce by design the vulnerability to MIA.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

Conference'17, Washington, DC, USA

© 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.

ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM

https://doi.org/10.1145/nnnnnnn.nnnnnnn

CCS Concepts

• **Computing methodologies** → **Distributed artificial intelligence**; • **Computer systems organization** → **Peer-to-peer architectures**; • **Security and privacy** → **Privacy-preserving protocols**.

Keywords

Gossip Learning, Membership Inference Attacks, Peer Sampling, Model Mixing

**ACM Reference Format:**

Ousmane Touat, Jezekael Brunon, Yacine Belal, Julien Nicolas, César Sabater, Mohamed Maouche, and Sonia Ben Mokhtar. 2025. Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixing. In. ACM, New York, NY, USA, 15 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn

1 Introduction

Federated learning [54], which allows training machine learning models while keeping users' data on their premises has gained significant interest in recent years both from academia and industry [2, 27, 60, 75]. However, this approach is often criticized because of the trust it requires on the aggregation server, which may be subject to failures [39], attacks [9], scalability issues [49], etc. In order to deal with these challenges, Decentralized Learning, which builds on the peer-to-peer exchange of model parameters between participants is gaining an increasing focus (e.g., [30, 50, 61]). Indeed, decentralized architectures have been heavily studied for decades and their scalability and resilience properties have been extensively documented both theoretically and experimentally in the literature [3, 53, 73]. This makes decentralized learning a natural avenue to explore for large-scale, distributed learning tasks [33, 51, 77].

However, whether federated or decentralized, collaborative learning has been shown to be vulnerable to privacy attacks and in particular to Membership Inference Attacks (MIA) [55, 58, 85]. This family of attacks aims at predicting whether a given data item has
been used or not in the model training set, which can reveal sensitive information about the data owner (e.g., a model trained to predict the risk of heart disease, could reveal sensitive information such as a patient's history of cholesterol levels, blood pressure, or family medical history [44]). Moreover, since MIA is closely related to other privacy attacks [82], the assessment of the vulnerability against such attack provides valuable insight into a model's overall vulnerability to other privacy attacks [62].

In this context, an active debate started in the research community to understand whether decentralization increases or decreases the vulnerability to MIA compared to FL architectures. For instance, Pasquini et al. [62] points out the vulnerability of decentralized topologies, while Ji et al. [36] contradicts this study by claiming that FL topologies are weaker. While these papers contribute to advancing the global knowledge on this topic, these studies are limited because they mainly focus on static graph topologies, ignoring decades of research on dynamic peer-to-peer systems enabled by their underlying random peer sampling protocols (RPS) [35, 74].

In this paper, we aim to contribute to this debate by analyzing the key factors that make decentralized architectures more or less vulnerable to MIA. Indeed, there are various ways of deploying a decentralized learning task, which differ according to the graph topology (e.g., a k-regular graph where each node's view size, or number of neighbors, is consistently k), graph dynamics (e.g., static graph, vs graph built over RPS protocols), communication synchronicity (synchronous vs asynchronous model exchange) and model mixing and dissemination strategy.

In this study, we focus on the MIA vulnerability of peer-to-peer based learning over asynchronous, k-regular graphs (also defined as Gossip Learning in [31, 61]) and vary all the remaining factors. Specifically, (i) we explore two local model mixing and dissemination strategies by introducing the send-all-merge-once (SAMO) strategy; (ii) we vary graph dynamics; (iii) we vary graph density through the number of neighbors. Beyond these key architectural features, we further consider four commonly used datasets and corresponding models and study the impact of i.i.d. and non-i.i.d. data distributions. We evaluate the MIA vulnerability of the overall decentralized architecture by considering that the attacker can observe all the messages exchanged in the system and run the MIA against each participant using messages received from its neighbors.

Our key finding is that MIA vulnerability is heavily correlated with two factors: (i) the local model mixing and dissemination strategy, which determines how a node aggregates models received from his neighbors and how he disseminates his updated model and (ii) the global mixing properties of the communication graph. For instance, we show that a static highly connected graph has a similar vulnerability to a dynamic graph of the same size. Instead, in weakly connected graphs (e.g., 2-regular graphs), graph dynamics brought by RPS protocols clearly decrease MIA vulnerability. Beyond local and global mixing properties, we also observe that learning tasks over non-i.i.d. data distributions clearly increase MIA vulnerability.

Summarizing, our paper makes the following contributions:

* **Comprehensive Study of MIA Vulnerabilities in Decentralized Learning:** To the best of our knowledge, we

propose the first study that scrutinizes how MIA vulnerability is affected by decentralized learning settings, focusing on the roles of graph connectivity, graph dynamics, model mixing, and data distributions.

* **Insights into Mixing Properties and Privacy:** We identify that both local and global model mixing properties are critical determinants of MIA vulnerability. Our findings reveal how dynamic graph settings and enhanced view sizes improve model mixing and mitigate privacy risks. Note that all our results can be reproduced using the code available in the following public repository<fnref n="1" />.

* **Recommendations for Privacy-Aware Decentralized Learning beyond generalization error:** Based on our findings, we provide design recommendations for decentralized learning systems that aim to balance utility with reduced MIA vulnerabilities, emphasizing the importance of dynamics and robust mixing protocols. We show that generalization error, which has been highlighted by previous works [62], is not the only key factor in determining the privacy risk.

The remainder of this paper is structured as follows: we first present preliminaries on decentralized learning, gossip learning, and a variant of gossip learning we consider in this paper as well as background on MIA and our threat model in Section 2. We then present our experimental analysis in Section 3. We complement our study with a theoretical analysis of graph mixing properties in Section 4. Then, we summarize our actionable recommendations in Section 5. Finally, we discuss related research efforts in Section 6 before concluding the paper in Section 7.

## 2 Preliminaries and Setup

In this section we present an overview of decentralized learning (Section 2.1) and the two learning protocols we consider, Base Gossip Learning (Section 2.2) and a variant of this protocol (Section 2.3). We detail the neighbor selection strategy for these two protocols in Section 2.4. Then, we introduce membership inference attacks (Section 2.5) and our considered threat model (Section 2.6).

### 2.1 Decentralized Learning

Decentralized Learning refers to a set $V = \{1, ..., n\}$ of nodes, which aim to collaboratively train a machine learning model without relying on a central parameter server. Given a set of attributes $Z$, a set of labels $Y$ and a model domain $M$, each node $i \in V$ has its local dataset denoted as $D_i \subset X = Z \times Y$ and their joint goal is to learn a global model $\theta^* \in M$ which minimizes the loss function:

$$ \mathcal{L}(\theta^*) = \frac{1}{n} \sum_{i=1}^{n} l_i(\theta^*, D_i), \qquad (1) $$

where $l_i(\cdot, \cdot) : M \times \phi(X) \to \mathbb{R}$ is the local loss function<fnref n="2" /> of node $i$ and $D = \bigcup_{i \in V} D_i$ as defined in [6].

In decentralized learning, each node performs three types of operations over models: local model updates, model exchanges, and model aggregation as described below.

<footnote n="1">¹https://gitlab.liris.cnrs.fr/otouat/exposingvulnerabilitydl</footnote>

<footnote n="2">²for a set $S$ we denote by $\phi(S)$ to the set of all subsets of $S$</footnote>
**Local Updates.** Each node $i \in \mathcal{V}$ updates its current model $\theta_i$ using its local training dataset $\mathcal{D}_i$ as follows:

$$ \theta_i = \theta_i - \eta \nabla l_i(\theta_i, \mathcal{D}_i), \qquad (2) $$

where $\eta$ is the learning rate and $\nabla l_i(\theta_i, \mathcal{D}_i)$ is the gradient of $l_i(\theta_i, \mathcal{D}_i)$.

As each node may update its model parameters by performing more than one local step, we will slightly abuse notation and still note $\eta \nabla l_i(\theta_i, \mathcal{D}_i)$ the sum of gradients obtained by sequentially applying a previously defined number of local steps with a given learning rate $\eta$.

**Model Exchange.** Each node $i \in \mathcal{V}$ exchanges its current model with one or many other nodes in its neighborhood. This neighborhood can be static or dynamic as further discussed in Section 2.4. Let $p \in \mathbb{N}$ be our time parameter, $N_i^{(p)} \subset \mathcal{V}$ is the view of party $i$ at time $p$. In practice, parties obtain their view through a random peer sampling service (RPS) [16, 35]. At each time $p \in \mathbb{N}$, the sets of views of all nodes $\{N_i^{(p)}\}_{i \in \mathcal{V}}$ define a graph $G^{(p)} = (\mathcal{V}, E^{(p)} \subset \mathcal{V} \times \mathcal{V})$ where an edge $(i, j) \in E^{(p)} \iff j \in N_i^{(p)}$.

**Local Aggregation.** To gain information from their neighbors, nodes aggregate their current model with received models. While for different purposes there exist many alternatives to perform this aggregation [41, 48, 64], we consider simple averaging of the current incoming models as in [46].

In the following section, we precisely describe how these operations are performed in the protocols we study.

## 2.2 Base Gossip Learning

In this paper, we focus on a particular type of decentralized learning, known as Gossip learning (GL) [31, 61], which assumes that nodes behave asynchronously. In Gossip Learning, each node initially starts with a common model $\theta_0$ and a set of neighbors also referred to as a view, and periodically and independently wakes up to perform the above three operations i.e., local updates, model exchange, and local aggregation.

Specifically, the GL algorithm defined in [61] is presented in Algorithm 1. In this algorithm, each node starts with a shared initial model $\theta_0$ (line 1). When a node $i \in \mathcal{V}$ wakes up at time $p \in \mathbb{N}$, it selects a random neighbor from $N_i^{(p)}$ and sends its current model $\theta_i$ (lines 3 and 4). When it receives a model $\theta_j$, it aggregates it with its current model $\theta_i$ (line 7) and performs a local update (line 8).

This algorithm is illustrated with an example depicted in the left part of Figure 1. In this figure, node $x$ receives a model from node $y_1$ (step ①), then performs a local update after aggregating the two models (step ②). Later, it receives a model from node $y_2$ (step ③) and performs again model aggregation and local update (step ④). Finally, upon waking up, it randomly selects one of its neighbors, i.e., node $z_1$ to which it sends its latest model (step ⑤).

We observe in this algorithm that models are aggregated in a pairwise manner, after which a local update is performed by the model owner. Furthermore, we observe that the model dissemination strategy is slow as each node sends its updated model to only one randomly selected neighbor. To increase model mixing, we investigate in this paper an alternative to the classical gossip learning algorithm, which we present in the next section.

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /><figcaption>Figure 1: Model Update and Aggregation in Gossip Learning (left) vs in SAMO (right)</figcaption></figure>

**Algorithm 1 Base GL Protocol**

1: **Input:** Communication topologies $N_i^{(p)}$ for each time $p$ and node $i \in \mathcal{V}$; initial model $\theta_i = \theta_0$, local data-sets $\mathcal{D}_i$, loss function $l_i(\cdot, \cdot)$ for all $i \in \mathcal{V}$; learning rate $\eta$

2: **for** each node $i \in \mathcal{V}$ that **wakes up** within time frame $p$ **do**

3: Select a node $j \in N_i^{(p)}$ at random

4: Send $\theta_i$ to node $j$

5: **end for**

6: **for** each node $i \in \mathcal{V}$ that receives a model $\theta_j$: **do**

7: Set $\theta_i \leftarrow \frac{\theta_i + \theta_j}{2}$ {aggregate with incoming model}

8: Set $\theta_i \leftarrow \theta_i - \eta \nabla l_i(\theta_i, \mathcal{D}_i)$ {perform local updates}

9: **end for**

## 2.3 Send-All-Merge-Once Protocol

We hereby introduce in Algorithm 2 a variant of the GL protocol, called Send-All-Merge-Once (SAMO) that is designed to improve the speed of model mixing over time. In SAMO, when a node receives a model, it stores it (as shown in line 11) instead of directly aggregating it and performing local updates. Later, when a node wakes up, if models were received, it aggregates them with its own (line 4) and performs local updates (line 5). Finally, it sends its current model to all its neighbors (line 8). This algorithm is illustrated with an example depicted in the right part of Figure 1. In this figure, node $x$ receives models from its neighbors $y_1, y_2$ and $y_3$ and stores them locally (steps ①, ② and ③). Later, upon waking up, node $x$ aggregates these models with its local model performs local updates (step ④), and sends the new model to all its neighbors (step ⑤).

As exchanges involve all the neighbors of a node, SAMO has a higher communication cost than the Base GL Protocol. Nevertheless, it better resists MIA (see Section 3).

## 2.4 Peer Sampling Service

In this paper, we aim to study both static and dynamic topologies. For both GL and SAMO, we assume that nodes rely on a Random Peer Sampling Service such as [35]. Specifically, in both protocols, neighbors of each node $i \in \mathcal{V}$, are chosen as follows. Initially, $N_i^{(1)}$ is determined by a k-regular graph $G^{(1)}$ chosen at random by the peer sampling service. In the static setting, neighbors do not change anymore. In the dynamic setting we adopt the PeerSwap method [24]: when node $i$ wakes up at time $p$, it swaps the view with a randomly selected neighbor before doing anything else. More precisely, $N_i^{(p)}$ is set to $(N_j^{(p-1)} \setminus \{i\}) \cup \{j\}$ through the peer
**Algorithm 2 SAMO Protocol**

1. **Input:** Communication topologies $\mathcal{N}_i^{(p)}$ for each time frame $p$ and node $i \in \mathcal{V}$, initial model $\theta_i = \theta_0$, local datasets $\mathcal{D}_i$, incoming model sets $\Theta_i = \{\theta_0\}$ and loss function $l_i(\cdot, \cdot)$ for all $i \in \mathcal{V}$; learning rate $\eta$

2. **for** each node $i \in \mathcal{V}$ that wakes up within time frame $p$ **do**

3. **if** $|\Theta_i| > 1$ **then**

4. Set $\theta_i \leftarrow \frac{1}{|\Theta_i|} \sum_{\theta \in \Theta_i} \theta$ {aggregate models}

5. Set $\theta_i \leftarrow \theta_i - \eta \nabla l_i(\theta_i, \mathcal{D}_i)$ {do local updates}

6. Set $\Theta_i \leftarrow \{\theta_i\}$

7. **end if**

8. Send $\theta_i$ to each node $j \in \mathcal{N}_i^{(p)}$

9. **end for**

10. **for** each node $i \in \mathcal{V}$ that receives a model $\theta_j$ **do**

11. Set $\Theta_i \leftarrow \Theta_i \cup \theta_j$

12. **end for**

sampling service, which also updates

$$ \mathcal{N}_j^{(p)} \leftarrow (\mathcal{N}_i^{(p-1)} \setminus \{j\}) \cup \{i\} $$

$$ \mathcal{N}_k^{(p)} \leftarrow (\mathcal{N}_i^{(p-1)} \setminus \{i\}) \cup \{j\} \text{ for all } k \in \mathcal{N}_i^{(p-1)} \setminus \{j\} $$

$$ \mathcal{N}_k^{(p)} \leftarrow (\mathcal{N}_i^{(p-1)} \setminus \{j\}) \cup \{i\} \text{ for all } k \in \mathcal{N}_j^{(p-1)} \setminus \{i\} $$

to keep views consistent. The swap is equivalent to changing the position of $i$ and $j$ in the current graph, which remains $k$-regular.

## 2.5 Membership Inference Attack

A Membership Inference Attack (MIA) is a privacy attack whose goal is to infer for a given model $\theta \in \mathcal{M}$ whether or not a sample $x \in \mathcal{X}$ belongs to the data used to train this model [68]. More formally, it is an algorithm $\mathcal{A}: \mathcal{M} \times \mathcal{X} \to \{0, 1\}$ such that $\mathcal{A}(\theta, x) = 1$ if it predicts that $x$ belongs to the training dataset of model $\theta$ and $\mathcal{A}(\theta, x) = 0$ otherwise.

Many MIAs have been proposed and range from expensive approaches that train ML models to predict membership such as neural shadow models [68] to information-theoretic estimators that leverage prediction entropy or confidence [67, 70]. In our work, we use the Modified Prediction Entropy Attack [70]. We first explain the Modified Prediction Entropy (MPE) measure and then the attack.

Let $P: \mathcal{Y} \to [0, 1]$ be a probability distribution over $\mathcal{Y}$ and $y \in \mathcal{Y}$. The MPE measure $M_{\text{MPE}}(\cdot, \cdot)$ is defined as follows:

$$ M_{\text{MPE}}(P, y) = -(1 - P(y)) \log(P(y)) - \sum_{y' \in \mathcal{Y} \setminus \{y\}} P(y') \log(1 - P(y')). \quad (3) $$

The MPE Attack used to discriminate if the sample $(z, y) \in \mathcal{X}$ was used to train model $\theta \in \mathcal{M}$ is defined as follows:

$$ \mathcal{A}_{\text{MPE}}(\theta, (z, y), \tilde{\tau}) = \begin{cases} 1 & \text{if } M_{\text{MPE}}(\theta[z], y) \le \tilde{\tau} \\ 0 & \text{otherwise} \end{cases}. \quad (4) $$

where $\theta[z]: \mathcal{Y} \to [0, 1]$ is the probability distribution that model $\theta$ assigns to each label in $\mathcal{Y}$ given input attributes $z \in \mathcal{Z}$ and $\tilde{\tau}$ is the attack threshold computed using training dataset $\mathcal{D}_i$ of a target node (or victim) $i \in \mathcal{V}$ as well as a test dataset $\mathcal{D}_i^t$ that was not used to train $\theta$. The idea is that the prediction entropy can discriminate

between member and non-member data, as the target model $\theta$ has a lower prediction entropy on train data.

It is unrealistic to deploy $\mathcal{A}_{\text{MPE}}$ in practice as it uses the training dataset of the victim to get the optimal threshold $\tilde{\tau}$. However, this approach offers an upper bound on the worst-case attacker whose information is the probability distributions $\{\theta[z]\}_{z \in \mathcal{Z}}$ generated by the target model $\theta$ and sample $x$. Therefore it is an informative privacy assessment metric. Moreover, it is significantly more practical to deploy than machine learning-based discriminators.

## 2.6 Threat Model

As an attacker, we consider an omniscient observer that at regular time intervals recovers the current models of all nodes $\{\theta_i\}_{i \in \mathcal{V}}$ and performs $\mathcal{A}_{\text{MPE}}$ on each one of them, targeting each data sample of each node. It allows us to evaluate the vulnerability of all nodes in the system wherever their position is in the communication graph. This attacker is similar to the passive attacker defined in [62] to assess the vulnerability of static decentralized learning against MIA. We do not consider other attack models that consider actively malicious nodes that can deviate from the protocol to gain information [62]. Finally, a more detailed explanation of the metrics we used to compute the vulnerability to MIA is provided in Section 3.2.

# 3 Experimental Study

In this section, we present the findings of our study, which investigates the factors influencing the tradeoff between utility and privacy in Gossip learning. Specifically, we address the following research questions:

* **RQ1:** What is the impact of using the SAMO protocol compared to Base Gossip?

* **RQ2:** Is there an advantage to using a dynamic setting compared to a static one?

* **RQ3:** Viewed through the lens of canary-based evaluation (probing worst-case memorization), what advantages does using a dynamic setting offer?

* **RQ4:** What is the impact of the view size on both static and dynamic settings?

* **RQ5:** What is the impact of non-i.i.d. data distribution?

* **RQ6:** Is the link between generalization error and MIA vulnerability maintained in our studied setting?

* **RQ7:** How using a dynamic setting can complement other privacy-preserving techniques such as Differential Privacy?

## 3.1 Experimental Setup & Configurations

*Implementation of Gossip Learning Protocol.* We consider two gossip learning protocols: the Base Gossip protocol and SAMO. Both protocols are implemented using our fork of the GossiPy python framework [63] with Python 3.11 and Pytorch.

To simulate the asynchronous nature of the gossip protocol, the execution is divided into discrete time units called ticks. Each round of communication is represented by 100 ticks and each node $i \in \mathcal{V}$ waits $\Delta_i$ ticks between wake-ups. The waiting time $\Delta_i$ is sampled from a normal distribution $N(\mu, \sigma^2)$ with $\mu = 100$ and $\sigma^2 = 100$ at the beginning of the execution.
**Communication Topology.** In our setup we simulate 150 nodes that are connected together in an initial k-regular graph topology with the view size $k \in \{2, 5, 10, 25\}$. We consider both static and dynamic settings.

**Datasets and Partitioning.** The efficacy of the proposed protocol is assessed on a number of datasets described in Table 1. For experiments on independent and identically distributed (i.i.d.) data, we utilize standard image classification datasets: CIFAR-10, CIFAR-100 [45] and FashionMNIST [78], alongside the Purchase100 dataset [68], which is commonly used in privacy attack research for tabular data. These datasets represent established benchmarks within the decentralized learning and privacy-focused research communities. In this configuration, the training data is distributed uniformly across all nodes in an equal partition. Specifically, this partitioning involves sampling each node's training data i.i.d. from the base dataset's training split. Furthermore, each node's respective test data is also sampled i.i.d. directly from this same base training split. The global test set (originating from the base dataset's distinct test split) is maintained separately to assess the global test accuracy of each model.

**Training Setup.** For our experiments, we trained each model with Gossip Learning (base algorithm and our variant), varying hyperparameters such as the learning rate, the number of local steps, or the used model for the dataset depending on its task complexity. Table 2 shows the training setup for all our datasets, including the used model and the number of parameters. For light image datasets such as CIFAR-10 or Fashion-MNIST, we use a light CNN architecture. For Purchase100 we follow the implementation form Nasr et al. [58], by using a 4-layer fully-connected neural network. For more challenging image datasets such as CIFAR-100, we opt to use ResNet models [29]. We initialize each node's model using Kaiming normal initialization function [28].

## 3.2 Metrics

**Utility: Global Test Accuracy.** In all experiments, we assess the utility of the node models using top-1 accuracy on the global test set. For a model $\theta$, we define this metric as follows:

$$ \text{acc}(\theta, \mathcal{D}_{\text{test}}) = \frac{1}{|\mathcal{D}_{\text{test}}|} \sum_{(z,y) \in \mathcal{D}_{\text{test}}} \mathbb{I}(\arg \max \theta(z) = y), \quad (5) $$

where $\mathbb{I}$ is the indicator function.

**Privacy: MIA Accuracy.** We measure the accuracy of the MIA attack. Let $\theta$ be the victim model, $\mathcal{A}_{\text{MPE}}$ be the attack described in Section 2.5 and $\mathcal{D}_{\text{att}} \subset \mathcal{Z} \times \mathcal{Y} \times \{0, 1\}$ a set of member and non-member data. Membership in $\mathcal{D}_{\text{att}}$ is defined with label $m$ (1 for member and 0 for non-member) and usually sampled equally from the local train and test set respectively. We then define the MIA vulnerability as below:

$$ \text{acc}_{\mathcal{A}}(\theta, \mathcal{D}_{\text{att}}) = \frac{1}{|\mathcal{D}_{\text{att}}|} \sum_{(z,y,m) \in \mathcal{D}_{\text{att}}} \mathbb{I}(\mathcal{A}_{\text{MPE}}(\theta, (z, y), \tilde{\tau}) = m), \quad (6) $$

The threshold $\tilde{\tau}$ is implicitly defined as the value-maximizing the attack accuracy.

**Privacy: MIA TPR@1%FPR.** We also compute this metric, using the MPE scores, $M_{\text{MPE}}(\theta[z], y)$, calculated for all samples in $\mathcal{D}_{\text{att}}$. By varying the decision threshold applied to these MPE scores (a lower score indicates member), we can generate a Receiver Operating Characteristic (ROC) curve, plotting the True Positive Rate (TPR) against the False Positive Rate (FPR). The TPR@1%FPR is the value of the TPR obtained from the ROC curve at the point where the FPR is less than or equal to 1%.

$$ \text{TPR@1\%FPR}(\theta, \mathcal{D}_{\text{att}}) = \frac{1}{|\mathcal{D}_{\text{att},m=1}|} \sum_{(z,y) \in \mathcal{D}_{\text{att},m=1}} \mathbb{I}(\mathcal{A}_{\text{MPE}}(\theta, (z, y), \tilde{\tau}_{1\%}) = 1) \quad (7) $$

The threshold $\tilde{\tau}_{1\%}$ is implicitly defined as the value yielding the point on the ROC curve where FPR $\le 1\%$.

**Generalization error.** We define a generalization error metric as the gap between the local train accuracy and the local test accuracy. Formally, for node $i$ that trains model $\theta_i$ with its local dataset $\mathcal{D}_i$, we define the generalization error as below:

$$ \text{gen\_error}(\theta_i, \mathcal{D}_i) = \text{acc}(\theta_i, \mathcal{D}_{i,\text{train}}) - \text{acc}(\theta_i, \mathcal{D}_{i,\text{test}}), \quad (8) $$

with $\mathcal{D}_{i,\text{train}}$ and $\mathcal{D}_{i,\text{test}}$ being respectively the local training and test dataset splits of node $i$.

At the end of each round of communication (defined in section 3.1), we record measurements on the model of each node and subsequently report the mean value aggregated across the nodes.

## 3.3 Comparing SAMO to Base Gossip (RQ1)

To start our evaluation, we want to emphasize why we propose the protocol SAMO (described in Algorithm 2). To do that, we show in Figure 2 the tradeoff between average MIA vulnerability (accuracy and TPR@1%FPR) and average global test accuracy for models trained on the CIFAR-10, CIFAR-100, Fashion-MNIST, and Purchase100 datasets. Each point in the plot represents an evaluation during a given round, with the x-axis showing the average model test accuracy and the y-axis showing the corresponding average MIA vulnerability. We set the view size to 5 to ensure a sufficient number of neighbors for SAMO to exhibit distinct behavior from Base Gossip, while still operating within a sparse graph topology.

Our results show that given a target test accuracy the SAMO protocol outperforms Base Gossip in most settings, especially in the regimes where the test accuracy is maximal. The low accuracy found on datasets such as CIFAR-10 is expected due to the constrained communication topology, observed similarly in other studies [10, 16]. More precisely, with SAMO, the maximum global test accuracy obtained ranges from 35.4% and 88.4% relative to an MIA vulnerability ranging from 62.7% to 92.9% accuracy, and 3.66% to 16.1% for TPR@1%FPR. In comparison, in the Base Gossip setting the maximum global test accuracy obtained ranges from 29.9% and 82.6% relative to an MIA vulnerability ranging from 63.5% and 94.1% accuracy, and 3.68% to 15.7% for MIA TPR@1%FPR. We explain this advantage by the fact that SAMO favors model mixing, as each user receives models from multiple neighbors and delays the aggregation in order to hide his or her models within more models.

**Takeaway:** Favoring model mixing by incorporating the SAMO protocol offers a better privacy/utility tradeoff.
<table><caption>Table 1: Dataset Characteristics</caption><thead><tr><td>Dataset</td><td>Train Set</td><td>Test Set</td><td>Input Size</td><td>Classes</td><td>Model</td><td>Description</td></tr></thead><tbody><tr><td>CIFAR-10</td><td>50,000</td><td>10,000</td><td>32 × 32 × 3</td><td>10</td><td>CNN</td><td>Color images across 10 classes including animals, vehicles</td></tr><tr><td>CIFAR-100</td><td>50,000</td><td>10,000</td><td>32 × 32 × 3</td><td>100</td><td>ResNet-8</td><td>Fine-grained color images with 100 object classes</td></tr><tr><td>Fashion-MNIST</td><td>60,000</td><td>10,000</td><td>28 × 28 × 1</td><td>10</td><td>CNN</td><td>Grayscale images of clothing and fashion accessories</td></tr><tr><td>Purchase-100</td><td>157,859</td><td>39,465</td><td>600</td><td>100</td><td>MLP</td><td>A tabular dataset of customer purchases to classify buying behavior</td></tr></tbody></table>

<table><caption>Table 2: Training Configuration</caption><thead><tr><td>Dataset</td><td>Model</td><td>Parameters</td><td>Learning Rate</td><td>Momentum</td><td>Weight Decay</td><td>Local Epochs</td><td>Rounds</td></tr></thead><tbody><tr><td>CIFAR-10</td><td>CNN</td><td>124k</td><td>0.01</td><td>0</td><td>5e-4</td><td>3</td><td>250</td></tr><tr><td>CIFAR-100</td><td>ResNet-8</td><td>1.2M</td><td>0.001</td><td>0.9</td><td>5e-4</td><td>5</td><td>500</td></tr><tr><td>Fashion-MNIST</td><td>CNN</td><td>124k</td><td>0.01</td><td>0.9</td><td>5e-4</td><td>3</td><td>250</td></tr><tr><td>Purchase-100</td><td>MLP</td><td>1.3M</td><td>0.01</td><td>0.9</td><td>5e-4</td><td>10</td><td>250</td></tr></tbody></table>

Given these results, we favor using the SAMO protocol for the remaining experiments.

## 3.4 Comparing Static and Dynamic Settings (RQ2)

Figure 3 shows the trade-off between average MIA vulnerability (accuracy and TPR@1%FPR) and average global test accuracy, comparing static and dynamic topologies. This experiment was conducted on a particularly challenging setting using a sparse graph with a view size of 2.

We observe that in all datasets, settings with dynamic topology outperform the static ones, achieving a better trade-off between MIA vulnerability and global test accuracy. More precisely, in the dynamic setting, the maximum global test accuracy obtained ranges from 30.4% and 80.9% relative to an MIA vulnerability ranging from 67.0% to 94.6% accuracy, and 2.0% to 20.3% for TPR@1%FPR. While, in the static setting, the maximum global test accuracy obtained ranges from 15.8% and 76.4% relative to an MIA vulnerability ranging from 72.0% and 98.8% accuracy, and 2.0% to 91.8% for TPR@1%FPR. We observe that models trained with dynamic topologies maintain lower MIA vulnerability at comparable accuracy levels.

**Takeaway:** Dynamic topologies have a positive impact on both the MIA vulnerability and model utility.

## 3.5 On auditing individual privacy with a canary-based approach (RQ3)

Evaluating individual privacy of worst-case vulnerabilities using standard metrics like TPR at low FPR is often computationally prohibitive, especially in large-scale scenarios involving hundreds of nodes and communication rounds, as in our setup. To overcome this, we employ an efficient alternative called "canaries" as proposed

in the work of Aerni et al. [1]. These crafted data samples simulate worst-case vulnerabilities and are designed to be readily memorized by models lacking privacy safeguards, thus serving as probes for potential leakage.

Leveraging our network's homogeneity (identical models and parameters per node), we generate canaries by applying a label flipping function to selected training samples and distribute them disjointly and evenly on all the nodes of the communication network. We then conduct a targeted, node-specific entropy-based attack on known canary samples, efficiently yielding numerous attack scores for metrics like TPR@1%FPR. We used 600 canaries for CIFAR-10/CIFAR-100/FashionMNIST and 1500 for the larger Purchase100 dataset.

Figure 4 presents the evolution of the maximum TPR@1%FPR across communication rounds, over 10 runs using distinct canary sets per run. We can see just how powerful this attack is, displaying results of up to 100% as these canaries are specially designed to be memorized by node models, making the attacker's task easier. However, we can see that using a dynamic topology marginally reduces maximum MIA vulnerability in the vast majority of datasets, with a very large reduction in the case of Purchase100. These experiments reveal that dynamic topologies tend to reduce the efficacy of the attack for outlier cases such as the one simulated here.

**Takeaway:** The positive impact of using a dynamic topology is also observed from the point of view of individual privacy.

## 3.6 Impact of the View Size on the Privacy/Utility Tradeoff (RQ4)

In this experiment, we aim to study the impact of the view size on the trade-off between utility and privacy. The results are presented in Figure 5 with the maximum average MIA vulnerabilities (accuracy and TPR@1%FPR) and the maximum average global test
<figure><img src="image_4.png" /><figcaption>Figure 2: Trade-off between MIA Accuracy and Global Test Accuracy and between MIA TPR@1%FPR and Global Test Accuracy across different datasets, comparing Base Gossip and SAMO, on a 5-regular graph with 150 nodes (60 nodes on CIFAR100).</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 3: Trade-off between MIA Accuracy and Global Test Accuracy and between MIA TPR@1%FPR and Global Test Accuracy across different datasets, comparing static and dynamic topology setups, on a 2-regular graph with 150 nodes (60 nodes on CIFAR100).</figcaption></figure>

accuracy on the CIFAR-10 dataset with various view sizes. The results show that the dynamic settings outperform the static settings in both utility and privacy. Interestingly, upon increasing the view size the differences between dynamic and static gets smaller. This is anticipated as increasing the view size gets the communication graph closer to a complete graph where both settings are equivalent. Also, the increase in view size is overall a strict benefit for the privacy/utility trade-off but this is at the cost of increasing the number of models sent per user (i.e., messages). Overall, profiting from both dynamic topology and increased view size there is a better privacy/utility tradeoff to be found. In particular, using a dynamic setting with a view size of 10 is quasi-similar to a static setting with a view size of 25 (with 2.5 times fewer models sent).

We conclude that both dynamicity and increased view size have a positive impact on MIA vulnerability as they both favor the model mixing. This intuition is further investigated in Section 4.

**Takeaway:** Increasing the view size closes the gap of dynamic over static but at the cost of more models sent.

## 3.7 Impact of non-i.i.d. Data Distribution (RQ5)

In this experiment, our goal is to examine the impact of non-iidness in Gossip Learning. To achieve this, we use the classical method for enforcing a non-i.i.d. setting [47], sampling from the Dirichlet distribution to apply label imbalance between each node. Specifically, using a Dirichlet distribution with parameter $\beta$, we sample the proportion $p_k \sim Dir_N(\beta)$ of records with label $k$ across the training sets of the $N$ nodes. The parameter $\beta > 0$ here controls the degree of heterogeneity, where a lower value of $\beta$ ($\le 0.1$) yields a higher label imbalance among nodes. In contrast, a higher $\beta$ value yields a distribution closer to the i.i.d. distribution.

The results are presented in Figure 6, showing the tradeoff between global test accuracy and MIA vulnerability (accuracy). As expected, the overall global test accuracy decreases with a higher non-i.i.d. setting from a maximum of 77.4% for the i.i.d. setting until a maximum of 52.4% for non-i.i.d. with $\beta = 0.1$, as the non-i.i.d. setting is known to make the collaborative learning process more challenging. However, more notably, the MIA vulnerability increases significantly across all training rounds. This is because the learned models tend to align a global model with data distribution
<figure><img src="image_6.png" /><figcaption>Figure 4: Maximum MIA TPR@1%FPR values on the canary set over communication rounds across different datasets, comparing static and dynamic topology setups, on a 2-regular graph with 150 nodes (60 nodes on CIFAR100).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 5: Comparison of maximum average MIA accuracy and TPR@1%FPR and according global test accuracy across different network view sizes and topology setups on the CIFAR-10 dataset using SAMO, with regular graph with 150 nodes.</figcaption></figure>

close to the overall global distribution, making local updates more
prone to leaking membership information.

Regarding the difference between static and dynamic settings, we
observe similar conclusions as in the previous experiment, with dy-
namic settings consistently outperforming static ones. However, the
increase in MIA vulnerability due to non-iidness remains prominent
regardless of the setting. Often, the dynamic setting successfully
approaches the results of the static setting with lower non-iidness
but never completely bridges the gap.

**Takeaway:** Non-i.i.d. data distribution among nodes significantly increases MIA vulnerability across all training rounds, and dynamic setting alone is insufficient to completely bridge this gap.

<figure><img src="image_8.png" /><figcaption>Figure 6: Trade-off between MIA Accuracy and Global Test Accuracy on the Purchase100 with varying degree of heterogeneity (i.i.d, dirichlet $\beta = 0.5$ and $\beta = 0.1$), comparing static and dynamic topology setups, on a 2-regular graph with 150 nodes</figcaption></figure>

## 3.8 Rethinking the Link Between Generalization Error and MIA Vulnerability (RQ6)

The relationship between generalization error (Equation (8)) and MIA vulnerability has been extensively studied in the literature, particularly in the centralized setting [68, 71]. This focus is understandable, as most MIA attacks rely on training overfitting, which is exposed through loss function evaluations on specific examples. The distinction between data points seen during training and those unseen is a crucial factor enabling the separation of members from non-members in MIA attacks. Intuitively, a model that generalizes well should be more resilient to MIA attacks. In our setting, however, we are not dealing with a centralized framework involving a single model. Instead, as demonstrated in previous experiments, models undergo various stages of mixing with others trained on different node data. This is why, we investigate the connection between generalization error and MIA vulnerability in our fully decentralized context. The results, depicted in Figure 7, illustrate the tradeoff between average MIA vulnerability (accuracy) and average generalization error. Each point in the plot represents an evaluation during a fixed round, with the x-axis showing the average generalization error and the y-axis showing the corresponding average MIA vulnerability. The findings reveal dataset-specific behaviors. For example, CIFAR-10 demonstrates an intuitive relationship between generalization error and MIA vulnerability. However, CIFAR-100 and Fashion-MNIST exhibit scenarios where, for a fixed generalization error, MIA vulnerability varies significantly. This phenomenon is even more pronounced for Purchase100, where distinct regimes of MIA vulnerability are observed for the same generalization error.

To delve deeper, Figure 8 highlights the tradeoff across training rounds. We observe that leakage introduced during an earlier round cannot be mitigated simply by improving the generalization error in subsequent rounds. Once a model overfits, increasing test accuracy and reducing generalization error do not eliminate the existing MIA vulnerability. Our interpretation suggests that the peak generalization error, typically occurring early in the training process, determines the maximum MIA vulnerability observed later.
**Takeaway:** Early overfitting drives persistent MIA vulnerability, which generalization improvements cannot fully mitigate.

This observation aligns with the concept of the Critical Learning Period phenomenon explored in FL [80]. This work emphasizes that changes during the early learning phases significantly impact the best accuracy achievable by the end of training. In our work, we find experimentally a similar pattern, but in terms of MIA vulnerability rather than accuracy.

## 3.9 Impact of Dynamicity on ensuring Differential Privacy (RQ7)

There remains the question of combining these design practices with more direct privacy protection techniques. In this experiment, we study the impact of the dynamicity of the communication topology while Differential Privacy (DP) is ensured. Formally, a stochastic algorithm $\mathcal{M}$ satisfies $(\epsilon, \delta)$-Differential Privacy (smaller parameters correspond to stronger guarantees) if for any two adjacent datasets $D, D'$ and any output $S$, we have $\Pr[\mathcal{M}(D) \in S] \le e^\epsilon \Pr[\mathcal{M}(D') \in S] + \delta$. In our decentralized setting, DP can be enforced on the node-level, by clipping local gradients and adding Gaussian noise with an adequate variance to the clipped gradient at each step, therefore spending a pre-determined privacy budget per step. We use the composition rule of Rényi Differential Privacy (RDP) [57] to track the overall privacy budget given per-iteration privacy budgets. In practice, our implementation uses Opacus’s DP-SGD with RDP accounting.

Figure 9 illustrates the tradeoff between the MIA efficiency and the average utility of the models trained by nodes. The results indicate that applying DP-SGD leads to a decrease in both average model utility and MIA attack efficiency. This effect is particularly pronounced under moderate to stricter DP guarantees (e.g., $\epsilon < 25$).

The dynamic setup yields a superior tradeoff between model utility and privacy compared to the static setup, potentially enabling relaxation of the local DP budget. For example, the dynamic setting experiment with $\epsilon = 50$ shows much better average model accuracy and lower MIA attack efficiency than the static setting experiment with $\epsilon = 25$.

**Takeaway:** While ensuring DP has an impact on model utility, using dynamic topologies yields a better MIA vulnerability/model utility tradeoff.

## 4 Mixing with Dynamic Graphs

In Section 3, we demonstrated the benefits of dynamic settings and increased view sizes in improving both privacy and utility. Our intuition is that the mixing properties of the learning protocol play a crucial role in reducing MIA vulnerability. Therefore, in this section, we aim to provide a deeper understanding of how dynamic settings and larger view sizes enhance the mixing rate of models. To investigate the mixing properties, we leverage existing findings on the influence of the spectral properties of the weighted adjacency matrix associated with gossip exchanges [11]. To conduct the analysis, we assume an idealized scenario with synchronous

iterations where all nodes simultaneously wake up to exchange their models and no local updates are performed.

The simplified analysis considers essentially a consensus task. Let $T \in \mathbb{N}$ be a fixed number of iterations. At each (synchronous) iteration $t \in \{1, \dots, T\}$, each node $i \in \mathcal{V}$ performs the following mixing step:

$$ \theta_i^{(t+1)} \leftarrow \frac{1}{k+1} \sum_{j \in \mathcal{N}_i^{(t)} \cup \{i\}} \theta_i^{(t)} \quad (9) $$

Where $\theta_i^{(t)} \in \mathcal{M}$ is the model of node $i$ at iteration $t$ and $\theta_i^{(0)}$ is equal to the initial model $\theta_0 \in \mathcal{M}$. While it is already evident that graphs $G^{(1)}, \dots, G^{(T)}$ defined by neighborhoods $\mathcal{N}_i^{(t)}$ for all $i \in \mathcal{V}$ and $t \in \{1, \dots, T\}$ impact convergence of consensus, this is also holds for decentralized learning [19, 43].

We now analyze mixing properties. For any iteration $t \in \{1, \dots, T\}$, let $\theta^{(t)} \in \mathcal{M}^n$ be the vector whose elements are the models of each node, and $W^{(t)} \in \mathbb{R}^{n \times n}$ be the weighted adjacency matrix (or mixing matrix) of the graph used at iteration $t$. Specifically, for our $k$-regular graphs $W_{i,j}^{(t)} = 1/(k+1)$ if and only if $i$ and $j$ are neighbors or $j = i$, and $W_{i,j}^{(t)} = 0$ else.

Exchanges at Equation (9) can be summarized by

$$ \theta^{(t+1)} \leftarrow W^{(t)} \theta^{(t)}. \quad (10) $$

It has been proven in the work of Boyd et al. [11] that if a mixing matrix $W \in \mathbb{R}^{n \times n}$ is symmetric and doubly stochastic (that is, the sum of the elements of any column or row are equal to 1), then for any vector $\theta \in \mathbb{R}^n$ we have that

$$ \|W\theta - \mathbb{1}\tilde{\theta}\|_2 \le \lambda_2(W) \|\theta - \mathbb{1}\tilde{\theta}\|_2, \quad (11) $$

where $\tilde{\theta} = \frac{1}{n} \sum_{i \in \mathcal{V}} \theta_i$ is the average of the elements of the vector, $\lambda_2(W)$ is the second largest eigenvalue of $W$ and $\mathbb{1}$ is the vector of all ones. Essentially, [11] shows that $\lambda_2(W)$ quantifies the reduction of the distance between values (in our case, models 3) of each node and the average of all values. The smaller $\lambda_2(W)$, the better the mixing. Given that we use undirected $k$-regular graphs and the weights of the exchanges is equal for all edges, we have that for all $t \in \{1, \dots, T\}$, $W^{(t)}$ is doubly stochastic and symmetric and hence we can use Equation (11) in our simplified consensus protocol.

We focus on the mixing properties of the matrix $W^*$ obtained by performing the matrix product $W^{(T)} \dots W^{(1)}$. For any vector $\theta \in \mathbb{R}^n$, $W^*\theta$ is the result of performing the sequence of mixing exchanges determined by $W^{(1)}, \dots, W^{(T)}$ over $\theta$.

An immediate bound given by

$$ \|W^*\theta - \mathbb{1}\tilde{\theta}\|_2 \le \prod_{t=1}^{T} \lambda_2(W^{(t)}) \|\theta - \mathbb{1}\tilde{\theta}\|_2. \quad (12) $$

can be obtained from sequentially applying Equation (11) to each factor $W^{(t)}$ of $W^*$. However, Equation (12) only takes into account individual mixing properties of each $W^{(t)}$ in isolation, ignoring the effects of varying the communication graph.

Since $W^*$ is also doubly-stochastic when $W^{(1)}, \dots, W^{(T)}$ are doubly-stochastic, a more accurate bound can be obtained by directly applying that Equation (11) using $\lambda_2(W^*)$. This captures the

<footnote n="3">³For simplicity, we perform our analysis treating models as scalars. However, if we consider models as vectors of scalars (i.e., $\mathcal{M} = \mathbb{R}^d$ where $d$ is the number of parameters of the model), the current analysis holds for each model parameter.</footnote>
<figure><img src="image_9.png" /><figcaption>Figure 7: Trade-off between MIA vulnerability and Generalization Error across different datasets, comparing Base Gossip and SAMO learning protocols.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Figure 8: Average MIA Accuracy and Generalization Error over communication rounds on Purchase100 dataset using the Merge Once Protocol, with a 2-regular graph of 150 nodes.</figcaption></figure>

overall mixing properties of the graph sequence. In the *static setting*, $W^{(t)} = W$ for all $t \in \{1, \dots, T\}$ and some $W$ initially chosen. In the *dynamic setting* all nodes are randomly permuted at each iteration. It is easy to see that for the static case $\lambda_2(W^*) = \lambda_2(W)^T$, while in the dynamic case the closed form of $\lambda_2(W^*)$ is not evident.

In Figure 10, we empirically show how $\lambda_2(W^*)$ decreases as iterations increase. We show it for the same topologies used in Section 3. All points in the curve are reported over an average of 50 runs of gossip simulation together with their standard deviation. We can see that for the same degree $k$, $\lambda_2(W^*)$ decreases much faster for dynamic graphs than for static graphs. Moreover, the standard deviation is negligible in the dynamic case, indicating that bad mixing scenarios occur with negligible probability.

From this analysis, we can better understand why the dynamic setting is favorable for reducing MIA vulnerability. Models in the dynamic setting tend to align more closely with consensual models and exhibit less bias toward the peculiar model of an individual client, which would otherwise increase MIA vulnerability.

# 5 Summary of Takeaways and Actionable Recommendations

Our experimental analysis provides several important insights into the factors influencing MIA vulnerability and utility in Gossip learning. Below, by summarizing the above takeaways, we offer actionable recommendations for designing privacy-aware decentralized learning architectures:

* **Static vs Dynamic Graph Topologies:** From our study, we recommend using dynamic graph structures to improve model mixing and mitigate privacy risks, especially as the communication network becomes large and sparse. These topologies should be paired with robust peer-sampling protocols. Furthermore, they complement privacy preserving techniques such as DP-SGD since dynamic graph topologies reduce the negative impact on model utility and decrease the vulnerability to MIA attacks.

* **Model Mixing:** Our study shows that incorporating advanced mixing strategies like SAMO reduces individual local biases in models and achieves better privacy-utility tradeoffs.

* **View Size:** While increasing view size is generally beneficial, it should be balanced against the increased communication cost. For practical implementations, we recommend considering dynamic settings with moderate view sizes to maximize efficiency.

* **Addressing Non-i.i.d. Data:** Our study shows the high impact of non-i.i.d. data distribution which asks for stronger protection mechanisms. We also recommend exploring adaptive learning protocols that account for data heterogeneity, such as adapting the ones introduced in FL [81]. For instance, using nodes pre-clustering [12] or knowledge distillation using public data [20].

* **Early Overfitting:** From our study, we recommend focusing on strategies to prevent early overfitting, such as regularization, dynamic learning rates, or delayed aggregation, to limit the persistent impact of initial vulnerabilities.

By implementing these recommendations in completion to privacy-preserving mechanisms (see Section 6.3), decentralized learning
<figure><img src="image_11.png" /><figcaption>Figure 9: Comparison of maximum average MIA Accuracy and TPR@1%FPR with global test accuracy across different DP-SGD privacy budget ($\epsilon$ for $\delta = 10^{-5}$) and topology setups on the Purchase100 dataset using SAMO, with regular graph with 150 nodes.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Figure 10: Values of $\lambda_2(W^*)$ depending on the number of iterations for k-regular graphs (k-reg) of different degrees in the dynamic (Dyn) and static (Stat) settings.</figcaption></figure>

systems can achieve a more favorable balance between utility and privacy, paving the way for safer and more effective collaborative learning environments.

# 6 Related Works

## 6.1 Decentralized Learning

While the literature on decentralized learning is significantly exhaustive, it shares one common objective: tackling the drawbacks of Federated Learning that stem from its centralized nature (e.g., single point of failure, governance, scalability ...). The first such attempts took the form of semi-decentralized FL. For instance, [26] assumes the presence of several servers while [52] envisions a hierarchical workflow consisting of several clusters of clients. While these approaches mitigate some of the previously mentioned drawbacks, they do not fully address issues related to governance and trust. To tackle these concerns, subsequent works have turned towards fully decentralized learning. Recent surveys [8, 83] distinguish these approaches based on (1) the roles played by nodes in the protocol and (2) the communication scheme, both in terms of topology and synchronicity. Regarding the first point, most of the literature centers

around the classical trainer and aggregator roles. In fully decentralized settings such as ours, clients are traditionally assumed to play both roles. This setup is often considered appealing as it enables nodes to retain full control over the training and aggregation processes, including their frequency, allowing them to adapt to their local resources [84]. However, some lines of work also consider the proxy role [40], while works leveraging blockchain systematically introduce a verifier role [65] that is relevant for resilience purposes.

While such additional layers are undoubtedly relevant for bridging the gap between theory and practical implementations, they do not significantly impact the privacy of decentralized learning and can always be integrated on top of protocols like ours. Secondly, the communication scheme varies considerably across frameworks. In terms of topology, most of the literature has focused on fully connected graphs due to their convergence properties, which are equivalent to FL [21, 79], and their resilience to network dynamics (e.g., churn and failures). However, such graphs incur quadratic communication costs [7], making them impractical for large-scale learning and, to some extent, defeating the purpose of decentralization. As a result, there have been increasing efforts toward learning over sparser graphs [25, 34]. In this context, Wu et al. [76] recently demonstrated that d-regular expander graphs are near-optimal both in terms of convergence and generalization. Since building d-regular expanders without global knowledge is non-trivial, dynamic regular graphs are often used instead, an approach we adopt in this work. In fact, such an approach has been shown to achieve graph-size-independent consensus rates [72] (See Section 6.4 for details). Finally, another critical aspect of the communication scheme relates to synchronicity. Much of the literature assumes settings that enforce synchronization points. While this is easier to model and thus attractive from a theoretical standpoint [43], it is suboptimal in real-world, resource-heterogeneous settings, as it introduces unnecessary idle time and slows convergence [13, 84]. In contrast, asynchronism; a core principle of what is commonly termed gossip learning, aligns well with real-world constraints, prompting recent efforts to establish its theoretical foundations [19]. It is from this principle that, in this work, we consider an asynchronous, fully decentralized learning setting, which accounts for practical requirements as well as the emerging necessary theoretical grounds.
## 6.2 Privacy in Decentralized and Gossip Learning.

From location-hiding [22] to source anonymity [5, 38], The inherent privacy properties of gossip protocols have captivated the research community over the past decade. However, studying how these properties translate to a learning protocol remains relatively underexplored. To the best of our knowledge, [62] is the first endeavor in this direction. In this work, the authors demonstrated that decentralized learning is at least as vulnerable as its federated counterpart to reconstruction and membership inference attacks. However, their approach relies on partial model aggregation, which we show leads to worse model mixing and, consequently, to more vulnerable models. Moreover, their results rely on assumptions such as adversaries possessing power equivalent to an FL parameter server in some contexts, or the notion that honest nodes have no honest neighbors within a two-hop radius ( [62], Section 4).

These assumptions are particularly unrealistic for dynamic networks such as ours. Additionally, important characteristics of gossip learning, such as asynchronous communications and the presence of a random peer sampling protocol, are not considered. The threat model of Pasquini et al. [62] was recently extended in [18], where the authors consider adversaries operating beyond their immediate neighborhoods. While their findings are compelling, they remain limited to static and synchronous settings and further assume i.i.d. data distributions. Concurrently, a recent study [36] examines the vulnerability of decentralized learning through a mutual information framework. While their theoretical results regarding the resilience of decentralized learning align with our findings, their analysis is constrained. Specifically, they exclusively consider an i.i.d. setting, with static topologies and synchronous communications. Our work alleviates the assumptions made in these works.

## 6.3 Defenses and Protection Mechanisms

Numerous attempts have been made to enhance privacy in decentralized/gossip learning using various mechanisms. For instance, Cyffers and Bellet [15] proposes a novel formulation of differential privacy (DP) that accounts for the position of nodes within the network. In such an approach, nodes send noisy model updates with formal privacy guarantees. Then Sabater et al. [66] considers a stronger model where adversarial parties can collude and share their view. Moreover, it substantially reduces the perturbations required to achieve privacy using a correlated noise approach. While DP-based solutions are holistic from several perspectives (e.g., adversary knowledge, types of attacks), they often result in significant utility loss [14, 42] and can impede other robustness objectives [23].

Other works explore secure multi-party computation primitives. For example, Dekker et al. [17] proposes a framework where each node performs a sequence of secure aggregations with its neighbors. Although effective, these solutions are often expensive to adapt to more general settings (e.g., Byzantine settings [69]). Finally, another line of research investigates hardware-based approaches, namely, Trusted Execution Environments (TEEs). For instance, GradSec [56] prevents privacy attacks in FL by incorporating sensitive model layers within secure compartments. However, this approach has demanding hardware requirements and has been shown to be vulnerable to side-channel attacks [37]. Overall, while we consider

each of these research directions promising, they are orthogonal to our work, which focuses specifically on understanding the vulnerabilities of models in gossip learning, particularly in relation to dynamicity and mixing rate.

## 6.4 Studying Dynamic Communication Topologies

There are mainly two lines of work that investigate dynamic topologies in gossip learning: (i) works that attempt to quantify the impact of dynamicity on the mixing rate and convergence properties, and (ii) works that propose practical protocols leveraging peer sampling algorithms [4, 31, 32]. In the mixing rate and convergence line of work, several studies have examined the use of dynamic topologies in decentralized learning, with the majority focusing on the synchronous case [59, 72]. A noteworthy example is Epidemic Learning [16], which employs a randomly formed communication topology where, in each round, a node transmits its model to a randomly selected set of peers without coordination between them. The study showed that randomness in peer sampling positively impacts convergence speed, aligning with previous results [72].

Corroborating these findings, many practical peer-sampling protocols have been proposed. For instance, Hegedűs et al. [32] demonstrate empirically that gossip learning, enhanced with a peer sampling protocol, can systematically compete with federated settings across various configurations and hyperparameters (e.g., message loss, data distributions). Similarly, Belal et al. [4] observe that a carefully designed peer sampling protocol can improve model personalization. Overall, while our work aligns to some extent with both research directions, it introduces a novel perspective by establishing a crucial connection between the mixing properties of a practical dynamic protocol and its vulnerability to membership inference attacks. To the best of our knowledge, such a connection has not been explored in prior research.

## 7 Conclusion

In this work, we presented a comprehensive study of MIA vulnerabilities in decentralized learning systems. By analyzing the interplay between model mixing properties, graph dynamics, and data distributions, we provided new insights into the privacy risks inherent in these systems.

Our findings revealed that mixing properties are critical in shaping MIA vulnerabilities. Dynamic graph topologies, coupled with increased view sizes, enhance mixing rates, leading to better privacy-utility tradeoffs. However, the effectiveness of these approaches is constrained when faced with non-i.i.d. data distributions, which amplify MIA risks across all training rounds. Importantly, we demonstrated that early overfitting creates persistent vulnerabilities that cannot be fully mitigated by improving generalization later in the training process.

These insights underline the need for careful design in decentralized learning protocols. By favoring dynamicity and improving model mixing, systems can achieve a better balance between utility and privacy. Future work should focus on developing adaptive mechanisms to address the challenges posed by heterogeneous data
distributions and early overfitting, as well as exploring complementary defense techniques, such as differential privacy and secure multi-party computation.

Through our analysis and experiments, we provide actionable recommendations for building more privacy-resilient decentralized learning systems, paving the way for safer collaborative learning environments.

## Acknowledgments

This work was supported by the French government managed by the Agence Nationale de la Recherche (ANR) through France 2030 program with the reference ANR-23-PEIA-005 (REDEEM project).

## References

[1] Michael Aerni, Jie Zhang, and Florian Tramèr. 2024. Evaluations of Machine Learning Privacy Defenses are Misleading. In *Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security* (Salt Lake City, UT, USA) (CCS '24). Association for Computing Machinery, New York, NY, USA, 1271–1284. doi:10.1145/3658644.3690194

[2] Syreen Banabilah, Mayad Aloqaily, Eitaa Alsayed, Nida Malik, and Yaser Jararweh. 2022. Federated learning review: Fundamentals, enabling technologies, and future applications. *Information processing & management* 59, 6 (2022), 103061.

[3] Paul Baran. 1964. On distributed communications networks. *IEEE transactions on Communications Systems* 12, 1 (1964), 1–9.

[4] Yacine Belal, Aurélien Bellet, Sonia Ben Mokhtar, and Vlad Nitu. 2022. PEPPER: Empowering User-Centric Recommender Systems over Gossip Learning. *Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.* 6, 3, Article 101 (Sept. 2022), 27 pages. doi:10.1145/3550302

[5] Aurélien Bellet, Rachid Guerraoui, and Hadrien Hendrikx. 2020. Who started this rumor? Quantifying the natural differential privacy of gossip protocols. In *34th International Symposium on Distributed Computing (DISC 2020)*. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, Virtual Only.

[6] Aurélien Bellet, Rachid Guerraoui, Mahsa Taziki, and Marc Tommasi. 2017. *Fast and differentially private algorithms for decentralized collaborative machine learning*. Research Report. INRIA Lille. https://inria.hal.science/hal-01665410

[7] Aurélien Bellet, Anne-Marie Kermarrec, and Erick Lavoie. 2022. D-cliques: Compensating for data heterogeneity with topology in decentralized federated learning. In *2022 41st International Symposium on Reliable Distributed Systems (SRDS)*. IEEE, 1–11.

[8] Enrique Tomás Martínez Beltrán, Mario Quiles Pérez, Pedro Miguel Sánchez Sánchez, Sergio López Bernal, Gérôme Bovet, Manuel Gil Pérez, Gregorio Martínez Pérez, and Alberto Huertas Celdrán. 2023. Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges. *IEEE Communications Surveys & Tutorials* 25, 4 (2023), 2983–3013.

[9] Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. 2019. Analyzing Federated Learning through an Adversarial Lens. In *Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97)*. Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Long Beach, California, USA, 634–643. https://proceedings.mlr.press/v97/bhagoji19a.html

[10] Sayan Biswas, Mathieu Even, Anne-Marie Kermarrec, Laurent Massoulié, Rafael Pires, Rishi Sharma, and Martijn de Vos. 2025. Noiseless Privacy-Preserving Decentralized Learning. *Proceedings on Privacy Enhancing Technologies* 2025, 1 (Jan. 2025), 824–844. doi:10.56557/popets-2025-0043

[11] Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah. 2006. Randomized gossip algorithms. *IEEE transactions on information theory* 52, 6 (2006), 2508–2530.

[12] Christopher Briggs, Zhong Fan, and Peter Andras. 2020. Federated learning with hierarchical clustering of local updates to improve training on non-IID data. In *2020 international joint conference on neural networks (IJCNN)*. IEEE, Glasgow, UK, 1–9.

[13] Jing Cao, Zirui Lian, Weihong Liu, Zongwei Zhu, and Cheng Ji. 2021. HADFL: Heterogeneity-aware decentralized federated learning framework. In *2021 58th ACM/IEEE Design Automation Conference (DAC)*. IEEE, 1–6.

[14] Alycia N Carey, Minh-Hao Van, and Xintao Wu. 2024. Evaluating the impact of local differential privacy on utility loss via influence functions. In *2024 International Joint Conference on Neural Networks (IJCNN)*. IEEE, IEEE, Yokohama, Japan, 1–10.

[15] Edwige Cyffers and Aurélien Bellet. 2022. Privacy Amplification by Decentralization. In *Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 151)*. Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera (Eds.). PMLR, A Virtual Conference, 5334–5353. https://proceedings.mlr.press/v151/cyffers22a.html

[16] Martijn De Vos, Sadegh Farhadkhani, Rachid Guerraoui, Anne-Marie Kermarrec, Rafael Pires, and Rishi Sharma. 2023. Epidemic Learning: Boosting Decentralized Learning with Randomized Communication. In *Advances in Neural Information Processing Systems*, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.). Vol. 36. Curran Associates, Inc., New Orleans, LA, USA, 36132–36164. https://proceedings.neurips.org/paper_files/paper/2023/file/7172e147d916ee4c1b1e30016ce725f-Paper-Conference.pdf

[17] Florine W Dekker, Zekeriya Erkin, and Mauro Conti. 2025. Topology-Based Reconstruction Prevention for Decentralised Learning. *Proceedings on Privacy Enhancing Technologies* (2025).

[18] Abdellah El Mrini, Edwige Cyffers, and Aurélien Bellet. 2024. Privacy attacks in decentralized learning. In *Proceedings of the 41st International Conference on Machine Learning (ICML'24)*. JMLR.org, Vienna, Austria, Article 1481, 15 pages.

[19] Mathieu Even, Anastasia Koloskova, and Laurent Massoulie. 2024. Asynchronous SGD on Graphs: a Unified Framework for Asynchronous Decentralized and Federated Optimization. In *Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 238)*. Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li (Eds.). PMLR, Palau de Congressos, Valencia, Spain, 64–72. https://proceedings.mlr.press/v238/even24a.html

[20] Xiuwen Fang and Mang Ye. 2022. Robust federated learning with noisy and heterogeneous clients. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. IEEE, New Orleans, LA, USA, 10072–10081.

[21] Evangelos Georgatos, Christos Mavrokefalidis, and Kostas Berberidis. 2022. Efficiently fully distributed federated learning with adaptive local links. *arXiv preprint arXiv:2203.12281* (2022).

[22] Karol Gotfryd, Marek Klonowski, and Dominik Pajak. 2017. On Location Hiding in Distributed Systems. In *Structural Information and Communication Complexity*, Shantanu Das and Sebastien Tixuil (Eds.). Springer International Publishing, Cham, 174–192.

[23] Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, Sébastien Rouault, and John Stephan. 2021. Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?. In *Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing* (Virtual Event, Italy) (PODC'21). Association for Computing Machinery, New York, NY, USA, 391–401. doi:10.1145/3465084.3467919

[24] Rachid Guerraoui, Anne-Marie Kermarrec, Anastasiia Kucherenko, Rafael Pinot, and Martijn de Vos. 2024. Peerswap: A Peer-Sampler with Randomness Guarantees. In *2024 43rd International Symposium on Reliable Distributed Systems (SRDS)*. IEEE, Charlotte, NC, USA, 271–281. doi:10.1109/SRDS64841.2024.00034

[25] Shunxin Guo, Jiaqi Lv, and Xin Geng. 2025. Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning. *arXiv preprint arXiv:2504.19103* (2025).

[26] Yeting Guo, Fang Liu, Zhiping Cai, Hui Zeng, Li Chen, Tongqing Zhou, and Nong Xiao. 2021. PREFER: Point-of-interest Recommendation with efficiency and privacy-preservation via Federated Edge Learning. *Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies* 5, 1 (2021), 1–25.

[27] Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. 2019. Federated Learning for Mobile Keyboard Prediction. *arXiv:1811.03604* [cs.CL] https://arxiv.org/abs/1811.03604

[28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In *Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV '15)*. IEEE Computer Society, USA, 1026–1034. doi:10.1109/ICCV.2015.123

[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In *Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR '16)*. IEEE, Las Vegas, NV, USA, 770–778. doi:10.1109/CVPR.2016.90

[30] István Hegedüs, Gábor Danner, and Márk Jelasity. 2019. Gossip Learning as a Decentralized Alternative to Federated Learning. In *Distributed Applications and Interoperable Systems (Lecture Notes in Computer Science, Vol. 11665)*. IFIP, Springer, Copenhagen, Denmark, 74–90. doi:10.1007/978-3-030-22496-7_5

[31] István Hegedüs, Gábor Danner, and Márk Jelasity. 2019. Gossip learning as a decentralized alternative to federated learning. In *Distributed Applications and Interoperable Systems: 19th IFIP WG 6.1 International Conference, DAIS 2019, Held as Part of the 14th International Federated Conference on Distributed Computing Techniques, DisCoTec 2019*. Springer, Springer, Kongens Lyngby, Denmark, 74–90.

[32] István Hegedüs, Gábor Danner, and Márk Jelasity. 2021. Decentralized Learning Works: An Empirical Comparison of Gossip Learning and Federated Learning. *J. Parallel and Distrib. Comput.* 148 (Feb. 2021), 109–124. doi:10.1016/j.jpdc.2020.10.006

[33] Hadrien Hendrikx, Francis Bach, and Laurent Massoulié. 2019. Accelerated Decentralized Optimization with Local Updates for Smooth and Strongly Convex Objectives. In *Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 89)*. Kamalika Chaudhuri and Masashi Sugiyama (Eds.). PMLR, 897–906. https://proceedings.mlr.press/v89/hendrikx19a.html
[34] Yifan Hua, Kevin Miller, Andrea L Bertozzi, Chen Qian, and Bao Wang. 2022. Efficient and reliable overlay networks for decentralized federated learning. *SIAM J. Appl. Math.* 82, 4 (2022), 1558–1586.

[35] Márk Jelasity, Spyros Voulgaris, Rachid Guerraoui, Anne-Marie Kermarrec, and Maarten Van Steen. 2007. Gossip-based peer sampling. *ACM Transactions on Computer Systems (TOCS)* 25, 3 (2007), 8-es.

[36] Changlong Ji, Richard Heusdens, Stephane Maag, and Qiongxiu Li. 2025. Re-Evaluating Privacy in Centralized and Decentralized Learning: An Information- Theoretical and Empirical Study. In *ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*. IEEE, Hyderabad, India, 1–5. doi:10.1109/ICASSP49660.2025.10888257

[37] Jianyu Jiang, Claudio Soriente, and Ghassan Karame. 2022. On the Challenges of Detecting Side-Channel Attacks in SGX. In *Proceedings of the 25th International Symposium on Research in Attacks, Intrusions and Defenses (Limassol, Cyprus) (RAID '22)*. Association for Computing Machinery, New York, NY, USA, 86–98. doi:10.1145/3545984.3545972

[38] Richeng Jin, Yufan Huang, Zhaoyang Zhang, and Huaiyu Dai. 2023. On the privacy guarantees of gossip protocols in general networks. *IEEE Transactions on Network Science and Engineering* 10, 6 (2023), 3114–3130.

[39] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Ben- nis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2021. Advances and open problems in federated learning. *Foundations and trends® in machine learning* 14, 1–2 (2021), 1–210.

[40] Shivam Kalra, Junfeng Wen, Jesse C Cresswell, Maksims Volkovs, and Hamid R Tizhoosh. 2023. Decentralized federated learning through proxy model sharing. *Nature communications* 14, 1 (2023), 2899.

[41] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, and Ananda Theertha Suresh. 2020. SCAFFOLD: stochastic controlled averaging for federated learning. In *Proceedings of the 37th International Conference on Machine Learning (ICML'20)*. JMLR.org, Virtual Only (formerly Vienna), Article 476, 12 pages.

[42] Muah Kim, Onur Günlü, and Rafael F Schaefer. 2021. Federated learning with local differential privacy: Trade-offs between privacy, utility, and communication. In *ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*. IEEE, Toronto, ON, Canada, 2650–2654.

[43] Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, and Sebastian Stich. 2020. A Unified Theory of Decentralized SGD with Changing Topology and Local Updates. In *Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119)*, Hal Daumé III and Aarti Singh (Eds.). PMLR, Virtual Only (formerly Vienna), 5381–5393. https://proceedings.mlr.press/v11/koloskova20a.html

[44] Noam Koren, Abigail Goldsteen, Guy Amit, and Ariel Farkash. 2025. Membership Inference Attacks Against Time-Series Models. In *Proceedings of the 16th Asian Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 260)*, Vu Nguyen and Hsuan-Tien Lin (Eds.). PMLR, Hanoi, Vietnam, 319–334. https://proceedings.mlr.press/v260/koren25a.html

[45] Alex Krizhevsky and Geoffrey Hinton. 2009. *Learning multiple layers of features from tiny images*. Technical Report 0. University of Toronto, Toronto, Ontario. https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf

[46] Chengxi Li, Gang Li, and Pramod K. Varshney. 2022. Decentralized Federated Learning via Mutual Knowledge Transfer. *IEEE Internet of Things Journal* 9, 2 (Jan. 2022), 1136–1147. doi:10.1109/IOT.2021.3078543

[47] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. 2022. Federated Learning on Non-IID Data Silos: An Experimental Study. In *2022 IEEE 38th International Conference on Data Engineering (ICDE)*. IEEE Computer Society, Los Alamitos, CA, USA, 965–978. doi:10.1109/ICDE53745.2022.00077

[48] Qinbin Li, Bingsheng He, and Dawn Song. 2021. Model-contrastive federated learning. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. IEEE, Nashville, TN, USA, 10713–10722.

[49] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated learning: Challenges, methods, and future directions. *IEEE signal processing magazine* 37, 3 (2020), 50–60.

[50] Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. 2017. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. In *Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS'17)*. Curran Associates Inc., Red Hook, NY, USA, 5336–5346.

[51] Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. 2017. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. *Advances in neural information processing systems* 30 (2017).

[52] Frank Po-Chen Lin, Seyyedali Hosseinalipour, Sheikh Shams Azam, Christopher G Brinton, and Nicolo Michelusi. 2021. Semi-decentralized federated learning with cooperative D2D local model aggregations. *IEEE Journal on Selected Areas in Communications* 39, 12 (2021), 3851–3869.

[53] NA Lynch. 1996. *Distributed Algorithms*. Morgan Kaufmann Publishers.

[54] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Y. Arcas. 2017. Communication-Efficient Learning of Deep Networks from Decentralized Data. In *Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 54)*, Aarti Singh and Jerry Zhu (Eds.). PMLR, Fort Lauderdale Florida, USA, 1273–1282.

[55] Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2019. Exploiting unintended feature leakage in collaborative learning. In *2019 IEEE symposium on security and privacy (SP)*. IEEE, IEEE, San Francisco, CA, USA, 691–706.

[56] Aghiles Ait Messaoud, Sonia Ben Mokhtar, Vlad Nitu, and Valerio Schiavoni. 2021. GradSec: a TEE-based Scheme Against Federated Learning Inference Attacks. In *Proceedings of the First Workshop on Systems Challenges in Reliable and Secure Federated Learning* (Virtual Event, Germany) (ResilientFL '21). Association for Computing Machinery, New York, NY, USA, 10–12. doi:10.1145/3477114.3488763

[57] Ilya Mironov. 2017. Rényi Differential Privacy. In *2017 IEEE 30th Computer Security Foundations Symposium (CSF)*. 263–275. doi:10.1109/CSF.2017.11

[58] Milad Nasr, Reza Shokri, and Amir Houmansadr. 2019. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In *2019 IEEE symposium on security and privacy (SP)*. IEEE, IEEE, San Francisco, CA, USA, 739–753.

[59] Angelia Nedić and Alex Olshevsky. 2015. Distributed Optimization Over Time-Varying Directed Graphs. *IEEE Trans. Automat. Control* 60, 3 (2015), 601–615. doi:10.1109/TAC.2014.2364096

[60] Julien Nicolas, César Sabater, Mohamed Maouche, Sonia Ben Mokhtar, and Mark Coates. 2024. Differentially private and decentralized randomized power method. arXiv:2411.01931 [cs.LG] https://arxiv.org/abs/2411.01931

[61] Róbert Ormándi, István Hegedűs, and Márk Jelasity. 2013. Gossip learning with linear models on fully distributed data. *Concurrency and Computation: Practice and Experience* 25, 4 (2013), 556–571. doi:10.1002/cpe.2858

[62] Dario Pasquini, Mathilde Raynal, and Carmela Troncoso. 2023. On the (In)security of Peer-to-Peer Decentralized Machine Learning. In *2023 IEEE Symposium on Security and Privacy (SP)*. IEEE, San Francisco, CA, USA, 418–436. doi:10.1109/SP46215.2023.10179291

[63] Mirko Polato. 2024. makgyver/gossipy. https://github.com/makgyver/gossipy

[64] Pian Qi, Diletta Chiaro, Antonella Guzzo, Michele Ianni, Giancarlo Fortino, and Francesco Piccialli. 2024. Model aggregation techniques in federated learning: A comprehensive survey. *Future Generation Computer Systems* 150 (2024), 272–293.

[65] Youyang Qu, Md Palash Uddin, Chenquan Gan, Yong Xiang, Longxiang Gao, and John Yearwood. 2022. Blockchain-enabled federated learning: A survey. *Comput. Surveys* 55, 4 (2022), 1–35.

[66] César Sabater, Aurélien Bellet, and Jan Ramon. 2022. An accurate, scalable and verifiable protocol for federated differentially private averaging. *Machine Learning* 111, 11 (2022), 4249–4293.

[67] Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. 2019. ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. In *NDSS. The Internet Society, San Diego, California, USA*. http://dblp.uni-trier.de/db/conf/ ndss/ndss2019.html#Salem0HBF019

[68] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership Inference Attacks Against Machine Learning Models. In *2017 IEEE Symposium on Security and Privacy (SP)*. IEEE, IEEE Computer Society, San Jose, CA, USA, 3–18. doi:10.1109/SP.2017.41

[69] Jinhyun So, Başak Güler, and A Salman Avestimehr. 2020. Byzantine-resilient secure federated learning. *IEEE Journal on Selected Areas in Communications* 39, 7 (2020), 2168–2181.

[70] Liwei Song and Prateek Mittal. 2021. Systematic Evaluation of Privacy Risks of Machine Learning Models. In *30th USENIX Security Symposium (USENIX Security 21)*. USENIX Association, Online, 2615–2632. https://usenix.org/conference/ usenixsecurity21/presentation/song

[71] Liwei Song, Reza Shokri, and Prateek Mittal. 2019. Membership Inference Attacks Against Adversarially Robust Deep Learning Models. In *2019 IEEE Security and Privacy Workshops (SPW)*. IEEE Computer Society, Los Alamitos, CA, USA, 507–56. doi:10.1109/SPW.2019.00021

[72] Zhuoqing Song, Weijian Li, Kexin Jin, Lei Shi, Ming Yan, Wotao Yin, and Kun Yuan. 2022. Communication-Efficient Topologies for Decentralized Learning with O(1) Consensus Rate. In *Advances in Neural Information Processing Systems*, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., Long Beach, California, USA, 1073–1085. https://proceedings.neurips.cc/paper_files/paper/2022/file/ 0790ef700dd0072f4940abda9b7d0005-Paper-Conference.pdf

[73] Robbert Van Renesse, Kenneth P Birman, and Werner Vogels. 2003. Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining. *ACM transactions on computer systems (TOCS)* 21, 2 (2003), 164–206.

[74] Spyros Voulgaris, Daniela Gavidia, and Maarten Van Steen. 2005. Cyclon: Inexpensive membership management for unstructured p2p overlays. *Journal of Network and systems Management* 13 (2005), 197–217.
[75] Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K Leung, Christian Makaya, Ting He, and Kevin Chan. 2019. Adaptive federated learning in resource constrained edge computing systems. *IEEE journal on selected areas in communications* 37, 6 (2019), 1205-1221.

[76] Jiajun Wu, Fan Dong, Henry Leung, Zhuangdi Zhu, Jiayu Zhou, and Steve Drew. 2024. Topology-aware federated learning in edge computing: A comprehensive survey. *Comput. Surveys* 56, 10 (2024), 1-41.

[77] Pengjun Wu, Zhanzhi Zhang, Xueyi Peng, and Ran Wang. 2024. Deep learning solutions for smart city challenges in urban development. *Scientific Reports* 14, 1 (2024), 5176.

[78] Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. *Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms*. arXiv:1708.07747 [cs.LG]

[79] Yue Xiao, Yu Ye, Shaocheng Huang, Li Hao, Zheng Ma, Ming Xiao, Shahid Mumtaz, and Octavia A Dobre. 2021. Fully decentralized federated learning-based on-board mission for UAV swarm system. *IEEE Communications Letters* 25, 10 (2021), 3296-3300.

[80] Gang Yan, Hao Wang, and Jian Li. 2022. Seizing Critical Learning Periods in Federated Learning. *Proceedings of the AAAI Conference on Artificial Intelligence*

36, 8 (Jun. 2022), 8788-8796. doi:10.1609/aaai.v36i8.20859

[81] Mang Ye, Xiuwen Fang, Bo Du, Pong C Yuen, and Dacheng Tao. 2023. Heterogeneous federated learning: State-of-the-art and research challenges. *Comput. Surveys* 56, 3 (2023), 1-44.

[82] Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. 2018. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. In *2018 IEEE 31st Computer Security Foundations Symposium (CSF)*. IEEE Computer Society, Los Alamitos, CA, USA, 268-282. doi:10.1109/CSF.2018.00027

[83] Liangqi Yuan, Ziran Wang, Lichao Sun, Philip S Yu, and Christopher G Brinton. 2024. Decentralized federated learning: A survey and perspective. *IEEE Internet of Things Journal* (2024).

[84] Shahryar Zehtabi, Seyyedali Hosseinalipour, and Christopher G Brinton. 2022. Decentralized event-triggered federated learning with heterogeneous communication thresholds. In *2022 IEEE 61st Conference on Decision and Control (CDC)*. IEEE, 4680-4687.

[85] Jingwen Zhang, Jiale Zhang, Junjun Chen, and Shui Yu. 2020. GAN Enhanced Membership Inference: A Passive Local Attack in Federated Learning. In *ICC 2020 - 2020 IEEE International Conference on Communications (ICC)*. IEEE, Dublin, Ireland, 1-6. doi:10.1109/ICC40277.2020.9148790