Graphs Help Graphs: Multi-Agent Graph Socialized Learning
Authors: Jialu Li
Yu Wang
Pengfei Zhu
Wanyu Lin
Xinjie Yao
Qinghua Hu
Venue: NeurIPS 2025 poster ()
Keywords: Graph Neural Network
Socialized Learning
Graph Collaborative Learning
Graph Lifelong Learning
Primary area: general_machine_learning

Abstract: Graphs in the real world are fragmented and dynamic, lacking collaboration akin to that observed in human societies. Existing paradigms present collaborative information collapse and forgetting, making collaborative relationships poorly autonomous and interactive information insufficient. Moreover, collaborative information is prone to loss when the graph grows. Effective collaboration in heterogeneous dynamic graph environments becomes challenging. Inspired by social learning, this paper presents a Graph Socialized Learning (GSL) paradigm. We provide insights into graph socialization in GSL and boost the performance of agents through effective collaboration. It is crucial to determine with whom, what, and when to share and accumulate information for effective GSL. Thus, we propose the ''Graphs Help Graphs'' (GHG) method to solve these issues. Specifically, it uses a graph-driven organizational structure to select interacting agents and manage interaction strength autonomously. We produce customized synthetic graphs as an interactive medium based on the demand of agents, then apply the synthetic graphs to build prototypes in the life cycle to help select optimal parameters. We demonstrate the effectiveness of GHG in heterogeneous dynamic graphs by an extensive empirical study. The code is available through https://github.com/Jillian555/GHG.

Graphs Help Graphs: Multi-Agent Graph Socialized Learning
Jialu Li jialuli@tju. edu.
Yu Wang 1.23, wang. yu@tju. edu, en
22,,. 1,23,, zhupengfei@tju.. edu cn
Pengfei Zhu
edu. en
Wanyu Lin' van-yu.lin@polppr edw.ppoolluuuuu........ edu. hk
Xinjie Yao 12.3
Qinghua Hu
Hu2.2,3 1.2,3
yaoxinjiestju. edu. en
huqinghuaSt ju edu. cn
CCollege of Intelligence and Computing, Tianjin University, Tianjin, China 'Enginerring Research Center of City Intelligence and Digital Governance, Ministry of Education of the People's Republic of China, Tianjin, China Haihe Lab of ITAI, Tianjin, China Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China
Abstract
Graphs in the real world are fragmented and dynamic, lacking collaboration akin to that observed in human societies. Existing paradigms present collaborative informa- tion collapse and forgetting, making collaborative relationships poorly autonomous and interactive information insufficient. Moreover, collaborative information is prone loss when the graph grows. Effective collaboration in heterogeneous dy- namic graph environments becomes challenging. Inspired by social learning. this paper presents a Graph Socialized Learning (GSL) paradigm. We provide insights into graph socialization in GSL and boost the performance of agents through effec- tive collaboration. is crucial to determine with whom, what, and when to share and accumulate information for effective GSL. Thus, propose the "Graphs Help Graphs" (GHG) method to solve these issues. Specifically, it uses graph-driven organizational structure select interacting agents and manage interaction strength autonomously. We produce customized synthetic graphs as an interactive medium based on the demand of agents, then apply the synthetic graphs to build prototypes in the life cycle to help select optimal parameters. We demonstrate the effectiveness of GHG in heterogeneous dynamic graphs by an extensive empirical study. The code is available through htttt:://// //gi eigithuu.....
Introduction
Graphs have become powerful tool for capturing complex relationships between real-world entities, leading to their widespread use in social networks (12) recommendation systems [3] and bioin- formatics [4]. However, real-world graphs tend to be fragmented and dynamic, and there often a lack of collaboration between different graphs. From the emergent behavior of ant colonies to the human collective intelligence, collaboration common across different societies S 6 It enables individuals to form social relationships and accomplish complex tasks. Inspired by social learning in human society, socialized learning has been extensively explored [ .. aims to have multiple agents interact, collaborate, and share knowledge. This acquired knowledge benefits other agents attempting to learn different yet related tasks, demonstrating capabilities beyond a single agent.
Yu Wang and Pengfei Zhu are both the corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Time
Time
Parameters
Before Afler
Heforr
After
Who? What?
Before
Paramctors
Information Collapse
Information
Information Forgetting
When!
Sharing and
Accumulation
A A/
Old
Old
New
A
Ao
A2
Old
Old
New
(a) Graph Federated Learning (b) Graph Lifelong Learinng
(c) Graph Socialized Learning
Figure 1: Comparison of different learning paradigms.
Existing paradigms strive to learn new graph knowledge collaboratively, just as human societies do.
As shown in Figure @ the star graph with a central server in graph centralized federated learning [9]
10 I [2] and the undirected topology in graph decentralized federated learning [3]
14
[5]
16
Decentralized methods address communication bottlenecks and system crash risks of centralized
methods 177. Despite their effectiveness, on the one hand, they rely on undirected symmetric communication topologies. where agents fail to capture agent dependencies accurately. On the other hand, merely interacting with parameters may not lead to knowledge compatibility, causing information collapse and difficulty in obtaining capabilities from heterogeneous agents. Additionally, the graph lifelong learning paradigm [18 19 20, 21 [22] can adapt to dynamically growing graphs in the same parameter space for single-agent setups, but partial old task information is forgotten, causing agents to lose their initial performance as training continues. Thus, current paradigms are limited in achieving effective collaboration in heterogencous dynamic graph environments.
Consider a real-world scenario: conducting collaborative training across growing citation networks on different platforms. Sharing elements like titles, author information, and citation relationships can boost the efficiency of academic paper retrieval, By re-visiting existing paradigms, we find that two significant challenges still exist: (1) Effectively managing graph heterogeneity among agents, thereby building autonomous collaborative relationships and sharing sufficient information to avoid collaborative information collapse. (2) Accurately capturing task dynamics in dynamic graphs to prevent the risk of forgetting accumulated collaborative information. Therefore, we raise the question: 1. How to realize effective collaboration in heterogeneous dynamic graph scenarios?
As shown in Figure []] we establish the Graph Socialized Learning (GSL) paradigm. It offers an effective way for agents to share and accumulate graph information in heterogencous dynamic environments. This enables agents to learn the abilities of other agents and preserve old information without loss through effective collaboration. This exploration leads us to consider: 2. How to determine with whom, what, and when to share and accumulate graph information accomplish effective GSL?
Based on property and generalization of graph socialization, we propose the method "Graphs Help Graphs" (GHG) to accomplish GSL from three Wlements-Whr (Organizational Structure): We employ graph-drive organizational structure to autonomously perceive collaborative relationship and strength among agents, thereby avoiding low-quality and redundant information from collaborators. Graph collaboration module models the collaboration graph by leveraging complementarity. as well as parameters (or features) and structure similarity between agents. What (Interactive Medium): We generate customized synthetic graphs as an interactive medium for information exchange. This enables the generation of the needed graphs based on agent states, thus addressing the issue of insufficient information from collaborators. A fine-grained customized collaboration evaluates the demand for different classes between agents. When (Life Cycle): We apply synthetic graphs to construct task prototypes, facilitating collaboration using optimal agent parameters at distinct tasks and preventing collaborative information from being forgotten. Our method achieves outstanding performance on seven datasets, surpassing the methods based on existing paradigms. Our main contributions can be summarized as follows:
We present practical learning paradigm called Graph Socialized Learning (GSI.), enabling each agent's 'ss growth via collaborative interaction.

---

Graph-driven organizational structure, customized interactive medium, and prototype-baser life cycle form three key elements of socialized collaboration.
Our method consistently achieves performance improvements on multiple datasets and demonstrates the effectiveness of all components.
2 Related Work
2.1 Graph Federated Learning
Graph Federated Learning (GFL) collaborative learning paradigm for graph neural networks that enables multiple agents to jointly train models by sharing model parameters 23 24 From the existence of centralized servers, GFL is divided into centralized graph federated learning (C-GFL) and decentralized graph federated learning (D-GFL.. C-GFL relies on a server to aggregate model
parameters, coordinating cross-client model training ( 10 12 25 26]. In contrast. D-GFL enables peer-to-peer parameter interaction by constructing communication topology [4 15 [6].
The server may be impractical in real-world collaborative training, and C-GFL can break down due to server failure or unreliability. The symmetric topology in D-GFL fails to capture agent dependencies. Moreover, model parameters can obtain sufficient interactive information. Our method solves these problems via graph-driven organizational structure and customized synthetic graph generation.
2.2 Graph Lifelong Learning
Graph Lifelong Learning (GLL) is an evolution paradigm that addresses the stable-plasticity dilemma,
requiring agents to learn new skills while retaining prior abilities [27 28 29 Existing methods can
be categorized into three types: Replay- Replay-based methods retain small number of nodes or subgraphs
from old classes, either use generator- generator-produced graphs, to prevent forgetting [19 21 22 30 31).
Eegulaaizzaionn-asee approaches introduce regularization terms into the loss function to enhance the preservation of prior knowledge [8
35). Parameter isolation methods fully or partially
preserve parameters of different tasks via approaches such as dynamic incremental feature extractors,
preventing interference between old and new information 22 36).
The proposed method captures more accurate task prototypes by interacting with synthetic graph information, preventing information forgetting and achieving excellent performance.
3 Problem Analysis
3.1 Problem Formulation
In heterogeneous dynamic multi-agent graph systems, consider the A-agent set, each agent receives graph data G at task f, which consists of a node set V and structure a A node is associated with node features X and labels Y?. Multiple agents jointly solve a supervised node classification task containing C² classes at task t, so all agents use the same test sets. The graph data G is heterogeneous distributed between agents, with overlapping classes, i.e., (Y² n Y # a + b). New graph data constantly emerges until time T, and each agent needs to learn new knowledge while retaining knowledge from previous tasks. Notably, old tasks are inaccessible and new and old tasks
have non-overlapping classes, i.e., (Y' nY'' = 0 1 t'), The relationships between agents can be represented by directed weighted graph G (A' EB,B", B3, M'), where A' is the set of agents, Ex denotes the edges between agents, B is structure, and M E RAXA represents edge weights. For each agent at task the goal is collaborate with other agents, share partial information, and learn graph neural network model that can distinguish new classes from previous classes, expressed as:
0% Fl (Gr, (
bJ 1 19t)
where set of model parameters 0:: = (0' Sa S A,1 < 5/5 T) are estimated, R is denoted as the information transmitted from agent b agent a. The model is updated through the function Ft based training data G and information from external agents.

---

3.2 Graph Socialization Property
Graph Sociability. Inspired by prior works 20
37
we provide graph sociability prediction for
agents as follows:
Yoraaa Gun, G.A) vo
Yonta Gua, 0r
where 0 = arg max IIA
( GIAA i arg max p (t Gua).
(2)
Proliction
Postetix
Tark
where Yiest.a is prediction results of test graph Geest and & denotes optimal task ID. This indicates that effective GSL can be achieved when precise intra-task node prediction, effective collaboration, and accurate task ID query are implemented.
Parameter Posterior. Due to complex correlations between agents, we model graph socialized collaboration by maximizing posterior distribution (0 G of model parameters, rather
than simply employing the likelihood function p (G1:7 (:A) [38] According to Bayes' rule, the (
posterior distribution can be decomposed as p (0]:4 G.A) x p 0::)) p (01:)) where the
first probability represents the likelihood of agent model parameters on their respective graph, while the second probability the prior distribution of parameters, which describes the collaborative patterns between agents.
We assume that the graph arrives independently at each agent and satisfies conditional in- dependence, so we get p (G (1:4) = I p 0%), and - llogp (G¹, (0) corresponds to the loss of agent's model parameters 0% on graph G In addition, we capture the pairwise correlations between multiple agent model parameters through collaborative graph optimization [39 [4).. Agents benefit from learning from others who are both com- plementary and similar, with greater weight given to those who are more complementary and similar. The prior distribution of model parameters can be expressed as logg (f%) x
Ed No (c% (EA,, Mabbf; G²) + we
+ Ed (Mas; G GT)) -ws Σx S (Mas; 8 f
where the relative size agents is denoted by N = |Va/NN and N the number of nodes of all agents. Cg (-) represents the empirical risk of its local graph. C and S measures complementarity and similarity between agents, as well as we and wi are weights to balance them.
Given the ambiguous nature of an agent's empirical loss, the relative graph size is adopted as a surrogate because graph size is key measure of credibility. The posterior distribution maximization of parameters max (0::7
G1:T) can be rewritten as:
Σ t= a=1
logp (Ga, (0)) +
Σ
(MLL
IV ) + wwe (M Gh, Gb) NO
min
b=1
A
-ws8 (Mari 0z: 03:
s.t.
Σ
Mas 1, Va; Ma 2 0, Va, b.
Methodology
We present single-agent graph learning to acquire outputs (Section 4.1 The proposed GHG consists of three phases: graph-driven organizational structure (Section 4.2) customized interactive medium (Section 4.3) and prototype-basee socialization (Section 4.5) is analyzed to provide insights into socialized collaboration. The overview of the proposed GHG is depicted in Figure 2
life cycle (Section 4.4). Morcover, the generalization of graph
The pseudo-code can be found in Appendix C11
4.1 Single-Agent Graph Learning
As stated in prompt-based graph learning method 20 41,, the graph prompts are represented as &¹ = = ..... ot E rxd for task f, where is the number of token vectors of and d' is the dimension of features. Node attributes are enhanced through weighted combinations of these tokens. The enhanced graph representations then passed through frozen pre-trained GNN model fa()) to generate node embeddings. Node classification loss for each agent is defined as:
ft
Σ iEV!
CE (F'. (G.), Vo.)) .V.)). Ft. (Gt) = 1 (Ja (o X +b)),
(4)
IV

---

Graph-Driven Organizational Structure
Oviginal Graph
Outpuss
Representative
Complemettaniy
Complemantuuttt
Subagaze
Le
Frese Course Finc
Similarity
Similazity
L
Collaburative
9.29
Clase
Class
Strength
Multi Agent
Synthetic Graphs
Outputs
Edge Weights Collaborative Groph.
Collaboritive Graph
Edge Weights
Patc
Collabontive Node
Eyn
Castomized Node
Label Distmbutton
pi
Pro
Clei
Label Distribution
P
Synthetic Graphs
Prototype
Pre
Cle
Synthetic Graph
...
Task Prototype Pool
Neighter Graph
Original Node
Time
Prototype-Batte Based Life Cycle
Customized Interactive Medium
Label Distabutton
Figure 2: overview the GHG method. Firstly, the collaborative graph is built by calculating complementarity and similarity. Next, coarse-grained agent collaboration is used to get collaborative strength for information transfer, while fine-grained customized collaboration aids in initializing label distribution of synthetic graphs. Then, synthetic graphs generated through distribution alignment. Finally, task prototypes are added gradually, and the task ID is queried to select optimal parameters.
where V is the training of G CE is the cross-entropy loss, Y is the predicted label, Ft denotes node classification function, and hy represents MLP-based classification head. The graph prompts { ...., x²) and classifiers {h' ...., h²" learn task-specific information while the frozen GNN leverages general cross-task knowledge acquired from the first task.
4.2 Graph-Driven Organizational Structure
Complementarity and Similarity Calculation. We employ singular value decomposition on agent outputs to indirectly capture the complementarity of the graph, i.e. F (.) U (V%) The matrix Ug holds the singular vectors of agent outputs, representing directions in the feature space. We treat the first & columns of Uh as the representative subspace. To measure the complementarity between two agents, we apply the average principal angles cos 3 between their corresponding subspaces. The complementarity metric is defined as:
C (Mabi Gh: G8) = Mrs cos
1L K
E K
max wEU REUM
u. K
k.
(5)
A large principal angle indicates high complementarity, whereas angle near 7/2 suggests subspaces are nearly orthogonal, reflecting substantial divergence in their feature spaces. To integrate topological structure and node information, we perform Laplacian smoothing [20] on the graph G to obtain topology-awe aware node embeddings:
1 (I-( - 1. (b')
))) x (D (B:)=
where is the number of Laplacian smoothing steps, I is an identity matrix, L' = Dí - a denotes graph Laplacian matrix of á af and (D. E, d:z)) is the diagonal degree matrix of a The parameters are enhanced by concatenating model parameters 0g { } and the mean of topology-awer node embeddings ( to promote structural information sharing. We utilize cosine distance between the enhanced parameters of agents to quantify similarity, formulated as:
O O
(M,, OM; 0 M |
O =0" MEAN( (..
|
5

---

Coarse-Grained Agent Collaboration. We present edge weights of the collaboration graph be- tween agents derived through coarse-grand gausee-rained collaboration, which are specified as follows:
[V No
min M
Mab
wcl (Masi G2 G wSS (Mas Oy; 0i)
(8)
A
s.L.
Mab ab = Va; Mab 2 Va,
b=1
Gt where -a.==1- - 11 -= ab represents the synthetic graph (graph synthesis is described in Section 4.3 of neighhor agents N₂ received by agent a. We calculate the relative size, complementarity. and similarity of agents based on the number of nodes, subspace representations, and enhanced parameters. sparse collaborative graph is generated based edge weights and participation ratio ò, represented as BBb = 1 6b E TOPK A, M [a, :]]]-
The denotes the joint distribution of the current task, making it an ideal graph for information transfer. In addition to the cross-entropy loss, we incorporate the KL. divergence loss to make the agent imitate the predictions of neighboring agents. The overall loss function for agent at task is defined as:
Call. Lee (e.(F.. F2 (G. Y. VY.
Wil
MarCu (F% (Gass) Fb
(Gab))
(9)
where Why is the loss weight of Lu-- The collaborative graph edge weight M serving measure of collaborative strength, is used as a weight to transfer information from others to specified agent.
4.3 Customized Interactive Medium
Fine-Grained Customized Collaboration. To generate sufficient interactive information, we pro- pose fine-grained collaboration, customizing the initial label distribution of synthetic graphs generated by neighboring agents based on their current states. Generally, the relative size of class counts is an important metric. We compute M for each class the current task by considering relative size, subspace representations, and enhanced embeddings between agents. The enhanced embeddings are concatenated by the mean of outputs and the mean of topology-aware node embeddings. The number of nodes in class of synthetic graph
ab.c generated by agent b for agent a is denoted as follows:
Vitel N
+ Weot Woot Mt (ab.e) ? V%
V ab.o
(10)
N
where N is the sum of the number of nodes of all agents in class G, Weal denotes the contribution of fine-grained collaborative graph edge weights, N is the sum of allocation proportions of all classes in agent 6, and y is the compression ratio.
Synthetic Graph Generation. We apply synthetic graphs as medium for information exchange. Mean and standard deviation statistics for embeddings can effectively capture the distribution each layer. We concatenate embeddings from O-layer to l-layer and form the concatenated (-step embed-
dings for original graph and synthetic graph: Z The synthesis loss expressed as:
[xt: (()) 20
R
1) ..... ((())
Clyn.ab ab - Natio (IIraro Par.n 112+
112 + wn 1o'snc - 0% 0abbrr122)
(11)
where Hes Re are the means of Ze. R. and oe. âe are the standard deviations of Ze, R. for every class, We is the standard deviation alignment loss weight. Xab.c = ( [ )²² 7>, Vis.cc assigns different
weights to different classes. G. and Vi aboc are the synthetic graph and node set generated by agent b for agent a. We use a unit matrix as the predefined adjacency matrix, i.e., oi I, which eliminates the need to train adjacency matrix generator and avoids encoding synthetic graphs [4] 43

---

4.4 Prototype-Based Life Cycle
On completing training for each task, each agent stores task-specific prompts and classifiers. Conse- quently, GHG focuses on forecasting the task ID. We construct task prototypes by averaging node representations from synthetic graphs generated for agent a by neighbor agents:
No
Pd
(12)
All task prototypes can be constructed and added P. (Pd P!' We apply the mean of 7 to calculate test prototype, denoted as Pies,a- Subsequently, we query task prototype pool using P P and retrieve task ID that exhibits the highest similarity to Prest.a We leverage the prompts and classifier corresponding to task f, together with the fixed GNN model, to make predictions.
4.5 Graph Socialization Generalization
We analyze the graph sociability influence of minimizing overall loss (9) on the generalization of multi-agent graph distribution for each task (t is omitted for simplicity) in Theorem [] Proposition | indicates that the generalization bound in Theorem i tighter than that of training on a single graph.
Theorem The overall graph distribution of agent a come from three aspects: De = wD,
" D (ws," wh) reflects the agent's demand on single graphs, synthetic graphs, and agent predictions from neighbors. Denote the model as Fa = Pa a ha E V. o Ha = Fa. Inspired by 45 46 47 the gemeralization error on multi-agent graph distribution is bounded as follows:
EDA (Pa) E +wC (D.. ED +W>D (v:)+
-1=++=---)) dn,,an, (v D.a, 2/ DA), (13)
(1-w) 2
where denotes error between model F and optimal labeling y under distribution D. C(.) represents a small distance term, and dH,AH. measures the distance between two distributions.
The first term is the error bound on Dall. The second term is a constant and related to a single graph. The third term quantifies the discrepancy between real labeling J and synthetic graph labeling Dtt while the fourth term is derived from the KL-divergence loss in the second term of Eq. ()) The synthetic graph generation makes T similar to DA, so y approximates V'A- The small EDA ()""T) shows that the model trained on DT closely learns the multi-agent graph distribution, and the task prototype is more holistic in Eq. 122) The last term dH,AH, emphasizes the need for distribution aligmment in Eq. (II..
Proposition Given the conditions outlined in Theorem 7 we can derive:
sup min { (DD"", 2 (D S inf (ED, (P) (F) (D)( + C (D., DA), FEF.
(14)
FEJ.
where Q(D) = lcD(F) CD
CD2 (F)] CD2 (DP), ). the usage of D and Dtd yields tighter general-
ization bound than using only single graphs.
When aaeti- graphs are highly heterogeneous, ies, (cD. (F) CD (F)) and C CD., DA
become large. Since D and Dt are close to the multi-agent distribution, the left side of Eq. (44 is small. Thus, Proposition (indicattend that when multi-agent graphs are extremely heterogeneous
and Te and Dil approximate multi-agent graphs, agents depend more on D and Del for better collaboration. Theorem [ and Proposition aare proved in Appendix B
Experiments
5.1 Datasets and Setups
Datasets and Settings. We assess the effectiveness of GHG by leveraging seven publicly available datasets, and their statistical details are presented in Appendix C22 CoraFull [88 Arxiv [49]. and Reddit [50] include 7, 4, and 4 tasks, respectively, each containing 10 classes. Cora [T]] and
CiteSeer [5I] both comprise 3 tasks with classes per task. SLAP [52] and Computers [3 include 3 and 2 tasks, respectively, each with classes. A is set 5 for all datasets except Cora and CiteSeer, where it is 2. We generate A heterogeneous agents using Dirichlet partitioning [44 where the
allocation ratio of nodes for each class denoted as Dir (het). We define heterogeneity level as
het = 0.1 (strong) and het 0.5 (weak). Implementation details provided in Appendix C.4
7

---

Table I: Performance comparison on five datasets in strong heterogeneity setups. Results are averaged among three trials. The best and second results are highlighted in bold and underline.
Dataset
Paradigm
CoraFull
Arxiv
Reddit
Cora
CiteSeer
Metric
MAP
MAKI
MAPT
MARL
MAPT
MARY
MAP
MAFU
MAP1
MAFL
Single
4.2.01
36.7s04
8.4
35.1
15.8
60.1
18.2
54.9.01
16.9000
54.9.0.
FedAvg
FL
4.6.0.1
38.3.02
8.9
36,1
26,6.02
47.9.0.7
18.2.0.1
58,7.0.
17.0.00
54.910.1
DFedGNN GFL
4.6.02
39.5.0.
9.0x0.1
36,0.0.4
14,8
62.7a06
18.0.0.0
54,4
17.0.00
55.4
Fed-PUB
GFL
4.0.0.1
34.0.03
7.7
15.1
13.5
18.1
16.9.0.1
53.0.02
16.7.00
55.9u0.2
FedGiTA
GFI.
4.6-01
38.1+04
8.7+0.1
35.8.0.4
14.9-0
62.4-0.7
18.0+a0
59.7-24
17.0-00
55.0
FedTAD
GFL
4.6:0a
37.9.0.
8.9
36.0.o.1
24.4:02
50.0e04
18.2.02
54.8:08
17.0:00
54.8.0.0
TWP
GLL
5.1.02
34,9.02
8.4
11.1
16.2.02
58.6 68.6.0.5
18.6
55.2.0.1
17.0.00
54.9.0.1
ERGNN
GLL
23.2.0.3
14.0.03
19.6.0.
18.2.0.6
36,5
31.6.00
24.0
42.5u5.7
17.9.0.1
53. 13.4
GSIP
GLL
26.3+0.9
12.8+08
22.5+0.9
9.7+02
48,4+06
15.3+04
33.2+09
34,5-24
19.6-05
50.6.0.7
TPP
GLLL
28.9.o2
4.8+03
16.4+0.3
6,0+o2
43,4+02
5,4+0.6
53.6-1.6
0.0+uo
51.6-0.
0.0+0.0
DMSG
GLL
27.3
9.0
21.8a0.1
8,8
47.9.02
13.2
37.5
27.7a53
33.4
28.7 28.7
Fed-TPP
GFLL
28.6.0.3
4.4
15.3+02
5.3
40.1
5.3
58.7+12
0.0:a.
51.7-01
0.0.0.0
Fed-DMSG
GFLL
37.3+03
0.1
22.7+0.
7.9
57.4.07
6.4404
39.9+1.6
18.1
40.1-10
14.1+1a
POWER
GFLL
25.3+0.
14.8-06
11.0+0.1
25.7+0.3
61.4-10
2.9+1.
40.9+0.
27.1
36.9-2.
24.0.a8
MASC SL
36.7+0.4
0.9-1.2
22.3+0.5
16.5
58.1
4.6
39.2+2.3
23.3-43
36.6-02
24.3
GHG
GSL
54.0.08
0.0.0.0
59.2.0.3
0.0
86.7=09
0.0
79.2:3.4 0.0:0.0
65.7.4.1
0.0.0.0
Baselines and Metrics. We compare GHG with following baselines: Single, five FL/GFL methods (i.e. FedAvg (55). DFedGNN [33. Fed-PUB (9) FedGTA (25), and FedTAD (TO),, five GLL methods
(i.e. TWP 18). ERGNN 19), GSIP 21). TPP (20). and DMSG (22). graph federated lifelong
learning methods (i.e. FedAvg combines with two representative GLL methods and POWER (56)), and socialized learning method MASC [7]. We evaluate the performance of compared methods using two widely adopted metrics: Average Performance (AP) and Average Forgetting (AF) 34 The final results are reported as mean values across all agents, denoted as MAP and MAF
5.2 Performance Comparison
GHG facilitates multi-agent graph socialized learning effectively. The effects of GSL for het=0.1 on seven datasets are presented in Tables and 2 and results for het=0.5 are in Appendix D.I GHG surpasses all baselines in performance. Results are discussed as follows:
(1) FL/GFL methods cannot sufficiently transmit information. Consequently, collaborative collapse of heterogeneity is exacerbated in the dynamic environment, resulting in lower performance. D-GFL method DFedGNN relies an undirected symmetric communication topology, which can not satisfy the collaborative demands of agents. In particular, GFL performs poorly compared with FedAvg. The reason might be that they focus on learning single fixed data and even overfit, thus lacking cooperation. We build an organizational structure based on directed weighted graphs and a customized interactive medium to achieve autonomous collaboration and sufficient information interaction.
(2) GLL methods are limited to single agent settings. Moreover, old information is forgotten after learning new tasks, so forgetting still occurs. TPP and Fed-TPP achieve high MAP and zero MAF on Cora and Citeseer datasets. This arises from agents only having two categories, leading to minimal divergence between individual and overall data distributions and enabling accurate task ID prediction. The prototypes formed by GHG, leveraging interactive information in strong heterogeneity, can still predict the task ID with 100% accuracy. Thus, it achieves zero MAF across all datasets, retaining old information losslessly.
(3) GFLL methods offer some mitigation for these issues, but a considerable performance disparity remains when compared with GHG. Merely integrating federated and lifelong learning fails to adequately address information collapse and forgetting. Conversely, GHG achieves information sharing and accumulation by designing organizational structure, interactive medium, and life cycle.
(4) SL method MASC forgot several old knowledge, and full-data interaction causes high transmis- sion. Our GHG successfully preserves old information and avoids interaction costs by leveraging synthetic graphs an effective medium.
8

---

Table 2: Performance comparison SLAP and Com- puters datasets in strong heterogeneity setups.
MAP (96)
52
51
Dataset
Para.
SLAP
Computers
Metric
MAPT
MAF
MAP
MAF
Single
8.2
34.1a18
13.6u20
30.8
WC
0.9
0.5
FedAvg
GFL
8,0s0.1
32.9206
10.5+0.
4.9,29
07
0.5
0.7
DFedGNN
GFL
8.1
33.8
19.8.22
21.8153
(a) Complementarity weight we and
Fed-PUB
GFL
11.6s0.1
42.3+1.9
27.5a1.1
64.7a29
similarity weight wi
FedGTA
GFL
8.3e02
33.7+23
13.5e5.
10.6
FodTAD
GFL
8.1
31.8,2.2
10.5.04
5.0u67
TWP
GLL
8.1
33.2+1.5
14.5e10
29.2+4.2
ERGNN
GLL
15.6 20.8a1.1
18.0a20
24.1
GSIP
GLL
15.7.05
15.9
19.2.1.0
36.4152
DosPal
Ands Clabeer
TPP
GLLL
24.8+13
15.8+20
36.7+06
0.0
MAP
DMSG
GLL
15.8.02
13.6
28,5+24
30.7a5.2
Fed-TPP
GFLL 27.2:12
12.3,07
36.5u1.5
0.0g0.
Fed-DMSG GFLL
18.0.03
9.8107
32.213.1
3.6128
POWER
GFLL
15.5
20.
10,7+3.
24.4
0.0 0.2 04
Mecll
MASC
SL
15.0.03
18.6.08
24.2
21.6u1.
(b) Collaborative weight Weol
GHG
GSL
62.4
0.0g0.0
82.5.02
0.0.0.0
Figure 3: The analysis of hyper-parameters,
5.3
Ablation Study
To explore the contributions of differ- ent components, we perform an abla- tion study in terms of the MAP met- ric. Table shows progressive improve- ments with each component, confirming their importance to GSL. We devise four variants for GHG-B: This variant uses TPP (20) as baseline; B+Layn This vari- ant constructs accurate task prototypes leveraging synthetic graph loss; C+C This variant incorporates synthetic graphs from neighbor agents into single graph training with cross-entropy loss: Cy tci This variant is the full model, trained with KL divergence loss to help agents mimic others' patterns, achieving the best performance.
Table 3: Ablation comparisons on five datasets.
Method
CoraFull
Arxiv
Reddit
Cora
CiteSeer
28.9.0.2
16.4.0.3
43.4.0.2
53,6216 S1.620.1
B+
33.2.0.3
26.410.3
48.3
54.1.02 52.6.0.7
B+L thee
51.6+10
58.940a
83.4+u9
78.8
63.6-22
54.0.0.8
59.2.0.3
86.7.0.9
79.2.3.4 65.7.4.1
5.4 Further Analysis
Hyper-Paramen Analysis. (1) We assess the performance CoraFull under diverse combi- nations of trade-off parameters. Specifically, we tune we varies in 0.5 to 1.7 and ws varies in 0.1 to 0.9. As shown in Figure B(a) the performance of GHG is stable across we and ws varies, indicating low sensitivity to hyper-parameters. (2) Figure ((b) presents performance as Weal varies, where "R" indicates random sampling. The customized graph synthesis significantly outperforms the random approach, with an almost 10% improvement on Cora, demonstrating the effectiveness of the customized interactive medium. The analysis of loss weights is presented in Appendix D.2
Graph Socialization Necessity. The performance of GHG with 5, 10, and 20 agents, as well as different participation ratios is shown in Table We can observe that the higher the participation ratio, the higher the performance, which proves the necessity of agent socialization. The more agents A. the more dispersed the graph becomes, thus resulting in lower performance. However, the only special case is when 8 equals 0.2. In the case of 5 or 10 agents, only I or agents cooperate, leading the inability to predict task ID accurately, so performance lower than that of 20 agents.
Graph Socialization Efficiency. (1) We compare GHG with baselines on Corafull across different interaction rounds in Figure|4(h) GHG outperforms baselines in all cases. It peaks four interactions, which shows GHG efficiently completes collaboration. In contrast, Fed-DMSG and MASC require

---

60
90-
50
0.5/g
MAP (
60
MAP (%)
401
MAP (%6)
70-
73
30-
50-
20
40-
10
10
20
Nimberaf Agure
Interaction Rounds
ComFull Arbi
Reddit
Coo
CiteSer
(a) Number of agents A
(b) Interaction rounds
(c) Compression ratio 7
Figure 4: The analysis of parameters.
more interactions for high performance. Fed- TPP shows little variation with different r. and POWER fails to save interaction overhead. (2) Figure44( illustrates the performance variation five datasets with different compression ratios. We observe that higher generally leads to better performance. However,
the results with "o limit interaction costs. Even when y is halved, method maintains high performance. We further analyze the time, memory, and communication costs in Appendix D.3 to demonstrate the efficiency of GHG over baselines.
Visulization. (1) The visualization collaborative graph edge weights is displayed in Figure
5 The first row shows three matrices of coarse-graines agent collaboration for Task 0 of Reddit, with those for the other three tasks in Appendix D.4 The second row presents three matrices of fine-grained customized collaboration for Class 0 of Task O. Agents almost abandon their initial state the complementarity matrix, impeding the subsequent training. In the similarity matrix, the insistence of agents their models hinders information acquisition. Our collaborative approach filters out redundant interaction information and gets what each agent needs, highlighting the role of GHG in enhancing autonomous socialization efficiency. (2) As shown in Figure 6 utilize I-SNE 57 to visualize node embeddings of three classes generated by random sampling and GHG on Arxiv. The reciprocal generation of node representations by two agents within GHG facilitates preservation of the category- node distributions inherent in the original graph and enables the customization of category quantities to meet agent demands. This demonstrates that GHG has superior ability to capture sufficient graph information.
1.0
1.0
Clas Oithiigee ('ripee
Elac (ihoped
666
64
4.6
34
2-
96
Claii I (drnteck
Elax clyckee)
0.0
0.0
50
1.0
1.6
86
866
96
444
40.0
Jon
(a) Complementarlyy (b) Similarity
(c) Collahoration
(a) Random
(b) Customized Synthesis
Figure 5: The visualization of collaborative graph Figure 6: The visualization of node embeddings edge weight matrices on Reddit dataset.
on Arxiv dataset.
6 Conclusion
Current paradigms face the predicament of information collapse and forgetting, struggling share and accumulate information in dynamic heterogeneous graph settings collaboratively. We present practical GSL paradigm and develop GHG approach enhance the performance of each agent. Graph-driven organizational structure, customized interactive medium, and prototype-base life cycle form three key elements GSL In future work, we aim to refine the organizational structure further to accommodate large-scale multi-agent systems and strengthen the scalability of the graph socialized learning model. Moreover, we plan to evaluate the effectiveness of the graph socialized learning method real-world application datasets.
10

---

Acknowledgement
This work was supported in part by the National Science and Technology Major Project under Grant 2022ZD01 16500, in part by the National Natural Science Foundation of China under Grants 62436002, 62476195, U23B2049, and 62222608, part by Tianjin Natural Science Funds for Distinguished Young Scholar under Grant 23J 23JCJQJC00270, in part by Zhejiang Provincial Natural Science Foundation of China under Grant LD24F020004, in part by Tianjin Young Scientific and Technological Talents Project under grant QN20230305, in part by Tianjin Science and Technology Plan Project under grant 24YDTPJC00150, and part by Natural Science Foundation of Tianjin under grant 24JCYBJC00950.
References
[1] Yunming Hui, Inez Maria Zwetsloot, Simon Trimborn, and Stevan Rudinac. negative sampling strategies for dynamic graph embedding in meme stock-related social net- works. In Proceedings of the ACM on Web Conference, pages 518-529, 2025.
[2] Yuefei Lyu, Chaozhuo Li, Sihong Xie, and Xi Zhang. Enhancing robustness of graph neural networks on social media with explainable inverse reinforcement learning. In Proceedings of Advances in Neural Information Processing Systems, pages 1-14, 2024.
[3] Huanyu Zhang, Xiaoxuan Shen, Baolin Yi, Jianfang Liu, and Yinao Xie. A plug-in critiquing approach for knowledge graph recommendation systems via representative sampling. In Proceedings of the ACM on Web Conference, pages 322-333, 2025.
[4] Xinlei Huang, Zhiqi Ma, Dian Meng, Yanran Liu, Shiwei Ruan, Qingqiang Sun, Xubin Zheng, and Ziyue Qiao. PRAGA: prototype-aware graph adaptive aggregation for spatial multi- modal omics analysis. In Proceedings of AAAI Conference on Artificial Intelligence, pages 326-333, 2025.
[5] Anita Williams Woolley, Christopher F Chabris, Alex Pentland, Nada Hashmi, and Thomas W Malone. Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004))686-688 2010.
[6] Bill Thompson, B Van Opheusden, T Sumers, and TL Griffiths. Complex cognitive algorithms preserved by selective social learning in experimental populations. Science, 376(6588):95 99-98, 2022.
[7] Xinjie Yao, Yu Wang, Pengfei Zhu, Wanyu Lin, Jialu Li, Weihao Li, and Qinghua Hu. Socialized learning: Making each other better through multi-agent collaboration. In Proceedings of International Conference on Machine Learning, pages 1-12, 2024.
[8] Jacob Eisenstein, Reza Aghajani, Adam Fisch, Dheeru Dua, Fantine Huot, Mirella Lapata, Vicky Zayats, and Jonathan Berant. Don't 't lie to your friends: Learning what you know from collaborative yelf-play. CoRR, abs/2503.1 110000...4811.. 2025.
[9] Jinheon Baek, Wonyong Jeong, Jiongdao Jin, Jachong Yoon, and Sung Ju Hwang. Personalized subgraph federated learning. In Proceedings of International Conference Machine Learning, volume 202, pages 1415, 2023.
[10] Yinlin Zhu, Xunkai Li, Zhengyu Wu, Di Wu, Miao Hu, and Rong-Hua Li. Fedtad: Topology- aware data-free knowledge distillation for subgraph federated learning. In Proceedings of International Joint Conference Artificial Intelligence, pages 5716-5724 2024.
[11] Guancheng Wan, Wenke Huang, and Mang Ye. Federated graph learning under domain shift with generalizable prototypes. In Proceedings of AAAI Conference Artificial Intelligence, pages 15429-15437, 2024.
[12] Zihan Tan, Guancheng Wan, Wenke Huang, and Mang Ye. Fedssp: Federated graph learning with spectral knowledge and personalized preference. In Proceedings of Advances Neural Information Processing Systems, pages 1-15, 2024.

---

[13] Yang Pei. Renxin Mao, Yang Liu, Chaoran Chen, Shifeng Xu, Feng Qiang, and Blue Elephant Tech. Decentralized federated graph neural networks. In International Workshop on Federated and Transfer Learning for Data Sparsity and Confidentiality in Conjunction with IJCAI, pages 1-7, 2021.
[14) Yuxing Tian, Lei Liu, Jie Feng, Qingqi Pei, Chen Chen, Jun Du, and Celinuge Wu. Toward robust and generalizable federated graph neural networks for decentralized spatial-temporal data modeling. IEEE Trans. Netw. Serv. Manag., 21 213))2637,2650, 2024.
[15] Chaoyang He, Emir Ceyani, Keshav Balasubramanian, Murali Annavaram, and Salman Aves- timehr. Spreadgnn: Decentralized multi-task federated learning for graph neural networks on molecular data. In Proceedings of AAAI Conference on Artificial Intelligence, pages 2022.
[16] Liang Qu, Ningzhi Tang, Ruiqi Zheng, Quoc Viet Hung Nguyen, Zi Huang, Yuhui Shi, and Hongzhi Yin. Semi-decentralized federated ego graph learning for recommendation. In Proceedings of the ACM Web Conference, pages 339-348, 2023.
[17] Xiaoyu Wang, Guojun Xiong, Houwei Cao, Jian Li, and Yong Liu. Decentralized federated learning with model caching on mobile agents. In Proceedings of AAAI Conference on Artificial Intelligence, pages
2025.
[18] Huihui Liu, Yiding Yang, and Xinchao Wang. Overcoming catastrophic forgetting in graph neural networks. In Proceedings of AAAI Conference on Artificial Intelligence, pages 8653- 8661, 2021.
[19] Fan Zhou and Chengtai Cao. Overcoming catastrophic forgetting graph neural networks with experience replay. In Proceedings of AAAI Conference Artificial Intelligence, pages 4714 411244472,222 2021.
[20] Chaoxi Niu, Guansong Pang, Ling Chen, and Bing Liu. aor-gyettd-foogt-frree graph class- incremental learning: task profiling and prompting approach. In Proceedings of Advances in Neural Information Processing Systems, pages 1-13,200 2024.
[21] Jialu Li, Yu Wang, Pengfei Zhu, Wanyu Lin, and Qinghua Hu. What matters in graph class incremental learning? an information preservation perspective. In Proceedings of Advances in Neural Information Processing Systems, pages 1-14, 2024.
[22] Ziyue Qiao, Junren Xiao, Qingqiang Sun, Meng Xiao, Xiao Luo, and Hui Xiong. Towards con- tinuous reuse of graph models via holistic memory diversificatio. In Proceedings of International Conference on Learning Representations, pages 1-13, 2025.
[23] Xunkai Li, Yinlin Zhu, Boyang Pang, Guochen Yan, Yeyu Yan, Zening Li, Zhengyu Wu, Wentao Zhang, Rong-Hun Hua Li, and Guoren Wang. Openfgl: A comprehensive benchmarks for federated graph learning. CoRR, abs/2408. 16288, 2024.
[24] Rui Liu, Pengwei Xing, Zichao Deng, Anran Li, Cuntai Guan, and Han Yu. Federated graph neural networks: Overview, techniques, and challenges. IEEE Trans. Neural Networks Learn. Syst., 3673):4779442955 2025.
[25] Xunkai Li, Zhengyu Wu, Wentao Zhang, Yinlin Zhu, Ronghua Li, and Guoren Wang. Fedgta: Topology-awar averaging for federated graph learning. Proc. VLDB Endow, 17(1):1-50, 2023.
[26] Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, and Siu-Ming Yiu. Subgraph federated learning with missing neighbor generation. In Proceedings of Advances in Neural Information Processing Systems, pages 6671-6682, 2021.
[27] Qiao Yuan, Sheng-Uei Guan, Pin Ni, Tianlun Luo, Ka Lok Man, Prudence W. H. Wong, and Victor Chang. Continual graph learning: A survey. CoRR, abs/2301.12230, 2023.
[28] Falih Gozi Febrinanto, Feng Xia, Kristen Moore, Chandra Thapa, and Charu Aggarwal. Graph lifelong learning: A survey. IEEE Comput. Intell. Mag., 18(1):32-51,200) 2023,
12

---

[29] Zonggui Tian, Du Zhang, and Hong-Ning Dai. Continual learning on graphs: A survey. CoRR, abs/2402.06330, 2024.
[30] Xikun Zhang, Dongjin Song, and Dacheng Tao. Sparsified subgraph memory for continual graph representation learning. In IEEE International Conference on Data Mining, pages 1335-1340, 2022.
[31] Yilun Liu, Ruihong Qiu, and Zi Huang. Cat: Balanced continual graph learning with graph condensation. In IEEE International Conference Data Mining, pages 1157-1162, 2023.
[32] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell., 40 (0(11))293552947 2018
[33] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, and Agnieszka Grabska-Barwinska. Overcoming catastrophic forgetting in neural networks. Proc. Narl. Acad. Sci., 114(13):3521- 3526, 2016.
[34] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In Proceedings of Advances in Neural Information Processing Systems, pages 6467-6476, 2017.
[35] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In Proceedings of European Conference Computer Vision, volume 12077 pages 144-161, 2018.
[36] Xikun Zhang, Dongjin Song, and Dacheng Tao. Hierarchical prototype networks for continual graph representation learning. IEEE Trans. Pattern Anal. Mach. Intell., 45(4):4622-4633 2023.
[37] Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu. A theoretical study on solving continual learning. In Proceedings of Advances in Neural Information Processing Systems, pages 1-15, 2022.
[38] Shuo Tang, Rui Ye, Chenxin Xu, Xiaowen Dong, Siheng Chen, and Yanfeng Wang. De- centralized and lifelong-adaattive multi-agent collaborative learning. CoRR, abs/2403.06535. 2024.
[39] Rui Ye, Zhenyang Ni, Fangzhao Wu, Siheng Chen, and Yanfeng Wang. Personalized federated learning with inferred collaboration graphs. In Proceedings of International Conference on Machine Learning, volume 202, pages 39801-398111 2023.
[40] Kunda Yan, Sen Cui, Abudukelimu Wuerkaixi, Jingfeng Zhang, Bo Han, Gang Niu, Masashi Sugiyama, and Changshui Zhang. Balancing similarity and complementarity for federated learning. In Proceedings of International Conference on Machine Learning, pages 1-11, 2024.
[41] Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang. and Lei Chen. Universal prompt tuning for graph neural networks. In Proceedings of Advances in Neural Information Processing Systems, pages 1-13, 2023.
[42] Xin Zheng, Miao Zhang, Chunyang Chen, Quoc Viet Hung Nguyen, Xingquan Zhu, and Shirui Pan. Structure- free graph condensation: From large-scale graphs to condensed graph-free data. In Proceedings of Advances in Neural Information Processing Systems, pages 1-15, 2023.
[43] Xinyi Gao, Tong Chen, Wentao Zhang, Yayong Li, Xiangguo Sun, and Hongzhi Yin. Graph condensation for open-world graph learning. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 851-862, 2024.
[44] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Mach. Learn., 79(1- 2):151-175, 2010.
[45] Haozhe Feng, Zhaoyang You, Minghao Chen, Tianye Zhang, Minfeng Zhu, Fei Wu, Chao Wu, and Wei Chen. KD3A: unsupervised multi-source decentralized domain adaptation via knowledge distillation. In Proceedings of International Conference on Machine Learning, volume 139, pages 3274-3283, 2021.
13

---

[46] Othmane Marfoq, Giovanni Neglia, Aurélien Bellet. Laetitia Kameni, and Richard Vidal. Federated multi-task learning under mixture of distributions. Proceedings of Advances in Neural Information Processing Systems, pages 15434-15447, 2021.
[47] Chun- Yin Huang, Kartik Srinivas, Xin Zhang, and Xiaoxiao Li. Overcoming data and model heterogeneities in decentralized federated learning via synthetic anchors. In Proceedings of International Conference Machine Learning, pages 1-12, 2024.
[48] Aleksandar Bojchevski and Stephan Günnemann. Deep gaussian embedding of graphs: Unsuper- vised inductive learning via ranking. In International Conference on Learning Representations, pages 1- 2018.
[49] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In Proceedings of Advances in Neural Information Processing Systems, pages 22 18 18-22213, 2020.
[50] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning large graphs. In Proceedings of Advances in Neural Information Processing Systems, pages 1024-1034, 2017.
[51] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, and Tina Eliassi- Rad. Collective classification in network data. Al Mag., 29(3):93-100 2008.
[52] Yuxin Xiao, Zecheng Zhang, Carl Yang, and Chengxiang Zhai. Non-local attention learning on large heterogeneous information networks. In Proceedings of IEEE International Conference on Big Data, pages 978-987, 2019.
[53] Oleksandr Shehur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. CoRR, abs/1811, 11.0568,, 2018.
[54] Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan H. Greenewald, Trong Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric federated learning of neural networks. In Proceedings of International Conference on Machine Learning, volume 97, pages 7252-7261, 2019.
[55] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas Communication--fficiee learning of deep networks from decentralized data. In Proceedings of International Conference on Artificial Intelligence and Statistics, volume 54, pages 1273-1282, 2017.
[56] Yinlin Zhu, Miao Hu, and Di Wu. Federated continual graph learning. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4203 4013,4213,1 2025.
[57] Laurens van der Maaten. Accelerating t-sne using tree-based algorithms. Mach. Learn. Res., 15211:322,33255, 2014.
[58] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Founda- tions and Trends in Theoretical Computer Science, (33-4):21-440) 2014.
14

---

A Appendix
The appendix contains supplementary proofs, together with comprehensive details of implementation and results mentioned in the main paper. It is organized as follows:
In Section B, we provide the proofs of Theorem 1 and Proposition 1 regarding graph socialization generalization.
In Section C, presents a thorough overview of baselines, detailed descriptions of datasets, and relevant experimental details in the main paper.
In Section D, offers complete experimental results alongside in-depth analysis to fully evaluate the performance of the proposed method, followed by time, memory, and commu- nication costs analysis.
In Section E, we discuss the related work, privacy issue, limitation, and potential broader impact of the method.
B Proof of Graph Socialization Generalization
B.1 Proof of Theorem
Proof. We leverage previous results tion generalization bound:
44 46 47 to get the inequality and multi-agent graph distribu-
CD (Fa) S ED. (Fa) + CDD..
(15)
S (F) (F)
+ d (v D DA) + tEDA (D),
(16)
where C (D.,, DA) = -dr DA) n (Da). and 7 (Da) minrer. ED.
(F)
+ ED (F) is
constant. These terms tend to be small except when graph heterogeneity is severe 44 Va. Ha F. are the space of prompts, classifier, and model on agent o. Y' represents the optimal labeling function of graph distribution D. Since the overall graph distribution of agent originates from single graphs, synthetic graphs, and predictions of neighboring agents on synthetic graphs, we sum three inequalities after multiplying them by their corresponding weights.
The support of " and D are identical. As a consequence, their proximity to the support of the multi-agent graph distribution also matches, which is denoted as:
dH,AH, (vb o Daya, + DA) = dH,AH, (v Del v DA).
(17)
By leveraging the findings from 45 and Pinsker inequality, we establish that the optimization of loss
leads to the minimization of EDan (F.). After gradually adding three inequalities, obtain:
cDA (Fa) Sepa (Fa) + CD,; DA) + wayn
2 dH.AH.
(w o Daya
Daya , 2o DA)
+ EDA (2)0m) + witl
d dH,SH. (10 o D5Th e DA) + CD2 (2
(18)
Scpu (Fa) + wC (Da, DA) + ". CDA (D
w CD. (2
w) + dH.sH. (v o Dzn 26 o DA).
Since wy wil 1, we can derive:
EDA (Fa) S epg (Fa) + WCC(D. DA) + w EDA (Dnr)) + w EDD ()a
(1 (1-w) w)
(19)
du (10 o Dayn v DA).
2
15

---

B.2 Proof of Proposition
Proof. We initiate with:
sup (D.T (F) + EDi (Jan)) PEF.
(20)
1
inf PEF.
(ED, (F) -Dx
2 -drAsF. (D),DA) + (Da),
where 2 |d Ar, (Da, DA) + (Da) ) = C (Da, DA). For any E Fa. we can derive:
"Den (F) (D, (F) (D, (1 - cD. (F) (D, C DA)
(21)
* Epg + ED (Da") S ED. (F) PA),,
The right side of Eq.22 corresponds to the original bound stated in [44], so we have:
DD (F) EDA () S CD_ (F) + - dr.AF. (Da, Da) + (Da)
(22)
Combine Eq. 21 and 22 and if weight 0,
w c (F) + w D (F) + ()% cc. dy.ar. (D, DA) + +
(23)
Thus, we determine that our overall graph generalization bound in Theorem is tighter than the single graph generalization bound in 4 provided that the condition in Proposition is satisfied.
Algorithm Training of GHG
Input: A series of multi-agent graphs: G [G', 1 S a A,ISITT) task number
T, interaction round r, agent number A, the graph neural networks f1:(()) (Sa)) SS S A), initial prompts 1:AA {
1 T) and classifiers h1:A
{hrs 1 S S A, T}, initial collaborative graph edge weights M.
for to A do
2: Pre-train f-(-) on G² with graph pre-training learning.
3: end for
4: for = 1 to T do
5:
for rnd 1 to do
6:
for 1 to A do
7:
rnd == then
8:
Optimize कC and hy by minimizing single-agent graph learning loss as Eq. 44
9:
else
10:
Receive interaction information R
{U Ue "',,',,,,,,A""",,,, MEAN(F(("I))))
MEAN(Z((0)
(MMANNZZ))MMMMAN & F(G')} sent from other agents. Optimize &e and hy by minimizing multi-agent graph learning loss as Eq. (9)
11:
12:
end if
13:
Calculate complementarity C and similarity S as Eq. (5) and Eq.
14:
Update collaboration graph edge weights M Eq. (8).
15:
Calculate initial label distribution of synthetic graphs for neighbor agents as Eq. (10).
Generate synthetic graphs for neighbor agents Eq. (TT).
17:
Send interaction information R. to other agents.
18:
if rnd = then
19:
Obtain task prototype P of task as Eq. 12
end if
21:
end for
22:
end for
23:
end for
16

---

Algorithm 2 Inference in GHG
Input: Graph neural networks (1:1() (fa() v a s A), graph prompts i:A
{Ør 1 Σ a v A, < 7), classifiers 1EA (Ma 1 $ A, st st task proto-
types P1:A = (Pa 1 S A}, and test graph Guss-
Output: Prediction results of agents for test graph.
1: for a a- 1=-10 to A do
2: Obtain task prototype P Peestaa
3: Infer the task ID of graph by querying P. with lest,a
4: Obtain predictions through Sa (-) and the corresponding graph prompts of and classifier hig
5: end for
6: Return Prediction results of agents for the test graph.
C Implementation Details
C.1
Algorithm
The proposed method is summarized in Algorithm for training and Algorithm 2 for inference.
C.2 Datasets and Baselines
The statistical properties of seven datasets are shown in Table On CoraFull dataset, the node label distribution five agents under two heterogeneous levels for each task is shown in Figure It can be seen that different agents have different classes and numbers of nodes in each task. We introduce the baselines from the main paper as follows:
Single serves as the lower-bound baseline, exclusively utilizing the most recent single graph to update the model.
Federated Averaging (FedAvg) [5
serves the foundational federated learning method, where the server aggregates the model parameters received from clients and distributes them back to clients.
Decentralized Federated Graph Neural Network (DFedGNN) 13 adopts decentralized parallel stochastic gradient descent algorithm for training the graph neural network model across scer-to-peer network topology.
Federated Personalized Subgraph Learning (Fed-PUB) [9] performs weighted averaging during server-side aggregation and learn personalized sparse masks to update the subgraph- relevant subset of the aggregation parameters.
Federated Graph Topology-ave -wware Aggregation (FedGTA) 25 aggregates through topology-
aware local smoothing confidence and mixed neighbor features.
Federated Topology-aware Data-free Knowledge Distillation (FedTAD) 10 measures class-wist knowledge reliability at clients and uses a generator produce pseudo-graphs. transferring reliable knowledge from clients to the global model.
Topology-awar aware Weight Preserving (TWP) [8 assesses parameter contribution to task per- formance and topology, and introduces regularization terms to preserve critical parameters.
Experience Replay Graph Neural Network (ERGNN) 19 achieves multiple node selection strategies to extract representative nodes for memory replay.
Graph Spatial Information Preservation (GSIP) 211 prevents catastrophic forgetting by preserving graph spatial information based on low- and high-frequency components.
Task Profiling and Prompting (TPP) 20 enables replay-free and forget-free learning through discriminative graph prompts and task ID predictions.
Diversified Memory Selection and Generation (DMSG) 22 employs buffer selection strategies considering intra-class and inter-class diversity, along with diversified memory generation for replay.
17

---

Table 4: Statistics datasets.
Datasets
CoraFull
Arxiv
Reddit
Cora
CiteSeer
SLAP
Computers
nodes
19,793
169,343 1,166,243
227,853
2,708
3,327
20,419 172,248
13,752
edges
130,622
114,615,892
5,429
4,732
245,778
class
70
40
40
6
15
10
agent
5
2
5
task
4
4
3
3
2
# novel class
10
10
10
2
5
5
400
300
500
600
400
600
200
300
400
300
130
Classes
200
-300
200
300
200
300
100
290
100
100
100
50
01
2
2.3 4
0 2
01.3 212.3
0
2
0 012 3
ТO
T1
T2
T3
T4
T5
T6
(a) het
0.1 (Strong Heterogeneity)
150
Classes
200
100
200
200
100
012 TO
012344 T1
0 1 2
0 01234
0 0
2134
0 0
2 T5
2.3
T2
T3
T4
T6
(b) het 0.5 (Weak Heterogeneity)
Figure 7: Node label distribution for each task five agents on CoraFull dataset under two heteroge- neous levels.
Graph Evolution Trajectory-aware Knowledge Transfer (POWER) [56 replays experience
nodes with maximum local-global coverage at clients and tackles global expertise conflict by trajectory-aware knowledge transfer.
Multi-Agent Socialized Collaboration (MASC) 7 constructs collective collaboration and reciprocal altruism and achieves socialized learning via knowledge distillation.
C.3 Metrics
We two widely adopted metrics evaluate the performance of the compared methods: Average Performance (AP) and Average Forgetting (AF) (3). Specifically, AP and AF are calculated from the lower triangular performance matrix E R jtl (L > t') represents the node classification accuracy on task after learning task t. After learning new task, average performance assesses the average performance on previous tasks, while average forgetting measures the average performance drop on old tasks. MAP and MAF denote the mean accuracy and forgetting rate across agents, respectively. Higher MAP and lower MAF correspond to superior performance. MAP and MAF are derived through the following formulas after completing T tasks:
MAP
TH MAF
J J.H).
(24)
A
T
C.4 Detailed Settings
Our model deployed in PyTorch and on an NVIDIA RTX 3090 GPU. We use Adam with weight decay, setting the learning rate to 0.005 and training for 50 epochs. For graph synthesis of each agent,
18

---

Table 5: Performance comparison on five datasets in weak heterogeneity setups. Results are averaged among three trials. The best and second results are highlighted in bold and underline.
Dataset
Paradigm
CoraFull
Arxiv
Reddit
Cora
CiteSeer
Metric
MAPT
MAFL
MAPT
MARL
MAPT
MARI
MAPT
MAFL
MAP1
MAFL
Single
6,2
55.3e01
11.9a0.
49.6sa1
21.3s02
82.
26.8
79.3
20,4x02
57.5
FedAvg
FL
7.2
59.3,0.2
12.3a02
50.7
21.6.0.1
83.6
29.6.03
81.0a0.4
20.4
60.0.02
DFedGNN GFL
7.3
60.01a.3
12.8:0.
53.0.04
21.7a0.0
83.4.0.
29.0.0a
81.8st.
23.6
64.1s00
Fed-PUB GFL
5.3
51.5w22
11.1am.5
50.7
21.2
84,3
16.7203
63.0ans
16.7a00
51.3at.1
FedGTA GFL
7.1
59.6.0.4
12.6:0.1
51.7.0.
21.5
83.3
29.8s0s
81.3
23.6x0.1
61.0.00
FedTAD
GFL
7.0.0.1
58.5.0.4
12.5a0.1
52.1s02
22.5.0.9
82.5
31.5a1.
80.3
20.1.0.)
59.6.04
TWP
GLL
6.7+0.2
51.9
11.9-0.2
50.1
21.3+03
81,8+0.5
31.3
79.8.0.
19.9.0.1
56.6-03
ERGNN
GLL
37.4:04
17.7
24.7.0.5
29.
55.6:04
36.3
36.3a38
70.6
23.2
16.1
GSIP
GLL
43,5
12.5:04
30.6:022
14.9a04
69.4
17.9
14.0u2.1
60.3
24.6:07
53.8
TPP
GLL
46.8.04
1.8.0.1
21.0a0.1
9.3
69.9.0.2
5,1
79.5.04
0.0.00
53.7a13.
0.0.0.0
DMSG
GLL
41.4
10.8
25,3a0.3
12.9.0.1
66.3at.1
16.2a14
58.3226
37.3
41,4
32.8
Fed-TPP
GFLL
47.1
2.2-0.1
19.5.0.
8,2+0.2
67.7+05
5.0+0.3
76.6-17
0.0
50.4
0.0
Fed-DMSG GFLL
56.1=07
0,4,09
29.9.0.
8.1
77.0stm
4.3
67.9a16
17.1
44,8a12
27.6:1.
POWER
GFLL
42.2=02
15.8
25.1
32.6a1.1
76.0s0x
10.6
63.4s49
30.1
49.9a07
21.3a00
MASC SL
54.7.08
2.7
31.7
24.9
78. 1004
7.7at.2
65. la12
26.6
48.5a13
10.6
GHG
GSL
60.7.002
0.0
61.0.0.2
0.0.n.e
92.7.0.0
0.0gn.0
93,9
0.0ga.
72.7
0.0
we employ Adam with a learning rate of 0.001 or 0.005 and 50 epochs per interaction round. The graph prompt settings follow those of TPP [20], with 3 Laplacian steps and three prompts. Results are reported the mean and standard deviation of three trials. For dataset splits, CoraFull, Arxiv, and Reddit have 60% training, 20% validation, and 20% testing, while Cora, CiteSeer, SLAP, and Computers have 20% training, 40% validation, and 40% testing. The compression ratio To is 0.1 for most datasets, except for Reddit, which has a value of 0.01. When applying SVD to agent outputs, the representative subspace column number & of CoraFull, Arxiv, and Reddit is 3, and the last four datasets use 2. For SVD on outputs of classes, & is set to 1. The number of interactions for the first three datasets is 4, and for the last four datasets, interaction round is set to 2.
D Experimental Results
D.1 Performance Comparison
The performance under weak heterogeneity conditions is presented in Table 5 Overall, it is better than that under the strong heterogeneity condition. Nevertheless, our method still shows remarkable performance. This indicates the effectiveness of our method in collaboration.
D.2 Hyper-Parameter Analysis
The KI. divergence loss weight wa takes values in [le 3, le - 2, ee- 1, le 0J. and the results on five datasets are shown in Figure [[a] Different datasets have different optimal values. We note that the model's 's performance doesn't 't vary significantly with different values. The synthetic standard deviation loss weight w, is set to [le-44 1e-3, 3, le 2, -e-eeel -l. 1]. and the performance on five datasets presented in Figure 8(b) The results indicate that setting wa le or le yields satisfactory outcomes.
D.3 Time, Memory, and Communication Costs Analysis
As shown in Table and Table 7 we provide an analysis of the time, memory, and communication costs. A, T, r,, and denote the number of agents, the number of tasks, the number of interactions, and the number of classes per task, respectively. d and dl represent the dimensions of original features and model outputs, respectively. Our method offers five advantages over baselines: shorter runtime, lightweight models, need for replay, lower memory usage, and reduced interaction cost.
19

---

60.
10
1c-3
3
70.
MAP
30
30
Comtaull
Ardiv
Baddd:
Coea
Cltbaa
Caalall
Andr
Radda
Cops
Canbier
(a) KL divergence loss weight wxi
Standard deviation loss weight we
Figure 8: The analysis of loss weights.
Table 6: Comparison of time, model parameters, memory size on CoraFull dataset.
Method
Time cost (s) Model content
Model params (M) Memory concent
Memory size (MB)
Fed-DMSG
881
A agents, Server
33.7
Replay graph
79.6
POWER
1403
A agents, Server
Replay graph, Global replay graph
86.2
MASC
A agents, Server
Replay graph
79.6
GHG
1092
agents
Prompts, Classifiers, Prototypes
7.5
Table 7: Comparison of communication costs on CoraFull dataset.
Method
Complexity
Content
Params (M)
Fed-DMSG O(ATrF)
Model parameters F
I 1885.0
POWER
O(ATrF ATCF
Model parameters F. Class prototype gradients Fgrad
632.5
Original graph Vid", Classifiers ht
MASC
O[ATr([V]d +) A))
Node emheddings [V
109.4
Prompts 4. Classifiers h. Synthetic graphs (Vdd,
O[AATr(1V]+20) +22 3)d
GHG
Node embeddings ([V) 2C 4 3)d.
105.0
+ +) +++)
Topology aware embeddings (C 1)d"
Our approach involves time cost facilitate information collaboration among agents, with the goal of enhancing each agent's performance. For each agent at every task of each interaction, the time consumption mainly stems from three aspects: graph-driven organizational structure, customized interactive medium, and prototype-baser life cycle. In the graph-driven organizational structure process, the complexity of using SVD to extract the representative subspace is (((VV + [V])d² for the complementarity measure. Similarity calculation related the Laplacian smoothing process. As solving for collaborative graph edge weights is convex optimization problem. it can be quickly solved with convex optimization solvers. Thus, the complexity of graph edge weight calculation is O((VVI + Vi)d l/ald"). where |a| returns of the number of edges. Given the number of epochs Egyn on synthetic process, the computational complexity of synthetic graph generation for neighbor agents and task prototype computation (AA(laa 1|V1 + C'dd Esyn) and O(AI)Vidd [V[d"). is linear to the number of nodes, the number of edges, and the number of original node attributes.
D.4 Visualization
As shown in Figure collaborative graph edge weights for the last three tasks on Reddit dataset displayed. The varying collaborative weights among agents across different tasks reveal autonomous collaborative relationships. which are beneficial for enhanced collaboration and promoting the performance improvement of agents.
E
Discussion
E.1
Discussion on Related Work
While our approach shares decentralization and lifelong learning considerations of recent work [38] differs in several key aspects. Unlike (38. which relies on full-parameter interac- tions and gradient-based lifelong adaptation, our method explicitly models not only agent similarity but also complementarity and structural relations. Moreover, GHG employs customized interactive
20

---

0.8
0.8
Agent
0.6 g
0.6
0.6
0.4
2
-.00
-0.0
0.0
Agonls
Agents
Agents
1.0
1.0
1.0
0.8
Agents
104
0.44
0.4
0.2
0.2
0.2
2
0.00
Agents
Agents
Agents
1.0
0.8
0.8
0.8
Agens
0.6
0.6
0.4
0.2
0.2
0.0
0.01
Agents
Agents
Agonts
Complementanty Matrix
Similantt Matrix
Collaboration Matrix
Figure The visualization collaborative graph edge weight matrices on Reddit dataset.
Table 8: Privacy analysis CoraFull dataset.
Method
MIA success rate (%)
SRA precision (%)
MAP (%)
GHG
9.7
1.0
54.0
GHG-DP
4.9
0.9
38.4
medium reduce communication overhead and enhance knowledge compatibility. GHG adopts prototypes and colation-bbsse strategies to achieve more stable and efficient lifelong learning.
E.2 Discussion on Privacy Issue
The primary application scenarios of GSL involve cases where multiple laboratories or institutions train models on different graph libraries while aiming to share knowledge collaboratively. Our goal is share information to enable effective socialized collaboration, unlike graph data across pharmaceutical companies or hospitals, which may not be shareable due to privacy concerns. We still provide brief discussion on the privacy issue. We conduct Membership Inference Attacks (MIA) and Structure Reconstruction Attacks (SRA) synthetic graphs transmitted between agents. As shown in Table g we report the average MIA success rate and SRA precision across all synthetic graphs. It can be observed that MIA scores are consistently higher than those of SRA. Although synthetic node features are generated based on the original topology, synthetic graphs do not directly transmit topology. Thus, the original structure is well protected. To enhance privacy protection of synthetic graphs, we apply feature perturbation using the Gaussian mechanism [58) to achieve Differential Privacy (DP). As shown in Table & both MIA success rate and SRA precision decrease. Although the performance of GHG slightly drops due to DP, it still significantly outperforms baselines.
E.3
Limitation
For large-scale multi-agent systems operating in complex environments, employ sparsification techniques to improve scalability. However, when number of agents grows very large, the current method may incur computational and communication overhead. An interesting direction for future work is to investigate more suitable organizational structures (i.e. hierarchical structures) that can better scale to large multi-agent systems. Moreover, plan to conduct experiments real-wo world datasets to further validate the potential of GSL methods in the future.
E.4 Broader Impact
Our work can positively impact society by contributing to fields that utilize graph-structured data, such as social networks and recommendation systems. can make information reuse and recommendations more efficient in these fields. However, lack of regulatory bodies might lead negative effects.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in Appendix E.3.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
22

---

Justification: See Section 3.2 and 4.5 for assumptions and the detailed proof is provided in Appendix B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All the information needed to reproduce the main experimental results is added in Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
23

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
They are enclosed in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ipss.cc/pill wttpcccpppppppppeeeeeeeeeeeeeeee httpa///ggiieee/eeeSebiiisssssss (Cooeooomiissssseeeeeeee sww.iissiiiiiiiiiico for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The experiment settings are provided in Appendix C.4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: See that Table 1, Table 2, Table 3, and Table 5 report the standard deviation of experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The detailed description is provided in Appendix C.4 and D.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
The research conducted in the paper conforms with the NeurIPS Code of Ethics.
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impacts are discussed in Appendix E.4.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
25

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does use existing assets. The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
26

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
27

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy buctps://neure wwwparippp...coooeeeeeeeee for what should or should be described.
28


Reviews:
[{"review_id": "C6slvcU0u5", "reviewer": "Reviewer_sYbT", "summary": "The authors address challenges of information collapse and catastrophic forgetting in distributed graph learning by introducing the Graph Socialized Learning paradigm. To realize this, the authors propose the Socialized Collaboration Paradigm, which formalizes “who to collaborate with, what information to share, and when to recall past knowledge” in a unified Bayesian framework for continual, heterogeneously distributed graph tasks.", "questions": "1. The paper states that “D-GFL fails to capture agent dependencies,” but recent variants (e.g., PushSum-GNN) incorporate structure-aware designs. Could the authors clarify whether this refers only to standard D-GFL methods?\n2. The current ablation study focuses on removing individual components in isolation, which limits insight into how different modules interact. Would it be beneficial to explore additional combinations—such as testing pairs of modules—to better understand their joint contributions and potential redundancy?\n\n3. How sensitive is GHG to the choice of hyperparameters, such as the collaboration threshold or prototype pool size? Has this been empirically evaluated?", "limitations": "see above", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\n1. The paper is well‑structured and logically sound, guiding readers smoothly from motivation through methodology to theoretical and empirical findings.\n\n2. Experiment 5.4 convincingly demonstrates the necessity of collaboration and confirms the algorithm’s ability to enable efficient information exchange.\n\n**Weakness**\n\n1.  The proposed method is relatively complex, incorporating multiple technical components such as subspace complementarity, model/structure similarity, distribution‑aligned synthetic graphs, and prototype‑based retrieval. While this design is principled, the number of interacting modules and hyperparameters may make the approach harder to tune and reproduce in practice.\n\n2.  The prototype pool may gradually grow as new tasks and classes are added, potentially introducing storage overhead. While pruning can mitigate this, it relies on carefully chosen strategies and thresholds.", "was_revised": true, "final_justification": "The authors have partially addressed my concerns. So, I keep my score unchanged.", "rebuttal": "We would like to thank you for acknowledging well-organization, sound logic, validity, and efficiency of proposed GHG. Thanks for your valuable suggestions!\n\n>W1: The proposed method is relatively complex. While this design is principled, the number of interacting modules and hyperparameters may make the approach harder to tune and reproduce in practice.\n\nA1: Our method introduces a practical graph socialized learning paradigm that leverages agent interactions to outperform individual agents. The modules are designed to answer the essential questions of socialized learning: with whom, what, and when to interact. It filters out collaborators’ low-quality, redundant, and insufficient information. Table 1 and Table 2 show that our approach incurs lower interaction costs than baselines and is insensitive to hyperparameters.\n\n* **Lower interaction costs.** The proposed method uses more interaction modules than baselines, yet achieves lower interaction costs and superior performance.\n \n**Table 1: Comparison of interaction memory size on CoraFull dataset.**\n| Method|Content|Memory size (MB) |MAP (%)|\n|:----|:----|:----|:----|\n|Fed-DMSG|Model parameters|1885.0|37.3|\n|POWER|Model parameters, Class prototype gradients|632.5|25.3|\n|MASC|Original graph, Classifiers, Node embeddings|109.4|36.7|\n|GHG|Prompts, Classifiers, Synthetic graphs, Node embeddings, Topology-aware embeddings |**105.0**|**54.0**|\n\n\n* **Insensitive hyperparameters.** The hyperparameters of our method are: the collaboration threshold **(analyzed in A2)**, the fine-grained collaborative weight $w_{col}$, the KL-divergence loss weight $w_{kl}$, and the synthetic standard-deviation loss weight $w_{\\sigma}$. Table 2 shows that these are largely insensitive. Due to space limits, these results are placed in the appendix. Hyperparameter analysis aids tuning and reproduction. We will include it in the main text for the revised version.\n\n**Table 2: Hyperparameter analysis (MAP (\\%)) on five datasets.**\n|$w_{col}$|0|0.2|0.4|0.6|0.8|1.0|1.2|1.4|1.6|\n|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|\n|CoraFull |49.0|50.5|52.4| 53.0| 53.1| 53.3| 53.1| 54.0| 53.4| 53.7|\n|Arxiv |56.3| 58.7| 59.0| 59.1| 59.2| 59.1| 59.2| 59.1| 59.1| 59.1|\n|Reddit |84.1| 86.0| 85.9| 86.1| 85.9| 86.2| 86.7| 86.1| 85.6| 85.7|\n|Cora |69.6| 79.2| 79.2| 79.1| 79.2| 79.1| 79.2| 79.0| 79.1| 79.2|\n|CiteSeer |59.4| 64.1| 64.8| 65.4| 65.7| 63.4| 63.6| 65.3| 63.8| 64.9|\n\n|$w_{kl}$|**1e-3**|**1e-2**|**1e-1**|**1e-0**| |$w_{\\sigma}$|**1e-4**|**1e-3**|**1e-2**|**1e-1**| \n|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|\n|CoraFull |51.5| 51.8| 51.9| 53.1|| |52.2|53.1|48.0|41.9| \n|Arxiv |59.0| 59.0| 59.2| 58.4||| 59.1|59.2|59.1|59.1| \n|Reddit |83.2| 83.7| 83.9| 86.7||| 86.5|86.7|86.1|85.7| \n|Cora |79.2| 78.6| 77.8| 74.2||| 78.5|78.8|79.2|75.9|\n|CiteSeer |56.2| 64.5| 62.3| 63.6|| |63.4|63.6|58.5|58.9|\n\n\n\n\n>W2: The prototype pool may gradually grow as new tasks and classes are added, potentially introducing storage overhead.\n\n>Q3: How sensitive is GHG to the choice of hyperparameters, such as the collaboration threshold or prototype pool size?\n\nA2: Each task prototype has a shape of $[1, d]$, so our method’s prototype pool is tiny and introduces negligible storage overhead. Thus, the prototype pool size is not treated as a hyperparameter. Table 3 shows that the collaboration threshold is largely insensitive.\n\n* **The prototype pool incurs negligible storage overhead when new tasks are added.** Each prototype is simply the mean of the received vectors, with dimension $[1, d]$. The entire pool occupies only $[T, d]$ after $T$ tasks, so we keep it unchanged. We believe that pruning might not be necessary.\n\n* **Low sensitivity of the collaboration threshold.** Table 3 summarizes the results under the adjusted thresholds: the vertical axis shows the complementary collaboration weight $w_c$ and the horizontal axis the similarity collaboration weight $w_s$. It can be observed that both thresholds are of low sensitivity.\n\n**Table 3: Collaboration threshold analysis (MAP (\\%)) on CoraFull dataset.**\n|$w_c$/$w_s$|0.1|0.3|0.5|0.7|0.9|\n|:----|:----|:----|:----|:----|:----|\n|0.5|51.0|51.7|50.7|51.2|52.1|\n|0.7|51.9|51.8|51.0|51.0|51.0|\n|0.9|51.4|51.2|51.9|51.9|51.5|\n|1.1|51.4|51.8|52.4|52.0|51.8|\n|1.3|51.7|52.9|51.7|51.9|52.0|\n|1.5|51.9|51.7|52.3|52.2|52.0|\n|1.7|51.9|52.0|52.0|52.3|51.8|\n\n\n\n\n\n\n>Q1: The paper states that “D-GFL fails to capture agent dependencies,” but recent variants (e.g., PushSum-GNN) incorporate structure-aware designs. Could the authors clarify whether this refers only to standard D-GFL methods?\n\nA3: We clarify that \"D-GFL fails to capture agent dependencies\" refers to standard D-GFL methods. These methods ignore the special design of the communication topology, and the recent topology-aware variants (e.g., PushSum-GNN) cannot be directly transferred to graph data.  \n\n* **Lack of dedicated designs for the communication topology in standard D-GFL approaches.**  D-GFL denotes classic methods (e.g., [1][2][3]), which rely solely on an undirected, symmetric topology that cannot meet individual agents’ collaboration needs. \n\n\n* **Inapplicability of recent variants (e.g., PushSum-GNN) to graph data.** Early structure-aware designs in vision attempt to build topologies from gradients, but the learned structures are task-specific, non-shareable of structure information, and disregard similarity or complementarity among agents, so they may still miss accurate agent dependencies.\n\n[1] Semi-decentralized federated ego graph learning for recommendation. WWW, 2023.\n\n[2] Spreadgnn: Decentralized multi-task federated learning for graph neural networks on molecular data. AAAI, 2022.\n\n[3] Decentralized federated graph neural networks. IJCAI Workshop, 2021.\n\n\n\n>Q2: Would it be beneficial to explore additional combinations—such as testing pairs of modules—to better understand their joint contributions and potential redundancy?\n\nA4: We evaluate the contributions of three module pairs across five datasets, replacing $L_{syn}$ with random sampling when $L_{syn}$ is omitted. Table 4 shows that every pair improves the model. Removing both $L_{ce}$ and $L_{kl}$ hurts most, confirming the necessity of interaction. Performance drops more with w/o $L_{syn} + L_{ce}$ than w/o $L_ {syn} + L_{kl}$, especially on Arxiv and Cora, underscoring the importance of task-specific information.\n\n**Table 4: Ablation comparisons (MAP (\\%)) on five datasets.**\n|Datasets |CoraFull|Arxiv|Reddit|Cora|Citeseer|\n|:----|:----|:----|:----|:----|:----|\n|w/o $L_{ce}+L_{kl}$|33.2±0.3|26.4±0.3|48.3±0.2|54.1±0.2|52.6±0.7|\n|w/o $L_{syn}+L_{ce}$|45.8±0.3|30.0±0.4|71.4±0.2|53.7±1.7|53.8±2.8|\n|w/o $L_{syn}+L_{kl}$|47.8±0.7|56.2±0.7|82.8±0.2|70.0±2.8|56.3±1.7|\n|Full|54.0±0.8|59.2±0.3|86.7±0.9|79.2±3.4|65.7±4.1|"}, {"review_id": "a0p2VYZfik", "reviewer": "Reviewer_LUDz", "summary": "Effective collaboration in heterogeneous dynamic graph environments becomes challenging. Inspired by social learning, the paper presents  Graph Socialized Learning (GSL) paradigm. The paper has following main contributions:\n• Authors present a practical learning paradigm called Graph Socialized Learning (GSL), enabling each agent’s growth via collaborative interaction.\n• Authors describe how Graph-driven organizational structure, customized interaction medium, and prototype-based life cycle form three key elements of socialized collaboration.\n• Proposed method is claimed to consistently achieves performance improvements on multiple datasets and demonstrates the effectiveness of all components.\n\nTo determine with whom, what, and when to share and accumulate information for effective GSL “Graphs Help Graphs” method is proposed (GHG). GHG uses graph-driven organizational structure to autonomously select interacting agents and manage interaction strength. In addition, synthetic graphs are built as interaction medium based on demand of agents. These help to build prototypes in life cycle to help select optimal parameters. An empirical study accompanies that demonstrates the effectiveness.", "questions": "1. what is good example of the scenario mentioned regarding complete new info being learned each time\n2. the data sets used for benchmarking have been out in the open and may be learned, how do we check for that\n3. isnt artificial partitioning using spectral partitioning giving away info on the labels\n4. It is unclear how many agents are sufficient and what is the interrelationship between the number of agents and algo perf.\n5. lot of hand waving in the paper than describing actual algo level details of how the social part happens", "limitations": "1. what is good example of the scenario mentioned regarding complete new info being learned each time\n2. the data sets used for benchmarking have been out in the open and may be learned, how do we check for that\n3. isnt artificial partitioning using spectral partitioning giving away info on the labels\n4. It is unclear how many agents are sufficient and what is the interrelationship between the number of agents and algo perf.\n5. lot of hand waving in the paper than describing actual algo level details of how the social part happens", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Authors should work on answering following to improve the quality of submission:\n1. what is good example of the scenario mentioned regarding complete new info being learned each time\n2. the data sets used for benchmarking have been out in the open and may be learned, how do we check for that\n3. isnt artificial partitioning using spectral partitioning giving away info on the labels\n4. It is unclear how many agents are sufficient and what is the interrelationship between the number of agents and algo perf.\n5. lot of hand waving in the paper than describing actual algo level details of how the social part happens and what is the inter-relationship of the Experimental tasks on the chosen five datasets vis-a-vis this social interaction.", "was_revised": true, "final_justification": "I still feel that \n1. the algorithm steps, while clarified (and should be in main body) have gaps. \n2. The trend on increasing agents and 0 forgetting was also not explained satisfactorily\n3. each new paper is said to be completely new info. We can have lot of overlap, and similar approaches developed independently. Each paper usually builds on previous ones and completely novel information that move the field are relatively smaller occurrence", "rebuttal": "Thanks for your constructive comments!\n\n> Q1: What is good example of the scenario mentioned regarding complete new info being learned each time.\n\nA1: We use real-world student learning and the Google Scholar citation network as concrete cases to illustrate concept of \"complete new information being learned each time\".\n\n* **Expansion of student knowledge network.**\n \n1. Old information corresponds to knowledge networks of subjects such as Chinese, English, and Politics.  \n\n2. Complete new information corresponds to separate knowledge networks of Mathematics, Physics, and Chemistry.  \n\n3. Students need to learn to correctly categorize knowledge of Chinese, English, and politics subjects. Then they learn to partition knowledge of Mathematics, Physics, and Chemistry correctly, while not forgetting the knowledge of previous subjects.\n\n* **Growth of Google Scholar citation network.**  \n1. Information for task $t-1$ is represented by papers and their citation relationships on topics such as graph neural networks, machine learning, and reinforcement learning.  \n\n2. Complete new information for task $t$ is represented by papers and their citation relationships on topics such as federated learning, generative models, and large language models. These topics and their graph data are entirely disjoint from those of task $t-1$.  \n\n3. Our model needs first to learn to assign the task $t-1$ papers to their correct topics, then learns to assign completely new papers of task $t$ to correct issues, without forgetting how to classify papers from task $t-1$ when training data from $t-1$ cannot be accessed.\n\n> Q2: The data sets used for benchmarking have been out in the open and may be learned, how do we check for that.\n\nA2: We sincerely clarify that our model has not been pre-trained on any public datasets, and all models are trained from scratch. We use benchmark datasets to evaluate performance of our proposed method, simulating real-world graph social learning process. Taking CoraFull citation network dataset as an example, which comprises 70 classes, we elaborate on graph data partitioning and our goal of graph socialized learning.\n\n* **Graph data partitioning setups.**\n\n1. Graph data is divided into 7 tasks according to class labels, with each task containing 10 distinct classes. Classes across tasks are non-overlapping.  \n\n2. For each task, data from the 10 classes are split into 5 non-overlapping subsets via Dirichlet partitioning, which are then assigned to 5 agents. Classes may overlap among agents, but individual nodes do not.\n\n* **Goal of graph socialized learning.**\n\n1. Five agents collaboratively learn 10 classes within each task. For every task, both training and test sets of each agent are drawn exclusively from these 10 classes. We demonstrate that post-collaboration performance of agents significantly surpasses pre-collaboration performance in experiments.  As shown in Table 1, the performance of collaborative method (GHG) significantly surpasses that of single agent (Single). \n\n2. During continual learning, at task $t$, training set of multi-agent system contains only data from task $t$, while test set includes data from tasks $1, 2, ..., t$. After learning task $t$, multi-agent model needs to not only achieve strong performance on current task but also retain all knowledge of previous tasks and maintain its classification ability on all old tasks. GHG achieves a forgetting rate of 0 in Table 1, indicating that it can retain all previously learned information without loss.\n\n**Table 1: Performance comparison (MAP, MAF (\\%)) of the proposed GHG and Single on five datasets.**\n| |CoraFull| |Arxiv| |Reddit| |Cora| |Citeseer| |\n|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|:----|\n| |MAP$\\uparrow$ |MAF$\\downarrow$ |MAP$\\uparrow$ |MAF$\\downarrow$ |MAP$\\uparrow$ |MAF$\\downarrow$ |MAP$\\uparrow$ |MAF$\\downarrow$ |MAP$\\uparrow$ |MAF$\\downarrow$|\n|Single|4.2|36.7|8.4|35.1|15.8|60.1|18.2|54.9|16.9|54.9|\n|GHG|**54.0**|**0.0**|**59.2**|**0.0**|**86.7**|**0.0**|**79.2**|**0.0**|**65.7**|**0.0**|\n\n> Q3: Isnt artificial partitioning using spectral partitioning giving away info on the labels.\n\nA3: We clarify that experiments do not use spectral partitioning but instead adopt Dirichlet partitioning [1] for data splitting. The node allocation ratio for each class is denoted as $Dir_A (het)$, and $A$ is the number of agents. A lower value of $het$ indicates more substantial data heterogeneity. This partitioning method is used to simulate heterogeneous environments commonly observed in real-world scenarios.\n\nEach agent has access only to its local training data and labels for the current task, with no additional information. Due to limited and diverse local data, individual agents struggle to perform well alone. Effective collaboration is essential to achieve strong performance on test set of tasks.\n\n[1] Bayesian nonparametric federated learning of neural networks. ICML, 2019.\n\n> Q4: It is unclear how many agents are sufficient and what is the interrelationship between the number of agents and algo perf.\n\nA4: We explain that in strongly heterogeneous environments, involving more agents is beneficial for performance. In contrast, in weakly heterogeneous environments, a sufficient number of agents may exist. We demonstrate this through experiments on CoraFull (strong heterogeneity) and Cora (weak heterogeneity) datasets with varying numbers of agents in Table 2. \n\n* **Experimental setup.** Our setting involves multiple agents collaboratively solving every task. The total amount of data per task is fixed. As the number of agents increases, data becomes more distributed, and each agent receives a smaller portion of the data.\n* **Experimental results.**\nTable 2 shows the performance with different numbers of agents on CoraFull and Cora datasets.\n1. On CoraFull dataset, due to strong heterogeneity among agents, increasing the number of agents makes collaboration more challenging, leading to lower performance. For example, in an extreme case, agent 1 may only have data from class 1, agent 2 from class 2, and agent 3 from class 3. It becomes challenging for agent 1 to improve performance across all three classes (classes 1, 2, and 3). \n\n2. On Cora dataset, each task contains only two classes. Even when more agents are involved, the degree of heterogeneity remains limited compared to other datasets. The performance reaches its peak when the number of agents is 6. Therefore, 6 may be a sufficient number of agents for the Cora dataset.\n\n* **Result analysis.**\nIn strongly heterogeneous environments, each agent has its strengths, and social collaboration among agents is essential for better performance. In this case, involving more agents tends to be beneficial. In weakly heterogeneous environments, there may exist a sufficient number of agents beyond which performance no longer improves.\n\n**Table 2: Analysis of the number of agents (MAP (%)) on CoraFull and Cora datasets.**\n|No. Agents|2|5|10|20| \n|:----|:----|:----|:----|:----|\n| CoraFull|**77.6**|54.0|44.6|43.0| \n\n|No. Agents|2|4|6|8|10|\n|:----|:----|:----|:----|:----|:----|\n|Cora|79.2|77.7|**83.6**|75.6|77.6|\n\n> Q5: Lot of hand waving in the paper than describing actual algo level details of how the social part happens.\n\nA5: Since our algorithm is inspired by human society, we intentionally adopt specific sociological and human society terms in our descriptions to help readers better understand motivation and the source of inspiration behind algorithm. As Reviewer sYbT also noted, this writing and organizational style helps guide readers smoothly from motivation through methodology to theoretical and empirical findings. As shown in Table 3,  we elaborate on the relationship from human society to machine society to algorithm design of our graph socialized learning method. \n\t\n* Who (Organizational structure): In human society, the question of whom to interact with is essential. Similarly, in machine society, the organizational structure of multi-agents plays a crucial role, as it determines collaboration relationships. **Algorithm details.** By considering complementarity and similarity, we construct a collaborative graph. The agents are updated using the cross-entropy loss computed from generated synthetic graphs and the KL divergence loss derived from imitating other agents’ outputs for information transfer.\n\n* What (Interaction medium): In human interactions, what is exchanged during communication is a valuable question. Analogously, in machine society, the medium of interaction between agents significantly affects the efficiency of communication. \n**Algorithm details.** The initial class distribution of the synthetic graphs is computed using the relative class sizes and fine-grained collaboration weights. Synthetic graphs are then generated through mean and variance alignment.\n\n* When (Life cycle): In human society, when interactions occur is a topic worthy of study. In machine society, the life cycle of agents is closely linked to their evolution and capacity for capability growth.\n**Algorithm details.** Task prototypes are added gradually and task ID is queried to select optimal prompt and classifier for inference.\n\n**Table 3: The relationship from human society to machine society to algorithm design.**\n\n|Human society|Machine society|Algorithm design|\n|:----|:----|:----|\n|Who|Organizational structure|Collaborative graph construction, Complementarity and similarity calculation, Collaborative information transfer|\n|What|Interaction medium|Synthetic graphs generation, Customized initial label distribution computation|\n|When|Life cycle|Task ID prediction based on prototypes, Prompts and classifiers selection|"}, {"review_id": "9yBb2YU2Ck", "reviewer": "Reviewer_4UrH", "summary": "This paper presents a Graph Socialized Learning (GSL) paradigm to address collaboration challenges in fragmented and dynamic graphs. The proposed method, Graphs Help Graphs (GHG), determines with whom, what, and when to share and accumulate information for effective multi-agent learning. GHG utilizes a graph-driven organizational structure to autonomously select interacting agents and manage interaction strength, generates customized synthetic graphs based on agent demands, and applies them to build prototypes over the life cycle. Extensive empirical studies validate the effectiveness of GHG in heterogeneous dynamic graph environments.", "questions": "1. Can the authors provide a more comprehensive analysis of the computational and communication costs of GHG?\n2. How robust is the task prototype mechanism to errors in task ID prediction, especially in more complex or overlapping tasks? And how does GHG perform under partial or unreliable collaboration—e.g., limited agent availability or communication failures?", "limitations": "While the authors acknowledge that \"the primary limitation of GHG lies in its insufficiency to consider relationships among graph tasks for multiple agents,\" I believe the paper would benefit from a broader discussion of the limitations of GSL and GHG compared to existing paradigms such as Graph Federated Learning (GFL) and Graph Lifelong Learning (GLL).", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The proposed GHG (Graphs Help Graphs) is both well-motivated and novel. It addresses real challenges in multi-agent graph learning with a fresh perspective.\n2. The GHG method combines graph structure, synthetic data generation, and task-aware collaboration in a unified and creative way. The method is structured around three intuitive questions—who to collaborate with, what to share, and when to share—making the approach easy to understand and practically meaningful.\n3. The approach shows consistently strong performance across several benchmarks, outperforming existing baselines in both accuracy and forgetting.\n4. The paper is well-written and clearly organized. The technical content is presented logically, and the mathematical derivations are sound and easy to follow.\n\nWeaknesses:\n1. Although the paper claims the method is efficient, it lacks detailed analysis and comparison of actual time, memory, and communication costs, which are important for real-world deployment. Simply reporting MAP across different interaction rounds is not sufficient to support the efficiency claim.\n2. The paper assumes agents can accurately identify task IDs using prototypes, but it doesn’t deeply discuss what happens if this step fails or is noisy.\n3. All experiments are on benchmark datasets. A real-world example (e.g., in citation networks or recommendation systems) would help show practical value.", "was_revised": true, "final_justification": "Post rebuttal, concerns regarding efficiency, robustness and real-world examples have been resolved. So I would like to recommend accepting the paper.", "rebuttal": "We appreciate your acknowledgment of our motivation, novelty, practicality, organization, and good performance. Thanks for your valuable suggestions!\n\n\n\n> W1: Although the paper claims the method is efficient, it lacks a detailed analysis and comparison of actual time, memory, and communication costs.\n\n> Q1: Can the authors provide a more comprehensive analysis of the computational and communication costs of GHG?\n\nA1: We provide a detailed analysis of the actual time, memory, and communication costs. As shown in Table 1, our method offers five advantages over baselines: **shorter runtime, lightweight model, no need for replay, lower memory usage, and reduced interaction cost.**\n\n**Table 1: Comparison of actual time, model params, memory size, and communication costs on CoraFull dataset. $A$, $T$, $R$, and $C$ denote the number of agents, the number of tasks, the number of interactions, and the number of classes per task, respectively. $d^O$ and $d$ represent the dimensions of original features and model outputs, respectively.**\n|Method |Time cost (s)|Model content|Model params (M)|Memory content|Memory Size (MB) |MAP (%)|\n|:----|:----|:----|:----|:----|:----|:----|\n|Fed-DMSG|**881**|$A$ agents, Server |33.7|Replay graph|79.6|37.3|\n|POWER|1403|$A$ agents, Server|20.4|Replay graph, Global replay graph|86.2|25.3|\n|MASC|1215|$A$ agents, Server|13.5|Replay graph|79.6|36.7|\n|GHG|1092|$A$ agents|**11.5**|Prompts, Classifiers, Prototypes |**7.5**|**54.0**|\n\n|Method |Communication overhead|Communication content|Communication params (M)|\n|:----|:----|:----|:----|\n|Fed-DMSG|$O(ATRM)$|Model parameters $M$|1885.0|\n|POWER|$O(ATRM+ATCM_{grad})$|Model parameters $M$, Class prototype gradients $M_{grad}$|632.5|\n|MASC|$O(ATR(N_Ad+M_h)+ATN_Ad^O)$|Original graph $N_Ad^O$, Classifiers $M_h$, Node embeddings $N_Ad$|109.4|\n|GHG|$O(AATR((N_{syn}+2C+3)d$|Prompts $M_{\\phi}$, Classifiers $M_h$, Synthetic graphs $N_{syn}d^O$, Node embeddings $(N_{syn}+2C+3)d$,|**105.0** |\n||$+(N_{syn}+C+1)d^O+M_\\phi+M_h))$|Topology-aware embeddings $(C+1)d^O$| |\n\n\n\n\n\n\n\n\n> W2: The paper assumes agents can accurately identify task IDs using prototypes, but it doesn’t deeply discuss what happens if this step fails or is noisy.\n\n> Q2: How robust is the task prototype mechanism to errors in task ID prediction, especially in more complex or overlapping tasks? And how does GHG perform under partial or unreliable collaboration—e.g., limited agent availability or communication failures?\n\n\nA2: We evaluate the accuracy of the task prototype mechanism under three scenarios: task overlap, partial collaboration, and unreliable collaboration. The results are presented in Tables 2, 3, and 4. We conclude that the task prototype mechanism is robust in the cases of task overlap and partial collaboration. However, in the scenario of unreliable collaboration, especially when the level of unreliability is high, the mechanism may produce some errors.\n\n* **Task overlap.**\n\n1. **Experimental setup.** To simulate a task overlap scenario, we evenly split the training nodes of each class and set the maximum number of repetitions for each class to 3. For the CoraFull, Arxiv, and Reddit datasets, we increase the number of classes per task from 10 to 15. For the Cora and Citeseer datasets, we adopt an extreme case of task overlap: each dataset contains only 6 total classes, and we increase the number of classes per task from 2 to 5. Taking the Citeseer dataset as an example, the class distributions for the 3 tasks are: [0,1,2,4,5], [0,1,3,4,5], and [1,2,3,4,5].\n\n2. **Experimental results.** The results on five datasets are shown in Table 2. We can observe that the task prototype mechanism remains robust under task-overlapping conditions.\n\n\n**Table 2: Performance of GHG under the scenario of task overlap on five datasets.**\n|Dataset|CoraFull|Arxiv|Reddit|Cora|Citeseer|\n|:----|:----|:----|:----|:----|:----|\n|MAP (%)|37.5|48.2| 82.3|48.1|40.1|\n|MAF (%)|0.0|0.0| 0.0|0.0|0.0|\n\n* **Partial collaboration.**\n\n1. **Experimental setup.** In each interaction round, we randomly select the number of collaborators for each agent based on a partial collaboration ratio to simulate limited agent availability or communication failures. The ratio is set to $[0,0.2,0.4,0.6,0.8]$, where a higher ratio indicates more limited agent availability.\n\n2. **Experimental results.** The results on five agents from the CoraFull dataset are shown in Table 3. The task prototype mechanism remains robust under partial collaboration conditions.\n\n\n**Table 3: Performance of GHG under the scenario of partial collaboration in five agents on CoraFull datasets. Please note that when ratio equals 0.8, the agent becomes a single agent and no collaboration is achieved, and the prototype in this case is inaccurate.**\n|Ratio|0|0.2|0.4|0.6|0.8 (Single agent, no collaboration)|\n|:----|:----|:----|:----|:----|:----|\n|MAP (%)|54.0|50.2|46.6|42.1|29.6|\n|MAF (%)|0.0|0.0| 0.0|0.0|*3.0*|\n\n\n\n* **Unreliable collaboration.**\n\n1. **Experimental setup.** To simulate unreliable collaboration in multi-agent system, we randomly select multiple agents according to a predefined unreliable collaboration ratio. For these selected agents, we inject noise into their graph data. Specifically, we add Gaussian noise to node features, as well as randomly add and remove edges to simulate noisy communication. The ratio is set to $[0,0.2,0.4,0.6,0.8]$, where a higher ratio indicates more noise present in the multi-agent system.\n\n\n2. **Experimental results.** The results in Table 4 show that under unreliable collaboration, when the ratio reaches 0.6 and 0.8, some task prototypes are incorrectly predicted. This may be due to the influence of noisy data received from other agents. In the future, we plan to explore further strategies for handling unreliable collaboration, such as graph adversarial robustness [1][2][3][4] and graph denoising learning [5], to enhance the robustness of our framework.\n\n[1] Can large language models improve the adversarial robustness of graph neural networks? KDD, 2025.\n\n[2] Are your models still fair? fairness attacks on graph neural networks via node injections. NeurIPS, 2024.\n\n[3] Adversarial training for graph neural networks: Pitfalls, solutions, and new directions. NeurIPS, 2023.\n\n[4] Gnnguard: Defending graph neural networks against adversarial attacks. NeurIPS, 2020.\n\n[5] Towards understanding and reducing graph structural noise for GNNs. ICML, 2023.\n\n**Table 4: Performance of GHG under the scenario of unreliable collaboration in five agents on CoraFull datasets.**\n|Ratio|0|0.2|0.4|0.6|0.8|\n|:----|:----|:----|:----|:----|:----|\n|MAP (%)|54.0|44.4|39.2|30.8|27.3|\n|MAF (%)|0.0|0.0| 0.0|*3.3*|*0.3*|\n\n\n\n\n\n\n\n> W3: All experiments are on benchmark datasets. A real-world example (e.g., in citation networks or recommendation systems) would help show practical value.\n\nA3: We use benchmark datasets followed by [6][7] to simulate real-world environments. Moreover, these datasets are reliable, publicly available, and allow subsequent researchers to follow our work. Testing on real-world datasets is significant. In the future, we plan to conduct experiments on real-world citation networks or recommendation-system datasets. For instance, collecting Google Scholar papers and their citation relationships, or gathering Amazon purchasers and their product data to construct a million-scale real graph data. If you have any recommended datasets, please let us know. We would be delighted to evaluate our methods on them.\n\n[6] CGLB: Benchmark tasks for continual graph learning. NeurIPS, 2022.\n\n[7] OpenFGL: A comprehensive benchmarks for federated graph learning. VLDB, 2025."}, {"review_id": "zKvgdOJgdH", "reviewer": "Reviewer_S53e", "summary": "The authors formalize **multi-agent graph learning** (MAGL), where multiple agents, each with its own graph, collaborate to train GNNs.\n\nThen, they show the limitations of existing paradigms for MAGL.\n\n- **Graph Federated Learning** breaks down under graph heterogeneity (it treats all peers equally and forces aggregation even when graphs differ wildly), and\n- **Graph Lifelong Learning** doesn’t leverage true peer-to-peer collaboration and still suffers from catastrophic forgetting.\n\nAs an alternative, they introduce **Graph Socialized Learning**, which lets agents learn **who** to collaborate with (via a learned collaboration graph), **what** to exchange (on-demand synthetic subgraphs tailored to each peer), and **when** to communicate (using a prototype-based lifecycle), thereby eliminating forgetting and handling heterogeneity.\n\nExtensive experiments on five citation/social-network benchmarks show that GSL achieves up to ~86% accuracy **with zero forgetting**, substantially outperforming federated, lifelong, and hybrid baselines.", "questions": "- **[Q1]** The zero-forgetting claim relies on storing and replaying task prototypes and synthetic subgraphs. Can you clarify the total memory footprint of these stored items for GSL and confirm that all baselines were given an equivalent memory budget when measuring forgetting? If not, how might unequal memory allowances bias the comparison?\n- **[Q2]** How does the complexity (compute and storage) of learning an A by A collaboration matrix scale as A grows, and have you considered sparse or hierarchical approximations for large-scale deployments?\n- **[Q3]** Have you evaluated whether a single average embedding per class/task sufficiently captures complex or multimodal distributions? Would using multiple prototypes per class improve performance?\n- **[Q4]** Given that all experiments use citation/social graphs, how do you expect GSL to perform on, say, biological networks, user-item graphs, or truly dynamic graphs with evolving topology?\n- **[Q5]** Beyond qualitative assertions, can you provide formal privacy bounds or adversarial-robustness analyses for your synthetic-graph exchanges to rule out potential information leakage?", "limitations": "Yes. In Appendix E.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "# Strengths\n\n- **[S1] Novel Paradigm**: Learning *who*, *what*, and *when* to communicate is a fresh take on MAGL, moving beyond “one-size-fits-all” aggregation.\n- **[S2] Effective Synthetic Messaging**: Tailored subgraphs offer targeted knowledge transfer while preserving privacy and reducing bandwidth.\n- **[S3] Zero-Forgetting Guarantee**: Prototype-driven replay convincingly prevents catastrophic forgetting without storing raw data.\n- **[S4] Empirical Rigor**: Thorough ablations, heterogeneity sweeps, and agent-scale analyses validate each component’s contribution.\n\n# Weaknesses\n\n- **[W1] Scalability of Collaboration Graph: It is unclear whether it is scalable to learn an A×A adjacency for large A, and related discussions are lacking.**\n- **[W2] Prototype effectiveness:** Using a single average embedding per class/task might be too coarse in highly heterogeneous or multimodal settings.\n- **[W3] Dataset Diversity:** All benchmarks are citation or social networks. It’d be good to see experiments on biological graphs, e-commerce interaction data, or truly evolving graphs to test generalization.\n- **[W4] Privacy and security guarantees:** Synthetic subgraphs are safer than raw data, but no formal privacy proofs or adversarial-robustness analyses are provided.", "was_revised": true, "final_justification": "The authors formalize multi-agent graph learning (MAGL), where multiple agents, each with its own graph, collaborate to train GNNs.\n\nThen, they show the limitations of existing paradigms for MAGL.\n\nGraph Federated Learning breaks down under graph heterogeneity (it treats all peers equally and forces aggregation even when graphs differ wildly), and\nGraph Lifelong Learning doesn’t leverage true peer-to-peer collaboration and still suffers from catastrophic forgetting.\nAs an alternative, they introduce Graph Socialized Learning, which lets agents learn who to collaborate with (via a learned collaboration graph), what to exchange (on-demand synthetic subgraphs tailored to each peer), and when to communicate (using a prototype-based lifecycle), thereby eliminating forgetting and handling heterogeneity.\n\nExtensive experiments on five citation/social-network benchmarks show that GSL achieves up to ~86% accuracy with zero forgetting, substantially outperforming federated, lifelong, and hybrid baselines.\n\n# Strengths\n\n- **[S1] Novel Paradigm**: Learning *who*, *what*, and *when* to communicate is a fresh take on MAGL, moving beyond “one-size-fits-all” aggregation.\n- **[S2] Effective Synthetic Messaging**: Tailored subgraphs offer targeted knowledge transfer while preserving privacy and reducing bandwidth.\n- **[S3] Zero-Forgetting Guarantee**: Prototype-driven replay convincingly prevents catastrophic forgetting without storing raw data.\n- **[S4] Empirical Rigor**: Thorough ablations, heterogeneity sweeps, and agent-scale analyses validate each component’s contribution.", "rebuttal": "We extend our appreciation to you for strong recognition of our novelty, effectiveness, zero-forgetting guarantee, empirical rigor. Thanks for your positive comments!\n\n> W1: Scalability of Collaboration Graph.\n\n> Q2: How does complexity (compute and storage) of learning an A by A collaboration matrix scale as A grows, and have you considered sparse or hierarchical approximations for large-scale deployments?\n\nA1: We analyze computational, storage, and communication complexity of multi-agent system. The proposed GHG already adopts a sparse approach. However, if adjacency matrix $A×A$ is particularly large, current method may indeed incur particular computational and storage overhead. In the future, we plan to explore hierarchical approaches [1] to address this issue.\n\n[1] Pc-agent: A hierarchical multi-agent collaboration framework for complex task automation on pc. Arxiv, 2025.\n\n* **Compute and storage complexity.** \n\n1. **Compute complexity** can be expressed as $O(\\textbf{A}TR((|V|+|\\overline{V}|)(4nd^oE_1+2dCE_1+d^2+ldE_2) + (|\\overline{V}|+2C+1)d + \\mathcal{P}))$. Time complexity is linearly related to $A$. Due to space constraints, detailed calculation of time complexity is presented in the appendix. \n\n2. **Storage complexity** is $O(\\textbf{A}(M+TM_\\phi+TM_h+Td^O))$, which consists of models $M$, prompts $M_{\\phi}$, classifiers $M_h$, and prototypes $d^O$. It is linearly related to $A$.\n\n3. **Communication complexity** is $O(\\textbf{AA}TR((N_{syn}+2C+3)d+(N_{syn}+C+1)d^O+M_\\phi+M_h))$, which consists of prompts $M_{\\phi}$, classifiers $M_h$, synthetic graphs $N_{syn}d^O$, node embeddings $(N_{syn}+2C+3)d$, and topology-aware embeddings $(C+1)d^O$. Communication complexity is related to the square of $A$.\n\n* **GHG adopts a sparse strategy to perform collaboration process.** The sparsity is defined as: $\\mathcal{B}_{ab} = \\mathbb{1}[b \\in \\operatorname{TOPK}(\\delta A, M[a,:])]$, meaning that each agent collaborates with only $\\delta A$ agents based on collaboration strength.\n\n> W2: Prototype effectiveness.\n\n> Q3: Have you evaluated whether a single average embedding per class/task sufficiently captures complex or multimodal distributions? Would using multiple prototypes per class improve performance?\n\nA2: Prototype is a highly condensed representation of both features and structure of graph data. We argue that a single average embedding can sufficiently capture complex distributions. We conduct experiments under heterogeneous, multimodal, and task-overlapping settings, and find that a single task prototype is still capable of accurately predicting task ID. In addition, we illustrate that assigning multiple prototypes per task may introduce interference and thus does not improve performance in task-overlapping scenario.\n\n* **A single task prototype is capable of capturing complex distributions.** \n\n1. **Experimental setup.** Inter-tasks themselves are strongly heterogeneous, and we enforce pronounced category-level heterogeneity across agents. We use text-attributed graphs Ele-Computers [2] to simulate a multimodal setting, with text features produced by RoBERTa language model serving as node features. Task overlap means that the same category can appear in different tasks. \n\n[2] A Comprehensive Study on Text-attributed Graphs: Benchmarking and Rethinking. NeurIPS, 2023.\n\n2. **Result analysis.** Results of three settings are shown in Table 1, demonstrating that a single task prototype can still accurately predict task ID without forgetting (MAF=0).\n\n**Table 1: Performance of GHG under three complex conditions on CoraFull and Ele-Computers datasets.**\n|Settings|Heterogeneous|Multi-modal|Task overlapping|\n|:----|:----|:----|:----|\n|MAP (%)|45.3|39.6|37.5|\n|MAF (%)|0.0|0.0|0.0|\n\n* **Using multiple prototypes per task may introduce interference.**\n\n1. **Experimental setup.** When tasks overlap, the risk of incorrectly predicting task ID increases. Therefore, we evaluate performance of using multiple prototypes for each task under such settings. Specifically, assuming each task has $n$ prototypes, we apply $k$-means clustering to synthetic graph features to obtain $n$ clusters, and use the mean feature vector of each cluster as a prototype. \n\n2. **Result analysis.** As shown in Table 2, a single prototype can accurately predict task ID, whereas using multiple prototypes leads to forgetting. This may be because multiple prototypes are too dispersed, causing interference with each other, which negatively impacts comparison between train and test prototypes.\n\n**Table 2: Performance of GHG under multiple prototypes per task on CoraFull dataset.**\n|No. Prototypes|1|2|3|\n|:----|:----|:----|:----|\n|MAP (%)|37.5 |30.2 |33.2 |\n|MAF (%)|0.0|*6.4*|*3.0*|\n\n> W3: Dataset Diversity.\n\n> Q4: How do you expect GSL to perform on, say, biological networks, user-item graphs, or truly dynamic graphs with evolving topology?\n\nA3: We add results on two additional datasets: SLAP (biological) [3] and Computers (e-commerce) [4]. Two datasets have 15 and 10 categories, respectively, and the number of categories for each task is set to 5. Results are presented in Table 3, and our method demonstrates superior performance. \n\nFollowed by [5][6], we simulate real-world scenarios using widely adopted benchmark datasets, which are reliable, publicly available, and facilitate future research to follow up on our work. Evaluating on real-world datasets is indeed valuable. In the future, we plan to conduct more experiments on such datasets. \n\n[3] Non-local attention learning on large heterogeneous information networks. IEEE International Conference on Big Data, 2019.\n\n[4] Pitfalls of graph neural network evaluation.  NeurIPS Workshop, 2018.\n\n[5] CGLB: Benchmark tasks for continual graph learning. NeurIPS, 2022.\n\n[6] OpenFGL: A comprehensive benchmarks for federated graph learning. VLDB, 2025.\n\n**Table 3: Performance comparison (MAP, MAF (%)) of GHG and baselines in five agents on SLAP and Computers datasets. Due to space constraints, we only present a portion of results.**\n|Dataset |SLAP| |Computers| |\n|:----|:----|:----|:----|:----|\n| |MAP $\\uparrow$|MAF $\\downarrow$|MAP $\\uparrow$|MAF $\\downarrow$|\n|Single|8.2|34.1|13.6|30.8|\n|Fed-TPP|27.2|12.3|36.5|0.0|\n|Fed-DMSG|18.0 |9.8|32.2|3.6|\n|POWER|15.5|20.1|10.7|24.4|\n|MASC|15.0|18.6|24.2|21.6|\n|GHG|**62.4**|**0.0**|**82.5**|**0.0**|\n\n> W4: Privacy and security guarantees.\n\n> Q5: Can you provide formal privacy bounds or adversarial-robustness analyses for your synthetic-graph exchanges to rule out potential information leakage?\n\nA4: We conduct Membership Inference Attacks (MIA) and Structure Reconstruction Attacks (SRA) on synthetic graphs. To prevent information leakage during graph synthesis, we apply Differential Privacy (DP), enabling an analysis of adversarial robustness. Finally, we provide the corresponding privacy bounds.\n\n* **Membership inference or structure reconstruction attack.**\n\nAs shown in Table 4, we report the average MIA success rate and SRA precision across all synthetic graphs. It can be observed that MIA scores are consistently higher than those of SRA. Although synthetic node features are generated based on original topology, synthetic graphs do not directly transmit topological structure. Therefore, original graph structure is well protected.\n\n* **Differential privacy for graph synthesis.**\n\nTo enhance privacy protection of shared synthetic graphs, we apply feature perturbation using Gaussian mechanism [7] to achieve differential privacy. As shown in Table 4, both MIA success rate and SRA precision decrease, indicating improved privacy. Although performance of GHG slightly drops due to DP mechanism, it still significantly outperforms baselines. This demonstrates that GHG is robust when synthetic graphs can roughly capture global graph distribution.\n\n**Table 4: Adversarial-robustness analyses on CoraFull dataset.**\n|Method|MIA success rate (%)|SRA precision (%)|MAP (%)|\n|:----|:----|:----|:----|\n|GHG|9.7|1.0|**54.0**|\n|GHG-DP| **4.9**|**0.9** | 38.4|\n\n* **Privacy bounds.**\nIf a function has $L_2$ sensitivity of $\\Delta f$, then by adding Gaussian noise with zero mean and standard deviation\n$$\n\\sigma \\geq \\frac{\\sqrt{2 \\ln (1.25 / \\delta)} \\cdot \\Delta f}{\\epsilon},\n$$\nthe mechanism satisfies $(\\epsilon, \\delta)$-differential privacy [7][8]. Here, $\\epsilon$ represents degree of privacy leakage, $\\delta$ is the probability of a privacy breach beyond $\\epsilon$, and $\\Delta f$ denotes $L_2$ sensitivity of function.\n\n[7] The Algorithmic Foundations of Differential Privacy. Foundations and trends in theoretical computer science, 2013. \n\n[8] The bounded Gaussian mechanism for differential privacy. Arxiv, 2022.\n\n> Q1: Can you clarify total memory footprint of these stored items for GSL and confirm that all baselines were given an equivalent memory budget when measuring forgetting? If not, how might unequal memory allowances bias the comparison?\n\nA5: We clarify that newly introduced prompts, classifiers, and task prototypes are preserved to address forgetting issue. The purpose of synthetic graphs is solely to compute task prototypes, and it does not need to be stored after each task is completed. Each task prototype has a shape of $[1, d]$, meaning that the proposed GHG introduces only minimal memory overhead. In contrast, replay-based methods need to store replay graphs, where feature dimension is $[n, d]$ (with $n \\gg  1$). If graph topology also needs to be stored, memory consumption increases even further. Table 5 presents memory usage comparison between baselines and GHG. We conclude that GHG achieves superior performance while consuming significantly less memory in mitigating forgetting.\n\n**Table 5: Comparison of memory size on CoraFull dataset.**\n|Method|Content|Memory Size (MB)|MAP (%)|\n|:----|:----|:----|:----|\n|Fed-DMSG|Replay graph|79.6|37.3|\n|POWER|Replay graph, Global replay graph|86.2|25.3|\n|MASC|Replay graph|79.6|36.7|\n|GHG|Prompts, Classifiers,Prototypes |**7.5**|**54.0**|"}]

Author rebuttal:

Meta-review: This paper proposes a graph socialized learning paradigm, instantiated as the "Graphs Help Graphs" method, to address multi-agent graph learning in heterogeneous and dynamic environments. The framework formalizes who to collaborate with, what to share, and when to communicate in a unified Bayesian paradigm. Key components include graph-driven organizational structures, synthetic subgraph exchange, and prototype-based life cycle management.

**Strengths:**
The paper is well-written, logically organized, and presents a novel and well-motivated paradigm for multi-agent graph learning. The idea of socialized collaboration (who/what/when) is intuitive, and the design integrates multiple techniques in a principled Bayesian way. Experiments are extensive, with ablations and heterogeneity sweeps, showing strong improvements over federated, lifelong, and hybrid baselines. The framework convincingly addresses both catastrophic forgetting and graph heterogeneity.

**Weaknesses and Suggestions:**
1. Despite its contributions, the approach is relatively complex, involving multiple modules and hyperparameters, which may raise reproducibility and scalability concerns. 
2. The proposed formulation is closely related to DeLAMA [38]. A more thorough discussion of the differences and contributions is needed in the final version.
3. In the original version, the experiments are limited to citation/social-network benchmarks. Results on real-world applications (as provided in the rebuttal) should be incorporated into the final version.
4. The original paper lacks analysis of efficiency in terms of time, memory, and communication. The corresponding discussions from the rebuttal should also be included in the final version.

**Conclusion:**
Overall, this is a novel and impactful contribution to multi-agent graph learning, addressing both theoretical and practical challenges. While some limitations remain, they do not overshadow the significance of the work. Therefore, I recommend acceptance.
Decision: Accept (poster)