Disentangled and Self-Explainable Node Representation Learning

## Abstract

Abstract Node representations, or embeddings, are low-dimensional vectors that capture node properties, typically learned through unsupervised structural similarity objectives or supervised tasks. While recent efforts have focused on explaining graph model decisions, the interpretability of unsupervised node embeddings remains underexplored. To bridge this gap, we introduce DiSeNE ( Di sentangled and Se lf-Explainable N ode E mbedding), a framework that generates self-explainable embeddings in an unsupervised manner. Our method employs disentangled representation learning to produce dimension-wise interpretable embeddings, where each dimension is aligned with distinct topological structure of the graph. We formalize novel desiderata for disentangled and interpretable embeddings, which drive our new objective functions, optimizing simultaneously for both interpretability and disentanglement. Additionally, we propose several new metrics to evaluate representation quality and human interpretability. Extensive experiments across multiple benchmark datasets demonstrate the effectiveness of our approach.

## 1 Introduction

Self-supervised and unsupervised node representation learning [] provide a powerful toolkit for extracting meaningful insights from complex networks, making them essential in modern AI and machine learning applications related to network analysis []. These methods offer flexible and efficient ways to analyze high-dimensional networks by transforming them into low-dimensional vector spaces. This transformation enables dimensionality reduction, automatic feature extraction, and the use of standard machine learning algorithms for tasks such as node classification, clustering, and link prediction []. Furthermore, self-supervised node representations, or embeddings, enable visualization of complex networks and can be transferred across similar networks, enhancing understanding and predictive power in fields ranging from social networks to biological interactions.

Although widely adopted, unsupervised representation learning methods often face substantial challenges in terms of interpretability, necessitating complex and indirect approaches to understand what the learned embeddings actually represent [, , ].
This raises a critical question: What information do these embeddings encode?

While there has been a large body of work on explainable GNN models, limited attention has been given to embedding methods, which are the fundamental building blocks of graph based models.
Existing efforts to explain embeddings are predominantly post-hoc [, , , ] and heavily dependent on the initial embedding techniques used.
Some approaches [] build upon existing literature for post-processing word embeddings [, ], focusing on minimizing reconstruction errors of node embeddings using over-complete auto-encoders to improve sparsity. Other works [, , ] focus solely on extracting meaningful explanations without addressing the learning process itself.

We propose DiSeNE (Disentangled and Self-Explainable Node Embedding), a framework that addresses these gaps by generating self-explainable unsupervised node embeddings.
Our methodology is based on disentangled representation learning to encode node embeddings that are interpretable on a dimension-wise basis. Disentangled representation learning [] aims to encode latent variables that represent separate factors of variation in data, ensuring that each latent dimension corresponds to a distinct, independent aspect.

Figure: Figure 1: DiSeNE generates dimension-wise disentangled representations in which each embedding dimension is mapped to a mesoscale substructure in the input graph. The vector represents the embedding for the node marked in blue and the bars depict feature values.
Refer to caption: x1.png

In graph data, node behaviour is strongly influenced by mesoscale structures like communities, which shape the network’s organization and drive dynamics []. By leveraging disentangled representation learning, we capture these topological substructures more effectively, with each embedding dimension reflecting an independent graph unit (see Figure 3). We achieve this by introducing a new objective function to ensure structural disentanglement. Specifically, we optimize the embeddings so that each dimension is predictive of a unique substructure in the input graph. To avoid degenerate solutions, we incorporate an entropy-based regularizer which ensures that the mapped substructures are non-empty.

Our paradigm represents a shift in the language of explanations compared to the ones often considered when dealing with GNNs []. Explainability for GNNs often involves understanding which parts of the local computation graph (nodes, edges) and node attributes significantly influence the model’s predictions [, , ]. On the other hand, the explanations that we aim to discover are inherently non-local, since they could involve mesoscale structures such as node clusters [], usually not included in the GNN computational graph.
To provide a comprehensive evaluation of the embeddings and uncover novel insights, we introduce several new metrics (for details refer to Section 3.3).
For example, our *overlap consistency* measure (depicted in Figure 3) confirms that the physical overlap between the topological substructures identified as explanations aligns with the actual correlation of the corresponding embedding features or dimensions.
This overlap offers insights into the interdependencies of node characteristics within the graph.
Further, we attribute meaning to actual feature values by showing that the magnitude of node embedding entries corresponding to a dimension is correlated with the proximity (depicted in Figure 3) of the corresponding nodes to the “topological” subgraphs associated with that dimension. This measurement aids in understanding the closeness of nodes to different graph features, thereby enhancing spatial awareness within the graph structure.

To summarize we (i) formalize new and essential criteria for achieving disentangled and explainable node representations, offering a fresh perspective on explainability in unsupervised graph-based learning, (ii) introduce novel evaluation metrics to help quantifying the goodness of node representation learning in disentangled and explainable settings (iii) perform extensive experimental analyses to establish state-of-the-art results in self-explainable node feature learning. We release our code and data at [https://github.com/simonepiaggesi/disene](https://github.com/simonepiaggesi/disene).

## 2 Preliminaries and Related Work

Given an undirected graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, node embeddings are obtained through an encoding function
$\mathbf{h}:\mathcal{V}\rightarrow\mathbb{R}^{K}$
that map each node to a points of a $K-$dimensional vector space $\mathbb{R}^{K}$, where typically $D<<|\mathcal{V}|$.
We denote the $K$-dimensional embedding of a node $v\in\mathcal{V}$ as $\mathbf{h}(v)=[h_{1}(v),\dots,h_{K}(v)]$, where $h_{d}(v)$ represents the value of the $d$-th feature of the embedding for node $v$.
Alternatively, we can represent all node embeddings collectively as a matrix $\mathbf{H}(\mathcal{G})\in\mathbb{R}^{V\times K}$, where each entry $\mathbf{H}_{vd}=\mathrm{h}_{d}(v)$ corresponds to the $d$-th feature for node $v$. We can also refer to columns of such matrix, $\mathbf{H}_{:,d}$, as the dimensions of the embedding model space.

#### Node embeddings interpretability.

Node embeddings are shallow encoding techniques, often based on matrix factorization or random walks []. Since the latent dimensions in these models are not aligned with high-level semantics [, ], interpreting embeddings typically involves post-hoc explanations of their latent features [, ]. Other works propose alternative methods to modify existing node embeddings, making them easier to explain with human-understandable
graph features [, ].
From a different viewpoint, [] explore how understandable are the embedded distances between nodes. Similarly, [] investigate whether specific topological features are predictable, and then encoded, in node representations.

#### Graph neural networks interpretability.

Graph Neural Networks (GNNs) [] are deep models that operate via complex feature transformations and message passing. In recent years, GNNs have gained significant research attention, also in addressing the opaque decision-making process. Several approaches have been proposed to explain GNN decision process [], including perturbation approaches [, , ], surrogate
model-based methods [, ], and gradients-based methods [, ]. In parallel, alternative research directions focused on concept-based explanations, i.e. high-level units
of information that further facilitate human understandability [, , ].

#### Disentangled learning on graphs.

Disentangled representation learning seeks to uncover and isolate the fundamental explanatory factors within data []. In recent years, these techniques have gained traction for graph-structured data [, , , ].
For instance, FactorGCN [] disentangles an input graph into multiple factorized graphs, resulting in distinct disentangled feature spaces that are aggregated afterwards. IPGDN [] proposes a disentanglement using a neighborhood routing mechanism, enforcing independence between the latent representations as a regularization term for GNN outputs. Meanwhile, DGCL [] focuses on learning disentangled graph-level representations through self-supervision, ensuring that the factorized components capture expressive information from distinct latent factors independently.

## 3 Our Proposed Framework: DiSeNE

In this section, we begin by outlining the key desiderata for achieving disentangled and self-explainable node representations. Next, we design a novel framework that meets these objectives by ensuring that the learned node representations are both disentangled and interpretable. Finally, we introduce new evaluation metrics to effectively assess the quality of node representation learning in both disentangled and explainable settings.

### 3.1 Core Objectives and Desiderata

In the context of self-supervised graph representation learning, we argue that learning self-explainable node embeddings amounts to reconstructing the input graph in a human-interpretable fashion.
Traditionally, dot-product models based on NMF [] and LPCA [] decompose the set of graph nodes
into clusters, where each entry of the node embedding vector represents the strength of the participation of the node to a cluster.
In this scenario, the dot-product of node embeddings becomes intuitively understandable, as it reflects the extent of shared community memberships between nodes–thereby providing a clear interpretation of edge likelihoods.
This concept is also related to distance encoding methods [, ], where a node feature $h_{d}(u)$ is expressed as a function of the node’s proximity $\zeta(u,\mathcal{S}_{d})=\mathrm{AGG}(\{\zeta(u,v),v\in\mathcal{S}_{d}\})$ to the anchor set $\mathcal{S}_{d}\subset\mathcal{V}$, using an aggregation function $\mathrm{AGG}$.
Typically, distance encodings are constructed by randomly sampling anchor sets [], and used as augmented node features to enhance expressiveness and improve performance on downstream tasks.

Inspired by this idea, our goal is to optimize self-supervised node embeddings encoded by a GNN function $\mathbf{h}:\mathcal{V}\rightarrow\mathbb{R}^{K}$ trained on the graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, such that node features resemble non-random, structurally meaningful anchor sets, thus improving human-interpretability.
To achieve this, we propose three key desiderata for learning general-purpose node representations: *(i)* structural faithfulness *(ii)*, dimensional interpretability, and *(iii)* structural disentanglement. These desiderata serve as the foundational components of our approach, as detailed below.

#### Structural faithfulness.

To implement the approach, we first train node embeddings in recovering the graph structure. We employ a random walk optimization framework based on the skip-gram model with negative sampling []. The loss function for this framework is defined as:

$$ $\mathcal{L}_{\text{rw}}=-\sum_{(u,v)\sim P_{rw}}\log\sigma\big{(}\mathbf{h}(u) ^{\top}\mathbf{h}(v)\big{)}-\sum_{(u\textquoteright,v)\sim P_{n}}\log\sigma \big{(}-\mathbf{h}(u\textquoteright)^{\top}\mathbf{h}(v)\big{)},$ $$

where
$\sigma(\cdot)$ is the sigmoid function,
$P_{rw}$ is the distribution of node pairs co-occurring on random walks (positive samples),
$P_{n}$ is a distribution over randomly sampled node pairs (negative samples),
and $\mathbf{h}(u)^{\top}\mathbf{h}(v)$ represents the dot product between the embeddings of nodes $u$ and $v$.
By optimizing this loss function, we encourage nodes that co-occur in random walks to have similar embeddings, effectively preserving the graph’s structural information in the embedding space.

#### Dimensional interpretability.

Given that our embeddings are structurally faithful–meaning they effectively encode the input graph’s structure–we should be able to interpret each embedding dimension in terms of the graph’s structural properties. We achieve this by
attributing local subgraphs to different latent dimensions.
Consider the likelihood of an edge $(u,v)$, defined as $\hat{y}(u,v;\mathbf{h})=\sigma\Big{(}\sum_{d=1}^{K}h_{d}(u)h_{d}(v)\Big{)}$.
To understand how each dimension $d$ contributes to this likelihood, we compute the edge-wise dimension importance $\phi_{d}(u,v;\mathbf{h})$ as the deviation of the dimension-specific contribution from its average over all edges:

$$ $\phi_{d}(u,v;\mathbf{h})=h_{d}(u)h_{d}(v)-\frac{1}{|\mathcal{E}|}\sum_{(u^{ \prime},v^{\prime})\in\mathcal{E}}h_{d}(u^{\prime})h_{d}(v^{\prime}).$ (1) $$

Since the dot-product is a linear function $\sum_{d=1}^{K}\alpha_{d}h_{d}(u)h_{d}(v)+\beta$ with unitary coefficients $\alpha_{d}\equiv 1$ and zero intercept $\beta\equiv 0$, Eq. (1) corresponds to the formulation of LinearSHAP attribution scores [], using the set of training edges as the background dataset.
Essentially, the attribution function $\phi_{d}(u,v;\mathbf{h})$
indicates whether a specific dimension $d$ contributes positively to an edge’s likelihood.
A positive attribution score means that the dimension increases the likelihood of predicting the edge. Leveraging this property, we generate dimension-wise explanations for the latent embedding model by collecting edge subsets with positive contributions:

$$ $\mathcal{E}_{d}=\{(u,v)\in\mathcal{E}:\phi_{d}(u,v;\mathbf{h})>0\}.$ (2) $$

These self-explanations take the form of global edge masks $\mathbf{M}^{(d)}\in\mathbb{R}_{\geq 0}^{|\mathcal{V}|\times|\mathcal{V}|}$, where each entry is defined as
$M^{(d)}_{uv}(\phi_{d};\mathbf{h})=\mathrm{max}\{0,\phi_{d}(u,v;\mathbf{h})\}$.
By applying these masks to the adjacency matrix $\mathbf{A}$ through Hadamard product ($\odot$), we obtain $\mathbf{A}^{(d)}=\mathbf{A}\odot\mathbf{M}^{(d)}$. Each masked adjacency matrix $\mathbf{A}^{(d)}$ highlights the subgraph associated with dimension $d$.
From these masked adjacency matrices, we construct edge-induced subgraphs $\mathcal{G}_{d}=(\mathcal{V}_{d},\mathcal{E}_{d})$, where $\mathcal{V}_{d}$ is the set of nodes involved in edges $\mathcal{E}_{d}$. These subgraphs act as anchor sets for the model, providing interpretable representations of how each embedding dimension relates to specific structural patterns within the graph. We will refer to edge-induced subgraphs computed as the aforementioned procedure as explanation subgraphs/substructures or topological components of the embedding model.

#### Structural disentanglement.

To enhance the effectiveness of structurally faithful encodings,
each dimension of the latent space should encode an independent structure of the input graph, effectively acting as an anchor subgraph.
Inspired by community-affiliation models [, ], we introduce a node affiliation matrix $\mathbf{F}\in\mathbb{R}^{{|\mathcal{V}|}\times K}$ that captures the association between each node $u\in\mathcal{V}$ and anchor subgraph $\mathcal{G}_{d}=(\mathcal{V}_{d},\mathcal{E}_{d})$.
Specifically, each entry $\mathbf{F}_{ud}$ is proportional to the magnitude of predicted meaningful connections between node $u$ and other nodes in $\mathcal{G}_{d}$, expressed using the per-dimension attribution scores from Eq. (1):

$$ $\mathbf{F}_{ud}(\{\mathbf{h}\})=\sum_{v\in\mathcal{V}_{d}}\phi_{d}(u,v;\mathbf {h})$ (3) $$

This aggregates the contributions of dimension $d$ to the likelihood of edges involving node $u$. To achieve structure-aware disentanglement, we enforce soft orthogonality among the columns of the affiliation matrix(^1^11Note that $\mathbf{F}_{ud}(\{\mathbf{h}\})\equiv\sum_{v\in\mathcal{V}}h_{d}(u)h_{d}(v)- \frac{|\mathcal{V}|}{|\mathcal{E}|}\sum_{(u^{\prime},v^{\prime})\in\mathcal{E} }h_{d}(u^{\prime})h_{d}(v^{\prime})$ and assuming $|\mathcal{V}|<<|\mathcal{E}|$, the second term becomes negligible, allowing us to approximate $\mathbf{F}_{ud}(\{\mathbf{h}\})$ and reduce computational costs.).
This ensures that different embedding dimensions capture independent structures, leading to nearly non-overlapping sets of predicted links for each dimension.
We express the columns of the affiliation matrix as
$\mathbf{F}_{:,d}$
and obtain the disentanglement loss function as:

$$ $\mathcal{L}_{\text{dis}}=\sum_{d=1}^{K}\sum_{l=1}^{K}\left[\cos\left(\mathbf{F }_{:,d},\mathbf{F}_{:,l}\right)-\delta_{d,l}\right]$ (4) $$

where $\cos\left(\mathbf{F}_{:,d},\mathbf{F}_{:,l}\right)$ denotes the cosine similarity between the $d$-th and the $l$-th columns of $\mathbf{F}$, and
and $\delta_{d,l}$ is the Kronecker delta function ($1$ if $d=l$ , 0 0 otherwise).
This approach enables us to obtain disentangled representations [], where each dimension corresponds to a single underlying structural factor in the graph. Although it is possible to obtain disentanglement at more coarse-grained levels (e.g., with groups of dimensions), we focus on single-feature disentanglement because it inherently leads to dimension-wise interpretability.

### 3.2 Our approach: DiSeNE

Building upon the above components, introduced to satisfy our desiderata for interpretable node embeddings, we present our approach, DiSeNE.
Specifically, DiSeNE takes as input the raw node attributes $\mathbf{X}\in\mathbb{R}^{|\mathcal{V}|\times F}$ and, depending on the encoder architecture, also the adjacency matrix $\mathbf{A}\in\mathbb{R}^{|\mathcal{V}|\times|\mathcal{V}|}$. The input is encoded into an intermediate embedding layer $\mathbf{Z}\in\mathbb{R}^{|\mathcal{V}|\times D}$.
Next, DiSeNE processes the embedding matrix $\mathbf{Z}$ to compute the likelihood of link formation between node pairs, given by $\hat{y}(u,v;\mathbf{h})=\sigma(\mathbf{h}(u)^{\top}\mathbf{h}(v))$ where $\mathbf{h}(v)=\rho(\mathbf{W}^{\top}\mathbf{z}(v))$ are the final node representations in $\mathbf{H}\in\mathbb{R}^{|\mathcal{V}|\times K}$, obtained by applying a linear transformation $\mathbf{W}\in\mathbb{R}^{D\times K}$ followed by a non-linear activation function $\rho$.
To encode $\mathbf{z}$, we employ architectures incorporating fully connected layers and graph convolutional layers []. This process can be further enhanced by integrating more complex message-passing mechanisms or MLP operations. For example, the message-passing could initiate from an MLP-transformed node attribute matrix, MLP($\mathbf{X}$), or incorporate more sophisticated architectures beyond simple graph convolutions for increased expressiveness [, ].

The embeddings are optimized by combining the previously described objective functions for preserving structural faithfulness and achieving structural disentanglement, thereby improving dimensional interpretability.
To avoid degenerate disentanglement solutions–specifically, the emergence of “empty” clusters characterized by near-zero columns in $\mathbf{F}$ that, while orthogonal to others, fail to convey meaningful information–we introduce a regularization strategy. This regularization ensures a minimal but significant level of connectivity within each topological substructure.
Specifically, we enforce that the total amount of predicted edges in each anchor subgraph $\mathcal{G}_{k}$ - $\sum_{u,v\in\mathcal{V}}\phi_{k}(u,v;\mathbf{h})$ - to be non-zero. We found a more stable and precise approach by enforcing that the aggregated node features of each embedding dimension are non-zero, achieved by maximizing the entropy:

$$ $\mathcal{H}=-\sum_{d=1}^{K}\left(\frac{\sum_{u}h_{d}(u)}{||\sum_{u}\mathbf{h}( u)||_{1}}\right)\log\left(\frac{\sum_{u}h_{d}(u)}{||\sum_{u}\mathbf{h}(u)||_{1 }}\right).$ $$

Thus, the model is optimized by minimizing the following comprehensive loss function:

$$ $\mathcal{L}=\mathcal{L}_{\text{rw}}+\mathcal{L}_{\text{dis}}+\lambda_{\text{ ent}}\left(1-\frac{\mathcal{H}}{\log K}\right)$ $$

The hyperparameter $\lambda_{\text{ent}}$ determines the strength of the regularization, controlling the stability for explanation subgraph sizes across the various latent dimensions.

### 3.3 Proposed Evaluation Metrics

In the following, we introduce novel metrics to quantify multiple aspects related to interpretability and disentanglement in node embeddings, which we use to compare models in our experiments.

#### Comprehensibility.

Comprehensibility measures how closely the identified topological explanations align with ground-truth clusters, which are crucial in the evolution of numerous complex real-world systems, i.e. community modules [, ].
We evaluate comprehensibility by treating edges in explanation masks $\{\mathbf{M}^{(d)}\}_{d=1,\dots,K}$ as retrieved items from a query, and measuring their overlap with the edges in the ground-truth communities using precision, recall, and $F_{1}$-score. Let $\mathcal{C}(\mathcal{E})=\{\mathcal{C}^{(1)},\dots,\mathcal{C}^{(m)}\}$ denotes the set of truthful link communities of the input graph(^2^22 Synthetic graphs can be constructed with ground-truth relevant sub-structures (like BA-Shapes [] or SBM graphs). In real-world graphs, it is usually reasonable to assume that the community structure [] can serve as ground-truth.).
Associated to partition $\mathcal{C}^{(i)}$, we define ground-truth edge masks $\mathbf{C}^{(i)}\in\{0,1\}^{V\times V}$ with binary entries $C^{(i)}_{uv}=[(u,v)\in\mathcal{C}_{i}]$.
Comprehensibility score is given by the maximum $F_{1}$-score across ground-truth index:

$$ $\mathrm{Comp}(\mathbf{M}^{(d)})=\mathrm{max}_{i}\left\{F_{1}(\mathbf{M}^{(d)}, \mathbf{C}^{(i)})\right\}=\mathrm{max}_{i}\left\{\frac{2}{\mathrm{prec}( \mathbf{M}^{(d)},\mathbf{C}^{(i)})^{-1}+\mathrm{rec}(\mathbf{M}^{(d)},\mathbf{ C}^{(i)})^{-1}}\right\}$ (5) $$

For precision, we weigh relevant item scores with normalized embedding masks values: $\mathrm{prec}(\mathbf{M}^{(d)},\mathbf{C}^{(i)})=\frac{\sum_{u,v}M^{(d)}_{uv}C
^{(i)}_{uv}}{\sum_{u,v}M^{(d)}_{uv}}$. For recall, we weigh binarized embedding masks values with normalized ground-truth scores(^3^33For the precision, we normalize with the sum of scores because they are continuous. For recall, we use the cardinality in place of the sum because the ground-truth has binary scores.): $\mathrm{rec}(\mathbf{M}^{(d)},\mathbf{C}^{(i)})=\frac{\sum_{u,v}[M^{(d)}_{uv}>
0]C^{(i)}_{uv}}{|\mathcal{C}^{(i)}|}$.

#### Sparsity.

We refer to sparsity as the amount of disorder in the dimension’s explanations, it is generally defined as the ratio of the number of bits needed to encode an explanation compared to those required to encode the input [].
Given that concise explanations are more effective in delivering clear insights, enhancing human understanding, we evaluate sparsity by measuring the normalized Shannon entropy over the mask distribution:

$$ $\mathrm{Sp}(\mathbf{M}_{d})=-\frac{1}{\log|\mathcal{E}|}\sum_{(u,v)\in\mathcal {E}}\left(\frac{M^{(d)}_{uv}}{\sum_{u^{\prime},v^{\prime}}M^{(d)}_{u^{\prime}v ^{\prime}}}\right)\log\left(\frac{M^{(d)}_{uv}}{\sum_{u^{\prime},v^{\prime}}M^ {(d)}_{u^{\prime}v^{\prime}}}\right).$ (6) $$

A lower entropy in the mask distribution indicates higher sparsity.

#### Overlap Consistency.

In explaining latent space representations, it is essential to comprehend how input space factors influence specific latent features. A well-structured, disentangled latent space should correspond to distinct, uncorrelated topological structures. We aim to quantify how different topological components affect pairwise feature correlations in the latent space. To achieve this, we propose a metric that measures the strength of association between the physical overlap of the explanation substructures $\{\mathcal{G}_{d}\}$ and the correlation among corresponding latent dimensions $\{\mathbf{H}_{:,d}\}$.
We compute the overlap between two subgraph components using the Jaccard Similarity Index (JSI) of their edge sets from Eq. (2):
$\mathrm{JSI}(d,l;\mathbf{h})=\frac{|\mathcal{E}_{d}\cap\mathcal{E}_{l}|}{|
\mathcal{E}d\cup\mathcal{E}_{l}|}$.
The overlap consistency (OvC) metric measures the linear correlation between the pairwise JSI values and the squared Pearson correlation coefficients ($\rho^{2}$) of the embedding features:

$$ $\mathrm{OvC}(\mathbf{h})=\rho\Big{(}[\mathrm{JSI}(d,l;\mathbf{h})]_{d<l},[\rho ^{2}(\mathbf{H}_{:,d},\mathbf{H}_{:,l})]_{d<l}\Big{)}$ (7) $$

where $[*]_{d<l}$ denotes the condensed list of pair-wise similarities. By using $\rho^{2}$ we remain agnostic about the sign of the correlation among latent features, since high overlaps could originate from both cases.
This metric provides a systematic way to assess the extent to which topological partitions align with the distribution of features in the embedding space, thus offering deeper insights into the interpretability and disentanglement of the learned representations.

#### Positional Coherence.

In unsupervised node embeddings, it is crucial to assess how well the latent space captures the graph’s structure by encoding the positional relationships of nodes. An effective representation should preserve meaningful spatial properties that reflect node proximity and connectivity patterns.
To achieve this, we propose to
measure the extent to which node entries reflect their positions within the graph.
Typically, positional encoding [, , ] involves the use of several sets of node anchors $\mathcal{S}_{d}\subset\mathcal{V}$ that establish an intrinsic coordinate system.
This system influences the node $u$’s features based on the node’s proximity $\zeta(u,\mathcal{S}_{d})=\mathrm{AGG}(\{\zeta(u,v),v\in\mathcal{S}_{d}\})$, where $\mathrm{AGG}$ denotes a specific pooling operation. As node proximity, we used the inverse of the shortest path distance
$\zeta_{spd}(u,v)\equiv{(1+d_{spd}(u,v))}^{-1}$.
As the anchor sets, we chose the embedding substructures used for explanations, $\mathcal{S}_{d}\equiv\mathcal{V}_{d}$.

For a specified pair of dimensions $(d,l)$, we assess the correlation between node features along dimension $d$ and the corresponding distances to the topological component indexed by $l$ via Feature-Proximity Correlation:
$\mathrm{FPC}(d,l;\mathbf{h})=\rho\Big{(}\big{[}\zeta_{spd}(u,\mathcal{V}_{d})
\big{]}_{u\in\mathcal{V}},\mathbf{H}_{:,l}\Big{)}$.
The positional coherence metric (PoC) is defined to specifically evaluate the degree to which each feature $d$ is uniquely correlated with its corresponding topological component $\mathcal{V}_{d}$, without being significantly influenced by correlations with other substructures.
This metric is calculated as the ratio of the average FPC for the given dimensions to the average FPC computed with pairs of permuted dimensions:

$$ $\mathrm{PoC}(\mathbf{h})=\frac{\sum_{d}\mathrm{FPC}(d,d;\mathbf{h})}{\bigl{ \langle}\sum_{d}\mathrm{FPC}(d,\pi(d);\mathbf{h})\bigr{\rangle}_{\pi}}$ (8) $$

where $\langle.\rangle_{\pi}$ denotes an empirical average over multiple permutations.
By comparing with random feature-subgraph pairs, the metric
avoid promoting models with redundancies in the latent features, where high correlations with other topological components are possible.

#### Plausibility

Figure: Figure 4: Schematic view of Plausibility metric computation. A high plausibility score indicates that the dimensions deemed more comprehensible also received higher importance scores from the post-hoc feature attribution technique.
Refer to caption: x4.png

Given the importance of node representations in downstream tasks, it is crucial to assess whether the features influencing predictions align with human expectations using synthetic benchmarks. To this end, we construct instance-wise explanations to determine if key features correspond to the topological structures behind the ground-truth labels.
Typical post-hoc explainers [] that produce feature importance scores often fail in this context because node embeddings are inherently uninterpretable, leading to useless explanations.
Our approach overcomes this limitation by mapping feature importance back to the graph’s structural components, enabling a more meaningful evaluation of how well the embeddings capture the underlying factors driving node behavior.

Consider training a downstream binary classifier $b:\mathbb{R}^{K}\rightarrow[0,1]$, such as for node classification or link prediction. We detail the procedure for link prediction here (see also Figure 4), but we will also report the methodology for node classification in Appendix A.4. For an edge instance $\mathbf{h}(u,v)$ (which could be derived from node-pair operations such as $\mathbf{h}(u)\odot\mathbf{h}(v)$), we employ post-hoc methods to determine
the feature relevance for the classifier prediction on the node pair instance $(u,v)$, $\{\Psi_{j}(u,v;b)\}_{j=1,\dots,K}$ that outputs important score for each of the embedding dimensions. Similarly to scores in Eq. (1), we define task-based masks $\mathbf{B}^{(j)}\in\mathbb{R}^{V\times V}_{\geq 0}$ which aggregate the individual logics of the classifier predictions:
$B^{(j)}_{uv}(\Psi_{j};b)=\mathrm{max}\{0,\Psi_{j}(u,v;b)\}.$
To evaluate the consistency of a prediction, we consider the $F_{1}$-score in Eq. (5) related to the ground-truth structure of the edge under study, indexed by $g(u,v)$. Specifically, we define plausibility for an individual prediction as the average comprehensibility relative to the instance ground-truth, weighted by the computed feature importance:

$$ $P\ell(u,v;b)=\frac{\sum_{d=1}^{K}f(\Psi_{j}(u,v;b))F_{1}(\mathbf{B}^{(j)}, \mathbf{C}^{g(u,v)})}{\sum_{j=1}^{K}f(\Psi_{j}(u,v;b))}$ (9) $$

where $f$ is a function guaranteeing the non-negativity of relevance weights.
This ensures that only the features that are both interpretable and significant to the local prediction contribute substantially to the score, penalizing instead those important features that are not human-comprehensible.

## 4 Experiments

We conduct large-scale experiments to answer the following research questions.

- •
Human understandability: How comprehensible and sparse are the explanation substructures generated by DiSeNE?
- •
Structural disentanglement:
Do the disentangled subgraphs reveal intrinsic properties of node embeddings, like feature correlations and latent positions?
- •
Utility for downstream tasks: Are the identified substructures plausible and coherent enough to serve as explanations in downstream tasks?

To address our research questions, we extract topological components of multiple embedding methods, trained on different graph-structured data, by computing edge subsets defined in Eq (2), and analyzing embedding metrics defined in Section 3.3.
In the following sections, we describe the data, models,
the experimental setup and results from the comparison.

### 4.1 Datasets and competitors

#### Datasets.

We ran experiments
on four real-world datasets (Cora, Wiki, FaceBook, PPI), and four synthetic datasets (RingOfCliques, StochasticBlockModel, BA-Cliques and ER-Cliques).
RingOfCliques and StochasticBlockModel are implemented in the NetworkX library(^4^44[https://networkx.org/documentation/stable/reference/generators.html](https://networkx.org/documentation/stable/reference/generators.html)).
Statistics for these datasets are provided in Table A1 in the Appendix. Additionally we employ several biological datasets (see Appendix A.3) for the evaluation on multi-label node classification).
BA-Cliques is a variation of the BA-Shapes [] where we randomly attach cliques instead of house motifs to a Barabási-Albert graph.
Similarly, ER-Cliques is generated by attaching cliques to an Erdős-Rényi random graph.
All synthetic graphs contain 32 cliques with 10 nodes each as ground-truth sub-structures. For synthetic data, we present only results for plausibility metrics, leaving the other findings in the Appendix A.4.

#### Methods.

We compare different node embedding methods. Competitors include DeepWalk [], Graph Autoencoder (GraphAE) [], InfWalk [], and GraphSAGE [].
Moreover, we apply the Dine retrofitting approach [] to post-process embeddings from DeepWalk and GraphAE.
We evaluate our method DiSeNE in two variants: a 1-layer fully-connected encoder (DiSe-FCAE) and a 1-layer convolutional encoder (DiSe-GAE). GNN-based methods are trained using the identity matrix as node features. Details on hyperparameters and training settings are provided in the Appendix A.2.

#### Setup.

In experiments on real-world graphs, we investigate latent space interpretability and disentanglement metrics by keeping the output embedding dimension fixed at 128. This dimensionality was chosen to ensure that all methods achieve optimal performance in terms of test accuracy, specifically for link prediction (see the Appendix A.3 for extensive downstream task results).
For synthetic data, since we investigated plausibility metric referred to a downstream classifier, thus we did not focus on a specific dimension but we selected the best score metric varying the output dimensions in the list $[2,4,8,16,32,64,128]$.
Each reported result is an average over 5 runs. For link prediction, we use a 90%/10% train/test split, and for node classification, we use an 80%/20% split. All results refer to the training set, except for downstream task experiments, where we present results for the test set.

### 4.2 Results and Discussion

#### Are the topological substructures both comprehensible and sparse to support human understandability?

**Table 1: Comprehensibility and sparsity results for real-world datasets. Best scores are in bold, second best results are underlined.**
| Method | Comprehensibility | Sparsity |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Cora | Wiki | FB | PPI | Cora | Wiki | FB | PPI |  |
| DeepWalk | .363$\pm$.003 | .356$\pm$.002 | .602$\pm$.004 | .281$\pm$.002 | .183$\pm$.001 | .165$\pm$.002 | .130$\pm$.004 | .136$\pm$.003 |
| GraphAE | .299$\pm$.001 | .248$\pm$.002 | .481$\pm$.014 | .263$\pm$.003 | .182$\pm$.002 | .164$\pm$.004 | .154$\pm$.003 | .135$\pm$.003 |
| InfWalk | .281$\pm$.002 | .288$\pm$.001 | .658$\pm$.006 | .312$\pm$.002 | .211$\pm$.003 | .185$\pm$.002 | .318$\pm$.010 | .177$\pm$.002 |
| GraphSAGE | .358$\pm$.004 | .307$\pm$.007 | .583$\pm$.003 | .306$\pm$.005 | .189$\pm$.001 | .189$\pm$.003 | .145$\pm$.002 | .172$\pm$.001 |
| DW+Dine | .511$\pm$.051 | .496$\pm$.014 | .813$\pm$.025 | .569$\pm$.022 | .317$\pm$.036 | .266$\pm$.007 | .226$\pm$.009 | .188$\pm$.002 |
| GAE+Dine | .569$\pm$.004 | .591$\pm$.004 | .843$\pm$.005 | .484$\pm$.007 | .290$\pm$.001 | .252$\pm$.001 | .195$\pm$.002 | .198$\pm$.002 |
| DiSe-FCAE | .822$\pm$.001 | .755$\pm$.003 | .971$\pm$.001 | .484$\pm$.001 | .504$\pm$.001 | .419$\pm$.001 | .297$\pm$.002 | .282$\pm$.001 |
| DiSe-GAE | .834$\pm$.003 | .762$\pm$.004 | .967$\pm$.001 | .515$\pm$.001 | .496$\pm$.001 | .418$\pm$.001 | .304$\pm$.003 | .254$\pm$.002 |

Here we explore how well the represented topological structures can serve as global explanations for the node embeddings, quantifying the Comprehensibility in the terms of associations between model parameters and human-understandable units of the input graph, as well as the Sparsity of these associations.
In Table 1 we show compact scores as the average values $\frac{1}{K}\sum_{d=1}^{K}\mathrm{Comp}(\mathbf{M}_{d})$ and $1-\frac{1}{K}\sum_{d=1}^{K}\mathrm{Sp}(\mathbf{M}_{d})$ over all the embedding features. For sparsity we report the value subtracted from 1 to have all the scores better with higher values.

DeepWalk and InfWalk show moderate performance in Comprehensibility, excelling slightly on FB but underperforming on PPI, while GraphAE consistently lags behind, particularly on Wiki and PPI. GraphSAGE shows good comprehensibility across Cora and FB. Incorporating DINE improves results, especially for GAE+Dine, which achieves improved scores on all datasets.
The proposed models, DiSe-FCAE and DiSe-GAE, deliver the highest overall performance.
DiSe-FCAE performs well on FB, while
DiSe-GAE excels across Cora and Wiki. However, both models show sub-optimal results on PPI, suggesting potential for further improvement on this dataset.

DeepWalk and GraphAE offer moderate Sparsity, peaking on Cora, but underperform on other datasets. InfWalk excels on FB but shows moderate results elsewhere, while
GraphSAGE performs poorly in terms of sparsity across all datasets. DeepWalk and GAE significantly improve their sparsity with Dine, particularly on Cora. For the proposed models, DiSe-FCAE performs best across datasets Cora, Wiki and PPI. Meanwhile, DiSe-GAE obtains the highest value on FB.

#### Can the identified subgraphs explain the intrinsic characteristics of the node embeddings?

Here we explore how well the defined topological units represent information in the node embedding space, providing insights into how the relative and absolute positioning of topological structures influences the feature encoding within a graph. By quantifying these relationships, we can better understand the underlying patterns and structural information in the graph, potentially leading to more robust and interpretable models. In Table 2 we report Positional Coherence and Overlap Consistency for the examined embedding methods.
For the second metric, as node proximity we used the inverse of the shortest path distance with sum as pooling.

**Table 2: Overlap consistency and positional coherence results for real-world datasets. Best scores are in bold, second best result are underlined.**
| Method | Overlap Consistency | Positional Coherence |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Cora | Wiki | FB | PPI | Cora | Wiki | FB | PPI |  |
| DeepWalk | .137$\pm$.009 | .143$\pm$.006 | .115$\pm$.007 | .015$\pm$.003 | 1.078$\pm$0.025 | 0.835$\pm$0.025 | 1.119$\pm$0.015 | 1.009$\pm$0.015 |
| GraphAE | .269$\pm$.002 | .295$\pm$.004 | .273$\pm$.017 | .452$\pm$.008 | 1.023$\pm$0.006 | 1.040$\pm$0.002 | 1.001$\pm$0.013 | 1.016$\pm$0.001 |
| InfWalk | .008$\pm$.003 | .023$\pm$.002 | .021$\pm$.002 | .134$\pm$.002 | 1.004$\pm$0.011 | 0.998$\pm$0.004 | 0.938$\pm$0.053 | 0.999$\pm$0.002 |
| GraphSAGE | .211$\pm$.003 | .136$\pm$.017 | .230$\pm$.007 | .097$\pm$.040 | 1.099$\pm$0.012 | 1.103$\pm$0.010 | 1.005$\pm$0.007 | 1.018$\pm$0.002 |
| DW+Dine | .900$\pm$.012 | .804$\pm$.032 | .851$\pm$.017 | .855$\pm$.016 | 1.790$\pm$0.076 | 2.126$\pm$0.065 | 1.792$\pm$0.058 | 1.247$\pm$0.043 |
| GAE+Dine | .560$\pm$.010 | .610$\pm$.006 | .801$\pm$.016 | .646$\pm$.003 | 2.317$\pm$0.028 | 2.551$\pm$0.048 | 1.783$\pm$0.037 | 1.098$\pm$0.004 |
| DiSe-FCAE | .885$\pm$.008 | .863$\pm$.006 | .939$\pm$.030 | .548$\pm$.001 | 5.210$\pm$0.080 | 3.540$\pm$0.082 | 3.348$\pm$0.085 | 1.283$\pm$0.004 |
| DiSe-GAE | .853$\pm$.007 | .811$\pm$.008 | .889$\pm$.007 | .887$\pm$.004 | 5.300$\pm$0.193 | 4.343$\pm$0.144 | 3.388$\pm$0.054 | 1.261$\pm$0.005 |

DeepWalk and InfWalk perform poorly for Overlap Consistency, while GraphAE shows moderate scores, particularly on PPI. GraphSAGE performs slightly worse, with the best overlap consistency on FB and Cora.
DW+Dine achieves strong scores across all datasets, while GAE+Dine performs solidly but slightly lower, with its best result on FB.
The proposed models, DiSe-FCAE and DiSe-GAE, outperform all others, achieving the highest consistency across all datasets except on Cora. DiSe-FCAE excels on FB and Wiki, while DiSe-GAE achieves the best overall score on PPI.

DeepWalk, GraphAE, and GraphSAGE demonstrate moderate Positional Coherence. InfWalk consistently scores around 1.0 on all datasets, indicating stable but unremarkable coherence. Incorporating DINE leads to substantial improvements for both DeepWalk and GAE, achieving notable gains on Cora, Wiki and FB.
The proposed models, DiSe-FCAE and DiSe-GAE, far outperform other methods, with DiSe-FCAE achieving top scores on PPI, while DiSe-GAE dominates on Cora, Wiki and FB (though with higher variance): both models show consistent superiority.

#### Are the identified latent structures sufficiently meaningful to serve as explanations for downstream tasks?

Node embeddings serve as versatile feature representations suitable for downstream tasks, though they typically function as "tabular-like" feature vectors without semantic labels for each feature. This limitation restricts the use of established post-hoc analysis methods [] like LIME, SHAP, etc. Our method allows us to link topological substructures with embedding features, thereby assigning semantic labels to node vectors. Consequently, we are able to explain a downstream classifier trained with unsupervised embeddings using feature attribution. Our goal is to assess whether the task-important features align with human understanding by measuring the Plausibility.

In these experiments we consider node classification and link prediction as binary downstream tasks, training a logistic regression classifier $b(x;\boldsymbol{\beta})=\sigma(\sum_{j=1}^{K}\beta_{j}h_{j}(x)+\beta_{0})$, where $x$ is a node/link instance.
We use SHAP [] to compute the instance-wise feature attribution values $\{\Psi_{j}(x;b)\}_{j=1\dots K}$.
For node classification, we consider positive instances as the nodes inside a clique in the synthetic graph.
Accordingly, the ground-truth explanation for a node is the clique it belongs to.
For link prediction, we focus on test edges that were inside a clique before removal, where the ground-truth explanation is again the clique itself. We used $f(*)=\mathrm{max(0,*)}$ as non-negative weighting function for computing plausibility scores.

Table 3 shows average Plausibility scores for downstream tasks over test instances.
DeepWalk, GraphAE, and InfWalk perform modestly, with DeepWalk scoring the highest among these on Ring-Cl and InfWalk showing relative strength on BA-Cl. GraphSAGE significantly underperforms across all tasks, especially in ER-Cl and BA-Cl.
The addition of Dine improves both DeepWalk and GAE. DW+Dine excels with strong performance on Ring-Cl and SBM, while GAE+Dine achieves solid results, particularly on node classification tasks, such as in ER-Cliques.
Within the proposed models, DiSe-FCAE and DiSe-GAE, DiSe-FCAE achieves the highest scores on Ring-Cl, BA-Cl, while DiSe-GAE closely follows, leading in SBM and ER-Cl.

**Table 3: Plausibility results for synthetic datasets with best scores in bold, second best underlined.**
| Method | Link Prediction | Node Classification |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Ring-Cl | SBM | BA-Cl | ER-Cl | BA-Cl | ER-Cl |  |
| DeepWalk | .234$\pm$.003 | .189$\pm$.003 | .173$\pm$.002 | .153$\pm$.006 | .146$\pm$.002 | .141$\pm$.003 |
| GraphAE | .183$\pm$.003 | .160$\pm$.002 | .145$\pm$.004 | .145$\pm$.005 | .130$\pm$.002 | .135$\pm$.006 |
| InfWalk | .224$\pm$.005 | .181$\pm$.005 | .218$\pm$.007 | .212$\pm$.008 | .129$\pm$.002 | .141$\pm$.004 |
| GraphSAGE | .252$\pm$.005 | .217$\pm$.003 | .186$\pm$.006 | .178$\pm$.005 | .016$\pm$.004 | .154$\pm$.002 |
| DW+Dine | .943$\pm$.012 | .904$\pm$.002 | .744$\pm$.008 | .722$\pm$.038 | .320$\pm$.031 | .325$\pm$.007 |
| GAE+Dine | .549$\pm$.005 | .547$\pm$.014 | .418$\pm$.011 | .387$\pm$.002 | .351$\pm$.011 | .394$\pm$.002 |
| DiSe-FCAE | .973$\pm$.002 | .908$\pm$.006 | .943$\pm$.009 | .898$\pm$.012 | .820$\pm$.011 | .791$\pm$.012 |
| DiSe-GAE | .969$\pm$.002 | .910$\pm$.006 | .936$\pm$.003 | .941$\pm$.005 | .813$\pm$.003 | .797$\pm$.009 |

## 5 Conclusions

We present DiSeNE, a novel framework for generating self-explainable unsupervised node embeddings. To build our framework, we design new objective functions that ensure structural faithfulness, dimensional explainability, and structural disentanglement. Unlike traditional GNN explanation methods that typically extract a subgraph from a node’s local neighborhood, DiSeNE introduces a paradigm shift by learning node embeddings where each dimension captures an independent structural feature of the input graph. Additionally, we propose new metrics to evaluate the human interpretability of explanations, analyze the influence of spatial structures and node positions on latent features, and apply post-hoc feature attribution methods to derive task-specific instance-wise explanations.

## Acknowledgments

S. Piaggesi acknowledges support by the European Community program under the funding schemes: G.A. 834756 “XAI: Science and technology for the eXplanation of AI decision making”.

## Appendix A Appendix / supplemental material

### A.1 Dataset statistics

**Table A1: Summary statistics of graph-structured data. In empirical data, we restrict our analysis to the largest connected component of any graph.**
|  | Cora | Wiki | FB | PPI | Ring-Cl | SBM | BA-Cl | ER-Cl |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| # nodes | 2,485 | 2,357 | 4,039 | 3,480 | 320 | 320 | 640 | 640 |
| # edges | 5,069 | 11,592 | 88,234 | 53,377 | 1,619 | 1,957 | 3,138 | 4,196 |
| # clusters | 28 | 18 | 16 | 9 | 32 | 32 | 32 | 32 |
| density | 0.002 | 0.004 | 0.011 | 0.009 | 0.032 | 0.038 | 0.015 | 0.021 |
| clust. coeff. | 0.238 | 0.383 | 0.606 | 0.173 | 0.807 | 0.561 | 0.486 | 0.456 |

### A.2 Training settings

- •
For DeepWalk [], we train Node2Vec(^5^55[https://github.com/eliorc/node2vec](https://github.com/eliorc/node2vec)) algorithm for 5 epochs with the following parameters: p$=1$, q$=1$, walk_length$=20$, num_walks$=10$, window_size$=5$.
- •
For InfWalk(^6^66[https://github.com/schariya/infwalk](https://github.com/schariya/infwalk)) [], a matrix factorization-based method linked to DeepWalk and spectral graph embeddings, we set the same value window_size$=5$ used for DeepWalk.
- •
In GraphAE [], we optimize a 1-layer GCN encoder with a random-walk loss setting analogous to DeepWalk. The model is trained for 50 iterations using Adam optimizer and learning rate of 0.01.
- •
in GraphSAGE(^7^77[https://github.com/pyg-team/pytorch_geometric/blob/master/examples/graph_sage_unsup.py](https://github.com/pyg-team/pytorch_geometric/blob/master/examples/graph_sage_unsup.py)) [], we optimize a 2-layer SAGE encoder with mean aggregation and with a random-walk loss setting analogous to DeepWalk.
The model is trained for 50 iterations using Adam optimizer, learning rate of 0.01.
- •
Dine(^8^88[https://www.github.com/simonepiaggesi/dine](https://www.github.com/simonepiaggesi/dine)) [], autoencoder-based post-processing process trained for $2000$ iterations, and learning rate of $0.1$. Input embeddings are from DeepWalk and GAE methods, tuning the input embedding size in the list $[8,16,32,64,128,256,512]$.
- •
DiSe-FCAE and DiSe-GAE trained for 50 iterations using Adam optimizer and learning rate of 0.01. Random walk sampling follows the same setting as DeepWalk, GraphAE and GraphSAGE.

### A.3 Downstream Tasks

**Table A2: Summary statistics of graph biological data used for multi-label node classification.**
|  | PPI | PCG | HumLoc | EukLoc |  |
| --- | --- | --- | --- | --- | --- |
| # nodes | 3,480 | 3,177 | 2,552 | 2,969 |  |
| # edges | 53,377 | 37,314 | 15,971 | 11,130 |  |
| # labels | 121 | 15 | 14 | 22 |  |
| density | 0.009 | 0.007 | 0.005 | 0.003 |  |
| clust. coeff. | 0.173 | 0.346 | 0.132 | 0.150 |  |

We tested link prediction for the datasets reported in the main paper. For node classification, we tested PPI and other benchmark biological datasets in multi-label setting []: the PCG dataset for the protein phenotype prediction, the HumLoc, and EukLoc datasets for the human and eukaryote protein subcellular location prediction tasks, respectively. Characteristics of additional biological datasets are reported in Table A2. In Figure A1 we report AUC-PR scores for link prediction and node classification in real-world graph data. Generally, scores increase with the number of latent embedding dimensions. Tables A3 and A4 show the maximum scores for link prediction and node classification, demonstrating that our approach can consistently achieve reasonable performances.

Figure: Figure A1: Downstream tasks results on real-world datasets with varying feature dimensions size.
Refer to caption: x5.png

**Table A3: Link prediction results (AUC-PR) on real-world datasets. Best scores are in bold, while scores with a relative performance loss of no more than 2% respect to the best score are underlined.**
|  | Cora | Wiki | FB | PPI |
| --- | --- | --- | --- | --- |
| DeepWalk | .892$\pm$.005 | .927$\pm$.002 | .990$\pm$.001 | .794$\pm$.002 |
| GraphAE | .911$\pm$.003 | .950$\pm$.001 | .994$\pm$.001 | .916$\pm$.001 |
| InfWalk | .923$\pm$.003 | .936$\pm$.002 | .941$\pm$.006 | .854$\pm$.003 |
| GraphSAGE | .913$\pm$.005 | .944$\pm$.002 | .991$\pm$.001 | .892$\pm$.003 |
| DW+Dine | .896$\pm$.004 | .931$\pm$.003 | .987$\pm$.001 | .817$\pm$.004 |
| GAE+Dine | .926$\pm$.001 | .957$\pm$.003 | .992$\pm$.002 | .919$\pm$.002 |
| DiSe-FCAE | .856$\pm$.007 | .911$\pm$.004 | .977$\pm$.001 | .884$\pm$.002 |
| DiSe-GAE | .885$\pm$.002 | .947$\pm$.002 | .993$\pm$.006 | .913$\pm$.001 |

**Table A4: Node classification results (AUC-PR) on real-world datasets. Best scores are in bold, while scores with a relative performance loss of no more than 5% respect to the best score are underlined.**
|  | PPI | PCG | HumLoc | EukLoc |
| --- | --- | --- | --- | --- |
| DeepWalk | .476$\pm$.003 | .210$\pm$.001 | .314$\pm$.012 | .241$\pm$.010 |
| GraphAE | .517$\pm$.003 | .241$\pm$.001 | .336$\pm$.004 | .249$\pm$.005 |
| InfWalk | .442$\pm$.001 | .207$\pm$.002 | .287$\pm$.004 | .212$\pm$.003 |
| GraphSAGE | .506$\pm$.001 | .231$\pm$.002 | .316$\pm$.004 | .237$\pm$.011 |
| DW+Dine | .488$\pm$.002 | .217$\pm$.001 | .308$\pm$.004 | .231$\pm$.008 |
| GAE+Dine | .526$\pm$.001 | .241$\pm$.001 | .333$\pm$.006 | .234$\pm$.008 |
| DiSe-FCAE | .498$\pm$.001 | .233$\pm$.003 | .291$\pm$.006 | .230$\pm$.006 |
| DiSe-GAE | .518$\pm$.001 | .242$\pm$.004 | .315$\pm$.003 | .238$\pm$.006 |

### A.4 Results for synthetic datasets

In Figure A2 we plot results for comprehensibility and sparsity, on the top and the bottom respectively, on synthetic datasets. Generally, DiSe-FCAE outperforms DiSe-GAE and the other competitors in all the datasets. In Figure A3 we plot results for overlap consistency and positional coherence, on the top and the bottom respectively, on synthetic datasets. For the overlap metric, DiSe-FCAE and DiSe-GAE consistently outperform the competitors, especially with more than 8 dimensions where they achieve almost perfect overlap. For the positional metric, the competitors GAE+Dine and DW+Dine slightly outperform DiSe methods, especially in large dimensions, while DeepWalk also show good results.

Figure: Figure A2: Comprehensibility and sparsity results on synthetic datasets with varying feature dimensions size.
Refer to caption: x7.png

Figure: Figure A3: Overlap consistency and positional coherence results on synthetic datasets with varying feature dimensions size.
Refer to caption: x9.png

In Figure A4 we plot results for the plausibility metric on link prediction and node classification, on the top and the bottom respectively.
Plausibility seems to benefit larger dimension values for DiSe methods and DW+Dine for link prediction. Figure A5 shows the corresponding downstream task accuracy results.

Figure: Figure A4: Plausibility results on synthetic datasets with varying feature dimensions size.
Refer to caption: x11.png

Figure: Figure A5: Downstream tasks results on synthetic datasets with varying feature dimensions size.
Refer to caption: x13.png

Local explanations for node embeddings are extracted by using post-hoc feature importance method SHAP. For a given embedding model $\mathbf{h}:\mathcal{V}\leftarrow\mathbb{R}^{K}$ we train a downstream classifier, e.g., in node classification task or link prediction. For simplicity, here we write the case when the classifier is a (binary) linear model, but it can be any arbitrary complex model. It is anyway reasonable to assume that node embeddings come from a deep graph model and downstream classifier is a simple 1-layer neural network on top of the embedding layers.

$$ $\text{{(node classification)}}~{}~{}~{}~{}~{}b(v)=\sigma(\sum_{j=1}^{K}\beta_{ j}h_{j}(v)+\beta_{0})$ $$

$$ $\text{{(link classification)}}~{}~{}~{}~{}~{}b(u,v)=\sigma(\sum_{j=1}^{K}\beta _{j}h_{j}(u,v)+\beta_{0})$ $$

Given an instance (can be a node embedding $\mathbf{h}(v)$ or a edge embedding $\mathbf{h}(u,v)$) we compute feature importance values with SHAP $\{\Psi^{(\mathcal{V})}_{j}\}_{j=1...K}$ or $\{\Psi^{(\mathcal{E})}_{j}\}_{j=1...K}$ and the corresponding task-based graph masks:

$$ $\mathbf{B}^{(j)}(\mathcal{V})\in\mathbb{R}_{\geq 0}^{V};~{}~{}~{}B^{(j)}(v;b)= \mathrm{max}\{0,\Psi^{(\mathcal{V})}_{j}(v;b)\}$ (10) $$

$$ $\mathbf{B}^{(j)}(\mathcal{E})\in\mathbb{R}_{\geq 0}^{E};~{}~{}~{}B^{(j)}(u,v;b )=\mathrm{max}\{0,\Psi^{(\mathcal{E})}_{j}(u,v;b)\}$ (11) $$

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. How does DISENE compare to graph interpretability methods like GNNExplainer in terms of explainability? A direct comparison with methods like GNNExplainer or PGExplainer would be helpful, not just in terms of performance but also in the quality of explanations provided. Can DISENE provide interpretable explanations at node or edge level similar to these methods?

2. How does DISENE scale to larger graphs? Could the authors provide more details about the computational complexity of the method? Specifically, how does the disentanglement objective affect training time and memory usage, and how does the method scale to larger graphs?

3. How do the proposed metrics relate to human interpretability? Since these metrics are novel, it would be useful to understand how well they align with human judgments of interpretability. While the proposed metrics help quantify disentanglement, it would be helpful to provide visualizations or examples where specific embedding dimensions correlate with interpretable graph structures (e.g., communities, topics). This would help demonstrate the practical interpretability of the method.
soundness: 3
strengths: The paper presents a well-structured approach with clearly defined disentanglement objectives and proposed evaluation metrics. The experiments cover multiple datasets and tasks, providing a comprehensive evaluation of the proposed method.
summary: This paper introduces **DISENE (Disentangled and Self-Explainable Node Embedding)**, a novel framework designed to generate self-explainable and disentangled node embeddings in graph-based learning tasks. The method leverages disentangled representation learning to ensure that each dimension of the learned embeddings corresponds to a unique topological substructure in the input graph, aiming to improve the interpretability of learned representations. The authors propose new evaluation metrics, such as overlap consistency and dimensional sparsity, to quantify the degree of disentanglement and interpretability of the embeddings.
weaknesses: 1. While the paper claims to generate "self-explainable" embeddings, the interpretability largely relies on proposed metrics (e.g., overlap consistency) without providing direct, human-interpretable explanations for individual embedding dimensions. The method lacks intuitive explanations for individual nodes or substructures, limiting the practical utility of the "self-explainable" claim, particularly for non-technical users or domain experts who might expect more direct insights from the embeddings.

2. The paper lacks comprehensive comparisons with other interpretability-focused methods in graph learning, such as **GNNExplainer** or **PGExplainer**, which provide more fine-grained explanations at node and edge levels. A detailed comparison with these methods in terms of interpretability (not just performance) would strengthen the paper's claims. Additionally, comparisons with simpler feature-based interpretability approaches would help highlight the unique advantages of the proposed method.

3. **Evaluation metrics may be subjective**: While novel, the new evaluation metrics are somewhat subjective and not widely used in the graph learning community. Calibrating these metrics against more established interpretability evaluation standards or providing more detailed justification for why these metrics are appropriate would be helpful. For instance, how does overlap consistency relate to human ability to interpret the learned embeddings? More human-centric evaluations, such as user studies or expert assessments, could help validate the practical interpretability of the embeddings.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: See the weakness above
soundness: 2
strengths: - The authors propose novel and interesting perspectives to consider node embedding interprebility. New metrics are also introduced to measure these perspectives.

- Experiment results seem to be promising.
summary: In this paper, the authors propose a method for interpreting node embeddings. Their method is based on several desiderata in terms of graph structures. In addition, they also propose several new criteria to evaluate the explainability. Experimental results seem to validate their method on these criteria.
weaknesses: - The proposed method for node interprebility is based on graph structures but semantic information is omitted, as verified by experiments that in GNN methods, node embeddings are initialized by identity matrix. However, in real-world applications this may not be true. For example, in social networks, if each node represents a user, it does not make sense that the user information is not considered in initial features. Whether or not the proposed method can be applied to semantic-rich graph scenarios are not discussed in the paper.

- The three key desiderata is not well explained. In corresponding paragraphs, the authors directly derive how they achieve these goals without explaining them. The definition of Equation 1 does not make sense to me as well. Say when we have $\phi_{d_1} (u_1, v_1; h) > 0$, it only means the the dimension $d_1$ is more important for embeddings of $u_1$ and $v_1$ (to compute the edge likelihood as in the paper, but what is the physical meaning of edge likelihood by the way?) than $d_1$ to other node embeddings with edges. Why could we use it to assign edges to dimensions? The dimension is shared across all node embeddings but the edge is not.

- While the authors propose several metrics for evaluation, I am confused why they do not compare their methods with previous metrics and representative node classification explanation methods such as GNN Explainer. If there is a discrepancy, please specify.

Ying, Zhitao, et al. "Gnnexplainer: Generating explanations for graph neural networks." Advances in neural information processing systems 32 (2019).

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: See Weaknesses
soundness: 3
strengths: 1. Innovative Self-Explanatory Node Embedding Method: This paper introduces the DISENE model, which leverages disentanglement and self-explanatory design to enable each embedding dimension to correlate directly with specific structural features within the graph. 
2. Disentangled Feature Representation: DISENE ensures that each embedding dimension independently captures a unique graph structure feature, minimizing overlap across dimensions. This disentangled representation improves both the interpretability and robustness of the embeddings, providing more granular structural information for downstream applications that require feature independence.
summary: The paper introduces DISENE, a novel approach to self-explanatory node embedding, addressing the increasing need for interpretability in graph representation learning. DISENE’s design emphasizes disentangled representations, allowing each embedding dimension to independently capture specific, non-overlapping structural features of the graph. This self-explanatory capability is achieved through a combination of disentangled representation learning and the embedding of interpretability directly within the model architecture, moving beyond traditional post-hoc interpretability approaches.
weaknesses: 1. Reliance on Interpretability Metrics with Ambiguous Definitions: The paper relies heavily on various interpretability metrics, such as consistency and sparsity, to evaluate model performance. However, these metrics lack clear, systematic definitions in the text. For instance, Section 4.2: Evaluation Metrics introduces these terms conceptually but fails to provide precise mathematical definitions or specific calculation methodologies. This lack of clarity may lead to inconsistencies in metric implementation across studies, thereby impacting the reproducibility of the results and the fairness of comparisons. For interpretability metrics to serve as robust evaluation tools, a more rigorous definition and operationalization are essential.
2. Ambiguity in Interpretability Metric Definitions: While the paper presents novel interpretability metrics, it leaves substantial gaps in detailing how these metrics are quantified. Section 3.4: Explanation and Evaluation discusses the interpretability aspects of the model but lacks the necessary formalism and computation process, making it challenging to assess the reliability and generalizability of these interpretability metrics. This ambiguity undermines the objective measurement of the model’s self-explanatory capabilities, calling into question the effectiveness of the proposed interpretability criteria.
3. Lack of Competitive Edge in Link Prediction and Node Classification Tasks: DISENE exhibits only moderate performance in standard tasks like link prediction and node classification, without showcasing significant improvements. As reported in Table 3: Link Prediction Results and Table 4: Node Classification Results, DISENE’s accuracy and AUC scores do not exhibit a marked advantage over other established models. This highlights a trade-off between interpretability and predictive power, suggesting that while DISENE makes strides in interpretability, it may fall short in delivering competitive performance in traditional graph embedding tasks, limiting its overall efficacy in practical applications.
4. Limited Dataset Scope, Focused on Small-Scale Citation Networks: The paper’s empirical analysis relies heavily on established citation networks such as Cora, CiteSeer, and PubMed (Section 4.1: Datasets). While these datasets are standard in graph embedding research, they are relatively small and structurally simple, which raises concerns about the model’s generalizability to larger, more complex graph structures. The limited dataset scope restricts the evaluation of DISENE’s performance on diverse, real-world graphs and challenges the model’s claims of broad applicability.
5. Outdated Baseline Comparisons: The baseline models selected for comparison—primarily traditional graph embedding methods such as DeepWalk, node2vec, and GraphSAGE (Section 4.3: Baselines)—do not reflect the recent advancements in graph representation learning. Given the rapid evolution of graph neural networks, the lack of comparisons with modern methods, including GAT, Graph Transformers, and Graph Autoencoders, as well as other interpretable models, undermines the robustness of the empirical validation. Comparisons with more recent and sophisticated models would provide a stronger benchmark, allowing for a more accurate assessment of DISENE’s strengths and weaknesses in both interpretability and performance.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: See the weaknesses.
soundness: 2
strengths: 1. The paper formalizes new and essential criteria for achieving disentangled and explainable node representations
2. The paper introduces novel evaluation metrics to help quantifying the goodness of node representation learning
summary: This paper introduces a framework that generates self-explainable embeddings in an unsupervised manner. The method employs disentangled representation learning to produce dimension-wise interpretable embeddings, where each dimension is aligned with distinct topological structure of the graph. The paper drives new objective functions and optimizes simultaneously for both interpretability and disentanglement. Additionally, the paper proposes new metrics to evaluate representation quality and human interpretability.
weaknesses: 1. The paper introduces comprehensibility as a metric for evaluating explanations. However, how does the method perform on widely-used explanation metrics like AUC, Fidelity-, and Fidelity+? I strongly recommend that the authors also consider including these popular metrics to thoroughly assess the method's performance.

2. The paper proposes using comprehensibility and sparsity to enhance human interpretability. This raises a natural question: do these metrics truly support human understandability? I strongly recommend that the authors conduct human evaluations to validate that these metrics align with human comprehension.

3. The baselines are too weak. The authors compare with the node embedding methods: DeepWalk, Deep AE, DISE-GAE, DISE-FACE. I suggest the authors consider more modern methods to have the node embedding, such as normalizing flows [R1], diffusion models [R2], self-supervised methods [R3].

[R1] RicciNet: Deep Clustering via A Riemannian Generative Model, WWW 2024

[R2] Directional diffusion models for graph representation learning, NeurIPS 2024

[R3] GraphMAE: Self-Supervised Masked Graph Autoencoders, KDD 2022.

4. The method is self-explainable, but how effective is the visualization of its explanations? I suggest that the authors include a case study to show the visualization.

5. The paper claims utility for downstream tasks like link prediction and node classification. However, the baselines used for evaluating these tasks are relatively weak. I recommend that the authors consider incorporating more widely-used contrastive learning methods as baselines, such as [R4], to assess whether the proposed node embeddings achieve superior performance in these downstream tasks.

[R4]. Provable Training for Graph Contrastive Learning, NeurIPS 2024.

Meta-review: This paper proposes a new framework named DISENE (Disentangled and Self-Explainable Node Embedding) for generating self-explainable and disentangled node embeddings in graph-based learning tasks. Reviewers agreed that this paper brings an interesting perspective by considering node embedding interoperability and also presents new evaluation metics. However, reviewers raised many concerns regarding the newly proposed metrics, the selection of baselines, limited datasets, etc. Overall, the current version of this work is not ready for publication at ICLR.

Decision: Reject