Title: Diffusion Representation for Asymmetric Kernels via Magnetic Transform
Venue: NeurIPS 2023
Accepted: yes
Keywords: Asymmetric kernels

diffusion maps

magnetic transform

dimension reduction
Abstract: As a nonlinear dimension reduction technique, the diffusion map (DM) has been widely used. 
In DM, kernels play an important role for capturing the nonlinear relationship of data. However, only symmetric kernels can be used now, which prevents the use of DM in directed graphs, trophic networks, and other real-world scenarios where the intrinsic and extrinsic geometries in data are asymmetric. A promising technique is the magnetic transform which  converts an asymmetric matrix to a Hermitian one. However, we are facing essential problems, including how diffusion distance could be preserved and how divergence could be avoided during diffusion process. Via theoretical proof, we successfully establish a diffusion representation framework with the magnetic transform, named MagDM. The effectiveness and robustness for dealing data endowed with asymmetric proximity are demonstrated on three synthetic datasets and two trophic networks.

Diffusion Representation for Asymmetric Kernels via Magnetic Transform

Abstract

As a nonlinear dimension reduction technique, the diffusion map (DM) has been widely used. In DM, kernels play an important role for capturing the nonlinear relationship of data. However, only symmetric kernels can be used now, which prevents the use of DM in directed graphs, trophic networks, and other real-world scenarios where the intrinsic and extrinsic geometries in data are asymmetric. A promising technique is the magnetic transform which converts an asymmetric matrix to a Hermitian one. However, we are facing essential problems, including how diffusion distance could be preserved and how divergence could be avoided during diffusion process. Via theoretical proof, we successfully establish a diffusion representation framework with the magnetic transform, named MagDM . The effectiveness and robustness for dealing data endowed with asymmetric proximity are demonstrated on three synthetic datasets and two trophic networks.

1 Introduction

The diffusion map (DM) [1, 2, 3] and its variations [4, 5, 6, 7] are typical dimension reduction (DR) techniques that has been widely used in the signal processing [8], computer vision [9], and textual network embedding [10]. DM is a powerful method that embeds high-dimensional data into a low-dimensional manifold while preserving diffusion distance, which helps describe meaningful geometric structures in data.

Kernels play a significant role in DM and strongly impact the quality of DR, as they allow for the identification of nonlinear relationships and underlying structures in data. In real-world scenarios such as directed graphs, trophic and social networks, the intrinsic and extrinsic geometries in data may be asymmetric, and asymmetric kernels [11, 12, 13, 14] are more suitable to depict asymmetric geometric structures. The advantages of integrating diffusion geometry and asymmetric kernels have been acknowledged by several researchers. [15] developed a framework to study function extension and approximation problems in the context of changing data and functions on directed graphs.

However, the diffusion distance and diffusion maps for vertices on directed graphs were unclear. [16] proposed the diffusion map for asymmetric kernels using the singular value decomposition, while the diffusion distance was calculated using only left singular vectors and right singular vectors were neglected, resulting in the underutilization of asymmetric information.

The magnetic transform (MT) [17] is a promising technique that transforms an asymmetric matrix into a Hermitian one, making it suitable for working with asymmetry. MT has been successfully applied in DR techniques based on asymmetric adjacency matrices on directed graphs, such as Laplacian eigenmaps [18, 19]. Nevertheless, connecting DM and MT still faces essential problems. Previous works have analyzed MT based on finite adjacency matrices of directed graphs, while the studies of applying MT on asymmetric kernel functions, the corresponding integral operators, and diffusion geometry have not been explored, which obstructs the application of MT to DM. Additionally, MT transforms real-valued asymmetric kernels that describe the connectivity between data into complex-valued function spaces, raising the question of how to define the diffusion distance and diffusion maps in those complex-valued function spaces. It is also crucial to investigate how to prevent divergence during the diffusion process when we are working in complex field. To address these challenges, we propose a novel diffusion representation framework with the magnetic transform, named MagDM , that enables the use of asymmetric kernels.

The contributions of this paper are as follows:

We develop a diffusion representation framework for data endowed with asymmetric kernels using the magnetic transform, where the corresponding diffusion distance and diffusion maps are defined.

We explore integral operators of MT, whose compactness is established if the asymmetric kernels are Hilbert-Schmidt kernels, helping us determine the range of usable asymmetric kernels.

We prove an important property that the spectral radius of the proposed integral operator is 1, which ensures that MagDM will not diverge during the diffusion process.

We present experiments that demonstrate the effectiveness and robustness of the MagDM algorithm on three synthetic datasets and two real-world trophic networks.

2 Related works

Dimension reduction techniques [20, 21, 22] have been extensively studied over the past few decades. Previous research has focused mainly on symmetric and positive semi-definite proximity. However, in recent years, researchers have become aware of benefits of asymmetric proximity and have proposed several insightful theories and algorithms. These methods fall into one of the following families.

Singular value decomposition. [15] presented a polar decomposition for the construction of kernels to embed directed graphs into manifolds to study harmonic analysis of functions on directed graphs. However, the diffusion representation for asymmetric kernels under this framework was not discussed. [16] utilized the singular value decomposition (SVD) for asymmetric kernels, and defined the diffusion distance for asymmetric kernels. There was a theoretical shortcoming in which diffusion distance was calculated using only left singular vectors and right singular vectors were neglected, meaning that only out-going proximity was considered, which resulted in the underutilization of asymmetric information. Although SVD embeds asymmetric kernels as an inner product of two feature mappings, it does not directly reflect the directional information, i.e., the skew-symmetric part, which is more important in asymmetric scenarios, in the feature mappings.

Magnetic Laplacian. The magnetic Laplacian [17] was originally derived from a quantum mechanical Hamiltonian of a particle under magnetic flux. [18, 19, 23] utilized magnetic Laplacian to embed asymmetric proximity as a complex-valued feature mapping, where symmetric and skew-symmetric parts were successfully separated into modules and phases, respectively. Magnetic Laplacian transformed an asymmetric matrix to a normalized Hermitian matrix H whose eigenvalues are contained within the interval [ -1 , 1] [24]. In Laplacian-typed methods, the feature spaces turned to be the (numerical) null spaces of I -H [25], in which only the positive part of the spectrum of H was used. However, the negative part of the spectrum also contains meaningful information which was ignored in previous works [18, 19], resulting in an inadequate use of asymmetric information.

3 Diffusion representation and diffusion distance

In this section, we briefly review the diffusion map. Assume ( X,µ ) is a σ -finite measure space where X ⊂ R M is a dataset and µ is an associated measure which is non-degenerate on X . Let K : X × X → R + be a non-negative symmetric kernel which measures the local connectivity between two samples x , y ∈ X . And the Markov kernel is defined as follows,

where ν ( x ) is the volume form defined as ν ( x ) = ∫ X K ( x , y )d µ ( y ) . Assuming that the volume form does not vanish and K ∈ L 2 ( X × X,µ ⊗ µ ) is a square integrable measurable function, then the operator A : L 2 ( X,µ ) → L 2 ( X,µ ) given by A ( f )( x ) = ∫ X ρ ( x , y ) f ( y )d µ ( y ) is compact and self-adjoint. We can write ρ ( x , y ) = ∑ n ∈ N λ n ϕ n ( x ) ϕ n ( y ) according to the spectral theorem, where { λ n , ϕ n } is the spectral decomposition of the operator A . For any positive natural number t ∈ N + , the diffusion distance at time t between two samples x , y can be defined as follows,

where ρ t is the kernel of the t powers of the integral operator A . Note that the diffusion distance D t is an average over all paths in time t connecting x to y , resulting in a diffusion distance that is robust to noise perturbation. The diffusion distance can be reformulated as follows,

The expression above enables dimension reduction in diffusion geometry, which is also known as diffusion maps. The dataset with the kernel structure ρ can be embedded into a lower dimensional space ψ t k : X → R k as follows,

4 Diffusion representation for asymmetric kernels

The definition (2) of the diffusion distance assumes symmetry, making it unsuitable for use with asymmetric kernels. To overcome this limitation, we introduce the magnetic transform in this section. With this transformation, we propose the MagDM algorithm for asymmetric kernels.

4.1 Magnetic transform

Assume that we have a set of data X ⊂ R M , and a non-negative distribution µ defined on this set. Let K : X × X → R + be a non-negative kernel that measures the directed connectivity between samples in X . Note that K does not need to be symmetric. The classical diffusion distance (2) fails to measure asymmetric distance from x to y because it only uses half of asymmetric information, treating both samples as sources. To handle asymmetry, the magnetic transform is defined to transform an asymmetric kernel to a new kernel with conjugate symmetry.

Suppose an asymmetric kernel K : X × X → R + is a Hilbert-Schmidt kernel, K ∈ L 2 ( X × X,µ ⊗ µ ) . The Magnetic transform kernel H ( q ) of K is defined by

where q ∈ R + is a scaling parameter and i = √ -1 . The kernel H ( q ) is conjugate symmetric, meaning that H ( q ) ( x , y ) = H ( q ) ( y , x ) . The modulus and phase terms of H ( q ) capture the symmetric and skew-symmetric parts of information in K , respectively.

Proposition 1. Suppose a real-valued asymmetric kernel K : X × X → R + is a Hilbert-Schmidt kernel K ∈ L 2 ( X × X,µ ⊗ µ ) where ( X,µ ) is a σ -finite measure space and X ⊂ R M . Then, the magnetic transform kernel H ( q ) : X × X → C is a Hilbert-Schmidt kernel.

The proof is shown in Appendix A. The aforementioned proposition allows us to define a HilberSchmidt kernel H ( q ) , which enables us to identify the range of usable asymmetric kernels. Then, we can define the corresponding Hilber-Schmidt operator that is also a compact operator [26] in the diffusion process.

4.2 Diffusion maps and diffusion distance

The Markov kernel ρ ( x , y , q ) ∈ L 2 ( X × X,µ ⊗ µ ) for the magnetic transform kernel H ( q ) is defined by

where m ( x ) = 1 2 ∫ X ( K ( x , y ) + K ( y , x ) ) d µ ( y ) is the volume form of the symmetric term of K . Assuming that m ( x ) does not vanish on X , the operator T ( q ) : L 2 ( X,µ ) → L 2 ( X,µ ) given by

is compact and self-adjoint. We can decompose ρ ( x , y , q ) = ∑ n ∈ N λ ( q ) n ϕ ( q ) n ( x ) ϕ ( q ) n ( y ) according to the spectral theorem, where { λ ( q ) n , ϕ ( q ) n } is the spectral decomposition of the operator T ( q ) . Next, we analyze the spectrum of the integral operator (5).

Proposition 2. Given an asymmetric kernel satisfying Proposition 1, the eigenvalues { λ ( q ) i } of the Hilbert-Schmidt operator defined by (5) are real and bounded by λ ( q ) i ∈ [ -1 , 1] .

The proof is provided in Appendix B. Because the operator is self-adjoint, its eigenvalues are real numbers and eigenfunctions are complex functions. According Proposition 2, as diffusion time t ∈ N increases, small eigenvalues converge to zero, which avoids divergence of MagDM during diffusion process. For any time t , MagDM defines the diffusion distance for the asymmetric kernel by

Running along the diffusion process for asymmetric kernels is equivalent to computing the powers of the operator T ( q ) . The diffusion distance (6) is a functional weighted L 2 distance between complexvalued proximities of two samples, thus it is also robust to noise perturbation. Both symmetry and skew-symmetry are embedded into the diffusion distance, which reflects the asymmetric geometry in data. (6) can be rewritten as the following,

Denote the eigenvalues in descending order as: 1 ≥ | λ ( q ) 1 | ≥ | λ ( q ) 2 | ≥ · · · ≥ 0 . D t ( x , y ) can be approximated by D t δ ( x , y ) with a preset accuracy δ ∈ (0 , 1) and finite terms as follows,

where s ( δ, t ) = max { n ∈ N + : | λ ( q ) n | t > δ | λ ( q ) 1 | t } . We define the diffusion map for asymmetric kernels which embeds data endowed with asymmetric kernels in a lower dimensional space as follows,

Proposition 3. The diffusion map ψ t, ( q ) embeds the data endowed with an asymmetric kernel K into a lower dimensional space C s ( δ,t ) , then the Euclidean distance in this space is equal to the diffusion distance D t δ ( x , y ) ,

When the fed kernel function is symmetric, the proposed method reverts back to the classical diffusion map. This is because exp ( i 2 πq · ( K ( x , y ) -K ( y , x ) )) becomes exp ( i 2 πq · 0 ) = 1 , and H ( q ) ( x , y ) = K ( x , y ) . The implementation details of this algorithm can be found in Appendix D.

4.3 Selection of the scaling parameter q.

The scaling parameter q is a key parameter for embedding the skew-symmetric part of an asymmetric kernel, which directly affects the representation ability. The selection of q has been previously discussed for unweighted directed graphs [18, 23] and in the context of graph signal processing [27]. However, the selection of q for asymmetric kernel functions has not been addressed so far. Here, we propose a simple and practical method for selecting q for asymmetric kernels. In (3), the skew-symmetry is embedded in the phase, which is a periodic function controlled by q . To select a suitable value for q , we recommend selecting a period that covers the extent of the skew-symmetric component, which means that a should be less than π where a := sup x , y ∈ X |K ( x , y ) -K ( y , x ) | , then we have q < 0 . 5 /a . In practice, we have access to the Gram matrix K ∈ R N × N where N is the size of the data. Then a is given by a = max N i,j =1 | K ij -K ji | , and the selection of q is

5 Experiments

In this section, we demonstrate the capability of the MagDM method to extract asymmetric information on three synthetic and two real-world trophic datasets. Additionally, we compare our method with five other methods: diffusion maps (DM) [1], kernel principal component analysis (KPCA) [28], asymmetric diffusion maps (ADM) [16], magnetic eigenmaps (ME) [18], and Markov normalized magnetic eigenmaps (MME) [19]. The experiments were conducted using MATLAB on a PC with an Intel i7-10700K CPU (3.8GHz) and 32GB of memory. Codes are available at https://github.com/AlexHe123/MagDM

5.1 Artificial networks

We begin by assessing the ability to extract asymmetric information. In this experiment, we create an artificial network following [29]. The network consists of three groups (A, B, and C), and we establish two types of flows (forward and backward) between them, which helps us construct an adjacency matrix to represent connections among nodes. Each node has a 50% chance of being connected to a node in its own group. Furthermore, each node has a 50% chance of being connected to a node in another group in the direction of the forward flow (i.e., interconnections from A to B, from B to C, and from C to A). Additionally, there is another type of interconnection in the direction of the backward flow with a probability of P . By adjusting the backward probability P , we can control the running flow. An illustration of this artificial network is shown in Appendix F.1.

In this experiment, we choose 5 probabilities for the backward running flow, P ∈ { 0 , 0 . 2 , 0 . 5 , 0 . 8 , 1 } . As P increases, the level of asymmetric information first decreases and then increases. Due to the fact that phases of eigenvectors contain most of asymmetric information [18, 19], Fig. 1 reports phases of the ME, MME and MagDM methods, and shows their capability of extracting asymmetric information. All the three methods distinguish three groups when the probability of backward flow P = 0 . As we increase the value of P , the level of asymmetric information decreases. This is reflected in the fact that the distance between samples in the same group becomes larger, as seen in the second row of Fig. 1. When P = 0 . 5 , the contribution of the forward and backward flows to adjacency connections is statistically the same, resulting in the disappearance of asymmetric information and the mixing of samples from different groups. As P continues to increase, asymmetric information gradually strengthens again, forming a scenario where the backward flow is stronger than the forward flow. The last two rows of Figure 1 demonstrate that the MagDM algorithm effectively distinguishes between the three groups, whereas the ME and MME algorithms are currently unable to differentiate between them, which illustrates the stronger robustness of our method.

The running flow of the second artificial network comprises four groups (A, B, C, and D) following [30]. The visualization of this directed graph can be seen in Appendix F.2. The structure of the flow is apparent, with Groups A and D serving as out-come and in-come flows, while Groups B and C function as intermediaries. For instance, Group A predominantly generates out-coming links with a high probability (strong flow), and only a few in-coming links from Group D are produced by weak flow with a low probability. Besides, each node has a high probability of being connected to a node in the same group.

-

/2

0

/2

ME phase

1

(q)

-

/2

0

/2

ME phase

2

(q)

-

-

/2

0

/2

MME phase

1

(q)

-

/2

0

/2

MME phase

2

(q)

-

-

/2

0

/2

MagDM phase

1

(q)

-

/2

0

/2

MagDM phase

2

(q)

-

/2

0

/2

ME phase

1

(q)

-

/2

0

/2

ME phase

2

(q)

-

-

/2

0

/2

MME phase

1

(q)

-

/2

0

/2

MME phase

2

(q)

-

-

/2

0

/2

MagDM phase

1

(q)

-

/2

0

MagDM phase

2

(q)

0

ME phase

1

(q)

-

/2

0

/2

ME phase

2

(q)

-0.06

-0.04

-0.02

0

MME phase

1

(q)

-

/2

0

/2

MME phase

2

(q)

-

-

/2

0

/2

MagDM phase

1

(q)

-

/2

0

/2

MagDM phase

2

(q)

-0.5

0

0.5

ME phase

1

(q)

-

/2

0

/2

ME phase

2

(q)

-

-

/2

0

/2

MME phase

1

(q)

-

/2

0

/2

MME phase

2

(q)

-

-

/2

0

/2

MagDM phase

1

(q)

-

/2

0

/2

MagDM phase

2

(q)

-0.6

-0.4

-0.2

0

ME phase

1

(q)

-

/2

0

/2

ME phase

2

(q)

-0.01

-0.005

0

0.005

0.01

MME phase

1

(q)

-

/2

0

/2

MME phase

2

(q)

-

-

/2

0

/2

MagDM phase

1

(q)

-

/2

0

/2

MagDM phase

2

(q)

P=0

P=0.2

P=0.5

P=0.8

P=1

5.2 Synthetic data using an asymmetric kernel

Next, we assume that the dataset is a set of 300 points distributed along the Möbius strip. The parametric form of the Möbius strip is defined by

-0.14

-0.13

-0.12

-0.11

DM

1

-0.05

0

0.05

DM

2

-6

-4

-2

0

ADM

1

-2

0

2

ADM

2

-2

-1

0

1

2

KPCA

1

-2

0

2

KPCA

2

-

-

/2

0

/2

ME phase

1

(q)

-

/2

0

/2

ME phase

2

(q)

-0.2

-0.1

0

0.1

ME Re

1

(q)

-0.1

0

0.1

ME Im

1

(q)

-0.2

-0.1

0

0.1

0.2

ME Re

2

(q)

-0.2

-0.1

0

ME Im

2

(q)

-0.2 -0.15 -0.1 -0.05

0

MME phase

1

(q)

-

/2

0

/2

MME phase

2

(q)

0.1

0.12

0.14

0.16

MME Re

1

(q)

-0.03

-0.02

-0.01

0

MME Im

1

(q)

-0.1

0

0.1

0.2

MME Re

2

(q)

-0.1

0

0.1

0.2

MME Im

2

(q)

-

-

/2

0

/2

MagDM phase

1

(q)

0

/2

MagDM phase

2

(q)

-0.1

-0.05

0

0.05

0.1

MagDM Re

1

(q)

-0.1

0

0.1

MagDM Im

1

(q)

-0.05

0

0.05

0.1

MagDM Re

2

(q)

0

0.05

0.1

MagDM Im

2

(q)

where 0 ≤ u ≤ 2 π and -0 . 5 ≤ v ≤ 0 . 5 . The dataset is with a color drift in the counterclockwise direction on the x-y plane, please see it in Appendix F.3. We define an asymmetric kernel as follows,

where ∠ xy is the angle between x , y on the x-y plane. The asymmetric kernel is composed of two truncated ℓ 1 kernels [31], which measures asymmetric similarity based on angles between x and y .

-0.06

-0.058

-0.056

DM

1

-0.02

-0.01

0

0.01

0.02

DM

2

-60

-40

-20

0

ADM

1

-20

-10

0

10

20

ADM

2

-10

-5

0

5

KPCA

1

-10

-5

0

5

10

KPCA

2

-

-

/2

0

/2

ME phase

1

(q)

-

-

/2

0

/2

ME phase

2

(q)

-10

-5

0

MME phase

1

(q)

10 -3

-

-

/2

0

/2

MME phase

2

(q)

-

/4

0

/4

/2

3

/4

MagDM phase

1

(q)

0

/2

MagDM phase

2

(q)

The dimension reduction results are illustrated in Fig. 3. DM and KPCA fail to distinguish the asymmetric geometry, as they only learn the symmetric part of (9). Although the ADM method contains the color drift after reducing the dimension of data, there is still a sharp turn on the green part of the data, which is not desired. The ME method distinguishes the color drift only on the phase of the first eigenvector, while the MME method fails to provide clear directed information. Notably, the MagDM method provides a clear and smooth result along both two phase axes.

5.3 Trophic networks

0

0.2

0.4

DM

1

-0.2

0

0.2

DM

2

-10

-5

0

ADM

1

0

10

20

ADM

2

0

2

4

6

KPCA

1

-5

0

5

KPCA

2

-

/2

0

ME phase

1

(q)

-

-

/2

0

/2

ME phase

2

(q)

-0.2

0

0.2

ME Re

1

(q)

-0.2

0

0.2

ME Im

1

(q)

-0.2

0

0.2

ME Re

2

(q)

-0.2

-0.1

0

ME Im

2

(q)

-0.2

-0.1

0

MME phase

1

(q)

-

/2

0

/2

MME phase

2

(q)

0

0.2

0.4

MME Re

1

(q)

-0.02

0

0.02

MME Im

1

(q)

-0.5

0

0.5

MME Re

2

(q)

-0.02

0

0.02

0.04

MME Im

2

(q)

-

-

/2

0

/2

MagDM phase

1

(q)

-

-

/2

0

/2

MagDM phase

2

(q)

-0.3

-0.2

-0.1

0

0.1

MagDM Re

1

(q)

-0.2

0

0.2

MagDM Im

1

(q)

-0.2

0

0.2

MagDM Re

2

(q)

-0.2

0

0.2

MagDM Im

2

(q)

To further illustrate the concept, we have chosen two specific real-world trophic networks from the Pajek datasets 1 : the Mondego network [32], which records trophic exchanges at the Mondego estuary, and the Florida network [33], which records trophic exchanges in Florida Bay during the wet season.

Based on the roles of the nodes in ecosystems, we classify them into different categories, as depicted in Appendix F.4. The different categories in the trophic networks consist of Producers, which generate their own food through photosynthesis or chemosynthesis (green nodes); Consumers, consisting of both low-level consumers (brown nodes) and high-level consumers (red nodes) that feed on other organisms for energy; and Organic Matters (blue and turquoise blue nodes). As for the Florida network, there is also a Decomposer that break down dead or decaying organic matter (purple node). The adjacency matrix W denotes the intensity of trophic exchanges and the asymmetric kernel matrix

for the trophic networks are defined by,

The dimension reduction results for two trophic networks are presented in Figs. 4 and 5. For the Florida network, MagDM outperforms other methods, as shown in Fig. 4. All DM, ADM, and KPCA methods fail to differentiate between different categories. On the phases results of the remaining three methods, ME and MME are unable to extract clear asymmetric information, while MagDM almost distinguishes between producers and consumers. For better illustration, the real and imaginary parts of the first two non-trivial eigenvectors of ME, MME, and MagDM are visually compared. It is observed that all three methods fail to distinguish categories on the first eigenvector axis, but on the second eigenvector axis, MagDM not only differentiates the categories but also tightly clusters high-level consumers together, resulting in a clear classification structure. Similar results are obtained for the Mondego network. These findings demonstrate that MagDM is a promising approach for dealing with asymmetric proximity in trophic networks.

0

0.2

0.4

DM

1

-0.2

0

0.2

0.4

DM

2

-15

-10

-5

0

ADM

1

-20

-10

0

ADM

2

0

2

4

6

8

KPCA

1

-4

-2

0

KPCA

2

-

/2

0

/2

ME phase

1

(q)

-

/2

0

/2

ME phase

2

(q)

0

0.2

0.4

ME Re

1

(q)

-0.2

0

0.2

ME Im

1

(q)

-0.5

0

0.5

ME Re

2

(q)

-0.3

-0.2

-0.1

0

ME Im

2

(q)

2.8

3

3.2

MME phase

1

(q)

-

/2

0

/2

MME phase

2

(q)

-0.6

-0.4

-0.2

0

MME Re

1

(q)

0

0.02

0.04

MME Im

1

(q)

0

0.2

0.4

MME Re

2

(q)

-0.1

-0.05

0

MME Im

2

(q)

-

-

/2

0

/2

MagDM phase

1

(q)

-

/2

0

/2

MagDM phase

2

(q)

-0.4

-0.2

0

MagDM Re

1

(q)

-0.2

0

0.2

MagDM Im

1

(q)

-0.4

-0.2

0

0.2

MagDM Re

2

(q)

-0.2

0

0.2

MagDM Im

2

(q)

6 Conclusion

In this paper, we have presented a novel framework named MagDM for representing data endowed with asymmetric kernels using the magnetic transform within the context of diffusion maps. We introduced the integral operator with the proposed magnetic transform kernel and investigated several

of its properties, including compactness, spectral decomposition, and spectral radius. Moreover, we proposed a diffusion distance that embeds asymmetric kernels into the diffusion map in complexvalued function spaces, allowing the identification and analysis of asymmetric geometries and patterns in data. This could lead to new insights into the structure and properties of asymmetric kernels, potentially leading to new applications or algorithmic improvements. Experiments demonstrated the effectiveness and robustness of MagDM on three synthetic datasets and two trophic networks.

There are several interesting avenues for future work in the study of asymmetric kernels. In terms of methodology, exploring variants of other dimension reduction schemes for asymmetric kernels such as MDS [34] and Isomap [35] would be appealing. In terms of theory, studying geometric harmonics of functions with magnetic transform kernels could provide a deeper understanding of the intrinsic and extrinsic geometries in data. In terms of applications, using MagDM as feature mappings for classification and regression tasks could be particularly useful for problems involving asymmetric proximity, such as social networks and commodity trading between economies.

Broader impacts. The proposed MagDM method described in this paper is not expected to have significantly different impacts on society compared to other dimension reduction techniques.

Acknowledgments

This work was partially supported by National Natural Science Foundation of China (62376155), Research Program of Shanghai Municipal Science and Technology Committee (22511105600), and Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102).

References

Ronald R Coifman and Stéphane Lafon. Diffusion maps. Applied and computational harmonic analysis , 21(1):5-30, 2006.

Ronald R Coifman and Matthew J Hirn. Diffusion maps for changing data. Applied and computational harmonic analysis , 36(1):79-107, 2014.

Stéphane S Lafon. Diffusion maps and geometric harmonics . Yale University, 2004.

Amit Singer and H-T Wu. Vector diffusion maps and the connection laplacian. Communications on pure and applied mathematics , 65(8):1067-1144, 2012.

Yifeng Fan and Zhizhen Zhao. Multi-frequency vector diffusion maps. In International Conference on Machine Learning , pages 1843-1852. PMLR, 2019.

Gabriele Cesa, Arash Behboodi, Taco S Cohen, and Max Welling. On the symmetries of the synchronization problem in cryo-em: Multi-frequency vector diffusion maps on the projective plane. In Advances in Neural Information Processing Systems , volume 35, pages 5446-5458, 2022.

Harbir Antil, Tyrus Berry, and John Harlim. Fractional diffusion maps. Applied and Computational Harmonic Analysis , 54:145-175, 2021.

Ronen Talmon, Israel Cohen, Sharon Gannot, and Ronald R Coifman. Diffusion maps for signal processing: A deeper look at manifold-learning techniques based on kernels and graphs. IEEE signal processing magazine , 30(4):75-86, 2013.

Deepti Pachauri, Risi Kondor, Gautam Sargur, and Vikas Singh. Permutation diffusion maps (pdm) with application to the image association problem in computer vision. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 1 , page 541-549. MIT Press, 2014.

Xinyuan Zhang, Yitong Li, Dinghan Shen, and Lawrence Carin. Diffusion maps for textual network embedding. In Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages 7598-7608, 2018.

Mark Mackenzie and A Kiet Tieu. Asymmetric kernel regression. IEEE Transactions on Neural Networks , 15(2):276-282, 2004.

Silvia L Pintea, Jan C van Gemert, and Arnold WM Smeulders. Asymmetric kernel in Gaussian processes for learning target variance. Pattern Recognition Letters , 108:70-77, 2018.

Baihe Huang, Xiaoxiao Li, Zhao Song, and Xin Yang. FL-NTK: A neural tangent kernel-based framework for federated learning analysis. In International Conference on Machine Learning , pages 4423-4434. PMLR, 2021.

Mingzhen He, Fan He, Lei Shi, Xiaolin Huang, and Johan A.K. Suykens. Learning with asymmetric kernels: Least squares and feature interpretation. IEEE Transactions on Pattern Analysis and Machine Intelligence , pages 1-12, 2023.

Hrushikesh N Mhaskar. A unified framework for harmonic analysis of functions on directed graphs and changing data. Applied and Computational Harmonic Analysis , 44(3):611-644, 2018.

Alvaro Almeida Gomez, Antônio J Silva Neto, and Jorge P Zubelli. Diffusion representation for asymmetric kernels. Applied Numerical Mathematics , 166:208-226, 2021.

Elliott H. Lieb and Michael Loss. Fluxes, laplacians, and kasteleyn's theorem. Duke Mathematical Journal , 71(2):337-363, 1993.

Michaël Fanuel, Carlos M Alaíz, Ángela Fernández, and Johan A. K. Suykens. Magnetic eigenmaps for the visualization of directed networks. Applied and Computational Harmonic Analysis , 44(1):189-199, 2018.

Alexander Cloninger. A note on markov normalized magnetic eigenmaps. Applied and Computational Harmonic Analysis , 43(2):370-380, 2017.

Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics , 2(4):433-459, 2010.

Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally linear embedding. science , 290(5500):2323-2326, 2000.

Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems , volume 14. MIT Press, 2001.

Michaël Fanuel, Carlos M Alaiz, and Johan A. K. Suykens. Magnetic eigenmaps for community detection in directed networks. Physical Review E , 95(2):022302, 2017.

XZhang, Y He, N Brugnone, M Perlmutter, and M Hirn. Magnet: A neural network for directed graphs. Advances in Neural Information Processing Systems , 34:27003-27015, 2021.

Jianzhong Wang. Mathematical analysis on out-of-sample extensions. International Journal of Wavelets, Multiresolution and Information Processing , 16(05):1850042, 2018.

John B Conway. A course in functional analysis , volume 96. Springer, 2019.

Satoshi Furutani, Toshiki Shibahara, Mitsuaki Akiyama, Kunio Hato, and Masaki Aida. Graph signal processing for directed graphs based on the hermitian laplacian. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16-20, 2019, Proceedings, Part I , pages 447-463. Springer, 2020.

Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Kernel principal component analysis. In Artificial Neural Networks-ICANN'97: 7th International Conference Lausanne, Switzerland, October 8-10, 1997 Proceeedings , pages 583-588. Springer, 2005.

Andrea Lancichinetti and Santo Fortunato. Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities. Physical Review E , 80(1):016118, 2009.

Elizabeth A Leicht and Mark EJ Newman. Community structure in directed networks. Physical review letters , 100(11):118703, 2008.

Xiaolin Huang, Johan A. K. Suykens, Shuning Wang, Joachim Hornegger, and Andreas Maier. Classification with truncated ℓ 1 distance kernel. IEEE transactions on neural networks and learning systems , 29(5):2025-2030, 2017.

R Pinto, J Patrício, F Salas, JM Neto, and JC Marques. Mondego catchment goods and services assessment under the millennium ecosystem assessment scope. Felicita Scapini, Jean-Marc Boffa, Louis F. Cassar , page 279, 2009.

Robert E Ulanowicz and Donald L DeAngelis. Network analysis of trophic dynamics in south florida ecosystems. US Geological Survey Program on the South Florida Ecosystem , page 114, 1999.

J Douglas Carroll and Phipps Arabie. Multidimensional scaling. Measurement, judgment and decision making , pages 179-250, 1998.

Mukund Balasubramanian and Eric L Schwartz. The isomap algorithm and topological stability. Science , 295(5552):7-7, 2002.

Kösaku Yosida. Functional analysis . Springer Science & Business Media, 2012.

Jihun Hamm and Daniel D Lee. Grassmann discriminant analysis: a unifying view on subspacebased learning. In Proceedings of the 25th international conference on Machine learning , pages 376-383, 2008.

A The proof of Proposition 1

Proof. Because K is a Hilbert-Schmidt kernel, we have ∫∫ X × X ∣ ∣ K ( x , y ) ∣ ∣ 2 d µ ( x )d µ ( y ) < ∞ . Then,

It holds that the symbol ' ≤ ′′ in the third row of the above equation because of the Cauchy-Schwarz inequality. The Cauchy-Schwarz inequality states that ∀ a i , b i ∈ R , i = 1 , · · · , k , then ( ∑ k i =1 a i b i ) 2 ≤ ( ∑ k i =1 a 2 i )( ∑ k i =1 b 2 i ) . Then, we have ∣ ∣ K ( x , y ) 2 + K ( y , x ) 2 ∣ ∣ 2 ≤ ( 1 4 + 1 4 ) · (∣ ∣ K ( x , y ) ∣ ∣ 2 + ∣ ∣ K ( y , x ) ∣ ∣ 2 ) , and measure µ is non-negative. Thus, symbol ' ≤ ′′ in the third row holds. We finish the proof.

B The proof of Proposition 2

Proof. The associated kernel µ ( x , y , q ) of the operator (5) is a Hermitian kernel, i.e., µ ( x , y , q ) = µ ( y , x , q ) , then the eigenvalues of (5) are real and the spectral radius is equal to the operator norm R ( ρ ) = ∥ T ( q ) ∥ [36]. Let f ∈ L 2 ( X,µ ) , we have

If we apply the Cauchy-Schwarz inequality as follows,

where S ( x , y ) = ∣ ∣ H ( q ) ( x , y ) ∣ ∣ = 1 2 ∣ ∣ K ( x , y ) + K ( y , x ) ∣ ∣ . The asymmetric kernel K is non-negative, thus S ( x , y ) = 1 2 ( K ( x , y ) + K ( y , x ) ) . Then we have

Consequently,

We can apply the Cauchy-Schwarz inequality again,

It can be noticed that the operator norm ∥ T ( q ) ∥ = 1 . Therefore, the spectral radius R ( ρ ) = 1 . We finish the proof.

C Selection of the scaling parameter q.

As illustrated in Fig. Ape.1, we suggest choosing a period that encompasses the range of the skew-symmetric component, i.e., a should be less than π , where a is defined as follows,

and the period of the phase function is T = 2 π 2 πq = 1 /q . Thus, we have

Im

2a

日

=

2πqaji

0

Re

-1-

0

Ⅱ

D Algorithm

To apply MagDM, we require a dataset and an asymmetric kernel or an asymmetric Gram matrix, as well as the scaling parameter q and the desired accuracy. Algorithm 1 outlines the MagDM procedure.

Algorithm 1 MagDM for asymmetric kernels

Input: The Gram matrix K of dataset X endowed with an asymmetric kernel K , the scaling parameter q and a preset accuracy δ .

Output: The diffusion map ψ t, ( q ) of X .

1: Calculate the Hermitian Gram matrix H of the asymmetric Gram matrix K by (3) and (4).

2: Calculate the t -powers kernel matrix H t .

3: Run eigen-decomposition of H t and denote its eigen-system as { λ ( q ) n , ϕ ( q ) n } .

4: s ( δ, t ) ← max { n ∈ N : | λ ( q ) n | > δ | λ ( q ) 1 |} .

5: Return the diffusion map ψ t, ( q ) by (8).

Limitations. Researchers should note that the MagDM method proposed in this paper has some limitations. One such limitation is its dependence on the choice of asymmetric kernel functions, which can impact its performance. Additionally, MagDM may be computationally expensive for large datasets, as it requires O ( N 2 ) memory and O ( N 3 ) computational complexity to derive the spectral decomposition. However, this limitation can be addressed through the use of out-of-sample extensions, which are discussed in Appendix E.

E Out-of-sample extensions.

̸

Out-of-sample extensions are useful in many applications where low-dimensional embeddings computed on the original dataset are extended to new data. The Nyström extension is a well-known technique used in the machine learning community to approximate the Gram matrix by a low-rank embedding. However, the out-of-sample extension of duffision maps for asymmetric kernels has not been studied before. Here, we present the corresponding Nyström-based extension for out-of-sample cases. As discussed earlier, the integral operator (5) is compact and self-adjoint, whose spectral decomposition is { λ ( q ) n , ϕ ( q ) n } . If λ ( q ) n = 0 , the following identity holds for x ∈ X :

The Nyström extension extends the equation above to new data Z such that X ⊆ Z as follows,

where z ∈ Z and ϕ ( q ) n ( z ) = ∑ y ∈ X ρ ( z , y ,q ) Nλ ( q ) n ϕ ( q ) n ( y ) is the empirical form of (a1) for X . This allows the eigenfunctions to be extended for new data, enabling the extension of MagDM (8) as follows,

F Descriptions and figures of datasets

In this section, we visualize the datasets using either expert knowledge or a force-directed layout. We hope that these visualizations will help readers gain a better understanding of the data.

F.1 The first artificial network

Fig. Ape.2(a) provides the running flow of the first artificial network and Fig. Ape.2(b) shows an example of the directed graphs generated with P = 0 , where the asymmetric adjacency connection can be considered as an asymmetric kernel.

(a) The running flow of three groups.

(b) Graph using the expert knowledge positions.

Forwardflow

Backward flow

B

F.2 The second artificial network

The running flow of the second artificial network comprises four groups (A, B, C, and D). The structure of the flow is apparent, with groups A and D serving as out-come and in-come nodes, respectively, while groups B and C function as communicators. Groups B and C are two dense sets containing 20 nodes and Group A and D are a pair of datasets whose interconnections are much more than Group B and C.

(a) The running flow of four groups.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

(b) Graph using a force-directed layout.

Strong flow

Weakflow

D

F.3 The Möbius strip

The Möbius strip dataset is a set of 300 points randomly distributed along the Möbius strip. The parametric form of the Möbius strip is defined by,

where 0 ≤ u ≤ 2 π and -0 . 5 ≤ v ≤ 0 . 5 . The dataset is with a color drift in the counterclockwise direction on the x-y plane in Fig. Ape.4.

0.3

0.2

0.1

8:8880888

N

0

888

（)

-0.1

8888888

?

-0.2

8

-0.3

1

0

1

0.5

0

-1

-0.5

y

-1

X

F.4 Two trophic networks

We have chosen two specific trophic networks: the Mondego [32] and Florida [33] networks, which are part of the Pajek datasets. These networks have recorded the trophic exchanges at Mondego estuary and Florida bay during the wet season, respectively. Based on the roles of the nodes in these ecosystems, we have classified them into different categories, as shown in Appendix Ape.5. The green nodes, such as 2um Spherical Phytoplankt and Phytoplankton, are producers that generate their own food through photosynthesis or chemosynthesis. The brown and red nodes are low-level consumers like littorina and high-level consumers like bonefish and crocodiles that feed on other organisms for energy. Additionally, the purple node in the Florida network represents decomposers that break down dead or decaying organic matter. Finally, the blue and turquoise blue nodes correspond to the input, output, and organic matter of the ecosystem, respectively.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

(a) Mondego.

(b) Florida.

G Quantitative experiments

We have incorporated quantitative experiments to measure the performance of the proposed method (MagDM) in comparison to two other methods (ME and MME). In the first network, we set the forward flow probability to 0.5 and the backward flow probability to P=0/0.2/1. We then cluster the low-dimensional embeddings of the three methods using the k-means algorithm with k=3. To evaluate performance for the clustering results, we consider two internal evaluation metrics: the silhouette coefficient (SC) and the Davies-Bouldin index (DB), as well as two external evaluation metrics: the adjusted Rand index (ARI) and the normalized mutual information (NMI). These metrics assess the quality of clustering.

The results of the quantitative experiments evaluating the performance of the proposed MagDM are presented in Tab. Ape.1 of the attached PDF file. It can be seen that MagDM achieves higher scores in the metrics. Particularly when P=1, where interconnections are more complicated, MagDM significantly outperforms other methods, which highlights the superior effectiveness and robustness of MagDM compared to other methods.

H Experiments regarding the importance of the parameter t

We have conducted additional experiments on MagDM, focusing on diffusion time t ∈ { 1 , 2 , 3 , 4 , 5 , 10 } . The results are presented in Fig. Ape.6, which can be found in the attached PDF file under the General Response section.

In Fig. Ape.6, we illustrate the clusters that evolve from different values of t ( t ∈ { 1 , 2 , 3 , 4 , 5 , 10 } ). At t = 1 , the local geometry becomes apparent with four clusters visible on the real and imaginary axes. Due to the presence of asymmetric connections, the diffusion distance between samples within the same group is small, while the diffusion distance between samples in different groups is large. Consequently, even with reduced dimensions, we can clearly distinguish the four clusters in the diffusion space. As the diffusion process progresses, the diffusion distance between different groups decreases. At t = 4 , the groups start to connect with each other, forming a single structure. Finally, at t = 10 , the four groups merge to create a single super-cluster structure, with very small diffusion distances between points. Interestingly, the clustering on the phase axes remains clear and preserved throughout the diffusion time. This observation demonstrates the strong ability of MagDM to capture asymmetric information during the diffusion process. The dynamic behavior we observe underscores the effectiveness of representing and visualizing the data.

I Discussion of the diffusion framework

The MagDM framework, proposed for asymmetric scenarios, employs a mathematical technique to transform a pair of real-valued asymmetric similarities into complex-valued and conjugate symmetric similarities. By leveraging the concept of random walks on graphs, the framework computes the diffusion distance D t ( x, y ) . This distance is a functional weighted L 2 distance between complexvalued proximities, enabling a more flexible and adaptive measure of similarity. D t ( x, y ) will be

0.04

T=1

0

0.01

0

-0.02

-T

-π/2

0

π/2

T

-0.02

0

0.02

-0.02

0

0.02

1

0.04

π/2

0.02

T=2

0

0

O

-0.02

-π

-π/2

0

T/2

-0.02

0

0.02

-0.02

0

0.02

(q)

0.04

0.03

T/2

0.02

0.02

T=3

0.01

0

0

O

-0.02E

-πT/2

0

π/2

T

-0.02

0

0.02

-0.02

0

0.02

MagDMphase

(9)

0.04

0.03

T=4

T/2

0.02

0.02

0

8

0.01

0

OE

-0.02E

-T

-T/2

0

π/2

-0.02

0

0.02

-0.02

0

0.02

0.04

0.03

T=5

T/2

0.02

0.02

0

0.01

OE

-0.02

0

-T/2

0

T/2

-0.02

0

0.02

-0.02

0

0.02

MagDMRe(q9)

#

0.04

MagDMRe

T=10

T/2

0.02

0.01-

0

O正

-0.02E

0

-T/2

0

T/2

π

-0.02

0

0.02

-0.02

0

0.02

MagDMphase

9)

MagDMRe

small if two samples are similar in the complex-valued plane and vice versa. Consequently, it becomes capable of capturing both local and global patterns within the data.

In this framework, the diffusion kernel, denoted as ρ ( x, y ) , represents the transition probabilities of the random walks. It signifies the likelihood of transitioning from one data point to another within the network. In summary, the proposed diffusion distance within the MagDM framework facilitates a geometric interpretation of asymmetric similarity. It encompasses the network structure and takes into account the asymmetric relationships between data points.

J The comparison with [4]

The MagDM is not a simple derivation of diffusion map for U (1) -connection graphs because there are three key differences between MagDM and the graph connection Laplacian (GCL) approach.

Definition of the similarity matrices. MagDM focuses on asymmetric similarity between samples and calculates the U (1) transporter for all pairs of samples. On the other hand, GCL focuses on symmetric cases and calculates the orthogonal O ( d ) transporter for graph edges. While there is an isomorphism between U (1) and SO (2) , they differ in implementation. MagDM measures scalar

similarity between two samples, denoted as x and y, while the similarity between x and y in GCL is a d ∗ d block. The corresponding Gram matrix in MagDM, denoted by ρ , has a size of N ∗ N , while the corresponding Gram matrix in GCL, denoted by S , is Nd ∗ Nd .

Definition of diffusion distance. In MagDM, the diffusion distance is defined as [ D t ( x, y )] 2 = ∫ X ( ρ t ( x, u ) -ρ t ( y, u )) 2 µ ( du ) . It measures the number of paths of length t connecting x and y in the complex-valued plane. The diffusion distance is calculated by the sum of squares of the difference between ρ t ( x, · ) and ρ t ( y, · ) . In GCL, diffusion distance has a different form: [ D t ( x, y )] 2 = Tr ( S t ( x, y ) S t ( x, y ) ⊤ ) , which measures the agreement between transporters and the number of paths of length t connecting x and y. Based on these two definitions, it can be observed that U (1) -connection graphs exhibit different diffusion distances in MagDM and GCL.

Definition of the diffusion map. Due to the differences in diffusion distances, the diffusion maps of the two methods also differ. In MagDM, eigenvalues and eigenvectors are defined as { λ i } N i =1 , { ϕ i } N i =1 , respectively. The diffusion map is defined as Φ t ( x ) = ( λ t 1 ϕ 1 ( x ) , λ t 2 ϕ 2 ( x ) , · · · , λ t N ϕ N ( x )) . For vector diffusion map, its eigenvalues and eigenvectors are defined as { η i } Nd i =1 , { ψ i } Nd i =1 , respectively. Its diffusion map is defined by Φ t ( x ) = (( η i η j ) t < ψ i ( x ) , ψ ( x ) > ) Nd i,j =1 . These two diffusion maps have different forms from each other. Overall, MagDM is not a derivation to a notion of diffusion map for U (1) -connection graphs.

K Eigenvalues and robustness

Indeed, H and I -H can be transferred to each other when considering both positive and negative eigenvalues. However, it should be noted that in [18, 19], the negative eigenvalues are truncated, which means that H and I -H are not identical in this regard. We focus on the k eigenvalues that include both positive and negative eigenvalues with the largest absolute magnitudes. These top k eigenvalues correspond to the slowest diffusion processes or the global structures in the data. It is also believed that the larger magnitude of eigenvalues corresponds to better discriminative information, as discussed in [37]. One notable advantage of this approach is its robustness when confronted with perturbed asymmetric similarity. Even if the similarity measure is distorted, we can still obtain the principal components using this method. However, there is no specific reference to a related study because most previous work has mainly focused on positive semi-definite matrices. When dealing with asymmetry, it becomes necessary to consider negative eigenvalues. This aspect is worth exploring in future research. Currently, we can observe this phenomenon experimentally in Fig. 1 of the manuscript. In the case where P=0, it represents a situation where there are only links in one direction, making it an asymmetric but relatively easy scenario. When P=1/0.8, the situation becomes more complicated and backward flow is perturbed by the forward, MagDM effectively distinguishes between the three groups while ME/MME struggles to capture the information, highlighting stronger robustness and effectiveness of MagDM.

In the experiments involving asymmetric similarity, it is indeed possible for the largest eigenvalues to have different signs. Here, we present the counts of eigenvalues for ϕ 1 and ϕ 2 that exhibit both positive signs (++) and opposite signs (+-) among 100 trials for the network generated by the forward and backward flows (Experiment 1). The forward flow probability is fixed at 0.5, while the backward flow probability P is selected from the following set: {0, 0.3, 0.4, 0.5, 0.6, 0.65, 0.675, 0.7, 0.8, 1}.

In Tab. Ape.2, it can be observed that as P increases, the number of opposite signs also increases. Particularly, when P is larger than 0.675, opposite signs occur more frequently. We have discovered that the principal components align with both positive and negative eigenvalues, highlighting the superior performance of MagDM. These results further substantiate the robustness of MagDM.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the concept of the magnetic transform to define diffusion representation and diffusion distance for asymmetric kernels. The key idea is to transform an asymmetric kernel into a Hermitian one, enabling the application of standard techniques such as eigen-decomposition.\n\nBy leveraging this magnetic transform approach, the authors conduct experimental validations to assess the effectiveness of their proposed method. The results demonstrate that their method outperforms other dimension reduction methods in terms of separating clusters in asymmetric data.\n\nSTRENGTHS:\n- The presentation of the paper is clear and the paper is easy to follow.\n- A proper discussion on the selection of the scaling parameter in included which is helpful for people to implement the method.\n- The experimental results support the effectiveness of the proposed method.\n\nWEAKNESSES:\nA major concern regarding the paper relates to its novelty and theoretical justification. Several points raise doubts regarding the originality and uniqueness of the proposed approach:\n  - The Magnetic transform, as presented in the paper, may not be considered entirely novel. Previous works, specifically [18] and [19], have already studied similar forms of the Magnetic transform, particularly when the kernel represents the adjacency matrix of a directed graph. This raises questions about the extent to which the proposed approach differs from previous research. While the paper focuses on a different matrix form ($H$ instead of $D-H$ or $I-H$), more theoretical justification is needed to demonstrate the significance and appropriateness of this choice.\n  - Besides, I would like to bring the authors' attention to the following paper: ```VECTOR DIFFUSION MAPS AND THE CONNECTION LAPLACIAN``` by Singer and Wu. It is well known that magnetic laplacian is the same as the connection laplacian when considering $SO(2)$ signatures. In this reference, Singer and Wu have already considered diffusion maps and diffusion distances for connection graphs which could well be a generalization already for what the authors are proposing. I think a comparison with this reference is needed.\n  \nAdditionally, there is a lack of clarity in the experiments regarding the importance of the parameter $t$ and the role of diffusion in the proposed method. The paper does not explicitly demonstrate how the choice of $t$ influences the results or why diffusion is significant. In fact, the current experiments only show that the eigenvectors of the normalized kernel is useful. This ambiguity hampers a thorough understanding of the method and its underlying principles.\n\nQUESTIONS:\n- line 86: why do you choose $t$ to be an integer instead of a real number?\n- What is the kernel involved in the experiments in Section 5.1?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-03"}

comments: SUMMARY:
This paper studies the asymmetric kernel case for the diffusion map. The authors utilize the magnetic transform technique to develop a diffusion representation framework for the asymmetric kernel case. They investigate several properties of the proposed magnetic transform kernel. The challenge lies in defining the corresponding integral operators and diffusion geometry. In their experiments, the authors show their proposed MagDM framework competitive and even superior to existing dimension reduction techniques like DM, KPCA, etc.

STRENGTHS:
This work combine the magnetic transform and the diffusion map techniques to handle the asymmetric kernel case of the diffusion map. It seems to be a pioneer research work by defining new concepts.

WEAKNESSES:
All the experiments present only qualitative results and some quantitative results would help figure out the advantages of the proposed MagDM framework over other existing techniques.

QUESTIONS:
I am not familiar with the topic and feel quite confused about Proposition 1: why do we want to assume a Hilbert-Schmidt kernel $X$ and define another Hilbert-Schmidt kernel $H^{(q)}$? The goal seems to define a kernel with conjugate symmetry.

LIMITATIONS:
The authors have mentioned some limitations of the proposed MagDM framework in appendix. For example, different choices of asymmetric kernel functions would impact the performance of the framework. Currently, there is no clue about how to handle it.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes proposes a new method called MagDM in which it connects Diffusion maps (DM) and the Magnetic transform (MT). DM is a nonlinear dimension reduction technique obtains a lower dimensional embedding by using the information of the diffusion distances that assume symmetry. However, in practice the intrinsic and extrinsic geometries of data can be asymmetric. MT is a promising technique that converts an asymmetric matrix to a Hermitian one, make it suitable for working with asymmetry, but the connection between DM and MT haven’t been explored much. Hence, the main contribution of the paper is to make such connections between DM and MT. Specifically, they developed a diffusion framework endowed with asymmetric kernels using MT. Also, the integral operators of MT, whose compactness is established if the asymmetric kernels are Hilbert-Schmidt kernels have been explored. They’ve also proven an important property that the spectral radius of the proposed integral operator is 1, which ensures that MagDM will not diverge during the diffusion process.\n\nSTRENGTHS:\nThe ideas presented here are original. The paper is generally well-written. There are numerous qualitative experiments. Since asymmetric data is common, the proposed approach could have a high significance and impact and could be useful in other methods that use the diffusion framework.\n\nWEAKNESSES:\nThe paper is missing quantitative experiments. While the qualitative experiments in the paper do suggest that MagDM is capturing the asymmetric geometry better than other approaches, the scatter plots are difficult to interpret without expert knowledge of the underlying data. It would be better to have some numerical evaluation of this. I recommend the authors come up with some metric that measures this performance for comparing the different methods.\n\nOther comments\nThe diffusion framework could be clarified further. For example, what is the geometric interpretation of the diffusion distances and how do they relate to $\\rho(x,y)$?\n\nQUESTIONS:\n1. Can the authors use quantitative measures of how well the methods perform?\n\n2. In Figure 1, it is shown that MagDM has better results compared to ME & MME when P has large values (greater than 0.5). On the other hand, when P is less than 0.5, specifically when P = 0, P = 0.2, all three methods are doing well to separate the groups. So if P = 0 and P = 0.2 are also considered to have high level of asymmetry information, how does MagDM outperform others? \n\n3. In Figure 3, from the perspective of dimension reduction, I would like to compare the result to the original structure . Since the original structure is connected for a M¨obius strip, I would expect the lower dimensional embedding to be connected in a loop as well. However you’ve mentioned in line 210 that you focus on the asymmetric geometry of the data but I am not sure how does the ”asymmetric structure” look like in the original space colored by the rainbow map. So I think you should add the additional subplots to your figures to show us how the original data looks like.\n\nLIMITATIONS:\nAddressed", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}