Extending Power of Nature from Binary to Real-Valued Graph Learning in Real World
Authors: Chunshu Wu
Ruibing Song
Chuan Liu
Yunan Yang
Ang Li
Michael Huang
Tong Geng
Venue: ICLR 2024 poster (2024)
Keywords: graph learning
nature-powered computing
dynamic physical system
Primary area: learning on graphs and other geometries & topologies
TL;DR: We upgrade a binary Ising Machine and its associated model to support real values in solving real-world problems, achieving orders of magnitude of speedup and energy efficiency in Graph Learning compared to baseline GNNs

Abstract: Nature performs complex computations constantly at clearly lower cost and higher performance than digital computers. It is crucial to understand how to harness the unique computational power of nature in Machine Learning (ML). In the past decade, besides the development of Neural Networks (NNs), the community has also relentlessly explored nature-powered ML paradigms. Although most of them are still predominantly theoretical, a new practical paradigm enabled by the recent advent of CMOS-compatible room-temperature nature-based computers has emerged. By harnessing a dynamical system's intrinsic behavior of chasing the lowest energy state, this paradigm can solve some simple binary problems delivering considerable speedup and energy savings compared with NNs, while maintaining comparable accuracy. Regrettably, its values to the real world are highly constrained by its binary nature. A clear pathway to its extension to real-valued problems remains elusive. This paper aims to unleash this pathway by proposing a novel end-to-end Nature-Powered Graph Learning (NP-GL) framework. Specifically, through a three-dimensional co-design, NP-GL can leverage the spontaneous energy decrease in nature to efficiently solve real-valued graph learning problems. Experimental results across 4 real-world applications with 6 datasets demonstrate that NP-GL delivers, on average, $6.97\times 10^3$ speedup and $10^5$ energy consumption reduction with comparable or even higher accuracy than Graph Neural Networks (GNNs).

Published as a conference paper at ICLR 2024
EXTENDING POWER OF NATURE FROM BINARY TO REAL- VALUED GRAPH LEARNING IN REAL WORLD
Chunshu Wu', Ruibing Song', Chuan Liu', Yunan Yang", Ang Li Michael Huang', Tong Geng'
Department of Electrical and Computer Engineering, University of Rochester'
Department of Mathematics, Comell University?. Physical & Computational Sciences Directorate, Pacific Northwestern National Laboratory
ABSTRACT
Nature performs complex computations constantly at clearly lower cost and higher performance than digital computers. It is crucial to understand how to harness the unique computational power of nature in Machine Learning (ML). In the past decade, besides the development of Neural Networks (NNs), the community has also relentlessly explored nature-powered ML paradigms. Although most of them are still predominantly theoretical, a new practical paradigm enabled by the re- cent advent of CMOS-compatible room-tempert temperatue nature-based computers has emerged. By harnessing a dynamical system's 's intrinsic behavior of chasing the lowest energy state, this paradigm can solve some simple binary problems deliv- ering considerable speedup and energy savings compared with NNs, while main- taining comparable accuracy. Regrettably, its values to the real world are highly constrained by its binary nature. A clear pathway to its extension to real-valued problems remains elusive. This paper aims to unleash this pathway by proposing novel end-to-end Nature-Powered Graph Learning (NP-GL) framework. Specif- ically, through a three-dimensional co-design, NP-GL can leverage the sponta- neous energy decrease in nature efficiently solve real-valued graph learning problems. Experimental results across 4 real-world applications with 6 datasets demonstrate that NP-GL delivers, on average, 6.97 102 speedup and 10 en- ergy consumption reduction with comparable or even higher accuracy than Graph Neural Networks (GNNs).
INTRODUCTION
Nature apparently performs complex computations constantly, e.g., solving differential equations and performing random sampling, at clearly lower cost and higher performance than digital com- puters. Nature's 's computation be powered by various phenomena in physies: entanglement and tunneling that drive quantum computing, the spontaneous energy decrease observed in dynamical systems that drives emergy-based computing, and so on.
The latter happens ubiquitously in daily life actually, many dynamical systems in nature spon- tancously and swiftly evolve towards the most stable states with the lowest energy, during which highly complex computation is performed. Intuitive examples are ink diffusion in water and chem- ical reactions among molecules. These natural processes extremely complex to be evaluated with acceptable accuracy on digital computers, chemical reactions, such as drastic combustion pro- cesses involving only 100K atoms, can be formulated as over 102 iterations of computation with each iteration requiring 10" FL.OPs (Wu et al. 2023). In contrast, nature could deliver accurate solutions to such complex problems within microseconds. That being said, if we can build com- puters that efficiently harness the power of nature, these computers, so-called "nature-based computers", " have the potential to solve complex real-world learning problems delivering substantial speedups and competitive accuracy compared with traditional solutions e.g., NNs GPUs.
Intrigued by the unique computational power in nature, in the past decade, besides the tireless efforts on the development of NNs, the community has also relentlessly explored ML paradigms that are powered by natural principles and can be realized on nature- based computers. Typical examples quantum MI. (King et al. 2022) quantum computers and optical MI. photonic optical

---

Published as a conference paper at ICLR 2024
The Ising Method with Binary Limitation $3
N1
y Jooos + E
NP-GL 54
A Hamiltonian System for Real Values 54.2 4
9
Binary Ising Model
Inference with New Nature-
N
1 jei
(1u + -Jy)oy 2h.0))
Σ
Computer 54.4
Training of NP-GL Hamiltonian Based on Maximum Conditional Likelihood $4.3
(1y Jyyos hi)
Diagonal Line Reinforcement
Fine tine-tuning& a Pre trained Spatial Model
Guaronsoed Æ>0
Iniiiaize
longana
Binary Ising Machine
Fine
Tune
Initialize
{11, +1
Jonatis
Apatiol
Pre-trained
J.Jopanto
Figure 1: Overview of the end-to-end NP-GL framework.
systems (Inagaki et al. 2016b; Lin et al. 2018). Regrettably, despite their perceived potential, many nature- powered ML methods are still predominantly theoretical and demand stringent operational conditions. Notably, the emergence of the "Ising machine" (Afoakwa al., 2021 [hhamma al.) 2022) is closing the gap between theoretical nature-powered methodologies and practical applica- tions. This novel computing paradigm, compatible with CMOS technology. can operate at room temperature with less than Watt of power consumption, signifying a major leap towards unleash- ing the computational power of nature into the real world.
Specifically, Ising machines as physical embodiment of the Ising model can be thought of as a dynamical system governed by the Hamiltonian (energy function of a dynamical system) of the Ising mode("] (detailed in Section 2.1). Driven by the spontaneous energy decrease in nature, CMOS-based Ising machine can swiftly and automatically chase and find its lowest-energy states at the "speed of electrons" with negligible costs mm power & ns latency. Unsurprisingly, the unique computational power of Ising machines has enabled a new nature- powwred ML paradigm - termed "Ising Graph Learning (IGL)" in this paper. In IGL, graph problem (e.g., time series forecasting) is formulated as an Ising model whose parameters are trained to map the problem's 's desired (e.g., prediction) results with higher probabilities to the lower energy states of the Ising machine, enabling the Ising machine to automatically find the desired solution at extreme speed (detailed in Section 2.2). It has been demonstrated that IGL outperforms GNNs with orders of magnitude speedups maintaining competitive accuracy in simple real-wortd graph learning applications with binary data, e.g., traffic congestion prediction (Pan al. 2023) and collaborative filtering (Liu et al. 2023).
Despite the early-stage successes, significant limitation persists within Ising Graph Learning: it only works with binary problems. The limitation is rooted in the definition of the Ising model a node (aka "spin") of the Ising model has only two states. As the physical implementation of the Ising model, Ising machines are also designed for binary objects. Regrettably, as most critical applications in the real world use real-valued data, the practical values of IGL remain highly limited. If the notable advantages of Ising Graph Learning, driven by the automatic energy decrease, can be extended to the real number field, we foresee the ensuing nature- sowered graph learning paradigm offering orders of magnitude speedups and energy savings compared to GNNs while preserving highly competitive accuracy for graph learning problems. A route to this extension remains elusive, and a clear pathway is highly desired.
To this end, this paper proposes end-to-end Powured Graph Learning (NP-GL) frame- work, extending nature's 's computing power inherent in dynamical systems to tackle real-valued and real-world graph learning problems. The framework's overview, encompassing four components, is depieted in Figure [ Limitation analysis of Ising Graph Learning: We theoretically analyze why the vanilla Ising Graph Learning (including both the Ising model and Ising machine) cannot be straightforwardly extended to support real values without upgrading the Ising Hamiltonian. Ex- ploration of NP-GL Hamiltonian: Based on the analysis, NP-GL incorporates a newly designed Hamiltonian that is highly werddware-fieddyy It inherits the strengths of the Ising model, ensuring high expressivity, and maintains distinct stable states with real values. Design of NP-GL train- ing algorithms: Similarly to Ising Graph Learning, the parameters of the new Hamiltonian are trained to construct an energy landscape, which the lowest-energy states correspond to the ground truth derived from historical data. In pursuit of high accuracy, NP-GL training adopts an improved conditional likelihood method with two optimizations: (1) Diagonal Line Reinforcement, which re-
'The Ising model is a probabilistic graphical model widely used to represent binary-state dynamical systems.
2

---

Published as a conference paper at ICLR 2024
inforces the self-reaction parameters for better temporal continuity and differentiates the training of self ceffrrecttonn (diagonal) and coupling parameters that convey distinct physical information. (2) Spa- tial Pre-training coupled with Temporal Fine-tuning, which leverages the similarity between spatial and temporal correlations to better learn from the temporal information. Design of NP-GL nature- based computer: Due the similarity between NP-GI. and Ising Hamiltonians, we build the new nature-based computer governed by the NP-GL Hamiltonian by slightly augmenting the circuitry of the Ising machine. The new nature-based computer, like Ising machines, leverages nature's power to swiftly find the lowest-energy states, but, unlike them, it does so with real values.
To the best of our knowledge, NP-GL is the first end-to-ent real-valued nature-poweed ML solution that outperforms NNs in the real world. Our contributions are summarized as follows.
We propose NP-GL,, end-to-end nature-poweres graph learning method, through codesign of Hamiltonian, training algorithms, and nature-based hardware. NP-GL lifts the binary limitation of existing nature-powered ML methods and extends their applicability to real-valued problems.
We design new hardware-friendl Hamiltonian for real-valued support, coupled with efficient training method with two optimizations that ensure high training speed and quality.
We develop a new nature-based computer for the new Hamiltonian, enabling nature's 's power in electronic dynamical systems solve real-valued learning problems with extremely high speed. Experimental results across four real-world applications and six datasets show that NP-GL deliv- ers 6.97 x 10 speedup and 10x x energy saving with even higher accuracy than GNNs.
2 BACKGROUND AND RELATED WORK
2.1 ISING MODEL AND ISING MACHINE
The Ising model is probabilistic graphical model rooted in the statistical physics of ferromag- netism and widely employed to represent complex dynamical systems. Its Hamiltonian is as follows:
N
Hassng
Jijoro Σ haos: (-1,+1}
Due to magnetic physical property, an Ising spin o; has two states "+1" and "-1", namely, spin up and spin down. Jıg the coupling parameter representing the spatial and temporal correlations among different spins; h; represents the self-reaction strength to the external impact and provides temporal continuity for time-series problems. The generalized Ising model goes heyond physical lattices and uses complete graphs to embed alll-to-all correlations among spins, therefore exhibiting strong connectivity, expressivity, and the long-range cascading ability to propagate information. A graph can be straightforwardly mapped to an Ising model, where the nodes are modeled as spins a,, and edges as a pair-wise coupling coefficient matrix Jij and the self-reaction h; of individual spins. In the realm of graph learning, these properties exceptionally valuable for accuracy.
Variations of the Ising model. A few other models are related to the (vanilla) Ising model. The quantum Ising model allows spins to be in state that is superposition of up and down, incor- porating quantum effects. It is used mostly for the study for quantum phase transitions (Kadowaki & Nishimori 1998 Dziarmaga 2005; Chakrabarti et al. 2008 and quantum magnetism (Labuhn et al. 2016 SchauB et al. 2015 Bitko et al. 1996). The XY model and the Heisenberg model extra degrees of Treedom for spins so that they are unit vectors in the 2D and 3D spaces, respectively. The former is usually used to study topological phase transitions (Leoncini et al. 1998 Song & Zhang 202T), and also acts as the model foundation for electronic oscillator-based Ising machines. The Heisenberg model allows spins to orient in three spatial dimensions, making it crucial for studying magnetism in certain materials (Fisher 1964 Arnesen et al.) (000). while 3D spin orientation significantly increases the model complexity. In spite of the number of Ising model variations, they largely remain theoretical with few practical realizations, as even realizing the vanilla Ising model very challenging (more details introduced in (Afoakwa et al. (02T)). Reasonably, it is vital to preserve the feature of easy manufacturing when adding real-value support to the Ising model.
Ising machine, a physical embodiment of the Ising model (Cipra 1987), can be thought of as dynamical system whose gradient is the Ising Hamiltonian. The inherent nature of gradient dynam- ical systems is to seek the lowest-energy most probable states. Like other dynamical systems, the
3

---

Published as a conference paper at ICLR 2024
Ising machine also naturally gravitates towards the lowest-energy state. What powers this inclina- tion of the Ising machine to find the lowest-energy states is inherently derived from nature, more specifically, the spontaneous energy decrease in nature. To show some Ising machine examples, we list three prominent technologies, including D-Wave's quantum annealers (Harris et al.) 2010), Coherent Ising Machines (CIMs) (Inagaki al.] 2016a), and coupled oscillators (Wang & Roy- chowdhury 2019). D-Wave's 's system takes advantage of quantum effects with its superconducting qubits, but also faces constraints in problem mapping and high power consumption due to cryogenic operational requirements. CIMs use Optical Parametric Oscillators, enabling simpler, all- spin coupling, but are challenged by scalability and temperature stability issues, as well as heavy com- putational demands for pulse modulation. In addition, emerging oscillator-based Ising machines, which form stable phase relationships and utilize LC tank oscillators suitable for analog circuits, face challenges in the oversized machine and the lack of high quality inductors. On the other hand, the Ising machine used as the backbone of the proposed NP-GL nature-based computer is BRIM (Afoakwa et al., 202T), the current SOTA. As discussed earlier, considering the difficulty in realiz- ing the Ising models, even the vanilla model, the practicality of deployment becomes exceptionally important. Fortunately, BRIM is constructed to operate at room temperature using low-power cir- cuits and in standard CMOS technology, inheriting decades of manufacturing experience. In such Ising machine, the states of objects, aka "spins", are effectively modeled as nano-scale capacitors and the coupling parameters representing the correlations among objects are modeled as resistors of varying resistance. Specifically, the electrons automatically traverse between capacitors through wires connected to resistors to establish stable electric charge distribution, enabling the "speed of electrons". Consequently, the overall energy is minimized, and the lowest-energy state solution is found with extremely low operating latency. It is not hard to imagine that if the coupling parameters in the Ising Hamiltonian can be trained to effectively map the desired results of a given ML problem to the lowest-energy states, the Ising machine can then perform nanosecond-seut nature-powered ML inference and find the expected results.
2.2 ISING GRAPH LEARNING: BINARY GRAPH LEARNING BASED ON ISING MACHINE
When it comes to solving a graph learning problem using an Ising machine, two steps are encom- passed. Firstly, the complete correlation graph is trained to construct an energy landscape, in which the energy ground state of the Ising model corresponds to the ground truth derived from historical data pertaining to the problem. Secondly, upon initializing the spins, the Ising machine undertakes a process of electron-speed annealing with asynchronous spin flipping, enabling it to efficiently converge towards the energy ground state that accurately represents the problem's 's ground truth. In essence, the computation is carried out by nature itself, resulting in an exceptionally rapid compu- tational process. Recent studies have demonstrated that Ising Graph Learning outperforms GNN in binary problems. However, this new ML paradigm only works with binary problems.
2.3 GRAPH NEURAL NETWORKS
GNNs represent a class of models designed to address graph-structured data, which are prevalent across diverse domains, including physics(Shlomi et al., 22020), social science(Yang et al. 2021). bioinformatics Yi et al.) 2021). combinatorial optimization LIU (022) and so on. GNNs effectively learn graph embeddings by iteratively propagating information between connected nodes in a graph. They usually employ formation from its neighbors, allowing the network to capture complex relational patterns within the graph. These learned embeddings contain valuable insights about the graph, making GNNs powerful tool (Xu et al. 2018 Zhhu et al. 2020) for various graph-related tasks such as node clas- sification(Kipf & Welling, 2016; Jiang & Luo, 2022), link Prediction(Zhangg & Chen 2018 Chen
pressage-passing mechanism, where each node aggregates and updates in-
et al. 2022), and graph classification Wu et al. 2019a).
3 MOTIVATION: THEORETICAL ANALYSIS OF THE BINARY LIMITATION
This section theoretically examines the inherent binary limitation of the Ising model. Specifi- cally, it demonstrates that the direct replacement of binary variables of the Ising Hamiltonian with real-valued ones without modifying the Hamiltonian formulation (namely, "naive real-valued Ising
4

---

Published as a conference paper at ICLR 2024
model") will result in energy landscape without local energy minima. Consequently, this model lacks the necessary expressivity to solve real-valued ML problems.
The stationary points of the naive real-valued Ising Hamiltonian obtained by solving the below equation for spins, with o, E IR being the only difference from Hising
i Oo:
N
Σ (Jis + JziJa - hy =0
which can be formulated into the following vector form:
-J'o = h; diag(J')
(3)
where J' = J + J For stationary point analysis, the Hessian Matrix of H is computed:
H H -J
Because J' is a constant matrix, all stationary points have the same concavity. J' is also diagonaliz- able due to its symmetry. Therefore, based the theorem in linear algebra, tr(J") Σ; X where X; is the i'th eigenvalue of J'. As the diagonal line J' is O, both positive and negative eigenvalues exist unless J' = ONXN- This suggests that all the stationary points local minima, indicating that there are always neighbor states with lower energy and therefore no local minima
saddle points instead of
be found by the Ising machine. In practice, both the energy of the Ising model and the values of the spins in the Ising model have boundaries (e.g.. +1 and -1 for the spins), the Ising machine always stops at the boundary energy state with highly polarized spin values. If used in ML, these spin values are uninformative, normally representing inaccurate results. To this end, we are convinced that the Hamiltonian must upgraded to support real values.
In addition to the Ising model, adjustments in the Ising machines are also desired. Constructed upon the Ising model, Ising machines specialize in accurately bisecting group of nodes into two parties with opposite features, putting the effort in maintaining polarized result. Nevertheless, for the sake of real-valued support, it is favorable to allow the spins to stabilize at intermediate states rather than being exclusively polarized at their boundaries. By implementing this adjustment, the machine is granted the ability to express a wider range of values and is therefore generally applicable in ML.
To bring the nature power to the realm of real values, by developing the NP-GL framework, which is described in depth in the following section.
motivated to break the binary limitations
THE PROPOSED NP-GL FRAMEWORK
This section introduces the details of the proposed NP-GL framework. We first provide overview of NP-GL workflow, and then step-by-stey present the design of NP-GL Hamiltonian, training algo- rithm, and nature-bar based hardware architecture.
4.1 OVERVIEW OF NP-GL
Figure shows three essential steps in solving real-world graph learning problems with NP-GL. First, real-world graph is mapped to Hamiltonian-based probabilistic graphical model, which upgraded from the Ising Hamiltonian to support real values (Sec. 4.2p. In the model, the nodes modeled spins, while the relations between spins, logical edges, modeled as coupling and self-reaction parameters J and h. Second, through training (Sec. (.3.. with real-valued historical data, energy landscape is constructed by learning the Hamiltonian parameters, during which the energy ground state is mapped to the maximum likelihood in the model determined by the observed samples. Third, to align with the upgrade of Hamiltonian, we make slight modifications to SOTA Ising machine to build a nature-based computer for NP-GL. After the trained parameters are deployed on our nature- based computer, the computer starts to search for the lowest energy state that represents the desired inference results (Sec. 4.4). The specific computing process is as follows: For the known spin values, we keep the spins fixed to the known real-valued data and let other spins evolve to find the lowest energy state conditioned on the known information. For temporal graph prediction, spins are uniformly divided into multiple partitions, representing the nodes from the previous, current, and future time steps, respectively. The nodes representing the previous and current time steps are fixed to historical data, while the others are predicted by evolving towards the spin configuration with the lowest energy.
5

---

Published as a conference paper at ICLR 2024
Probabilistic Graphical Model Mapping
Model Training
Inference the Dynamic System
Hav
Hamiltorian
Energy Landscapes Determined by and
Energy Landscape
Dynamic System
Program
Predicted Node
andd
Values
coupling
Seff-ltaction
Hsv
To Lowest,
Geaph Fdie
<
Sel--oop
Verterz
Observvdd Sample Lowwest line State
01101
Energy
N-11 dimensions
Inittel Nede Valuue
Figure 2:
The workflow of NP-GL, involving Hamiltonian mapping, training, and inference steps.
4.2 THE PROPOSED HAMILTONIAN
The new Hamiltonian is proposed for three major reasons, (1) To solve the spin polarization problem in Sec. 3 (2) To enable effective training techniques. (3) To minimize the adjustment of our back- bone hardware. We hereby use a pure quadratic term to replace the linear term in Ising Hamiltonian:
N
N
HRV
Jijoiaz + Σ hoo? oz ER
(5)
It can be inferred that J and h still represent spin coupling and self-reaction similar to the Ising model, inheriting the connectivity, expressivity, and the information propagation ability from it. Additionally, this Hamiltonian can be formulated more compactly as HRv = - Eis Jiyora, with Ja -h;, indicating that (-h) is embedded in J' as the diagonal line. If h is positive, the convexity of the Hamiltonian is ensured, enabling the existence of local minima. As we will discuss shortly, h is guaranteed positive with our training method, and the adjustment in Hamiltonian can be easily implemented on top of our backbone hardware.
4.3 THE TRAINING METHOD OF NP-GL HAMILTONIAN
The training for Hamiltonian parameters aims map the energy ground state the maximum likelihood with vellev data. consists of two major components: the conditional likelihood method as the backbone, and two optimization methods enhance training quality.
4.3.1 THE CONDITIONAL LIKELIHOOD METHOD AS BACKBONE
To perform real-valued training with affordable computing power, conditional likelihood method is proposed. The spin configuration of our model denoted as {oi..... NN The joint probability density function of spin configuration S is defined under the Boltzmann distribution
P[S] h) z exp(-HRv}
where Z is the partition function. Instead of maximizing the likelihood for the observed samples altogether which is computationally infeasible, conditional likelihood is maximized an approx- imation. That is to say, we optimize the parameters by focusing on one spin (say, o,) at a time, while treating other spin values as conditions. In particular, we split HRV into HRV and HRV. Ar to represent the terms dependent and independent of o, The joint probability density then becomes:
p(o;, σ.) Z exp( HRV, HRV
consider all o as conditions, HRV,Y becomes a constant. Subsequently, we obtain:
p(a. vlla))) x expl- HHVV,
(8)
Specifically,
HRV. HRV -EEH + hi12
(9)
where z is the spin variable, and E; - Eyys Jejar for conciseness. It is apparent that HRV, is minimized at μ = s 2//2h, which is determined by the trained parameters and the other conditional spin values. This indicates that the value of a spin can converge between its upper and lower bounds, fundamentally supporting valled data. To train this model, either the Mean Square Error (MSE) loss the Li loss between μ 2//2h; and the ground truth is utilized to maximize the conditional likelihood. Once the training is complete, the maximum conditional likelihood is obtained, which is automatically mapped to the energy ground state according to equation g

---

Published as a conference paper at ICLR 2024
Spatial Coupling
(t. t) J sparial
(1. spitial NxN
(t, tj J spatial NxN
(t. I+1)
J temporal NxN
Initialized
N*N
Fine Tune
2N
as
(t, t) J spatial N
(s, t)
(t-1, temporal
( 11
1a+
J spatial
spatial NxN
Temporal Coupling
NxN
NxN
Intuition: (a)(t =Jaa 1) LLVV)) Pre-trained Spatial Model
Target Model
Figure 3: Fine-tuning of pre-trained spatial model. JAB: the coupling between Spin A and B.
Coupling Units
11
13
12
N1
Node Control Units
Voltage Regulator
ZiV
Coupling currents other nodes
Programming Units
4x
24
23
21
N2
Vi
Original Node
Upgraded to support real values
334
33
331
N3
I
V
JuH'
43
J42
J41
N4
Variable Resistor
Coupling currents other nodes
NP-GL Hardware
Upgraded Node
Network Control Units
R: resistance-related scaling factor
Figure 4: The nature-based computer upgraded from its binary predecessor support real values.
This training approach is applicable to both spatial and spatial-temporal models. In the former scenario, the spins only represent the nodes the same time step, while in the latter case, more spins are necessary to represent the nodes from past, current, and future time steps.
4.3.2 TRAINING ENHANCEMENT TECHNOLOGIES
With the training backbone established, there are still two questions remain to be answered. (1) J and h have distinct physical meanings. Is there a way differentiate them during training? (2) How do we leverage the similarity between spatial and temporal correlations?
A. Diagonal Line Reinforvement (DLR): and are trained jointly with the training method described above. However, these two types of parameters have distinct physical meanings: coupling and self-reaction. In practice, they also usually differ by orders of magnitude. To differentiate these parameters in training, reinforce h on the diagonal line of J by multiplying scaling factor hyperparameter. This method is applicable to both spatial and spatial-temporal models. As Figure illustrates, the diagonal line of J spattial is directly multiplied by the factor to reinforce the self- reaction. Similarly, a spatial-temporal model, all four submatrices have their individual diagonal lines reinforced, enhancing both the self-reaction and the temporal coupling of the same node.
B. Spatial Pre-training Coupled with Temporal Fine-tuning: Intuitively, Spin A at time t and I+1 may have similar properties. For instance, they may have similar coupling relations with respect to Spin B, as Figure 3 shows. Based on this, we speculate that the temporal couplings may have similar values as the spatial couplings. To exploit this property, we pre-train spatial model featured as JNNN and map it onto spatial-tempore model as initial values. The model is subsequently fine- tuned to obtain a final model. Through this approach, the similarity between spatial and temporal correlations is leveraged, resulting in improved accuracy and convergence speed.
4.4 INFERENCE WITH A NEW NATURE-BASHR COMPUTER
Aiming to harness the power of nature and achieve an end-to-end solution for graph learning, the inference process is carried out nature-based computer modified from SOTA Ising machine to align with the upgrade in the Hamiltonian. The overall layout of the computer shown in Figure in which the nodes connected through the coupling units in an all-to-all manner. Specifically,

---

Published as a conference paper at ICLR 2024
Table 1: Accuracy comparison (in MAE/ RMSE): the lower the better best results are in bold.
Application
Traffic Flow
Air Quality
Taxi Demand
Pandemic Progression
Dataset
PEMS04
PEMS08
PM2.5
PMIO
NYC Taxi
Texas COVID
Graph WaveNet
20.84733.00
24.07
182373.106
195473.4990
10.22 022270 21.24
82.967 430.1
ASTGCN
20.79//3.39
33.68/23.73
1883/3.133
2.265/3.662
N/A4
N/A
MTGNN
19.96/31.64
13.13/22.79
1.83572.933
19900/3.303
7.0797
15.40
84.177414,2
DDGCRN
18.97730.59
14.647
1.71173.004
73315
3,0397
10.21
23.94 188.8
MegaCRN
17.65/29.25
13.707 21,03
1.64672.866
1.74173.098
6.0827
15:01
83.73 423.6
NP-GL- MI-Gl-VVnnill-MM
VanillaMMSE MSE
18.37729.10
16.027 7 24.31
1.71072.868
L860/3.146
3.741 / 11.41
41.46/ 268.9
NP-GL-MSE
18.09/28.06
13.1172340
1.7027
13.42/3.14
3.19579.610
37,60 268.8
NP-GL-LI
17.07/27.66
13.51/21.9
1.624/29144
1730/3.013
3,0317 10.08
22.04/ 117.7
the spin values are represented as the voltage on the capacitors in the nodes, while the parameters J' and h are represented as conductance. On the right-hand side of the figure, the modifications are colored green, showing that we merely replace the voltage regulator "ZIV" with a variable resistor in each node import the quadratie term Σ, h V22 into our Hamiltonian. The electric current of Node i is written as equation TO where Jıj" is the effective conductance of the coupling between nodes and 2h, is the effective conductance of the added variable resistor in the node.
dV. 1 (lin -or)
1 C
Σ 1t1
Joy's - - Vi) = C 0V;
87ay
(10)
dt
which satisfies
d'HR dt
Σ HyRy dV. di
)2 <0
(11)
OV.
dt
This suggests the Hamiltonian described by the CMOS compatible circuit spontaneously decreases. In other words, the lowest energy state is automatically pursued.
After the Hamiltonian parameters J and h are obtained from training, they are deployed on our nature- nature-based computer for inference. In practice, the spins representing the nodes from the previous and current time steps are fixed to the known data, allowing the spins representing the nodes in future to evolve and reach the spin configuration with the lowest energy, thus performing the prediction.
EXPERIMENTAL RESULTS
5.1 EXPERIMENTAL SETUP
Applications and Datasets. We evaluate the NP-GL framework with four real-world spatial-
temporal applications and six real-world datasets including Traffic Flow, Air Quality, Taxi Demand, and Pandemic Progression follows. More information is reported in Appendix [A.]]
Application: Traffic Flow- Prediction of the number of vehicles passing through detectors per unit time; Datasets: PEMS04 & PEM08 Traffic flow data in metropolitan areas of California. Application: Air Quality - Prediction of air pollution levels; Datasets: CAQRA-PM2.5 & CAQRA- PM/0 Chinese historical PM2.5 and PM10 data from 2019.5 to 2019.12 sampled from the Chinese Air Quality Reanalysis (CAQRA) database.
Application: Taxi Demand - Prediction of the hourly number of taxi trips; Dataset: NYC Taxi
The hourly number of taxi trips in New York City in 2022.
Application:
Pandemic Progression Prediction of the daily number of new cases; Dataset: Texas COVID - -OVID-2020--20 2023 daily case increment of COVID-19 in Texas.
Baselines. For fair evaluation, we compare NP-GL accuracy and inference latency with five SOTA spatial-temppral GNNs including Graph WaveNer (Wu et al. 2019b), ASTGCN (Guo et al. 2019) MTGNN (Wu et al., 2020), DDGCRN Weng et al.) (2023), and MegaCRN Diang et al. (203). Plat- forms. NVDIA A 40GB GPUs and AMI) Ryzen Threadripper PRO 3975WX CPUs
used to
measure the inference latency of the baseline GNNs. The nature-based computer utilized to evaluate the performance of NP-GL is adapted from the current SOTA Ising machine, BRIM (Afoakwa et al.) 2021 based on Cadence Analog Design Environment.
Graph topology is unavailable for NYC Taxi and Texas COVID) datasets. Different from MTGNN and Graph WaveNet, ASTGCN does not support graph topology generation.
8

---

Published as a conference paper at ICLR 2024
10:
2 10
Latenor
10
10.5
10.
10,
10
CAQRA-PM2.5
CAQRA-PMIO
PEM1S04
PEM508
Texas COVID
NYC Taxi
GraphWaveNet-FUU ASTGCN-GPU
M MTGGN CPU RegaRRN-GPU
ASTGGN- CPU
Graph Net- GPU
MITGNN- GPU
DOGCRN GPU
Nature Based Computer
Figure 5: The latency comparison of CPU, GPU, and NP-GL nature-based computer.
60
50-
CAORA-PM2.5
250
300
PEMS04
2000
1750
Texas
1500
MAE
200
w1250
2 1000
750
10-
50
250
500
0-
0.5
1.0
1.5
30xx10
0.5
1.0
200
0.5
1.0
1.5
2.0x10
70
600
CAQRA-PM10
300
PEM508
500
NYC Taxi
.50
250
400
-40-
30
150
300
20
100
10
50
100
0
0
x110'
x10
0.5
1.0
1.5
2.0x10
Processing Time
Processing Time
(s)
Processing Time (s)
Figure 6: Natural process of energy decrease in NP-GL computer to find desired inference results.
5.2
EVALUATION OF ACCURACY
Table 5.2 compares the accuracy of predicting the l'th snapshot based on the (I-1)'th snapshot with five GNN baselines and two NP-GL. variants. NP-GL-VaniIIa-MSE is equipped with the DLR op- timization. NP-GL-MSE is further improved with Spatial Pre-training and Temporal Fine-tuning. NP-GL-LI uses LI loss instead of MSE loss compared with NP-GL-MAE, as we realize that the data outliers may significantly affect the accuracy. Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) that reflect the average and the deviation of the result quality are adopted as the accuracy metries for comparison. For the MAE metric, NP-GL outperforms all baselines across all datasets. For RMSE, except for PEMS08 where MegaCRN is slightly better, NP-GL outperforms all other baselines.
5.3 EVALUATION OF LATENCY
This section evaluates the inference latency of NP-GL. Figure compares the inference latency of GNNs and NP-GL., highlighting the extraordinarily short latency introduced by our nature-based computer that is powered by nature and operates at extremely high speed. In the figure, the nature- based computer delivers orders of magnitude speedup (from 100s to 10*) compared to CPU and GPU results across all datasets and baselines. Specifically. 1.70 X 10a speedup is achieved on average for the Texas COVID dataset, while the dataset with the least speedup (NYC Taxi) still impressively reaches 1.01 x 10 To highlight a comprehensive result, we take the mean value of all 6 average speedups obtain the overall average speedup 6.97 10 Figure 6 shows spin evolution processes of the nature-based computer throughout inference. As the spins evolve towards the lowest energy state, the еπтог of the results found by the computer decreases accordingly. The curves show the nature- based computer rapidly optimizes the spin configurations to approach the lowest energy states which are also the desired learning solutions. In terms of energy consumption, our nature-based based computer operates at power of ~ 500 mW, more than 100x lower than modern GPUs and CPUs. The overall energy consumption is approximately 10" less than GPUs and CPUs.
CONCLUSION AND FUTURE INSIGHTS
This paper extends the computational power of nature to real-world graph learning problems by proposing NP-GL. Specifically, NP-GL is nature-powered graph learning model and solves real- valued graph learning problems by harnessing the natural phenomenon of energy decrease. Experi- mental results across 4 real-world applications with 6 datasets demonstrate that NP-GL delivers, on average, 6.97 x 10 speedup and 10" energy consumption reduction with comparable or even higher accuracy than GNNs.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
This work was supported by NSF under Awards SHF-2326494 and No.2231036; by NYS Center of Excellence under Awards No. 2089009;00 and by META Reality Lab. This research was also partially supported by the U.S. DOE Office of Science, Office of Advanced Scientific Computing Research, under award No.66150: "CENATE Center for Advanced Architecture Evaluation" and No.78284: "ComPort: Rigorous Testing Methods Safeguard Software Porting".
REFERENCES
Gon: End-to-end optimization framework for constraint graph
optimization problems.
Knowledge-Basse Based Systems, 254:109697, 2022.
Richard Afoakwa, Yiqiao Zhang, Uday Kumar Reddy Vengalam, Zeljko Ignjatovic, and Michael Huang. Brim: Bistable resistively-coupled ising machine. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pp. 749-760. IEEE, 2021.
MC Arnesen, Bose, and Vedral. Natural thermal and magnetic entanglement in the Id heisen- berg model. Physical Review Letters, 87(1):017901, 2001.
D Bitko, TF Rosenbaum, and G Aeppli. Quantum critical behavior for a model magnet. Physical review letters, 77(5):940, 1996.
Bikas K Chakrabarti, Amit Dutta, and Parongama Sen. Quantum Ising phases and transitions in transverse Ising models, volume 41. Springer Science & Business Media, 2008.
Jinyin Chen, Xueke Wang. and Xuanheng Xu. Gc-lstm: Graph convolution embedded Istm for dynamic network link prediction. Applied Intelligence, pp. 1-16, 2022.
Barry A Cipra. An introduction to the ising model. The American Mathematical Monthly, 94(10): 937-959, 1987.
Jacek Dziarmaga. Dynamics of a quantum phase transition: Exact solution of the quantum ising model. Physical review letters, 95524), 2005.
Michael E Fisher. Magnetism in one-dimensional systems- the heisenberg model for infinite spin. American Journal of Physics, (2(5):343-446 1964.
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the Thirty- -hiry-Third--o AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, 809-1. doi: 10. w3000//aai.v3311133333.33 v3310 01 3301922 922]
1AAT19/AAAF1//AATTT AAAI Press, 2019. ISBN 978-1-57735- URL https://doi.org/ 1tt0/////....1/..a.....
R. Harris, M. W. Johnson, Lanting, A. J. Berkley, J. Johansson, P. Bunyk, E. Tolkacheva, E. Ladizinsky, N. Ladizinsky, T. Oh, F. Cioata, I. Perminov, P. Spear, C. Enderud, C. Rich, S. Uchaikin, M. C. Thom, E. M. Chapple, J. Wang, B. Wilson, M. H. S. Amin, N. Dick- son, K. Karimi, B. Macready, C. J. S. Truncik, and G. Rose. Experimental investigation of an eight-qubit unit cell in a superconducting optimization processor. Phys. Rev. B, 82:024511, B...c......2251.111 Jul 2010. doi: 10. 10.1103/
URL https:://iiiie aps. wptp.aaa...rr///........
1103/PhysReve 82, 024511
Takahiro Inagaki, Yoshitaka Haribara, Koji Igarashi, Tomohiro Sonobe, Shuhei Tamate, Toshimori Honjo, Alireza Marandi, Peter L. McMahon, Takeshi Umeki, Koji Enbutsu, Osamu Tadanaga, Hi- rokazu Takenouchi, Kazuyuki Aihara, Ken ichi Kawarabayashi, Kyo Inoue, Shoko Utsunomiya, and Hiroki Takesue. A coherent ising machine for 2000-node optimization problems. Science, 354(6312):603 606, 2016a. doi: 10. acw.sececnnceaa224333333 URL https: //www... science. arg/doi/abar / 1126 sc ence aah4243
10

---

Published as a conference paper at ICLR 2024
Takahiro Inagaki, Yoshitaka Haribara, Koji Igarashi, Tomohiro Sonobe, Shuhei Tamate, Toshimori Honjo, Alireza Marandi, Peter L McMahon, Takeshi Umeki, Koji Enbutsu, et al. A coherent ising machine for 2000-node optimization problems. Science, 354(6312):603- 6060 2016b.
Renhe Jiang, Zhaonan Wang, Jiawei Yong, Jeph, Quanjun Chen, Yasumasa Kobayashi, Xuan Song, Shintaro Fukushima, and Toyotaro Suzumura. Spatio-temporal meta-graph learning for traffic forecasting. In Proceedings of the AAAT Conference Artificial Intelligence, volume 37, pp. 8078-8086, 2023.
Weiwei Jiang and Jiayun Luo. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications, 207:1 17921, 2022.
Tadashi Kadowaki and Hidetoshi Nishimori. Quantum annealing in the transverse ising model. Physical Review E, 58(5)5355,, 1998.
Andrew King, Sei Suzuki, Jack Raymond, Alex Zucca, Trevor Lanting, Fabio Altomare, Andrew Berkley, Sara Ejtemaee, Emile Hoskinson, Shuiyuan Huang, al. Coherent quantum annealing in a programmable 2,000 qubit ising chain. Nature Physics, 18(1 (((1)):332--322) 2022.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In International Conference Learning Representations, 2016.
Henning Labuhn, Daniel Barredo, Sylvain Ravets, Sylvain De Léséleuc, Tommaso Macri, Thierry Lahaye, and Antoine Browaeys. Tunable two-dimensional arrays of single rydberg atoms for realizing quantum ising models. Nature, 534(7609):667-670 2016.
Xavier Leoncini, Alberto D) Verga, and Stefano Ruffo. Hamiltonian dynamics and the phase transi- tion of the xy model. Physical Review E, 57(6):6377, 1998.
Xing Lin, Yair Rivenson, Nezih T. Yardimci, Muhammed Veli, Yi Luo, Mona Jarrahi, and Aydogan Ozcan. All-optical machine learning using diffractive deep neural networks. Science, (6406) 1004-10000 1008, 2018. doi: 10.1 126/sciennc..aat88844
URLpttps: www. www.seciecce.oom wrg/doroooooooooooooo doi/
aat abs Abs/10.1126/cciee aat 8084
Zhuo Liu, Yunan Yang, Zhenyu Pan, Anshujit Sharma, Amit Hasan, Caiwen Ding, Ang Li, Michael Huang, and Tong Geng. Ising-ef: A pathbreaking collaborative filtering method through efficient ising machine learning. In Proceedings of the 60th ACM/IEEE Design Automation Conference. of DAC, 2023.
Zhenyu Pan, Anshujit Sharma, Jerry Yao-Chieh Hu, Zhuo Liu, Ang Li, Han Liu, Michael Huang, and Tony Geng. Ising-traffice traffic: Using ising machine learning to predict traffic congestion under uncertainty. Proceedings of the AAAI Conference on Artificial Intelligence, 37(4):935- (tLhtttps://
Jun. 2023. doi: 10. 10.1609aaai. wwwp:///aaa............ v3718.26121.
aaai www./orrggerreeeeeeeeeeeeeoeeeee index.iexx.////// h
AAAI
26121
Peter SchauB, Johannes Zeiher, Takeshi Fukuhara, Sebastian Hild, Marc Cheneau, Tommaso Macri, Thomas Pohl, Immanuel Bloch, and Christian GroB. Crystallization in ising quantum magnets.
Science, 4455
2015.
Anshujit Sharma, Richard Afoakwa, Zeljko Ignjatovic, and Michael Huang. Increasing ising ma- chine capacity with multi-chip architectures. In Proceedings of the 49th Annual International Symposium Computer Architecture, pp. 508-521, 2022.
Jonathan Shlomi, Peter Battaglia, and Jean-Roch Vlimant. Graph neural networks in particle physics. Machine Learning: Science and Technology, 2(2):021001, 2020.
Feng-Feng Song and Guang-Ming Zhang. Hybrid berezinskii-kosteriizz -thuuless-ztouless and ising topologi- cal phase transition in the generalized wwo-dimensional xy model using tensor networks. Physical Review B. 08,51,, 2021.
Tianshi Wang and Jaiject Roychowdhury. Oim: Oscillator- based ising machines for solving com- binatorial optimisation problems. In Ian McQuillan and Shinnosuke Seki (eds.), Unconventional Computation and Natural Computation, pp. 232-256, Cham, 2019. Springer International Pub- lishing. ISBN

---

Published as a conference paper at ICLR 2024
Wenchao Weng, Jin Fan, Huifeng Wu, Yujie Hu, Hao Tian, Fu Zhu, and Jia Wu. de-
composition dynamic graph convolutional recurrent network for traffic forecasting. Pattern Recognition, 142:109670, 2023. ISSN 0031-3203. doi: http:///olorr/10000111..ppggg
1023.109670. URL https: httpps::/wwwwwsetc. www.sccinced. rect. wwwtrretttooooo cttp///cciieeeeaatticle/llili 50031320322003777
Wu, Geng, A Guo, S Bandara, P Haghi, C Liu, A Li, and M Herbordt. Fasda: An fpga- aided, scalable and distributed accelerator for range-limited molecular dynamics, In International Conference for High Performance Computing, Networking, Storage and Analysis, 2023.
Jun Wu, Jingrui He, and Jiejun Xu. Net: Degree-specific graph neural networks for node and graph classification. In Proceedings of the 25th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 406-415, 2019a.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. 2019b.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Con-
necting the dots: Multivariate time series forecasting with graph neural networks, 2020.
Keyulu Xu, Weihua Hu, Jure Leskovee, and Stefanie Jegelka. How powerful graph neural networks? arXiv preprint arXiv: 181000080. 2018.
Liangwei Yang, Zhiwei Liu, Yingtong Dou, Jing Ma, and Philip S. Yu. Consisrec: Enhancing gnn for social recommendation via consistent neighbor aggregation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2141-2144, 2021.
Hai-Cheng Yi, Zhu-Hong You, De-Shuang Huang, and Chee Keong Kwoh. Graph representation learning in bioinformatics: trends, methods and applications. Briefings in Bioinformatics, 23(1): bbab340, 2021.
Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. Advances in neural information processing systems, 31, 2018.
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang. Changeheng Li, and Maosong Sun. Graph neural networks: A review of methods and applica- tions. Al open, 1:57-81, 2020.
12

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1 DATASETS
NP-GL is evaluated with four real-world spatial-temporal applications and six real-world datasets listed below. Their specification details is reported Table All datasets are partitioned into 70/20/10 percentages for training, validation, and test purposes.
Table A.1: Dataset specifications. A snapshot is the state of the data at a specific point time.
Application
Traffic
Flow
Air Quality
Taxi Demand
Pandemnic Progression
Dutuset
PEMS04
PEMS08
CAQRA-PM2.3
CAQRA-PMIU
NYC
Texas COVID
Snapshots
16992
17856
5856
5856
8759
1100
of Nodes
307
170
400
400
259
256
Data Range (Max-Min)
919
1147
167.0
186,9
11I5
25790
A.2 HYPERPARAMETERS
For all experiments carried out on the six datasets (detailed specifications reported in Table A.I). the Rprop optimizer is adopted with a custom early-stop mechanism. The same hyperparameters are used across the experiments except for the DLR factors used reinforce diagonal lines of J, which are listed in Table A.2
Table A.2: DLR Factors as the Hyperparameters for Different Datasets.
Application
Traffic Flow
Air Quality
Taxi Demand
Pandemic Progression
Dataset
PEMS04
PEMS08
PM2.5
PM10
NYC Taxi
Texas COVID 5.89
DLR Factor
12.28
8.50
16.00
6.00
12.95
Table A.3: The universal hyperparameters used for all 6 datasets.
Minibatch
LR
Etas
Step Sizes
Decay
Step Decay
Early Stop
64
0.01
(0.5, 1.2)
(IE-4, 50)
0.5
50
1E-6
The rest the hyperparameters are listed in Table A.3 where LR is the initial learning rate, Etas and Step Sizes are the hyperparameters intrinsic to the Rprop optimizer. To implement early stop mechanism, the learning rate is multiplied by 0.5 (LR Decay) if the loss function does not decrease in 50 epochs (Step Decay). Once the learning rate below 1E-6 (Early Stop), the training process is terminated.
A.3 DATA DISTRIBUTION PROFILING
100
80
CAORA-PM2.5
PEMS04
80
60
200
Predicted
Node
40
60
150
Ground Truth
Prodicted Average
Average
50
100
125
150
400
800
500
2000
Node Count
100
CCAQRA-P110
40
PEMS08
250
NYC Taxi
30
200
60
150
20
40
100
10:
0
25
100
125
150
175
400
600
800
1000
1200
50
100
150
200
250
300
Data Values
Data Valuee
Data Values
Figure A.1: The data distribution comparison of the predicted data and ground truth data.
The example predicted data distribution and the ground truth distribution across all 6 datasets are illustrated Figure A.1 The examples are randomly sampled from the snapshots used for graph prediction. It can be observed that despite the datasets have distinct data ranges and distributions, the predicted data distribution obtained from our approach is very close to the ground truth, with the mean data values (dashed lines) almost identical.
13


Reviews:
[{"review_id": "Veu3d9hK4x", "reviewer": "Reviewer_UYh4", "summary": "This paper propose a novel graphic model architecture by improving the existing ising model. The authors further propose the efficient training and inference algorithm to search for the local minimum of the proposed solution. The paper also shows the potential hardware architecture to implement this novel graphical model. The results demonstrate that the proposed solution can achieve comparable performance as GNN over multiple tasks.", "questions": "Please see the weakness section and solve the problem accordingly.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "+ A novel graphical model architecture\n+ Inference and training methods to achieve the local min\n+ Efficient hardware implementation", "weaknesses": "- To me, the most significant problem of this work is insufficient review for the prior work. Section 2.1 and 2.2 is a good background introduction, but not too many prior works on the variation of Ising model are discussed, making the contribution of this work hard to justify.\n- A section is missing to introduce the prior work on hardware implementation for Ising model.\n- Technically, this paper simply propose a modified version of Ising model by using a pure quadratic term to replace the linear term in Ising Hamiltonian. \n- Graphic model may not be well-suited for ICLR, which mostly focus on deep learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fzKDHMqC7L", "reviewer": "Reviewer_w2U6", "summary": "**Objective:** The paper aims to extend the capabilities of nature-powered computations from binary problems to real-valued graph learning problems. The authors propose a novel end-to-end Nature-Powered Graph Learning (NP-GL) framework, which leverages the natural power of entropy increase to efficiently solve real-valued graph learning problems.\n\n**Methodology:** The NP-GL framework is designed through a three-dimensional co-design, incorporating a new Hamiltonian that is hardware-friendly, maintains distinct stable states with real values, and ensures high expressivity. The training algorithms of NP-GL adopt an improved conditional likelihood method with optimizations for complexity reduction, convergence expediation, and better learning from temporal information. Additionally, a new nature-based computer is developed to support the NP-GL Hamiltonian, enabling the solution of real-valued graph learning problems.\n\n**Results:** Experimental results across four real-world applications and six datasets demonstrate that NP-GL delivers, on average, a 6.97 × 10^3 speedup and 10^5× energy consumption reduction, with comparable or even higher accuracy than Graph Neural Networks (GNNs).\n\nThe contribution lies in\n\n**Extending Nature-Powered ML to Real-Valued Problems:** The paper introduces NP-GL, an end-to-end nature-powered graph learning method that breaks the binary limitation of existing nature-powered ML methods, extending their applicability to real-valued problems.\n\n**New Hardware-Friendly Hamiltonian:** A new Hamiltonian is designed for real-valued support, coupled with an efficient training method that ensures high training speed and quality.\n\n**Development of a New Nature-Based Computer:** A new nature-based computer is developed for the NP-GL Hamiltonian, using the Ising machine as a backbone, enabling the solution of real-valued graph learning problems using nature’s power.\n\n**Significant Speedup and Energy Savings:** NP-GL demonstrates a substantial speedup (6.97 × 10^3) and energy savings (10^5×) compared to GNNs, with even higher accuracy across various real-world applications and datasets.", "questions": "- Could you elaborate on the adaptation for NP-GL on top of SOTA Ising machine? The hardware implementation, challenges, and potential optimizations could provide valuable insights for readers interested in the practical aspects of the work.\n- To confirm I assume the the engergy and latency are based on the simulation in CAD software?\n- Would this machine be able to generalize beyond GNN?\n\n- How it performs as the size of the graph increases and any potential strategies for handling large-scale graph learning problems?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Originality:\n- Innovative Approach: The paper introduces a novel end-to-end Nature-Powered Graph Learning (NP-GL) framework, extending the capabilities of nature-powered computations from binary problems to real-valued graph learning problems. This is a significant departure from existing nature-powered ML methods, showcasing a high level of originality.\n- Unique Integration: The three-dimensional co-design integrating a new Hamiltonian, training algorithms, and a nature-based computer is a unique approach that has not been explored extensively in previous works.\n2. Quality:\n- Robust Methodology: The paper employs a robust methodology, incorporating a hardware-friendly Hamiltonian, efficient training methods with optimizations, and the development of a new nature-based computer.\n- Comprehensive Evaluation: The experimental results across four real-world applications and six datasets provide a comprehensive evaluation of the NP-GL framework, demonstrating its effectiveness in delivering significant speedup, energy savings, and high accuracy compared to GNNs.\n3. Clarity:\n- Well-Structured: The paper is well-structured, with a clear introduction, background, methodology, results, and conclusion sections. This structure aids in the reader’s understanding of the content.\n- Detailed Explanations: The authors provide detailed explanations of the NP-GL framework, the new Hamiltonian, the training algorithms, and the nature-based computer, ensuring that readers can grasp the complexities of the work.\n4. Significance:\n- Addressing Real-World Problems: By extending the applicability of nature-powered ML methods to real-valued problems, the paper addresses a significant gap in the field, making it highly relevant to real-world applications.\n- Potential for Impact: The demonstrated speedup, energy savings, and accuracy improvements have the potential to make a substantial impact in the field of graph learning, showcasing the significance of the work.", "weaknesses": "- Insufficient Discussion on Challenges: While the paper provides a comprehensive overview of the NP-GL framework and its benefits, there could be a more in-depth discussion on the potential challenges and limitations of the proposed approach. Providing such insights would offer a balanced view and help guide future research. \n\nMinor: Add one-liner for the future insights summary to main text from Appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zp22oxj2lx", "reviewer": "Reviewer_qast", "summary": "This paper tackles the issue of the binary nature of existing Ising graph learning by proposing a new Hamiltonian for real-value graph learning problems.", "questions": "*  Could the authors provide more comparison with recent SOTA GNN spatial-temporal GNNs? I found one recent paper,\n    * Jiang, Renhe, et al. \"Spatio-temporal meta-graph learning for traffic forecasting.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 7. 2023.\n* What’s the physical meaning of asymmetric weight decomposition? As it is a natural process (Hamitonian), is this symmetric weight decomposition meaningful for the physical system besides the efficiency consideration?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is well-written and easy to follow, more like a report.\n* Solve an important problem in current Ising graph learning using nature-law-based machines, i.e., only supporting binary values.\n*  Efficient training methods with three optimizations that ensure training speed and quality.\n* Achieves the best accuracy compared to three baselines", "weaknesses": "* Over-exaggerated descriptions in the main context. The paper seems to overclaim some parts, like mentioning, “Regrettably, despite their perceived potential, most\nnature-powered ML methods are still predominantly theoretical, outperforming NNs only in toy problems under highly idealized conditions” while the Ising machine is also not quite partial and only applicable for some problems.\n* Out-dated baselines. In comparison, the authors chose three “SOTA” spatial-temporal GNNs, while the earliest was published in 2020. The author should compare with more recent advances.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)