InstaTrain: Adaptive Training via Ultra-Fast Natural Annealing within Dynamical Systems
Authors: Chuan Liu
Ruibing Song
Chunshu Wu
Pouya Haghi
Tong Geng
Venue: ICLR 2025 Poster (2025)
Keywords: Time-Series Prediction
Online Learning
Novel Computing Paradigm
Primary area: learning on time series and dynamical systems
TL;DR: A novel ultra-fast learning paradigm for time-series prediction tasks through iterative natural annealing within dynamical systems.

Abstract: Time-series modeling is broadly adopted to capture underlying patterns present in historical data, allowing prediction of future values. However, one crucial aspect of such modeling is often overlooked: in highly dynamic environments, data distributions can shift drastically within a second or less. Under these circumstances, traditional predictive models, and even online learning methods, struggle to adapt to the ultra-fast and complex distribution shifts present in highly dynamic scenarios. To address this, we propose InstaTrain, a novel learning approach that enables ultra-fast model updates for real-world prediction tasks, thereby keeping pace with rapidly evolving data distributions. In this work, (1) we transform the slow and expensive training process into an ultra-fast natural annealing process within a dynamical system. (2) Leveraging a recently proposed electronic dynamical system, we augment the system with parameter update modules, extending its capabilities to encompass both rapid training and inference. Experimental results on highly dynamic datasets demonstrate that our method achieves orders-of-magnitude improvements in training speed and energy efficiency while delivering superior accuracy compared to baselines running on GPUs.

Published as a conference paper at ICLR 2025
INSTATRAIN: ADAPTIVE TRAINING VIA ULTRA-FAST NATURAL ANNEALING WITHIN DYNAMICAL SYSTEMS
Chuan Liu, Ruibing Song, Chunshu Wu, Pouya Haghi, Tony (Tong) Geng
University of Rochester, Rochester, NY, USA (chuan. liu, ruibing.. song, chunshu. wu, tong. (enggggggg@@rrcceeeeeee edu {phaghi [iraghi)euu... Jeur. rochester.edu
ABSTRACT
Time-series modeling broadly adopted to capture underlying patterns present in historical data, allowing prediction of future values. However, one crucial as- pect of such modeling often overlooked: in highly dynamic environments, data distributions can shift drastically within second or less. Under these circum- stances, traditional predictive models, and even online learning methods, struggle to adapt to the ultra-fast and complex distribution shifts present in highly dynamic scenarios. To address this, we propose Insta Train, a novel learning approach that enables ultra-fast model updates for real-world prediction tasks, thereby keeping pace with rapidly evolving data distributions. In this work, (1) we transform the slow and expensive training process into an ultra-fast natural annealing process within a dynamical system. (2) Leveraging recently proposed electronic dynam- ical system, we augment the system with parameter update modules, extending its capabilities to encompass both rapid training and inference. Experimental re- sults on highly dynamic datasets demonstrate that our method achieves orders-of- magnitude improvements in training speed and energy efficiency while delivering superior accuracy compared to baselines running on GPUs.
INTRODUCTION
Time-series prediction lies at the heart of artificial intelligence, powering applications ranging from weather forecasting (Karevan & Suykens 2020, Bochenek & Ustrnul 2022) to product and con- tent recommendation (He et al.) 2017,Zhang et al. (021). Current neural network methods have achieved remarkable success by learning the joint distribution between inputs and predictions (Lim) & Zohren (222). However, these methods often implicitly assume that the learned joint distribution remains stable over a considerably long period, an assumption that can easily be violated when the underlying distribution undergoes severe shifts, consequently causing significant failures in trained models. In response to this challenge, the community has pivoted toward more adaptive learning strategies, such online learning and continual learning approaches (Hoi al., 2021 Chen al. 2021 Pham et al., [022] Wen et al. 2024).. These methodologies are designed to incrementally adjust model parameters, thereby maintaining alignment with current data trends. Despite their advancements, they struggle to adapt to the circumstances where data distribution evolves rapidly due their insufficient adaptation speed. This underscores the pressing need for more agile and responsive approaches that can swiftly adapt to shifts in distribution and ensure model effectiveness.
In the poot-Moore's 's Law era, the limitations of speed improvements in digital processors (such as CPUs and GPUs) have become more pronounced, attracting growing attention in novel computing substrates, promising yet largely untapped area of research (Momeni et al. 2024). As a promising candidate, recently developed electronic dynamical system (Afoakwa et al., [021]] Sharma et al.) 2022) stands out due to its timely applicability, low power consumption, and exceptionally fast computational speed. Similar to natural dynamical systems, where particles naturally move toward lower energy states following the Second Law of Thermodynamics, the behavior of this electronic dynamical system is governed by its energy function (Hamiltonian). with lower energy states being rapidly reached through natural annealing a process where electrons spontaneously move among capacitors seek equilibrium at the "speed of electrons" and on the milliwatt scale.

---

Published as a conference paper at ICLR 2025
However, despite this system having been utilized to accelerate graph learning inference (Wu al. 2024). the model training process still relies on conventional digital processors. which cannot keep up with rapidly evolving data distributions in real-world applications. Consequently, more ad- vanced learning approach is critically needed to fully exploit the exceptional computational power of this dynamical system. Since the system specializes in performing natural annealing, it holds the potential to meet the ultra-fast model learning demand if the sluggish offline training process can be transformed into the natural annealing process. This idea also aligns with the concept of "mor- tal computation" proposed by (Hinton) 2022), which advocates for the integration of algorithms and hardware, thereby offering significantly lower costs compared to conventional neural networks running on CPUs and GPUs.
In response to this opportunity, we propose Traia, which extends the extraordinary computational efficiency of the electronic dy- namical system from inference to training, ad- dressing the need for capturing rapidly evolving data distributions. The overall framework of this approach is illustrated in Fig. [ compris- ing two major components. (1) Training Algo- rithm: Formulated as dynamical system, our model is determined by the trainable param- eters in the Hamiltonian, or energy function. The proposed algorithm accomplishes training through an iterative natural annealing process, which pushes the lowest energy state of the dy- namical system to match the ground truth pro-
Training Algorithm
Hardware
Grodiohutth
native
Anreolnn
Result
Armesiing
Anneal
Iteratively
Reratively
Compare
Update
Update
Parameter Updater
Gradients
@ High Accuracy
Low Cost
Ultra Fast
vided by training data. (2) Hardware Aug-
Figure 1: Overview of InstaTrrain.
mentation: We enhance the dynamical system
with parameter update modules realize the self-training mechanism. This allows both training and inference be carried
the same hardware, resulting in outstanding computational efficiency and, essentially, achieving real-time model adaptation upon highly dynamic data distributions.
The core contributions of this paper summarized as follows:
We propose InstaTrain, a novel learning approach designed to address the demands for ultra- fast model adaptation in highly dynamic time-series prediction.
We transform the sluggish offline training into an iterative natural annealing process within dynamical system, enabling ultra-fast model training and updating.
We augment the original dynamical system with parameter update modules, extending its ca- pabilities to encompass both rapid training and inference.
Experimental results across highly dynamic datasets show that the proposed method achieves orders-of -fannnitude improvements in training speed and energy efficiency while delivering superior accuracy compared to baselines running on GPUs.
BACKGROUND
This section provides background on the targeted electronic dynamical system, discussing its math- ematical model, physical embodiment, and offline training method.
Mathematical Model. Mathematically, the dynamical system describes how components (nodes)
interact and influence each other's states over time, driving the system's 's evolution toward equilib- rium. The system has been utilized to accelerate graph learning inference (Wu et al.) 2024), with a real-valued Hamiltonian function:
N H(s) = Σ Jyyooa + > ho? inf
N
(1)
where o; E IR. 5 (0,, 02 x denotes the nodes in the dynamical system, J.g represents the relationship between node a, and node oj, and h, refers the self-reaction strength, and is forced
2

---

Published as a conference paper at ICLR 2025
positive. This Hamiltonian function H. is derived from the classic binary Ising Hamiltonian (Cipra 1987) rooted in ferromagnetism physics, but extends it to overcome the binary limitations of nodes restricted values of +1 or -1. Specifically, the binary limitation of the Ising model refers to the failure of naively extending its binary nodes to real values. The Hamiltonian of binary Ising model is (s) = - -Etz Jijā,a; Σ: h,o;, If nodes are real-valued, they evolve to =oo to pursue the lowest energy state, which is -cc. Even if boundaries are applied to o,, 0; is only intercepted along its way infinity, resulting in polarized values and essentially a binary model. In contrast, in the real-valued Hamiltonian function (i.e.. H), the quadratic term acts as an energy regulator, which prevents the energy from going down to -o,, allowing nodes be localized at certain values. This extension gives nodes the ability to take on real values, thus making it feasible to perform more precise modeling of real-valued systems in real life. this work, the Hamiltonian that supports real-valued nodes H. is employed.
Physical Embodiment. The dynamical system is physically realized as an electronic cir- cuit (Afoakwa al., 2021), where electronic components facilitate the system's spontaneous ergy decrease toward equilibrium (i.c., dHH/dt S 0). Specifically, each node o; is represented voltage a capacitor C, while coupling parameters J and implemented resistor conductance. According to Lyapunov stability analysis, the node dynamics can be designed as /dt/ -OHr./0o;:
do, dl
E + Jz)); 2h1o:
C
571
guaranteeing the system evolves toward lower energy state:
dHl N OH do, Σ 07l 2 S 0.
dt Σ
(3)
do, dt
0o,
Here, C' denotes the capacitance, a positive constant. The node dynamics indicate that the value of each node is influenced by its input electric currents (Jaz + Jji)a from others nodes as well as its local current 2h,0;, Therefore, through charging or discharging the capacitors "speed of electrons", the node values adjust rapidly, driving the system toward equilibrium with minimal power consumption.
To train the parameters J and h in H. prior research (Wu et al. 2024 employs a conditional likelihood method implemented on conventional digital processors. This approach fo- cuses on one node a; at a time, treating other nodes as conditions. An estimated value for node o, computed
Offline Training.
2 Eift (Jij + Jji) oj.- The difference between d, and the corresponding ground truth is then evaluated using loss metrics such as Mean Absolute Error (MAE). By mini- mizing these losses, the parameters are optimized to align the ground truth with the system's lowest energy state. Consequently, during inference, the inherent process of spontaneous energy decrease drives the system toward the lowest energy state, producing the desired solution.
3 METHODOLOGYY: INSTATRAIN
In this section, we present InstaTraii, a novel learning approach that leverages the natural annealing process within a dynamical system enable ultra-fast model training, capturing rapidly evolving data distribution for prediction tasks. We first introduce our Iterative Natural Annealing Training (INAT) algorithm, including how to formulate the prediction problem using the dynamical system and the detailed training process. Furthermore, grating update modules to enable the self self-training feature.
augment the original electronic hardware, inte-
3.1 ITERATIVE NATURAL ANNEALING TRAINING (INAT)
3.1.1 FORMULATING PREDICTION VIA NATURAL ANNEALING
Consider a time-series prediction task where we aim to predict a system's 's future states s++1 based on its historical states s i.e., s!+1 fo(s'). The goal is optimize the parameters 0 so that so
3

---

Published as a conference paper at ICLR 2025
accurately captures the system's 's evolution over time. We can reformulate this prediction problem using a dynamical system. Specifically, let s' and s represent the node configurations of the dynamical system at consecutive time steps. Without loss of generality, we clamp the first 2/2 node values to the input state (0)...., = s', and allow the remaining N/2 nodes (/244+1=-- NN) anneal freely. If the Hamiltonian parameters J and h perfectly capture the dependencies between inputs and predictions, then the ground truth configuration s = (s", s s corresponds to the lowest energy of the dynamical system. Consequently, by clamping the input nodes s² and letting the remaining nodes evolve according to the designed dynamics (Eq. 2 the natural annealing process will drive the system toward the lowest energy state (equilibrium), resulting in the remaining nodes converging to the desired solution s s
We can further interpret this annealing process using the Boltzmann distribution, which defines a mapping from energy to probability. Specifically, the lowest energy node configuration corresponds to the maximum probability state through the following:
1 H((5") Ps- Z
(4)
where Z is the partition function defined as J H do. functioning as normalizing constant. Therefore, the system's 's evolution towards the lowest energy state is equivalent to finding the desired prediction sf+1 with the
High Energy Low Probabilty
Initial State
highest probability under the Hamiltomian H. To clu- cidate more clearly, we visualize the whole process in Fig. 2 Clamping the input s' confines the entire en- ergy landscape to subspace compatible with the given input data. Within this constrained landscape, the remain- ing unclamped nodes undergo natural annealing, sponta- neously evolving toward the lowest energy state. This process efficiently yields the desired solution s'+1, lever-
Annealing
Low energy High Probubiiity
Desired Solution
Energy Landscape
Figure 2: Prediction via annealing.
aging the extraordinary computational power inherent in the dynamical system.
3.1.2 TRAINING THROUGH ITERATIVE NATURAL ANNEALING
Through the above description, we can perform efficient prediction on the dynamical system given the optimal Hamiltonian parameters J and h. In terms of training, instead of undergoing costly training processes on digital processors, is much more preferable that model training is also avail- able on the dynamical system. To address this, describe how to obtain the target parameters from the training data through an iterative natural annealing process, the same process used for inference.
Specifically, we seek to maximize the likelihood of the training set under the model:
arg max II J,h
Ps:
(5)
SET
where T is the training set. This is equivalent to minimizing the negative log-likelihood loss:
arg min L (s: J.h) M
1
Σ (In (Z) 1 In (c (c-*)),
J
SET
where M is the number of training samples. The gradients of L with respect to Jij are given by
aC(s)
dln(Z)
8H
aJ₁ aJg
M
0J.j'
SET
where the two terms essentially expectations of node multiplications:
dln(Z)
02 = - o, - do
_ do (0:0,)
(8)
OJi
Z aJ,
1
Σ OH
= M Σ Lo.a5= 0,0, = ((,
(9)
M SET
OJi
SET
4

---

Published as a conference paper at ICLR 2025
Bo
111
Hy
High Energy
Annealing
gradient
Annealing
Annealing
Anneallig
Law energy
Calc gradient
gradient
Minimum Gap
Initial Energy Landscape
Well-trained Energy Landscape
Annealing result
Ground truth
Parameter update Hi.l H, +
Figure 3: Model training through iterative natural annealing.
Particularly, (a; denotes the expectation over the training data, which tractable, and (0:0z) model corresponds the expectation of oxa, given by the current model. Consequently, the gradient for the coupling parameter Jıý is
OC (s) = (0:05) model (0:07))aas JJ
(10)
In the same way, the gradients of h2 are given by OC(s) 0h;
Øl)(2)
OH
Øln(Z)
Σ SET
of
- model + (o,)) data
(11)
Oh;
M
Oh,
Oh;
M
SET
Therefore, to update the parameters, we need (0.0)) and (o, model" which correspond to the expectation under the current model parameters. To estimate (.) we leverage the computational power of the electronic dynamical system achieve remarkable efficiency. As described in $3.1.1 we can estimate the current model's 's prediction g through clamping s' to input nodes and allowing the dynamical system to perform natural annealing. By measuring the node configurations at the end of the annealing process, we can estimate the required model expectations mode) model and (o (mode))) In this way, the training process is transformed into an iterative natural annealing process, as described in Algorithm I and illustrated in Fig. 3 This innovative training process eliminates the need for computationally expensive offline training. Instead, it harnesses the natural energy decrease feature to perform efficient computations, enabling ultra-fast model training.
To summarize, the outcome of the natural annealing process depends on the accuracy of the current Hamiltonian parameters in capturing the dependencies between the inputs and predictions. When the values of some nodes are fixed to s', two scenarios occur: (1) If the parameters properly
Algorithm Iterative Natural Annealing Training
Input: Training set == (s1, 82, S2...., SM) initial Jo, ho, learning n, and training epochs Nieer Output: Trained Hamiltonian parameters J,
1: Initialize J <- JO, - h..
2: for i = to Naer do + in do
3: for each S, (5),
4:
Clamp the first half nodes to s) 5: Perform natural annealing to obtain $/,t1
6: Get (o, and (af medel based on s/j S +1
7: Get (a,or)) and (a data based on sy. s
8: Update Jij <- Jíj (a;0;) model (
9: Update h; +- h; 17 (- model + data)
10: end for
11: end for
12: return J.
5

---

Published as a conference paper at ICLR 2025
describe the dependencies, the annealing process will converge the desired solution s repre- senting the ideal case where the model has successfully learned the correct relationships between the imputs and predictions. (2) If the parameters do not accurately capture these dependencies, the annealing process will instead yield results that align with the current model's expectations, denoted by (a:0ÿ) model model and (o modal" m This outcome indicates that the model's parameters require further optimization to better represent the underlying dependencies. Regardless of the parameter accuracy, both scenarios correspond to the equilibrium state of the dynamical system.
3.2 HARDWARE AUGMENTATION
Hardware for Training & Inference tem is achieved by mapping the node values the voltages applied on nano-scale capacitors C, and modeling the Hamiltonian parameters J and as the conductance of resistors. More specifically, referring Fig. 4 the value of node o, corresponds to the voltage V,, the
Node
The physical realization of this dynamical sys-
Programiming Units
13
Capacttor
C
Node Control
Variable Resistor R
4n
R
effective conductance of the coupling between node o; and node Jj is Jig (ycllow blocks). The effective conductance of the added resis- tor for node a, is 2h, which is embedded the node (green blocks). This mapping enables the construction of the dynamical system using a mesh of programmable resistors, which are in- terconnected and span across all nodes. By ex- ploiting the intrinsic dynamics (Eq. [2) of this resistor-capacioor network, the natural annealing process can be physically implemented, allow- ing for rapid convergence toward the equilibrium state that corresponds to the desired results.
Network Control
From Other Nodes
Positive Path
Node
Preset
Cp
V2
Positive
X
Negalive Path
Negative
Positive
Enable
Enable
Negative
Update
Vi
Juy
Analog
4
Update
Cr
Multiplier
Node
Updater integrated with Jij
Figure 4: Redesigned InstaTrain hardware.
dV. dV dt
0H aV.
EJ 2/, Lin IR. j71
(12)
After implementing the natural anncaling process in this electronic system, we need to further make the system self-trainable. This requires integrating update modules compared the original design. In particular, the update modules take the values of nodes d; and dy as input, compute V.V,, and update the voltage V, applied the capacitor C. The programmable parameter Jıy is then updated according to the value of V,. As depicted in Fig. 4 embed the update modules (purple blocks) into coupling units (for updating Jız) and nodes (for updating h;). The detailed steps are:
Initialize J,y and h2 through preset, giving V, an initial value.
2. Initialize V; and Vy- Set known ground truth values for input nodes, and arbitrary values for remaining nodes.
3. Perform natural annealing to get the updated voltages for unclamped nodes.
Obtain (VVV)) using the analog multiplier, storing the results as voltages in capacitors.
5. Load the ground truths for unclamped nodes.
6. Obtain (VVV using the analog multiplier, storing the results voltages in capacitors.
7. Based on the voltage difference between (VVV)) and (V;V,) data the positive path enabled if the former is larger; otherwise, the negative path is enabled.
8. The difference between the two voltages is used to adjust V,, thereby updating Jij.
9. Repeat steps (2)-(8) for the next iteration.
Through these steps, the entire training process is transformed into an iterative natural annealing process within the dynamical system, enabling ultra-fast training for highly dynamic applications.

---

Published as a conference paper at ICLR 2025
4 EVALUATION
EXPERIMENTAL SETUP
Datasets. We evaluate InstaTrain on five high-frequence datasets, each providing 100 samples per second. Carbon-Oxide consists of sampled time series data collected from gas delivery plat- form facility, capturing readings from chemical sensors exposed to varying concentrations of carbon oxide and ethylene mixtures (Fonollosa al., 2015b). Similarly, Methane includes sampled data from chemical sensors exposed to mixtures of methane and ethylene at varying concentration lev- els (Fonollosa et 2015b). Stock contains sampled stock data of S&P-500 (Nasdaq). Ammonia includes sampled time series recordings from chemical detection platform, featuring data from 72 metal-oxide sensors across six different locations, all maintained under consistent wind speed and operating temperatures (Fonollosa et al.) 2015a). Toluene comprises sampled time series recordings from 72 sensors one location, collected under ten varying conditions (two wind speeds and five operating temperatures) from chemical detection platform (Fonollosa et al. 2015a).
Baselines. We evaluate InstaTTrain in three scenarios: static scenarios, low-frequency update scenar- ios, and high-frequency update scenarios.
Static Scenarios: Models are trained on the first 25% of each dataset and evaluated on the re- maining 75%. We compare against Graph Neural Networks (GNNs), Transformer-based time series prediction models, and NPGL. (Wu al., 2024). The GNNs include: Graph WaveNet (Wul et al. 2019), MTGNN (Wu et al., (222), and MegaCRN (Jiang et al., 2023). The Transformer- based prediction models include: Autoformer (Wu et al. 2021). DLinear (Zeng et al. (023), iTransformer (Liu et al. 2023a). All methods are implemented following the experimental se- tups detailed in their respective original papers.
Low-Frequency Update Scenarios: Building upon the pre-trained static models above, the GNNs, Transformer-based prediction models, NPGL, and InstaTrain updated new data become available. In particular, the models are updated once after observing 1,000 snapshots, equivalent to 10 seconds in the real world. After each update, the model is tested on the subse- quent 1,000 snapshots.
Table Accuracy comparison using MAE. Lower values indicate better performance. LF/ Low / High Frequency. Grayes results indicate "Not Achievable" results due to slow processing.
Dataset
Carbon-Oxide
Methane
Stock
Ammonia
Toluene
GWN
14.40
19.34
3.34
19.35
13.26
MTGNN
24.47
19.31
2.85
13.43
18.74
MegaCRN
25.94
23.65
3.45
18.12
20.15
Informer
14.16
19.37
2.76
13.59
13.07
Static
DLinear
14.08
19.32
2.31
12.74
12.82
iTransformer
14.02
19.29
2.27
12.41
11.95
NPGL
13.90
19.22
2.01
12.15
11.43
InstaTrain
13.88
19.25
2.02
12.08
11.37
GWN
10.28
11.84
1.85
4.72
5.82
MTGNN
12.51
11.57
1.70
4.95
5.19
MegaCRN
12.34
13,49
1.87
5,41
5.93
Informer
9.21
10.41
1.64
4.39
5.25
LF Update
DLinear
8.82
10.25
1.39
4.16
4.96
iTransformer
8.53
9.72
1.22
3.94
4.85
NPGL
8.25
9.26
1.18
3.81
4.68
InstaTrain
8.28
9.22
1.20
3.72
4.67
FSNet
7.11
7.14
0.79
1.48
2.07
PatchTST
7.05
7.09
0.80
1.46
2.02
HF Update
OneNet
6.93
7.11
0.77
1.42
1.93
InstaTrain
6.79
7.05
0.68
1.36
1.86

---

Published as a conference paper at ICLR 2025
High-Frequens Update Scenarios: Similar to the Low-frequency setup, but with models allowed to update more frequently to further improve accuracy. Specifically, InstaTrain is updated once every 100 snapshots (equivalent to second in real time), leveraging its rapid adaptation capabil- ity. After each update, the model is evaluated on the subsequent 100 snapshots. In this case, we compare against online learning models, including FSNet (Pham al. (022), online-adapted PatchTST (Nic et 2022) proposed in (Wen et al. (024). and OneNet (Wen et 2024). These online learning models are implemented based on the setup detailed in their original papers.
Platforms. We evaluate the accuracy and inference latency of GNNs, Transformer- based mod- els, and online learning models using an NVIDIA A100-40GB GPU. Training latency for NPGL, models, and online learning models also measured on the same GPU. The accuracy and inference lateney of NPGL,, along with the accuracy, training latency, and infer- ence latency of Insta'rrain, are assessed using a CUDA-accelerated Finite Element Analysis (FEA) software simulator implemented based BRIM (Afoakwa et al. 202I). Furthermore, the Cadence Mixed-Signal Design Environment is employed to evaluate the power consumption of InstaTrain.
GNNs, Transformer-
4.2
MAIN RESULTS
Accuracy Evaluations. Table presents accuracy results across five datasets using MAE as the evaluation metric. Results for dynamic scenarios are averaged across test sets. The comparison across different scenarios indicates that high-frequency updates crucial for achieving better per- formance. However, due to computational limitations, GNNs, Transformer-base prediction mod- els, and NPGL can only accommodate the low-frequency update schedule (as shown in Fig. 5). The high- figh-frequency update schedule requires completing updates within 1 second, which is infeasible for these methods. Although selected online learning methods also cannot meet high high-frequeney up- date requirements, we still calculate their accuracy under the high-frequency setup for the sake of comparison. The results show that InstaTrain outperforms GNNs and Transformer-bant tion models in both static scenarios and low-frequency update scenarios across all datasets, with comparable accuracy versus NPGL. On average, InstaTrain reduces MAE by 14.45% compared to baselines in static scenarios and by 15.14% in low-frequency update scenarios. In high- high-frequency update scenarios, InstaTrain outperforms online learning methods with a 6.29% MAE reduction.
predic-
Notably, when comparing across different update scenarios, InstaTrain with high fegg-freueency up- dates achieves a substantial 74.17% MAE reduction compared to all static models, and a 49.51 % MAE reduction versus all low-frequency update models, underscoring the critical importance of high-frequency model updating. In summary, the limitation baselines positions Insta'rrain as the optimal solution in the cases where data distributions change rapidly and require frequent adaptation.
Latency Evaluations. Figure 5 illustrates the update latency of each method across all datasets. InstaTrriin achieves microsecond-level (10-4 seconds) update latency, significantly outperforming other approaches that operate at the second level. The orange dashed line represents the update latency requirement for low-frequency model updates (10 seconds). All models perform below this threshold, enabling them to realize the accuracy improvements associated with transitioning from static to low-frequency updating. In contrast, the red dashed line indicates the latency require- ment for high-frequency model updates (1 second). Under this criterion, only InstaTrain satisfies the requirement, while all other models fail to achieve the necessary speed for further accuracy enhancements from high-frequency updating.
Update Latency (9)
10a
10a
20-4
10-3
Carbon-Owe Oxide
Methane
Stock
Ammonla
Toluene
LF Update Reg. Update
GWN
MegaCRN
DLinear
FSNet
OneNet
InstaTrin
Figure Update latency of methods across datasets.
8

---

Published as a conference paper at ICLR 2025
Inference Latency
per Snapshot (5)
1
10-4
1
10
10:?
10-4
Carbon-Oxide
Methane
Stock
Ammonia
Toluene
McgaCRN
InstaTrain
MTGNN
Autoformer
ITransformer
PatchTST
NPGL
Figure 6: Average inference latency per snapshot.
In addition, the inference latency results in Fig. 6 show that InstaTrain also benefits from the ceptional efficiency brought by the electronic dynamical system, resulting in a similar latency with respect NPGL. On average, InstaTrain achieves ~4,000x speedup online learning compared to all baseline models across all tasks, while achieving a ~3,000x speedup in inference compared all baselines except NPGL. These results underscore the superior computational efficiency of the proposed Trainn
Power and Energy Consumption. Table presents hardware comparison. Compared with the A100 GPU, InstaTrain operates with ultra-low power consumption, requir- ing approximately 950mW. For a reasonable reference, we assume the average power for the A100 GPU used in this work is 250W. In terms of overall energy consumption, taking into account the exceptional speedups achieved training and inference across the selected datasets, InstaTraii achieves more than 10y greater energy efficiency compared to A100 GPUs. Compared to NPGL, despite higher power utilized, Insta supports ultra-fast fast online training, still resulting in orders-of-magnitude training energy reduction.
Table 2: Hardware comparison.
Hardware
Power
Fast Training
Fast Inference
A100
250W
No
No
NPGL
500mW
No
Yes
InstaTrain
950mW
Yes
Yes
Ablation Study. Update frequency plays crucial role in balancing model accuracy and computa- tional efficiency. To investigate its impact, vary the update interval of Insta'Train from every 50 snapshots to every 1,000 snapshots across all datasets. The results, presented in Table 3 demon- strate that generally, higher-frequency updating achieves better accuracy, reflected by lower MAE values. This finding further underscores the significance of InstaTrain, which enables extremely feggffreyeeccy online updates.
Table 3: Ablation study on update interval.
Update Interval
Methane Carbon-Oxide
Stock
Ammonia
Toluene
1000
8.28
9.22
1.20
3.72
4.67
500
7.26
8.46
0.84
2.41
3.57
100
6.79
7.05
0.68
1.36
1.86
50
6.73
7.02
0.67
1.33
1.85
RELATED WORK
Dynamical Systems in Machine Learning. Dynamical systems have gained increasing attention in the machine learning community due to their unique properties and their potential to enhance com- putational efficiency. These systems, which model the evolution states over time, offer an alterna- tive framework for solving complex optimization problems and machine learning tasks. However, the majority of studies showcasing their potential have been confined to relatively straightforward applications, primarily within the binary domain. For instance, the binary Ising model has been em- ployed to formulate optimization problems (Lucas) 2014)), which can be efficiently solved on Ising machines (Mohseni al. 2022). Additionally, several real-world problems, such satisfiability

---

Published as a conference paper at ICLR 2025
(SAT) problems (Sharma et al.] 2023a b). traffic congestion prediction (Pan et al. 2023 uplink MIMO detection (Singh et al_ 2023) and collaborative filtering (Liu et al. 2023b). have also been formulated and addressed using the binary Ising model. While these works have offered valuable insights of using dynamical systems for practical problem solving, they remain constrained by the inherent binary nature of the Ising model, which impedes further progress in real-valued applications in the real world.
Recent efforts, such as those by (Wu al. 2024), have sought to extend the traditional binary Ising model to accommodate real-valued nodes. Despite this advancement, the practical impact of these extensions has been limited. Firstly, the acceleration achieved by these approaches confined to the inference phase, leaving the primary bottleneck of training unaddressed. Secondly, the bene- fits of accelerated inference are diminished if the model static and cannot be promptly updated, especially in highly dynamic applications where patterns evolve rapidly. In summary, while dynam- ical systems have proven effective for certain machine learning problems, the slow training process continues to hinder their broader applicability.
Novel Computing Paradigms. The limitations of conventional von Neumann architectures have been bottleneck in modern computing. The separation between processing and memory units, commonly referred to as the "von Neumann bottleneck", limits the performance of CPUs and GPUs. With Moore's Law decelerating and conventional transistor scaling nearing its physical limits, there is an urgent need to explore alternative computing paradigms capable of overcoming these con- straints and sustaining performance gains.
In response to these challenges, researchers have investigated several novel computing paradigms. For instance, quantum computing, such as D-Wave (Bunyk et al., 2014), leverages quantum me- chanical effects to solve combinatorial optimization problems. Despite their impressive speedups in certain domains, these systems generally require cryogenic environments and are highly special- ized, which limits their scalability and broad applicability. Optical computing, such as coherent Ising machines (Yamamoto et al. 2017), exploits the inherent parallelism of light to perform com- putations at remarkable speeds. However, integration, stability, and reliability issues have hindered their widespread adoption. Inspired by the structure and functionality of biological neural networks, neuromorphic computing has led to the development of systems such TrueNorth (Akopyan al.) 2013) and Loihi (Davies et al. 2018 Although these systems offer substantial energy efficiency benefits for neural network applications, they often lack the programming flexibility and face chal- lenges in general-purpose computation beyond specific neural algorithms.
Within this diverse landscape, the CMOS compatible Ising machine BRIM (Afoakwa et al. 2021 has emerged as particularly compelling alternative. BRIM leverages the natural dynamics of elec- tronic circuits perform computation via natural annealing. Its design is based analog compo- nents such as capacitors and resistors, which allow the system to achieve ultra-fast computation by exploiting the inherent parallelism of physical processes. Moreover, BRIM operates extremely low power levels-typicall on the order of milliwatts-make sides, its compatibility with standard CMOS fabrication techniques facilitates seamless integration into existing semiconductor workflows, which further enhances its practical applicability.
it an energy-efficient solution. Be-
6 CONCLUSION
This paper presents Insta Train, a novel ultra-fast model learning approach for prediction tasks. By transforming the expensive training process into an iterative natural annealing process within dy- namical system, our method enables the model to rapidly adapt to the ever-changing correlations be- tween inputs and predictions, addressing the pressing need for agility and responsiveness in highly dynamic applications. This pioneering approach transcends the limitations of conventional methods and paves the way for a new era of ultra-fast and energy-efficient learning, empowering applications in domains characterized by high data volatility and stringent latency requirements. Further explo- rations could focus on incorporating advanced online learning strategies that are compatible with average, ~4,000 x speedup in training supporting microsecond-level model updates and a ~ 10 x reduction in training energy cost, along with lower MAE compared to baselines running on GPUs.
the hardware, which might yield better solutions. To highlight, InstaTTrain achieves,
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work is supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, in support of the MEERCAT Microelectronics Science Research Center, under Contract DE-ACO5-76RLO18330 This work is also supported by NSF under Award No. 2326494.
REFERENCES
Richard Afoakwa, Yiqiao Zhang, Uday Kumar Reddy Vengalam, Zeljko Ignjatovic, and Michael Huang. Brim: Bistable resistivelyccoupled ising machine. In 2021 IEEE International Symposium on High-Perfonee Performerf Computer Architecture (HPCA), pp. 749-760. IEEE, 2021.
Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. Truenorth: Design and tool flow of 65 mw million neuron programmable neurosynaptic chip. IEEE transactions on computer-aidel design of integrated circuits and systems, 34( 1537-1555, 2015.
Bogdan Bochenek and Zhigniew Ustrnul. Machine learning weather prediction and climate anal- yses applications and perspectives. Atmosphere, 13(2):180) 2022.
Paul Bunyk, Emile M Hoskinson, Mark Johnson, Elena Tolkacheva, Fabio Altomare, Andrew Berkley, Richard Harris, Jeremy Hilton, Trevor Lanting, Anthony Przybysz, et al. Archi- tectural considerations in the design superconducting quantum annealing processor. IEEE Transactions Applied Supercondactivitt. 24(4):1-10, 2014.
Xu Chen, Junshan Wang, and Kunqing Xie. Trafficstream: streaming traffic flow fore- casting framework based on graph neural networks and continual learning. arXiv preprint arXiv:2106.06272 2021.
Barry A Cipra. An introduction to the ising model. The American Mathematical Monthly, 94(10): 937-959, 1987.
Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: neuromorphic manycore processor with on-chip learning. leee Micro, 38(1):82 99, 2018.
Jordi Fonollosa, Irene Rodríguez-Luján, Marco Trincavelli, and Ramón Huerta. Dataset from chem- ical gas sensor array in turbulent wind tunnel. Data in Brief, 3:169-14 1144 2015a.
Jordi Fonollosa, Sadique Sheik, Ramón Huerta, and Santiago Marco. Reservoir computing compen- sates slow response of chemosensor arrays exposed to fast varying gas concentrations in continu- ous monitoring. Sensors and Actuators B: Chemical, 215:618 629, 2015b.
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural col- laborative filtering. In Proceedings of the 26th international conference on world wide web, pp- 173-182, 2017.
Geoffrey Hinton. The haward-forwait algorithm: Some preliminary investigations. arXiv preprint 13345:2212.1335. 2022.
Steven CH Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao. Online learning: A comprehensive survey. Neurocomputing, 459:249-288 2021.
Renhe Jiang, Zhaonan Wang, Jiawei Yong. Puneet Jeph, Quanjun Chen, Yasumasa Kobayashi, Xuan Song, Shintaro Fukushima, and Toyotaro Suzumura. Spatio-temporaa meta-graph learning for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8078- 8086, 2023.
Zahra Karevan and Johan AK Suykens. Transductive Istm for time-series prediction: An application to weather forecasting. Neural Networks, 125:1-9, 2020.

---

Published as a conference paper at ICLR 2025
Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: survey. Philosophical Transactions of the Royal Society A, 379(2119)) 2021.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint M.Xiv:3100006022. 2023a.
Zhuo Liu, Yunan Yang, Zhenyu Pan, Anshujit Sharma, Amit Hasan, Caiwen Ding, Ang Li, Michael Huang, and Tong Geng. Ising-cf: A pathbreaking collaborative filtering method through efficient ising machine learning. In 2023 60th ACM/IEEE Design Automation Conference (DAC), 2023b.
Andrew Lucas. Ising formulations of many up problems. Frontiers in physics, 2:5, 2014.
Naeimeh Mohseni, Peter L. McMahon, and Tim Byrnes. Ising machines as hardware solvers of combinatorial optimization problems. Nature Reviews Physics, 4(6):363-379, 2022.
Ali Momeni, Babak Rahmani, Benjamin Scellier, Logan G Wright, Peter L McMahon, Clara C Wanjura, Yuhang Li, Anas Skalli, Natalia G Berloff, Tatsuhiro Onodera, et al. Training of physical neural networks. arXiv preprint arXin::400.003332, 2024.
Nasdaq. Nasdaq data link.
Yuqi Nie, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730,, 2022.
Zhenyu Pan, Anshujit Sharma, Jerry Yao-Chieh Hu, Zhuo Liu, Ang Li, Han Liu, Michael Huang, and Tony Geng, Ising-traffic: Using ising machine learning to predict traffic congestion under uncertainty. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 9354-9363, 2023.
Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven CH Hoi. Learning fast and slow for online time series forecasting. arXiv preprint arXiv:2202. 16772, 2022.
Anshujit Sharma, Richard Afoakwa, Zeljko Ignjatovic, and Michael Huang. Increasing ising ma- chine capacity with multi-chip architectures. In Proceedings of the 49th Annual International Symposium on Computer Architecture, ISCA '22, pp. 508-521, New York, NY, USA, 2022. As- sociation for Computing Machinery. ISBN 9781 1784850386.
Anshujit Sharma, Matthew Burns, Andrew Hahn, and Michael Huang. Augmenting an electronic ising machine to effectively solve boolean satisfiability. Scientific Reports, 13(1):22858, 2023a.
Anshujit Sharma, Matthew Burns, and Michael C Huang. Combining cubic dynamical solvers with make/break heuristics to solve sat. 26th International Conference on Theory and Applications of Satisfiability Testing (SAT 2023). Schloss- Schloss-Dasstuhle Dagstuhl-Leibniz Zentrum für Informatik, 2023b.
Abhishek Kumar Singh, Ari Kapelyan, Davide Venturelli, and Kyle Jamieson. Uplink mimo de- tection using ising machines: A multi-stage ising approach. arXiv preprint arXiv:2304. 12830, 2023.
Qingsong Wen, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, Tieniu Tan, et al. Onenet: Enhancing time series forecasting models under concept drift by online ensembling. Advances in Neural Information Processing Systems, 36, 2024.
Chunshu Wu, Ruibing Song, Chuan Liu, Yunan Yang, Ang Li, Michael Huang, and Tong Geng. Extending power of nature from binary to real-valued graph learning in real world. In The Twelfth International Conference Learning Representations, 2024.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting, Advances in neural information processing systems, 34:22419-2242 2021.
Zonghan Wu. Shirui Pan, Guodong Long, Jing Jiang, and Chenggi Zhang. Graph wavenet for deep spatial-temporal graph modeling, 2019.
12

---

Published as a conference paper at ICLR 2025
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Con- necting the dots: Multivariate time series forecasting with graph neural networks, 2020.
Yoshihisa Yamamoto, Kazuyuki Aihara, Timothee Leleu, Ken-ichi Kawarabayashi, Satoshi Kako, Martin Fejer, Kyo Inoue, and Hiroki Takesue. Coherent ising machinees-ppical neural networks operating at the quantum limit. upj Quantum Information, 3(1):49, 2017.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, pp. 11121-11128, 2023.
Qian Zhang, Jie Lu, and Yaochu Jin, Artificial intelligence in recommender systems. Complex & Intelligent Systems, 7(1):439-455) 457, 2021.
13


Reviews:
[{"review_id": "fc8CTz6arr", "reviewer": "Reviewer_VHTd", "summary": "The manuscript proposes an improved electronic dynamical system---InstraTrain---to tackle the rapidly evolving nature of time series in a highly dynamic environment: it enables frequent model updates with microsecond-level intervals for real-world predictions.\n\nThe manuscript extends the extraordinary computational efficiency of the electronic dynamical system from inference to training, using two major components. It first formulates the time series prediction problem via the natural annealing procedure of a dynamical system, enabling performing efficient prediction. It further considers a physical realization of the proposed dynamical system.", "questions": "In lines 365-366, the authors claim that \"the more accurate high-frequency update is unattainable due to the sluggish training speed of NP-GL, GNNs, and even SOTA online-training methods\", as well as the \"'Not achievable' are due to slow training\" in the caption of Figure 5, the reviewer is a bit confused regarding these results, as they are not achievable. The authors are encouraged to explain this point.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The manuscript is well-written. It starts from the statistical mechanics in physics and models the time-series prediction problem as a dynamical system. It then leverages the natural annealing process of a dynamical system to enable ultra-fast model training. Furthermore, it redesigns the electronic dynamical system from a hardware aspect, enabling the self-training feature. The reviewer enjoys reading the paper, especially section 3.\n* The manuscript considers three types of baselines, covering 7 methods. It also uses an FEA software simulator to measure the accuracy and latency of the proposed method. Numerical results show that InstaTrain can achieve a significant speedup over all baseline models, while maintaining a $10^5$ reduction in energy consumption.", "weaknesses": "* The datasets used for the evaluation are not carefully explained, making the significance of the performance gain unclear. It would be great if the authors could elaborate on the dataset statistics. It would be great if the authors could include results on more challenging datasets.\n* One intuition of the paper is that the algorithm could converge to the equilibrium state of the dynamical system. It would be great if the authors could justify this point, either theoretical or empirical.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YOc1lXX5KP", "reviewer": "Reviewer_2k7C", "summary": "This paper presents a novel approach that transforms traditional model training into an iterative natural annealing process within an electronic dynamical system. The proposed approach operates through Iterative Natural Annealing-based Training (INAT), which performs training directly on an electronic dynamical system rather than on conventional processors. The key innovation lies in extending existing electronic dynamical system hardware from inference-only to support both training and inference on the same hardware. The authors claim significant improvements in training speed, energy efficiency, and prediction accuracy compared to state-of-the-art methods, particularly for time-series prediction tasks with rapidly evolving data distributions.", "questions": "See the weaknesses part.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The novel transformation of traditional training into a natural annealing process is interesting.\n- Hardware-algorithm co-design that extends existing electronic dynamical systems.\n- Clear theoretical foundation.\n- Well-structured presentation with clear problem motivation.", "weaknesses": "1. Some concerns about the experimental datasets. As far as I know, the three datasets used are time series data with relatively low-dimension. There is no discussion of scalability to larger, more complex datasets. How would the system perform on higher-dimensional data or more complex tasks? Are there theoretical or practical limitations?\n\n2. The experimental results of the proposed IntraTrain approach are based on CUDA-based software simulator rather than actual hardware implementation. Although the author claimed that the real hardware has been manufactured in the reproducibility part.\n\n3. In Table 1, the power consumption of InstaTrain (950mW) is actually higher than baseline BRIM (250mW) and NP-GL (260mW), could you provide detailed calculations for the claimed energy reduction?\n\n4. The paper compares performance with \"not achievable\" baseline results for high-frequency scenarios. Could you explain the methodology behind these comparisons since they are \"not achievable\"?\n\nI'd be happy to raise the score if the author could address my concerns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EM2I9HdjFJ", "reviewer": "Reviewer_hPWn", "summary": "This paper introduces InstaTrain, a novel approach for ultra-fast model training in highly dynamic environments where data distributions can change within microseconds. Traditional models, including online learning approaches, struggle to adapt quickly enough to such rapid changes. The authors transform the typically slow training process into an ultra-fast \"natural annealing\" process implemented on an electronic dynamical system. Their key contribution is twofold: they develop a new training algorithm called Iterative Natural Annealing based Training (INAT) that reformulates model training as an energy minimization problem, and they augment existing electronic hardware with parameter update modules to enable both training and inference on the same system. Evaluated on three high-frequency datasets including stock market and chemical sensor data, InstaTrain achieves dramatic improvements over state-of-the-art methods, demonstrating orders of magnitude faster training and inference speeds, significantly lower energy consumption, and better prediction accuracy. The work effectively bridges physics-inspired computing with machine learning to address the challenge of rapid distribution shifts in real-world applications.", "questions": "Can you add more comparison results in the form of tables?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Originality: The paper creatively combines statistical physics with machine learning in a novel way, transforming traditional model training into a physical annealing process. While both Ising models and online learning existed previously, extending electronic hardware to support microsecond-level training and inference together represents a genuine innovation. The work effectively removes the speed limitations that have constrained previous approaches to dynamic model adaptation.\nQuality: The technical work is rigorous, with thorough mathematical foundations and experimental validation. The evaluation compares against multiple baselines across several high-frequency datasets, demonstrating improvements in speed, energy efficiency, and accuracy. The inclusion of ablation studies further strengthens the empirical results.\nClarity: The paper effectively communicates complex concepts through clear writing.\nSignificance: The work has substantial practical impact by enabling model adaptation at previously impossible speeds while reducing energy costs by orders of magnitude. It also opens promising research directions by demonstrating how physics-based computing can be effectively combined with machine learning to overcome fundamental limitations of traditional approaches.", "weaknesses": "- The baseline comparison is incomplete. The paper should compare to more regular time series prediction models.\n- The presentation of the results in bar plot is not ideal. Tables of numbers are much better.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tz6yPlj0iF", "reviewer": "Reviewer_3cwj", "summary": "The paper introduces InstaTrain, a novel adaptive learning paradigm designed to operate in highly dynamic environments where data distributions shift rapidly. InstaTrain leverages a natural annealing process on an electronic dynamical system, enabling model training and updates at microsecond intervals. Experimental results across high-frequency datasets showcase substantial improvements in speed, energy efficiency, and prediction accuracy compared to state-of-the-art (SOTA) methods.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The application of natural annealing as a training mechanism in a dynamical system is novel, addressing limitations in existing adaptive models in terms of speed and energy consumption.\n\n2. InstaTrain demonstrates significant training speedups (4,135×) and energy reductions (105×) while achieving superior accuracy on high-frequency datasets. This supports the paper’s claims of real-time adaptability and efficiency.\n\n3. The integration of parameter update modules within the dynamical system facilitates simultaneous training and inference, a promising approach for handling rapid data shifts.", "weaknesses": "1. The model appears to primarily handle binary data (+1, -1). How does the proposed InstaTrain method address continuous data in time series applications?\n\n2. The paper omits several widely used time series datasets, including ETTm1, Electricity, Traffic, Exchange, and Weather.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)