DS-LLM: Leveraging Dynamical Systems to Enhance Both Training and Inference of Large Language Models

Authors: Ruibing Song, Chuan Liu, Chunshu Wu, Ang Li, Dongfang Liu, Ying Nian Wu, Tong Geng
Primary area: foundation or frontier models, including LLMs
Keywords: Large Language Model;  nature-powered computing; dynamic physical system

Abstract:
The training of large language models (LLMs) faces significant computational cost challenges, limiting their scalability toward artificial general intelligence (AGI) and broader adoption. With model sizes doubling approximately every 3.4 months and training costs escalating from 64 million USD for GPT-4 in 2020 to 191 million USD for Gemini Ultra in 2023, the economic burden has become unsustainable. While techniques such as quantization offer incremental improvements, they fail to address the fundamental computational bottleneck. In this work, we introduce DS-LLM, a novel framework that leverages dynamical system (DS)-based machines, which exploit Natural Annealing to rapidly converge to minimal energy states, yielding substantial efficiency gains. Unlike traditional methods, DS-LLM maps LLM components to optimization problems solvable via Hamiltonian configurations and utilizes continuous electric current flow in DS-machines for hardware-native gradient descent during training. We mathematically demonstrate the equivalence between conventional LLMs and DS-LLMs and present a method for transforming a trained LLM into a DS-LLM. Experimental evaluations across multiple model sizes demonstrate orders-of-magnitude improvements in speed and energy efficiency for both training and inference while maintaining consistent accuracy. Additionally, we provide an in-depth analysis of the challenges and potential solutions associated with this emerging computing paradigm, aiming to lay a solid foundation for future research.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730232112128

thread_type:
official_review

<thread_object>
creator: reviewer_hawr

<content>
summary:
This paper proposes a novel approach to training and inference for large language models on dynamic systems, a new type of computing mechanism. Inference is facilitated through Hamiltonian configurations of the dynamic system machines, whereas training necessitates additional analog hardware for gradient computations. The authors assert that their approach yields substantial benefits in terms of speedup and energy reduction compared to traditional GPU-based methods.

strengths:
(1) New computing paradigm. The authors explore the use of DS machines, a novel computing paradigm for both training and inference of large language models, a timely contribution given the substantial resources currently consumed by LLM pretraining.

(2) Significant gains. The authors claim 1000x speedups and 100,000x energy reduction for training while maintaining consistent model accuracy.

(3) Mathematically rigorous. Although the reviewer may not have understood every detail, the problem formulation of LLM inference into Hamiltonian and Natural Annealing in dynamical systems appears mathematically solid and promising.

weaknesses:
(1) **Lack of clear differentiation from prior work**: One of the major contributions of using dynamical systems for machine learning appears to be extremely similar to prior work [1]. The reviewer struggles to understand the significant differences and contributions of this work compared to [1], particularly given that some graph neural networks in [1] also employ attention mechanisms. A clearer explanation of the novelty and significance of this work is necessary.

(2) **Inadequate consideration of circuit non-idealities**: The training process relies on specialized analog circuits to achieve gradient calculations and backpropagation. However, analog circuits are prone to non-idealities such as thermal noise and non-linearity [2]. The authors seem to overlook these non-idealities and their potential impact on model accuracy. Furthermore, the exponential approximation using Taylor expansion in Section 3.3 lacks justification, and the choice of a 3rd-order expansion is not fully explained. The authors should provide more insight into how this choice affects model accuracy and hardware requirements.

(3) **Limited analysis of model scalability**: The authors conduct analysis on model sizes of 124M, 355M, and 1.3B, but recent works have demonstrated the importance of scaling up model sizes, such as the 405B model in llama3. Can the authors provide more theoretical analysis or insight into how the speedups and energy consumption scale with the size of the models?

(4) **Unfair/Unclear comparisons**: The authors compare their energy calculations to A100 GPUs (800W), but the basis for the claimed 1000x speedup is unclear. Specifically, did the authors consider the latest techniques for fully utilizing GPU resources, such as vLLM[3]? Moreover, was batched inference used, considering that only 1.3B models were tested? It is unclear how/why this was not done, as four 40GB A100s would not be required for 124M models, for example. Especially in terms of LLM inference benchmarking, what is the comparision results on meaningful metrics, such as Time to First Token (TTFT),Token Generation Rate , and Energy in Token/KWh [4,5].?


References

[1] EXTENDING POWER OF NATURE FROM BINARY TO REAL-VALUED GRAPH LEARNING IN REAL WORLD

[2] Design of Analog CMOS Integrated Circuits

[3] Efficient Memory Management for Large Language Model Serving with PagedAttention

[4] https://www.anyscale.com/blog/comparing-llm-performance-introducing-the-open-source-leaderboard-for-llm

[5] https://huggingface.co/spaces/optimum/llm-perf-leaderboard

questions:
**Regarding novelty and differentiation from prior work**

1. Can you provide a detailed comparison of your work with [1], highlighting the specific contributions and differences in your approach?
How does your use of dynamical systems for machine learning improve upon or depart from existing methods, particularly those that also employ attention mechanisms?
2. What are the key advantages of your approach over prior work (and other emerging compute alternatives such as compute-in-memory), and how do you demonstrate these advantages in your experiments?

**Regarding circuit non-idealities**

1. How do you plan to address the potential impact of thermal noise and non-linearity in analog circuits on model accuracy?

2. Can you provide a justification for the exponential approximation using Taylor expansion in Section 3.3, and explain why a 3rd-order expansion was chosen?How does the choice of Taylor expansion order affect model accuracy and hardware requirements, and can you provide experimental or theoretical evidence to support your choice?

**Regarding model scalability**

1. Can you provide a theoretical analysis of how your approach scales with increasing model sizes, such as those demonstrated in recent works (e.g., 405B model in llama3)?How do you expect the speedups and energy consumption to change as model sizes increase, and what are the implications for hardware requirements? Can you provide additional experiments or simulations to demonstrate the scalability of your approach?

**Regarding comparisons and metrics**

1. Can you clarify the basis for the claimed 1000x speedup, and provide more details on the experimental setup and metrics used?

2. Did you consider using the latest techniques for fully utilizing GPU resources, such as vLLM[3], and if not, why not? Why was batched inference not used, particularly for smaller models (e.g., 124M), and how would this affect the energy calculations and comparisons with A100 GPUs?

3. In terms of LLM inference/training benchmarking, what is the comparision results on meaningful metrics, such as Time to First Token (TTFT), Token Generation Rate, and Energy (Token/KWh).

4. Can the authors also compare with (to a rough level) on dynamical systems with other emerging computing candidates (as introduced in the second paragraph of the introduction), such as quantum computing, optical computing, computing in memory etc? 

**Minor recommendations**: The abstract of the paper consists of 3 paragraphs and is quite long, can the authors abbreviate the abstract into a single paragraph to breifly explain the papers novelty and contributions?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730366197329

thread_type:
official_review

<thread_object>
creator: reviewer_v14i

<content>
summary:
This paper proposes a novel approach for constructing large language models (LLMs) using analog dynamic system (DS) machines, offering an alternative to conventional CMOS-based hardware. The authors present two methods for mapping existing Transformer-based models onto DS machines, thoroughly analyzing key components within Transformers to facilitate a complete transformation compatible with DS architecture. The proposed DS-based models achieve significantly faster training and inference times, along with notably reduced energy consumption compared to traditional CMOS-based systems.

strengths:
* The paper introduces a creative application of analog DS machines for LLM construction, advancing research on energy-efficient AI model deployment.
* By analyzing and adapting key Transformer components for DS architecture, the authors provide a clear methodology for how LLMs can be mapped effectively, which contributes to the feasibility of DS-based machine learning.
* The substantial reductions in training and inference time and energy consumption demonstrate the practical advantages of DS machines over CMOS for LLMs.

weaknesses:
* The paper could benefit from a broader benchmarking analysis that includes different LLM sizes. 
* While DS machines are promising in terms of efficiency, their limitations (e.g., stability and precision constraints) are not fully discussed.

questions:
* As LLMs scale up, are additional considerations or modifications necessary for DS machines to handle larger model architectures effectively?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730689306971

thread_type:
official_review

<thread_object>
creator: reviewer_kykq

<content>
summary:
This work proposes a method for mapping LLM inference and training to Ising machines (which the authors refer to as "dynamical system machines"). They build upon existing work that introduces a room temperature Ising machine built with CMOS technology and work that maps graph learning problems to these substrates. The resulting DS-LLM is orders of magnitude more energy efficient than conventional hardware.

strengths:
The problem tackled in this paper—the energy consumption of LLMs—is both timely and well-motivated. The approach taken by the authors is unique and the results appear quite significant. The methodology is clearly explained despite covering a broad range of background fields.

weaknesses:
- Evaluation of inference improvements focus on a comparison against GPUs instead of low-precision, quantized implementations for edge devices. For example, comparing against low-precision CPU inference implementations (https://arxiv.org/pdf/2311.00502) or novel edge devices (https://arxiv.org/pdf/2409.15654, https://arxiv.org/pdf/2305.18691)
- There is no discussion of the possible limitations which is extremely relevant for ICLR readers as they may not be familiar with the hardware used in this paper. In particular, focus on scalability issues (like described in Question 1) or model flexibility issues (are there operations that would be difficult to map to the proposed substrate, and are all existing transformer variants mappable?).
- Small suggestion: the introduction would be clearer if the authors removed hyperbolic/unnecessary language surrounding the capabilities of natural annealing
    - e.g. Line 067: "Recently, Dynamical-System-based (DS) machines have emerged as a promising solution of such, which leverages nature itself as a computer!"
    - Paragraph on Lines 077-090 can be shortened as it mostly restates the previous paragraph with longer phrasing. Instead, simply state that no existing framework maps LLMs to DS machines, the prior work is specific to graph learning, and the difference in computation and scale between LLMs and graph learning poses a challenge for finding such a mapping.

questions:
1. The prior work on NP-GL utilized relatively small graphs, resulting in circuits with 100s of nodes. This is not the case for a transformer model where the number of nodes is much larger. Is it possible to manufacture Ising machines at this scale? If not, would we need to rely on a multi-chip solution?
2. The description of back-propagation implemented on the Ising machine is not clear. What is computed by the auxiliary circuit and what is computed by the Ising machine? Is there a separate "reverse" network for computing gradients? If so, how are the reverse and forward networks kept aligned (typically referred to as the "weight transport problem" in brain-inspired learning settings)?
3. Does the time and energy evaluation refer to the total training / inference time over the full dataset or a single sample?
4. Does your proposed hardware solution perform training with batches? If not, wouldn't the total training time be prohibitive?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730818131109

thread_type:
official_review

<thread_object>
creator: reviewer_xu44

<content>
summary:
The paper introduces DS-LLM, a novel framework that employs Dynamical System (DS)-based machines to optimize the training and inference processes for Large Language Models (LLMs). By incorporating an electrodynamics-based model and a Natural Annealing process, DS-LLM dramatically reduces the computational costs associated with these models. The approach is mathematically and empirically validated, demonstrating significant speedups and energy reductions without sacrificing accuracy compared to conventional LLMs.

strengths:
1. The paper pioneers the use of DS machines in the context of LLMs, leveraging their inherent Natural Annealing processes to optimize both training and inference, which is both novel and potentially transformative for the field.
2. The reported results show a 1,090× speedup and a 102,559× reduction in energy during training, along with a 127× speedup and a 37,545× reduction during inference. These improvements are substantial and address critical cost and efficiency barriers in LLM deployment.
3. The evaluation is thorough, with tests across multiple models and datasets demonstrating that DS-LLM achieves comparable accuracy to traditional LLMs. This thorough testing helps validate the practicality of the proposed approach.

weaknesses:
1. While the paper demonstrates impressive theoretical and simulated results, the practical implementation of such systems might be challenging due to specialized hardware requirements. To better understand the feasibility, could you discuss specific hardware requirements, potential adoption challenges, and necessary adaptations to existing ML infrastructures for implementing DS-LLM?
2. The paper discusses the theoretical and simulated benefits of DS machines but lacks concrete examples of these machines being implemented in real-world scenarios.

questions:
1. Can you provide specific examples of Dynamical System-based machines being used in real-world applications? 
2. What are the main challenges in integrating DS machines with the existing computational infrastructures used for LLMs?
3. Could you provide more specifics about the Finite Element Analysis (FEA) software emulator used for the DS machine simulations? 
4. How closely do the results from the FEA simulations align with what you would expect from real hardware implementations? Are there any known discrepancies or performance variations between the simulated outcomes and actual hardware tests?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732179244840

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Thank you to all reviewers

comment:
### We sincerely thank all the reviewers for their insightful and constructive feedback, which has greatly helped us improve the quality of this paper. We have uploaded a revised manuscript and labeled it corresponding to reviewer questions. Due to space constraints, most detailed responses to the comments are added in the Appendix.

### Here is a summary of the major revisions:
1.	Comprehensive Discussion on DS Hardware: We have included an extensive discussion in the Appendix covering aspects such as stability, feasibility, robustness, scalability, non-idealities, and potential challenges, along with future exploration directions.
2.	Extended Model Evaluation: We have added evaluations on larger models, up to 7B, to better demonstrate the scalability and effectiveness of our solution.
3.	Clarifications and Experimental Details: More details have been provided to clarify key concepts and experimental setups for better transparency.
4.	Fairer and Clearer Comparisons: Additional experimental results have been added to ensure fair and clear comparisons.

### We also want to emphasize that this work, as a cutting-edge exploration, introduces a promising solution to address the growing computational demands of LLMs. While DS machines are not yet mature for commercial applications, they hold immense potential to become a critical component in next generation computing paradigms, fundamentally tackling the challenges of LLM computing.

### Once again, we deeply appreciate the reviewers for their time and invaluable efforts. Please feel free to reach out if you have any further questions regarding the revisions. Thank you!
</content>
</thread_object>

</thread>
<thread>
creation date:
1733196181897

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Part2 of Summary of Rebuttal and Discussion

comment:
We would like to emphasize the contributions of DS-LLM as recognized by reviewers:

1. *Introducing a new computing paradigm to LLM*: DS-LLM is the first to losslessly equate neural network (NN) kernels to the natural annealing processes of DS machines, transforming expensive and explicitly programmed NN computation into significantly more efficient nature-powered processes. By achieving this lossless equivalence, DS-LLM extends the computing power of DS machine to complex NN models including large language models (LLMs). 
2. *On-device training support*: DS-LLM is the first to enable on-device training on a DS machine, demonstrating that DS machines have the ability to perform not only inference but also efficient training, bringing the computing power of DS machine to LLM training. In contrast, prior works restricted DS machines to inference tasks, requiring all parameter training still to be conducted on GPUs. 
3. *Significantly reducing LLM computing cost*: DS-LLM achieves orders of magnitude speedup and energy reduction over traditional GPU baselines, showcasing the impressive potential to significantly reduce the cost of LLM computing on both inference and training. 

We extend our sincere gratitude to the reviewers for their invaluable feedback, which has greatly improved the quality of this work. We hope DS-LLM provides meaningful insights and serves as a foundation for further exploration in this promising area. 

Sincerely,

Authors
</content>
</thread_object>

</thread>
<thread>
creation date:
1733196263909

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Part1 of Summary of Rebuttal and Discussion

comment:
### Dear Area Chair and Reviewers,

We deeply appreciate the constructive feedback and insightful suggestions provided throughout the review process. Your thoughtful comments have been instrumental in strengthening our work. We are particularly encouraged by the positive consensus across all reviewers during the discussion phase: 

1. Reviewer xU44 retained their positive score. Their comments are very insightful and constructive, and we have addressed each point thoroughly. Our responses have been incorporated into the revised manuscript. 
2. Reviewer kykQ raised their rating, recognizing the enhancements made to our introduction, the comprehensive analysis of hardware scalability, and the comparison with realistic baselines. They also highlighted the potential importance of this work for ongoing research in applying DS machines to LLMs. 
3. Reviewer v14i maintained their positive score, appreciating our efforts to address concerns about stability, precision, and scalability. They commended the work as intriguing and novel, expressing enthusiasm about its potential to inspire further research. 
4. Reviewer HAWr significantly increased their score, acknowledging the improved discussions on limitations, enhanced baselines and metrics, and more detailed analyses of scalability and nonidealities. They noted that this work is both interesting and likely the first to tailor DS machines specifically for LLMs.

As the discussion concludes, we summarize below the key revisions made in response to your feedback:
### 1. Enhanced Related Works
To address Reviewer xU44’s request for more real-world context, we added concrete examples in the introduction illustrating DS machines’ applications to practical problems.

In response to Reviewer HAWr, we expanded the introduction to differentiate our approach from prior work. In Appendix Sec. A.1, we elaborated on the unique advantages of DS-LLM compared to other emerging computing technologies, providing a nuanced perspective on its contributions to the field.

### 2. Discussion on Hardware limitations
Following Reviewer xU44’s recommendation, we analysis and discuss the feasibility of DS machines in a new added discussion section in Appendix Sec. A.1, highlighting the fundamental feasibility of DS machines on integration with existing computing infrastructures. 

In response to Reviewer kykQ’s request, we included a new section in Appendix Sec. A.1 discussing the scalability and model flexibility issues. We provided supportive reference for the scalability potential of DS machines and highlighted the inherent model flexibility by theoretically analysis. 

Reviewer v14i raised concerns on limitations of DS machines specifically on stability, precision, and scalability issues. We provided a new experiment demonstrating DS machines' high stability and discussed precision and scalability challenges and solutions in detail within Appendix Sec. A.1. 

Following Reviewer HAWr’s request, we revised the abstract, introduction, and evaluation section to reflect the potential challenges on DS machines and solutions, providing a reliable and practical analysis of the approach. We also added a new discussion section as Sec. 5 to highlight the challenges and solutions on physical practicality and system integration of DS machines, and we addressed other discussions on stability, non-idealities, and trade-offs on Taylor expansion in Appendix Sec A.1. Additionally, we provided a better visualization with theoretical projections on scaling curve on training/inference performance in Fig. 7, with detailed cost of scaling in Table. 3, showing a comprehensive analysis of the scalability of our approach. 

### 3. Improvements on model evaluation
Reviewer kykQ suggested benchmarking against low-precision, quantized implementations for edge devices. To this end, we included a comparison with low-precision CPUs and emerging edge devices, showing that DS machines outperform these solutions.

As requested by Reviewer v14i, we included evaluations on two larger-sized models including OPT-2.7B and Llama2-7B to better assess our solution’s performance. The results in Table. 1 and Table. 2 demonstrate that DS-LLM continues to offer comparable accuracy and significant speedup & energy reduction. 

Following the suggestion from Reviewer HAWr, we have incorporated vLLM to enhance GPU inference performance and updated Table. 2 with metrics such as token/s and token/KWh to provide a fairer and more comprehensive comparison. 

### 4. Clarification on experimental details
Addressing Reviewer xU44’s request, we expanded the experimental setup section to include more details on the FEA emulator and its stability. 

Addressing concerns from Reviewer uXT1, we added detailed explanations of time and energy evaluations and clarified batch training capabilities. 

Reviewer KNeJ sought clarification on experimental setup and metrics. We have incorporated these details and updated the metrics based on their suggestions.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This work introduces DS-LLM, a framework that employs Dynamical System (DS)-based machines to optimize Large Language Models (LLMs), significantly reducing computational costs through an electrodynamics model and Natural Annealing process. The approach achieves substantial speedups and energy reductions without compromising accuracy. The paper demonstrates the potential of DS machines to transform LLM training and inference, offering a promising path toward more efficient and sustainable AI model deployment.
All the reviewers give positive comments on this work, thus I recommend the acceptance of this work.