Title: Analog Foundation Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: analog in-memory computing

robustness

large language models

foundation models
Abstract: Analog in-memory computing (AIMC) is a promising compute paradigm to improve speed and power efficiency of neural network inference beyond the limits of conventional von Neumann-based architectures. However, AIMC introduces fundamental challenges such as noisy computations and strict constraints on input and output quantization. Because of these constraints and imprecisions, off-the-shelf LLMs are not able to achieve 4-bit-level performance when deployed on AIMC-based hardware. While researchers previously investigated recovering this accuracy gap on small, mostly vision-based models, a generic method applicable to LLMs pre-trained on trillions of tokens does not yet exist. In this work, we introduce a general and scalable method to robustly adapt LLMs for execution on noisy, low-precision analog hardware. Our approach enables state-of-the-art models — including Phi-3-mini-4k-instruct and Llama-3.2-1B-Instruct — to retain performance comparable to 4-bit weight, 8-bit activation baselines, despite the presence of analog noise and quantization constraints. Additionally, we show that as a byproduct of our training methodology, analog foundation models can be quantized for inference on low-precision digital hardware. Finally, we show that our models also benefit from test-time compute scaling, showing better scaling behavior than models trained with 4-bit weight and 8-bit static input quantization. Our work bridges the gap between high-capacity LLMs and efficient analog hardware, offering a path toward energy-efficient foundation models. Code is available at [github.com/IBM/analog-foundation-models](https://github.com/IBM/analog-foundation-models).


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of deployment of LLM on Analog In Memory Computing (AIMC) hardware. The authors propose a methodology to make a pre-trained LLM robust to the non-idealities encountered in AIMC hardware (quantization, static ranges, noise). Inspired by LLM-QAT, the method is based on fine-tuning a pre-trained model using knowledge distillation from the full precision baseline, on a large amount of samples (20B tokens) generated by the LLM itself, using an existing analog hardware-aware training and evaluation methods and tools developed by previous work. The approach is extensively evaluated on numerous benchmarks, using 2 standard pre-trained LLM (3B parameters), showing better accuracy than LLM-QAT on the simulated noisy hardware.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths :\n\n-\tQuality : The work appears in a very mature form. The approach is evaluated on numerous benchmarks. The claims are in general well supported.\n- Clarity : The paper is very clear, easy-to-read, and gives sufficient information to reproduce the experiments.\n- Significance : The work is significant as it is the first work to investigate the scaling of LLM (of moderate size) for AIMC accelerators. It is an important step towards making possible the implementation of LLM on AIMC.\n- Originality : The originality lies in the combination of existing methods and the demonstration at large scale.\n\nWeaknesses :\n\n-\tQuality :\n1) In my opinion, the authors should spend more time on the strengths and weaknesses of their method depending on various factors (see questions below).\n2) I have doubts on several assumptions about the hardware model that should be clarified (see question 1).\n3) One of the claims is that the method shows a better scaling behavior under test-time compute scaling than models trained with 4-bit per-channel weight and 8-bit static input quantization. Although this is supported by an experiment, I find the difference in accuracy between the 2 not very important, moreover the authors do not provide at least intuitions on why this should be the case, therefore I would mitigate this claim.\n-\tSignificance : The re-usability of the work could be improved if the paper would study various sizes of models as well as various levels of hardware noise, to better understand the extent of the method. However, I understand that this could be left for future work.\n- Originality : \n1)\tThis works builds upon the work [52] developping the analog hardware-aware training method and toolkit. The link with this work should be clearly stated in the background and related works section.\n2)\tThe paper shows many property of their method by empirical demonstration, however there is rather limited explanation / intuitions on the why. The paper could be improved with more analysis (see my questions).\n\nQUESTIONS:\n1)\tAssumptions on the hardware model. I would consider increase my rating if the authors could clarify / justify these points.\n- The model based on [52] use weights in 16 bits : is this precision really achievable with the AIMC chip of [52] using 4 devices per weight ? In my understanding of AIMC, as the precision is represented by the number of encoded levels, I have difficulty to imagine having 2^16 distinct levels with only 4 devices. \n- The authors assume that weights with 0 value have no noise, while the model in LeGallo show that they do have noise (2% according to Fig 8). What is the justification ? I believe this can highly impact the results due to a large proportion of weights having zero value, and zero is very sensitive to noise (low SNR).\n- The authors aknowledge there exist various types of noise in AIMC, such as temporal variability or device-to-device. However in the paper, the type of noise modelled is not mentioned. I believe the noise is dynamic (sampled at each inference independently), and the authors do not model static device-to-device noise. Could the authors clarify this point ?\n2)\tAnalysis of the method. The authors show that weight clipping is more important than noise injection to increase robustness to noise. What is the explanation for that ? Moreover, the authors suggest that smaller weights are less robusts than bigger weights (in the experiments on additive gaussian noise). Why is that, knowing that bigger weights have a lower SNR (according to Fig 8) ? Isn’t in contradicton with the fact that their method is made to favor small range of weights ? I would consider increasing my rating if the authors could answer these questions.\n3)\tHow are the results dependent on factors such as LLM model size, quantization precision and noise level ? Indeed, all the paper is performed assuming these prameters are fixed, while I believe they play an important role. For instance, I expect bigger LLM to be more robust to noise and aggressive quantization. Moreover, the noise model used seem to have low level of noise. What would happen on AIMC with higher noise level, or with lower weights precision ? I would consider increasing my rating if the authors could provide hints on the sensitivity of their method to these parameters.\n4)\tHow is the method impacting the training speed and ressources ? I would expect noisy training to slower the learning. It seems so according to Tables 7 and 8 : while the Analog FM plateaus after 20B training tokens, the LLM-QAT plateaus already after 10B tokens. I think this should be highlighted in the limitations section.\n5)\tComparison with LLM-QAT for quantization on digital hardware. The fact that it yields better results without noise on digital quantization is surprising and deserves more explanation. Is it the noise injection, the clipping in the weight quantization, or the output static quantization that makes the model more robust to quantization error? Does this mean that QAT methods would improve by introducing these features ?\n\nLIMITATIONS:\nI think the authors could spend more time on the limitations of their work (e.g. see my questions 3 and 4), in particular explaining the sensitivity of their method to parameters such as model size, quantization precision and noise level, and highlighting potential limitations in terms of training ressources.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a  method for adapting pretrained LLMs to AIMC constraints without requiring original training data. Their approach maintains performance similar to 4-bit weight/8-bit activation quantization approaches on digital hardware, despite the presence of noise. They show this approach is robust with realistic noise/quantization scenarios and that the approach is robust on a range of benchmarks, and in fact outperforms quantized models with compute scaling at test time. This is a significant innovation as it clearly demonstrates the potential of AIMC in SOTA learning settings enabling major energy savings in settings of great interest.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper and results are strong and well motivated. This has clear impact and utility for future AIMC applications in a realm that is of great and direct interest. Experiments extend the SOTA for Analog models considerably. \n\nStrengths: the paper takes an important step toward extending the scope of AIMC hardware to billion parameter models for the first time, showcasing performance in a language setting. This bridges a major gap between extant literature on AIMC housed models (mostly CNNs/ RNNs with under 50m parameters) and models approaching a scale of interest in modern ML literature. The authors also do fairly comprehensive evaluation on 12 different benchmarks, so results  reported are clearly robust. The results show that n-best sampling can be leveraged toward superior performance on these models than standard quantized models, suggesting good tradeoffs for inference heavy use cases which are increasingly an area of great interest at scale. \n\nWeaknesses: Despite these points some limitations persist. While the researchers do address a few of these, namely the gap in performance compared to off the shelf models. There are certainly some areas where a paper such as this could benefit from further experiments. For example, it is fairly commonplace in research on foundation models in general to investigate scalability and robustness. While the training cost of further bridging the gap between these and actual foundation models is very likely prohibitively expensive, once can still assess metrics of scalability across a range of smaller model sizes. This would help to support the claim (as yet unproven) that such an approach can extend to the 100B parameter regime. Additionally, the researchers could have further explored non-gaussian or drift-based non-idealities. In my opinion insufficient attention was given to the various additional realistic sources of potential error one might encounter on chip. Furthermore, given the understandable lack of on-chip deployment, it would be interesting to provide more estimation of hardware metrics (throughput, on-chip latency, energy use), ADC/DAC overhead for example is an important consideration alongside noise derived therein. \n\nIn addition to further discussion of the complications encountered in real world on-chip deployment of such models, further ablations and theoretical investigation of for example, which aspects of the HWA training scheme (clipping vs noise injection or quantization) contribute most to robustness would be interesting and provide a basis for further investigations in this area to build upon.\n\nNevertheless, this paper is very strong empirically and provides a significant step forward in bridging the gap between extant AIMC results and SOTA ML.\n\nQUESTIONS:\nWhile the paper was strong, many of the contributions are more engineering than theoretical contributions. Stronger theoretical contribution would result in a higher score. For example, there is no theoretical justification given for clipping and noise synergy. Authors could develop this insight by drawing links to other literature e.g. quantile based regularization or stochastic smoothing , likewise there is a lack of robustness guarantees provided or analysis of error propagation (via. e.g. perturbation theory). Having guaranteed for regimes  where models are provably robust would make this paper significantly stronger. As mentioned before, some account of how number of training tokens, model size, or noise variance influence convergence would also boost this paper's overall quality.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to develop a general and scalable adaptation methodology called hardware-aware training for successfully running pre-trained LLMs on analog in-memory computing hardware with severe analog noise and quantization constraints. In so doing, it aims to minimize the performance degradation due to physical hardware limitations and achieve performance equivalent to that of a digital quantization model with 4-bit weights and 8-bit activations.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: This paper describes a specific and precise process for hardware-aware training, applying AIMC to LLM, and demonstrates that learning results based on Gaussian Noise show excellent performance under a given quantization environment on various benchmarks. It is thought that this could be a very interesting topic, considering the possibility of Analog in-memory computing in inference engines. In the field of application of AMIC, I believe this paper offers novelty in advancing low-power LLMs. \n\nWeaknesses: Although it claims to learn in consideration of the environment of AMIC, it is mostly a combination of existing algorithms and does not suggest solving new hypotheses, etc. This paper does not suggest a technique for fine-tuning AMIC, but in the case of actual Analog in-memory computing, it is greatly affected by various semiconductor processes and environments, so it seems that a fine-tuning approach is more realistic, and it is thought that a mention of this is necessary.\nIn AMIC, errors will occur due to Digital-to-Analog Converters and Analog-to-Digital Converters, as well as higher-dimensional errors resulting from semiconductor PVT characteristics. Therefore, a discussion is needed on how to address these issues.\n\nQUESTIONS:\nPlease refer to the above weaknesses. \nThere are various references in AMIC about what error characteristics AMIC has compared to FP32 model, but I think it is necessary to add to these. Can you add more references about that?\n\nLIMITATIONS:\nThere is no negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAnalog in-memory computing (AIMC) can deliver reduced energy consumption for LLMs but imposes 4 to 8-bit quantization and device-level noise that reduce LLM accuracy. The authors present a  hardware-aware training pipeline combining synthetic data distillation, learned static 8-bit input ranges, globally static 8-bit output quantization, iterative weight-clipping, and small additive per-channel Gaussian noise. Their method adapts pre-trained models (Phi-3-mini-4k-instruct and Llama-3.2-1B-Instruct) using <1% of the original pre-training tokens. After retraining on 20B synthetic tokens, the analog foundation models retain within ~4 % of their FP16 baselines and match state-of-the-art 4-bit-weight/8-bit-activation digital quantization when evaluated under realistic PCM AIMC noise across several reasoning, knowledge and safety benchmarks. They also quantize via round-to-nearest to run on conventional 4-bit digital accelerators, and show superior test-time compute-scaling on MATH-500 compared with QAT-quantized models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n- In contrast to several prior AIMC works that focus on retaining accuracy in CNNs on AIMC hardware, this paper tackles the much larger LLMs. The problem of reducing energy consumption in LLM workloads while retaining accuracy is especially relevant today as the energy costs of LLMs continue to skyrocket.\n- While prior works only focused on accuracy impact of ADCs (output quantization), this paper models the accuracy impact under weight noise, input DAC quantization, and output ADC quantization.\n- The authors evaluate many benchmarks, confirming that their method works for the given LLMs and PCM AIMC noise model. \n- The method achieves high accuracy for AIMC LLMs while retraining on only 1% of the training data\n- The retraining method yields LLM models that retain accuracy and are deployable on both analog and digital hardware. \n\n\nWeaknesses:\n\n\n- The paper assumes that the entire LLM dot product is performed in one-shot on an AIMC crossbar. LLM dot products may require as many as 8192 MAC operations. However, typical AIMC architectures usually support only ~128 MACs in one crossbar column. Hence, the MVM must be tiled and intermediate partial results are often accumulated in the digital domain using wide adders without any noise. For this reason, the dot product noise in the tiled setting may not be as severe as accumulating the entire dot product in the analog domain in one shot.\n- While the study demonstrates that billion-parameter LLMs can be made AIMC-ready, it still requires substantial retraining compute and leaves a noticeable accuracy gap on the hardest reasoning tasks, highlighting the need for lighter post-training schemes and further robustness research. The authors acknowledge this limitation.\n- Results are reported only for two relatively small LLMs (~1B parameters). It is unclear whether the method will scale to cutting-edge model sizes.\n- The method is not validated for popular ReRAM and MRAM AIMC architectures and noise models.\n\nQUESTIONS:\n- What is the method's impact on accuracy in the tiled AIMC setting?\n- Can the method be adapted to modeling and reducing the accuracy impact of other AIMC non-idealities such as sneak paths (undesired paths for current, parallel to the intended path)\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-08"}