ICML 2025

BARNN: A Bayesian Autoregressive and Recurrent Neural Network
Dario Coscia 1 2 Max Welling 2 Nicola Demo 3 Gianluigi Rozza 1
Abstract
Autoregressive and recurrent networks have
achieved remarkable progress across various
fields, from weather forecasting to molecular gen-
eration and Large Language Models. Despite
their strong predictive capabilities, these mod-
els lack a rigorous framework for addressing un-
certainty, which is key in scientific applications
such as PDE solving, molecular generation and
Machine Learning Force Fields. To address this
shortcoming we present BARNN: a variational
Bayesian Autoregressive and Recurrent Neural
Network. BARNNs aim to provide a principled
way to turn any autoregressive or recurrent model
into its Bayesian version. BARNN is based on
the variational dropout method, allowing to apply
it to large recurrent neural networks as well. We
also introduce a temporal version of the “Vari-
ational Mixtures of Posteriors” prior (tVAMP-
prior) to make Bayesian inference efficient and
well-calibrated. Extensive experiments on PDE
modelling and molecular generation demonstrate
that BARNN not only achieves comparable or
superior accuracy compared to existing methods,
but also excels in uncertainty quantification and
modelling long-range dependencies.
1. Introduction
Autoregressive and recurrent models have demonstrated
impressive advancements in different disciplines, from
weather prediction (Bodnar et al., 2024; Lam et al., 2022),
to molecules generation (Shi et al., 2020; Simm et al., 2020)
and Large Language Models (LLMs) (Bengio et al., 2000;
Vaswani, 2017; Radford et al., 2019). However, while au-
toregressive models show strong predictive abilities, they are
also prone to overfitting to the specific tasks they are trained
1Mathematics Area, International School of Advanced Studies,
Italy 2Informatics Institute, University of Amsterdam, The Nether-
lands 3FAST Computing Srl, Italy. Correspondence to: Dario
Coscia <dario.coscia@sissa.it>...