Learning Machine Learning with Lorenz-96

Dhruv Balwada, Ryan Abernathey, Shantanu Acharya, Alistair Adcroft,
Judith Brener, V Balaji, Mohamed Aziz Bhouri, Joan Bruna, Mitch Bushuk,
Will Chapman, et al.

► To cite this version:

Dhruv Balwada, Ryan Abernathey, Shantanu Acharya, Alistair Adcroft, Judith Brener, et al. Learning Machine Learning with Lorenz-96. JOSE - Journal of Open Source Education, 2024, 7 (82), pp.241. 10.21105/jose.00241. hal-05410428

HAL Id: hal-05410428

https://hal.science/hal-05410428v1

Submitted on 11 Dec 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Learning Machine Learning with Lorenz-96

Dhruv Balwada D¹, Ryan Abernathey D¹, Shantanu Acharya D², Alistair Adcroft D³, Judith Brener D¹², V Balaji D¹⁵, Mohamed Aziz Bhouri D⁴, Joan Bruna D²,¹³, Mitch Bushuk D⁹, Will Chapman D¹², Alex Connolly D⁴, Julie Deshayes D¹⁰, Carlos Fernandez-Granda D²,¹³, Pierre Gentine D⁴,¹⁴, Anastasiia Gorbunova D⁶, Will Gregory D³, Arthur Guillaumin D⁵, Shubham Gupta D⁸, Marika Holland D¹², J Emmanuel Johnsson D⁶, Julien Le Sommer D⁶, Ziwei Li², Nora Loose D³, Feiyu Lu D⁹, Paul O’Gorman D¹¹, Pavel Perezhogin D², Brandon Reichl D⁹, Andrew Ross D², Aakash Sane D³, Sara Shamekh D⁴,², Tarun Verma D³, Janni Yuval D¹¹, Lorenzo Zampieri D⁷, Cheng Zhang D³, and Laure Zanna D²

1 Lamont Doherty Earth Observatory, Columbia University 2 Courant Institute of Mathematical Sciences, New York University 3 Program in Atmospheric and Oceanic Sciences, Princeton University 4 Earth and Environmental Engineering, Columbia University 5 Queen Mary University of London 6 Univ. Grenoble Alpes, CNRS, IRD, Grenoble INP, INRAE, IGE, 38000 Grenoble, France 7 Ocean Modeling and Data Assimilation Division, Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici - CMCC 8 Tandon School of Engineering, New York University 9 NOAA Geophysical Fluid Dynamics Laboratory 10 Sorbonne Universités, LOCEAN Laboratory, Paris, France 11 Department of Earth, Atmospheric, and Planetary Sciences, Massachusetts Institute of Technology 12 National Center for Atmospheric Research 13 Center for Data Science, New York University 14 Columbia Climate School, Columbia University 15 Schmidt Futures

DOI: 10.21105/jose.00241

Software

- Review ⚖

- Repository ⚖

- Archive ⚖

Submitted: 13 October 2023

Published: 26 December 2024

License

Authors of papers retain copyright
and release the work under a
Creative Commons Attribution 4.0
International License (CC BY 4.0).

Summary

Machine learning (ML) is a rapidly growing field that is starting to touch all aspects of our lives, and science is not immune to this. In fact, recent work in the field of scientific ML, i.e. combining ML and with conventional scientific problems, is leading to new breakthroughs in notoriously hard problems, which might have seemed too distant till a few years ago. One such age-old problem is that of turbulence closures in fluid flows. This closure or parameterization problem is particularly relevant for environmental fluids, which span a large range of scales from the size of the planet down to millimeters, and remains a big challenge in the way of improving forecasts of weather and projections of climate.

The climate system is composed of many interacting components (e.g., ocean, atmosphere, ice) and is described by complex nonlinear equations. To simulate, understand, and predict climate, these equations are solved numerically under a number of simplifications, therefore leading to errors. The errors result from numerics used to solve the equations and the lack of appropriate representations of processes occurring below the resolution of the climate model grid (i.e., sub-grid processes).

This book aims to conceptualize the problems associated with climate models within a
simple and computationally accessible framework, and show how some basic ML methods
can be used to approach these problems. We will introduce the readers to climate modeling
using a simple tool, the Lorenz (1995) (L96) two-timescale model. We discuss the numerical
aspects of the L96 model, the approximate representation of sub-grid processes (known
as parameterizations or closures), and simple data assimilation problems (a data-model
fusion method). We then use the L96 results to demonstrate how to learn sub-grid
parameterizations from data with ML, and then test the parameterizations offline (apriori) and online (aposteriori), with a focus on the interpretability of the results. This book is written primarily for climate scientists and physicists, who are looking for a gentle introduction to how they can incorporate ML into their work. However, it may also help ML scientists learn about the parameterization problem in a framework that is relatively simple to understand and use.

The material in this Jupyter book is presented over five sections. The first section, Lorenz 96 and General Circulations Models, describes the Lorenz-96 model and how it can work as a simple analog to much more complex general circulation models used for simulating ocean and atmosphere dynamics. This section also introduces the essence of the parameterization or closure problem. In the second section, Neural Networks with Lorenz-96, we introduce the basics of ML, how fully connected neural networks can be used to approach the parameterization task, and how these neural networks can be optimized and interpreted. No model, even the well parameterized ones, is perfect, and the way we keep computer models close to reality is by guiding them with the help of observational data. This task is referred to as data assimilation, and is introduced in the third section, Data Assimilation with Lorenz-96. Here, we use the L96 model to quickly introduce the concepts from data assimilation, and show how ML can be used to learn data assimilation increments to help reduce model biases. While neural networks can be great functional approximators, they are usually quite opaque, and it is hard to figure out exactly what they have learnt. Equation discovery is a class of ML techniques that tries to estimate the function in terms of an equation rather than as a set of weights for a neural network. This approach produces a result that is far more interpretable, and can potentially even help discover novel physics. These techniques are presented in the fourth section, Equation Discovery with Lorenz-96. Finally, we describe a few more ML in section five, Other ML approaches for Lorenz-96, with the acknowledgment that there are many more techniques in the fast-growing ML and scientific ML literature and we have no intention of providing a comprehensive summary of the field.

The book was created by and as part of M2LInES, an international collaboration supported by Schmidt Futures, to improve climate models with scientific ML. The original goal for these notebooks in this Jupyter book was for our team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, ML, data assimilation, uncertainty quantification) and to develop new ideas. This was done as a series of tutorials, each of which was led by a few team members and occurred with a frequency of roughly once every 2 weeks for about 6-7 months. This Jupyter book is a collection of the notebooks used during these tutorials, which have only slightly been edited for continuity and clarity. Ultimately, we are happy to share these resources with the scientific community to introduce our research ideas and foster the use of ML techniques for tackling climate science problems.

## Statement of Need

Parameterization of sub-grid processes is a major challenge in climate modeling. The details of this problem may often be very context dependent (Christensen & Zanna (2022)), but much can be learned by addressing the issue in a general and simpler sense. Also, a general approach allows non-domain experts, e.g. ML researchers, to engage and contribute more meaningfully. This JupyterBook aims to achieve this target with the help of a simple dynamical system model - Lorenz 96, such that the reader is introduced to the basic concepts with minimal superfluous complexity. It is possible to extend the concepts that are presented here to other dynamical systems, and even to more complex parameterization tasks (some examples can be found at https://m2lines.github.io/publications/), and we hope that researchers and learners aiming to do this find the concepts presented here as a useful stepping stone in this pursuit.
As described above, these notebooks were originally created to introduce non-domain experts to ideas from the parameterization aspects of climate modeling and how ML could be used to potentially address these. Now they have been adapted to act as a pedagogical tool for self-learning, be used as a reference manual, or for teaching some modules in an introductory class on ML. The book is organized in sections that are relatively independent; with the exception that the first section provides a general overview to the parameterization problem in climate models. Each notebook covers material that can be discussed in roughly an hour-long lecture, and sections can be mixed and matched or ordered as needed depending on the overall learning objectives.

## Acknowledgements

This work is supported by the generosity of Eric and Wendy Schmidt by recommendation of Schmidt Futures, as part of its Virtual Earth System Research Institute (VESRI). MAB acknowledges support from National Science Foundation's AGS-PRF Fellowship Award (AGS2218197).

## References

* Christensen, H., & Zanna, L. (2022). *Parametrization in weather and climate models*. https://doi.org/10.1093/acrefore/9780190228620.013.826

* Lorenz, E. N. (1995). Predictability: A problem partly solved. *Seminar on Predictability*, 1, 1–18. https://doi.org/10.1017/cbo9780511617652.004