# New Frontiers in Associative Memories

## About This Workshop
Associative Memory (AM) is a core notion in psychology responsible for our ability to link people's names to their faces and to remember the smell of a strawberry when we see one. Mathematical formalizations of AM date back to the 1960s-1980s [...] . For instance, the celebrated Hopfield Networks of Associative Memory have made a significant impact on the communities of machine learning researchers, neuroscientists, and physicists. A recent surge of novel theoretical and practical developments [...] have reinvigorated this seemingly established field and placed it in the spotlight of modern ideas in deep learning [...] . and contemporary artificial network models of the brain [...] (see also this Quanta Magazine Article), culminating in the 2024 Nobel Prize in Physics "for foundational discoveries and inventions that enable machine learning with artificial neural networks".

However, there still remain significant gaps between the language, methods, and ideas that are used in the theoretical work pertaining to this topic and mainstream machine learning literature. The main goal of our workshop is to bring together key researchers and developers working on AM from the perspectives of machine learning, computational neuroscience, statistical physics, and software engineering, to build upon the first iteration of this workshop at NeurIPS 2023 towards closing the gaps and converging to a common language, methods, and ideas.

We would consider our workshop a success if it sparks enough interest from the communities of AM theorists, LLM practitioners, computational neuroscientists, and software developers, which are largely disjoint, to work together towards understanding the language and methods used by each of the sub-fields. We hope that this convergence will lead to efforts towards the development of novel architectures and algorithms uniquely suitable for Associative Memory networks, and to the integration of these modules into modern large scale AI systems.

Recent developments have opened up a New Frontier for Associative Memory and Hopfield Networks. The announcement of the Nobel Prize is Physics 2024 has further placed this area of research in the spotlight. We believe that 2025 is the right time to bring this topic to ICLR.

## Scope and Related Work
Associative memory is defined as a network that can link a set of features into high-dimensional vectors, called memories. Prompted by a large enough subset of features taken from one memory, an animal or an AI network with an associative memory can retrieve the rest of the features belonging to that memory. The diverse human cognitive abilities which involve making appropriate responses to stimulus patterns can often be understood as the operation of an associative memory, with the memories often being distillations and consolidations of multiple experiences rather than merely corresponding to a single event.

In the world of artificial neural networks a canonical mathematical model of this phenomenon is the Hopfield network. Although often narrowly viewed as a model that can store and retrieve predefined verbatim memories of past events, its contemporary variants make it possible to store consolidated memories turning individual experiences into useful representations of the training data. Such modern variants are often trained using the backpropagation algorithm and often benefit from superior memory storage properties. Contemporary Hopfield networks can be used as submodules in larger AI networks solving a diverse set of tasks. The goal of this workshop is to discuss the existing and emerging developments of these ideas. The research topics of interest at this workshop include (but are not limited to):

- Novel architectures for associative memory, Hopfield Networks, Dense Associative Memories, and related models (e.g., Krotov & Hopfield (2016), Demircigil et al. (2017), Ramsauer et al. (2020), Millidge et al. (2022), Krotov (2021), Hoover et al. (2023), Zhang et al. (2024), Krotov (2023), Dohmatob (2023))
- Hybrid memory augmented architectures, e.g., memory augmented Transformers and RNNs, networks with fast weight updates (e.g., Rae et al. (2019), Wu et al. (2022), Wang et al. (2023), He et al. (2023), Wang et al. (2024), Bulatov et al. (2024))
Energy-based models and their applications (e.g., Hoover et al. (2023a), Hoover et al. (2022), Ota & Taki (2023))
Associative Memory and Diffusion Models (e.g., Hoover et al. (2023b), Ambrogioni (2024), Pham et al. (2024), Achilli et al. (2024), Ambrogioni (2023), Biroli et al. (2024))
- Training algorithms for energy-based, or memory-based architectures (e.g., Du & Mordatch (2019), Scellier & Bengio (2017), Goemaere et al. (2023))
- The connection between associative memory and neuroscience (both insights from neuroscience for better AI, and AI-inspired neurobiological work) (e.g., Krotov & Hopfield (2021), Whittington et al. (2021), Sharma et al. (2022), Tyulmankov et al. (2023), Kozachkov et al. (2023), Kozachkov et al. (2023), Spens & Burgess (2023))
- Kernel methods and associative memories (e.g., Choromanski et al. (2020), Hoover et al. (2024), Hu et al. (2024), Iatropoulos et al. (2022))
- Theoretical properties of associative memories with insights from statistical physics, contraction analysis, control theory, etc. (e.g., Lucibello & Mezard (2024), Fachechi et al. (2018), Agliari et al. (2022))
- Multimodal architectures with associative memories
- Lyapunov Functions (e.g., Cohen & Grossberg (1983), Hopfield (1984), Krotov (2021))
Sequential Hopfield networks for temporal sequences (e.g., Karuvally et al. (2022), Chaudhry et al. (2023), Wu et al. (2023))
- Other machine learning tasks (such as clustering, dimensionality reduction) with associative memories (e.g., Saha et al. (2023), Hu et al. (2024), Hu et al. (2023), Saha et al. (2024), Cabannes et al. (2023), Bhandarkar & McClelland (2023), Davydov et al. (2023))
- Energy-based Transformers (e.g., Hoover et al. (2023a))
- Applications of associative memories and energy-based models to various data domains, such as language, images, sound, graphs, temporal sequences, computational chemistry and biology, etc. (e.g., Widrich et al. (2020), Liang et al. (2022), Fürst et al. (2022), Bricken et al. (2023), Tang & Kopp (2021))