Attention Approximates Sparse Distributed Memory

Introduction Used most notably in the Transformer , Attention has helped deep learning to arguably approach human level performance in various tasks with larger models continuing to boost performance [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] . However , the heuristic motivations that produced Attention leave open the question of why it performs so well [ 1 , 10 ] . Insights into why Attention is so effective would not only make it more interpretable but also guide future improvements . Much has been done to try and explain Attention ’ s success , including work showing that Transformer representations map more closely to human brain recordings and inductive biases than other models [ 11 , 12 ] . Our work takes another step in this direction by showing the potential relationship between Attention and biologically plausible neural processing at the level of neuronal wiring , providing a novel mechanistic perspective behind the Attention operation . This potential relationship is created by showing mathematically that Attention closely approximates Sparse Distributed Memory ( SDM ) . SDM is an associative memory model developed in 1988 to solve the “ Best Match Problem ” , where we have a set of memories and want to quickly find the “ best ” match to any given query [ 13 , 14 ] . In the development of its solution , SDM respected fundamental biological constraints , such as Dale ’ s law , that synapses are fixed to be either excitatory or inhibitory and can not dynamically switch ( see Section 1 for an SDM overview and [ 13 ] or [ 15 ] for a deeper review ) . Despite being developed independently of neuroanatomy , SDM ’ s biologically plausible solution maps strikingly well onto the cerebellum [ 13 , 16 ] .1 Abstractly , the relationship between SDM and Attention exists because SDM ’ s read operation uses intersections between high dimensional hyperspheres that approximate the exponential over sum of exponentials that is Attention ’ s softmax function ( Section 2 ) . Establishing that Attention approximates SDM mathematically , we then test it in pre-trained GPT2 Transformer models [ 3 ] ( Section 3 ) and simulations ( Appendix B.7 ) . We use the Query-Key Normalized Transformer variant [ 22 ] to directly show that the relationship to SDM holds well . We then use original GPT2 models to help confirm this result and make it more general . 1This cerebellar relationship is additionally compelling by the fact that cerebellum-like neuroanatomy exists in many other organisms including numerous insects ( eg . the Drosophila Mushroom Body ) and potentially cephalopods [ 17 , 18 , 19 , 20 , 21 ] . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Using the SDM framework , we are able to go beyond Attention and interpret the Transformer architecture as a whole , providing deeper intuition ( Section 4 ) . Motivated by this mapping between Attention and SDM , we discuss how Attention can be implemented in the brain by summarizing SDM ’ s relationship to the cerebellum ( Section 5 ) . In related work ( Section 6 ) , we link SDM to other memory models [ 23 , 24 ] , including how SDM is a generalization of Hopfield Networks and , in turn , how our results extend work relating Hopfield Networks to Attention [ 25 , 26 ] . Finally , we discuss limitations , and future research directions that could leverage our work ( Section 7 ) . 1 Review of Kanerva ’ s SDM . Here , we present a short overview of SDM . A deeper review on the motivations behind SDM and the features that make it biologically plausible can be found in [ 13 , 15 ] . SDM provides an algorithm for how memories ( patterns ) are stored in , and retrieved from , neurons in the brain . There are three primitives that all exist in the space of n dimensional binary vectors : Patterns ( p ) - have two components : the pattern address , pµa 2 { 0 , 1 } n , is the vector representation of a memory ; the pattern “ pointer ” , pµp 2 { 0 , 1 } n , is bound to the address and points to itself when autoassociative or to a different pattern address when heteroassociative . A heteroassociative example is memorizing the alphabet where the pattern address for the letter a points to pattern address b , b points to c etc . For tractability in analyzing SDM , we assume our pattern addresses and pointers are random . There are m patterns and they are indexed by the superscript µ 2 { 1 , . . . , m } . Neurons ( x ) - in showing SDM ’ s relationship to Attention it is sufficient to know there are r neurons with fixed addresses x⌧a 2 { 0 , 1 } n that store a set of all patterns written to them . Each neuron will sum over its set of patterns to create a superposition . This creates minimal noise interference between patterns because of the high dimensional nature of the vector space and enables all patterns to be stored in an n dimensional storage vector denoted x⌧v 2 Zn+ , constrained to the positive integers . Their biologically plausible features are outlined in [ 13 , 15 ] . When we assume our patterns are random , we also assume our neuron addresses are randomly distributed . Of the 2n possible vectors in our binary vector space , SDM is “ sparse ” because it assumes that r ⌧ 2n neurons exist in the space . Query ( ⇠ ) - is the input to SDM , denoted ⇠ 2 { 0 , 1 } n. The goal in the Best Match Problem is to return the pattern pointer stored at the closest pattern address to the query . We will often care about the maximum noise corruption that can be applied to our query , while still having it read out the correct pattern . An autoassociative example is wanting to recognize familiar faces in poor lighting . Images of faces we have seen before are patterns stored in memory and our query is a noisy representation of one of the faces . We want SDM to return the noise-free version of the queried face , assuming it is stored in memory . SDM uses the Hamming distance metric between any two vectors defined : d ( a , b ) : = 1Tn |a b| . The all ones vector 1n is of n dimensions and |a b| takes the absolute value of the element-wise difference between the binary vectors . When it is clear what two vectors the Hamming distance is between , we will sometimes use the shorthand dv : = d ( a , b ) . The Hamming distance is crucial for determining how many neurons read and write operations are distributed across . The optimal Hamming distance for the read and write circles denoted d⇤ , depends upon the number and distribution of patterns in the vector space and what the memories are being used for ( e.g . maximizing the number of memories that can be stored versus the memory system ’ s robustness to query noise ) . We provide three useful reference d⇤ values , using equations outlined in Appendix B.5 . The Signal-to-Noise Ratio ( SNR ) optimal d⇤SNR maximizes the probability a noise-free query will return its target pattern [ 15 ] . The memory capacity optimal d⇤Mem maximizes the number of memories that can be stored with a certain retrieval probability and also assumes a noise-free query . The critical distance d⇤CD maximizes , for a given number of patterns , the amount of noise that can be applied to a query such that it will converge to its correct pattern [ 15 ] . These d⇤s are only approximate reference points for later comparisons to Transformer Attention , first and foremost because they assume random patterns to make their derivations tractable . In addition , Transformer Attention will not be optimizing for just one of these objectives , and likely interpolates between these optimal d⇤s as it wants to have both a good critical distance to handle noisy queries and a reasonable memory capacity . These optimal d⇤ are a function of n , r and m. For the Transformer Attention setting [ 1 ] , where n = 64 , r = 2n and m 1024 , d⇤SNR = 11 , d ⇤ Mem = 5 , d⇤CD = 15 , as derived in Appendix B.5 . 1.1 SDM Read Operation . For the connection to Attention we focus on the SDM read operation and briefly summarize the write operation : all patterns write their pointers pp in a distributed fashion to all neuron addresses located within Hamming distance d. This means that each neuron will store a superposition of pattern pointers from those pattern addresses within d : x⌧v = P { p : d ( pµa , x⌧a ) d,8µ } pp . Having stored patterns in a distributed fashion across nearby neurons , SDM ’ s read operation retrieves stored pattern pointers from all neurons within distance d of the query and averages them . This average is effectively weighted because the same patterns have distributed storage across multiple neurons being read from . The pattern weighting will be higher for those patterns with addresses nearer the query because they have written their pointers to more neurons the query reads from . Geometrically , this weighting of each pattern can be interpreted as the intersection of radius d circles2 that are centered on the query and each pattern address pµa for all µ . A high level overview of the SDM read and write operations is shown in Fig . 1 . The 2n possible neurons that have both stored this pattern ’ s pointer pp and been read by ⇠ is : |On ( pa , d ) \ On ( ⇠ , d ) | , where | · | is the cardinality operator and On ( ⇠ , d ) = { xa 2 { 0 , 1 } n : d ( ⇠ , xa ) d } is the set of all possible neuronal addresses xa within radius d of ⇠ . Mathematically , 2In this binary space , the Hamming distance around a vector is in fact a hypercube but the vertices of an n dimensional unit cube lie on the surface of an n dimensional sphere with radius p n/2 and we refer to this as a circle because of our two dimensional diagrams . We adopt this useful analogy , taken from Kanerva ’ s book on SDM [ 13 ] , throughout the paper . SDM ’ s read operation sums over each pattern ’ s pointer , weighted by its query circle intersection : ⇠new = g P p2P |On ( pa , d ) \On ( ⇠ , d ) |ppP p2P |On ( pa , d ) \On ( ⇠ , d ) | ! , g ( e ) = ( 1 , if e > 1 2 0 , else , ( 1 ) and g acts elementwise on vectors . The denominator normalizes all of the weights so they sum to 1 in the numerator and enables computing if the element-wise majority value is a 0 or 1 , using the function g ( · ) . Intuitively , the query will converge to the nearest “ best ” pattern because it will have the largest circle intersection weighting . The output of the SDM read operation is written as updating the query ⇠ ! ⇠new so that it can ( but is not required to ) apply the read operation iteratively if full convergence to its “ best match ” pattern is desired and was not achieved in one update . The circle intersection ( derived in Appendix B.1 ) is calculated as a function of the Hamming radius for the read and write operations d , the dimensionality n , and the vector distance between the query and pattern : dv = d ( pa , ⇠ ) , so we use the shorthand I ( dv , d , n ) : I ( dv , d , n ) : = |On ( pa , d ) \On ( ⇠ , d ) | = n dvX a=n d b dv2 c dv ( n d a ) X c=max ( 0 , n d a ) ✓ n dv a ◆ · ✓ dv c ◆ ! ( 2 ) Eq . 2 sums over the number of possible binary vectors that can exist at every location inside the circle intersection . Taking inspiration from [ 27 ] , this is a new and more interpretable derivation of the circle intersection than that originally developed [ 13 ] . Eq . 2 is approximately exponential for the closest , most important patterns where d ( pa , ⇠ ) 2d , which is crucial to how SDM approximates Attention . This is shown for a representative instance of SDM in Fig . 2 . The details of this approximation are provided in Appendix B.2 , but at a high level the binomial coefficients can be represented as binomial distributions and then approximated by normal distributions that contain exponentials . With the correctly chosen constants , c1 and c2 , that are independent of the vector distance d ( pa , ⇠ ) , we can make the following approximation : I d ( pa , ⇠ ) , d , n ⇡ c1 exp c2 · d ( pa , ⇠ ) ( 3 )

Summary: This paper shows that Transformer Attention approximates Sparse Distributed Memory (SDM) model under given conditions.  It further confirms that those conditions are satisfied on pre-trained GPT2 Transformer models.  In addition to theoretical analysis, some results are provided using various datasets including MNIST.  Furthermore, there is a section on biological interpretation of Attention and SDM.