Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks

1 INTRODUCTION Artificial neural networks have been increasingly used to study biological neural circuits . In particular , recent work in vision demonstrated that convolutional neural networks ( CNNs ) trained to perform visual object classification provide state-of-the-art models that match neural responses along various stages of visual processing ( Yamins et al. , 2014 ; Khaligh-Razavi & Kriegeskorte , 2014 ; Yamins & DiCarlo , 2016 ; Cadieu et al. , 2014 ; Güçlü & van Gerven , 2015 ; Kriegeskorte , 2015 ) . Recurrent neural networks ( RNNs ) trained on cognitive tasks have also been used to account for neural response characteristics in various domains ( Mante et al. , 2013 ; Sussillo et al. , 2015 ; Song et al. , 2016 ; Cueva & Wei , 2018 ; Banino et al. , 2018 ; Remington et al. , 2018 ; Wang et al. , 2018 ; Orhan & Ma , 2019 ; Yang et al. , 2019 ) . While these results provide important insights on how information is processed in neural circuits , it is unclear whether artificial neural networks have converged upon similar architectures as the brain to perform either visual or cognitive tasks . Answering this question requires understanding the functional , structural , and mechanistic properties of artificial neural networks and of relevant neural circuits . We address these challenges using the brain ’ s internal compass - the head direction system , a system that has accumulated substantial amounts of functional and structural data over the past few decades in rodents and fruit flies ( Taube et al. , 1990a ; b ; Turner-Evans et al. , 2017 ; Green et al. , 2017 ; Seelig & Jayaraman , 2015 ; Stone et al. , 2017 ; Lin et al. , 2013 ; Finkelstein et al. , 2015 ; Wolff et al. , 2015 ; Green & Maimon , 2018 ) . We trained RNNs to perform a simple angular velocity ( AV ) integration task ( Etienne & Jeffery , 2004 ) and asked whether the anatomical and functional features that have emerged as a result of stochastic gradient descent bear similarities to biological networks sculpted ∗equal contribution †Correspondence : ccueva @ gmail.com , weixxpku @ gmail.com by long evolutionary time . By leveraging existing knowledge of the biological head direction ( HD ) systems , we demonstrate that RNNs exhibit striking similarities in both structure and function . Our results suggest that goal-driven training of artificial neural networks provide a framework to study neural systems at the level of both neural activity and anatomical organization . 2 MODEL 2.1 MODEL STRUCTURE We trained our networks to estimate the agent ’ s current head direction by integrating angular velocity over time ( Fig . 1f ) . Our network model consists of a set of recurrently connected units ( N = 100 ) , which are initialized to be randomly connected , with no self-connections allowed during training . The dynamics of each unit in the network ri ( t ) is governed by the standard continuous-time RNN equation : τ dxi ( t ) dt = −xi ( t ) + ∑ j W recij rj ( t ) + ∑ k W inik Ik ( t ) + bi + ξi ( t ) ( 1 ) for i = 1 , . . . , N . The firing rate of each unit , ri ( t ) , is related to its total input xi ( t ) through a rectified tanh nonlinearity , ri ( t ) = max ( 0 , tanh ( xi ( t ) ) ) . Every unit in the RNN receives input from all other units through the recurrent weight matrix W rec and also receives external input , I ( t ) , through the weight matrix W in . These weight matrices are randomly initialized so no structure is a priori introduced into the network . Each unit has an associated bias , bi which is learned and an associated noise term , ξi ( t ) , sampled at every timestep from a Gaussian with zero mean and constant variance . The network was simulated using the Euler method for T = 500 timesteps of duration τ/10 ( τ is set to be 250ms throughout the paper ) . Let θ be the current head direction . Input to the RNN is composed of three terms : two inputs encode the initial head direction in the form of sin ( θ0 ) and cos ( θ0 ) , and a scalar input encodes both clockwise ( CW , negative ) and counterclockwise , ( CCW , positive ) angular velocity at every timestep . The RNN is connected to two linear readout neurons , y1 ( t ) and y2 ( t ) , which are trained to track current head direction in the form of sin ( θ ) and cos ( θ ) . The activities of y1 ( t ) and y2 ( t ) are given by : yj ( t ) = ∑ i W outji ri ( t ) ( 2 ) 2.2 INPUT STATISTICS Velocity at every timestep ( assumed to be 25 ms ) is sampled from a zero-inflated Gaussian distribution ( see Fig . 5 ) . Momentum is incorporated for smooth movement trajectories , consistent with the observed animal behavior in flies and rodents . More specifically , we updated the angular velocity as AV ( t ) = σX + momentum∗AV ( t−1 ) , whereX is a zero mean Gaussian random variable with standard deviation of one . In the Main condition , we set σ = 0.03 radians/timestep and the momentum to be 0.8 , corresponding to a mean absolute AV of ∼100 deg/s . These parameters are set to roughly match the angular velocity distribution of the rat and fly ( Stackman & Taube , 1998 ; Sharp et al. , 2001 ; Bender & Dickinson , 2006 ; Raudies & Hasselmo , 2012 ) . In section 4 , we manipulate the magnitude of AV by changing σ to see how the trained RNN may solve the integration task differently . 2.3 TRAINING We optimized the network parameters W rec , W in , b and W out to minimize the mean-squared error in equation ( 3 ) between the target head direction and the network outputs generated according to equation ( 2 ) , plus a metabolic cost for large firing rates ( L2 regularization on r ) . E = ∑ t , j ( yj ( t ) − ytargetj ( t ) ) 2 ( 3 ) Parameters were updated with the Hessian-free algorithm ( Martens & Sutskever , 2011 ) . Similar results were also obtained using Adam ( Kingma & Ba , 2015 ) . 3 FUNCTIONAL AND STRUCTURAL PROPERTIES EMERGED IN THE NETWORK We found that the trained network could accurately track the angular velocity ( Fig . 1g ) . We first examined the functional and structural properties of model units in the trained RNN and compared them to the experimental data from the head direction system in rodents and flies . 3.1 EMERGENCE OF HD CELLS ( COMPASS UNITS ) AND HD × AV CELLS ( SHIFTERS ) Emergence of different classes of neurons with distinct tuning properties We first plotted the neural activity of each unit as a function of HD and AV ( Fig . 2a ) . This revealed two distinct classes of units based on the strength of their HD and AV tuning ( see Appendix Fig . 6a , b , c ) . Units with essentially zero activity are excluded from further analyses . The first class of neurons exhibited HD tuning with minimal AV tuning ( Fig . 2f ) . The second class of neurons were tuned to both HD and AV and can be further subdivided into two populations - one with high firing rate when animal performs CCW rotation ( positive AV ) , the other favoring CW rotation ( negative AV ) ( CW tuned cell shown in Fig . 2g ) . Moreover , the preferred head direction of each sub-population of neurons tile the complete angular space ( Fig . 2b ) . Embedding the model units into 3D space using t-SNE reveals a clear compass-like structure , with the three classes of units being separated ( Fig . 2c ) . Mapping the functional architecture of RNN to neurophysiology Neurons with HD tuning but not AV tuning have been widely reported in rodents ( Taube et al. , 1990a ; Blair & Sharp , 1995 ; Stackman & Taube , 1998 ) , although the HD * AV tuning profiles of neurons are rarely shown ( but see Lozano et al . ( 2017 ) ) . By re-analyzing the data from Peyrache et al . ( 2015 ) , we find that neurons in the anterodorsal thalamic nucleus ( ADN ) of the rat brain are selectively tuned to HD but not AV ( Fig . 2d , also see Lozano et al . ( 2017 ) ) , with HD * AV tuning profile similar to what our model predicts . Preliminary evidence suggests that this might also be true for ellipsoid body ( EB ) compass neurons in the fruit fly HD system ( Green et al. , 2017 ; Turner-Evans et al. , 2017 ) . Published as a conference paper at ICLR 2020 Integrated angular input ( 0 to 360 degrees ) An gu la r i np ut ( - 13 .8 63 5 to 1 4 . 56 69 d eg re es ) Different scale for each unit , Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % A 22 5 0 -2 00 0 2ππ an gu la r v el oc ity ( ° /s ) heading ( rad ) 22 5 0 -2 00 an gu la r v el oc ity ( ° /s ) 05 spike rate ( Hz ) 0 5 sp ik e ra te ( H z ) 0 2ππ R2 = 0 .8 7 R2 = 0 .9 4 C 16 06 21 _1 , 5 5 m in ut es , 6 57 2 sp ik es 15 11 29 _1 , 6 m in ut e ep oc h , 2 33 s pi ke s heading ( rad ) 0 2ππ R2 = 0 .7 6 R2 = 0 .9 6 12 5 0 -1 75 an gu la r v el oc ity ( ° /s ) 0 2ππ heading ( rad ) 12 5 0 -1 75 an gu la r v el oc ity ( ° /s ) 010 spike rate ( Hz ) 0 5 sp ik e ra te ( H z ) 10 Ri gh t Le ft Ri gh t Le ft heading ( rad ) 048 spike rate ( Hz ) 12 B 0 20 0 co un t count 050 sh uffl ed c on tr ol -2 00020 0 2π -6 0 -4 0 -2 0π0 D ex pe rim en t v R ot ( ° /s ) he ad in g ( r ad ) V m ( m V ) 2s m ax m in m ax m in 0 0 . 2 0 . 4 0 0 . 5 1 ro ta tio n tu ni ng in de x heading tuning index Fi g ur e 5 . PEN s ar e co nj un ct iv el y tu ne d to an gu la rv el oc ity an d he ad in g. ( A ) T w odi m en si on al tu ni ng pr op er tie s of a PEN ’ s sp ik e ou tp ut av er ag ed ov er th e en tir e du ra tio n of th e ex pe rim en t. Th e ce ll w as lo ca te d in th e le ft he m is ph er e ( s ch em at ic at to p ) .T he fly w al ke d w ith cl os ed lo op vi su al fe ed ba ck .T he he at m ap ( u pp er le ft ) sh ow s sp ik e ra te s as a fu nc tio n of an gu la rv el oc ity an d vi rt ua lh ea di ng .A tu ni ng cu rv e w as fit te d to th e fly ’ s vi rt ua l he ad in g an gl e ( u pp er rig ht ) a s w el la s th e fly ’ s an gu la rv el oc ity ( lo w er le ft ) ( s ee M at er ia ls an d m et ho ds ) . B la ck : m ea n ra te an d 95 % co nf id en ce in te rv al s. Tu ni ng cu rv es ar e le as ts qu ar es fit to th e su m of tw o vo n M is es fu nc tio ns fo rh ea di ng ( g re en ) a nd to a si gm oi da lf un ct io n fo rr ot at io ns ( re d ) . ( B ) Q ua nt ifi ca tio n of th e he ad in g tu ni ng in de x ( m ea n ve ct or le ng th of sp ik e ra te s ac ro ss vi rt ua lh ea di ng an gl es ) a nd th e ro ta tio n tu ni ng in de x ( R 2 va lu es fo r si gm oi da lt un in g cu rv e fit ) f or th e tw o ex am pl e ce lls pl ot te d in A an d C ( b ol d re d op en ci rc le an d bl ue op en ci rc le , r es pe ct iv el y ) .S hu ffl ed co nt ro ls ( s ee M at er ia ls an d m et ho ds ) a re pl ot te d as do ts an d do tt ed lin es m ar k 95 th pe rc en til e bo un da rie s ( p lo tt ed in re d an d bl ue fo rt he tw o ex am pl e ce lls ) . H is to gr am s of sh uf fle d da ta in di ce s ar e pl ot te d at to p an d at rig ht fo llo w in g th e sa m e co lo rc od e. Tu ni ng in di ce s fo ra ll ce lls ar e sh ow n in Fi g ur e 5— fi g ur e su p p le m en t 1C . ( C ) T w odi m en si on al tu ni ng pr op er tie s of a PEN ’ s sp ik e ou tp ut du rin g a br ie fe po ch th at st ill su ffi ci en tly sa m pl ed th e tw o- di m en si on al pa ra m et er sp ac e. Th e ce ll w as lo ca te d in th e rig ht he m is ph er e ( s ch em at ic at to p ) .T he fly w al ke d w ith a cl os ed lo op vi su al fe ed ba ck .T he he at m ap ( u pp er le ft ) sh ow s sp ik e ra te s as a fu nc tio n of an gu la rv el oc ity an d vi rt ua lh ea di ng .C ol or ba ra tr ig ht .T un in g cu rv es to th e fly ’ s vi rt ua lh ea di ng an gl e an d an gu la rv el oc ity ar e pl ot te d an al og ou s to A . ( D ) E xa m pl e tr ac es fo rt he ep oc h pl ot te d in C ill us tr at in g tu ni ng to ro ta tio na lv el oc ity as w el la s vi rt ua lh ea di ng .T he tw o pa ra m et er s ar e co lo rc od ed ac co rd in g to th e ne ur al ac tiv ity ex pe ct ed fro m th e tu ni ng cu rv es pl ot te d in pa ne lC . D O I : 10 .7 55 4/ eL ife .2 34 96 .0 09 Fi gu re 5 co nt in ue d on ne xt pa ge Tu rn er -E va ns et al .e Li fe 20 17 ; 6 : e 23 49 6 . D O I : 10 .7 55 4/ eL ife .2 34 96 9 of 39 Re se ar ch ar tic le N eu ro sc ie nc e 0 An gu la r v el oc ity ( - 14 ° t o 14 ° ) a c d Head direc8on ( 0° to 360° ) Ring CW shiAer CCW shiAe t-SNE Pr ef er re d he ad d ire c8 on ( 0 ° t o 36 0° ) Unit Integrated angular input ( 0 to 360 degrees ) An gu la r i np ut ( - 13 .8 63 5 to 1 4 . 56 69 d eg re es ) Different scale for each unit , Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % shift neg ring shift pos shift pos ring shift pos shift neg shift posshift pos shift neg shift pos shift neg weakly tuned ring shift neg ring ring ring ringring shift neg shift pos shift neg shift pos shift pos shift pos shift pos ring ringring shift pos shift neg shift pos ring ring shift neg ring ring shift neg ringring shift pos shift neg ring ring shift pos shift pos shift pos ring shift neg shift pos ring ring shift pos ring shift pos ring ring shift pos ringring shift neg shift posshift pos shift neg shift pos shift neg ring shift neg shift neg ring shift neg shift neg shift neg shift neg shift pos weakly tunedshift neg shift pos shift pos shift pos ring shift neg weakly tuned ring ring shift neg ring A 2 2 5 0 -2 0 0 0 2 ππ an g u la r ve lo ci ty ( ° /s ) heading ( rad ) 2 2 5 0 -2 0 0 an g u la r ve lo ci ty ( ° /s ) 05 spike rate ( Hz ) 0 5 sp ik e ra te ( H z ) 0 2 ππ R 2 = 0 .8 7 R 2 = 0 .9 4 C 1 6 0 6 2 1 _1 , 5 5 m in u te s , 6 5 7 2 s p ik es 1 5 1 1 2 9 _1 , 6 m in u te e p o ch , 2 3 3 s p ik es heading ( rad ) 0 2 ππ R 2 = 0 .7 6 R 2 = 0 .9 6 1 2 5 0 -1 7 5 an g u la r ve lo ci ty ( ° /s ) 0 2 ππ heading ( rad ) 1 2 5 0 -1 7 5 an g u la r ve lo ci ty ( ° /s ) 0 1 0 spike rate ( Hz ) 0 5 sp ik e ra te ( H z ) 1 0 R ig h t Le ft R ig h t Le ft heading ( rad ) 048 spike rate ( Hz ) 1 2 B 0 2 0 0 co u n t count 0 5 0 sh u ffl ed c o n tr o l -2 0 00 2 0 0 2 π -6 0 -4 0 -2 0π0 D ex p er im en t v R o t ( °/ s ) h ea d in g ( r ad ) V m ( m V ) 2 s m ax m in m ax m in 0 0 .2 0 .4 0 0 .5 1 ro ta ti o n t u n in g in d e x heading tuning index F ig u re 5 . P -E N s ar e co n ju n ct iv e ly tu n e d to an g u la r ve lo ci ty an d h e ad in g . ( A ) T w o -d im e n si o n al tu n in g p ro p e rt ie s o f a P -E N ’ s sp ik e o u tp u t av e ra g e d o ve r th e e n ti re d u ra ti o n o f th e e xp e ri m en t. T h e ce ll w as lo ca te d in th e le ft h e m is p h e re ( s ch e m at ic at to p ) . T h e fly w al ke d w it h cl o se d lo o p vi su al fe e d b ac k. T h e h e at m ap ( u p p e r le ft ) sh o w s sp ik e ra te s as a fu n ct io n o f an g u la r ve lo ci ty an d vi rt u al h e ad in g .A tu n in g cu rv e w as fit te d to th e fly ’ s vi rt u al h e ad in g an g le ( u p p e r ri g h t ) as w e ll as th e fly ’ s an g u la r ve lo ci ty ( lo w e r le ft ) ( s e e M at e ri al s an d m e th o d s ) .B la ck : m e an ra te an d 95 % co n fid e n ce in te rv al s. T u n in g cu rv e s ar e le as t sq u ar e s fit to th e su m o f tw o vo n M is e s fu n ct io ns fo r h e ad in g ( g re e n ) an d to a si g m o id al fu n ct io n fo r ro ta ti o n s ( r e d ) . ( B ) Q u an ti fic at io n o f th e h e ad in g tu n in g in d e x ( m e an ve ct o r le n g th o f sp ik e ra te s ac ro ss vi rt u al h e ad in g an g le s ) an d th e ro ta ti o n tu n in g in d e x ( R 2 va lu e s fo r si g m o id al tu n in g cu rv e fit ) fo r th e tw o e xa m p le ce lls p lo tt e d in A an d C ( b o ld re d o p e n ci rc le an d b lu e o p e n ci rc le , r e sp e ct iv e ly ) . S h u ff le d co n tr o ls ( s e e M at e ri al s an d m e th o d s ) ar e p lo tt e d as d o ts an d d o tt e d lin e s m ar k 95 th p e rc e n ti le b o u n d ar ie s ( p lo tt e d in re d an d b lu e fo r th e tw o e xa m p le ce lls ) . H is to g ra m s o f sh u ff le d d at a in d ic e s ar e p lo tt e d at to p an d at ri g h t fo llo w in g th e sa m e co lo r co d e . T u n in g in d ic e s fo r al lc e lls ar e sh o w n in F ig u re 5 — fi g u re su p p le m e n t 1 C . ( C ) T w o -d im e n si o n al tu n in g p ro p e rt ie s o f a P -E N ’ s sp ik e o u tp u t d u ri n g a b ri e f e p o ch th at st ill su ff ic ie n tl y sa m p le d th e tw o - d im e n si o n al p ar am e te r sp ac e . T h e ce ll w as lo ca te d in th e ri g h t h e m is p he re ( s ch e m at ic at to p ) . T h e fly w al ke d w it h a cl o se d lo o p vi su al fe e d b ac k. T h e h e at m ap ( u p p e r le ft ) sh o w s sp ik e ra te s as a fu n ct io n o f an g u la r ve lo ci ty an d vi rt u al h e ad in g .C o lo rb ar at ri g h t. T u n in g cu rv e s to th e fly ’ s vi rt u al h e ad in g an g le an d an g u la r ve lo ci ty ar e p lo tt e d an al o g o u s to A . ( D ) E xa m p le tr ac e s fo r th e e p o ch p lo tt e d in C ill u st ra ti n g tu n in g to ro ta ti o n al ve lo ci ty as w e ll as vi rt u al h e ad in g .T h e tw o p ar am e te rs ar e co lo r co d e d ac co rd in g to th e n e u ra la ct iv it y e xp e ct e d fr o m th e tu n in g cu rv e s p lo tt e d in p an e lC . D O I : 10 .7 55 4/ e Li fe .2 34 96 .0 09 F ig u re 5 co n ti n u e d o n n e xt p ag e T ur ne rE va ns et al . eL if e 20 17 ; 6 : e 23 49 6 . D O I : 10 .7 55 4/ eL if e. 23 49 6 9 o f 39 R es ea rc h ar ti cl e N eu ro sc ie nc e b Head direc8on ( 0° to 360° ) A ng ul ar v el oc ity Pr ef er re d he ad d ire c8 on Dimension 2 Training epoch 415 errormain = 0.0014384 , normalized error overall = 0.57538 % Dimension 1 D im en si on 3 33 ring nits 28 s iftneg units 24 shiftpos units Di m en sio n 3 Training epoch 415 errormain = 0.0014384 , normalized error overall = 0.57538 % Dimension 2 Dimension 1 Dimension 1 Trainin epoch 415 errormain = 0.0014384 , norma ized error overall = 0.57538 % Dimension 2 D im en si on 3 33 ring units 28 shiftneg units 24 shiftpos units 10 20 30 0 180 360 0° 180° 360° 5 10 15 20 25 30 Unit ( sorted ) 0 50 100 150 200 250 300 350 Pr ef er re d in te gr at ed -a ng le ( d eg re es ) Training epoch 415 , 2000 trials , 1000 timesteps in simulation errormain = 0.0014384 , normalized error overall = 0.57538 % 33 ring units 24 shiftneg units 28 shiftpos units 3 weakly tuned units Ring CW shiAer CCW shiAer 20 10 30 An gu la r v el oc ity ( - 14 ° t o 14 ° ) a c d Head direc8on ( 0° to 360° ) Ring CW shiAer CCW shiAer t-SNE Pr ef er re d he ad d ire c8 on ( 0 ° t o 36 0° ) Unit Integrated angular input ( 0 to 360 degrees ) An gu la r i np ut ( - 13 .8 63 5 to 1 4 . 56 69 d eg re es ) Different scale for each unit , Training epoch 415 , 2000 trials , 1000 timestep in imulation normalized error overall = 0.57538 % shift neg ring shift pos shift pos ring shift pos shift neg shift posshift pos shift neg shift pos shift neg weakly tuned ring shift neg ring ring ring ringring shift neg shift pos shift neg shift pos shift pos shift pos shift pos ring ringring shift pos shift neg shift pos ring ring shift neg ring ring shift neg ringring shift pos shift neg ring ring shift pos shift pos shift pos ring shift neg shift pos ring ring shift pos ring shift pos ring ring shift pos ringring shift neg shift posshift pos shift neg shift pos shift neg ring shift neg shift neg ring shift neg shift neg shift neg shift neg shift pos weakly tunedshift neg shift pos shift pos shift pos ring shift neg weakly tuned ring ring shift neg ring A 2 2 5 0 -2 0 0 0 2 ππ an g u la r ve lo ci ty ( ° /s ) heading ( rad ) 2 2 5 0 -2 0 0 an g u la r ve lo ci ty ( ° /s ) 05 spie rate ( Hz ) 0 5 sp ik e ra te ( H z ) 0 2 ππ R 2 = 0 .8 7 R 2 = 0 .9 4 C 1 6 0 6 2 1 _1 , 5 5 m in u te s , 6 5 7 2 s p ik es 1 5 1 1 2 9 _1 , 6 m in u te e p o ch , 2 3 3 s p ik es heading ( rad ) 0 2 ππ R 2 = 0 .7 6 R 2 = 0 .9 6 1 2 5 0 -1 7 5 an g u la r ve lo ci ty ( ° /s ) 0 2 ππ heading ( rad ) 1 2 5 0 -1 7 5 an g u la r ve lo ci ty ( ° /s ) 0 1 0 spike rate ( Hz ) 0 5 sp ik e ra te ( H z ) 1 0 R ig h t Le ft R ig h t Le ft heading ( rad ) 048 spike rate ( Hz ) 1 2 B 0 2 0 0 co u n t count 0 5 0 sh u ffl ed c o n tr o l -2 0 00 2 0 0 2 π -6 0 -4 0 -2 0π0 D ex p er im en t v R o t ( °/ s ) h ea d in g ( r ad ) V m ( m V ) 2 s m ax m in m ax m in 0 0 .2 0 .4 0 0 .5 1 ro ta ti o n t u n in g in d e x heading tuning index F ig u re 5 . P -E N s ar e co n ju n ct iv e ly tu n e d to an g u la r ve lo ci ty an d h e ad in g . ( A ) T w o -d im e n si o n al tu n in g p ro p e rt ie s o f a P -E N ’ s sp ik e o u tp u t av e ra g e d o ve r th e e n ti re d u ra ti o n o f th e e xp e ri m e n t. T h e ce ll w as lo ca te d in th e le ft h e m is p h e re ( s ch e m at ic at to p ) . T h e fly w al ke d w it h cl o se d lo o p vi su al fe e d b ac k. T h e h e at m ap ( u p p e r le ft ) sh o w s sp ik e ra te s as a fu n ct io n o f an g u la r ve lo ci ty an d vi rt u al h e ad in g . A tu n in g cu rv e w as fit te d to th e fly ’ s vi rt u al h e ad in g an g le ( u p p e r ri g h t ) as w e ll as th e fly ’ s an g u la r ve lo ci ty ( lo w e r le ft ) ( s e e M at e ri al s an d m e th o d s ) . B la ck : m e an ra te an d 95 % co n fid e n ce in te rv al s. T u n in g cu rv e s ar e le as t sq u ar e s fit to th e su m o f tw o vo n M is e s fu n ct io n s fo r h e ad in g ( g re e n ) an d to a si g m o id al fu n ct io n fo r ro ta ti o n s ( r e d ) . ( B ) Q u an ti fic at io n o f th e h e ad in g tu n in g in d e x ( m e an ve ct o r le n g th o f sp ik e ra te s ac ro ss vi rt u al h e ad in g an g le s ) an d th e ro ta ti o n tu n in g in d e x ( R 2 va lu e s fo r si g m o id al tu n in g cu rv e fit ) fo r th e tw o e xa m p le ce lls p lo tt e d in A an d C ( b o ld re d o p e n ci rc le an d b lu e o p e n ci rc le , re sp e ct iv e ly ) . S h u ff le d co n tr o ls ( s e e M at e ri al s an d m e th o d s ) ar e p lo tt e d as d o ts an d d o tt e d lin e s m ar k 95 th p e rc e n ti le b o u n d ar ie s ( p lo tt e d in re d an d b lu e fo r th e tw o e xa m p le ce lls ) . H is to g ra m s o f sh u ff le d d at a in d ic e s ar e p lo tt e d at to p an d at ri g h t fo llo w in g th e sa m e co lo r co d e . T u n in g in d ic e s fo r al lc e lls ar e sh o w n in F ig u re 5 — fi g u re su p p le m e n t 1 C . ( C ) T w o -d im e n si o n al tu n in g p ro p e rt ie s o f a P -E N ’ s sp ik e o u tp u t d u ri n g a b ri e f e p o ch th at st ill su ff ic ie n tl y sa m p le d th e tw o - d im e n si o n al p ar am e te r sp ac e . T h e ce ll w as lo ca te d in th e ri g h t h e m is p h e re ( s ch e m at ic at to p ) . T h e fly w al ke d w it h a cl o se d lo o p vi su al fe e d b ac k. T h e h e at m ap ( u p p e r le ft ) sh o w s sp ik e ra te s as a fu n ct io n o f an g u la r ve lo ci ty an d vi rt u al h e ad in g . C o lo rb ar at ri g h t. T u n in g cu rv e s to th e fly ’ s vi rt u al h e ad in g an g le an d an g u la r ve lo ci ty ar e p lo tt e d an al o g o u s to A . ( D ) E xa m p le tr ac e s fo r th e e p o ch p lo tt e d in C ill u st ra ti n g tu n in g to ro ta ti o n al ve lo ci ty as w e ll as vi rt u al h e ad in g . T h e tw o p ar am e te rs ar e co lo r co d e d ac co rd in g to th e n e u ra la ct iv it y e xp e ct e d fr o m th e tu n in g cu rv e s p lo tt e d in p an e lC . D O I : 10 .7 55 4/ e Li fe .2 34 96 .0 09 F ig u re 5 co n ti n u e d o n n e xt p ag e T ur ne rE va ns e t al . eL if e 20 17 ; 6 : e 23 49 6 . D O I : 10 .7 55 4/ eL if e. 23 49 6 9 o f 39 R es ea rc h ar ti cl e N eu ro sc ie nc e b Head direc8on ( 0° to 360° ) A ng ul ar v el oc ity Pr ef er re d he ad d ire c8 on Dimension 2 Training epoch 415 errormain = 0.0 14384 , normalized rror overall = 0.57538 % Dimension 1 D im en si on 3 33 ring units 28 s iftneg units 24 shif pos units D im en si on 3 Training epoch 415 errormain = 0.0014384 , normalized error ov rall = 0.57538 % Dimension 2 Dimension 1 Di ension 1 Training epoch 415 errormain = 0.0014384 , normaliz d error v rall = 0.57538 % Dimension 2 D im en si on 3 33 ring units 28 shiftneg units 24 shiftpos units 10 20 30 0 180 360 0° 180° 360° 5 10 15 20 25 30 Unit ( sorted ) 0 50 100 150 200 250 300 350 Pr ef er re d in te gr at ed -a ng le ( d eg re es ) Training epoch 415 , 2000 trials , 1000 timesteps in s mulation errormain = 0.0 1438 , normalized rror overall = 0.57538 % 33 ring units 24 shiftneg units 28 shiftpos units 3 weakly tuned units Ring CW shiAer CCW shiAer 20 10 30 10 20 30 0 180 360 5 10 15 20 25 30 Unit ( sorted ) 0 50 100 150 200 250 300 350 P re fe rr e d in te g ra te d -a n g le ( d e g re e s ) Training epoch 415 , 2000 trials , 1000 timesteps in simulation errormain = 0.0014384 , normalized error overall = 0.57538 % 33 ring units 26 shiftneg units 29 shiftpos units Training epoch 415 errormain = 0.0014384 , normalized error overall = 0.57538 % D im en si on 3 Dimension 2Dimension 1 Dimension 2 Training epoch 415 errormain = 0.0014384 , normalized error overall = 0.57538 % Dimension 1 D im en si on 3 Dimension 1 D im en si on 2 Training epoch 415 errormain = 0.0014384 , normalized error overall = 0.57538 % 33 ring units 29 shiftneg units 26 shiftpos units Integrated angular input ( 0 to 360 degrees ) A ng ul ar in pu t ( -1 3 . 86 35 to 1 4 . 56 69 d eg re es ) Different scale for each unit , Training epoch 415 , 2000 trials , 1000 timesteps in s mul tion normalized error overall = 0.57538 % shift negringshift posshift posringshift posshift neg shift posshift posshift negshift posshift negshift posringshift neg ringringringringringshift negshift posshift negshift pos shift posshift posshift posringringringshift posshift negshift pos ringringshift negringringshift negringringshift pos shift negringringshift posshift posshift posringshift neg shift posringringshift posringshift posringringshift posring ringshift negshift posshift posshift negshift posshift negringshift negshift neg ringshift negshift negshift negshift negshift posshift negshift negshift pos shift posshift posringshift negshift negringringshift negring Integrated angular input ( 0 to 360 degrees ) A ng ul ar in pu t ( -1 3 . 86 35 to 1 4 . 56 69 d eg re es ) Different scale for each unit , Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % shift negringshift posshift posringshift posshift neg shift posshift posshift negshift posshift negshift posringshift neg ringringringringringshift negshift posshift negshift pos shift posshift posshift posringringringshift posshift negshift pos ringringshift negringringshift negringringshift pos shift negringringshift posshift posshift posringshift neg shift posringringshift posringshift posringringshift posring ringshift negshift posshift posshift negshift posshift negringshift negshift neg ringshift negshift negshift negshift negshift posshift negshift negshift pos shift posshift posringshift negshift negringringshift negring joint HD * AV tuning ( Model ) an gu la r v el oc ity ( - 55 0 to 5 50 d eg /s ) joint HD * AV tuning ( Data ) cell , HD cell discharge in these cortical areas was modulated over all directional headings ( compare Fig . 4A , B with Fig . 8C of Chen et al. , 1994 ) . The head turn modulation of HD cell activity in these distinct brain areas suggests that LMN HD cells are head turn-modulated HD cells , whereas those in the cortical regions re resent a dual code , signaling both head turn and directional heading . LMN HD cells exhibit anticipatory directional firing Optimal time shif t analyses Previous studies have shown that the temporal relationship of cell firing to the rat ’ s head direction differs across HD cells recorded from the PoS and ADN ( Blair and Sharp , 1995 , Blair et al. , 1997 , Taube and Muller , 1998 ) . Specifically , HD cells in the ADN appear to encode the rat ’ s future directional heading by ! 25 msec , whereas HD cells in the PoS encode the rat ’ s present or recent past directional h ading . Of the 20 HD cells recorded from the LMN , a second HD cell was recorded simultaneously on the same electrode wire for hree ce ls . Because the tuning curves of these simultaneously recorded cells partially overlapped one another , the accuracy of the time shift analysis is likely to be compromised for these cells , and they were therefore excluded from the following analyses . Time shift analyses were conducted on a repre entative session for each of the remaining 17 LMN HD cells . Figure 5A–C illustrates an exampl of the optimal time shifts of an LMN HD cell for each of the three parameters . For this cell , the optimal time shifts for peak firing rate , range width , and information Figure 4 . Differences in influence of turning direction on HD cells recorded from the left and right LMN . Most HD cells recorded from the left LMN exhibited higher firing rates during CCW head turns , whereas most HD cells recorded from the right LMN exhibited higher firing rates during CW head turns . A , B , Representative firing rate by HD-tuning curves during CCW and CW head turns for an LMN HD cell recorded from the left side ( A ) , and from the right side ( B ) . For each HD cell a CW : CCW ratio was calculated by dividing the CW peak firing rate by the CCW peak firing rate . C , D , Representa ive CW and CCW firing rate versus HD tuning functions for a PoS HD cell ( C ) and an ADN HD cell ( D ) . E , The frequency distribution of CW : CCW ratios across both sides ( open bars , lef t side ; filled bars , right side ) for LMN HD cells . F , The frequency distributions of CW : CCW ratios for PoS and ADN cells ( open bars , ADN ; filled bars , PoS ) . Note that the histogram for ADN cells reveals little preference for higher firing rates for CW or CCW turns , whereas the distribution for PoS cells indicates that six cells ( 30 % ) exhibited higher firing rates for CCW turns than CW turns . 9028 J . Neurosci. , November 1 , 1998 , 18 ( 21 ) :9020–9037 Stackman and Taube • Single Unit Analysis of Lateral Mammillary Neurons cell , HD cell discharge in these cortical areas was modulated over all directional headings ( compare Fig . 4A , B with Fig . 8C of Chen et al. , 1994 ) . The head turn modulation of HD cell activity in these distinc brain areas suggests th t LMN HD cells are head turn-modulated HD cells , whereas those in the cortical regions represent a dual code , signaling b th head turn and directional heading . LMN HD cells exhibit anticipatory directional firing Optimal time shif t analyses Previous studies have show t at the temporal rel tionship of c ll firing to the r t ’ s head direction differs across HD cells recorded from the PoS and ADN ( Blair nd Sharp , 1995 , Blair et al. , 1997 , Taub and Muller , 1998 ) . Specifically , HD c ll n the ADN appear to encode the rat ’ s future directional heading by ! 25 msec , whereas HD cells in the PoS encode the rat ’ s present or recent past directional heading . Of the 20 HD cells recorded from the LMN , a second HD c ll was recorded simultaneously on the same lectrode wire for three ells . Because the tuning curves of these simult neously r corded cells partially overlapped one anoth r , the accurac of the time shift analysis is likely to be compromised for thes cells , and th y were therefore excluded from the following analyses . Time shift a lyses were conducted on a representative session for each of the remaining 17 LMN HD cells . Figure 5A–C illustrates an example of the optimal time shifts of an LMN HD cell for each of the three parameters . For this cell , the optimal time shifts for peak firing rate , range width , and information Fig re 4 . Differences in influence of turning direction on HD cells recorded from the left and right LMN . Most HD cells recorded from the left LMN exhibited higher firing rates during CCW head turns , whereas most HD cells recorded from the right LMN exhibited higher firing rates during CW head turns . A , B , Represe tative firing rate by HD-tuning curves d ing CCW and CW head turns for an LMN HD cell recorded from the lef A , and from the right side ( B ) . For each HD cell a CW : CCW ratio was calculated by dividing the CW peak firing rate by the CCW peak firing rate . C , D , Rep esentative CW and CCW firing rate versus HD tuning functions for a PoS HD cell ( C ) and an ADN HD cell ( D ) . E , The frequency distribution of CW : CCW rati acro s both sides ( open bars , lef t side ; filled bars , right side ) for LMN HD cells . F , The frequency distributions of CW : CCW ratios for PoS and ADN cells ( open bars , ADN ; filled bars , PoS ) . Note that the histogram for ADN cells reveals little pr ference for higher firing rates for CW or CCW turns , whereas the distribution for PoS cells indicates that six cells ( 30 % ) exhibited higher firing rates for CCW turns than CW turns . 9028 J . Neurosci. , November 1 , 1998 , 18 ( 21 ) :9020–9037 Stackman and Taube • Single Unit Analysi of Lateral Mammillary Neurons cell , HD cell discharge in these cortical areas was modulated over all directional headings ( compare Fig . 4A , B with Fig . 8C of Chen et al. , 1994 ) . The head turn modulation of HD cell activity in these distinct brain areas suggests that LMN HD cells are head turn-modulated HD cells , whereas those in the cortical regions represent a dual code , signaling both head turn and directional heading . LMN HD cells exhibit anticipatory directional firing Optimal time shif t analyses Previous studies have shown that the temporal relationship of cell firing to the rat ’ s head direction differs across HD cells recorded from the PoS and ADN ( Blair and Sharp , 1995 , Blair et al. , 1997 , Taube and Muller , 1998 ) . Specifically , HD cells in the ADN appear to encode the rat ’ s future directional heading by ! 25 msec , whereas HD cells in the PoS encode the rat ’ s present or recent past directional heading . Of the 20 HD cells recorded from the LMN , a second HD cell was recorded simultaneously on the same electrode wire for three cells . Because the tuning curves of these simultaneously recorded cells partially overlapped one another , the accuracy of the time shift analysis is likely to be compromised for these cells , and they were therefore excluded from the following analyses . Time shift analyses were conducted on a representative session for each of the remaining 17 LMN HD cells . Figure 5A–C illustrates an example of the optimal time shifts of an LMN HD cell for each of the three parameters . For this cell , the optimal time shifts for peak firing rate , range width , and information Figure 4 . Differences in i fluence of turning direction on HD cells recorded from the left and right LMN . Most HD cells recorded from the left LMN exhib- ited higher firing rates during CCW head turns , whereas most HD cells recorded from the right LMN exhibited higher fir- ing rates during CW head turns . A , B , Representative firing rate by HD-tuning curves during CCW and CW head turns for an LMN HD cell recorded from the left side ( A ) , and from the right side ( B ) . For each HD cell a CW : CCW ratio was calculated by dividing the CW peak firing rate by the CCW peak firing rate . C , D , Representative CW and CCW firing rate versus HD tuning functions for a PoS HD cell ( C ) and an ADN HD cell ( D ) . E , The frequency distribution of CW : CCW ra- tios across both sides ( open bars , lef t side ; filled bars , right side ) for LMN HD cells . F , The frequency distributions of CW : CCW ratios for PoS and ADN cells ( open bars , ADN ; filled bars , PoS ) . Note that the histogram for ADN cells reveals little preference for higher firing rates for CW or CCW turns , whereas the distribution for PoS cells indicates that six cells ( 30 % ) exhibited higher firing rates for CCW turns than CW turns . 9028 J . eurosci. , November 1 , 1998 , 18 ( 21 ) :9020–9037 Stackman and Taube • Single Unit Analysis of Lateral Mammillary Neurons cell , HD cell discharge in these cortical areas was modulated over all directional headings ( compare Fig . 4A , B with Fig . 8C of Chen et al. , 1994 ) . The head turn modulation of HD cell activity in these distinct brain areas suggests that LMN HD cells are head turn-modulated HD cells , whereas those in the cortical regions represent a dual code , signaling both head turn and directional heading . LMN HD cells exhibit anticipatory directional firing Optimal time shif t analyses Previous studies have shown that the temporal relationship of cell firing to the rat ’ s head direction differs across HD cells recorded from the PoS and ADN ( Blair and Sharp , 1995 , Blair et al. , 1997 , Taube and Muller , 1998 ) . Specifically , HD cells in the ADN appear to encode the rat ’ s future directional heading by ! 25 msec , whereas HD cells in the PoS encode the rat ’ s present or recent past directional heading . Of the 20 HD cells recorded from the LMN , a second HD cell was recorded simultaneously on the same electrode wire for three cells . Because the tuning curves of these simultaneously recorded cells partially overlapped one another , the accuracy of the time shift analysis is likely to be compromised for these cells , and they were therefore excluded from the following analyses . Time shift analyses were conducted on a representative session for each of the remaining 17 LMN HD cells . Figure 5A–C illustrates an example of the optimal time shifts of an LMN HD cell for each of the three parameters . For this cell , the optimal time shifts for peak firing rate , range width , and information Fig re 4 . Differences in influence of turning direction on HD cells recorded from the left and right LMN . Most HD cells recorded from the left LMN exhib- ited higher firing rates during CCW head turns , whereas most HD cells recorded from t e right LMN exhibited higher fir- ing rates during CW head turns . A , B , Represe tative firing rate by HD-tuning curves d ring CCW and CW head turns for an LMN HD cell recorded from the left si ( A ) , and from the right side ( B ) . For each HD cell a CW : CCW ratio was calculated by dividing the CW peak firing rate by th CCW peak firing rate . C , D , Representative CW and CCW firing rate versus HD tuning functions for a PoS HD cell ( C ) and an ADN HD cell ( D ) . E , The frequency distribution of CW : CCW ra- tios across both sides ( open bars , lef t side ; filled bars , right side ) for LMN HD cells . F , The frequency distributions of CW : CCW ratios for PoS and ADN cells ( open bars , ADN ; filled bars , PoS ) . Note that the histogram for ADN cells reveals little preference for higher firing rates for CW or CCW turns , whereas the distribution for PoS cells indicates that six cells ( 30 % ) exhibited higher firing rates for CCW turns than CW turns . 9028 J . Neurosci. , November 1 , 1998 , 18 ( 21 ) :9020–9037 Stackman and Taube • Single Unit Analysi of Lateral Mammillary Neurons a b c d e f g h i 0 unit pr ef er re d HD t-SNE HD HD HD HD example neuron 1 from rat area Pos xample neuron 2 r m r t ar a Pos example neuron 3 from rat leJ LMN example neuron 4 from rat right LMN Compass units CW ShiJers CCW ShiJers 10 20 30 180 360 [ deg ] Compass units CW ShiJers CCW ShiJers HD HD head direcTon ( 0 to 360 deg ) fir in g ra te ( s pi ke s/ s ) fir in g ra te ( s pi ke s/ s ) fir in g ra te ( a .u . ) fir in g ra te ( a .u . ) shiJ = 12° s iJ = 12° 0 360 360 0 0 360 360 0 0 50 0 25 0 400 0 90 HD HD HD HD D HD CW/CCW CW/CCW AV AV AV AV joint HD * AV tuning ( Data ) Shi , er Compass unit Figure 2 : Emergence of different functional cell types in the ain d RNN . a ) Joint HD * AV tuning plots for individual units in the RNN . Units are arranged by fu cti nal yp ( Compass , CCW Shifters , CW Shifters ) . The ctivity of each unit is shown as a functi of he d direction ( x-axis ) and angular velocity ( y-axis ) . For each unit , AV ranges from -550 to 550 eg/s , an HD ranges from 0 to 360 deg . b ) Preferred HD of eac unit within each fu ctional type . Ap oxim t ly uniform tiling of preferred HD for each functional type of model neurons is observed . c ) 3D embedding of model neurons using t-SNE , with the distance between two units defined as one minus heir fi ng r t orr lation , xhibits a compass-like structure from one view angle ( left ) and are segr gated according to AV in another view angle ( right ) . Each dot represents one unit . d ) Joint HD * AV tu ing pl ts fo tw xample HD neurons in the rat anterodorsal thalamic nucleus , or ADN ( plotted based on data from Peyrache et al . ( 2015 ) , downloaded from CRCNS website ) . White indicates high firing rate . e ) Joint HD * AV tuning plots for two example neurons from the PB of of the fly central complex , adapted from Turner-Evans et al . ( 2017 ) . Red indicates high firing rate . ( f , g , h , i ) Detailed tuning properties of model neurons match neural data . f ) HD tuning curves for model Compass units exhibit shifted peaks at high CW ( green ) and CCW rotations ( blue ) . g ) HD tuning curves for model Shifters exhibit peak shift and gain changes when comparing CW ( green ) and CCW ( blue ) rotations . h ) HD tuning curves for CW ( solid ) and CCW ( dashed ) conditions for two example neurons in the ostsubiculum of rats , adapted from Stackman & Taube ( 1998 ) . i ) HD tuning curves for CW ( solid ) and CCW ( dashed ) conditions for two example neurons in the lateral mammillary nuclei of rats , adapted from Stackman & Taube ( 1998 ) . Neurons tuned to both HD and AV tuning have also been reported previously in rodents and fruit flies ( Sharp et al. , 2001 ; Stackman & Taube , 1998 ; Bassett & Taube , 2001 ) , although the joint HD * AV tuning profiles of neurons have only been documented anecdotally with a few cells ( Turner-Evans et al . ( 2017 ) ) . In rodents , certain cells are also observed to display HD and AV tuning ( Fig . 2e ) . In addition , in the fruit fly heading system , neurons on the two sides of the protocerebral bridge ( PB ) ( Pfeiffer & Homberg , 2014 ) are also tuned to CW and CCW rotation , respectively , and tile the complete angular space , much like what has been observed in our trained network ( Green et al. , 2017 ; 4 Turner-Evans et al. , 2017 ) . These observations collectively suggest that neurons that are HD but not AV selective in our model can be tentatively mapped to `` Compass '' units in the EB , and the two sub-populations of neurons tuned to both HD and AV map to `` Shifter '' neurons on the left PB and right PB , respectively . We will correspondingly refer to our model neurons as either ‘ Compass ’ units or ‘ CW/CCW Shifters ’ ( Further justification of the terminology will be given in sections 3.2 & 3.3 ) Tuning properties of model neurons match experimental data We next sought to examine the tuning properties of both Compass units and Shifters of our network in greater detail . First , we observe that for both Compass units and Shifters , the HD tuning curve varies as a function of AV ( see example Compass unit in Fig . 2f and example Shifter in Fig . 2g ) . Population summary statistics concerning the amount of tuning shift are shown in Appendix Fig . 7a . The preferred HD tuning is biased towards a more CW angle at CW angular velocities , and vice versa for CCW angular velocities . Consistent with this observation , the HD tuning curves in rodents were also dependent upon AV ( see example neurons in Fig . 2h , i ) ( Blair & Sharp , 1995 ; Stackman & Taube , 1998 ; Taube & Muller , 1998 ; Blair et al. , 1997 ; 1998 ) . Second , the AV tuning curves for the Shifters exhibit graded response profiles , consistent with the measured AV tuning curves in flies and rodents ( see Fig . 1b , d ) . Across neurons , the angular velocity tuning curves show substantial diversity ( see Appendix Fig . 6b ) , also consistent with experimental reports ( Turner-Evans et al. , 2017 ) . In summary , the majority of units in the trained RNN could be mapped onto the biological head direction system in both general functional architecture and also in detailed tuning properties . Our model unifies a diverse set of experimental observations , suggesting that these neural response properties are the consequence of a network solving an angular integration task optimally . 3.2 CONNECTIVITY STRUCTURE OF THE NETWORK 20 40 60 80 Unit ( 33 ring , 29 shiftpos , 26 shiftneg 10 20 30 40 50 60 70 80U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g Training epoch 415 , 2000 trials , 1000 timesteps in simulation errormain = 0.0014384 , normalized error overall = 0.57538 % -1.5 -1 -0.5 0 0.5 1 1.5 W hh a fte r r em ov in g 12 u nr es po ns iv e an d 0 w ea kl y tu ne d un its so rte d by p re fe rre d in te gr at ed -a ng le a b Protocerebral bridge Ellipsoid body angular velocity -180 -90 0 90 180 Preferred integrated-angle relative to shiftpos unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 29 shiftpos units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° Compass units CCW Shi0ers CW Shi0ers Co m pa ss u ni ts CC W S hi 0e rs CW S hi 0e rs -1 -180 -90 0 90 180 Preferred integrated-angle relative to ring unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 33 ring units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° Compass units CCW Shi0ers CW Shi0ers -180 180 0 0 0.5 -0.5 0 0.5 -0.5 0 0.5 -0.5 Co nn ec : v ity Difference in preferred head direc : on ( deg ) 0 0 180 -180 180 -180 would be interesting for future work to und rstand how such tuning diversity can facilit te the angular108 integration.109 3.3 Structural properties of the network110 20 40 60 80 Unit ( 33 ring , 28 shiftpos , 24 shiftneg 10 20 30 40 50 60 70 80U ni t ( 33 ri ng , 2 8 sh ift po s , 2 4 sh ift ne g Training epoch 415 , 2000 trials , 1000 timesteps in simulation errormain = 0.0014384 , normalized error overall = 0.57538 % -1.5 -1 -0.5 0 0.5 1 1.5 W hh a fte r r em ov in g 12 u nr es po ns iv e an d 3 w ea kl y tu ne d un its so rte d by p re fe rre d in te gr at ed -a ng le Ring neurons CCW shi.ers CW shi.ers Ri ng n eu ro ns CC W s hi . er s CW s hi . er s a b 0 -1 1 -180 -90 0 90 180 Preferred integrated-angle relative to ring unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 33 ring units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° Ring neurons CCW shi.ers CW shi.ers -180° 180° 0° 0 0.5 -0.5 -180 -90 0 90 180 Preferred integrated-angle relative to ring unit ( degrees ) -1 -0.5 0 M ea n co nn ec tio n st re ng th ( W hh ) From 33 ring units to 24 shiftneg units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° -.5 0 -1 -180 -90 0 90 180 Preferred integrated-angle relative to shiftneg unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 24 shiftneg units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° 0 0.5 -0.5 -180 -90 0 90 180 Preferred integrated-angle relative to shiftneg unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 24 shiftneg units to 24 shiftneg units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° 0 0.5 -0.5 -180 -90 0 90 180 Preferred integrated-angle relative to ring unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 33 ring units to 28 shiftpos units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° 0 0.5 -0.5 -180 -90 0 90 180 Preferred integrated-angle relative to shiftpos unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 28 shiftpos units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° 0 0.5 -0.5 -180 -90 0 90 180 Preferred integrated-angle relative to shiftpos unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 28 shiftpos units to 28 shiftpos units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° 0 0.5 -0.5 -180 -90 0 90 180 Preferred integrated-angle relative to shiftneg unit ( degrees ) -1 -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 24 shiftneg units to 28 shiftpos units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° 0 0.5 -0.5 -180 -90 0 90 180 Pr ferr d integrated-angle relative to shiftpos unit ( degrees ) -1 -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 28 shiftpos units to 24 shiftneg units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° 0 0.5 -0.5 Figure 4 : Connectivity of the trained network are structured , and exhibit similarity to the connectivity in fly central complex . a ) the connectivity of the network trained with mid-speed . We sort the neuron according to the functional classes , and further arrange them according to the preferred HD with each class . There is visible structural connectivity within and across different cell types . b ) mapping the connectivity structure onto fly central complex . Inserted panels represent the average connectivity ( shading area for one s.d . ) as a function of the difference between preferred HD for within and between different class of neurons . Network with highly structured connectivity have been proposed to perform integration . It ’ s shown111 that these models with hand-crafted connectivity can give rise to the head direction tuning curves [ 17,112 14 , 18 ] . However , it is unclear whether less structured network may also lead to robust angular113 velocity integration , and what connectivity structure would achieve best performance in integration114 of inputs.115 To address these issues , we analyze the structural properties of the trained RNN . We took advantage116 of division of the functional neural types in the trained network , and examined the structure of the117 connectivity matrix with and across each types of cells . Overall , we found that the basic connectivity118 in the network is highly structured albeit with variability , and is qualitatively consistent across the119 different simulations with the same input statistics . Fig . 5 show the detailed connectivity structure.120 Below we highlight some features of the connectivity in light of previous experimental and theoretical121 work.122 First of all , we found the HD neurons exhibit local excitation and long-range inhibition . Second,123 the connectivity from the CW ( or CCW ) shifters to the ring neurons exhibit an asymmetric pattern,124 which excite the HD neurons which tuned to more CW ( CCW ) head direction . This is reminiscent of125 the connectivity pattern discovered in fly between the PB and the Ellipsoid Body , which has been126 proposed to implement a shifting mechanism [ 18 ] . However , our pattern differ from the previous127 proposal in that , not only excitation , but there is also inhibition that ’ s lagging in the direction of the128 movement . Now mention something about the P-EN2 neurons here . Furthermore , the connectivity129 between two sets of shifters also exhibit a specific connectivity structure . In the fly , this might be130 implemented by 7 neurons of the PB . However , in general , lateral connection in the PB as well131 as within the EB are not well studied and documented so far . Based on our modeling results , we132 speculate that systematic EM studies might reveal much richer pattern of connectivity compared to133 the simple connectivity structure which has been proposed/revealed previously.134 -180 -90 0 90 180 Preferred integrated-angle relative to shiftneg unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 26 shiftneg units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° Figure 3 : Connectivity of the trained network is structured and exhibits similarities with the connectivity in the fly central complex . a ) Pixels represent connections from the units in each column to the units in each row . Excitatory connections are in red , and inhibitory connections are in blue . Units are first sorted by functional classes , and then are further sorted by their preferred HD within each class . The black box highlights recurrent connections to the Compass units from Compass units , from CCW Shifters , and from CW Shifters . b ) Ensemble connectivity from each functional cell type to the Compass units as highlighted in a ) , in relation to the architecture of the PB & EB in the fly central complex . Plots show the average connectivity ( shaded area indicates one s.d . ) as a function of the difference between the preferred HD of the cell and the Compass unit it is connecting to . Compass units connect strongly to units with similar HD tuning and inhibit units with dissimilar HD tuning . CCW Shifters connect strongly to Compass units with preferred head directions that are slightly CCW-shifted to its own , and CW Shifters connect strongly to Compass units with preferred head directions that are slightly CW-shifted to its own . Refer to Appendix Fig . 8b for the full set of ensemble connectivity between different classes . 5 Previous experiments have detailed a subset of connections between EB and PB neurons in the fruit fly . We next analyzed the connectivity of Compass units and Shifters in the trained RNN to ask whether it recapitulates these connectivity patterns - a test which has never been done to our knowledge in any system between artificial and biological neural networks ( see Fig . 3 ) . Compass units exhibit local excitation and long-range inhibition We ordered Compass units , CCW Shifters , and CW Shifters by their preferred head direction tuning and plotted their connection strengths ( Fig . 3a ) . This revealed highly structured connectivity patterns within and between each class of units . We first focused on the connections between individual Compass units and observed a pattern of local excitation and global inhibition . Neurons that have similar preferred head directions are connected through positive weights and neurons whose preferred head directions are anti-phase are connected through negative weights ( Fig . 3b ) . This pattern is consistent with the connectivity patterns inferred in recent work based on detailed calcium imaging and optogenetic perturbation experiments ( Kim et al. , 2017 ) , with one caveat that the connectivity pattern inferred in this study is based on the effective connectivity rather than anatomical connectivity . We conjecture that Compass units in the trained RNN serve to maintain a stable activity bump in the absence of inputs ( see section 3.3 ) , as proposed in previous theoretical models ( Turing , 1952 ; Amari , 1977 ; Zhang , 1996 ) . Asymmetric connectivity from Shifters to Compass units We then analyzed the connectivity between Compass units and Shifters . We found that CW Shifters excite Compass units with preferred head directions that are clockwise to its own , and inhibit Compass units with preferred head directions counterclockwise to its own ( Fig . 3b ) . The opposite pattern is observed for CCW Shifters . Such asymmetric connections from Shifters to the Compass units are consistent with the connectivity pattern observed between the PB and the EB in the fruit fly central complex ( Lin et al. , 2013 ; Green et al. , 2017 ; Turner-Evans et al. , 2017 ) , and also in agreement with previously proposed mechanisms of angular integration ( Skaggs et al. , 1995 ; Green et al. , 2017 ; Turner-Evans et al. , 2017 ; Zhang , 1996 ) ( Fig . 3b ) . We note that while the connectivity between PB Shifters and EB Compass units are one-to-one ( Lin et al. , 2013 ; Wolff et al. , 2015 ; Green et al. , 2017 ) , the connectivity profile in our model is broad , with a single CW Shifter exciting multiple Compass units with preferred HDs that are clockwise to its own , and vice versa for CCW Shifters . In summary , the RNN developed several anatomical features that are consistent with structures reported or hypothesized in previous experimental results . A few novel predictions are worth mentioning . First , in our model the connectivity between CW and CCW Shifters exhibit specific recurrent connectivity ( Fig . 8 ) . Second , the connections from Shifters to Compass units exhibit not only excitation in the direction of heading motion , but also inhibition that is lagging in the opposite direction . This inhibitory connection has not been observed in experiments yet but may facilitate the rotation of the neural bump in the Compass units during turning ( Wolff et al. , 2015 ; Franconville et al. , 2018 ; Green et al. , 2017 ; Green & Maimon , 2018 ) . In the future , EM reconstructions together with functional imaging and optogenetics should allow direct tests of these predictions . 3.3 PROBING THE COMPUTATION IN THE NETWORK We have segregated neurons into Compass and Shifter populations according to their HD and AV tuning , and have shown that they exhibit different connectivity patterns that are suggestive of different functions . Compass units putatively maintain the current heading direction and Shifter units putatively rotate activity on the compass according to the direction of angular velocity . To substantiate these functional properties , we performed a series of perturbation experiments by lesioning specific subsets of connections . Perturbation while holding a constant head direction We first lesioned connections when there is zero angular velocity input . Normally , the network maintains a stable bump of activity within each class of neurons , i.e. , Compass units , CW Shifters , and CCW Shifters ( see Fig . 4a , b ) . We first lesioned connections from Compass units to all units and found that the activity bumps in all three classes disappeared and were replaced by diffuse activity in a large proportion of units . As a consequence , the network could not report an accurate estimate of its current heading direction . Furthermore , when the connections were restored , a bump formed a k he ad d ire c * on ( d eg ) * mestep 0 180 360 75 225 C om pa ss C CW C W 75 225 * mestep zero connec * ons from Compass units to all units Storing constant HD zero connec * ons from CW & CCW ShiFers to all units zero connec * ons from CW ShiFers to all units zero connec * ons from CCW ShiFers to all unit ShiFing HD 230 50 zero connec * ons from CCW ShiFers to all units CCW rota * on * mestep 0 180 360 Co m pa ss C CW C W zero connec * ons from CW ShiFers to all units 230 50 * mestep CW rota * on CCW rota * on CW rota * on CCW rota * on CW rota * on 50 230 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 2.7003 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 U ni t a ct iv ity 50 230 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 2.7003 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 U ni t a ct iv ity 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 2.7003 % 50 230 Timestep 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 2.7003 % no perturba * on 50 230 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 123.839 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 U ni t a ct iv ity 50 230 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 123.839 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 U ni t a ct iv ity 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 123.839 % 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 123.839 % 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 153.85 % 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 153.85 % 50 230 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 153.85 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 U ni t a ct iv ity 50 230 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 153.85 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 U ni t a ct iv ity he ad d ire c * on ( d eg ) 75 225 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 75 225 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 0 0.5 1 U ni t a ct iv ity start end ( of perturba * on ) * mestep l m n o p q r s t u v no perturba * on b c d e f g h i j 75 225 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 75 225 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 0 0.5 1 U ni t a ct iv ity 75 225 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 0 0.5 1 U ni t a ct iv ity 75 225 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 75 225 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 75 225 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 0 0.5 1 U ni t a ct iv ity 75 225 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 75 225 Timestep U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 1 trials = Inf % 0 0.5 1 U ni t a ct iv ity start end ( of perturba * on ) * mestep Under review as a conference paper at ICLR 2020 0 0.5 1 Un it ac tiv ity 0 180 360 Target output RNN output Target output RNN output Target output RNN output Target output RNN output 75 225 timestep 50 230 Timestep 0 18 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 2.7003 % 50 230 Timestep Un it ( 3 3 rin g , 2 8 sh ift po s , 2 4 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 2.7003 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Un it ac tiv ity 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 2.7003 % 50 230 Timestep Un it ( 3 3 rin g , 2 8 sh ift po s , 2 4 sh ift ne g ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 2.7003 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Un it ac tiv ity 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 123.058 % 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 123.058 % 50 230 Timestep Un it ( 3 3 rin g , 2 8 sh ift po s , 2 4 sh ift ne g ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 123.058 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Un it ac tiv ity 50 230 Timestep Un it ( 3 3 rin g , 2 8 sh ift po s , 2 4 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 123.058 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Un it ac tiv ity 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 186.736 % 50 230 Timestep 0 180 360 An gl e ( d eg re es ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 186.736 % 50 230 Timestep Un it ( 3 3 rin g , 2 8 sh ift po s , 2 4 sh ift ne g ) Trial 1 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 186.736 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Un it ac tiv ity 50 230 Timestep Un it ( 3 3 rin g , 2 8 sh ift po s , 2 4 sh ift ne g ) Trial 2 , epoch 415 , 300 timesteps in simulation normalized error for all 2 trials = 186.736 % -0.2 0 0.2 0.4 0.6 0.8 1 1.2 Un it ac tiv ity he ad d ire ct io n [ d eg ] Storing constant HD Shifting HD Ri ng CC W CW 50 230 timestep 0 180 360 zero connections from Ring units to all units zero connections from CW Shifters to all units zero on ections from CCW Shifters to all units zero connections from CW & CCW Shifters to all units zero connections from CW Shifters o all units o pertubati n no pertubation zero connections from CCW Shifters to all units CCW rotation CW rotation CCW rotation CW rotation CCW rotation CW rotation Ri ng CC W CW Ri ng CC W CW a c b d e f g h i j k l m n o p q r s t u v start end ( of perturbation ) start end ( of perturbation ) timestep timestep Figure 4 : Probing the functional role of different classes of model neurons . a-j ) P rturbation an lysis in the case of maintaining a constant HD . a ) Under normal conditions , the RNN output matches the target output . b ) Population activity for the trial shown in a ) , sorted by the preferred HD and the class of each unit , i.e. , Ring units , CCW Shifters , CW Shifters . c-j ) RNN output and population activity when a specific set f connections are set to zero during the period indicated by blue arrows in i ) and j ) . k-v ) Perturbation analysis in case of a shifting HD . For each manipulation , CW rotation and CCW rotation are tested , resulting in two trials . k-n ) Normal case without any perturbation . o-v ) RNN output and population activity when connections from CW and CCW Shifters are set to zero during the period indicated by blue arrows in u ) and v ) . Refer to the main text for the interpretation of the results . Perturbation while integrating constant angular velocity We next lesioned connections during either constant CW or CCW angular velocity . Normally , the network can integrate AV accurately ( Fig . 4k-n ) . As expected , during CCW rotation , we observe a corresponding rotation of the activity bump in Ring units and in CCW Shifters , but CW Shifters display low levels of activity . The converse is true during CW rotation . We first lesioned connections from CW Shifters to all units , and found that it significantly impaired rotation in the CW direction , and also increased the rotation speed in the CCW direction . Lesioning of CCW Shifters to all units had the opposite effect , significantly impairing rotation in the CCW direction . These results are consistent with the hypothesis that CW/CCW Shifters are responsible for shifting the bump in a CW and CCW direction , respectively , and are consistent with the data in Green et al . ( 2017 ) , which shows that inhibition of Shifter units in the PB of the fruit fly heading system impairs the integration of HD . Our lesion experiments further support the segregation of units into modular components that function to separately maintain and update heading during angular motion . 7 arget output N o tput We then lesioned connections from CW Shifters to all units and found that all three bumps exhibit a CCW rotation , and the read-out units correspondingly reported a CCW rotation of heading direction ( Fig . 4e , f ) . Analogous results were obtained with lesions of CCW Shifters , which resulted in a CW drifting bump of activity ( Fig . 4g , h ) . These results are consistent with the hypothesis that CW and CCW Shifters simultaneously activate the compass , with mutually cancelling signals , even when the heading direction is stationary . When connections are lesioned from both CW and CCW Shifters to all units , we observe that Compass units are still capable of holding a stable HD activity bump ( Fig . 4i , j ) , consistent with the predictions that while CW/CCW Shifters are necessary for updating heading during motion , Compass units are responsible for maintaining heading . Perturbation while integrating constant angular velocity We next lesioned connections during either constant CW or CCW angular velocity . Normally , the network can integrate AV accurately ( Fig . 4k-n ) . As expected , during CCW rotation , we observe a corresponding rotation of the activity bump in Compass units and in CCW Shifters , but CW Shifters display low levels of activity . The converse is true during CW rotation . We first lesioned connections from CW Shifters to all units , and found that it significantly impaired rotation in the CW direction , and also increased the rotation speed in the CCW direction . Lesioning of CCW Shifters to all units had the opposite effect , significantly impairing rotation in the CCW direction . These results are consistent with the hypothesis that CW/CCW Shifters are responsible for shifting the bump in a CW and CCW direction , respectively , and are consistent with the data in Green et al . ( 2017 ) , which shows that inhibition of Shifter units in the PB of the fruit fly heading system impairs the integration of HD . Our lesion experiments further support the segregation of units into modular components that function to separately maintain and update heading during angular motion . -20 0 20 Angular input ( -1.33 to 1.43 degrees ) Distribution of angular inputs to RNN 500 training trials , 500 timesteps distribu ( on of AV -0.04 0.04 0 -0.04 -0.02 0 0.02 0.04 0.06Slope ( firing rate/degree ) 0 5 10 N um be r o f u ni ts Training epoch 415 , 2000 trials , 1000 timesteps in simulation errormain = 0.0014384 , normalized error overall = 0.57538 % abs median/mean of 88 units that do not have constant activity = 0.0173355/0.0175643 angular velocity slope a b h distribu ( on of AV distribu ( on of AV histogram of slope of AV tuning 0 -800 800 angular velocity -0.03 -0.02 -0.01 0 0.01 0.02 0.03 Slope ( firing rate/degree ) 0 2 4 6 N um be r o f u ni ts Training epoch 380 , 2000 trials , 1000 timeste s in simulation errormain = 0.0089381 , normalized error overall = 3.5752 % abs median/mean of 90 units that do not have constant activity = 0.0120826/0.0115089 -0.02 0.02 0 slope angular velocity ( deg/s ) fr eq ue nc y -0.05 0.05 0 -0.05 0 0.05 0.1Slope ( firing rate/degree ) 0 5 10 N um be r o f u ni ts Training epoch 500 , 2000 trials , 1000 timesteps in simulation errormain = 0.0015722 , normalized error overall = 0.62978 % abs median/mean of 61 units that do not have constant activity = 0.00247316/0.00942536 nu m be r o f u ni ts slope ( a.u . ) -20 0 20 Angular input ( -39.8 to 43 degrees ) Distribution of angular inputs to RNN 500 training trials , 500 timesteps histogra of slope of AV tuning histogram of slop of AV uning Low AV Medium AV ( Main condi1on ) High AV -20 0 20 Angular input ( -13.3 to 14.3 degrees ) Distribution of angular inputs to RNN 500 training trials , 500 timesteps 0 -800 800 0 -800 800 d e g 4 ADAPTATION OF NETWORK PROPERTIES TO INPUT STATISTICS Optimal computation requires the system to adapt to the statistical structure of the inputs ( Barlow , 1961 ; Attneave , 1954 ) . In order to understand how the statistical properties of the input trajectories affect how a network solves the task , we trained RNNs to integrate inputs generated from low and high AV distributions . When networks are trained with small angular velocities , we observe the presence of more units with strong head direction tuning but minimal angular velocity tuning . Conversely , when networks are trained with large AV inputs , fewer Compass units emerge and more units become Shifter-like and exhibit both HD and AV tuning ( Fig . 5c , f , i ) . We sought to quantify the overall AV tuning under each velocity regime by computing the slope of each neuron ’ s AV tuning curve at its preferred HD angle . We found that by increasing the magnitude of AV inputs , more neurons developed strong AV tuning ( Fig . 5b , e , h ) . In summary , with a slowly changing head direction trajectory , it is advantageous to allocate more resources to hold a stable activity bump , and this requires more Compass units . In contrast , with quickly changing inputs , the system must rapidly update the activity bump to integrate head direction , requiring more Shifter units . This prediction may be relevant for understanding the diversity of the HD systems across different animal species , as different species exhibit different overall head turning behavior depending on the ecological demand ( Stone et al. , 2017 ; Seelig & Jayaraman , 2015 ; Heinze , 2017 ; Finkelstein et al. , 2018 ) . 5 DISCUSSION Previous work in the sensory systems have mainly focused on obtaining an optimal representation ( Barlow , 1961 ; Laughlin , 1981 ; Linsker , 1988 ; Olshausen & Field , 1996 ; Simoncelli & Olshausen , 2001 ; Yamins et al. , 2014 ; Khaligh-Razavi & Kriegeskorte , 2014 ) with feedforward models . Several recent studies have probed the importance of recurrent connections in understanding neural computation by training RNNs to perform tasks ( e.g. , Mante et al . ( 2013 ) ; Sussillo et al . ( 2015 ) ; Cueva & Wei ( 2018 ) ) , but the relation of these trained networks to the anatomy and function of brain circuits are not mapped . Using the head direction system , we demonstrate that goal-driven optimization of recurrent neural networks can be used to understand the functional , structural and mechanistic properties of neural circuits . While we have mainly used perturbation analysis to reveal the dynamics of the trained RNN , other methods could also be applied to analyze the network . For example , in Appendix Fig . 10 , using fixed point analysis ( Sussillo & Barak , 2013 ; Maheswaranathan et al. , 2019 ) , we found evidence consistent with attractor dynamics . Due to the limited amount of experimental data available , comparisons regarding tuning properties and connectivity are largely qualitative . In the future , studies of the relevant brain areas using Neuropixel probes ( Jun et al. , 2017 ) and calcium imaging ( Denk et al. , 1990 ) will provide a more in-depth characterization of the properties of HD circuits , and will facilitate a more quantitative comparison between model and experiment . In the current work , we did not impose any additional structural constraint on the RNNs during training , asides from prohibiting self-connections . We have chosen to do so in order to see what structural properties would emerge as a consequence of optimizing the network to solve the task . It is interesting to consider how additional structural constraints affect the representation and computation in the trained RNNs . One possibility would to be to have the input or output units only connect to a subset of the RNN units . Another possibility would be to freeze a subset of connections during training . Future work should systematically explore these issues . Recent work suggests it is possible to obtain tuning properties in RNNs with random connections ( Sederberg & Nemenman , 2019 ) . We found that training was necessary for the joint HD * AV tuning ( see Appendix Fig . 9 ) to emerge . While Sederberg & Nemenman ( 2019 ) consider a simple binary classification task , our integration task is computationally more complicated . Stable HD tuning requires the system to keep track of HD by accurate integration of AV , and to stably store these values over time . This computation might be difficult for a random network to perform and , more generally , completely random networks may lead to different memory representations than the attractor geometry we observe in Fig . 10 ( Cueva et al. , 2019 ) . Our approach contrasts with previous network models for the HD system , which are based on hand-crafted connectivity ( Zhang , 1996 ; Skaggs et al. , 1995 ; Xie et al. , 2002 ; Green et al. , 2017 ; Kim et al. , 2017 ; Knierim & Zhang , 2012 ; Song & Wang , 2005 ; Kakaria & de Bivort , 2017 ; Stone et al. , 2017 ) . Our modeling approach optimizes for task performance through stochastic gradient descent . We found that different input statistics lead to different heading representations in an RNN , suggesting that the optimal architecture of a neural network varies depending on the task demand - an insight that would be difficult to obtain using the traditional approach of hand-crafting network solutions . Although we have focused on a simple integration task , this framework should be of general relevance to other neural systems as well , providing a new approach to understand neural computation at multiple levels . Our model may be used as a building block for AI systems to perform general navigation ( Pei et al. , 2019 ) . In order to effectively navigate in complex environments , the agent would need to construct a cognitive map of the surrounding environment and update its own position during motion . A circuit that performs heading integration will likely be combined with another circuit to integrate the magnitude of motion ( speed ) to perform dead reckoning . Training RNNs to perform more challenging navigation tasks such as these , along with multiple sources of inputs , i.e. , vestibular , visual , auditory , will be useful for building robust navigational systems and for improving our understanding of the computational mechanisms of navigation in the brain ( Cueva & Wei , 2018 ; Banino et al. , 2018 ) . ACKNOWLEDGMENTS Research supported by NSF NeuroNex Award DBI-1707398 and the Gatsby Charitable Foundation . We would like to thank Kenneth Kay for careful reading of an earlier version of the paper , and Rong Zhu for help preparing panel d in Figure 2 . REFERENCES Shun-ichi Amari . Dynamics of pattern formation in lateral-inhibition type neural fields . Biological cybernetics , 27 ( 2 ) :77–87 , 1977 . Fred Attneave . Some informational aspects of visual perception . Psychological review , 61 ( 3 ) :183 , 1954 . Andrea Banino , Caswell Barry , Benigno Uria , Charles Blundell , Timothy Lillicrap , Piotr Mirowski , Alexander Pritzel , Martin J Chadwick , Thomas Degris , Joseph Modayil , et al . Vector-based navigation using grid-like representations in artificial agents . Nature , 557 ( 7705 ) :429 , 2018 . Horace B Barlow . Possible principles underlying the transformation of sensory messages . Sensory communication , pp . 217–234 , 1961 . Joshua P Bassett and Jeffrey S Taube . Neural correlates for angular head velocity in the rat dorsal tegmental nucleus . Journal of Neuroscience , 21 ( 15 ) :5740–5751 , 2001 . John A Bender and Michael H Dickinson . A comparison of visual and haltere-mediated feedback in the control of body saccades in drosophila melanogaster . Journal of Experimental Biology , 209 ( 23 ) :4597–4606 , 2006 . Hugh T Blair and Patricia E Sharp . Anticipatory head direction signals in anterior thalamus : evidence for a thalamocortical circuit that integrates angular head motion to compute head direction . Journal of Neuroscience , 15 ( 9 ) :6260–6270 , 1995 . Hugh T Blair , Brian W Lipscomb , and Patricia E Sharp . Anticipatory time intervals of head-direction cells in the anterior thalamus of the rat : implications for path integration in the head-direction circuit . Journal of neurophysiology , 78 ( 1 ) :145–159 , 1997 . Hugh T Blair , Jeiwon Cho , and Patricia E Sharp . Role of the lateral mammillary nucleus in the rat head direction circuit : a combined single unit recording and lesion study . Neuron , 21 ( 6 ) : 1387–1397 , 1998 . Charles F Cadieu , Ha Hong , Daniel LK Yamins , Nicolas Pinto , Diego Ardila , Ethan A Solomon , Najib J Majaj , and James J DiCarlo . Deep neural networks rival the representation of primate it cortex for core visual object recognition . PLoS computational biology , 10 ( 12 ) : e1003963 , 2014 . Christopher J Cueva and Xue-Xin Wei . Emergence of grid-like representations by training recurrent neural networks to perform spatial localization . ICLR , 2018 . Christopher J Cueva , Alex Saez , Encarni Marcos , Aldo Genovesio , Mehrdad Jazayeri , Ranulfo Romo , C Daniel Salzman , Michael N Shadlen , and Stefano Fusi . Low dimensional dynamics for working memory and time encoding . bioRxiv doi : 10.1101/504936 , 2019 . Winfried Denk , James H Strickler , and Watt W Webb . Two-photon laser scanning fluorescence microscopy . Science , 248 ( 4951 ) :73–76 , 1990 . Ariane S Etienne and Kathryn J Jeffery . Path integration in mammals . Hippocampus , 14 ( 2 ) :180–192 , 2004 . Arseny Finkelstein , Dori Derdikman , Alon Rubin , Jakob N Foerster , Liora Las , and Nachum Ulanovsky . Three-dimensional head-direction coding in the bat brain . Nature , 517 ( 7533 ) :159 , 2015 . Arseny Finkelstein , Nachum Ulanovsky , Misha Tsodyks , and Johnatan Aljadeff . Optimal dynamic coding by mixed-dimensionality neurons in the head-direction system of bats . Nature communications , 9 ( 1 ) :3590 , 2018 . Romain Franconville , Celia Beron , and Vivek Jayaraman . Building a functional connectome of the drosophila central complex . Elife , 7 : e37017 , 2018 . Jonathan Green and Gaby Maimon . Building a heading signal from anatomically defined neuron types in the drosophila central complex . Current opinion in neurobiology , 52:156–164 , 2018 . Jonathan Green , Atsuko Adachi , Kunal K Shah , Jonathan D Hirokawa , Pablo S Magani , and Gaby Maimon . A neural circuit architecture for angular integration in drosophila . Nature , 546 ( 7656 ) : 101 , 2017 . Umut Güçlü and Marcel AJ van Gerven . Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream . Journal of Neuroscience , 35 ( 27 ) :10005–10014 , 2015 . Stanley Heinze . Unraveling the neural basis of insect navigation . Current opinion in insect science , 24:58–67 , 2017 . James J Jun , Nicholas A Steinmetz , Joshua H Siegle , Daniel J Denman , Marius Bauza , Brian Barbarits , Albert K Lee , Costas A Anastassiou , Alexandru Andrei , Çağatay Aydın , et al . Fully integrated silicon probes for high-density recording of neural activity . Nature , 551 ( 7679 ) :232 , 2017 . Kyobi S Kakaria and Benjamin L de Bivort . Ring attractor dynamics emerge from a spiking model of the entire protocerebral bridge . Frontiers in behavioral neuroscience , 11:8 , 2017 . Seyed-Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte . Deep supervised , but not unsupervised , models may explain it cortical representation . PLoS computational biology , 10 ( 11 ) : e1003915 , 2014 . Sung Soo Kim , Hervé Rouault , Shaul Druckmann , and Vivek Jayaraman . Ring attractor dynamics in the drosophila central brain . Science , 356 ( 6340 ) :849–853 , 2017 . D P Kingma and J L Ba . Adam : a method for stochastic optimization . International Conference on Learning Representations , 2015 . James J Knierim and Kechen Zhang . Attractor dynamics of spatially correlated neural activity in the limbic system . Annual review of neuroscience , 35:267–285 , 2012 . Nikolaus Kriegeskorte . Deep neural networks : a new framework for modeling biological vision and brain information processing . Annual Review of Vision Science , 1:417–446 , 2015 . Simon Laughlin . A simple coding procedure enhances a neuron ’ s information capacity . Zeitschrift für Naturforschung c , 36 ( 9-10 ) :910–912 , 1981 . Chih-Yung Lin , Chao-Chun Chuang , Tzu-En Hua , Chun-Chao Chen , Barry J Dickson , Ralph J Greenspan , and Ann-Shyn Chiang . A comprehensive wiring diagram of the protocerebral bridge for visual information processing in the drosophila brain . Cell reports , 3 ( 5 ) :1739–1753 , 2013 . Ralph Linsker . Self-organization in a perceptual network . Computer , 21 ( 3 ) :105–117 , 1988 . Yave Roberto Lozano , Hector Page , Pierre-Yves Jacob , Eleonora Lomi , James Street , and Kate Jeffery . Retrosplenial and postsubicular head direction cells compared during visual landmark discrimination . Brain and neuroscience advances , 1:2398212817721859 , 2017 . Niru Maheswaranathan , Alex H Williams , Matthew D Golub , Surya Ganguli , and David Sussillo . Universality and individuality in neural dynamics across large populations of recurrent networks . arXiv preprint arXiv:1907.08549 , 2019 . Valerio Mante , David Sussillo , Krishna V Shenoy , and William T Newsome . Context-dependent computation by recurrent dynamics in prefrontal cortex . Nature , 503 ( 7474 ) :78–84 , 2013 . James Martens and Ilya Sutskever . Learning recurrent neural networks with hessian-free optimization . pp . 1033–1040 , 2011 . Bruno A Olshausen and David J Field . Emergence of simple-cell receptive field properties by learning a sparse code for natural images . Nature , 381 ( 6583 ) :607 , 1996 . A Emin Orhan and Wei Ji Ma . A diverse range of factors affect the nature of neural representations underlying short-term memory . Nature neuroscience , 22 ( 2 ) :275 , 2019 . Jing Pei , Lei Deng , Sen Song , Mingguo Zhao , Youhui Zhang , Shuang Wu , Guanrui Wang , Zhe Zou , Zhenzhi Wu , Wei He , et al . Towards artificial general intelligence with hybrid tianjic chip architecture . Nature , 572 ( 7767 ) :106–111 , 2019 . Adrien Peyrache , Marie M Lacroix , Peter C Petersen , and György Buzsáki . Internally organized mechanisms of the head direction sense . Nature neuroscience , 18 ( 4 ) :569 , 2015 . Keram Pfeiffer and Uwe Homberg . Organization and functional roles of the central complex in the insect brain . Annual review of entomology , 59:165–184 , 2014 . Florian Raudies and Michael E Hasselmo . Modeling boundary vector cell firing given optic flow as a cue . PLoS computational biology , 8 ( 6 ) : e1002553 , 2012 . Evan D Remington , Devika Narain , Eghbal A Hosseini , and Mehrdad Jazayeri . Flexible sensorimotor computations through rapid reconfiguration of cortical dynamics . Neuron , 2018 . Audrey J Sederberg and Ilya Nemenman . Randomly connected networks generate emergent selectivity and predict decoding properties of large populations of neurons . arXiv preprint arXiv:1909.10116 , 2019 . Johannes D Seelig and Vivek Jayaraman . Neural dynamics for landmark orientation and angular path integration . Nature , 521 ( 7551 ) :186 , 2015 . Patricia E Sharp , Hugh T Blair , and Jeiwon Cho . The anatomical and computational basis of the rat head-direction cell signal . Trends in neurosciences , 24 ( 5 ) :289–294 , 2001 . Eero P Simoncelli and Bruno A Olshausen . Natural image statistics and neural representation . Annual review of neuroscience , 24 ( 1 ) :1193–1216 , 2001 . William E Skaggs , James J Knierim , Hemant S Kudrimoti , and Bruce L McNaughton . A model of the neural basis of the rat ’ s sense of direction . In Advances in neural information processing systems , pp . 173–180 , 1995 . H Francis Song , Guangyu R Yang , and Xiao-Jing Wang . Training excitatory-inhibitory recurrent neural networks for cognitive tasks : a simple and flexible framework . PLoS computational biology , 12 ( 2 ) : e1004792 , 2016 . Pengcheng Song and Xiao-Jing Wang . Angular path integration by moving “ hill of activity ” : a spiking neuron model without recurrent excitation of the head-direction system . Journal of Neuroscience , 25 ( 4 ) :1002–1014 , 2005 . Robert W Stackman and Jeffrey S Taube . Firing properties of rat lateral mammillary single units : head direction , head pitch , and angular head velocity . Journal of Neuroscience , 18 ( 21 ) :9020–9037 , 1998 . Thomas Stone , Barbara Webb , Andrea Adden , Nicolai Ben Weddig , Anna Honkanen , Rachel Templin , William Wcislo , Luca Scimeca , Eric Warrant , and Stanley Heinze . An anatomically constrained model for path integration in the bee brain . Current Biology , 27 ( 20 ) :3069–3085 , 2017 . David Sussillo and Omri Barak . Opening the black box : low-dimensional dynamics in highdimensional recurrent neural networks . Neural computation , 25 ( 3 ) :626–649 , 2013 . David Sussillo , Mark M Churchland , Matthew T Kaufman , and Krishna V Shenoy . A neural network that finds a naturalistic solution for the production of muscle activity . Nature neuroscience , 18 ( 7 ) : 1025–1033 , 2015 . Jeffrey S Taube . Head direction cells recorded in the anterior thalamic nuclei of freely moving rats . Journal of Neuroscience , 15 ( 1 ) :70–86 , 1995 . Jeffrey S Taube and Robert U Muller . Comparisons of head direction cell activity in the postsubiculum and anterior thalamus of freely moving rats . Hippocampus , 8 ( 2 ) :87–108 , 1998 . Jeffrey S Taube , Robert U Muller , and James B Ranck . Head-direction cells recorded from the postsubiculum in freely moving rats . i. description and quantitative analysis . Journal of Neuroscience , 10 ( 2 ) :420–435 , 1990a . Jeffrey S Taube , Robert U Muller , and James B Ranck . Head-direction cells recorded from the postsubiculum in freely moving rats . ii . effects of environmental manipulations . Journal of Neuroscience , 10 ( 2 ) :436–447 , 1990b . Alan Turing . The chemical basis of morphogenesis . Philosophical Transactions of the Royal Society of London . Series B , Biological Sciences , 237 ( 641 ) :37–72 , 1952 . Daniel Turner-Evans , Stephanie Wegener , Herve Rouault , Romain Franconville , Tanya Wolff , Johannes D Seelig , Shaul Druckmann , and Vivek Jayaraman . Angular velocity integration in a fly heading circuit . Elife , 6 : e23496 , 2017 . Jing Wang , Devika Narain , Eghbal A Hosseini , and Mehrdad Jazayeri . Flexible timing by temporal scaling of cortical responses . Nature Neuroscience , 2018 . Tanya Wolff , Nirmala A Iyer , and Gerald M Rubin . Neuroarchitecture and neuroanatomy of the drosophila central complex : A gal4-based dissection of protocerebral bridge neurons and circuits . Journal of Comparative Neurology , 523 ( 7 ) :997–1037 , 2015 . Xiaohui Xie , Richard HR Hahnloser , and H Sebastian Seung . Double-ring network model of the head-direction system . Physical Review E , 66 ( 4 ) :041902 , 2002 . Daniel LK Yamins and James J DiCarlo . Using goal-driven deep learning models to understand sensory cortex . Nature neuroscience , 19 ( 3 ) :356–365 , 2016 . Daniel LK Yamins , Ha Hong , Charles F Cadieu , Ethan A Solomon , Darren Seibert , and James J DiCarlo . Performance-optimized hierarchical models predict neural responses in higher visual cortex . Proceedings of the National Academy of Sciences , 111 ( 23 ) :8619–8624 , 2014 . Guangyu Robert Yang , Madhura R Joglekar , H Francis Song , William T Newsome , and Xiao-Jing Wang . Task representations in neural networks trained to perform many cognitive tasks . Nature neuroscience , 22 ( 2 ) :297 , 2019 . Kechen Zhang . Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble : a theory . Journal of Neuroscience , 16 ( 6 ) :2112–2126 , 1996 . A APPENDIX -180 -90 0 90 180 Preferred integrated-angle relative to ring unit ( degrees ) -1 -0.5 0 M ea n co nn ec tio n st re ng th ( W hh ) From 33 ring units to 26 shiftneg units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° -180 -90 0 90 180 Preferred integrated-angle relative to ring unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 33 ring units to 29 shiftpos units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° Compass units CCW Shi/ers CW Shi/ers Co m pa ss u ni ts CC W S hi /e rs CW S hi /e rs a b 0 -1 1 -180 -90 0 90 180 Preferred integrated-angle relative to ring unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 33 ring units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° CCW Shi/ers CW Shi/ers -180 180 0 0 0.5 -0.5 -.5 0 -1 0 0.5 -0.5 0 0.5 -0.5 0 0.5 -0.5 Co nn ec : v ity Difference in preferred head direc : on ( deg ) -0.5 0.5 0 0.5 -0.5 0 0 0.5 -0.5 -180 180 0 0 0.5 -0.5 -180 180 0 Compass units 20 40 60 80 Unit ( 33 ring , 29 shiftpos , 26 shiftneg 10 20 30 40 50 60 70 80U ni t ( 33 ri ng , 2 9 sh ift po s , 2 6 sh ift ne g Training epoch 415 , 2000 trials , 1000 timesteps in simulation errormain = 0.0014384 , normalized error overall = 0.57538 % -1.5 - -0.5 0.5 1.5 W hh a fte r r em ov in g 12 u nr es po ns iv e an d 0 w ea kl y tu ne d un its so rte d by p re fe rre d in te gr at ed -a ng le -180 -90 0 90 180 Preferred integrated-angle relative to shiftpos unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 29 shiftpos units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° -180 -90 0 90 180 Preferred integrated-angle relative to shiftneg unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 26 shiftneg units to 33 ring units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° -180 -90 0 90 180 Preferred integrated-angle relative to shiftpos unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 29 shiftpos units to 29 shiftpos units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° -180 -90 0 90 180 Preferred integrated-angle relative to shiftneg unit ( degrees ) -1 -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 26 shiftneg units to 29 shiftpos units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° -180 -90 0 90 180 Pref rred integrated-angle relative to shiftpos unit ( degrees ) -1 -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 29 shiftpos units to 26 shiftneg units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° -180 -90 0 90 180 Preferred integrated-angle relative to shiftneg unit ( degrees ) -0.5 0 0.5 M ea n co nn ec tio n st re ng th ( W hh ) From 26 shiftneg units to 26 shiftneg units Training epoch 415 , 2000 trials , 1000 timesteps in simulation normalized error overall = 0.57538 % , 10° bins slid by 1° Figure 8 : Connectivity of the trained network . a ) Pixels represent connections from the units in each column to the units in each row . Units are first sorted by functional classes , and then are further sorted by their preferred HD within each class . Excitatory connections are in red , and inhibitory connections are in blue . b ) Ensemble connectivity from each functional cell type . Plots show the average connectivity ( shaded area indicates one standard deviation ) as a function of the difference in preferred HD . Published as a conference paper at ICLR 2020 360 ) Trial 25 , training epoch 415 , 1000 timesteps in simulation errormain = 0.0062855 , normalized error overall = 2.6264 % 0 180 360 H ea d di re ct io n ( d eg re es ) Trial 136 , training epoch 415 , 1000 timesteps in simulation errormain = 0.0062855 , normalized error overall = 2.5142 % Timesteps 500 1000 Timesteps 0 180 360 He ad d ire ct io n ( d eg re es ) Trial 25 , training epoch 415 , 1000 timesteps in simulation errormain = 0.0062855 , normalized error overall = 2.6264 %

Summary: This paper examines head direction representations in a RNN. The RNN was trained to report current head direction using initial head direction and angular velocity information as inputs. The authors compare the representations in the RNN to the representations found in the head direction systems of mice and flies. They find that the representations and connectivity matrices of the RNN recapitulate many aspects of the real brains’ head direction systems. These include the presence of cells tuned only for head direction (“ring cells”) and cells with combined head direction and angular velocity tuning (“shift cells”). These cells tile the space in a functional torus structure, as revealed by t-SNE, which resembles the structures seen in real brains. Similarly, the connectivity profiles and functional role of these cell types matches what is observed in the brains. Altogether, these results demonstrate that RNNs optimized for this task can capture both the functional and anatomical aspects of the systems found in animals. This validates the use of RNNs for studying these systems. In my opinion, it also suggests that information about the structure of brains could inform strong priors for ANNs.