Title: Brain development dictates energy constraints on neural architecture search: cross-disciplinary insights on optimization strategies
Venue: NeurIPS 2023
Accepted: no
Keywords: NAS

brain development

glial-neural networks

metabolic optimization

dynamic coordination
Abstract: Today’s artificial neural architecture search (NAS) strategies are essentially prediction-error-optimized. That principle also holds true for AI functions in general. From the developmental neuroscience perspective, I present evidence for the central role of energy-cost-, rather than prediction-error-, optimized neural architecture search (NAS). Supporting evidence is drawn from the latest insights into the glial-neural organization of the human brain and the dynamic coordination theory which provides a mathematical foundation for the functional expression of this optimization strategy. This is relevant to devising novel NAS strategies in AI, especially in AGI. Additional implications arise for causal reasoning from deep neural nets. Together, these insights from developmental neuroscience offer a new perspective on NAS and the foundational assumptions in AI modeling.

1

Abstract

Today's artificial neural architecture search (NAS) strategies are essentially prediction-error-optimized. That holds true for AI functions in general. From the developmental neuroscience perspective, I present evidence for the central role of metabolically, rather than prediction-error-, optimized neural architecture search (NAS). Supporting evidence is drawn from the latest insights into the glial-neural organization of the human brain and the dynamic coordination theory which provides a mathematical foundation for the functional expression of this optimization strategy. This is relevant to devising novel NAS strategies in AI, especially in AGI. Additional implications arise for causal reasoning from deep neural nets. Together, the insights from developmental neuroscience offer a new perspective on NAS and the foundational assumptions in AI modeling.

1 Introduction

This work is written by a neuroscientist, not a computer scientist. Apologies in advance for any 13

naïveté. This article is theoretical in nature and based on evidence from developmental neuroscience. 14

The aim is to generate new testable hypotheses about the design of artificial intelligence (AI) from 15

the physiological point of view. The manuscript is organized as follows. First, I briefly review the 16

general process of the neural architecture search (NAS). Second, I discuss the neuroanatomical and 17

neurophysiological evidence for glial-neural network architecture. Third, I add evidence for the 18

fundamental role of metabolic constraints in such brain architectures for brain development, i.e., 19

biological NAS, and brain function, i.e., multimodal multitask deep neural networks. Fourth, from the 20

perspective of physics, we consider the dynamic coordination theory which provides a mathematical 21

bridge between the energy-driven coordination of weakly coupled non-linear oscillators and brain 22

networks. Finally, we synthesize these insights into a revised version of the NAS process. The 23

manuscript is concluded with a discussion of limitations and future directions. 24

25

2 From artificial to brain-inspired NAS

2.1 Neural architecture search: status quo 26

For the context of the present work, it is important to summarize the steps of NAS. I reproduce these 27

steps as defined by Elsken et al. [2019]. The credit for these steps goes entirely to the cited authors. 28

Briefly, the search space is defined and then explored using search strategies in an iterative manner as 29

part of the performance estimation strategy. 30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

The search space defines which architectures can be represented in principle. Incorporating prior knowledge about typical properties of architectures well-suited for a task can reduce the size of the search space and simplify the search. However, this also introduces a human bias, which may prevent finding novel architectural building blocks that go beyond the current human knowledge.

The search strategy details how to explore the search space. In the conventional view of NAS, the search space can be exponentially large or even unbounded, while in the proposed energy-constrained view of NAS, this is not the case. It encompasses the classical exploration-exploitation trade-off since, on the one hand, it is desirable to find wellperforming architectures quickly, while on the other hand, premature convergence to a region of suboptimal architectures should be avoided.

The conventional objective of NAS is typically to find architectures that achieve high predictive performance on unseen data. Performance estimation represents the process of estimating the predictive performance of a deep neural net. The simplest option is to perform standard training and validation of the architecture on data, but this is unfortunately computationally expensive and limits the number of architectures that can be explored. Much recent research, therefore, focuses on developing methods that reduce the cost of these performance estimations.

In each of these three steps, I show in the following how the energy-driven glial-neuronal NAS approach can provide useful constraints. To do that, we first need to review the evidence from neuroscience and dynamic coordination theory that formalizes mathematically the relationship between energy optimization and network communication.

2.2 Energy-constrained glial-neural nets

The early notion that neurons compute to effectively reduce the prediction error does not hold true in biological brains, especially in the human brain on two accounts: first, neurons are dynamically and spatiotemporally organized with as numerous glial cells to perform their functions, and, second, these functions are energy-constrained first and, most likely, prediction-error-minimizing second. We observe the paramount role of metabolic optimization, i.e., neural nets develop and are sustained under energy constraints. It is then questionable that error prediction should be the primary objective of a deep neural net evolution and function. This has implications throughout the NAS. In the following, I review some evidence that biological parallels of deep neural nets are in fact glial-neural nets in which energy optimization drives the NAS.

First, neurons should not continue to be viewed as the sole foundation for biologically inspired AI. In fact, when it comes to brain development and function in health and disease, they are at best an equal partner of the glial cells, with a complex, spatiotemporally dynamic relationship between these two major cell categories. The estimates for the human brain glia-to-neuron ratio range between 1:1 - 6:1 (von Bartheld et al. [2016], Herculano-Houzel [2014]). Of the 170.68 billion cells, 84.6 billion are glial cells and 86.1 billion are neurons (Collman [2022]). Moreover, from the perspective of artificial general intelligence (AGI), it is noteworthy that the ratio of glia to neurons is brain region specific. This ratio is evolutionarily conserved by brain regions across species that diverged as far as 90 million years ago.

This evidence indicates that task-specific glia-neuron ratios in deep neural networks are meaningful and fundamental while imposing constraints on brain function. With regard to synaptic complexity between glia cells and neurons, human astrocytes, the most populous glia cell, differ dramatically from astrocytes in other primates as well as in rodents: 2 million synapses versus 120,000 are found in the human astrocytes and the degree of this synaptic plasticity is driven by astrocytes-neuronal interactions in the developing brain (Cheng et al. [2023]).

Second, energy constraints, mediated via glia, enforce sparse neural coding, a property conserved 78 evolutionarily (von Bartheld et al. [2016], Herculano-Houzel [2014]). Perhaps the most widely known 79 example of this on a network level is the synaptic consolidation during sleep with some synapses 80 being eliminated and some reinforced. On a cellular level, larger neurons show reduced rates of 81 excitatory synaptic transmission. 82

Third, in developmental neuroscience, there is evidence that energy constraints also drive brain 83 development and pathophysiologies underlying Autism Spectrum Disorder and neurodegenerative 84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

conditions like Alzheimer's (Desplats et al. [2019], Frasch et al. [2019]). This is the subject of the next subsection.

2.3 Metabolic constraints on brain development: biological glial-neuronal NAS

Glial cells, such as astrocytes and microglia, participate in energy management, synaptic pruning, and plasticity of developing, adult, and aging brain. Glial functional alterations are hallmarks of the brains of people with Autism Spectrum Disorder or neurodegenerative conditions such as Alzheimer's (Desplats et al. [2019], Frasch et al. [2019]). Notably, in both conditions, the glial function itself is energy-dependent. The glial cells adapt to energy availability in early life which in turn reflects intrauterine adversity the fetal brain may experience (Desplats et al. [2019], Frasch et al. [2019]). Put differently, biological NAS can be seen as a process of spatiotemporal integration of earlylife adversities and developmental programming, such as prenatal stress and the accompanying or independently occurring systemic- and neuro-inflammation, on glial energy reserves that modulate the risk for neurodegeneration via modulation of the pace or extent of immunosenescence, i.e., neural function and plasticity (Desplats et al. [2019]).

Such energy-driven network architecture selection suggests a template for NAS in AI. Conversely, artificial NAS that incorporates these relationships has the potential to yield insights into developmental neuroscience and the neuroscience of aging, especially the aforementioned neurological conditions.

This raises the question of how exactly one would go about redefining the relationship between the artificial neurons to include the metabolic cost of computation. To tackle this question, let us consider first another bit of evidence from integrative neuroscience.

2.4 Dynamic coordination during behavioral states points to metabolic optimization

Brain's behavioral states, such as the NREM and REM states, also known as quiet and active states in neonates, are known examples of state-specific system-wide energy management (Figure 1) (Schmidt [2014]). The relevant insight in Figure 1 is that the systemic metabolic states are also reflected in or driven by dynamic relationships between the participating oscillatory networks, the one generating heart beats fluctuations and the one responsible for fluctuations in breathing movements. These phenomena can be described mathematically in simple constructs of Farey trees or Arnold tongues: low energy state of quiet sleep is accompanied by a 3:1 ratio of dynamic coordination between heartbeats and breathing movements, while the higher energy state of active sleep is accompanied by a break-down of such ratios into those more off center (or higher hierarchy) of the ratio distribution (Hoyer et al. [2001], Gebber et al. [1997]). In the following section, we review the more general case of the theory of dynamic coordination which provides a generic mathematical formulation of the observed connection between energy consumption and network dynamics.

2.5 Dynamic coordination and metabolic optimization: linking meta-/multi-stability to the energy landscape

One could conceptualize energy-driven systems' optimization as the optimal solution to the 'too many degrees of freedom' state of biological systems. In their 1988 Science paper, Schoener and Kelso express the solution via the following simple equation for a basic case of two weakly coupled nonlinear oscillators (Schöner and Kelso [1988]). A candidate collective variable that succinctly captures the dynamics of such coordinative patterns is the relative phase between the two rhythmically moving components. In the case of an artificial neural network, this could be a coordinative dynamics of two neurons or a glia-neuron pair. The collective variable ϕ (relative phase) describes the system's dynamic behavior on the energy landscape V as follows:

The attractors are thus the minima of V, whereas the maxima of V are unstable fixed points that separate different basins of attraction. An intrinsic feature of this dynamic pattern approach is the invariance of function under a change of material substrate - a reconfiguration of the connections or couplings among 'neural' elements (Schöner and Kelso [1988], Kelso [1995], Tognoli and Kelso [2014], Kelso [2012]). That is, the function is not rigidly coded into the neural network. This can encode multimodal and multitask capabilities while also optimizing for energy. Further reading can be found here: Tognoli and Kelso [2014], Kelso [2012], Chauhan et al. [2022], Kelso [2021]. In the

Quiet sleep: low energy expenditure

10

02

0

1000

2000

3000

4000

5000

6000

7000

8000

0006

0

5:1

6:1

-20

1000

2000

3000

4000

5000

6000

7000

8000

9000

21

1000

2000

3000

4C00

5000

↓

6000

i.

7000

80C0

9000

1:1-

--..

人

↓

1

Y

↓.

L

-.

1000

2000

3000

4000

5000

6000

7000

8000

0006

0.5

1

0.5

O

1:1

Phsse Coordination Histogram

2:1

3:1

4:1

5:1

6:1

1:1

Phsse Synchronization Histogram

2:1

3:1

4:1

5:1

6:1

Active sleep: higher energy expenditure

100

1000

2000

3000

4000

5000

0009

7000

8000

9000

100

6:1

1000

2000

3000

4000

5000

6000

7000

8000

0006

oweufg

N....

......

0

1000

2000

3000

4000

5000

6000

7000

8000

0006

1:1

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

sweiEo9H

0.15

0.1

0.5

1F

0.05

1:1

Phase Coordination Histogram

2:1

3:1

4:1

.l..

5:1

6:1

0

1:1

Phese Synchronization Histogram

2:1

3:1

4:1

5:1

6:1

proposed context, the application of the dynamic pattern approach to modeling deep glial-neural 136 networks can be seen as an extension of the well-established free energy principle, now constrained 137 on the available energy for predictive coding (Kirchhoff et al. [2018], Friston and Kiebel [2009], 138 Isomura et al. [2022]). 139

2.6 Revisiting neural architecture search: brain-inspired at every step 140

Deep Glial-

Task-specific & metabolic

Free Energy

Neuron Nets

constraints

Principle/Dynamic

architecture

Patterns

AEA

Search Space

Performance

Search Strategy

Estimation

A

Strategy

performance

estimate of A

The novel perspective the developmental neuroscience contributes to this NAS strategy is the evolu149 tionary, task- and modality-specific function of glial/neuronal distribution that is solved for during 150 the Search Strategy step. 151

152

153

154

Such NAS strategy will yield deep glial-neural networks (DGNNs) which are multi-stable, hence capable of representing multiple behaviors with multistability arising from Emin constraints to represent multitask/multimodality with Imax predictive performance.

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

3 Limitations and future directions

Current AI is based on optimization strategies for information (Imax) without metabolic constraints (Emin). Developmental neuroscience is a natural yet seemingly neglected starting point for learning how to evolve energy- and computationally efficient and resilient pattern recognition in AI, including AGI.

In neuroscience, the free energy principle and the dynamic coordination theories provide mathematical formalisms to bridge this gap. Explicit metabolic constraints minimize surprisal maximizing predictive performance and optimizing energy utilization for information processing.

Neurons are but part of the family of brain cells involved in this process in a) space: different brain regions specialize in different tasks, albeit this is dynamically regulated; and b) time: as a function of developmental cues and allostatic load - metastability.

Future research should leverage deeper glial-neural networks. Some precedents exist (Mesejo et al. [2015]).

We need NAS algorithms incorporating Emin x Imax. Is Emin for AI just the cost of electricity or also an intrinsic system's property as exemplified by the theory of dynamic coordination? Evolutionary considerations and observations from developmental neuroscience suggest the latter should be considered if we are to further learn from biological brains. Future research will address the question if the collective variable ϕ (relative phase) can augment (multi-step or quantum?) or replace the traditional weights.

Related to the above question of phase versus weight optimization is the question if Emin is more 'important' than Imax. Modeling will be able to address this question.

What are the implications of biomimetic NAS design for generative DGNNs behavior? Can such architecture help design safer AGIs?

We started out with biological brains, let's return to them. The order parameters identified in multistable DGNNs can inform a new generation of neuroscience models and experiments. Moreover, causal explainable DNNs for modeling brain behavior would be possible thanks to such intrinsically energy-driven NAS design.

References

Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search. In Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren, editors, Automated Machine Learning: Methods, Systems, Challenges , pages 63-77. Springer International Publishing, Cham, 2019.

Christopher S von Bartheld, Jami Bahney, and Suzana Herculano-Houzel. The search for true numbers of neurons and glial cells in the human brain: A review of 150 years of cell counting. J. Comp. Neurol. , 524(18):3865-3895, December 2016.

Suzana Herculano-Houzel. The glia/neuron ratio: how it varies uniformly across brain structures and species and what that means for brain physiology and evolution. Glia , 62(9):1377-1391, September 2014.

Forrest Collman. SciShots: The brain's cleanup crew and paramedics. https://alleninstitute. org/news/scishots-the-brains-cleanup-crew-and-paramedics/?utm_source=

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

social&utm_medium=organic&utm_campaign=bestofforrest , October 2022. Accessed: 2023-5-7.

Yi-Ting Cheng, Estefania Luna-Figueroa, Junsung Woo, Hsiao-Chi Chen, Zhung-Fu Lee, Akdes Serin Harmanci, and Benjamin Deneen. Inhibitory input directs astrocyte morphogenesis through glial GABABR. Nature , pages 1-8, April 2023.

Paula Desplats, Ashley M Gutierrez, Marta C Antonelli, and Martin G Frasch. Microglial memory of early life stress and inflammation: susceptibility to neurodegeneration in adulthood. Neurosci. Biobehav. Rev. , November 2019.

Martin G Frasch, Gal Snir, and Marta C Antonelli. Autism spectrum disorder: a neuro-immunometabolic hypothesis of the developmental origins. arXiv https://doi.org/10.48550/arXiv.1909.05198 , September 2019.

Markus H Schmidt. The energy allocation function of sleep: a unifying theory of sleep, torpor, and continuous wakefulness. Neurosci. Biobehav. Rev. , 47:122-153, November 2014.

D Hoyer, M G Frasch, M Eiselt, O Hoyer, and U Zwiener. Validating phase relations between cardiac and breathing cycles during sleep. IEEE Eng. Med. Biol. Mag. , 20(2):101-106, March 2001.

G L Gebber, S Zhong, S Y Zhou, and S M Barman. Nonlinear dynamics of the frequency locking of baroreceptor and sympathetic rhythms. Am. J. Physiol. , 273(6 Pt 2):R1932-45, 1997.

Giulio Tononi and Chiara Cirelli. Sleep and synaptic down-selection. Eur. J. Neurosci. , 51(1): 413-421, January 2020.

G Schöner and J A Kelso. Dynamic pattern generation in behavioral and neural systems. Science , 239(4847):1513-1520, March 1988.

J A S Kelso. Dynamic Patterns: The Self-organization of Brain and Behavior . MIT Press, Cambridge, MA, 1995.

Emmanuelle Tognoli and J A Scott Kelso. The metastable brain. Neuron , 81(1):35-48, January 2014.

J A Scott Kelso. Multistability and metastability: understanding dynamic coordination in the brain. Philos. Trans. R. Soc. Lond. B Biol. Sci. , 367(1591):906-918, April 2012.

Kanishk Chauhan, Ali Khaledi-Nasab, Alexander B Neiman, and Peter A Tass. Dynamics of phase oscillator networks with synaptic weight and structural plasticity. Sci. Rep. , 12(1):15003, September 2022.

J A Scott Kelso. Unifying large- and Small-Scale theories of coordination. Entropy , 23(5), April 2021.

Michael Kirchhoff, Thomas Parr, Ensor Palacios, Karl Friston, and Julian Kiverstein. The markov blankets of life: autonomy, active inference and the free energy principle. J. R. Soc. Interface , 15 (138), January 2018.

Karl Friston and Stefan Kiebel. Predictive coding under the free-energy principle. Philos. Trans. R. Soc. Lond. B Biol. Sci. , 364(1521):1211-1221, May 2009.

Takuya Isomura, Hideaki Shimazaki, and Karl J Friston. Canonical neural networks perform active inference. Commun Biol , 5(1):55, January 2022.

Pablo Mesejo, Oscar Ibáñez, Enrique Fernández-Blanco, Francisco Cedrón, Alejandro Pazos, and Ana B Porto-Pazos. Artificial neuron-glia networks learning approach based on cooperative coevolution. Int. J. Neural Syst. , 25(4):1550012, June 2015.

Peer reviews:
comments: SUMMARY:
This is a theoretical paper that proposes novel ideas for neural architecture search based on neuroscientific research. The author argues for the fundamental role of metabolic contains in the development of biological neural networks, which should inform the optimization strategies used for artificial neural networks.

STRENGTHS:
- The paper points out interesting and neglected constraints that should be investigated for NAS

WEAKNESSES:
- The authors do not present any experiments that could strengthen their hypothesis
- The paper could give a more comprehensive overview of the current state of NAS, also including early work from the evolutionary community that already did incorporate energy constraints in the form of connection costs, e.g. "Clune, Jeff, Jean-Baptiste Mouret, and Hod Lipson. "The evolutionary origins of modularity." Proceedings of the Royal Society b: Biological sciences 280.1755 (2013): 20122863."
- Some of the concept, such as glia-neural networks, could be explained in more detail

QUESTIONS:
What would be concrete experimental setups that can be used to test the hypothesis proposed in this paper? 

Currently, it is hard to judge how useful and impactful this paper would be for the wider machine-learning community. Initial proof-of-concept results would go a long way in pursuing others about this new proposed direction for NAS. Even though the presented ideas are interesting, In its current form, the paper seems better suited as a workshop paper and not quite ready for the main conference.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-04

comments: SUMMARY:
This paper proposes an approach to neural architecture search that optimizes for energetic efficiency and draws inspiration from the role of glial cells in biological brains.

STRENGTHS:
The idea of energetic efficiency objective functions for neural architecture search may have merit.

WEAKNESSES:
The paper does not really make a technical contribution -- there are no experiments or theoretical results.  The paper is essentially a proposal for a high-level idea about neuroscience-inspired neural architecture search, which is not adequate for a conference like NeurIPS.  Morevoer, the survey of neuroscience literature, while potentially interesting, is not presented in a way that would be comprehensible or actionable for machine learning researchers hoping to leverage these ideas.  I encourage the author(s) to make more concrete their proposed approach and evaluate it empirically if their goal is to improve upon current NAS methods.

QUESTIONS:
Could the authors describe in detail (e.g. with equations or an algorithm) a proposed NAS method using their ideas?

LIMITATIONS:
I found the discussion of limitations hard to follow and so cannot evaluate it properly.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 1
DATE: 2023-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article suggests that the concept of energy-driven network architecture selection in the brain can be applied to artificial neural architecture search (NAS). It proposes redefining the relationship between artificial neurons to include the metabolic cost of computation. The article also discusses the relationship between brain behavioral states and energy management, as well as the theory of dynamic coordination and metabolic optimization in biological systems. It suggests that the optimization of energy-driven systems can be seen as the optimal solution to the complexity of biological systems. The article concludes by proposing an updated NAS strategy that incorporates glial-neuronal ensembles and energy-constrained architecture search.\n\nSTRENGTHS:\nI believe that this kind of interdisciplinary perspective should be encouraged. AI has benefited greatly from biological inspiration and insight, and I commend the authors for their effort. The article proposes that two aspects of biological brains receive higher focus in modern deep learning: the contribution of glial cells and metabolism. I do believe that it is useful to clarify the differences between the biological brain and deep learning on these aspects.\n\nWEAKNESSES:\nThe article does not sufficiently cover contemporary deep learning literature, which does make links to biological neural networks or their properties. While this is less evident for the inclusion of glial cells, there is substantial literature on network efficiency, including in NAS. It is common practice in NAS to compare architectures not only on error on a test dataset, but also measures of efficiency like parameter count or number of floating point operations. NSGA-Net, for example, explicitly searches using two objectives: one of accuracy and one of efficiency. \n\nLu, Zhichao, et al. \"Nsga-net: neural architecture search using multi-objective genetic algorithm.\" Proceedings of the genetic and evolutionary computation conference. 2019.\n\nHardware-aware NAS methods take into account the energy limitations of different hardware, aiming to find a balance between energy efficiency and performance:\n\nChitty-Venkata, Krishna Teja, and Arun K. Somani. \"Neural architecture search survey: A hardware perspective.\" ACM Computing Surveys 55.4 (2022): 1-36.\n\nA common motivation to compress networks, through pruning or reducing precision, is to reduce their energy use:\n\nYang, Tien-Ju, Yu-Hsin Chen, and Vivienne Sze. \"Designing energy-efficient convolutional neural networks using energy-aware pruning.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.\n\nThe proposed NAS method which takes \"metabolic\" constraints into account is in the same line, however the article does not acknowledge the existence of this common consideration in the current NAS literature. The claim that \"Current AI is based on optimization strategies for information (Imax ) without metabolic constraints (Emin )\" is false if we consider measures of floating point operations or computational time on specialized hardware as equivalents to metabolic constraints. The article does not detail how metabolic constraints would be calculated, asking instead \"Is Emin for AI just the cost of electricity or also an intrinsic system’s property as exemplified by the theory of dynamic coordination?\". Given that measuring energy cost in terms of computational complexity (number of floating point operations) is standard in NAS, the proposed definition of Emin should be clarified in the article.\n\nThe article is very short for a NeurIPS submission at less than 5 full pages and could have gone much further in detail both on the proposed ideas and their context in contemporary deep learning.\n\nQUESTIONS:\nWhat is the appropriate definition for Emin for artificial neural networks? For example, is it FLOPs, computational time on specific hardware, energy consumed during inference passes?\n\nFor non-spiking neural networks, how would the influence of glial cells be represented? BatchNorm regulates the activation of a group of neurons, similar to the regulatory behavior of glia. The use of pruning methods during training direct learning towards sparse networks, similar to the energy constraint enforcement of glia. While backpropagation is not considered biologically plausible, the memory of past synaptic modulation stored in the momentum terms of common optimizers like Adam simulate the different timescales of neural action and glial modulation. Are any of these mechanisms sufficient to consider contemporary deep learning architectures as already being \"glial-neural nets\"?\n\nLIMITATIONS:\nAs previously mentioned, the main limitations of this article are the scope of work considered and the lack of detail of the proposed ideas. The proposed ideas should be more fully realized and more firmly grounded in the current literature. The Limitations section on page 5 does not adequately address these limitations.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2023-07-13"}