user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Flexible degrees of connectivity under synaptic weight constraints

Paper content:
Biological neural networks face homeostatic and resource constraints that restrict the allowed configurations of connection weights.
If a constraint is tight it defines a very small solution space, and the size of these constraint spaces determines their potential overlap with the solutions for computational tasks.
We study the geometry of the solution spaces for constraints on neurons' total synaptic weight and on individual synaptic weights, characterizing the connection degrees (numbers of partners) that maximize the size of these solution spaces.
We then hypothesize that the size of constraints' solution spaces could serve as a cost function governing neural circuit development.
We develop analytical approximations and bounds for the model evidence of the maximum entropy degree distributions under these cost functions.
We test these on a published electron microscopic connectome of an associative learning center in the fly brain, finding evidence for a developmental progression in circuit structure.
Computation in neural networks is constrained by their architecture [14] .
The capacity of a network (the number of computations it can successfully learn) depends on a number of factors.
For simple associative memory models, the capacity depends on the structure of the inputs [15] , the learning rule [26] , and constraints on the connectivity [6] .
In biological neural networks, the cost function, learning rule, and structure of input activity are often unknown.
Increasingly, however, high-throughput connectomics studies are revealing the architecture of neural circuits (e.g., [18, 23, 1, 13, 27, 32] ).
This allows us to examine biological circuit structures for signatures of developmentally inspired cost functions governing network architectures.
Biological circuit structure is shaped by developmental programs and slow structural plasticity, which construct a scaffold for and stabilize learning and memory on faster timescales [22] .
Motivated by this, we hypothesize that developmental programs that structure circuits might aim for flexibility: to optimize the number of available weight configurations under given constraints.
The total strength of synaptic connections between two neurons is limited by the amount of receptor and neurotransmitter available and the size of the synapse [19] .
Pyramidal neurons of mammalian cortex and hippocampus undergo synaptic scaling, regulating their total synaptic input strengths to stabilize postsynaptic activity levels [29] .
We consider simple models of resource limitations and homeostatic constraints on total and individual synaptic weights.
We examine how the size of the solution space for these constraints depends on the number of connections (the degree) and compute the optimally flexible degrees under different constraints on total and individual connection strengths.
We then develop the maximum entropy degree distributions under these constraints.
We derive the Laplace approximation for the evidence of these degree distribution models.
Finally, we apply these models to a recently characterized connectome of a learning and memory center of the larval Drosophila melanogaster [13] , asking which constraints best explain the degree distributions of neurons at different developmental stages.
We find that overall, a homeostatically fixed net weight best predicts the degree distributions of Kenyon cell inputs and outputs.
The most mature Kenyon cells, however, are better explained by a simple binomial random wiring model, suggesting a developmental progression in the cost functions governing mushroom body wiring.
Most of the results of this abstract are presented in more detail in a preprint [24] .
We hypothesized that under a particular constraint, the probability of a neuron having degree K is proportional to the size of the space of allowed circuit configurations with K partners.
This corresponds to the degree distribution of the maximum entropy synaptic weight configurations under a constraint.
The general idea of considering the space of allowed configurations can be traced back to Elizabeth Gardner's pioneering work examining the storage capacity of the perceptron for random input patterns [15] .
In the limit of infinitely many connections and input patterns, the idea that a neuron performs associations leads to predictions for the distributions of synaptic weights [3, 5, 2, 6] .
Here, in contrast, we examined the hypothesis that the size of the space of allowed configurations governs the distribution of the number of connections.
We examined constraints on the total strength of connections to (or from) a neuron and on individual connection strengths.
The results with constraints on total connection strengths are a summary of results shown in more detail in [24] .
Connectivity constraints Previous studies have shown that minimizing the amount of wire used to connect neural circuits can predict the spatial layout of diverse neural systems (e.g., [12, 8, 21, 9, 7, 31, 4] ) and pyramidal neurons' dendritic arborizations [10, 11] .
Here we examined, in contrast, the idea that the number of synaptic partners to a neuron might be structured to make constraints flexible: to allow many different connectivity configurations under a constraint.
We hope that this focus on models of synaptic weight configurations and degrees, rather than on physical wire and physical space, may expedite links with theories of computation.
We discussed constraints that limit or fix neurons' total input or output synaptic weight.
We are not aware of experimental studies directly measuring synaptic scaling or resource limitations in Kenyon cells.
There is evidence of homeostatic regulation of total synaptic weights in other Drosophila melanogaster neurons.
Growth from the first instar larva to the third instar larva is accompanied by a homeostatic regulation of mechanosensory receptive fields [17] and functional motor neuron outputs [20] and nociceptive projections [16] .
In addition, changes in inputs to the central aCC neuron elicit structural modifications to its dendritic tree that homeostatically maintain input levels [28] .
Regularization In machine learning, regularizing weights is a common way to reduce generalization errors.
L2 regularization pressures the weights to lie in a L2-ball, and L1 pressures them to lie in an L1-ball; if the weights are also only positive, L1 regularization pressures weights to lie on the surface of a simplex.
We examined regularization on its own, and observed that the sizes of solution spaces for simplicial weight constraints depends on the degree.
This motivated us to consider cost functions (equivalently, probability distributions) for the degrees.
We hope that these biologically inspired cost functions for connectivity degrees might be useful for architecture search.
Computational capacity The Rademacher complexity of a set is bounded by its covering number: the number of spheres of radius r that are required to cover a set [25] .
The measure of configuration flexibility we used are Haussdorff measures of the solution spaces for different constraints.
The Haussdorff measure has a similar flavor to the covering number.
We have not yet formalized a relation between our approach and the Rademacher complexity, but believe this to be a promising direction.

assistant: text: We examine the hypothesis that the entropy of solution spaces for constraints on synaptic weights (the "flexibility" of the constraint) could serve as a cost function for neural circuit development.