Sparsely Connected, Hebbian Networks with Strikingly Large Storage Capacities
Conspicuous problems confront the use of fully-connected networks (F-nets) in the construction of realistic partial models of biological memory. These problems include the high synaptic densities of F-nets, and the low information storage capacities of F-nets with simple, biologically realistic learning rules. Most auto-associative networks constructed with low connectivities have employed random projections of path length 1. Projective networks (P-nets) are nonrandom, multilayer networks which achieve extremely low connectivities by linking all neurons in the same layer through paths of length 2. In this paper we derive a lower bound on the storage capacities of a class of simple, two-layer P-nets with binary Hebbian synapses. Given a 1% tolerance for spurious neurons, we find that the P-net with 1000 synapses per neuron (2  106 neurons) will store more than 1.5  106 training vectors with 20 active neurons per vector (0.25 bits per synapse).  1997 Elsevier Science Ltd.
sparse connectivity projective p-net lower bound block design associative biased code