System for performing weighted summations

Abstract: Plural-bit digital input signals to be subjected to weighted summation in a neural net layer are bit-sliced; and a number N of respective first through N th weighted summations of the bits of the digital input signals in each bit slice are performed, resulting in a respective set of first through N th partial weighted summation results. Weighted summations of the partial weighted summation results of similar ordinal number are then performed to generate first through N th final weighted summation results. Each weighted summation of a bit slice of the digital input signals is performed using a capacitive network that generates partial weighted summation results in the analog regime. The weighted summation to generate a final weighted summation result can be done in the analog regime, which facilitates the analog final weighted summation result being non-linearly processed in an analog amplifier with sigmoidal response. This non-linear processing generates an analog axonal output response for a neural net layer, which analog axonal output response can then be digitized. Alternatively, the weighted summation to generate a final weighted summation result can be done in the digital regime and the digital final weighted summation result can then be non-linearly processed in digital circuitry with sigmoidal response. Where the neural net layer is to be trained using a back propagation training algorithm, the digital circuitry with sigmoidal response can be of a type where response slope is readily determined.

Description: The invention relates to analog computer structures for performing a plurality of weighted summations of a plurality of input signals (as are used in neural nets, for example) and more particularly, to adapting such analog computer structures for use with digital electronic circuits. Computers of the von Neumann type-architecture have limited computational speed owing to the communication limitations of the single processor. These limitations can be overcome if a plurality of processors are utilized in the calculation and are operated at least partly in parallel. This alternative architecture, however, generally leads to difficulties associated with programming complexity. Therefore, it is often not a good solution. Recently, an entirely different alternative that does not require programming has shown promise. The networking ability of the neurons in the brain has served as a model for the formation of a highly interconnected set of analog processors, called a "neural network" or "neural net" that can provide computational and reasoning functions without the need of formal programming. The neural nets can learn the correct procedure by experience rather than being preprogrammed for performing the correct procedure. The reader is referred to R.P. Lippmann's article "An Introduction to Computing With Neural Nets" appearing on pages 4-21 of the April 1987 IEEE ASSP MAGAZINE (0740-7467/87/0400-0004/$10.00 " 1987 IEEE), incorporated herein by reference, for background concerning neural nets. Neural nets are composed of a plurality of neuron models, analog processors each exhibiting "axon" output signal response to a plurality of "synapse" input signals. In a type of neural net called a "perceptron", each of these processors calculates the weighted sum of its "synapse" input signals, which are respectively weighted by respective weighting values that may be positive- or negative-valued, and responds nonlinearly to the weighted sum to generate the "axon" output response. This relationship may be described in mathematical symbols as follows. Here, i indexes the input signals of the perceptron, of which there are an integral number M, and j indexes its output signals, of which there are an integral number N. W i,j is the weighting of the i th input signal as makes up the j th output signal at such low input signal levels that the function is approximately linear. At higher absolute values of its argument, the function no longer exhibits linearity but rather exhibits a reduced response to This type of non-linear response is termed "sigmoidal". The weighted summation of a large number of sampled-data terms can be viewed as the process of correlating the sampled-data function described by those sampled-data terms with the sampled-data function described by the pattern of weights; and the analog processor used as a neuron model in a "perceptron" can be viewed as a correlator with non-linear circuitry exhibiting a sigmoidal response connected thereafter. A more complex artificial neural network arranges a plurality of perceptrons in hierarchic layers, the output signals of each earlier layer providing input signals for the next succeeding layer. Those layers preceding the output layer providing the ultimate output signal(s) are called "hidden" layers. The training of an artificial neural network comprising hierarchic layers of perceptrons is accomplished in the prior art by using a back-propagation training algorithm. The back-propagation training algorithm is an iterative gradient algorithm designed to minimize the mean square error between the actual output of a multi-layer feed-forward neural net and the desired output. It requires continuous, differentiable non-linearities in the transfer function of the non-linear circutry used after the weighted summation circuits in the neural net layer. A recursive algorithm starting at the output nodes and working back to the first hidden layer is used iteratively to adjust weights in accordance with the following formula. W i,j (t+1) = W ij (t) - ηδ j x i In this equation W i,j (t) is the weight from hidden node i (or, in the case of the first hidden layer, from an input node) to node j at time t; x i is either the output of node i (or, in the case of the first hidden layer, is an input signal); η is a gain term introduced to maintain stability in the feedback procedure used to minimize the mean square errors between the actual output(s) of the perceptron and its desired output(s); and δ j is a derivative of error. The general definition of δ j is the change in error energy from output node j of a neural net layer with a change in the weighted summation of the input signals used to supply that output node j. Lippman presumes that a particular sigmoid logistic non-linearity is used. Presuming the non-linearity of processor response is to be defined not as restrictively as Lippmann does, then δ j can be more particularly defined as in equation (2), following, if node j is an output node, or as in equation (3), following, if node j is an internal hidden node. δ j = z j ' (d j - z j ) In equation (3) δ j and z j are the desired and actual values of output response from the output layer and z j ' is differential response of z j to the non-linearity in the output layer - i.e., the slope of the transfer function of that non-linearity. In equation (4) k is over all nodes in the neural net layer succeeding the hidden node j under consideration and W j,k is the weight between node j and each such node k. The term z j' is defined in the same way as in equation (3). The general definition of the z j' term appearing in equations (3) and (4), rather than that general term being replaced by the specific value of z j' associated with a sigmoid logistic non-linearity, is the primary difference between the training algorithm as described here and as described by Lippmann. Also, Lippmann defines d j in opposite polarity from equations (1), (3) and (4) above. During training of the neural net, prescribed patterns of input signals are sequentially repetitively applied, for which patterns of input signals there are corresponding prescribed patterns of output signals known. The pattern of output signals generated by the neural net, responsive to each prescribed pattern of input signals, is compared to the prescribed pattern of output signals to develop error signals, which are used to adjust the weights per equation (2) as the pattern of input signals is repeated several times, or until the error signals are detected as being negibly valued. Then training is done with the next set of patterns in the sequence. During extensive training the sequence of patterns may be recycled. In the present-day development of the integrated electronic circuitry art, the weighted summation of a large number of terms, each of which has resolution that would require plural-bit digital sampling, can be done appreciably faster and at less cost in integrated circuit die area by processing in the analog regime rather than in the digital regime. Using capacitors to perform weighted summation in accordance with Coulomb's Law provides neural nets of given size operating at given speed that consume less power than those the analog processors of which use resistors to implement weighted summation in accordance with Ohm's Law. There is a class of problems that can usefully employ neural nets for their solution, but for which synapse input signal samples are plural-bit digital samples. Particularly where there are numerous plural-bit digital synapse input signal samples, difficulties are encountered in getting the plural-bit digital samples into the monolithic integrated circuit in which a neural net layer reposes, because of practical limitations on the number of pins available on the integrated circuit package. An example of when this type of problem is encountered is when one attempts to use a neural net to perform the two-dimensional spatial filtering associated with the recognition of shapes in video signals generated by raster scanning a focal plane array of photosensors in a video camera. In such two-dimensional spatial filtering it is desirable to provide for the temporary storage of the raster-scanned video signal samples, so that they may be temporally aligned for application to the neural net. In the present state of technology the temporary storage of a substantial number of video samples is most advantageously done in digital memory. The synapse input signals made available to the neural net from the digital memory are in digital, not analog, form. The problem of the synapse input signals made available to a neural net being in plural-bit digital form rather than in analog form can arise in other contexts, too. In systems where the neural net layers are not located in proximity to each other the axon output response ofa hidden neural net layer can be digitized in an analog-to-digital converter and transferred without error over substantial distances to another neural net layer. Using digital rather than analog signals to communicate between neural net layers can also be useful when back-propagation training algorithms are used to train a neural net, since it easier in the digital regime to alter the training algorithm in specified ways to take into account whether a new task is being learned or whether instead an old task is being modified to take into account a change in the operating environment. Streams of digital samples have to be converted to respective analog input signals in order to be used with the neural nets known in the prior art; and, since the samples requiring conversion are parallel in time, a respective digital-to-analog converter is required for each input signal to the neural net. Except where the streams of digital samples are single-bit in nature, the well-known problems of avoiding the conversion non-linearity associated with converting a multiple-bit digital number code stream to a respective analog signal have to be faced with regard to each of these digital-to-analog converters, and these problems are known to be more difficult to solve as the size of the converters has to be made smaller to accommodate a large number of them on a monolithic integrated circuit. Document : TWENTY-SECOND ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS & COMPUTERS vol. 1, 31 October 1988, PACIFIC GROVE , USA pages 649 - 653 EBERHARDT 'Considerations for hardware implementations of neural networks' describes the performance of a single-layer feedback network incorporating simple binary interconnects being extended by using an analog prompt method. Each download interface is parallel, and consists of a memory for digital storage of the weights or network input values, a D/A converter for generating the analog refresh voltage, and an address counter for selecting the capacitor to be refreshed. A digital I/O bus under control of a host processor is used to program weights and network inputs. Document : NEURAL INFORMATION PROCESSING SYSTEMS 1987, DENVER, USA pages 573 - 583 MURRAY 'Bit - serial neural network' discusses the importance of bit-serial architecture, bit-serial versus bit-parallel arithmetic. When an actual implementation is discussed, the bit-serial concept is used in its computational units. Input to these units are by parallel buses. Document : ELECTRONIC ENGINEERING vol. 62, no. 759, March 1990, LONDON GB page 70 'A bit-slice boost for neural networks' discloses a digital neural network element containing eight neurons. Data is processed in the neuron in a bit-serial fashion in the synapse computation. Document : 1990 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS vol. 4, 1 May 1990, NEW ORLEANS, USA pages 2982 - 2985 CILINGIROGLU 'Capacitive synapses for microelectronic neural networks' refers to the implementation of the synaptic function of non-adaptive neural networks with a single capacitor. The resulting capacitive synaptic matrix give an opportunity to exploit hybrid microelectronic implementation styles. Document : 1989 INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS vol. 3, 8 May 1989, PORTLAND, USA pages 2177 - 2180 HANSEN 'A time-multiplexed switched-capacitor circuit for neural network applications' discloses a time-multiplexed switched-capacitor neural computational block. Digital signals for application to an artificial neural network can be processed on a bit-sliced basis, so there is no need for digital-to-analog converters to convert multiple-bit digital number code streams each to a respective analog signal. A system for performing N weighted summations of input signals, M in number, identified by respective ones of consecutive first through M th ordinal numbers, N being an integer greater than zero and M being an integer greater than one, that embodies the invention is characterized as set out in claim 1. The final weighted summation of the partial weighted summation results, which are analog in nature, can be carried out in the analog regime (and thereafter digitized, if so desired). Alternatively, the partial weighted summation results may be digitized and then summed with appropriate shifts to obtain final weighted summation results in digital form. The various figures of the drawing will be described briefly, before considering them in greater detail. FIGURE 1 is a schematic diagram of a prior-art single-ended charge sensing amplifier comprising a Miller integrator with resetting circuitry that compensates for input offset error in the differential-input operational amplifier the Miller feedback capacitor provides degenerative feedback to; this type of single-ended charge sensing amplifier and a balanced version thereof are preferred over simpler Miller integrators for the charge sensing amplifiers used in the circuitry for performing weighted summations in the analog regime. FIGURES 2, 3 and 4 are schematic diagrams of three different apparatuses, each for performing a plurality of weighted summation procedures in parallel on a bit slice of input signals. FIGURE 5 is a schematic diagram of apparatus for performing a plurality of weighted accumulation procedures, each of which weighted accumulation procedures weights and sums sequentially supplied data. FIGURE 6 is a schematic diagram of a neural net layer including any of the FIGURE 2, FIGURE 3 and FIGURE 4 apparatuses for performing a plurality of weighted summation procedures; possibly a FIGURE 5 apparatus for performing a plurality of weighted accumulation procedures; an array of non-linear processors and an array of analog-to-digital converters. FIGURE 7 is a schematic diagram of a voltage amplifier that can be used as a non-linear processor in neural net layers per FIGURE 6. FIGURE 8 is a schematic diagram of apparatus for digitizing analog partial weighted summation results as sequentially supplied from one of the FIGURE 2, FIGURE 3 and FIGURE 4 apparatuses and then digitally performing a plurality of weighted accumulation procedures, each of which weighted accumulation procedures weights and sums a respective succession of digitized sequentially supplied data. FIGURES 9A and 9B, arranged respectively on the left and on the right of the other, together form a schematic diagram (generally referred to as FIGURE 9) of a neural net layer including any of the FIGURE 2, FIGURE 3 and FIGURE 4 apparatuses for performing a plurality of partial weighted summation procedures; an array of analog-to-digital converters for digitizing the partial weighted summation results; possibly a FIGURE 8 apparatus for digitally performing a plurality of digital final weighted summation procedures; and an array of digital non-linear processors generating digital axonal responses to the digital final weighted summation results. FIGURE 10 is a schematic diagram of a digital non-linear processor that can be used in neural net layers. FIGURE 11 is a graph showing the non-linear transfer function of the FIGURE 10 digital non-linear processor. FIGURES 12 and 13 are schematic diagrams of two further apparatuses, each for performing a plurality of weighted summation procedures in parallel on a bit slice of input signals. FIGURE 14 is a schematic diagram of analog apparatus for performing a plurality of weighted accumulation procedures, each of which weighted accumulation procedures weights and sums sequentially supplied data. FIGURE 15 is a schematic diagram of a neural net layer including either of the FIGURE 12 and FIGURE 13 apparatuses for performing a plurality of weighted summation procedures; possibly a FIGURE 14 apparatus for performing a plurality of weighted accumulation procedures; an array of non-linear processors and an array of analog-to-digital converters. FIGURES 16 and 17 are schematic diagrams of respective modifications of the FIGURE 15 neural net layer to adapt it for operation with one's complement synapse input signals rather than with two's complement synapse input signals. FIGURE 18 is a schematic diagram of apparatus for digitizing analog partial weighted summation results as sequentially supplied from one of the FIGURE 12 and FIGURE 13 apparatuses and then digitally performing a plurality of weighted accumulation procedures, each of which weighted summation procedures weights and sums a respective succession of digitized sequentially supplied data. FIGURES 19A and 19B, arranged respectively on the left and on the right of the other, together form a schematic diagram (generally referred to as FIGURE 19) of a neural net layer including two ranks of either of the FIGURE 12 and FIGURE 13 apparatuses for performing a plurality of partial weighted summation procedures; two ranks of analog-to-digital converters for digitizing the partial weighted summation results; two ranks of the FIGURE 18 apparatus connected for digitally performing a plurality of digital final weighted summation procedures; and an array of digital non-linear processors generating digital axonal responses to the digital final weighted summation results. FIGURES 20A and 20B, arranged respectively on the left and on the right of the other, together form a FIGURE 20 which is a schematic diagram of a modification that can be made manifold times to the neural net shown in FIGURES 2, 8 and 9; FIGURES 3, 8 and 9; FIGURES 12, 18 and 19; or FIGURES 13, 18 and 19 for the programmable weighting of the capacitances used in performing weighted summations of synapse signals. FIGURES 21A and 21B, arranged respectively on the left and on the right of the other, together form a FIGURE 21 which is a schematic diagram of a modification that can be made manifold times to the neural net shown in FIGURES 4, 8 and 9, for the programmable weighting of the capacitances used in performing weighted summations of synapse signals. FIGURES 22A and 22B, arranged respectively on the left and on the right of the other, together form a FIGURE 22 which is a schematic diagram of a modification that can be made manifold times to the neural net shown in FIGURES 12, 18 and 19 or in FIGURES 13, 18 and 19, for the programmable weighting of the capacitances used in performing weighted summations of synapse signals. FIGURES 23A and 23B, arranged respectively on the left and on the right of the other, together form a FIGURE 23 that is a schematic diagram of a neural net layer in which both forward propagation and back propagation through a capacitive network are carried out with balanced signals, in which digital capacitors have their capacitance values programmed from respective word storage locations within a digital memory, and in which the signs of the digital signals being weighted and summed are taken into account in the final weighted summation procedure performed on a digital basis. FIGURES 24A and 24B, arranged respectively on the left and on the right of the other, together form a FIGURE 24 that is a schematic diagram of an alternative neural net layer in which both forward propagation and back propagation through a capacitive network are carried out with balanced signals, and in which digital capacitors have their capacitance values programmed from respective word storage locations within a digital memory. In the FIGURE 24 neural net layer the signs of the digital signals being weighted and summed are taken into account in the partial weighted summation procedure performed on an analog basis. FIGURE 25 is a schematic diagram of a back-propagation processor used for training a neural net having layers as shown in FIGURES 2, 8 and 9 or in FIGURES 3, 8 and 9 modified manifoldly per FIGURE 20; or a neural net having layers as shown in FIGURES 4, 8 and 9 modified manifoldly per FIGURE 21; or a neural net having layers as shown in FIGURES 12, 14 and 15 or in FIGURES 13, 14 and 15 modified manifoldly per FIGURE 22; or a neural net having layers as shown in FIGURE 23; or or a neural net having layers as shown in FIGURE 24. FIGURE 26 is a schematic diagram of a system having a plurality of neural net layers as shown in FIGURES 2, 8 and 9 or in FIGURES 3, 8 and 9 modified manifoldly per FIGURE 20; or in FIGURES 4, 5 and 6 modified manifoldly per FIGURE 21; or in FIGURES 12, 14 and 15 or in FIGURES 13, 14 and 15 modified manifoldly per FIGURE 22; or in FIGURE 23; or in FIGURE 24 each of which neural net layers has a respective back-propagation processor per FIGURE 23 associated therewith. FIGURES 27A and 27B, arranged respectively on the left and on the right of the other, together form a FIGURE 27 that is a schematic diagram of an alternative modification of that can be made manifold times to the neural net shown in FIGURES 2, 8 and 9 or in FIGURES 3, 8 and 9, for the programmable weighting of the capacitances used in performing weighted summations of synapse signals. FIGURES 28A and 28B, arranged respectively on the left and on the right of the other, together form a FIGURE 28 that is a schematic diagram of an alternative modification of that can be made manifold times to the neural net shown in FIGURES 4, 8 and 9, for the programmable weighting of the capacitances used in performing weighted summations of synapse signals. FIGURES 29A and 29B, arranged respectively on the left and on the right of the other, together form a FIGURE 29 that is a schematic diagram of an alternative modification of that can be made manifold times to the neural net shown in FIGURES 12, 14 and 15 or in FIGURES 13, 14 and 15, for the programmable weighting of the capacitances used in performing weighted summations of synapse signals. FIGURES 30A and 30B, arranged respectively on the left and on the right of the other, together form a FIGURE 30 that is a schematic diagram of a neural net layer in which both forward propagation and back propagation through a capacitive network are carried out with balanced signals, in which digital capacitors have their capacitance values programmed from counters, and in which the signs of the digital signals being weighted and summed are taken into account in the final weighted summation procedure performed on a digital basis. FIGURES 31A and 31B, arranged respectively on the left and on the right of the other, together form a FIGURE 31 that is a schematic diagram of an alternative neural net layer in which both forward propagation and back propagation through a capacitive network are carried out with balanced signals, and in which digital capacitors have their capacitance values programmed from counters. In the FIGURE 31 neural net layer the signs of the digital signals being weighted and summed are taken into account in the partial weighted summation procedure performed on an analog basis. FIGURE 32 is a schematic diagram of the arrangement of stages in the counters shown in FIGURES 27-31. FIGURE 33 is a schematic diagram of the logic elements included in each counter stage in the counters shown in FIGURES 27-32. More particularly, FIGURE 1 shows a single-ended charge sensing amplifier QS k of a preferable type for implementing the single-ended charge sensing amplifiers used in modifications of the FIGURE 2, 3, 12, 14, 20B, 21A, 22B, 27 and 29 B circuitry. The charge sensing amplifier QS K is essentially a Miller integrator that includes a differential-input operational amplifier OA having a Miller feedback capacitor MC connecting from its output connection OUT back to its inverting (-) input connection during normal operation. The non-inverting (+) input connection of the operational amplifier OA is connected to a fixed potential in the single-ended charge sensing amplifier QS k , which fixed potential is shown as having a value (V SS + V DD )/2 midway between a relatively high potential (V DD ) and a relatively low potential (V SS ). During normal charge-sensing operation a relatively low potential (V SS ) is applied via RESET terminal to a logic inverter INV that responds to apply a relatively high potential (V DD ) to a transmission gate TG1. The transmission gate TG1 is rendered conductive to connect the output connection OUT of operational amplifier OA to capacitor MC to complete its connection as Miller feedback capacitor. The relatively low potential applied via RESET terminal conditions a transmission gate TG2 and a transmission gate TG3 both to be non-conductive. QS K is a preferred charge sensing amplifier because differential input offset error in the operational amplifier OA is compensated against, owing to the way the Miller integrator is reset. During periodic reset intervals for the integrator a relatively high potential (V DD ) is applied via RESET terminal to condition transmission gates TG2 and TG3 each to be conductive and to condition the logic inverter INV output potential to go low, which renders transmission gate TG1 non-conductive. The conduction of transmission gate TG2 connects the output connection OUT of operational amplifier OA directly to its inverting (-) input connection, completing a feedback connection that forces the inverting (-) input connection to the differential input offset error voltage, which voltage by reason of transmission gate TG3 being conductive is stored on the Miller capacitor MC. When normal charge-sensing operation is restored by RESET terminal going low, the differential input offset error bias remains stored on the Miller capacitor MC, compensating against its effect upon charge sensing. Supposing the operational amplifier OA to be a differential output type having balanced output connections OUT and OUTBAR, a balanced version of the charge sensing amplifier QS K can be formed by disconnecting the non-inverting (+) input connection of the operational amplifier OA from a point of fixed potential having a value (V SS + V DD )/2. Instead, the non-inverting (+) input connection of the operational amplifier OA is arranged to have a feedback connection from the OUTBAR output connection of the operational amplifier OA similar to the feedback connection from the OUT output connection of the operational amplifier OA to its inverting (-) input connection. This balanced version of the charge sensing amplifier QS K is shown throughout the remaining figures of the drawing. M is a positive plural integer indicating the number of input signals to the FIGURE 2, 3, 4, 12 or 13 weighted summation apparatus, and N is a positive plural integer indicating the number of output signals the FIGURE 2, 3, 4, 12 or 13 apparatus can generate. To reduce the written material required to describe operation of the weighted summation apparatuses in FIGURES 2, 3, 4, 12 or 13 of the drawing, operations using replicated elements will be described in general terms; using a subscript i ranging over all values one through M for describing operations and circuit elements as they relate to the (column) input signals x 1 , x 2 ,.. x M ; and using a subscript j ranging over all values one through N for describing operations and apparatus as they relate to the (row) output signals y 1 , y 2 ,...y N . That is, i and j are the column and row numbers used to describe particular portions of the FIGURES 2, 3, 4, 12 and 13 weighted summation apparatuses and modifications of those apparatuses. The FIGURE 2 apparatus performs a plurality of weighted summation procedures in parallel on each successive bit slice of input signals, which input signals comprise a plurality M in number of parallel bit streams x 1 , x 2 ,... x M . This apparatus is assumed to receive a first operating voltage V DD , a second operating voltage V SS , and a third operating voltage (V SS + V DD )/2 midway between V SS and V DD. V DD and V SS are presumed to be relatively positive and relatively negative respective to each other. Each input voltage signal x i is applied as control signal to a respective multiplexer MX i and to a respective multiplexer MX (i+M) . Multiplexer MX i responds to x i being a ONE to apply the V DD first operating voltage to an input line IL i and responds to x i being a ZERO to apply the V SS second operating voltage to the input line IL i . Multiplexer MX (i+M) responds to x i being a ONE to apply the V SS second operating voltage to an input line IL (i+M) and responds to x i being a ZERO to apply the V DD first operating voltage to the input line IL (i+M) . A capacitor C i,j has a first plate connected to the input line IL i and has a second plate connected to an output line OL j . A capacitor C (i+M),j has a first plate connected to the input line IL (i+M) and has a second plate connected to the output line OL j . Capacitor C i,j and capacitor C (i+M),j together are considered as a pair, for providing in effect a single weighting capacitor, the capacitance of which is the difference in the capacitances of capacitor C i,j and capacitor C (i+M),j between their respective first and second plates. The charge placed on the output line OL j by all the weighting capacitors connecting thereto is sensed on a single-ended basis by a respective charge-sensing amplifier QS j . Each of the charge-sensing amplifiers QS j is shown as a respective Miller integrator comprising an operational amplifier and Miller feedback capacitors. During reset or zeroing of all the charge-sensing amplifiers QS j , each of the x i input voltages is a logic ZERO. This applies V SS to the plates of capacitors C i,j connected from the multiplexers MX i and applies V DD to the plates of capacitors C (i+M),j connected from the multiplexers MX (i+M) . The total capacitance on each output line OL j is caused to be the same as on each of the other output lines by a respective shunt capacitor C 0,j to signal ground, which capacitor either has capacitance that is so large as to overwhelm the other capacitances on the output line OL j or preferably that complements the other capacitances on the output line OL j . Causing the total capacitance on each output line OL j to be the same as on each of the other output lines makes the sensitivities of the charge-sensing amplifiers QS j to their respective inputs uniform, presuming them to be Miller integrators of identical design. If the capacitances of capacitor C i,j and capacitor C (i+M),j between their respective first and second plates sum to a prescribed standard value, for the complete selection range of i and j, the sensitivities of the charge-sensing amplifiers QS j to their respective inputs are uniform withou need for a respective shunt capacitor C 0,j to signal ground for each output line OL j , presuming the charge-sensing amplifiers QS j to be Miller integrators of identical design. After reset or zeroing, when x i bits for different i may each be ZERO or ONE, each x i bit that is a ZERO creates no change in charge condition on any of the output lines OL j . A bit x i that is a ONE creates an incremental change in charge on an output line OL j that, in accordance with Coulomb's Law, is equal to (V SS - V DD )/2 times the difference in the capacitances of capacitors C i,j and C (i+M),j between their respective first and second plates. The sum of these incremental charges accumulates on the output line OL j and is sensed by the charge-sensing amplifier QS j . FIGURE 3 shows a modification of the FIGURE 2 apparatus in which the multiplexer MX i and the multiplexer MX (i+M) respond to x i being a ZERO to apply the third operating voltage (V SS + V DD )/2 to the input line IL i and to the input line IL (i+M) , respectively. During reset, the weighting capacitors C i,j and C (i+M) will be charged to relatively small bias voltages between their plates, rather than to bias voltages of amplitudes close to (V SS + V DD )/2. The FIGURE 2 apparatus is advantageous over the FIGURE 3 apparatus in that, in the FIGURE 2 apparatus, accuracy of the the third operating voltage (V SS + V DD )/2 being exactly midway between the first operating voltage V DD and the second operating voltage V SS is not necessary for accuracy of the partial weighted summation results. The FIGURE 4 apparatus also performs a plurality of weighted summation procedures in parallel on each successive bit slice of input signals, which input signals comprise a plurality M in number of parallel bit streams x 1 , x 2 , x 3 , ... x M . Logic inverters INV 1 , INV 2 , INV 3 , ...INV M respond to the current bits x 1 , x 2 , x 3 , ... x M respectively with their respective logic complements. (The current bits x 1 , x 2 , x 3 , ... x M are assumed to be supplied in accordance with the positive logic convention.) The FIGURE 4 apparatus also is assumed to receive a relatively positive first operating voltage V DD , a relatively negative second operating voltage V SS , and a third operating voltage (V SS + V DD )/2 midway between V SS and V DD . The logic inverter INV i responds to x 1 being a ZERO to apply V DD to an input line IL i and responds to x 1 being a ONE to apply V SS to the input line IL i . As in FIGURES 2 and 3, the charge-sensing amplifier QS j is one of a plurality, N in number, of identical charge-sensing amplifiers for sensing the difference in charges accumulated on a respective pair of output lines. In FIGURE 4 the charge-sensing amplifier QS j is arranged for differentially sensing charge and is connected to sense the difference in charges accumulated on output lines OL j and OL (j+N) . The output lines OL j and OL (j+N) are charged from each input line input line IL i via a capacitor C i,j and via a capacitor C i,(j+N) , respectively. Capacitor C i,j and capacitor C i,(j+N) together are considered as a pair, for providing in effect a single weighting capacitor, the capacitance of which is the difference in the capacitances of capacitor C i,j and capacitor C i,(j+N) between their respective first and second plates. The total capacitance on each output line OL j is maintained the same as on each of the other output lines by a respective shunt capacitor C 0,j to signal ground; and the total capacitance on each output line OL (j+N) is maintained the same as on each of the other output lines by a respective shunt capacitor C 0,(j+N) to signal ground. Where the capacitances of capacitor C i,j and capacitor C i,(j+N) between their respective first and second plates are to be alterable responsive to digital programming signals, it is preferred that the capacitances of capacitor C i,j and capacitor C i,(j+N) between their respective first and second plates sum to a prescribed standard value for the complete selection range of i and j. It is further preferred that each pair of capacitors C i,j and C i,(j+N) have a corresponding further pair of capacitors C i,(j+2N) and C i,(j+3N) associated therewith, capacitor C i,(j+2N) having a capacitance equal to that of capacitor C i,(j+N) and connecting output line OL j to a point of connection P i,j , and capacitor C i,(j+3N) having a capacitance equal to that of capacitor C i,j and connecting output line OL (j+N) to the same point of connection P i,j . If all the points of connection P i,j connect to signal ground, the capacitors C i,(j+2N) for all values of i together provide for each value of j the respective shunt capacitor C 0,j to signal ground, and the capacitors C i,(j+3N) for all values of i together provide for each value of j the respective shunt capacitor C 0,(j+N) to signal ground. This is taught in greater detail in U. S. patent application serial No. 526,470. The FIGURE 4 apparatus may be modified to replace logic inverters INV 1 , INV 2 , INV 3 , ... INV M with non-inverting driver amplifiers. In such case the other output connections of the differential-output operational amplifiers in the charge sensing amplifiers QS 1 , QS 2 , QS 3 , ... QS N are used to supply the y 1 , y 2 , y 3 ... y N partial weighted summations. The FIGURE 4 apparatus may alternatively be modified to augment logic inverters INV 1 , INV 2 , INV 3 , ... INV M with non-inverting driver amplifiers DA 1 , DA 2 , DA 3 , ... DA M respectively and to use each non-inverting driver amplifier DA i to drive the points of connection P i,j for all values of j. This provides for full-capacitor bridge drives to each charge-sensing amplifier QS j , rather than half-capacitor bridge drives. The advantage of doing this is that the common-mode voltage on the output lines OL j and OL (j+N) is zero, so one does not have to rely as much on the common-mode suppression of the charge-sensing amplifier QS j to keep the integration of charge within the operating supply range of that amplifier. Where the input signals to the FIGURE 2, FIGURE 3 or FIGURE 4 weighted summation apparatus comprise single-bit words, the N signals y 1 , y 2 , y 3 , ...y N can, per FIGURE 6, be applied to an array of respective non-linear processors NLP 1 , NLP 2 , NLP 3 ,... NLP N to generate a plurality, N in number, of analog axonal output responses. As shown in FIGURE 6, these analog axonal output responses can be digitized in respective analog-to-digital converters ADC 1 , ADC 2 , ADC 3 ,...ADC N , for application to the succeeding circuitry NEXT. For example, this succeeding circuitry may be an output neural net layer where the neural net layer thusfar described is a hidden neural net layer. Oversampling analog-to-digital converters of the sigma-delta type are preferred for analog-to-digital converters ADC 1 , ADC 2 , ADC 3 ,...ADC N when passing digitized axonal output responses to another weighted summation apparatus of one of the types illustrated in FIGURES 2-4, since the sigma-delta type of converter is inherently adapted to supplying its digitized response in bit-serial form. FIGURE 5 shows further weighted summation apparatus that can be used with the weighted summation apparatus of FIGURE 2 or 3 to provide for the weighted summation of digital signals having plural-bit words. The plurality B in number of bits per digital word is assumed in FIGURE 5 to be four bits per digital word. The digital signals are placed in bit-serial form and in word alignment for application to the weighted summation apparatus of FIGURE 2 or 3, which processes the bit-serial digital signals on a bit-slice basis to generate B partial weighted summation results, one per bit slice, during each successive word interval. A SIGN BIT FLAG signal is provided to signal the parallel occurrence of the sign bits in each successive group of M input signal words. The SIGN BIT FLAG signal is used to synchronize the operation of a sequencer SEQ in the further weighted summation apparatus of FIGURE 5. (Where the synapse input signals x 1 , x 2 , ... x M are unsigned, the SIGN BIT FLAG signal is replaced by a MOST SIGNIFICANT BIT FLAG signal, of course.) The FIGURE 5 weighted summation apparatus can be described as weighted accumulation apparatus since it accumulates the B partial weighted summation results supplied to it from the weighted summation apparatus of FIGURE 2 or 3, the accumulated partial weighted summation result term being weighted in accordance with the significance of the bit slice involved in its generation. The FIGURE 5 apparatus includes a plurality, N in number, of weighted summers generating respective ones of N final weighted summation results z 1 , z 2 , ... z N . Each of these weighted summers weights the four partial weighted summation results y j sequentially supplied during a word interval as part of one of the output signals of the weighted summation apparatus of FIGURE 2 or 3, the weighting being done in accordance with the significance of the bits used to generate each respective partial weighted summation result, and sums the weighted partial weighted summation results to generate a final weighted summation result z j . More particularly, when the sequencer SEQ determines that the least significant of the four partial weighted summation results y j associated with a digital word interval is being generated by the weighted summation apparatus of FIGURE 2 or 3, the sequencer SEQ directs sample-and-hold circuits SH1 j and SH5 j to sample and hold that y j value. When the sequencer SEQ determines that the next to least significant of the four partial weighted summation results y j associated with a digital word interval is being generated by the weighted summation apparatus of FIGURE 2 or 3, the sequencer SEQ directs sample-and-hold circuits SH2 j and SH6 j to sample and hold that y j value. When the sequencer SEQ determines that the next to most significant of the four partial weighted summation results y j associated with a digital word interval is being generated by the weighted summation apparatus of FIGURE 2 or 3, the sequencer SEQ directs sample-and-hold circuits SH3 j and SH7 j to sample and hold that y j value. When the sequencer SEQ determines that the most significant of the four partial weighted summation results y j associated with a digital word interval is being generated by the weighted summation apparatus of FIGURE 2 or 3, the sequencer SEQ directs a sample-and-hold circuit SH4 j and SH8 j to sample and hold that y j value. The sequencer SEQ may be designed to accommodate the particular bit-serial format being used. The common bit-serial formats arrange the bits in order of their significance, one such format beginning each digital word with its least significant bit and ending each digital word with its most significant (sign) bit, and another such format beginning each digital word with its most significant (sign) bit and ending each digital word with its least significant bit. At the end of the word interval, the four partial weighted summation results y j are held in parallel at the output ports of the sample-and-hold circuits SH1 j , SH2 j , SH3 j and SH4 j and the four complementary partial weighted summation results -y j are held in parallel at the output ports of the sample-and-hold circuits SH5 j , SH6 j , SH7 j and SH8 j . Weighting capacitors C1 j , C2 j , C3 j and C4 j are used to weight the four partial weighted summation results held by the sample-and-hold circuits SH1 j , SH2 j , SH3 j and SH4 j , respectively. The four partial weighted summation results held by the the sample-and-hold circuits SH1 j , SH2 j , SH3 j and SH4 j are weighted in 1:2:4:8 ratio by weighting capacitors C1 j , C2 j , C3 j and C4 j , in accordance with the amplitudes of the weights +1, +2, +4 and -8 of the respective groups of bits generating them. (The most significant group of bits are assumed to be sign bits of two's complement numbers, which is the reason for the negative sign of the weighting by eight.) Similarly, the four partial weighted summation results held by the the sample-and-hold circuits SH5 j , SH6 j , SH7 j and SH8 j are weighted in 1:2:4:8 ratio by weighting capacitors C5 j , C6 j , C7 j and C8 j , in accordance with the amplitudes of the weights +1, +2, +4 and -8 of the respective groups of bits generating them. Accordingly, weighting capacitors C1 j , C2 j , C3 j , C4 j , C5 j , C6 j , C7 j , and C8 j have respective capacitances in 1:2:4:8:1:2:4:8 ratio. The four partial weighted summation results held by the the sample-and-hold circuits SH1 j , SH2 j , SH3 j and SH4 j charge the weighting capacitors C1 j , C2 j , C3 j and C4 j in respective amounts depending on their voltages and the capacitances of the capacitors to which they are applied, in accordance with Coulomb's Law. The four partial weighted summation results held by the the sample-and-hold circuits SH5 j , SH6 j , SH7 j and SH8 j charge the weighting capacitors C5 j , C6 j , C7 j and C8 j in respective amounts depending on their voltages and the capacitances of the capacitors to which they are applied, in accordance with Coulomb's Law. A charge sensing amplifier QS( N+j) then differentially senses the charge on the plates of the weighting capacitors C5 j , C6 j , C7 j and C4 j connecting to its inverting (-) input connection and the combined charge on the plates of the weighting capacitors C1 j , C2 j , C3 j and C8 j connecting to its non-inverting (+) input connection, to generate a final weighted summation result as a sample of its output signal z j . Connecting plates of the weighting capacitors C4 j and C8 j to the inverting (-) input connection of the charge sensing amplifier QS( N+j) and to its non-inverting (+) input connection, respectively, provides for the negativeness associated with the minus eight weight of the sign bit in the two's complement numbering system. Reset or zeroing of the charge sensing amplifiers QS( N+j) in FIGURE 5 takes place at the same time as the reset or zeroing of the charge sensing amplifiers QS j in the preceding FIGURE 2, 3 or 4 weighted summation circuitry. During reset the (V SS + V DD )/2 third operating voltage at the y j and -y j output connections of the charge sensing amplifiers QS j are applied to the plates of the weighting capacitors C1 j , C2 j , C3 j , C4 j , C5 j , C6 j , C7 j , and C8 j by the sample-and-hold circuits SH1 j , SH2 j , SH3 j , SH4 j , SH5 j , SH6 j , SH7 j and SH8 j . Where the input signals to the FIGURE 2, FIGURE 3 or FIGURE 4 weighted summation apparatus comprise plural-bit words, the N signals z 1 , z 2 ,...z N from a succeeding FIGURE 5 weighted summation apparatus can then, per FIGURE 6, be applied to an array of respective non-linear processors NLP 1 , NLP 2 ,... NLP N to generate a plurality, N in number, of analog axonal output responses. As shown in FIGURE 6, these analog axonal output responses can be digitized in respective analog-to-digital converters ADC 1 , ADC 2 , ADC 3 ,...ADC N , for application to the succeeding circuitry NEXT. For example, this succeeding circuitry may be an output neural net layer where the neural net layer thusfar described is a hidden neural net layer. FIGURE 7 shows non-linear voltage amplifier circuitry that can be used to implement each non-linear processor NLP j in FIGURE 6. The FIGURE 7 non-linear voltage amplifier is a cascade connection of two source-follower transistors, one (Q 20A ) being an n-channel MOSFET and the other (Q 20B ) being a p-channel MOSFET. Q 20A is provided a constant-current generator source load by an n-channel MOSFET Q 21 , which is the slave or output transistor of a current mirror amplifier including as its master or input transistor an n-channel MOSFET Q 22 self-biased by drain-to-gate feedback. Q 20B is provided a constant-current generator source load by a p-channel MOSFET Q 23 , which is the slave or output transistor of a current mirror amplifier including as its master or input transistor a p-channel MOSFET Q 24 self-biased by drain-to-gate feedback. Q 22 and Q 24 are connected as diodes by their respective drain-to-gate feedback connections, and these diodes are connected in series with another diode-connected n-channel MOSFET Q 25 and with another diode-connected p-channel MOSFET Q 26 between V SS and V DD potentials to implement a bias network. In this bias network a quiescent input current flows from the input port of the current mirror amplifier comprising Q 23, Q 24 into the input port of the current mirror amplifier comprising Q 21 , Q 22 . Q 21 and Q 23 drain current flows are similar-valued by current mirror amplifier action. All the n-channel MOSFETs Q 20A , Q 21 , Q 22 and Q 25 have similar channel widths and lengths and exhibit similar operating characteristics. All the p-channel MOSFETs Q 20B , Q 23 , Q 24 and Q 26 have similar channel widths and lengths and exhibit similar operating characteristics, which are complementary to those of the n-channel MOSFETs. The bias network MOSFETs Q 22 , Q 24 , Q 25 and Q 26 may be shared by a plurality of the FIGURE 7 non-linear voltage amplifier circuits to conserve hardware and operating power. Non-linearity of response in the FIGURE 7 voltage amplifier comes about because (1) source-follower action of Q 20A for positive-going excursions of its gate electrode potential becomes limited as its source potential approaches its drain potential V HI and (2) source-follower action of Q 20B for negative-going excursions of its gate electrode potential becomes limited as its source potential approaches its drain potential V LO . At the source electrode of source-follower Q 20B there is a sigmoidal response to a linear ramp potential applied to the gate electrode of source-follower Q 20A . The voltages V LO and V HI can be programmed to control the limiting properties of the FIGURE 7 non-linear amplifier, and the voltages V LO and V HI may be selected to provide for symmetry of response or for asymmetry of response. FIGURE 7 shows representative values for V HI and V LO that provide a substantially symmetrical response about +2.5 volts. Performing the final weighted summation processes in the analog regime, as thusfar described, facilitates also performing in the analog regime the sigmoidal non-linear processing subsequently required in a neural network. Usually, a further array of capacitive structures is required for storing the partial weighted summation results, then performing the final weighted summation in the analog regime. Performing the final weighted summation processes in the digital regime avoids the need for such a further array of capacitive structures. A digital accumulator circuit is used instead, which tends to be more economical of area on a monolithic integrated-circuit die. In a neural network, performing the sigmoidal non-linear processing in the digital regime tends to be less straightforward. However, the determination of the slope of the non-linear transfer function, which determination is necessary when training a neural net layer using a back-propagation training algorithm, is simply accomplished in certain digital non-linear circuitry as will be described further on in this specification. This contrasts with the greater difficulty of determining the slope of the non-linear transfer function in analog non-linear circuitry. FIGURES 9A and 9B when arranged as shown in FIGURE 9 provide a schematic diagram of a neural net layer wherein the the final weighted summation processes are performed in the digital regime. The weighting capacitors of the FIGURE 9 neural net layer are preferably constructed on a binary-digit-sliced basis as described in allowed U. S. patent application serial No. 525,931, filed 21 May 1990 by W. E. Engeler, entitled "WEIGHTED SUMMATION CIRCUITS HAVING DIFFERENT-WEIGHT RANKS OF CAPACITIVE STRUCTURES" and assigned to General Electric Company. In FIGURE 9A the bit-sliced (column) input signals x 1 , x 2 ,... x M are applied to a weighted summation network WSN 1 that comprises a rank of capacitive weighting structures of relatively high significance; and in FIGURE 9B the bit-sliced (column) input signals x 1 , x 2 ,... x M are applied to a weighted summation network WSN 3 that comprises a rank of capacitive weighting structures of relatively low significance. In FIGURE 9A the analog weighted sums of the bit-sliced input signals x 1 , x 2 ,... x M supplied from the weighted summation network WSN 1 , which are of relatively higher significances, are digitized in respective analog-to-digital (or A-to-D) converters RADC 1 , RADC 2 , RADC 3 , ...RADC N to generate bit-sliced digital signals y 1 , y 2 , y 3 , ...y N , respectively. Analogously, in FIGURE 9B the analog weighted sums of the bit-sliced input signals x 1 , x 2 ,... x M supplied from the weighted summation network WSN 3 , which are of relatively lower significances, are digitized in respective analog-to-digital (or A-to-D) converters RADC( N+1 ), RADC (N+2) , RADC (N+3) ,...RADC 2N to generate bit-sliced digital signals y (N+1) , y (N+2) , y (N+3) , ...y 2N , respectively. In FIGURE 9A the bit-sliced digital signals y 1 , y 2 , y 3 ,...y N , which are partial weighted summations, are applied to a weighted summation network WSN 2 (as will be described in greater detail further on with reference to FIGURE 8). The weighted summation network WSN 2 combines each set of partial weighted summations y j to generate a respective relatively more significant component Sy j of the final weighted summation supplied as the sum signal output of a digital adder ADD j , which respective relatively more significant component Sy j is applied as augend input signal to the adder ADD j . Analogously, in FIGURE 9B the bit-sliced digital signals y (N+1) , y (N+2) , y (N+3) , ...y 2N , which are partial weighted summations, are applied to a weighted summation network WSN 4 (as will be described in greater detail further on with reference to FIGURE 8). The weighted summation network WSN 4 combines each set of partial weighted summations y (N+j) to generate a respective relatively less significant component Σy (N+j) of the final weighted summation supplied as the sum signal output of a digital adder ADD j , which respective relatively less significant component Σy (N+j) is applied after appropriate wired shift right WSR j as addend input signal to the adder ADD j . (Right shift of a digital signal by P binary places is reckoned as being a division of the digital signal by 2 P .) The sum signals from the adders ADD 1 , ADD 2 , ADD 3 , ... ADD N are supplied as input signals to respective digital non-linear processors NLP 1 , NLP 2 , NLP 3 ,... NLP N which respond to their respective input signals to generate a plurality, N in number, of digital axonal output responses. As shown in FIGURE 9A, these digital axonal output responses can be applied to succeeding circuitry NEXT. For example, this succeeding circuitry NEXT may be an output neural net layer where the neural net layer thusfar described is a hidden neural net layer. Where the input signals to the FIGURE 2, FIGURE 3 or FIGURE 4 weighted summation apparatus comprise single-bit words, the N signals y 1 , y 2 , y 3 ,...y N can, per FIGURE 9A, be digitized in respective analog-to-digital (or A-to-D) converters RADC 1 , RADC 2 , RADC 3 ,...RADC N , for application to adders ADD 1 , ADD 2 , ADD 3 ... ADD N , which supply their sum signals to respective digital non-linear processors NLP 1 , NLP 2 , NLP 3 ,... NLP N to generate a plurality, N in number, of digital axonal output responses. There is no need for the further, digital weighted summation apparatus WSN 2 . Analogously,the N signals y (N+1) , y (N+2) , y (N+3) ,...y 2N can per FIGURE 9B, be digitized in respective analog-to-digital (or A-to-D) converters RADC (N+1) , RADC (N+2) , RADC (N+3) , ...RADC 2N , for application with appropriate wired shifts right WSR 1 , WSR 2 , WSR 3 , ... WSR N to adders ADD 1 , ADD 2 , ADD 3 ... ADD N . There is no need for the further, digital weighted summation apparatus WSN 4 . Where only a single rank of capacitive weighting structures -- that rank in the weighted summation network WSN 1 -- is used, all the elements shown in FIGURE 9B are dispensed with. The adders ADD 1 , ADD 2 , ADD 3 ... ADD N are supplied zero-valued addend signals rather than being supplied Σy (N+1) , Σy (N+2) , Σy (N+3) , ... Σy 2N as respective addend signals. Alternatively, the adders ADD 1 , ADD 2 , ADD 3 , ... ADD N may be omitted, with the digital non-linear processors NLP 1 , NLP 2 , NLP 3 ,... NLP N receiving their respective input signals directly from the weighted summation network WSN 2 . Or directly from the A-to-D converters RADC (N+1) , RADC (N+2) , RADC (N+3) ,...RADC 2N where the input signals x 1 , x 2 ,... x M to the weighted summation network WSN 1 comprise single-bit words so the weighted summation network WSN 1 is not used. FIGURE 8 shows further, digital weighted summation apparatus that can be used with the analog weighted summation apparatus of FIGURE 2, 3 or 4 to provide for the weighted summation of digital input signals x 1 , x 2 ,... x M having plural-bit words. The digital signals are placed in bit-serial form and in word alignment for application to the weighted summation apparatus of FIGURE 2, 3 or 4, which processes the bit-serial digital signals on a bit-slice basis to generate B partial weighted summation results, B being the number of bits per bit-serial arithmetic word. The successive bit slices of the plural-bit words of the digital input signals x 1 , x 2 ,... x M are processed in order of increasing significance, with the sign bit slice being processed last in each word. A SIGN BIT FLAG that is normally ZERO is provided, which SIGN BIT FLAG goes to ONE to signal the parallel occurrence of the sign bits in each successive group of M input signal words at the time the partial weighted summation of greatest significance is being generated, supposing the digital signals to be flows of two's complement arithmetic words. FIGURE 8 shows the plurality N in number of A-to-D converters RADC 1 , RADC 2 ... RADC N used to convert the analog y 1 , y 2 , ... y N partial weighted summations from the charge sensing amplifiers RQS 1 , RQS 2 , RQS 3 , ... RQS N to digital format and a weighted summation network for combining the sequences of digitized y 1 , y 2 , ... y N partial weighted summations to generate final weighted summations Σy 1, Σy 2 ... Σy N . More particularly, the digitized y 1 , y 2 , ... y N partial weighted summations from the FIGURE 8 A-to-D converters RADC 1 , RADC 2 ... RADC N are assumed to be respective two's complement numbers. The bit resolution afforded by the A-to-D converters RADC 1 , RADC 2 ... RADC N preferably increases by one bit place each successive bit slice of the digital word; in any case, each successive partial weighted summation within the same digital word is weighted by an additional factor of two respective to the previous partial weighted summation. A plurality, N in number, of accumulators RACC 1 , RACC 2 , ... RACC N are used for generating respective ones of N respective final weighted summation results Σy j . Each of the successive partial weighted summation results y j sequentially supplied by the A-to-D converter RADC i is routed through a respective selective complementor RCMP i . The selective complementor RCMP i complements each bit of the partial weighted summation result the A-to-D converter RADC i generates when the SIGN BIT FLAG is ONE, then augments the result of this one's complementing procedure by adding to it a number that has a ONE as its least significant bit and has more significant bits all of which are ZEROs. This augmentation is simply carried out by applying the SIGN BIT FLAG as a carry to the adder in the accumulator RACC i . When the SIGN BIT FLAG is ZERO, the selective complementor RCMP i transmits to the accumulator RACC i , without complementation, each bit of the partial weighted summation result the A-to-D converter RADC i generates. Each of these accumulators RACC 1 , RACC 2 , ... RACC N accumulates for each word the successive partial weighted summation results y j sequentially supplied as one of the output signals of the weighted summation apparatus of FIGURE 2, 3 or 4, to generate a final weighted summation result Σy j . (In alternative embodiments of the invention where the bit-serial digital input signals supplying the parallel bit streams x 1 , x 2 , x 3 , ... x M are invariably of one polarity and are processed as unsigned numbers, of course, the selective complementors RCMP 1 , RCMP 2 , ... RCMP N need not be included in the respective connections of the A-to-D converters RADC 1 , RADC 2 ... RADC N to the accumulators RACC 1 , RACC 2 , ... RACC N .) The respective Σy j final weighted summation result generated by each accumulator RACC i is latched by a respective clocked latch RLCH i to be temporarily stored until such time as the next final weighted summation result Σy j is generated by the accumulator RACC i . The respective Σy j final weighted summation result from latch RLCH i is supplied as augend input signal to a respective digital adder ADD j , which may receive as an addend a final weighted summation result Σy j ' of lesser significance from a further, less significant rank of capacitors for weighting the plurality M in number of parallel bit streams x 1 , x 2 ,... x M , which final weighted summation result Σy j ' of lesser significance is appropriately attenuated respective to Σy j by a wired shift right WSR i . The sum output signal z i from the adder ADD i is supplied to the ensuing digital non-linear processor NLP i ; and , as shown in FIGURE 6A, the non-linear response from the digital non-linear processor NLP i is applied to the succeeding circuitry NEXT. Where the input signals to the FIGURE 2, FIGURE 3 or FIGURE 4 weighted summation apparatus comprise plural-bit words, the N signals z 1 , z 2 ,...z N from a succeeding FIGURE 8 weighted summation apparatus can then, per FIGURE 9, be applied to an array of respective digital non-linear processors NLP 1 , NLP 2 ,... NLP N to generate a plurality, N in number, of digital axonal output responses for application to the succeeding circuitry NEXT. For example, this succeeding circuitry may be an output neural net layer where the neural net layer thusfar described is a hidden neural net layer. FIGURE 10 shows circuitry that can be used to implement a digital non-linear processor NLP j as may be used for each of the digital non-linear processors NLP 1 , NLP 2 , NLP 3 , ... NLP N of FIGURE 9. The FIGURE 10 digital non-linear processor is used to obtain a transfer function as diagrammed in FIGURE 11. The FIGURE 11 transfer function approximates a sigmoidal symmetrically non-linear response by five straight line segments 11, 13, 15, 17 and 19. One skilled in the art of digital circuit design will after being acquainted with the following description of the FIGURE 10 digital non-linear processor be able to design more complex digital non-linear processors approximating a sigmoidal transfer characteristic using an odd number greater than five of straight line segments. In the FIGURE 11 graph of the transfer function of the FIGURE 10 digital non-linear processor, the straight line segment 15 passes through zero input signal, extending between inflection points 14 and 16 which occur at input signals of values -k 1 and k 1 , respectively. The input signal has relatively small absolute magnitudes in the range defined by the straight line segment 15, which segment of the transfer characteristic has the steepest slope and passes through the origin of the transfer characteristic graph. The straight line segment 11 extending left and down from inflection point 12 and the straight line segment 19 extending right and up from inflection point 18 have the shallowest slope. The input signal has relatively large absolute magnitudes in the ranges defined by the straight line segments 11 and 19. If the straight line segment 11 were extended without change in its direction right and up from inflection point 12 it would cross the input axis at an intercept value of -k 3 ; and if the straight line segment 19 were extended without change in its direction left and down from inflection point 18 it would cross the input axis at an intercept value of +k 3 . The input signal has intermediate absolute magnitudes in the ranges defined by the straight line segments 13 and 17. The straight line segment 13 extends between the inflection points 12 and 14 and, if it were extended without change in its direction beyond inflection point 14, would cross the output signal axis at an intercept value of -k 4 . The straight line segment 17 extends between the inflection points 16 and 18 and, if it were extended without change in its direction beyond inflection point 16, would cross the output signal axis at an intercept value of +k 4 . In the FIGURE 7 digital non-linear processor the digital final weighted summation result Σy j is supplied to a respective absolute value circuit AVC j where the most significant, sign bit conditions a selective complementor RCMP (N+j) to complement the less significant bits of Σy j when z j is negative. The output signal from the selective complementor RCMP (N+j) is supplied as augend signal to a digital adder ADD (N+j) , which is supplied the most significant bit of Σy j as an addend signal. The sum output of the adder ADD (N+j) is the |Σy j | output signal of the absolute value AVC j . A digital subtractor SUB j is used for subtracting a positive constant value k 1 from |Σy j | to determine whether |Σy j | being < k 1 should be on a portion of the non-linear transfer characteristic having steepest slope or whether |Σy j | being ≥ k 1 should be on a portion of the non-linear transfer characteristic having a shallower slope. If the most significant, sign bit of the difference output signal of the subtractor SUB i is a ONE, indicating |Σy j | < k 1 , a slope multiplexer SMX j selects the steepest transfer characteristic slope as the multiplier signal for a digital multiplier MULT j that is supplied |Σy j | as multiplicand signal. The digital product is then added in a digital adder ADD (2N+j) to a zero-value intercept, as selected by an intercept multiplexer IMX j responsive to the sign bit of the difference output signal of subtractor SUB i being a ONE, to generate the non-linear response |z j | to |Σy j |. A digital subtractor SUB (N+j) is used for subtracting a positive constant value k 2 from |Σy j | to determine whether |Σy j | being ≥ k 2 should be on a portion of the non-linear transfer characteristic having shallowest slope or whether |Σy j | being < k 2 should be on a portion of the non-linear characteristic having a steeper slope. It is most significant, sign bit of the difference output signal of the subtractor SUB (N+j) is a ZERO, indicating |Σy j | ≥ k 2 , the slope multiplexer SMX j selects the shallowest transfer characteristic slope as the multiplier signal for the digital multiplier MULT j ; and the intercept multiplexer IMX j selects an intercept of value k 3 somewhat smaller than k 2 to be added to the product from multiplier MULT j in adder ADD (N+j) to generate |z j |. Responsive to the most significant bits of the difference signals from the subtractors SUB j and SUB (N+j) being ZERO and ONE respectively, indicative that |Σy j | should be on a portion of the non-linear transfer characteristic have a slope intermediate between the steepest and shallowest slopes, the slope multiplexer SMX j selects that intermediate slope as multiplier signal to the multiplier MULT j , and the intercept multiplexer IMX j selects an intercept of value k 4 somewhat smaller than k 1 as the addend signal supplied to the adder ADD (2N+j) . The non-linear response |z j | to |Σy j | is converted to z j response to Σy j by processing |z j | through a selective complementor CMP (2N+j) that complements each bit of |z j | when the most significant, sign bit of Σy j is a ONE indicating its negativity; inserting the most significant bit of Σy j as sign bit before the selective complementor CMP (2N+j) output signal; and adding the resultant in a digital adder ADD (3N+j) to the most significant bit of Σy j . The sum signal from the digital ADD (3N+j) is then used to load a parallel-in/serial-out register RPISO j for the j th row, which supplies z j in bit-serial format to the succeeding circuitry NEXT (per FIGURE 6). During back-propagation training, the value of the slope selected by the slope multiplexer SMX j responsive to the |Σy j | generated by a set of prescribed synapse input signals can be temporally stored in a latch SLCH j responsive to a LATCH SLOPE command. This operation will be considered as part of an overall description of back-propagation training further on in this specification. In FIGURES 20-23 and 27-31, the slope latch SLCH j is shown external to the digital non-linear processor NLP j with which it is associated. Thusfar, the reversal of sign in the weighting of the parallel sign bits of the bit-serial synapse input signals has been described as being done in the final weighted summation of the partial weighted summation results. Alternatively, in other embodiments of the invention, such as those now to be described, the reversal of sign in the weighting of the parallel sign bits of the bit-serial synapse input signals is done while performing the partial weighted summations. FIGURE 12 shows a modification of the FIGURE 2 or FIGURE 3 apparatus in which the multiplexers MX 1 ... MX M, MX (M+1) ... MX 2M that each select from between two operating voltages the voltage applied to a corresponding one of the input lines IL 1 ... IL M, IL( M+1 ) ... IL 2M are replaced by multiplexers MX 1 ' ... MX M ', MX (M+1) ' ... MX 2M ' that each select from among the V DD , V SS and (V SS + V DD )/2 operating voltages the voltage applied to a corresponding one of the input lines IL 1 ... IL M, IL (M+1) ... IL 2M . The current condition of the SIGN BIT FLAG is applied to each of the multiplexers MX i ' and MX (i+M) ' as its first control bit, and the current bit of a respective input voltage signal x i is applied to each of the multiplexers MX i ' and MX (i+M) ' as its second control signal. For all bits of x i except its sign bits, the SIGN BIT FLAG is a ZERO. The SIGN BIT FLAG being a ZERO conditions multiplexer MX i ' to respond to x i being a ONE to apply the V DD first operating voltage to an input line IL i and to respond to x i being a ZERO to apply the third operating voltage (V SS + V DD )/2 to the input line IL i . The SIGN BIT FLAG being a ZERO conditions multiplexer MX (i+M) ' to respond to x i being a ONE to apply the V SS second operating voltage to an input line IL (i+M) and to respond to x i being a ZERO to apply the third operating voltage (V SS + V DD )/2 to the input line IL (i+M) . When the sign bits of x i occur, the SIGN BIT FLAG is a ONE. The SIGN BIT FLAG being a ONE conditions multiplexer MX i ' to respond to x i being a ONE to apply the V SS second operating voltage to an input line IL i and to respond to x i being a ZERO to apply the third operating voltage (V SS + V DD )/2 to the input line IL i . The SIGN BIT FLAG being a ONE conditions multiplexer MX (i+M) ' to respond to x i being a ONE to apply the V DD first operating voltage to an input line IL (i+M) and to respond to x i being a ZERO to apply the third operating voltage (V SS + V DD )/2 to the input line IL (i+M) . Accordingly,the reversal of sign in the weighting of the parallel sign bits of the bit-serial synapse input signals is done while performing the partial weighted summations. FIGURE 13 shows a modification of the FIGURE 4 apparatus in which the multiplexers MX 1 , MX 2 , MX 3 , ... MX M, that each select from between two operating voltages the voltage applied to a corresponding one of the input lines IL 1 , IL 2 , IL 3 , ... IL M are replaced by multiplexers MX 1 ', MX 2 ', ... MX M ' that each select from among the V DD , V SS and (V SS + V DD )/2 operating voltages the voltage applied to a corresponding one of the input lines IL 1 , IL 2 , IL 3 , ... IL M . The multiplexers MX 1 ', MX 2 ... MX M ' are controlled in the FIGURE 13 apparatus similarly to the way they are in FIGURE 12 apparatus, again to provide the reversal of sign in the weighting of the parallel sign bits of the bit-serial synapse input signals while performing the partial weighted summations. FIGURE 14 shows analog apparatus for performing the final weighted summation of partial weighted summation results from the FIGURE 12 or 13 apparatus. The FIGURE 14weighted summation apparatus can be described as weighted accumulation apparatus since it accumulates the B partial weighted summation results supplied to it from the weighted summation apparatus of FIGURE 12 or 13, the accumulated partial weighted summation result term being weighted in accordance with the significance of the bit slice involved in its generation. The FIGURE 14 apparatus differs from the FIGURE 5 apparatus in that, for each value of j, one through N, the weighting capacitors C1 j , C2 j , C3 j and C4 j all connect to a single respective output line OL j , so the sum of their respective charge conditions can be sensed on a single-ended basis. Single-ended charge sensing allows dispensing with the weighting capacitors C5 j , C6 j , C7 j and C8 j . Single-ended charge sensing will also allow each of the charge sensing amplifiers QS (N+j) to be replaced by a simpler single-ended charge sensing amplifier such as that shown in FIGURE 1. Where the FIGURE 14 apparatus performs the final weighted summation of partial weighted summation results from the FIGURE 12 apparatus, each of the charge sensing amplifiers QS j can also be replaced by a simpler single-ended charge sensing amplifier such as that shown in FIGURE 1. FIGURE 15 is a schematic diagram of a neural net layer similar to the neural net layer of FIGURE 6, but includes either of the FIGURE 12 and FIGURE 13 apparatuses as a weighted summation network WSN 1 ' for performing a plurality of weighted summation procedures, so the reversal of sign in the weighting of the parallel sign bits of the bit-serial synapse input signals takes place in the weighted summation network WSN 1 '. Where the weighted summation procedures performed in the weighted summation network WSN 1 ' generate partial weighted summation results, a weighted summation network WSN 2 ' comprising the FIGURE 14 apparatus is used to generate final weighted summation results for application to an array of non-linear processors NLP 1 , NLP 2 , ... NLP N . The non-linear processors NLP 1 , NLP 2 , ... NLP N have their analog axonal output signals digitized in respective analog-to-digital converters ADC 1 , ADC 2 ,...ADC N , for application to the succeeding circuitry NEXT. FIGURE 16 shows a modification made to the FIGURE 12 apparatus in the FIGURE 15 neural net layer in respect to each synapse input signal port. These modifications are made to adapt the FIGURE 15 neural net layer for operation with one's complement, rather than with two's complement, bit-serial synapse input signals. The one's complement bit-serial synapse input signals are presumed to be ones in which the sign bit is the initial bit in a bit-serial word. Responsive to the SIGN BIT FLAG being a ONE, a respective bit latch SBL i latches the sign bit of x i to be held for the remainder of the bit-serial word when the SIGN BIT FLAG is a ZERO. Another bit latch ABL i clocked at bit rate latches each successive bit of x i . The outputs of the bit latches SBL i and ABL i are applied as the two inputs of a two-input exclusive-OR gate XOR i . If the sign bit stored in sign bit latch SBL i is a ZERO indicating x i to be positive, the exclusive-OR gate XOR i applies bits corresponding to the successive bits of x i as second control bits to the multiplexers MX i ' and MX (i+M) '. Alternatively, if the sign bit stored in sign bit latch SBL i is a ONE indicating x i to be negative, the exclusive-OR gate XOR i applies bits corresponding to the bit complements of the successive bits of x i as second control bits to the multiplexers MX i ' and MX (i+M) '. When the SIGN BIT FLAG is a ONE during the initial bit of each bit-serial word, the bit latches SBL i and ABL i latch the same sign bit to each of the two inputs of exclusive-OR gate XOR i , which responds with a ZERO output. This ZERO output is applied as the second control bit of each of the multiplexers MX i ' and MX (i+M) ', conditioning them to select (V SS + V DD )/2 to each of the input lines IL i and IL (i+M) . This facilitates the resetting of the charge sensing amplifiers QS (N+1) , QS (N+2) , ... QS 2N during the sign bit interval. When adapting the FIGURE 15 neural net layer for operation with one's complement bit-serial synapse input signals, the FIGURE 14 sequencer SEQ is modified to omit weighting capacitors SH4 j for the sign bit slice and the associated sample-and-hold circuits SH4 j . The FIGURE 14 sequencer SEQ in the FIGURE 15 neural net layer is also arranged to sequence sampling in the sample-and-hold circuits in accordance with the significances of the successive bits of the bit-serial x i words which follow. FIGURE 17 shows a modification made to the FIGURE 12 apparatus in the FIGURE 15 neural net layer in respect to each synapse input signal port to facilitate operation with one's complement bit-serial synapse input signals in which the final bit in each bit-serial word is the sign bit. It differs from the FIGURE 16 modification in that x i is delayed for the duration of one bit-serial word in a digital delay line DDL i before its application to the exclusive-OR gate XOR i bit latch ABL i , so the sign bit latch SBL i can be written for temporarily storing the final sign bit of each bit-serial word to be applied to one input of the exclusive-OR gate XOR i throughout the interval that the other bits of that bit-serial word are successively applied to the other input of the exclusive-OR gate XOR i . The clocking of the digital delay line DDL i and of the sign bit latch SBL i in FIGURE 17 is arranged so that the exclusive-OR gate XOR i invariably provides a ZERO for application as second control bits to the multiplexers MX i ' and MX (i+M) ' during the time the SIGN BIT FLAG is a ONE, facilitating the resetting of the charge sensing amplifiers QS (N+1) , QS (N+2) , ... QS 2N . Again, the FIGURE 14 sequencer SEQ in the FIGURE 15 neural net layer is modified to omit weighting capacitors SH4 j for the sign bit slice and the associated sample-and-hold circuits SH4 j . The FIGURE 14 sequencer SEQ in the FIGURE 15 neural net layer is also arranged to sequence sampling in the sample-and-hold circuits in accordance with the significances of the successive bits of the bit-serial x i words which follow. FIGURE 18 shows apparatus for performing the final weighted summation of partial weighted summation results from the FIGURE 12 or 13 apparatus digitally after digitizinging the analog partial weighted summation results. The FIGURE 18 apparatus differs from the FIGURE 8 apparatus in that: there are no selective complementors RCMP 1 , RCMP 2 , ... RCMP N included in the respective connections of the A-to-D converters RADC 1 , RADC 2 ... RADC N to the accumulators RACC 1 , RACC 2 , ... RACC N ; and the accumulators invariably have ZERO carry inputs. FIGURE 19 is a schematic diagram of a neural net layer similar to the neural net layer of FIGURE 9, but includes either of the FIGURE 12 and FIGURE 13 apparatuses as a weighted summation network WSN 1 ' for performing a plurality of weighted summation procedures, so the reversal of sign in the weighting of the parallel sign bits of the bit-serial synapse input signals takes place in the weighted summation network WSN 1 '. Where the weighted summation procedures performed in the weighted summation network WSN 1 ' generate partial weighted summation results, a weighted summation network WSN 2 ' comprising the FIGURE 18 apparatus is used to generate final weighted summation results for application to an array of non-linear processors NLP 1 , NLP 2 , ... NLP N . The non-linear processors NLP 1 , NLP 2 , ... NLP N have their analog axonal output signals digitized in respective A-to-D converters RADC 1 , RADC 2 ... RADC N , for application to the succeeding circuitry NEXT. The nature of the analog-to-digital converters RADC i has thusfar not been considered in detail. Indeed, a wide variety of A-to-D converters can be used in implementing the invention. Some A-to-D converter structures are capable of accepting input signals in charge form, and in such a structure the charge sensing means and the analog-to-digital converting means specified in certain of the claims appended to this application will both be provided. The bit-slicing of input signals already slows data word throughput rate, so oversampling A-to-D converters, as of the sigma-delta type will not be favored over successive-approximation A-to-D converters unless quite slow data processing rates are acceptable. Where fairly fast data processing rates are required, flash A-to-D converters will be preferred oversuccessive-approximation A-to-D converters. The resistive ladder used to scale reference voltage in a flash A-to-D converter can be shared with other flash A-to-D converters within the same monolithic integrated circuit, economizing the amount of hardware associated with this approach to analog-to-digital conversion. FIGURE 20, comprising component FIGURES 20A and 20B, shows a representative modification that can be manifoldly made to a neural net as shown in FIGURES 2, 8 and 9 or in FIGURES 3, 8 and 9. Each neural net layer comprises a capacitive network viewed as having a plurality M in number of columns of capacitive weights and a plurality N in number of rows of capacitive weights. The description that follows focuses on the intersection of an i th of these columns with a j th of these rows, where i is a column index from 1 to M and j is a row index from 1 to N. A respective modification is made near each set of intersections of an output line OL j with input lines IL i and IL (M+i) driven by opposite senses of a synapse input signal x i . Such modifications together make the neural net capable of being trained. Each capacitor pair C i,j and C (M+i),j of the FIGURE 2 or 3 portion of the neural net is to be provided by a pair of digital capacitors DC i,j and DC (M+i),j . Capacitors DC i,j and DC (M+i),j are preferably of the type described by W. E. Engeler in his allowed U. S. patent application serial No. 526,470 filed 21 May 1990, entitled "CAPACITIVE STRUCTURES FOR WEIGHTED SUMMATION, AS USED IN NEURAL NETS". The capacitances of DC i,j and DC (M+i),j are controlled in complementary ways by a digital word and its one's complement, as drawn from a respective word-storage element WSE i,j in an array of such elements located interstitially among the rows of digital capacitors and connected to form a memory. This memory may, for example, be a random access memory (RAM) with each word-storage element WSE i,j being selectively addressable by row and column address lines controlled by address decoders. Or, by way of further example, this memory can be a plurality of static shift registers, one for each column j. Each static shift register will then have a respective stage WSE i,j for storing the word that controls the capacitances of each pair of digital capacitors DC i,j and DC (M+i),j . The word stored in word storage element WSE i,j may also control the capacitances of a further pair of digital capacitors DC i,(N+j) and DC (M+i),(N+j) , respectively. Capacitors DC i,(N+j) and DC (M+i),(N+j) are also preferably of the type described by in U. S. patent application serial No. 526,470. The capacitors DC i,(N+j) and DC (M+i),(N+j) connect between "ac ground" and input lines IL i and IL (M+i) , respectively, and form parts of the loading capacitors CL i and CL (M+i) , respectively. The capacitances of DC (M+i),(N+j) and DC i,j are similar to each other and changes in their respective values track each other. The capacitances of DC i,(N+j) and DC (M+i),j are similar to each other and changes in their respective values track each other. The four digital capacitors DC i,j , DC (M+i) , DC i,(N+j) and DC (M+i),(N+j) are connected in a bridge configuration having input terminals to which the input lines IL i and IL (M+i) respectively connect and having output terminals connecting to output line OL j and to ac ground respectively. This bridge configuration facilitates making computations associated with back-propagation programming by helping make the capacitance network bilateral insofar as voltage gain is concerned. Alternatively, where the computations for back-propagation programming are done by computers that do not involve the neural net in the computation procedures, the neural net need not include the digital capacitors DC i,(N+j) and DC (M+i),(N+j) · These digital capacitors DC i,(N+j) and DC (M+i),(N+j) are not needed either if very large loading capacitors are placed on the output lines OL j , but this alternative undesirably reduces sensitivity of the row charge-sensing amplifier RQS j . A respective column driver CD i applies complementary bit-sliced digital signals to each pair of input lines IL i and IL (M+i) . The column driver CD i comprises the multiplexers MX i and MX (M+i) connected per FIGURE 2 or per FIGURE 3. When the neural net modified per FIGURE 20 is being operated normally, following programming, the  P signal applied to a mode control line MCL is a logic ZERO. This ZERO conditions a respective input line multiplexer ILM i to connect the non-inverting output port of each column driver CD i to input line IL i . The  P signal on mode control line MCL being a ZERO also conditions a respective input line multiplexer ILM (M+i) to connect the inverting output port of each column driver CD i to input line IL (M+i) . A ZERO on mode control line MCL also conditions each output line multiplexer OLM j of an N-numbered plurality thereof to select the output line OL j to the input port of a respective row charge-sensing amplifier RQS j that performs a charge-sensing operation for output line OL j . When  P signal on mode control line MCLis a ZERO, the input signal x i induces a total change in charge on the capacitors DC i,j and DC (M+i),j proportional to the difference in their respective capacitances. The resulting displacement current flow from the input port of the row charge-sensing amplifier RQS j requires that there be a corresponding displacement current flow from the Miller integrating capacitor CI j in the row charge-sensing amplifier RQS j charging that capacitor to place thereon a voltage v j defined as follows. The voltage V j is supplied to an analog-to-digital converter RADC j for the j th row of the capacitive network, which digitizes voltage V j to generate a digital signal y j for the j th row that is assumed to be a two's complement digital signal. The digitized V j signal is supplied to a selective complementor RCMP j for the j th row, which passes the digitized V j signal without change for each of the bit slices of the input signals x 1 , x 2 , ... x M except the sign bit slices. For the sign bit slices the selective complementor RCMP j one's complements the bits of the digitized V j signal and adds unity thereto. The selective complementor RCMP j supplies its output signal to an accumulator RACC j for the j th row of the capacitive network. The accumulator RACC j accumulates the bit slices of each set of samples of the input signals x 1 , x 2 , ... x M as weighted for the output line OL j , sensed and digitized; the accumulator result Sy j is latched by a respective row latch RLCH j for the j th row; and the accumulator RACC j is then reset to zero. At the same time the accumulator RACC j is reset to zero, to implement dc-restoration a reset pulse  R is supplied to the charge sensing amplifier RQS j for the j th row and to the charge sensing amplifiers for the other rows. During the dc-restoration all x i are "zero-valued". During training, the  P signal applied to mode control line MCL is a logic ONE, which causes the output line multiplexer OLM j to disconnect the output line OL j from the charge sensing amplifier RQS j and to connect the output line OL j to receive a bit-sliced δ j error term. This δ j error term is generated by a row driver RD j responsive to the bit-serial digital product output signal of a digital multiplier BSM j , responsive to a signal Δ j and to the slope stored in the slope latch SLCH j . The term Δ j for the output neural net layer is the difference between z j actual value and its desired value d j . The term Δ j for a hidden neural net layer is the Δ j output of the succeeding neural net layer during the back-propagation procedure. The row driver RD j is shown as a type having differential outputs which can be provided by a pair of multiplexers similar to input multiplexers MX i and MX (M+i) connected as in FIGURE 2 or as in FIGURE 3. Since output of only one sense is required to drive the single-ended output line OL j , as used in the neural net layers shown in FIGURES 20 and 27, however, it is preferable from the standpoint of simplifying hardware to use row drivers RD j that consist of but a single multiplexer. Indeed, the row driver function may be subsumed in the bit-serial multiplier BSM j . In the neural net layers shown in FIGURES 21, 23, 28 and 30 the row driver RD j must be of a type having differential outputs, as can be provided by a pair of multiplexers similar to input multiplexers MX i and MX (M+i) connected as in FIGURE 2 or as in FIGURE 3. During training, the  P signal applied to the mode control line MCL being a ONE also causes the input line multiplexers ILM i and ILM (M+i) to disconnect the input lines IL i and IL (M+i) from the column driver CD i output ports and connect them instead to the non-inverting and inverting input terminals of a column charge-sensing amplifier CQS j . Each successive partial summation of the weighted δ 1 , δ 2 ... δ N , signals induces a differential change in charge between input lines IL j and IL (M+i) proportional to (C i,j - C (M+i),j ), which differential change in charge is sensed using the column charge sensing amplifier CQS j , then digitized in a respective column analog-to-digital converter CADC i for the i th column. The digital output signal from the column A-to-D converter CQS i is supplied to a selective complementor CCMP i for the i th column, which responds to a SIGN BIT FLAG to selectively one's complement the bits of that digital signal as applied to an accumulator CACC i for the i th column. The selective complementor CCMP i otherwise supplies the accumulator CACC i an input signal the bits of which correspond to the bits of the digital output signal from the column A-to-D converter CQS i . The accumulator CACC i accumulates the successive partial summations of the weighted δ 1 , δ 2 , ... δ N signals to generate the final weighted summation result Δ i supplied to a parallel-in/serial-out register CPISO i for the i th column. The register CPISO i supplies the final weighted summation result Δ i in bit-serial format for the bit-serial digital multiplier of the preceding neural net layer if such there be. Resetting of the column charge sensing amplifier CQS i is normally done shortly after a ZERO to ONE transition appears in the  P signal applied to mode control line MCL and may also be done at other times. This procedure corrects for capacitive unbalances on the input lines IL i and IL (M+i) during back-propagation computations that follow the resetting procedure. FIGURE 21, comprising component FIGURES 21A and 21B, shows a representative modification that can be manifoldly made to a neural net as shown in FIGURES 4, 8 and 9. The four digital capacitors DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) are connected in a bridge configuration having input terminals connecting from the input line IL i and from a-c ground respectively and having output terminals connecting to output lines OL j and OL (N+j) respectively. When the neural net layer per FIGURES 4, 8 and 9 is being operated normally, following programming, the  P signal applied to a mode control line MCL is a logic ZERO. This ZERO on mode control line MCL conditions each output line multiplexer OLM j of an N-numbered plurality thereof to select the output line OL j to the inverting input terminal of a fully differential amplifier in an associated differential charge sensing amplifier RQS j for the j th row. This ZERO on mode control line MCL also conditions each output line multiplexer OLM (N+j) to the non-inverting input terminal of the fully differential amplifier in the associated differential charge sensing amplifier RQS j . When  p signal on mode control line MCL is a ZERO, the input signal x i induces a total differential change in charge on the capacitors DC i,j and DC i,(N+j) proportional to the difference in their respective capacitances. The resulting displacement current flows needed to keep the input terminals of differential amplifier DA j substantially equal in potential requires that there be corresponding displacement current flow from the integrating capacitor CI j and CI (N+j) differentially charging those charging capacitors to place thereacross a differential voltage v j defined as follows. The A-to-D converter RADC j for the j th row is of a type capable of digitizing the balanced output signal from the differential charge sensing amplifier RQS j . The selective complementor RCMP j , the accumulator RACC j , the row latch RLCH j for each j th row, the digital adder ADD j , the digital non-linear processor NLP j , the slope latch SLCH j , and the digital multiplier BSM j correspond with those elements as described in connection with FIGURE 20. The bit-serial product signal from the digital multiplier BSM j is supplied to the row driver RD j , which can take either of the forms the column driver CD i can take in FIGURE 20. During training, the  P signal applied to the mode control line MCL being a ONE causes the output line multiplexers OLM j and OLM (N+j) to connect the output lines OL j and OL (N+j) for receiving from the row driver RD j complementary digital responses to the bit-serial product from the digital multiplier BSM j . The ONE on the mode control line MCL also causes each input line multiplexer ILM i to disconnect each input line IL i from the output port of the logic inverter INV i and connect it instead to supply charge on a single-ended basis to the input port of a column charge-sensing amplifier CQS i . FIGURE 22, comprising FIGURES 22A and 22B, shows a representative modification that can be made manifold times to the neural net shown in FIGURES 12, 18 and 19 or in FIGURES 13, 18 and 19, for the programmable weighting of the capacitances used in performing weighted summations of synapse signals. In this type of neural net the signs of the digital signals being weighted and summed are taken into account in the partial weighted summation procedure performed on an analog basis, rather than being taken into account in the final weighted summation procedure performed on an digital basis as is the case in the type of neural net shown in FIGURE 20. Accordingly, respective straight-through digital bussing replaces the selective complementor RCMP j for each j th row and the selective complementor CCMP i for each i th column. Accordingly, also, each column driver CD i comprises the multiplexers MX i and MX (M+i) connected per FIGURE 12 or per FIGURE 13, rather than the multiplexers MX i and MX (M+i) connected per FIGURE 2 or per FIGURE 3. The row driver RD j ' is shown as a type having differential outputs which can be provided by a pair of multiplexers similar to input multiplexers MX i ' and MX (M+i) ' connected as in FIGURE 12 or as in FIGURE 13. Since output of only one sense is required to drive the single-ended output line OL j , as used in the neural net layers shown in FIGURES 22 and 29, however, it is preferable from the standpoint of simplifying hardware to use row drivers RD j ' that consist of but a single multiplexer. In the neural net layers shown in FIGURES 24 and 31 the row driver RD j ' must be of a type having differential outputs, as can be provided by a pair of multiplexers similar to input multiplexers MX i ' and MX (M+i) ' connected as in FIGURE 12 or as in FIGURE 13. The row driver RD j ' provides in the neural net layers shown in FIGURES 22, 24, 29 and 31 for taking care of the sign bit in the digital product δ i in the partial weighted summation procedure, so it need not be taken care of in the final weighted summation procedure. Accordingly, the selective complementors CCMP i for each i th column are replaced by respective straight-through bussing. In a variant of the FIGURE 21 neural net layer not shown in the drawing, each row driver RD j is also replaced by a row driver RD j ' that takes either of the forms the column driver CD i can take in FIGURE 22. In this variant, too, the row driver RD j ' provides for taking care of the sign bit in the digital product δ i in the partial weighted summation procedure, so it need not be taken care of in the final weighted summation procedure. Accordingly, the selective complementors CCMP i for each i th column are replaced by respective straight-through bussing. FIGURE 23, comprising FIGURES 23A and 23B, shows a neural net layer in which both forward propagation and back propagation through a capacitive network are carried out with balanced signals, in which digital capacitors have their capacitance values programmed from respective word storage locations within a digital memory, and in which the signs of the digital signals being weighted and summed are taken into account in the final weighted summation procedure performed on a digital basis. The FIGURE 23 neural net layer shares features with both the FIGURE 20 and the FIGURE 21 neural net layers. A respective set of four digital capacitors DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) is used at each intersection of an i th column with a j th row of the array of sets of digital capacitors, for weighting of forward propagated input signals during periods of normal operation, and for weighting of back propagated error signals during training periods. Each set of four digital capacitors DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) have their capacitance values programmed by a word stored in a word storage element WSE i,j of an interstital memory array IMA. Each set of four digital capacitors DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) are connected in a bridge configuration having input terminals respectively connecting from paired input lines IL i and IL (M+i) as in FIGURE 20A and having output terminals respectively connecting to output lines OL j and OL (N+j) as in FIGURE 21A. During periods of normal operation, a respective column driver CD i that comprises the multiplexers MX i and MX (M+i) connected per FIGURE 2 or per FIGURE 3 applies complementary bit-sliced digital input signals to each pair of input lines IL i and IL (M+i) , as in FIGURE 20A; and a charge sensing amplifier RQS j for each j th row of the array of sets of digital capacitors, is connected to differentially sense the charges on paired output lines OL j and OL (N+j) as in FIGURE 21B. During training periods a respective row driver RD j constructed similarly to each column driver CD i applies to each pair of output lines OL j and OL (N+j) complementary bit-serial responses to digital product signal from a bit-serial digital multiplier BSM j , as in FIGURE 21B; and a charge sensing amplifier CQS j for each i th column of the array of sets of digital capacitors is connected to differentially sense the charges on paired input lines IL i and IL (M+i) , as in FIGURE 20A. FIGURE 24, comprising FIGURES 24A and 24B, shows another neural net layer in which both forward propagation and back propagation through a capacitive network are carried out with balanced signals, and in which digital capacitors have their capacitance values programmed from respective word storage locations within a digital memory. In the FIGURE 24 neural net layer the signs of the digital signals being weighted and summed are taken into account in the partial weighted summation procedure performed on an analog basis, rather than being taken into account in the final weighted summation procedure performed on a digital basis. The FIGURE 24 neural net layer shares features with both the FIGURE 21 and the FIGURE 22 neural net layers. A respective set of four digital capacitors DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) is used at each intersection of an i th column with a j th row of the array of sets of digital capacitors, for weighting of forward propagated input signals during periods of normal operation, and for weighting of back propagated error signals during training periods. Each set of four digital capacitors DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) have their capacitance values programmed by a word stored in a word storage element WSE i,j of an interstital memory array IMA. Each set of four digital capacitors DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) are connected in a bridge configuration having input terminals respectively connecting from paired input lines IL i and IL (M+i) as in FIGURE 22A and having output terminals respectively connecting to output lines OL j and OL (N+j) as in FIGURE 21A. During periods of normal operation, a respective column driver CD i that comprises the multiplexers MX i and MX (M+i) connected per FIGURE 12 or per FIGURE 13 applies complementary bit-sliced digital input signals to each pair of input lines IL i and IL (M+i) , as in FIGURE 22A; and a charge sensing amplifier RQS j for each j th row of the array of sets of digital capacitors, is connected to differentially sense the charges on paired output lines OL j and OL (N+j) as in FIGURE 21B. During training periods a respective row driver RD j constructed similarly to each column driver CD i applies to each pair of output lines OL j and OL (N+j) complementary bit-serial responses to digital product signal from a bit-serial digital multiplier BSM j , as in FIGURE 21B; and a charge sensing amplifier CQS j for each i th column of the array of sets of digital capacitors is connected to differentially sense the charges on paired input lines IL i and IL (M+i) , as in FIGURE 22A. FIGURE 25 shows apparatuses for completing the back-propagation computations, as may be used with a neural net having layers as shown in FIGURES 2, 8 and 9 or in FIGURES 3, 8 and 9 modified manifoldly per FIGURE 20; or having layers as shown in FIGURES 4, 8 and 9 modified manifoldly per FIGURE 21; or having layers as shown in FIGURES 12, 18 and 19 or in FIGURES 13, 18 and 19 modified manifoldly per FIGURE 22; or having layers as shown in FIGURE 23; or having layers as shown in FIGURE 24. The weights at each word storage element WSE i,j in the. interstitial memory array IMA are to be adjusted as the i column addresses and j row addresses are scanned row by row, one column at a time. An address scanning generator ASG generates this scan of i and j addresses shown applied to interstitial memory array IMA, assuming it to be a random access memory. The row address j is applied to a row multiplexer RM that selects bit-serial δ j to one input of a bit-serial multiplier BSM 0 , and the column address i is applied to a column multiplexer CM that selects bit-serial x i to another input of the multiplier BSM 0 . Multiplier BSM 0 generates in bit-serial form the product x i δ j as reduced by a scaling factor η, which is the increment or decrement to the weight stored in the currently addressed word storage element WSE ij in the memory array IMA. A serial-in/parallel-out register SIPO is used to convert the η x i δ j signal from bit-serial to the parallel-bit form of the signals stored in the memory array IMA. The former value of weight stored in word storage element WSE ij is read from memory array IMA to a temporary storage element, or latch, TS. This former weight value is supplied as minuend to a digital subtractor SUB, which receives as subtrahend η x i δ j from the serial-in/parallel-out register SIPO. The resulting difference is the updated weight value which is written into word storage element WSE i,j in memory array IMA to replace the former weight value. FIGURE 26 shows how trained neural net layers L 0 , L 1 and L 2 are connected together in a system that can be trained. Each of the neural net layers is similar to neural net layers as shown in FIGURES 2, 8 and 9 or in FIGURES 3, 8 and 9 modified manifoldly per FIGURE 20; or in FIGURES 4, 8 and 9 modified manifoldly per FIGURE 21; or in FIGURES 12, 18 and 19 or in FIGURES 13, 18 and 19 modified manifoldly per FIGURE 22; or in FIGURE 23; or in FIGURE 24. Each of the neural net layers has a respective back-propagation processor per FIGURE 25 associated therewith. L 0 is the output neural net layer that generates z j output signals and is provided with a back-propagation processor BPP 0 with elements similar to those shown in FIGURE 25 for updating the weights stored in the interstitial memory array of L 0 . L 1 is the first hidden neural net layer which generates z h output signals supplied to the output neural net layer as its x i input signals. These z i output signals are generated by layer L 1 as its non-linear response to the weighted sum of its x h input signals. This first hidden neural net layer L 1 is provided with a back-propagation processor BPP 1 similar to BPP 0 . L 2 is the second hidden neural net layer, which generates z h output signals supplied to the first hidden neural net layer as its x h input signals. These z h output signals are generated by layer L 2 as its non-linear response to a weighted summation of its x g input signals. This second hidden layer is provided with a back-propagation processor similar to BPP 0 and to BPP 1 . FIGURE 26 presumes that the respective interstitial memory array IMA of each neural net layer L 0 , L 1 , L 2 has a combined read/write bus instead of separate read input and write output busses as shown in FIGURE 25. FIGURE 26 shows the Δ j , Δ i and Δ h signals being fed back over paths separate from the feed forward paths for z j , z i and z h signals, which separate paths are shown to simplify conceptualization of the neural net by the reader. In actuality, as previously described, a single path is preferably used to transmit z j in the forward direction and Δ j in the reverse direction, etc. Back-propagation processor BPP 0 modifies the weights read from word storage elements in neural net layer L 0 interstitial memory array by η x i δ j amounts and writes them back to the word storage elements in a sequence of read-modify-write cycles during the training procedure. Back-propagation processor BPP 1 modifies the weights read from word storage elements in neural net layer L 1 interstitial memory array by η x h δ i amounts and writes them back to the word storage elements in a sequence of read-modify-write cycles, during the training procedure. Back-propagation processor BPP 2 modifies the weights read and storage elements in neural net layer L 2 interstitial memory array by η x g δ h amounts and writes them back to the word storage element in a sequence of read-modify-write cycles during the training procedure. FIGURE 27, comprising component FIGURES 27A and 27B shows an alternative modification that can be manifoldly made to a neural net as shown in FIGURES 2, 8 and 9 or in FIGURES 3, 8 and 9 to give it training capability. This alternative modification seeks to avoid the need for a high-resolution bit-serial multiplier BSM 0 and complex addressing during back-propagation calculations in order that training can be implemented. A respective up/down counter UDC i,j is used instead of each word storage element WSE i,j . Correction of the word stored in counter UDC i,j is done a count at a time; and the counter preferably has at least one higher resolution stage in addition to those used to control the capacitances of digital capacitors DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) . Each up/down counter UDC i,j has a respective counter control circuit CON i,j associated therewith. Each counter control circuit CON i,j may, as shown in FIGURE 27A and described in detail further on in this specification, simply consist of an exclusive-OR gate XOR i,j . Responsive to a SIGN BIT FLAG, a row sign bit latch RBL j latches the sign bit of δ j , indicative of whether a row of weights should in general be decremented or incremented, to be applied via a respective row sign line RSL j to all counter control circuits (CON i,j for i=1, ...,M) in the j th row associated with that row sign bit latch RBL j . Before making a back-propagation calculation, a respective column sign bit latch CBL i latches the sign bit of x i for each i th columnar position along the row which is to be updated, to provide an indication of whether it is likely the associated weight should be decremented or incremented. Each column sign bit latch CBL i is connected to broadcast its estimate via a respective column sign line CSL i to all counter control circuits (CON i,j for j=1, ...N) in the i th column associated with that column sign bit latch CBL i . Responsive to these indications from sign bit latches CBL i and RBL j , each respective counter control circuit CON i,j decides in which direction up/down counter UDC i,j will count to adjust the weight control signals D i,j and D i,j stored therein. The counter control circuitry CON i,j should respond to the sign of +δ j being positive, indicating the response v j to be too positive, to decrease the capacitance to output line OL j that is associated with the signal x i or - x i that is positive and to increase the capacitance to output line OL j that is associated with the signal - x i or x i that is negative, for each value of i. The counter control circuitry CON i,j should respond to the sign of +δ j being negative, indicating the response v to be too negative, to increase the capacitance to output line OL j that is associated with the signal - x i or x i that is negative and to decrease the capacitance to output line OL j that is associated with the signal x i or - x i that is positive. Accordingly, counter control circuitry CON i,j may simply consist of a respective exclusive-OR gate XOR i,j as shown in FIGURE 27A, if the following presumptions are valid. Each of the digital capacitors DC i,j and DC (M+i),(N+j) is presumed to increase or decrease its capacitance as D i,j is increased or decreased respectively. Each of the digital capacitors DC (M+i),j and DC i,(N+j) is presumed to increase or decrease its capacitance as D i,j is increased or decreased respectively. A ZERO applied as up/down signal to up/down counter UDC i,j is presumed to cause counting down for D i,j and counting up for D i,j . A ONE applied as up/down signal to up/down counter UDC i,j is presumed to cause counting up for D i,j and counting down for D i,j . Column sign detector CSD i output indication is presumed to be a ZERO when x i is not negative and to be a ONE when x i is negative. Row sign detector RSD j output indication is presumed to be a ZERO when δ j is not negative and to be a ONE when δ j is negative. Since the condition where x i or δ j is zero-valued is treated as if the zero-valued number were positive, forcing a false correction which is in fact not necessary, and thus usually creating the need for a counter correction in the next cycle of back-propagation training, there is dither in the correction loops. However, the extra stage or stages of resolution in each up/down counter UDC i,j prevent high-resolution dither in the feedback correction loop affecting the capacitances of DC i,j , DC (M+i),j , DC i,(N+j) and DC (M+i),(N+j) . FIGURE 28, comprising component FIGURES 28A and 28B shows an alternative modification that can be manifoldly made to a neural net as shown in FIGURES 4, 8 and 9 to give it training capability. A respective up/down counter UDC i,j is used instead of each word storage element WSE i,j in this FIGURE 28 alternative modification; and FIGURE 28 differs from FIGURE 21 in substantially the same respects that FIGURE 27 differs from FIGURE 20. FIGURE 29, comprising component FIGURES 29A and 29B shows an alternative modification that can be manifoldly made to a neural net as shown in FIGURES 12, 18 and 19 or in FIGURES 13, 18 and 19 to give it training capability. A respective up/down counter UDC i,j is used instead of each word storage element WSE i,j in this FIGURE 29 alternative modification; and FIGURE 29 differs from FIGURE 22 in substantially the same respects that FIGURE 27 differs from FIGURE 20. In FIGURE 29 as in FIGURE 22 the signs of the digital signals being weighted and summed are taken into account in the partial weighted summation procedure performed on an analog basis, rather than being taken into account in the final weighted summation procedure performed on an digital basis as is the case in the types of neural nets shown in FIGURES 20 and 27. Accordingly, a neural net manifoldly using the FIGURE 29 modification differs from a neural net manifoldly using the FIGURE 27 modification in that respective straight-through digital bussing replaces the selective complementor RCMP j for each j th row and the selective complementor CCMP i for each i th column. Accordingly, also, in a neural net manifoldly using the FIGURE 29 modification differs from a neural net manifoldly using the FIGURE 27 modification in that each column driver CD i comprises the multiplexers MX i and MX (M+i) connected per FIGURE 12 or per FIGURE 13, rather than the multiplexers MX i and MX (M+i) connected per FIGURE 2 or per FIGURE 3. Except for these differences, a neural net manifoldly using the FIGURE 29 modification resembles, in both structure and operation, a neural net manifoldly using the FIGURE 27 modification. In a variant of the FIGURE 28 neural net layer not shown in the drawing, the row driver RD j takes either of the forms the column driver CD i can take in FIGURE 29. This provides for taking care of the sign bit in the digital product δ i in the partial weighted summation procedure, so it need not be taken care of in the final weighted summation procedure. Accordingly, the selective complementors CCMP i for each i th column are replaced by respective straight-through bussing. The extraction of the sign bit of the digital product δ i for application to the row sign line RSL j is done before the row driver RD j , of course. FIGURE 30, comprising component FIGURES 30A and 30B shows an alternative modification that can be manifoldly made to a neural net using layers as shown in FIGURE 23 to give it training capability. A respective up/down counter UDC i,j is used instead of each word storage element WSE i,j in this FIGURE 30 alternative modification; and FIGURE 30 differs from FIGURE 23 in similar respects as FIGURE 27 differs from FIGURE 20 and as FIGURE 28 differs from FIGURE 21. FIGURE 31, comprising component FIGURES 31A and 31B shows an alternative modification that can be manifoldly made to a neural net using layers as shown in FIGURE 24 to give it training capability. A respective up/down counter UDC i,j is used instead of each word storage element WSE i,j in this FIGURE 31 alternative modification; and FIGURE 31 differs from FIGURE 24 in similar respects as FIGURE 28 differs from FIGURE 21 and as FIGURE 29 differs from FIGURE 22. Hybrids of the FIGURE 30 and 31 neural net layers are possible, handling the sign bits of the digital input signals as in either one of the FIGURE 30 and 31 neural net layers and handling the sign bits of the digital error signals as in the other of the FIGURE 30 and 31 neural net layers. FIGURE 32 shows the construction of counter UDC i,j being one that has a plurality of binary counter stages BCS 1 , BCS 2 , BCS 3 that provide increasingly more significant bits of the weight control signal D i,j and of its one's complement D i,j . FIGURE 33 shows the logic within each binary counter stage which is implemented with MOS circuitry that is conventional in the art. FIGURES 32 and 33 make it clear that the opposite directions of counting for D i,j and D i,j can be controlled responsive to a ZERO or ONE up/down control signal in either of two ways, depending on whether D i,j is taken from Q outputs of the flip-flops and D i,j is taken from their Q outputs, as shown, or whether D i,j is taken from the Q outputs of the flip-flops and D i,j is taken from their Q outputs. if the latter choice had been made instead, each counter control circuit CON i,j would have to consist of a respective exclusive-NOR circuit, or alternatively the CSD i and RSD j sign detectors would have to be of opposite logic types, rather than of the same logic type. While bit-slicing the plural-bit digital synapse samples and processing the bit slices through the same capacitive weighting network is advantageous in that it provides good guarantee that the partial weighted summation results track are scaled in exact powers of two respective to each other, speedier processing can be required which necessitates bit-slicing be done on a spatial-division-multiplexing, rather than time-division multiplexing, basis. Each bit slice is then processed to generate a respective partial weighted summation result in a respective capacitive weighting network similar to the capacitive weighting networks respectively used to generate respective partial weighted summation results for the other bit slices, and a network of digital adders is used instead of an accumulator for generating each final weighted summation result by combining appropriately shifted partial weighted summation results.

Claims: A system for performing a plurality of weighted summations, N in number, of a plurality of input signals (x i ), M in number, N being an integer greater than zero, and M being an integer greater than 1; each of said input signals being a digital signal; said system being CHARACTERIZED IN THAT: said system includes apparatus for supplying a multi-bit word of each of said M input signals one bit at a time on a bit-slice basis; and said system includes apparatus (WSN 1 , WSN 2 ) for performing said N weighted summations on a bit-slice basis, said apparatus including a plurality N in number of analog weighted summers (C i,j ) identified by respective ones of first through N th , consecutive ordinal numbers, each said analog weighted summer being connected for receiving in parallel M analog input signals obtained from said M input digital signals (x i ) as identified by respective ones of first through M th , consecutive ordinal numbers and each said analog weighted summer generating a respective analog output signal (y i , z i ) as first through N th partial weighted summation results, identified by the same one of the first through N th consecutive ordinal numbers as said analog weighted summer generating it, weighted summations of the partial weighted summation results of similar ordinal number being then performed, in accordance with the significance of said bit-slices with which they are respectively associated, to generate first through N th final weighted summation results. A system as set forth in Claim 1 for performing N weighted summations of single-bit digital input signals, said system CHARACTERIZED BY said plurality N of analog weighted summers identified by respective ones of consecutive first through N th ordinal numbers together comprising: M pairs of input lines, said input lines being identified by respective ones of consecutive first through 2M th , ordinal numbers, each pair including input lines being identified by one of the M ordinal numbers and each pair being identified by the same ordinal number as the lower of the ordinal numbers identifying the input lines in said pair; respective circuitry respectively responding to said first through M th single-bit digital input signals being ONE, for placing a first prescribed voltage level on the input line identified by the corresponding ordinal number and placing a second prescribed voltage level on the input line identified by the ordinal number that is M higher; respective circuitry respectively responding to said first through M th single-bit digital input signals being ZERO, for placing said second prescribed voltage level on the input line identified by the corresponding ordinal number and placing said first prescribed voltage level on the input line identified by the ordinal number that is M higher; N output lines identified by respective ones of consecutive first through N th ordinal numbers; a plurality of weighting capacitors having respective first plates and second plates, each of said weighting capacitors having its first plate connected to one of said first through 2M th input lines and having its second plate connected to one of said first through N th output lines; and N charge sensin circuits identified by respective ones of consecutive first through N th ordinal numbers, each connected for sensing the charge on said output line identified by the corresponding ordinal number and responding to generate the respective analog output signal identified by the corresponding ordinal numbers. A system as set forth in Claim 1 for performing N weighted summations of single-bit digital input signals, said system CHARACTERIZED BY said plurality N in number of analog weighted summers identified by respective ones of consecutive first through N th ordinal numbers together comprising: M pairs of input lines, said input lines identified by respective ones of consecutive first through 2M th ordinal numbers, each pair including input lines identified by one of ordinal numbers the M and each pair being identified by the same ordinal number as the lower of the ordinal numbers identifying the input lines in said pair; respective circuitry respectively responding to said first through M th single-bit digital input signals being ONE, for placing a first prescribed voltage level on the input line identified by the corresponding ordinal number and placing a second prescribed voltage level on the input line identified by the ordinal number M that is higher; respective circuitry respectively responding to said first through M th single-bit digital input signals being ZERO, for placing a third voltage level both on the input line identified by the corresponding ordinal number and on the input line identified by the ordinal number M that is higher, said third voltage level being midway between said first and second prescribed voltage levels; N output lines identified by respective ones of consecutive first through N th ordinal numbers; a plurality of weighting capacitors having respective first plates and second plates, each of said weighting capacitors having its first plate connected to one of said first through 2M th input lines and having its second plate connected to one of said first through N th output lines; and N charge sensing circuits identified by respective ones of consecutive first through N th ordinal numbers, each said charge sensing circuit connected for sensing the charge on said output line identified by the corresponding ordinal number and responding to generate the respective analog output signal identified by the corresponding ordinal number. A system as set forth in Claim 1 for performing N weighted summations of single-bit digital input signals, said system CHARACTERIZED BY said plurality N in number of analog weighted summers identified by respective ones of consecutive first through N th ordinal numbers together comprising: N pairs of output lines, said output lines identified by respective ones of consecutive first through 2N th ordinal numbers, each air including output lines identified by one of the N ordinal numbers and each pair being identified by the same ordinal number as the lower of the ordinal numbers identifying the output lines in said pair; M input lines, said input lines identified by respective ones of consecutive first through N th ordinal numbers; respective circuitry respectively responding to said first through M th single-bit digital input signals being ZERO, for placing a first prescribed voltage level on the input line identified by the corresponding ordinal number; respective circuitry respectively responding to said first through M h single-bit digital input signals being ONE, for placing a second voltage level on the input line identified by the corresponding ordinal number; a plurality of weighting capacitors having respective first plates and second plates, each of said weighting capacitors having its first plate connected to one of said first through 2N th output lines and having its second plate connected to one of said first through M th input lines; N charge sensing circuits identified by respective ones of consecutive first through N th ordinal numbers, each of said first through N th charge sensing circuits connected for sensing the differential charge between said output line identified by the corresponding ordinal number and said output line identified by the ordinal number that is N higher, each of said first through N th charge sensing circuits responding to the differential charge it senses to generate the respective analog output signal identified by the corresponding ordinal number; and circuitry for periodically zeroing said system during which periods of zeroing said system said first prescribed voltage level is applied to the first and second plates of each of said plurality of weighting capacitors. A system for performing N weighted summations of input signals as set forth in any of Claims 1-4 CHARACTERIZED BY being included in a neural network layer with: N non-linear processors, each having a sigmoidal transfer characteristic, being receptive of a respective one of said first through N th analog output signals, and responding to generate a respective axonal response. A system for performing N weighted summations of input signals as set forth in any of Claims 1-4 CHARACTERIZED BY being included in a neural network layer with: N non-linear processors, each having a sigmoidal transfer characteristic, being receptive of a respective one of said first through N th analog output signals, and responding to generate a respective axonal response; and analog-to-digital conversion circuitry for digitizing each of said N axonal responses. A system for performing N weighted summations of input signals as set forth in any of Claims 1-4 CHARACTERIZED BY being included in combination with: analog-to-digital conversion circuitry for digitizing said first through N th analog output signals. A system for performing N weighted summations of input signals as set forth in Claim 7 CHARACTERIZED BY: each of said M digital input signals having B bits in each successive word thereof, B being an integer greater than one, each of said M digital input signals having its words aligned in time with the words of each other of said M digital input signals; and a plurality N in number of digital weighted accumulators identified by respective ones of consecutive first through N th ordinal numbers, each of said digital weighted accumulators being of a type for receiving sequentially via said analog-to-digital conversion circuitry from said analog weighted summer identified by the same ordinal number as its own B digital input signals each representative of a respective partial weighted summation result, which respective partial weighted summation results are shifted in accordance with the weights of the bit slices respectively giving rise to them and then summed to generate a respective digital output signal representative of a respective final weighted summation result. A system for performing N weighted summations of input signals as set forth in Claim 8 CHARACTERIZED BY being included in a neural network layer with: N non-linear digital circuits, each having a sigmoidal transfer characteristic and receptive of a respective one of said digital output signals representative of final weighted summation results, to generate a respective axonal response. A system for performing N weighted summations of input signals as set forth in any of Claims 1-4 CHARACTERIZED BY being included in a neural network layer with: analog-to-digital conversion circuitry for digitizing said first through N th analog output signals; and N non-linear digital circuits, each having a sigmoidal transfer characteristic and receptive of a respective one of said digitized first through N th analog output signals, to generate a respective axonal response. A system for performing N weighted summations of input signals as set forth in any of Claims 1-4 CHARACTERIZED BY: each of said M digital input signals having B bits in each successive word thereof, B being an integer greater than one, and each of said M digital input signals having its words aligned in time with the words of each other of said M digital input signals. A system for performing N weighted summations of input signals as set forth in Claim 11 CHARACTERIZED BY being included in combination with: a plurality N in number of analog weighted accumulators identified by respective ones of consecutive first through N th ordinal numbers, each connected for receiving sequentially the B successive values of said analog output signal identified by the corresponding ordinal number, as generated responsive to respective bit slices of said M input signals and weighting those B sequentially received values of said analog output signal in accordance with the weights of the bit slices respectively giving rise to them to generate a final weighted summation signal. A system for performing N weighted summations of input signals as set forth in Claim 12 CHARACTERIZED BY being included in a neural network layer with: N non-linear processors, each having a sigmoidal transfer characteristic, being receptive of a respective one of said final weighted summation signals, and responding to generate a respective axonal response. A system for performing N weighted summations of input signals as set forth in Claim 12 CHARACTERIZED BY being included in a neural network layer with: N non-linear processors, each having a sigmoidal transfer characteristic, being receptive of a respective one of said final weighted summation signals, and responding to generate a respective axonal response; and analog-to-digital conversion circuitry for digitizing each of said N axonal responses. A system for performing N weighted summations of input signals as set forth in Claim 12 CHARACTERIZED BY being included in combination with: analog-to-digital conversion circuitry for digitizing each of said final weighted summation signals. A system for performing N weighted summations of input signals as set forth in Claim 12 CHARACTERIZED BY being included in a neural network layer with: analog-to-digital conversion circuitry for digitizing each of said final weighted summation signals; and N non-linear digital circuits, each having a sigmoidal transfer characteristic and receptive of a respective one of said digitized final weighted summation signals, to generate a respective axonal response. A system for performing N weighted summations of digital input signals connected in a neural net layer, as set forth in any of Claims 9, 10, 13, 14 and 16, CHARACTERIZED BY said neural net layer being one layer of a plural-layer neural net.

Inventors: ENGELER WILLIAM ERNEST, ENGELER, WILLIAM ERNEST
Applicants: GEN ELECTRIC, GENERAL ELECTRIC COMPANY
Application year: 1991
Main classification: G06F15
IPC codes: G06G7, G06N3, G06F15, G06N99