Weighted summation circuitry with digitally controlled capacitive structures

An array of weighted summation circuits, N in number, each generate a weighted sum response to the same plurality of input signals, M in number. Each weighted summation circuit includes at least one corresponding capacitive element for determining the weighting of each of the input signals within that weighted summation circuit. At least one corresponding capacitive element is of a programmable type having its capacitance value determined in accordance with the bits of a digital word received at a control word port thereof. The array of weighted summation circuits are preferably constructed in integrated circuit form together with an interstitial memory having respective word storage elements for temporarily storing the digital words applied to the control word ports of nearby capacitive elements in the integrated circuitry.

The invention generally concerns neural nets the processors of which use 
capacitors to perform weighted summation in accordance with Coulomb's Law. 
BACKGROUND OF THE INVENTION 
U.S. Pat. No. 5,039,871 issued 13 Aug. 1991 to W. E. Engeler, entitled 
"CAITIVE STRUCTURES FOR WEIGHTED SUMMATION AS USED IN NEURAL NETS" and 
assigned to General Electric Company is incorporated herein by reference. 
U.S. Pat. No. 5,039,870 issued 13 Aug. 1991 to W. E. Engeler, entitled 
"WEIGHTED SUMMATION HAVING DIFFERENT-WEIGHT RANKS OF CAITIVE 
STRUCTURES" and assigned to General Electric Company is incorporated 
herein by reference. These patents explain in their backgrounds of 
invention the desirability of using weighted summation networks in neural 
network layers, which weighted summation networks use capacitive elements 
to determine in accordance with Coulomb's Law the weighting accorded their 
synapse input signals. Y. P. Tsividis and D. Anastassion in a letter 
"Switched-Capacitor Neural Networks" appearing in ELECTRONICS LETTERS, 
27th Aug. 1987, Vol. 23, No. 18, pages 958,959 (IEE) describe a switched 
capacitor method of implementing weighted summation in accordance with 
Coulomb's Law that is useful in analog sampled-data neural net systems. 
U.S. Pat. Nos. 5,039,870 and 5,039,871 describe methods of implementing 
weighted summation in accordance with Coulomb's Law that do not require 
capacitors to be switched. 
It not only is desirable to use capacitive elements to determine in 
accordance with Coulomb's Law the weighting accorded analog synapse input 
signals, as described specifically in U.S. patent applications Ser. Nos. 
366,838 and 366,839, it is desirable to use capacitive elements to 
determine in accordance with Coulomb's Law the weighting accorded digital 
synapse input signals. U.S. patent application Ser. No. 546,970 filed 1 
Aug. 1990 by W. E. Engeler, entitled "NEURAL NETS SUPPLIED DIGITAL SYNAPSE 
SIGNALS DIGITAL ON A BIT-SLICE BASIS" and assigned to General Electric 
Company describes digital synapse input signals being so weighted. So does 
U.S. patent application Ser. No. 628,257 filed 14 Dec. 1990 by W. E. 
Engeler, entitled "DIGITAL CORRELATORS INCORPORATING ANALOG COMPUTER 
STRUCTURES OPERATED ON A BIT-SLICED BASIS" and assigned to General 
Electric Company. 
SUMMARY OF THE INVENTION 
Apparatus embodying the invention includes an array of weighted summation 
circuits, N in number, each weighted summation circuit generating a 
weighted sum response to the same plurality of input signals, M in number, 
and including at least one corresponding capacitive element for 
determining the weighting of each of the input signals within that 
weighted summation circuit. At least one corresponding capacitive element 
is of a programmable type having its capacitance value determined in 
accordance with the bits of a digital word received at a control word port 
thereof. In preferred embodiments of the invention the array of weighted 
summation circuits are constructed in integrated circuit form together 
with an interstitial memory having respective word storage elements for 
temporarily storing the digital words applied to the control word ports of 
nearby capacitive elements in the integrated circuitry.

DETAILED DESCRIPTION 
FIG. 1 shows a neural net comprising a plurality, N in number, of 
non-linear amplifiers OD.sub.1, OD.sub.2, . . . OD.sub.(N-1), OD.sub.N. 
Each of a plurality, M in number, of input voltage signals x.sub.1, 
x.sub.2, . . . x.sub.(M-1), x.sub.M supplied as "synapse" signals is 
weighted to provide respective input voltages for the non-linear voltage 
amplifiers OD.sub.1, OD.sub.2, . . . OD.sub.(N-1 ), OD.sub.N, which 
generate respective "axon" responses y.sub.1, y.sub.2, . . . y.sub.(N-1), 
y.sub.N. This weighting is, as will be described in detail further on in 
this specification, done using capacitive structures in accordance with 
the invention. 
M is a positive plural integer indicating the number of input synapse 
signals to the FIG. 1 neural net, and N is a positive plural integer 
indicating the number of output axon signals the FIG. 1 net can generate. 
To reduce the written material required to describe operation of the FIG. 
1 neural net, operations using replicated elements will be described in 
general terms; using a subscript i ranging over all values one through M 
for describing operations and apparatuses as they relate to the (column) 
input signals x.sub.1, x.sub.2, . . . x.sub.(M-1), x.sub.M ; and using a 
subscript j ranging over all values one through N for describing 
operations and apparatus as they relate to the (row) output signals 
y.sub.1, y.sub.2, . . . y.sub.(N-1), y.sub.N. That is, i and j are the 
column and row numbers used to describe particular portions of the neural 
net. 
Input voltage signal x.sub.i is applied to the input port of an input 
driver amplifier ID.sub.i that is a voltage amplifier which in turn 
applies its voltage response to an input line IL.sub.i. Respective output 
lines OL.sub.j and OL.sub.(j+N) connect to the non-inverting input port of 
output driver amplifier OD.sub.j and to its inverting input port. The 
non-linear output driver amplifier OD.sub.j is shown in FIG. 1 as simply 
being a differential-input non-linear voltage amplifier with the quiescent 
direct potential applied to its (+) and (-) input signal terminals via 
output lines OL.sub.j and OL.sub.(j+N) being adjusted by clamping to a 
desired bias voltage at selected times using a respective direct-current 
restorer circuit DCR.sub.j. (The direct-current restorer circuit DCR.sub.j 
is shown at the left of FIG. 1 for drafting reasons, but is normally 
associated with the input port of the output driver amplifier OD.sub.j.) 
Output driver amplifier OD.sub.j generates at its output port a non-linear 
voltage response to the cumulative difference in charge on that respective 
pair of output lines OL.sub.j and OL.sub.(j+N). 
A respective capacitor C.sub.i,j connects each of the input lines IL.sub.i 
to each of the OL.sub.j output lines OL.sub.j, and a respective capacitor 
C.sub.i,(j+N) connects each of the input lines IL.sub.i to each of the 
output lines OL.sub.(j+N). Since at its output terminal the output driver 
amplifier OD.sub.j responds without inversion to x.sub.i input signal 
voltage applied to its non-inverting (+) input terminal via capacitor 
C.sub.i,j and responds with inversion to x.sub.i input signal voltage 
applied to its inverting (-) input terminal via capacitor C.sub.i,(j+N), 
respectively, the electrically equivalent circuit is x.sub.i signal 
voltage being applied to a single output line QL.sub.j by a capacitor 
having a capacitance that equals the capacitance of C.sub.i,j minus the 
capacitance of C.sub.i,(j+N). This technique of single-ended output signal 
drive to paired output lines that are differentially sensed avoids the 
need for switched-capacitance techniques in order to obtain inhibitory (or 
negative) weights as well as excitory (or positive) weights. Thus, this 
technique facilitates operating the neural net with analog signals that 
are continuous over sustained periods of time, if so desired. 
FIG. 1 shows each of the input lines IL.sub.i as being provided with a 
respective load capacitor CL.sub.i to cause that capacitive loading upon 
the output port of the input driver amplifier ID.sub.i to be substantially 
the same as that upon each output port of the other input driver 
amplifiers. This is desirable for avoiding unwanted differential delay in 
responses to the input signals x.sub.i. Substantially equal capacitive 
loading can be achieved by making the capacitance of each of the input 
line loading capacitors, CL.sub.1 through CL.sub.M, very large compared to 
the total capacitance of the capacitors C.sub.i,j connecting thereto. 
Preferably, however, this result is achieved by making the capacitance of 
each of the input line loading capacitors complement the combined value of 
the other capacitances connecting thereto. This procedure reduces the 
amount of line loading capacitance required. Where the voltages appearing 
on the output lines OL.sub.j and OL.sub.(i+N) are sensed directly by the 
non-linear output driver amplifiers OD.sub.1, . . . OD.sub.N, as shown in 
FIG. 1, this procedure makes the voltage division ratio for each input 
voltage x.sub.i, . . . x.sub.m independent of the voltage division ratios 
for the other input voltages. 
FIG. 1 also shows each of the output lines OL.sub.j being loaded with a 
respective Icad capacitor CL.sub.(M+j) and each of the output lines 
OL.sub.(N+j) being loaded with a respective load capacitor CL.sub.(M+N+j). 
This is done so that the total capacitance on each output line remains 
substantially the same as on each of the other output lines. This can be 
done by choosing CL.sub.(M+j) to be much larger than other capacitances to 
output line OL.sub.j, and by choosing CL.sub.(M+N+j) to be much larger 
than other capacitances to output line OL.sub.N+j). Alternatively, this 
can be done by choosing CL.sub.(M+j) and CL.sub.(M+N+j) to complement the 
combined value of the other capacitances connecting the same output line. 
The input voltage to output driver amplifier OD.sub.j will (to good 
approximation) have the following value, v.sub.j, in accordance with 
Coulomb's Law. 
##EQU1## 
The generation of voltage v.sub.j can be viewed as the superposition of a 
plurality of capacitive divisions between, on the one hand, the effective 
capacitance (C.sub.(i,j) -C.sub.i,(j+N)) each input voltage has to output 
line OL.sub.j and, on the other hand, the total capacitance C.sub.j of the 
output line to its surroundings. That is, C.sub.j is the total capacitance 
on output line OL.sub.j or the total capacitance on output line 
OL.sub.(N+j), which capacitances should be equal to each other and fixed 
in value. 
Each non-linear output driver amplifier OD.sub.j in the FIG. 1 neural net 
layer can be implemented using linear voltage amplifier circuitry followed 
by non-linear voltage amplifier circuitry. Each output driver amplifier 
can comprise a long-tailed pair connection of transistors having a current 
mirror amplifier load for converting their output signal voltage to 
single-ended form for application to an ensuing non-linear voltage 
amplifier. The long-tailed pair connection of transistors is a 
differential amplifier connection where their source electrodes have a 
differential-mode connection to each other and to a constant-current 
generator. An ensuing non-linear voltage amplifier can, as described in 
patent applications Ser. Nos. 366,838 and 366,839, comprise a cascade 
connection of two source-follower transistors, one an n-channel MOSFET and 
the other a p-channel MOSFET, each provided with a respective 
suitably-valued constant-current generator source load. Non-linearity of 
response in such a cascade connection comes about because (I) 
source-follower action of the n-channel MOSFET for positive-going 
excursions of its gate electrode potential becomes limited as its source 
potential approaches its drain potential V.sub.HI and (2) source-follower 
action of the p-channel MOSFET for negative-going excursions of its gate 
electrode potential becomes limited as its source potential approaches its 
drain potential V.sub.LO. At the source electrode of the output 
source-follower of the cascade connection, there is a sigmoidal response 
to a linear ramp potential applied to the gate electrode of the input 
source-follower of the cascade connection. 
Alternatively, the difference in charge appearing on the output lines 
OL.sub.j and OL.sub.(i+N) can be sensed by fully differential 
charge-sensing amplifiers preceding the non-linear voltage amplifiers in 
the output driver amplifiers. In such case the output signals from the 
charge. sensing amplifiers will be balanced with reference to a reference 
VBIAS potential. This alternative will be described presently in 
connection with FIG. 2. 
Consider now how neuron model behavior is exhibited by input driver 
amplifier ID.sub.i, capacitors C.sub.i,j and C.sub.i,(j+N), and non-linear 
output driver amplifier OD.sub.j for particular respective values of i and 
j. If the capacitance of capacitor C.sub.i,j is larger than the 
capacitance of capacitor C.sub.i,(j+N) for these particular values of i 
and j, then the output voltage y.sub.j for that j will exhibit "excitory" 
response to the input voltage x.sub.i. If the capacitances of C.sub.i,j 
and C.sub.i,(j+N) are equal for these i and j values, then the output 
voltage y.sub.j for that j should exhibit no response to the input voltage 
y.sub.j. If the capacitance of capacitor C.sub.i,j is smaller than the 
capacitance of capacitor C.sub.i(j+N) for those i and j values, then the 
output voltage y.sub.j for that j will exhibit "inhibitory" response to 
the input voltage x.sub.i. 
In some neural nets using capacitors for weighting synapse signals, the 
capacitors C.sub.i,j and C.sub.i,(j+N) for all i and j may be fixed-value 
capacitors, so there is never any alteration in the weighting of input 
voltages x.sub.i where i=1, . . . M. However, such neural nets lack the 
capacity to adapt to changing criteria for neural responses--which 
adaptation is necessary, for example, in a neural network that is to be 
connected for self-learning. It is desirable in certain applications, 
then, to provide for altering the capacitances of each pair of capacitors 
C.sub.i,j and C.sub.i,(j+N) associated with a respective pair of values of 
i and j. This alteration is to be carried out in a complementary way, so 
the sum of the capacitances of C.sub.i,j and of C.sub.i,(j+N) remains 
equal to C.sub.k. In accordance with the invention, each of a set of 
component capacitors with capacitances related in accordance with powers 
of two is selected to be a component of one or the other of the pair of 
capacitors C.sub.i,j and C.sub.i,(j+N), the selecting being done by field 
effect transistors operated as transmission gates. 
FIG. 2, comprising component FIGS. 2A and 2B, shows a representative 
modification that can be made to the FIG. 1 neural net near each set of 
intersections of output lines OL.sub.j and OL.sub.(j+N) with an input line 
IL.sub.i from which they receive with differential weighting a synapse 
input signal x.sub.i. Such modifications together make the neural net 
capable of being trained. Each capacitor pair C.sub.i,j and C.sub.i,(j+N) 
of the FIG. 1 neural net is to be provided by a pair of digital capacitors 
DC.sub.i,j and DC.sub.i,(j+N). The capacitances of DC.sub.i,j and 
DC.sub.i,(j+N) are controlled in complementary ways by a weighting factor 
and its one's complement as described by a digital word stored in a 
respective word-storage element WSE.sub.i,j of an array of such elements 
located interstitially among the rows of digital capacitors and connected 
to form a memory. This memory may, for example, be a random access memory 
(RAM) with each word-storage element WSE.sub.i,j being selectively 
addressable by row and column address lines controlled by address 
decoders. Or, by way of further example, this memory can be a plurality of 
static shift registers, one for each column j. Each static shift register 
will then have a respective stage WSE.sub.i,j for storing the word that 
controls the capacitances of each pair of digital capacitors DC.sub.i,j 
and DC.sub.i,(j+N). 
The word stored in word storage element WSE.sub.i,j may also control the 
capacitances of a further pair of digital capacitors DC.sub.(i+M),j and 
DC.sub.(i+M),(j+N), respectively. The capacitors DC.sub.(i+M),j and 
DC.sub.(i+M),(j+N) connect between "ac ground" and output lines OL.sub.j 
and OL.sub.(j+N), respectively, and form parts of the loading capacitors 
CL.sub.(M+j). The capacitances of DC.sub.(i+2M,j) and DC.sub.i,j are 
similar to each other and changes in their respective values track each 
other. The capacitances of DC.sub.(i+M),(j+N) and DC.sub.i,(j+N) are 
similar to each other and changes in their respective values track each 
other. The four digital capacitors DC.sub.i,j, DC.sub.i,(j+N), 
DC.sub.(i+M),j and DC.sub.(ik+M),(j+N) are connected in a quad or bridge 
configuration having input terminals connecting from the input line 
IL.sub.i and from a-c ground respectively and having output terminals 
connecting to output lines OL.sub.j and OL.sub.(j+N) respectively. This 
configuration facilitates making computations associated with 
back-propagation programming by helping make the capacitance network 
bilateral insofar as voltage gain is concerned. Alternatively, where the 
computations for back-propagation programming are done by computers that 
do not involve the neural net in the computation procedures, the neural 
net need not include the digital capacitors DC.sub.(i+M),j and 
DC.sub.(i+M),(j+N). 
When the FIG. 2 neural net is being operated normally, following 
programming, the .phi..sub.P signal applied to a mode control line MCL is 
a logic ZERO. This ZERO on mode control line MCL conditions each output 
line multiplexer OLM.sub.j of an N-numbered plurality thereof to select 
the output line OL.sub.j to the inverting input terminal of a respective 
associated fully differential amplifier DA.sub.j. This ZERO on mode 
control line MCL also conditions each output line multiplexer 
OLM.sub.(j+N) to select the output line OL.sub.(j+N) to the non-inverting 
input terminal of the respective associated fully differential amplifier 
DA.sub.j, which is included in a respective differential charge-sensing 
amplifier DQS.sub.j that performs a charge-sensing operation for output 
line OL.sub.i. A fully differential amplifier constructed of MOS 
field-effect transistors, as may serve for any one of the fully 
differential amplifiers DA.sub.j for j=1, 2, . . . N, is described on 
pages 255-257 of the book Analog MOS Integrated Circuits for Signal 
Processing by R. Gregorian and G. C. Temes, copyright 1986 , published by 
John Wiley & Sons, Inc., of New York, Chichester, Brisbane, Toronto and 
Singapore. 
In furtherance of this charge-sensing operation, a transmission gate 
TG.sub.j responds to the absence of a reset pulse .phi..sub.R to connect 
an integrating capacitor CI.sub.j between the (+) output and (-) input 
terminals of amplifier DA.sub.j ; and a transmission gate TG.sub.(j+5N) 
responds to the absence of the reset pulse .phi..sub.R to connect an 
integrating capacitor CI.sub.(j+N) between the (-) output and (+)input 
terminals of amplifier DA.sub.j. With integrating capacitors CI.sub.j and 
CI.sub.(j+N) so connected, amplifier DA.sub.j functions as a differential 
charge amplifier. When .phi..sub.p signal on mode control line MCL is a 
ZERO, the input signal x.sub.i induces a total differential change in 
charge on the capacitors DC.sub.i,j and DC.sub.i,(j+N) proportional to the 
difference in their respective capacitances. The resulting displacement 
current flows needed to keep the input terminals of differential amplifier 
DA.sub.j substantially equal in potential requires that there be 
corresponding displacement current flow from the integrating capacitor 
CI.sub.j and CI.sub.(j+N) differentially charging those charging 
capacitors to place thereacross a differential voltage v.sub.j defined as 
follows. 
##EQU2## 
The half V.sub.j signal from the non-inverting (+) output terminal of 
amplifier DA.sub.j is supplied to a non-linear voltage amplifier circuit 
NL.sub.j which can be the non-linear voltage amplifier circuit using a 
cascade connection of p-channel and n-channel source-follower field effect 
transistors as previously described. The non-linear voltage amplifier 
circuit NL.sub.j responds to generate the axon output response y.sub.j. It 
is presumed that this non-linear voltage amplifier NL.sub.j supplies 
y.sub.j at a relatively low source impedance as compared to the input 
impedance offered by the circuit y.sub.j is to be supplied to--e.g. on an 
input line in a succeeding neural net layer. If this is so there is no 
need in a succeeding neural net layer to interpose an input driver 
amplifier ID.sub.i as shown in FIG. 1. This facilitates interconnections 
between successive neural net layers being bilateral. An output line 
multiplexer OLM.sub.j responds to the .phi..sub.P signal appearing on the 
mode control line MCL being ZERO to apply y.sub.j to an input line of a 
succeeding neural net layer if the elements shown in are in a hidden 
layer. If the elements shown in FIG. 2 are in the output neural net layer, 
output line multiplexer OLM.sub.j responds to the .phi..sub.P signal on 
the mode control line being ZERO to apply y.sub.j to an output terminal 
for the neural net. 
From time to time, the normal operation of the neural net is interrupted; 
and, to implement dc-restoration a reset pulse .phi..sub.R is supplied to 
the differential charge sensing amplifier. DQS.sub.j. Responsive to 
.phi..sub.R, the logic complement of the reset pulse .phi..sub.R, going 
low when .phi..sub.R goes high, transmission gates TG.sub.j and 
TG.sub.(j+5N) are no longer rendered conductive to connect the integrating 
capacitors CI.sub.j and CI.sub.(j+N) from the output terminals of 
differential amplifier DA.sub.j. Instead, transmission gates TG.sub.(j+N) 
and TG.sub.(j+4N) respond to .phi..sub.R going high to connect to 
V.sub.BIAS the plates of capacitor CI.sub.j and CI.sub.(j+N) normally 
connected from those output terminals, V.sub.BIAS being the 2.5 volt 
intermediate potential between the V.sub.SS =0 volt and V.sub.DD =5 volt 
operating voltages of differential amplifier DA.sub.j. Other transmission 
gates TG.sub.(j+2N) and TG.sub.(j+3N) respond to .phi..sub.R going high to 
apply direct-coupled degenerative feedback from the output terminal of 
differential amplifier DA.sub.j to its input terminals, to bring the 
voltage at the output terminals to that supplied to its inverting input 
terminal from output lines OL.sub.j and OL.sub.(j+N). During the 
dc-restoration all x.sub.i are "zero-valued". So the charges on 
integrating capacitors CI.sub.j and CI.sub.(j+N) are adjusted to 
compensate for any differential direct voltage error occurring in the 
circuitry up to the output terminals of differential amplifier DA.sub.j. 
Dc-restoration is done concurrently for ail differential amplifiers 
DA.sub.j (i.e., for values of j ranging from one to N). 
During training, the .phi..sub.P signal applied to mode control line MCL is 
a logic ONE, which causes the output line multiplexer OLM.sub.j to 
disconnect the output lines OL.sub.j and OL.sub.(j+N) from the (+) and (-) 
input terminals of differential amplifier DA.sub.j and to connect the 
output lines OL.sub.j and OL.sub.(j+N) to receive +.delta..sub.j and 
-.delta..sub.j error terms. These +.delta..sub.j and -.delta..sub.j error 
terms are generated as the balanced product output signal of a analog 
multiplier AM.sub.j, responsive to a signal .DELTA..sub.j and to a signal 
y'.sub.j which is the change in output voltage y.sub.j of non-linear 
amplifier NL.sub.j for unit change in the voltage on output line OL.sub.j. 
The term .DELTA..sub.j for the output neural net layer is an error signal 
that is the difference between y.sub.j actual value and its desired value 
d.sub.j. The term .DELTA..sub.j for a hidden neural net layer is also an 
error signal, which is of a nature that will be explained in detail 
further on in this specification. 
Differentiator DF.sub.j generates the signal y'.sub.j, which is a 
derivative indicative of the slope of y.sub.j change in voltage on output 
line OL.sub.j, superposed on V.sub.BIAS. To determine the y'.sub.j 
derivative, a pulse doublet comprising a small positive-going pulse 
immediately followed by a similar-amplitude negative-going pulse is 
introduced at the inverting input terminal of differential amplifier 
DA.sub.j (or equivalently, the opposite-polarity doublet pulse is 
introduced at the non-inverting input terminal of differential amplifier 
DA.sub.j) to first lower y.sub.j slightly below normal value and then 
raise it slightly above normal value. This transition of y.sub.j from 
slightly below normal value to slightly above normal value is applied via 
a differentiating capacitor CD.sub.j to differentiator DF.sub.j. 
Differentiator DF.sub.j includes a charge sensing amplifier including a 
differential amplifier DA.sub.(j+N) and an integrating capacitor 
CI.sub.(j+2N). During the time y.sub.j that is slightly below normal 
value, a reset pulse .phi..sub.S is applied to transmission gates 
TG.sub.(j+7N) and TG.sub.(j+8N) to render them conductive. This is done to 
drain charge from integrating capacitor CI.sub.(j+N), except for that 
charge needed to compensate for DA.sub.(j+N) input offset voltage error. 
The reset pulse .phi..sub.S ends, rendering transmission gates 
TG.sub.(j+7N) and TG.sub.(j+5N) no longer conductive, and the 
complementary signal .phi..sub.S goes high to render a transmission gate 
TG.sub.(j+6N) conductive for connecting integrating capacitor 
CI.sub.(j+2N) between the output and inverting-input terminals of 
differential amplifier DA.sub.(j+N). 
With the charge-sensing amplifier comprising elements DA.sub.(j+N) and 
CI.sub.(j+N) reset, the small downward pulsing of y.sub.j from normal 
value is discontinued and the small upward pulsing of y.sub.j from normal 
value occurs. The transition between the two abnormal conditions of 
y.sub.j is applied to the charge-sensing amplifier by electrostatic 
induction via differentiating capacitor CD.sub.j. Differential amplifier 
DA.sub.(j+N) output voltage changes by an amount y'.sub.j from the 
V.sub.BIAS value it assumed during reset. The use of the transition 
between the two pulses of the doublet, rather than the edge of a singlet 
pulse, to determine the derivative y'.sub.j makes the derivative-taking 
process treat more similarly those excitory and inhibiting responses of 
the same amplitude. The doublet pulse introduces no direct potential 
offset error into the neural net layer. 
Responsive to a pulse .phi..sub.T, the value y'.sub.j +V.sub.BIAS from 
differentiator DF.sub.j is sampled and held by row sample and hold circuit 
SH.sub.j for application to analog multiplier AM.sub.j as an input signal. 
This sample and hold procedure allows y.sub.j to return to its normal 
value, which is useful in the output layer to facilitate providing y.sub.j 
for calculating (y.sub.j -d.sub.j). The sample and hold circuit SH.sub.j 
may simply comprise an L-section with a series-arm transmission-gate 
sample switch and a shunt-leg hold capacitor, for example. The analog 
multiplier AM.sub.j accepts a first push-pull input signal between input 
terminals IN1 and IN1, accepts a second push-pull input signal between 
terminals IN2 and IN2, and supplies product output signal in balanced form 
at its output terminals POUT and POUT. The difference between y'.sub.j 
+V.sub.BIAS and V.sub.BIAS voltages is applied as a differential input 
signal to the analog multiplier AM.sub.j, which exhibits common-mode 
rejection for the V.sub.BIAS term. In U.S. patent applications Ser. Nos. 
366,838 and 366,839 respectively entitled "NEURAL NET USING CAITIVE 
STRUCTURES CONNECTING INPUT LINES AND DIFFERENTIALLY SENSED OUTPUT LINE 
PAIRS" and "NEURAL NET USING CAITIVE STRUCTURES CONNECTING OUTPUT LINES 
AND DIFFERENTIALLY DRIVEN INPUT LINE PAIRS", the four-quadrant analog 
multiplier AM.sub.j is described as being a push-pull-output analog 
multiplier formed by modifying a single-ended-output analog multiplier 
described by K. Bultt and H. Wallinga in their paper "A CMOS Four-quadrant 
Analog Multiplier" appearing on pages 430-435 of the IEEE JOURNAL OF SOLID 
STATE CIRCUITS, Vol. SC-21, No. 3, June 1986, incorporated herein by 
reference. 
During training, the .phi..sub.P signal applied to mode control line MCL is 
a logic ONE, as previously noted. When the FIG. 2 elements are in the 
output layer, the ONE on mode control line MCL conditions an output 
multiplexer OM.sub.j to discontinue the application of y.sub.j signal from 
non-linear amplifier NL.sub.j to an output terminal. Instead, the output 
multiplexer OM.sub.j connects the output terminal to a charge-sensing 
amplifier QS.sub.j. Charge sensing amplifier QS.sub.j includes a 
differential amplifier DA.sub.(j+2N) and an integrating capacitor 
CI.sub.(j+2N) and is periodically reset responsive to a reset pulse 
.phi..sub.U. Reset pulse .phi..sub.U can occur simultaneously with reset 
pulse .phi..sub.S, for example. Output signal .DELTA..sub.j from 
charge-sensing amplifier QS.sub.j is not used in the output layer, 
however. Analog multiplier AM.sub.j does not use .DELTA..sub.j +V.sub.BIAS 
and V.sub.BIAS as a differential input signal in the output layer, 
(y.sub.j -d.sub.j) being used instead. 
When the FIG. 2 elements are in a hidden neural net layer, .phi..sub.P 
signal on the mode control line MCL being a ONE conditions output 
multiplexer OM.sub.j to discontinue the application of y.sub.j signal from 
non-linear amplifier NL.sub.j to the input line IL.sub.j of the next 
neural net layer. Instead, output multiplexer OM.sub.j connects the input 
line IL.sub.j to charge-sensing amplifier QS.sub.j. Charge-sensing 
amplifier QS.sub.j senses change in the charge on input line IL.sub.j 
during training to develop a .DELTA..sub.j error signal superposed on 
V.sub.BIAS direct potential. The difference between .DELTA..sub.j 
+V.sub.BIAS and V.sub.BIAS voltages is used as a differential input signal 
to analog multiplier AM.sub.j, which multiplier exhibits common-mode 
rejection for the V.sub.BIAS term. 
Charge-sensing amplifier QS.sub.j employs differential-input amplifier 
DA.sub.(j+2N) and integrating capacitor CI.sub.(j+2N). Transmission gates 
TG.sub.(i+gN.sub.), TG.sub.(j+10N) and TG.sub.(j+11N) cooperate to provide 
occasional resetting of charge conditions on the integrating capacitor 
CI.sub.j+2N responsive to the reset pulse .phi..sub.U. 
FIG. 3 shows a neural net in which the input driver amplifier ID.sub.i 
applies, in response to input voltage signal x.sub.i, not only a 
non-inverted voltage response from its (+) output port to an input line 
ILi, but also an inverted voltage response from its (-) output port to an 
input line IL.sub.(i+M). A respective degenerative feedback connection 
from its (+) output terminal to its (-) input terminal conditions each of 
the input driver amplifiers ID.sub.i in the FIG. 3 neural net to provide 
x.sub.i voltage-follower response at its (+) output terminal to x.sub.i 
signal applied to its (+) input terminal and to provide inverted, -x.sub.i 
response at its (-) output terminal. A respective single output line 
OL.sub.j connects to the input port of output driver amplifier OD.sub.j 
,which generates at its output port a non-linear voltage response to the 
cumulative charge on that respective output line OL.sub.j. 
The non-linear output driver amplifier OD.sub.j is shown in FIG. 3 as being 
just a non-linear voltage amplifier with the quiescent direct potential 
applied to its input signal terminal via output line OL.sub.j being 
adjusted by clamping to a desired bias voltage at selected times using a 
respective direct-current restorer circuit DCR.sub.j. A respective 
capacitor C.sub.i,j connects each of the input lines IL.sub.i to each of 
the output lines OL.sub.j, and a respective capacitor C.sub.(i+M),j 
connects to each of the output lines OL.sub.j the one of the input lines 
IL.sub.(i+M) paired with that IL.sub.i. Since the paired IL.sub.i and 
IL.sub.(i+M) input lines are driven with x.sub.i and -x.sub.i signal 
voltages respectively, the electrically equivalent circuit is x.sub.i 
signal voltage being applied to output line OL.sub.j by a capacitor having 
a capacitance that equals the capacitance of C.sub.i,j minus the 
capacitance of C.sub.(i+M),j. This balanced input signal drive to paired 
input lines technique avoids the need for switched-capacitance techniques 
in order to obtain inhibitory as well as excitory weights, and thus 
facilitates operating the neural net with analog signals that are 
continuous over sustained periods of time, if so desired. 
FIG. 3 shows each of the input lines IL.sub.i or IL.sub.(i+M) as being 
provided with a respective load capacitor CL.sub.i or CL.sub.(i+M) to 
cause that capacitive loading upon each of the output terminals of the 
input driver amplifier ID.sub.i to be substantially the same as that upon 
each output port of the other input driver amplifiers. This is desirable 
for avoiding unwanted differential delay in responses to the input signals 
x.sub.i. Substantially equal capacitive loading can be achieved by making 
the capacitance of each of the input line loading capacitors CL.sub.1 
-CL.sub.2M very large compared to the total capacitance of the capacitors 
C.sub.i,j or C.sub.(i+M),j connecting thereto. Preferably, however, this 
result is achieved by making the capacitance of each of the input line 
loading capacitors complement the combined value of the other capacitances 
connecting thereto. This procedure reduces the amount of line loading 
capacitance required. Where the voltage appearing on the output lines is 
sensed directly by the non-linear output driver amplifiers OD.sub.1, . . . 
OD.sub.N, as shown in FIG. 3, this preferable procedure makes the voltage 
division ratio for each input voltage x.sub.i, . . . x.sub.M independent 
of the voltage division ratios for the other input voltages. Where the 
charge appearing on the output lines is sensed by charge-sensing 
amplifiers preceding the non-linear output driver amplifiers, as will be 
described later on in this specification in connection with FIG. 4, this 
latter consideration is not as important. 
FIG. 3 also shows each of the output lines OL.sub.j being loaded with a 
respective load capacitor CL.sub.(2M+j) to cause the total capacitance on 
that line to remain substantially the same as on each of the other output 
lines. Again, this can be done either by choosing CL.sub.(2M+j) to be much 
larger than other capacitances to output line OL.sub.j, or by choosing 
CL.sub.(2M+j) to complement the combined value of the other capacitances 
connecting thereto. The input voltage to output driver amplifier OD.sub.i 
will (to good approximation) have the following value, v.sub.j, in 
accordance with Coulomb's Law. 
##EQU3## 
Here C.sub.j is the total capacitance on output line OL.sub.j. The 
generation of voltage v.sub.j can be viewed as the superposition of a 
plurality of capacitive divisions between, on the one hand, the effective 
capacitance (C.sub.(i,j) -C.sub.(i+M),j) each input voltage has to output 
line OL.sub.j and, on the other hand, the total capacitance C.sub.j of the 
output line to its surroundings. 
Consider now how neuron model behavior is exhibited by input driver 
amplifier ID.sub.i, capacitors C.sub.i,j and C.sub.(i+M),j, and non-linear 
output driver amplifier OD.sub.j for particular respective values of i and 
j. The voltage responses input driver amplifier ID.sub.i applies to input 
lines IL.sub.i and IL.sub.(i+M) are the same in amplitude but are of 
opposing polarity as referred to a common-mode voltage that is designed to 
be nominally the same as a bias voltage V.sub.BIAS midway between the 
0-volt V.sub.SS and +5-volt V.sub.DD supply voltages. If the capacitance 
of capacitor C.sub.i,j is larger than the capacitance of capacitor 
C.sub.(i+M),j for these particular values of i and j, then the output 
voltage y.sub.j for that j will exhibit "excitory" response to the input 
voltage x.sub.i. If the capacitances of C.sub.i,j and C.sub.(i+M),j are 
equal for these i and j values, then the output voltage y.sub.j for that j 
should exhibit no response to the input voltage y.sub.j. If the 
capacitance of capacitor C.sub.i,j is smaller than the capacitance of 
capacitor C.sub.(i+M),j for those i and j values, then the output voltage 
y.sub.j for that j will exhibit "inhibitory" response to the input voltage 
x.sub.i. 
In some neural nets constructed in accordance with the invention the 
capacitors C.sub.i,j and C.sub.(i+M)j for all i and j may be fixed-value 
capacitors, so there is never any alteration in the weighting of input 
voltages x.sub.i where i=1, . . . M. However, such neural nets lack the 
capacity to adapt to changing criteria for neural responses--which 
adaptation is necessary, for example, in a neural network that is to be 
connected for self-learning. It is desirable in certain applications, 
then, to provide for altering the capacitances of each pair of capacitors 
C.sub.i,j and C.sub.(i+M),j associated with a respective pair of values of 
i and j. This alteration is to be carried out in a complementary way, so 
the sum of the capacitances of C.sub.i,j and of C.sub.(i+M),j remains 
equal to C.sub.k. In accordance with the invention, each of a set of 
component capacitors with capacitances related in accordance with powers 
of two is selected to be a component of one or the other of the pair of 
capacitors C.sub.i,j and C.sub.(i+M),j, the selecting being done by field 
effect transistors operated as transmission gates. 
FIG. 4, comprising component FIGS. 4A and 4B, shows a representative 
modification that can be made to the FIG. 3 neural net near each set of 
intersections of an output line OL.sub.j with input lines IL.sub.i and 
IL.sub.(i+M) driven by opposite senses of a synapse input signal x.sub.i. 
Such modifications together make the neural net capable of being trained. 
Each capacitor pair C.sub.i,j and C.sub.(i+M),j of the FIG. 3 neural net 
is to be provided by a pair of digital capacitors DC.sub.i,j and 
DC.sub.(i+M),j. The capacitances of DC.sub.i,j and DC.sub.(i+M),j are 
controlled in complementary ways by a digital word, as drawn from a 
respective word-storage element WSE.sub.i,j in an array of such elements 
located interstitially among the rows of digital capacitors and connected 
to form a memory. This memory may, for example, be a random access memory 
(RAM) with each word-storage element WSE.sub.i,j being selectively 
addressable by row and column address lines controlled by address 
decoders. Or, by way of further example, this memory can be a plurality of 
static shift registers, one for each column j. Each static shift register 
will then have a respective stage WSE.sub.i,j for storing the word that 
controls the capacitances of each pair of digital capacitors DC.sub.i,j 
and DC.sub.(i+M),j. 
The word stored in word storage element WSE.sub.i,j may also control the 
capacitances of a further pair of digital capacitors DC.sub.i,(j+N) and 
DC.sub.(i+M),(j+N), respectively. The capacitors DC.sub.i,(j+N) and 
DC.sub.(i+M),(j+N) connect between "ac ground" and input lines IL.sub.i 
and IL.sub.(i+M), respectively, and form parts of the loading capacitors 
CL.sub.i and CL.sub.(i+M), respectively. The capacitances of 
DC.sub.(i+M,(j+N) and DC.sub.i,j are similar to each other and changes in 
their respective values track each other. The four digital capacitors 
DC.sub.i,j, DC.sub.(i+M),j, DC.sub.i,(j+N) and DC.sub.(i+M),(j+N) are 
connected in a quad or bridge configuration having input terminals to 
which the input lines Il.sub.i and IL.sub.(i+M) respectively connect and 
having output terminals connecting to output line QL.sub.j and to ac 
ground respectively. The capacitances of DC.sub.i,(j+N) and DC.sub.(i+M),j 
are similar to each other and changes in their respective values track 
each other. This configuration facilitates making computations associated 
with back-propagation programming by helping make the capacitance network 
bilateral insofar as voltage gain is concerned. Alternatively, where the 
computations for back-propagation programming are done by computers that 
do not involve the neural net in the computation procedures, the neural 
net need not include the digital capacitors DC.sub.i,j +N and 
DC.sub.(i+M),(j+N). These digital capacitors DC.sub.i,(j+N) and 
DC.sub.(I+M),(j+N) are not needed either where very large loading 
capacitors are placed on the output lines OL.sub.j, but this alternative 
undesirably reduces sensitivity of the output driver amplifier OD.sub.j. 
When the FIG. 4 neural net is being operated normally, following 
programming, the .phi..sub.P signal applied to a mode control line MCL.is 
a logic ZERO. This ZERO conditions a respective input line multiplexer 
ILMi to connect the non-inverting output port at each input driver 
amplifier ID.sub.i to input line IL.sub.i. The .phi..sub.P signal on mode 
control line MCL being a ZERO also conditions a respective input line 
multiplexer ILM.sub.(i+M) to connect the inverting output port of each 
input driver amplifier ID.sub.i to input line IL.sub.(i+M). 
A ZERO on mode control line MCL also conditions each output line 
multiplexer OLM.sub.j of an n-numbered plurality thereof to select the 
output line OL.sub.j to the inverting input terminal of a respective 
associated differential-input amplifier DA.sub.j, included in a respective 
charge-sensing amplifier QS.sub.j that performs a charge-sensing operation 
for output line OL.sub.j. In furtherance of this charge-sensing operation, 
a transmission gate TG.sub.j responds to the absence of a reset pulse 
Q.sub.R to connect an integrating capacitor CI.sub.j between the output 
and inverting-input terminals of differential-input amplifier DA.sub.j. 
Amplifier DA.sub.j may be an operational amplifier of the conventional 
voltage amplifier type or may be an operational transconductance 
amplifier. With integrating capacitor CI.sub.j so connected, amplifier 
DA.sub.j functions as a charge amplifier. When .phi..sub.P signal on mode 
control line MCL is a ZERO, the input signal x.sub.i induces a total 
change in charge on the capacitors DC.sub.i,j and DC.sub.(i+M),j 
proportional to the difference in their respective capacitances. The 
resulting displacement current flow from the inverting input terminal of 
differential-input amplifier DA.sub.j requires that there be a 
corresponding displacement current flow from the integrating capacitor 
CI.sub.j charging that capacitor to place thereon a voltage v.sub.j 
defined as follows. 
##EQU4## 
The voltage V.sub.j is supplied to a non-linear voltage amplifier circuit 
NL.sub.j, which non-linear voltage amplifier circuit responds to generate 
the axon output response y.sub.j. 
From time to time, the normal operation of the neural net is interrupted, 
and to implement dc-restoration a reset pulse .phi..sub.R is supplied to 
each charge sensing amplifier QS.sub.j. Responsive to .phi..sub.R the 
logic complement of the reset pulse .phi..sub.R, going low when 
.phi..sub.R goes high, transmission gate TG.sub.j is no longer rendered 
conductive to connect the integrating capacitor CI.sub.j from the output 
terminal of differential amplifier DA.sub.j. Instead, a transmission gate 
TG.sub.(j+N) responds to .phi..sub.R going high to connect to V.sub.BIAS 
the plate of capacitor CJ.sub.j normally connected from that output 
terminal, V.sub.BIAS being the 2.5 volt intermediate potential between the 
V.sub.SS =0 volt and V.sub.DD =5 volt operating voltages of differential 
amplifier DA.sub.j. Another transmission gate TG.sub.(j+2N) responds to 
.phi..sub.R going high to apply direct-coupled feedback from the output 
terminal of differential amplifier DA.sub.j to its inverting input 
terminal, to bring the voltage at the output terminal to that supplied to 
its inverting input terminal from output line OL.sub.j. During the 
dc-restoration all x.sub.i are "zero-valued". So the charge on integrating 
capacitor CI.sub.j is adjusted to compensate for any direct voltage error 
occurring in the circuitry up to the output terminal of differential 
amplifier DA.sub.j. DC-restoration is done concurrently for all 
differential amplifiers DA.sub.j (i.e., for values of j ranging from one 
to N). 
During training, the .phi..sub.P signal applied to mode control line MCL is 
a logic ONE, which causes the output line multiplexer OLM.sub.j to 
disconnect the output line OL.sub.j from the inverting input terminal of 
differential amplifier DA.sub.j and to connect the output line OL.sub.j to 
receive a .delta..sub.j error term. This .delta..sub.j error term is 
generated as the product output signal of an analog multiplier AM.sub.j, 
responsive to a signal .DELTA..sub.j and to a signal y'.sub.j which is the 
change in output voltage y.sub.j of non-linear amplifier NL.sub.j for unit 
change in the voltage on output line OL.sub.j. The term .DELTA..sub.j is 
for the output neural net layer the difference between y.sub.j actual 
value and its desired value d.sub.j. The term .DELTA..sub.j is for a 
hidden neural net layer the .DELTA..sub.j output of the succeeding neural 
net layer during the back-propagation procedure. 
Differentiator DF.sub.j generates the signal y'.sub.j, which is a 
derivative indicative of the slope of y.sub.j change in voltage on output 
line OL.sub.j, superposed on V.sub.BIAS. To determine the y'.sub.j 
derivative, a pulse doublet comprising a small positive-going pulse 
immediately followed by a similar-amplitude negative-going pulse is 
introduced at the inverting input terminal of differential amplifier 
DA.sub.j (or equivalently, the opposite-polarity doublet pulse is 
introduced at the non-inverting input terminal of differential amplifier 
DA.sub.j) to first lower y.sub.j slightly below normal value and then 
raise it slightly above normal value. This transition of y.sub.j from 
slightly below normal value to slightly above normal value is applied via 
a differentiating capacitor CD.sub.j to differentiator DF.sub.j. 
Differentiator DF.sub.j includes a charge sensing amplifier including a 
differential amplifier DA.sub.(j+N) and an integrating capacitor 
CI.sub.(j+N). During the time that y.sub.j is slightly below normal value, 
a reset pulse .phi..sub.S is applied to transmission gates TG.sub.(j+4N) 
and TG.sub.(j+5N) to render them conductive. This is done to drain charge 
from integrating capacitor CI.sub.(j+N), except for that charge needed to 
compensate for DA.sub.(j+N) input offset voltage error. The reset pulse 
.phi..sub.S ends, rendering transmission gates TG.sub.(j+4N) and 
TG.sub.(j+5N) no longer conductive, and the complementary signal 
.phi..sub.S goes high to render a transmission gate TG.sub.(j+3N) 
conductive for connecting integrating capacitor CI.sub.(i+N) between the 
output and inverting-input terminals of differential amplifier 
DA.sub.(j+N). 
With the charge sensing amplifier comprising elements DA.sub.(j+N) and 
CI.sub.(j+N) reset, the small downward pulsing of y.sub.j from normal 
value is discontinued and the small upward pulsing of y.sub.j from normal 
value occurs. The transition between the two abnormal conditions of 
y.sub.j is applied to the charge sensing amplifier by electrostatic 
induction via differentiating capacitor CD.sub.j. Differential amplifier 
DA.sub.(i+N) output voltage changes by an amount y'.sub.j from the 
V.sub.BIAS value it assumed during reset. The use of the transition 
between the two pulses of the doublet, rather than the edge of a singlet 
pulse, to determine the derivative y'.sub.j makes the derivative-taking 
process treat more similarly those excitory and inhibiting responses of 
the same amplitude. The doublet pulse introduces no direct potential 
offset error into the neural net layer. 
Responsive to a pulse .phi..sub.T, the value y'.sub.j +V.sub.BIAS from 
differentiator DF.sub.j is sampled and held by (row) sample and hold 
circuit RSH.sub.j for application to the analog multiplier AM.sub.j as an 
input signal. This sample and hold procedure allows y.sub.j to return to 
its normal value, which is useful in the output layer to facilitate 
providing y.sub.j for calculating (y.sub.j -d.sub.j). The sample and hold 
circuit RSH.sub.j may simply comprise an L-section with a series-arm 
transmission-gate sample switch and a shunt-leg hold capacitor, for 
example. The difference between y.sub.j +V.sub.BIAS and V.sub.BIAS 
voltages is used as a differential input signal to the analog multiplier 
AM.sub.j, which exhibits common-mode rejection for the V.sub.BIAS term. 
During training, the .phi..sub.P signal applied to the mode control line 
MCL is a ONE, as previously noted, and this causes the input line 
multiplexers ILM.sub.i and ILM.sub.(i+M) to disconnect the input lines 
IL.sub.i and IL.sub.(i+M) from the input driver amplifier ID.sub.i output 
terminals and connect them instead to the non-inverting and inverting 
input terminals of a differential charge-sensing amplifier BDQSi. The 
voltage .delta..sub.j induces a differential change in charge between 
input lines IL.sub.j and IL.sub.(i+M) proportional to .delta..sub.j 
(C.sub.i,j -C.sub.(i+M),j), which differential change in charge is sensed 
using the differential charge sensing amplifier BDQSi. 
Differential charge-sensing amplifier BDQSi includes a fully differential 
amplifier provided with integrating capacitors IC.sub.i and IC.sub.(i+M) 
in respective degenerative feedback connections from each of its output 
terminals to each of its input terminals. Resetting of differential 
charge-sensing amplifier BDQSi is similar to the resetting of a 
single-ended amplifier such as QS.sub.j, except for involving two 
integrating capacitors IC.sub.i and IC.sub.(i+M), rather than just the one 
integrating capacitor C.sub.ij. Resetting of differential charge-sensing 
amplifier BDQSi is done responsive to a pulse .phi..sub.U, which occurs 
during the time when mode control line MCL has a ONE thereon conditioning 
input line multiplexers ILM.sub.i and IM.sub.(i+M) to connect input lines 
IL.sub.i and IL.sub.(i+M) to the differential charge-sensing amplifier 
BDQSi. Resetting is normally done shortly after a ZERO to ONE transition 
appears in the .phi..sub.P signal applied to mode control line MCL and may 
also be done at other times. This procedure corrects for capacitive 
unbalances on the input lines IL.sub.i and IL.sub.(i+M) during 
back-propagation computations that follow the resetting procedure. In 
these computations voltages +.DELTA..sub.i +V.sub.BIAS and -.DELTA..sub.i 
V.sub.BIAS are developed at the (+) and (-) output terminals of the fully 
differential amplifier included in differential charge-sensing amplifier 
BDQSi. The voltage +.DELTA..sub.i +V.sub.BIAS is used by the preceding 
neural net layer during the back-propagation training procedure, if such a 
preceding neural net layer exists. The use of single-ended +.DELTA..sub.i 
and +.DELTA..sub.j drive is shown in FIG. 4A, presuming the neural net 
layers to be integrated within separate monolithic integrated circuits, 
and presuming the limitation on number of pin-outs is restrictive. Where a 
plurality of neural net layers are integrated within the same monolithic 
integrated circuitry, or where maximum pin-out count is not a restrictive 
design factor, balanced .DELTA. signals may be applied from one neural net 
layer to the preceding one. So, too, if the non-linear voltage amplifier 
NL.sub.j is of a correct type (for example, a long-tailed pair connection 
of transistors) y.sub.j +V.sub.BIAS and -y.sub.j +V.sub.BIAS balanced 
output signals may be supplied to the next neural net layer. Indeed, the 
y'.sub.j signals applied to the analog multiplier AM.sub.j may be 
generated in balanced form, replacing differentiator DF.sub.j and 
sample-and-hold circuit SHj with balanced circuitry. 
FIG. 5 shows how in either FIG. 2 or 4 each output line OL.sub.j for j=1, . 
. . N may be pulsed during calculation of y'.sub.j terms. Each output line 
OL.sub.j is connected by a respective capacitor CO.sub.j to the output 
terminal of a pulse generator PG, which generates the doublet pulse. FIG. 
5 shows the doublet pulse applied to the end of each output line QL.sub.j 
remote from the--terminal of the associated differential amplifier 
DA.sub.j in the charge-sensing amplifier QS.sub.j sensing the charge on 
that line. It is also possible to apply the doublet pulses more directly 
to those--terminals by connecting to these terminals respective ones of 
the plates of capacitors CO.sub.j that are remote from the plates 
connecting to pulse generator PG. 
Each output line OL.sub.j has a respective capacitor CO.sub.j connected 
between it and a point of reference potential, and each output line 
OL.sub.(j+N) has a respective capacitor CO.sub.(j+N) connected between it 
and the point of reference potential, which capacitors are not shown in 
the drawing. The respective capacitances of the capacitors CO.sub.j and 
CO.sub.(j+N) are all of the same value, so that the back-propagation 
algorithm is not affected by the presence of these capacitors. 
Arrangements for adding the doublet pulse to v.sub.j before its 
application to the non-linear amplifier NL.sub.j can be used, rather than 
using the FIG. 5 arrangement. 
FIG. 6 comprising component FIGS. 6A and 6B shows further modification that 
can be made to the FIG. 4 modification for the FIG. 3 neural net. This 
modification, as shown in FIG. 6A provides for a pair of input lines 
IL.sub.i and IL.sub.(i+M) for driving each quad configuration of digital 
capacitors DC.sub.i,j, DC.sub.i,(j+N), DC.sub.(i+M),j and 
DC.sub.(i+M),(j+N) push-pull rather than single-ended. Push-pull, rather 
than single-ended drive is provided to the differential charge sensing 
amplifier DQSj, doubling its output response voltage. Push-pull drive also 
permits differential charge sensing amplifier DQSj to be realized with 
differential-input amplifiers that do not provide for common-mode 
suppression of their output signals, if one so desires. 
FIG. 6B differs from FIG. 4B in that the single-ended charge-sensing 
amplifier. QS.sub.j does not appear, being inappropriate for sensing 
differences in charge appearing on a pair of input lines. Instead, 
.DELTA..sub.j +B.sub.BIAS is developed in the following neural net layer 
and is fed back to analog multiplier AM.sub.j via the output multiplexer 
OM.sub.j when the .phi..sub.P signal on mode control line MCL is a 
FIG. 4A shows circuitry that may be used in each neural net layer to 
provide balanced input signal drive to a pair of input lines IL.sub.i and 
IL.sub.(i+M) during normal operation and to differentially sense the 
charge on those input lines during back-propagation calculations. Both 
function may be implemented with separate apparatus. 
During normal operation the .phi..sub.P Signal appearing on mode control 
line MCL is a ZERO, conditioning an input multiplexer IM.sub.i (designated 
output multiplexer OM.sub.j of the previous layer) to apply x.sub.i signal 
to the non-inverting (+) input terminal of differential amplifier ID.sub.i 
and conditioning input line multiplexers ILM.sub.i and ILM.sub.(i+M) to 
connect the non-inverting (+) and inverting (-) output terminals of 
differential amplifier ID.sub.i to input lines IL.sub.i and IL.sub.(i+M) 
respectively. A signal .phi..sub.P is a ONE during normal operation and 
appears in the .phi..sub.U +.phi..sub.P control signal applied to a 
transmission gate between the non-inverting (+) output terminal of 
differential amplifier ID.sub.i and its inverting (-) input terminal, 
rendering that transmission gate conductive to provide direct-coupled 
feedback between those terminals. This d-c feedback conditions 
differential amplifier ID.sub.i to provide x.sub.i and -x.sub.i responses 
at its (+) and (-) output terminals to the x.sub.i signal applied to its 
(-) input terminal. Other transmission gates within the duplex circuitry 
DPX.sub.i are conditioned to be non-conductive during normal operation. 
During back-propagation calculations, the .phi..sub.P signal appearing on 
mode control line MCL is a ONE, conditioning multiplexer OM.sub.j (shown 
in FIG. 6B) to apply .DELTA..sub.i signal from the non-inverting (+) 
output terminal of differential amplifier ID.sub.i to the preceding neural 
net layer, if any, and conditioning input line multiplexers ILM.sub.i and 
ILM.sub.(i+M) to connect the input lines IL.sub.i and IL.sub.(i+M) to 
respective ones of the non-inverting (+) and inverting (-) input terminals 
of differential amplifier BDQS.sub.j. Integrating capacitors IC.sub.i and 
IC.sub.(i+M) connect from the (+) and (-) output terminals of differential 
amplifier BDQS.sub.j to its (-) and (+) input terminals when transmission 
gates in duplex circuitry DPX.sub.i that are controlled by .phi..sub.U 
.multidot..phi..sub.P signal receive a ZERO during back-propagation 
calculations. The charge conditions on integrating capacitors IC.sub.i and 
IC.sub.(i+M) are reset when .phi..sub.U occasionally pulses to ONE during 
back-propagation calculations. This happens in response to transmission 
gates in duplexer circuitry DPX.sub.i receptive of .phi..sub.U and 
.phi..sub.U +.phi..sub.P control signals being rendered conductive 
responsive to .phi..sub.U being momentarily a ONE, while transmission 
gates in duplexer circuitry DPX.sub.i receptive of .phi..sub.U control 
signal being rendered non-conductive. 
A column sign detector (not shown) receives output signal from differential 
amplifier ID.sub.i directly as its input signal and can simply be a 
voltage comparator and sample and hold for the x.sub.i and -x.sub.i output 
signals from the differential amplifier ID.sub.i. This signal is applied 
to line CSL.sub.i shown in FIG. 6A. 
FIG. 7 shows apparatuses for completing the back-propagation computations, 
as may be used with the FIG. 1 neural net manifoldly modified per FIG. 2, 
with the FIG. 3 neural net manifoldly modified per FIG. 4, or with the 
FIG. 1 neural net manifoldly modified per FIG. 6. The weights at each word 
storage element WSE.sub.i,j in the interstitial memory array IMA are to be 
adjusted as the column addresses and j row addresses are scanned row by 
row, one column at a time. An address scanning generator ASG generates 
this scan of i and j addresses shown applied to interstitial memory array 
IMA, assuming it to be a random access memory. The row address j is 
applied to a row multiplexer RM that selects .delta..sub.j to one input of 
a multiplier MULT, and the column address i is applied to a column 
multiplexer CM that selects x.sub.i to another input of the multiplier 
MULT. 
Multiplier MULT is of a type providing a digital output responsive to the 
product of its analog input signals. Multiplier MULT may be a multiplying 
analog-to-digital converter, or it may comprise an analog multiplier 
followed by an analog-to-digital converter, or it may comprise an 
analog-to-digital converter for each of its input signals and a digital 
multiplier for multiplying together the converted signals. Multiplier MULT 
generates the product x.sub.i .delta..sub.j as reduced by a scaling factor 
.eta., which is the increment or decrement to the weight stored in the 
currently addressed word storage element WSE.sub.ij in the memory array 
IMA. The former value of weight stored in word storage element WSE.sub.ij 
is read from memory array IMA to a temporary storage element, or latch, 
TS. This former weight value is supplied as minuend to a digital 
subtractor SUB, which receives as subtrahend .eta. x.sub.i .delta..sub.j 
from multiplier MULT. The resulting difference is the updated weight value 
which is written into word storage element WSE.sub.i,j in memory array IMA 
to replace the former weight value. 
FIG. 8 shows how trained neural net layers L.sub.0, L.sub.1 and L.sub.2 are 
connected together in a system that can be trained. L.sub.0 is the output 
neural net layer that generates y.sub.j output signals; is similar to that 
described in connection with FIGS. 1 and 2, in connection with FIGS. 3 and 
4, or in connection with FIG. 6; and is provided with a back-propagation 
processor BPP.sub.0 with elements similar to those shown in FIG. 2, 4 or 6 
for updating the weights stored in the interstitial memory array of 
L.sub.0. L.sub.1 is the first hidden neural net layer which generates 
y.sub.i output signals supplied to the output neural net layer as its 
x.sub.i input signals. These y.sub.i output signals are generated by layer 
L.sub.1 as its non-linear response to the weighted sum of its x.sub.h 
input signals. This first hidden neural net layer L.sub.1 is provided with 
a back-propagation processor BPP.sub.1 similar to BPP.sub.0. L.sub.2 is 
the second hidden neural net layer, which generates y.sub.h output signals 
supplied to the first hidden neural net layer as its x.sub.h input 
signals. These y.sub.h output signals are generated by layer L.sub.2 as 
its non-linear response to a weighted summation of its x.sub.g input 
signals. This second hidden layer is provided with a back-propagation 
processor similar to BPP.sub.0 and to BPP.sub.1. 
FIG. 8 presumes that the respective interstitial memory array IMA of each 
neural net layer L.sub.0, L.sub.1, L.sub.2 has a combined read/write bus 
instead of separate read input and write output busses as shown in FIG. 2, 
4 or 6. FIG. 8 shows the .delta..sub.j, .DELTA..sub.i and .delta..sub.h 
signals being fed back over paths separate from the feed forward paths for 
y.sub.j, y.sub.i and y.sub.h signals, which separate paths are shown to 
simplify conceptualization of the neural net by the reader. In actuality, 
as shown in FIG. 2, 4 or 6, a single path may be used to transmit y.sub.j 
in the forward direction and Aj in the reverse direction, etc. 
Back-propagation processor BPPo modifies the weights read from word 
storage elements in neural net layer L.sub.0 interstitial memory array by 
.eta. x.sub.i .delta..sub.j amounts and writes them back to the word 
storage elements in a sequence of read-modify-write cycles during the 
training procedure. Back-propagation processor BPP1 modifies the weights 
read from word storage elements in neural net layer L.sub.1 interstitial 
memory array by .eta. x.sub.h .delta..sub.i amounts and writes them back 
to the word storage elements in a sequence of read-modify-write cycles, 
during the training procedure. Back-propagation processor BPP.sub.2 
modifies the weights read and storage elements in neural net layer L.sub.2 
interstitial memory array by .eta. x.sub.g .delta..sub.h amounts and 
writes them back to the word storage element in a sequence of 
read-modify-write cycles during the training procedure. 
The neural nets thusfar described make extensive use of pairs of capacitors 
wherein the capacitances of each pair of capacitances are determined in 
response to a digital word, sum to a prescribed constant value, and differ 
so as to determine the weighting to be applied to a synapse input signal. 
The foregoing specification also describes the usefulness in a neural net 
of two pairs of capacitors, wherein the capacitances of each pair of 
capacitances are determined in response to the same digital word. While 
the capacitors C.sub.i,i and C.sub.i,(j+N) for all i and j could in some 
instances be fixed-value capacitors, so there would never be any 
alteration in the weighting of input voltages x.sub.i where i=1, . . . M, 
such neural nets lack the capacity to adapt to changing criteria for 
neural responses. Such adaptation is necessary, for example, in a neural 
network that is to be connected for self-learning. 
So it is desirable to provide for altering the capacitances of each pair of 
capacitors C.sub.i,j and C.sub.(i+M),j associated in FIG. 1 or 2 neural 
net with a respective pair of values of i and j. This alteration is to be 
carried out in a complementary way, so the sum of the capacitances of 
C.sub.i,j and of C.sub.(i+M),j remains equal to C.sub.k. In FIG. 2 neural 
net it is also desirable to provide for altering the capacitances of each 
pair of capacitors C.sub.i,(j+N) and C.sub.(i+M),(j+N) in a complementary 
way, so the sum of their capacitances remains equal to C.sub.k. 
Similarly, it is desirable to provide for altering the capacitances of each 
pair of capacitors C.sub.i,j and C.sub.i,(j+N) associated in FIG. 3 or 4 
neural net with a respective pair of values of i and j. This alteration is 
to be carried out in a complementary way, so the sum of the capacitances 
of C.sub.i,j and of C.sub.i,(j+N) remains equal to C.sub.k. In FIG. 4 
neural net it is also desirable to provide for altering the capacitances 
of each pair of capacitors C.sub.(i+M),j and C.sub.(i+M),(j+N) in a 
complementary way, so the sum of their capacitances of remains equal to 
C.sub.k. 
Altering the capacitances of each pair of capacitors can be implemented 
along the lines of W. E. Engeler's teachings in regard to "digital" 
capacitors having capacitances controlled in proportion to binary-numbers 
used as control signals, as particularly disclosed in connection with FIG. 
11 of his U.S. Pat. No. 3,890,635 issued 17 June 1975, entitled "VARIABLE 
CAITANCE SEMICONDUCTOR DEVICES" and assigned to General Electric 
Company. Each pair of capacitors is then two similar ones of these 
capacitors and their capacitances are controlled by respective control 
signals, one of which is the one's complement of the other. Another way of 
realizing the pair of capacitors is to control the inverted surface 
potentials of a pair of similar-size metal-oxide-semiconductor (MOS) 
capacitors with respective analog signals developed by digital-to-analog 
conversion. 
Such methods of constructing a pair of capacitors use separate capacitive 
element structures for each capacitor, portions of which capacitive 
element structures are unused when weighting values are chosen to be low. 
This undesirably tends to make the capacitive element structures take up 
nearly twice as much area on an integrated circuit die than is necessary, 
it is here pointed out. 
In the method of constructing a pair of capacitors in accordance with the 
invention, each of a set of component capacitors with capacitances related 
in accordance with powers of two is selected to be a component of one or 
the other of the pair of capacitors, the selecting being done by field 
effect transistors (FETs) operated as transmission gates. This method, 
which requires a minimum capacitor size providing capacitance half as 
large as the capacitance associated with the minimum weight, easily 
provides for a 2.sup.4 :1 range of capacitive weights without requiring 
much concern about unbalanced stray capacitances on the balanced input 
lines or balanced output lines affecting the accuracy of the scaling of 
the differential capacitance between those lines. With present day design 
rules a minimum-area capacitor of three square microns is feasible, which 
makes a capacitor eight times as large have an area of twenty-four square 
microns. There is no unused portion of the capacitive element structures 
in this method. 
FIG. 9 is a conceptual schematic diagram of a pair of capacitors with 
digitally programmed capacitances, designed in accordance with the 
invention to be operated as a half bridge. The two capacitors are provided 
by selective connection of component capacitive elements C0, C1, C2, C3 
and C4, which have a shared first plate labelled COMMON LINE and have 
respective second plates POSITIVE LINE and NEGATIVE LINE. As related to 
FIG. 1, to FIG. A or to FIG. 6A, COMMON LINE corresponds to the 
single-ended input line IL.sub.i ; and POSITIVE LINE and NEGATIVE LINE 
correspond to the balanced output lines OL.sub.j and OL.sub.(j+N), 
respectively. As related to FIG. 3 or to FIG. 4A, COMMON LINE corresponds 
to the single-ended output line OL.sub.j ; and POSITIVE LINE and NEGATIVE 
LINE correspond to the balanced input lines IL.sub.i and IL.sub.(i+M), 
respectively. 
Component capacitive elements C0, C1, C2, C3 and C4 of the FIG. 9 capacitor 
pair have respective capacitances weighted in 2.sup.0 :2.sup.0 :2.sup.1 
:2.sup.2 :2.sup.3 ratio; have respective first plates each connected to 
COMMON LINE; and have respective second plates connected by respective 
ones of single-pole-double-throw electronic switches SW1, SW2, SW3 and SW4 
each to POSITIVE LINE or to NEGATIVE LINE. Single-pole-double-throw 
electronic switches SW1, SW2, SW3 and SW4 each provide for connection to 
POSITIVE LINE or to NEGATIVE LINE, as determined by a respective bit of a 
weighting word, which word is stored in a respective word storage element 
WSE.sub.i,j. The least significant bit of the weighting word is stored in 
a bit store BS1 and successively more significant bits are stored in bit 
stores BS2, BS3 and BS4 FIG. 9 shows bit stores BS1, BS2, BS3 and BS4 as 
respective square boxes within the rectangular box representing word 
storage element WSE.sub.i,j. 
A tabulation of the various connections that can be made responsive to a 
four-bit weighting word, as shown in FIG. 10, suffices to indicate that a 
continuous set of incremental weights extending over a range with both 
positive and negative values is made possible by the method of 
constructing a pair of capacitors in accordance with the invention. The 
most significant bit of the four-bit weighting word governs connection of 
capacitive element C4 to POSITIVE LINE or to NEGATIVE LINE by electronic 
switch SW4 in the reverse sense that the less significant bits of the 
weighting word govern connections of capacitive elements C1, C2 and C3 to 
POSITIVE LINE or to NEGATIVE LINE by electronic switches, or multiplexers 
SW1, SW2 and SW3. A feeling of why this is done (which is to accommodate 
the use of two's complement numbers) and a feeling of why the NEGATIVE 
LINE is provided with the bias capacitive element C0 can be gotten from 
study of the FIG. 10 table. 
The first, second, third and fourth component terms in the NEGATIVE LINE 
and POSITIVE LINE capacitances (which are to the COMMON LINE) are 
determined by which of the NEGATIVE LINE and POSITIVE LINE capacitive 
elements with weights of four, two, one, and one-half are switched to, 
responsive to the most significant bit, the secondmost significant bit, 
the thirdmost significant bit and the least significant bit of the 
four-bit weighting word, respectively, as read from left to right. The 
fifth component term in the NEGATIVE LINE capacitance is the capacitance 
of the bias capacitive element corresponding to the bias capacitive 
element CO of FIG. 9. This fifth component term is constant in its 
application and is not switched responsive to the weighting word or any 
bit thereof. 
FIG. 11 shows another pair of capacitors with digitally programmed 
capacitances, designed in accordance with the invention for half bridge 
operation, which pair is composed of component capacitive elements C20, 
C21, C22, C23 and C24. Capacitive elements C20, C21, C22, C23 and C24 have 
respective capacitances weighted in 2.sup.0 :2.sup.0 :2.sup.1 :2.sup.2 
:2.sup.3 ratio; have respective first plates each connected to COMMON 
LINE; and have respective second plates connected by respective ones of 
single-pole-double-throw electronic switches SW20, SW21, SW22, SW23 and 
SW24 each to POSITIVE LINE or to NEGATIVE LINE. Single-pole-double-throw 
electronic switches SW20, SW21, SW22, SW23 and SW24 each provide for 
connection to POSITIVE LINE or to NEGATIVE LINE, as determined by a 
respective bit of a weighting word, which word is stored in a respective 
word storage element WSE.sub.i,j '. FIG. 11 shows bit stores BS14, BS10, 
BS13, BS12 and BS11 as respective square boxes arranged from left to right 
within the rectangular box representing word storage element WSE.sub.i,j 
'. This arrangement from left to right is in accordance with the bit order 
of a binary diminished radix number system known as a one's complement 
system because the sign of the number can be reversed simply by replacing 
each bit in the number with the bit complement. The one's complement 
system codes arithmetic zero in two ways. The arrangement of bits shown in 
FIG. 11 allows conventional digital adders to be used for signed addition 
in one's complement arithmetic through the expedient of end-around carry, 
wherein the carry out from the single-bit adder generating the leftmost 
bit-place of the plural-bit sum is returned as carry in to the single-bit 
adder generating the rightmost bit-place of the plural-bit sum. It is this 
arrangement of bits that will be presumed to be used for one's complement 
arithmetic in the remainder of this specification. The bits in bit stores 
BS14, BS10, BS13, BS12 and BS11 control electronic switches SW24. SW20, 
SW23, SW22 and SW21, respectively. The leftmost bit of the five-bit 
weighting word governs connection of capacitive element C24 to POSITIVE 
LINE or to NEGATIVE LINE by electronic switch SW24 in the reverse sense 
that the bits of the weighting word to the right govern connections of 
capacitive elements C20, C23, C22 and C21 to POSITIVE LINE or to NEGATIVE 
LINE by electronic switches, or multiplexers SW20, SW23, SW22 and SW21. 
Study of the FIG. 12 table, which tabulates the capacitances between COMMON 
LINE and each of the POSITIVE LINE and NEGATIVE LINE connections for the 
various one's complement numbers used as weighting words in the FIG. 11 
pair of digitally programmable capacitors, helps provide a feeling of how 
this accommodates the use of one's complement numbers. The first, second, 
third, fourth and fifth component terms in the NEGATIVE LINE and POSITIVE 
LINE capacitances (which are to the COMMON LINE) are determined by which 
of the NEGATIVE LINE and POSITIVE LINE capacitive elements with weights of 
four, one-half two, one, and one-half are switched to, responsive to the 
respective bits of the five-bit weighting word as read from left to right. 
The arithmetic of the one's complement number system for the set of 
weighting words used in the FIG. 11 capacitor pair has two zeroes, a 
"positive" zero of 00000 and a "negative" zero of 11111. When the 
interstitial memory array IMA has its contents modified by the increment 
.eta.x.sub.i .delta..sub.j supplied in two's complement form from 
multiplier MULT as shown in FIG. 7, the effects on the neural net training 
of this double zero arithmetic can be suppressed by the subtractor SUB 
being of the type using end-around carry. Alternatively, the effects of 
the double zero arithmetic may be left unsuppressed by subtractor SUB to 
make change between excitory and inhibitory weights less inclined to 
happen. 
The FIG. 11 capacitor pair has the advantage for half bridge operation that 
the range of excitory weights is as wide as the range of inhibitory 
weights, which may overweigh in design considerations the disadvantages of 
the extra bit storage and somewhat more complicated arithmetic used in 
training. Symmetry in the ranges of available excitory and inhibitory 
weights is an especially important design consideration where the range of 
available weights is very restricted--such as just -1, 0 and +1 in a 
modification of the FIG. 11 capacitor pair wherein elements C22, SW22, 
C23, SW23, C24, SW24, BS12, BS13 and BS14 are dispensed with. Also, though 
there is extra bit storage in the FIG. 11 capacitor pair than in the FIG. 
9 capacitor pair, there is greater regularity in the weighting capacitor 
pair structure with all component MOS capacitors being switched. 
AMOS capacitor in a monolithic integrated circuit inherently has a 
substantial stray capacitance to substrate ground from the one of its two 
plates adjacent to the substrate. To prevent such stray capacitance 
appearing in unbalanced form, more on one of the POSITIVE LINE and 
NEGATIVE LINE than on the other, the component capacitive elements C0, C1, 
C2, C3 and C4 of FIG. 9 are poled so as to place their stray capacitances 
to substrate on the COMMON LINE. Similarly, in FIG. 11 the component 
capacitive elements C20, C21, C22, C23 and C24 are poled so as to place 
their stray capacitances to substrate on the COMMON LINE. This balancing 
out of stray capacitance is also done in the capacitor quads of FIGS. 24 
and 25. 
The multiplexers employed in various portions of the circuits described in 
this specification are customarily constructed of single-pole switch 
elements, similar to the SW1, SW2, SW3 and SW4 switch elements of FIG. 9 
and to the SW21, SW22, SW23 and SW24 switch elements of FIG. 11. Each of 
these single-pole switch elements is conventionally a pair of so-called 
"transmission gate" connections of one or more field effect transistors in 
CMOS design. A suitable transmission gate is provided by the paralleled 
channels of a p-channel FET and an n-channel FET having oppositely 
swinging control voltages applied to their respective gate electrodes to 
control the selective conduction of those paralleled channels. 
FIG. 9 shows the SW4 switch element connected to POSITIVE LINE and NEGATIVE 
line oppositely from the SW1, SW2 and SW3 switch elements; and FIG. 11 
shows the SW24 switch element connected to POSITIVE LINE and NEGATIVE line 
oppositely from the SW1, SW2 and SW3 switch elements. In an actual 
integrated circuit layout of the FIG. 9 capacitor pair, rather than this 
being done for the various word storage elements, the bit lines can write 
to the opposite halves of the flip-flops providing bit stores BS4 that the 
bit lines write to the flip-flops providing bit stores BS1, BS2 and BS3. 
Similarly, in an actual integrated circuit layout of the FIG. 11 capacitor 
pair, rather than this being done for the various word storage elements, 
the bit lines can write to the opposite halves of the flip-flops providing 
bit stores BS24 that the bit lines write to the flip-flops providing bit 
stores BS21, BS22 and BS23. 
FIG. 13 shows in detail the electrical connections to a corresponding 
memory element MEX of capacitive element CX that is one of the FIG. 9 
switched capacitive elements C1, C2, C3 and C4. The memory element MEX is 
a flip-flop connection of enhancement-mode field effect transistors (FETs) 
Q1, Q2, Q3 and Q4 and is supplied a relatively positive operating supply 
voltage V.sub.DD and a relatively negative operating supply voltage VSS. 
One plate of capacitive element CX is shown with fixed connection to 
COMMON LINE. The switch element SWX, used to connect the other plate of 
capacitive element CX selectively either to POSITIVE LINE or to NEGATIVE 
LINE, comprises enhancement-mode FETs Q5, Q6, Q7 and Q8. Enhancement-mode 
n-channel FETs Q10 and Q11 are selectively rendered conductive by a WRITE 
command applied to their gate electrodes to impose the bit conditions D 
and DBAR on the complementary output connections Q and QBAR of memory 
element MEX to switch element SWX. 
If D is high (e.g., V.sub.DD) and DBAR is low (e.g., VSS), when the WRITE 
command is no longer applied to the gate electrodes of Q10 and Q12, DBAR 
being low conditions Q1 to be conductive and Q2 to be nonconductive to 
maintain Q high at V.sub.DD potential; and D being high conditions Q3 to 
be nonconductive and Q4 to be conductive to maintain QBAR low at V.sub.SS 
potential. In switch element SWX, Q being high conditions p-channel FET Q5 
for non-conduction and n-channel FET Q6 for conduction, and QBAR being low 
conditions p-channel FET Q7 for conduction and n-channel FET Q8 for 
non-conduction. The NEGATIVE LINE is connected to capacitive element CX by 
FETs Q6 and Q7 being conditioned for conduction; and the POSITIVE LINE is 
disconnected from capacitive element CX by FETs Q5 and Q8 being 
conditioned for nonconduction. 
If D is low (e.g., V.sub.SS) and DBAR is high (e.g., V.sub.DD), when the 
WRITE command is no longer applied to the gate electrodes of Q10 and Q12, 
DBAR being high conditions Q1 to be nonconductive and Q2 to be conductive 
to maintain Q low at Vss potential; and D being low conditions Q3 to be 
conductive and Q4 to be nonconductive to maintain QBAR high at V.sub.DD 
potential. In switch element SWX, Q being low conditions p-channel FET Q5 
for conduction and n-channel FET Q6 for non-conduction, and QBAR being 
high conditions p-channel FET Q7 for non-conduction and n-channel FET Q8 
for conduction. The POSITIVE LINE is connected to capacitive element CX by 
FETs Q5 and Q8 being conditioned for conduction; and the NEGATIVE LINE is 
disconnected from capacitive element CX by FETs Q6 and Q7 being 
conditioned for nonconduction. 
FIGS. 14-23 depict two basic monolithic structure cells, the one at the 
left of these figures and the one at the left of these figures being laid 
out as mirror duplicates of each other. The basic monolithic structure 
cell can be replicated a small number of times to form a pair of weighting 
capacitors composed of switched capacitive elements. A network of such 
pairs of weighting capacitors formed by row and column array of the basic 
monolithic structure cells, with adjacent column structures mirroring each 
other, can be used in constructing weighted summation circuitry in 
accordance with the invention. The smallest switched capacitive element(s) 
can each consist of one basic monolithic structure cell, and the larger 
switched capacitive elements can each be a combination of a plurality of 
the basic monolithic structure cells having electrical connections in 
parallel. The bit storage element need not be duplicated if a single bit 
storage element is used to control both sets of electronic switches in the 
combination, however, and modifications of the monolithic circuit layout 
that eliminate the redundant bit storage element are possible. A quad of 
weighting capacitors composed of switched capacitive elements can simply 
comprise two pairs of weighting capacitors, each pair composed of switched 
capacitive elements as just described, but again there are redundant bit 
storage elements. Layout modifications are possible that can remove these 
redundant bit storage elements, so there is the possibility of layout 
reconfiguration to save area on the monolithic die. Reconfigurations where 
the first and second switched capacitive elements controlled by the same 
bit storage element adjoin each other can give. rise to T-shaped or 
L-shaped basic monolithic structure cells. A row of T-shaped or L-shaped 
basic monolithic structure cells exhibits teeth along one edge of the row. 
Where there are rows of T-shaped or L-shaped basic monolithic structure 
cells, a technique to save area on the monolithic die is to flip alternate 
rows of cells about their row axes so their teeth can be interleaved. 
Monolithic structure cells as shown in FIGS. 14--23 have been used in 
implementing a network of pairs of weighting capacitors where two-bit 
digital words encode weighting factors in one's complement arithmetic. 
Each weighting capacitor in the pair has an effective weight of 0.5. 
In neural net layers of the type using capacitive structures connecting 
input lines and differentially sensed output line pairs, the bits of the 
digital word stored in the two bit storage elements of a basic monolithic 
structure cell both being ZEROs or both being ONEs causes the pair of 
weighting capacitors to be connected to opposite ones of a pair of 
differentially sensed output lines, so an effective weight of zero is 
provided by the capacitor pair. The digital word stored in the two bit 
storage elements being 01 causes both weighting capacitors to be connected 
to the positive output line, so the capacitor pair provides an effective 
weight of +1. The digital word being 10 causes, both weighting capacitors 
to be connected to the negative output line, so the capacitor pair 
provides an effective weight of -1. 
In neural net layers of the type using capacitive structures connecting 
output lines and differentially driven input line pairs, the digital word 
stored in the two bit storage elements being either 00 and 11 causes the 
two weighting capacitors to be connected to opposite ones of a pair of 
differentially driven input lines, so an effective weight of zero is 
provided by the capacitor pair. The digital word stored in the two bit 
storage elements being 01 causes both weighting capacitors to be connected 
to the positive input line, so the capacitor pair provides an effective 
weight of +1. The digital word stored in the two bit storage elements 
being 10 causes both weighting capacitors to be connected to the negative 
input line, so the capacitor pair provides an effective weight of -1. 
FIG. 24 is a conceptual schematic diagram of how a quad connection of four 
capacitors having their capacitances digitally controlled by two's 
complement weighting factor words can be constructed in accordance with 
the invention. Lines LINE1, LINE2, LINE3 and LINE4 of FIG. 24 may be 
considered to correspond to POSITIVE LINE, to NEGATIVE LINE, to COMMON 
LINE and to ac ground, respectively, of FIG. 9. One pair of the four 
capacitors in the quad is provided by selective connection of component 
capacitive elements C0, C1, C2, C3 and C4 to lines LINE1 and LINE2. 
Capacitive elements C0, C1, C2, C3 and C4 have respective capacitances 
weighted in 2.sup.0 :2.sup.0 :2.sup.1 :2.sup.2 :2.sup.3 ratio; have 
respective first plates each connected to LINE3; and have respective 
second plates connected by respective ones of single-pole-double-throw 
electronic switches SW1, SW2, SW3 and SW4 each to LINE1 or to LINE2. 
Single-pole-double-throw electronic switches SW1, SW2, SW3 and SW4 each 
provide for connection to LINE1 or to LINE2, as determined by a respective 
bit of a weighting word, which word is stored in a respective word storage 
element WSE.sub.i,j. The other pair of the four capacitors in the quad is 
provided by selective connection of component capacitive elements C10, 
C11, C12, C13 and C14. Capacitive elements C10, C11, C12, C13 and C14 have 
respective capacitances weighted in 2.sup.0 :2.sup.0 :2.sup.1 :2.sup.2 
:2.sup.3 ratio; have respective first plates each connected to LINE4; and 
have respective second plates connected by respective ones of 
single-pole-double-throw electronic switches SW11, SW12, SW13 and SW14 
each to LINE 1 or to LINE2. Component capacitive elements C10, C11, C12, 
C13 and C14 have respective capacitances which correspond to the 
respective capacitances of component capacitive elements C0, C1, C2, C3 
and C4, respectively. Single-pole-double-throw electronic switches SW11, 
SW12, SW13 and SW14 each provide for connection to LINE1 or to LINE2 in a 
way complementary to the way the electronic switches SW1, SW2, SW3 and SW4 
do, as determined by a respective bit of the weighting word stored in a 
respective word storage element WSE.sub.i,j. 
Lines LINE1, LINE2, LINE3 and LINE4 of FIG. 24 have thusfar been considered 
to correspond to POSITIVE LINE, to NEGATIVE LINE, to COMMON LINE and to ac 
ground, respectively. In such case, to prevent stray capacitance to 
substrate ground appearing in unbalanced form on the POSITIVE LINE and the 
NEGATIVE LINE--i.e., more on one of the lines LINE1 and LINE2 than on the 
other--the component capacitive elements C0, C1, C2, C3 and C4 are poled 
so as to place their stray capacitances to substrate on LINE3, the COMMON 
LINE. The component capacitive elements C10, C11, C12, C13 and C14 are 
poled so as to connect to LINE4, and thence to substrate ground, their 
stray capacitances to substrate, which avoids their having to be charged 
and discharged at all. 
Alternatively, lines LINE1, LINE2, LINE3 and LINE4 of FIG. 24 can be 
considered to correspond to COMMON LINE, to ac ground, to POSITIVE LINE 
and to NEGATIVE LINE, respectively. In such case, to prevent stray 
capacitance to substrate ground appearing in unbalanced form, more on one 
of the lines POSITIVE LINE and NEGATIVE LINE than on the other, the 
component capacitive elements C0, C1, C2, C3 and C4 are poled so as to 
place their stray capacitances to substrate on the POSITIVE LINE, LINE3; 
and the component capacitive elements C10, C11, C12, C13 and C14 are poled 
so as to place their stray capacitances to substrate on the NEGATIVE LINE, 
LINE4. However, this method of avoiding unbalanced stray capacitance to 
substrate appearing on the POSITIVE LINE and NEGATIVE LINE relies on 
matching between the stray capacitances of component capacitive elements 
C0, C1, C2, C3 and C4 and the stray capacitances of component capacitive 
elements C10, C11, C12, C13 and C14, unless the stray capacitances to 
substrate of the POSITIVE LINE and of the NEGATIVE LINE is shunted by 
relatively low source impedance balanced drive from differential amplifier 
ID.sub.i. So, the former method of avoiding unbalanced stray capacitance 
to substrate appearing on the POSITIVE LINE and NEGATIVE LINE is preferred 
over the latter method, at least where these lines are to differentially 
sensed by an output driver amplifier OD.sub.j. 
The FIG. 6 type of neural net uses pairs of input lines driven by balanced 
input signals for connection to the pairs of differentially sensed output 
lines by weighting capacitors connected in quad configurations and 
operated as full bridges. In the FIG. 6 type of neural net, lines LINE3 
and LINE4 of FIG. 24 can be chosen to correspond to one of the pairs of 
input lines driven by balanced input signals, and LINE1 and LINE2 can be 
considered to correspond to one of the the pairs of differentially sensed 
output lines. This choice arranges for the unbalanced stray capacitance to 
substrate appearing on LINE3 and LINE4 to be shunted by relatively low 
source impedance balanced drive from differential amplifier ID.sub.i. 
FIG. 25 is a conceptual schematic diagram of how a quad connection of four 
capacitors having their capacitances digitally controlled by one's 
complement weighting factor words can be constructed in accordance with 
the invention. One pair of the four capacitors is provided by selective 
connection of component capacitive elements C20, C21, C22, C23 and C24. 
Capacitive elements C20, C21, C22, C23 and C24 have respective 
capacitances weighted in 2.sup.0 :2.sup.0 :2.sup.1 :2.sup.2 :2.sup.3 
ratio; have respective first plates each connected to LINE3; and have 
respective second plates connected by respective ones of 
single-pole-double-throw electronic switches SW20, SW21, SW22, SW23 and 
SW24 each to LINE1 or to 2LINE. Single-pole-double-throw electronic 
switches SW24, SW20, SW23, SW22 and SW21 each provide for connection to 
LINE1 or to LINE2, as determined by successive bits of a weighting word 
stored in bit stores BS14, BS10, BS13, BS12 and BS11 of a respective word 
storage element WSE.sub.i,j '. The other pair of the four capacitors is 
provided by selective connection of component capacitive elements C30, 
C31, C32, C33 and C34. Capacitive elements C30, C31, C32, C33 and C34 have 
respective capacitances weighted in 2.sup.0 :2.sup.0 :2.sup.1 :2.sup.2 
:2.sup.3 ratio; have respective first plates each connected to LINE 4; and 
have respective second plates connected by respective ones of 
single-pole-double-throw electronic switches SW30, SW31, SW32, SW33 and 
SW34 each to LINE1 or to LINE2. Component capacitive elements C30, C31, 
C32, C33 and C34 have respective capacitances which correspond to the 
respective capacitances of component capacitive elements C20, C21, C22, 
C23 and C24, respectively. Single-pole-double-throw electronic switches 
SW34, SW30, SW33, SW32 and SW31 each provide for connection to LINE1 or to 
LINE2 in a way complementary to the way the electronic switches SW24, 
SW20, SW23, SW22 and SW21 do,, as determined by successive bits of a 
weighting word stored in bit stores BS14, BS10, BS13, BS12 and BS11 of a 
respective word storage element WSE.sub.i,j '. 
In the FIG. 25 quad connection of capacitors, as in the FIG. 24 quad 
connection of capacitors, lines LINE1, LINE2, LINE3 and LINE4 can be 
considered to correspond to POSITIVE LINE, to NEGATIVE LINE, to COMMON 
LINE and to ac ground, respectively, or alternatively, can be considered 
to correspond to COMMON LINE, to ac ground, to POSITIVE LINE and to 
NEGATIVE LINE, respectively. In the FIG. 6 type of neural net lines LINE1 
and LINE2 of FIG. 25 can be considered to correspond to one of the pairs 
of input lines driven by balanced input signals, and LINE3 and LINE4 can 
be considered to correspond to one of the the pairs of differentially 
sensed output lines. In any case, the component capacitive elements C20, 
C21, C22, C23 and C24 are poled so as to place their stray capacitances to 
substrate on LINE3; and the component capacitive elements C30, C31, C32, 
C33 and C34 are poled so as to place their stray capacitances to substrate 
on LINE4. The reasons for this are the same as offered in regard to the 
FIG. 24 quad connection of capacitors. 
FIG. 26 illustrates how, in order to provide greater resolution in 
weighting, at every i,j crosspoint in the neural net layer a plurality of 
similar weighting capacitor quads are employed rather than just a single 
weighting capacitor quad comprising digital capacitors DC.sub.i,j, 
DC.sub.(i+m),j, DC.sub.i,(j+n) and DC.sub.(i+m),(j+n). In FIG. 26, one 
weighting capacitor quad MSWC.sub.i,j is used to provide weighting 
responsive to the leftmost (generally more significant) bits of the 
eight-bit weighting word stored in the word storage element WSE.sub.i,j of 
the interstitial memory array, and another weighting capacitor quad 
LSWC.sub.i,j is used to provide weighting responsive to the rightmost 
(generally less significant) bits of the weighting word stored in the word 
storage element WSE.sub.ij of the interstitial memory array. The relative 
significances of the weighting provided by the weighting capacitor quad 
MSWC.sub.i,j and by the other weighting capacitor quad LSWC.sub.i,j are in 
a prescribed ratio. 
Much as previously, weighting capacitor quad MSWC.sub.i,j supplies to a 
charge-sensing amplifier FDQS.sub.j weighted response to x.sub.i input 
signal. Another charge-sensing amplifier FDQS.sub.(j+n) besides charge 
sensing amplifier FDQS.sub.j is used for the forward propagation of 
signals at the i,j crosspoint of the neural net layer and is supplied via 
weighting capacitor quad LSWC.sub.i,j with another weighted response to 
x.sub.i input signal. The output port of analog multiplier AM.sub.j 
supplies analog signal to the input port of an analog scaling amplifier 
ASA.sub.j, and the analog scaling amplifier ASA.sub.j responds with an 
analog signal at its output port that is scaled down from the analog 
signal at its input port by a factor equal to the ratio of the relative 
significances of the weighting provided by the weighting capacitor quad 
MSWC.sub.i,j and by the other weighting capacitor quad LSWC.sub.i,j. The 
input port of charge-sensing amplifier FDQS.sub.(j+N) and the output port 
of the analog scaling amplifier ASA.sub.j are multiplexed to the weighting 
capacitor quad LSWC.sub.i,j via output lines OL.sub.(j+ 2N) and 
OL.sub.(j+3N) by multiplexers OLM.sub.(j+2N) and OLM.sub.(j+3N). 
Differential-input charge-sensing amplifiers FDQS.sub.j and FDQS.sub.(j+N) 
are both linear charge-sensing amplifiers; and in a weight and sum circuit 
FW&S.sub.j their output responses are added after being scaled in 
accordance with the ratio of the relative significances of the weighting 
provided by the weighting capacitor quad MSWC.sub.i,j and by the other 
weighting capacitor quad LSWC.sub.i,j. Weight and sum circuit FW&S.sub.j 
thereby generates the input signal for non-linear amplifier NL.sub.i,j. 
For example, suppose the word storage element WSE.sub.i,j stores an 
eight-bit word describing a weight in two's complement arithmetic, four 
bits of which control the weighting afforded by the weighting capacitor 
quad MSWC.sub.i,j and four bits of which control the weighting afforded by 
the weighting capacitor quad LSWC.sub.i,j. Then, the output signal from 
charge-sensing amplifiers FDQS.sub.j and FDQS.sub.(j+N) are weighted in 
16:1 ratio before being summed in the weight and sum circuit FW&Sj to 
generate the input signal for the non-linear amplifier NL.sub.j. 
While FIG. 26 shows portions of a neural net layer in which neither the 
input lines to or the output lines from the weighting capacitance network 
are single-ended, modifications to provide for single-ended operation per 
FIGS. 2A and 2B or per FIGS. 4A and 4B are possible, of course, without 
affecting the way in which different magnitudes of weighting are achieved 
with groups of switched capacitor elements which groups are alike. While 
FIG. 26 shows two ranks of switched capacitor elements, one for responding 
to the more significant bits of the weighting words stored in the 
interstitial memory array and the other for responding to the less 
significant bits of the weighting words stored in the interstitial memory 
array, there may instead be three or more such ranks of switched capacitor 
elements in accordance with the invention. Indeed, bit-slicing the 
weighting factors may be a preferred structure for neural net layers, 
owing to the facts that each portion of a neural net layer associated with 
a single bit slice of the weighting factors is like the other portions of 
the neural net layer respectively associated with the other bit slices of 
the weighting factors; and that that single design can use the 
minimum-size digitally controlled capacitors throughout. 
FIG. 27 shows more particularly the nature of two corresponding pairs of 
capacitors in the weighting capacitor quads MSWC.sub.i,j and LSWC.sub.i,j 
when the word storage element WSE.sub.i,j stores an eight-bit weight in 
two's complement arithmetic. One pair of capacitors in the quad is formed 
from the capacitive elements C0, C1, C2, C3, and C4, which determine the 
differential capacitance to COMMON LINE from lines POSITIVE LS LINE and 
NEGATIVE LS LINE in the weighting capacitor quad LSWC.sub.i,j, and from 
the capacitive elements C5, C6, C7, and C8, which determine the 
differential capacitance to COMMON LINE from lines POSITIVE MS LINE and 
NEGATIVE MS LINE in the weighting capacitor quad MSWC.sub.i,j. The 
capacitive elements C5, C6, C7 and C8 in the weighting capacitor quad 
MSWC.sub.i,j have capacitances respectively similar to those of capacitive 
elements C1, C2, C3, and C4 in the weighting capacitor quad LSWC.sub.i,j. 
The other pair of capacitors in the quad is formed from the capacitive 
elements C10, C11, C12, C13, and C14, which determine the differential 
capacitance to signal ground from lines POSITIVE LS LINE and NEGATIVE LS 
LINE in the weighting capacitor quad LSWC.sub.i,j, and from the capacitive 
elements C15, C16, C17, and C18, which determine the differential 
capacitance to signal ground from lines POSITIVE MS LINE and NEGATIVE MS 
LINE in the weighting capacitor quad MSWC.sub.i,j. The capacitive elements 
C15, C16, C17 and C18 in the weighting capacitor quad MSWC.sub.i,j have 
capacitances respectively similar to those of capacitive elements C11, 
C12, C13, and C14 in the weighting capacitor quad LSWC.sub.i,j. Capacitive 
elements in the capacitor quad that have the same last digit in their 
call-out have capacitances that are similar to each other. 
FIG. 27 shows bit stores BS8, BS7, BS6, BS5, BS4, BS3, BS2 and BS1, which 
respectively store progressively less significant bits of a weighting 
word, as respective square boxes arranged from left to right within the 
rectangular box representing the word storage element WSE.sub.i,j. The 
most significant bit of the eight-bit weighting word governs connection of 
capacitive element C8 to POSITIVE MS LINE or to NEGATIVE MS LINE by 
electronic switch SW8 in the reverse sense that the three next most 
significant bits of the eight-bit weighting word govern connection of 
capacitive elements C7, C6 and C5 to POSITIVE MS LINE or to NEGATIVE MS 
LINE by electronic switches SW7, SW6, and SW5 and in the reverse sense 
that the four least significant bits of the weighting word govern 
connections of capacitive elements C1, C2, C3 and C4 to POSITIVE LS LINE 
or to NEGATIVE LS LINE by electronic switches SW1, SW2, SW3 and SW4. The 
four most significant bits of the eight-bit weighting word govern 
connection of capacitive elements C18, C17, C16 and C15 to POSITIVE MS 
LINE or to NEGATIVE MS LINE by electronic switches SW18, SW17, SW16, and 
SW15 in the reverse sense that it governs connection of capacitive 
elements C8, C7, C6 and C5 to POSITIVE MS LINE or to NEGATIVE MS LINE by 
electronic switches SW8, SW7, SW6 and SW5. The four least significant bits 
of the eight-bit weighting word govern connection of capacitive elements 
C14, C13, C12 and C11 to POSITIVE LS LINE or to NEGATIVE LS LINE by 
electronic switches SW14, SW13, SW12, and SW11 in the reverse sense that 
it governs connection of capacitive elements C4, C3, C2 and C1 to POSITIVE 
LS LINE or to NEGATIVE LS LINE by electronic switches SW4, SW3, SW2 and 
SW1. 
In the FIG. 26 neural net layer using capacitor quads per FIG. 27, then, 
when respective capacitive elements C8 and C18 are included in each quad 
of capacitors in the weighting capacitor quad MSWC.sub.i,j ; these 
respective capacitive elements C8 and C18 are switched in reverse sense 
from all other capacitive elements in that quad of capacitors to implement 
two's complement arithmetic in regard to weighting value. Respective 
non-switched C0 and C10 minimum-weight capacitive elements are included in 
each quad of capacitors in the weighting capacitor quad LSWC.sub.i,j, to 
bias the zero-capacitance condition to correspond to the all-ZERO 
condition of the two's complement numbers that can be stored in the word 
storage element WSE.sub.i,j of the interstitial memory array. However, the 
weighting capacitor quad MSWC.sub.i,j includes no non-switched 
minimum-weight capacitive elements. 
FIG. 28 shows more particularly the nature of a quad of capacitors in the 
weighting capacitance networks MSWC.sub.i,j and LSWC.sub.i,j when the word 
storage element WSE.sub.i,j stores nine-bit weights in one's complement 
form in bit stores BS10, BS11, BS12, BS13, BS14, BS15, BS16, BS17 and 
BS18. One pair of capacitors in the quad is formed from the capacitive 
elements C20, C21, C22, C23, and C24, which determine the differential 
capacitance to COMMON LINE from lines POSITIVE LS LINE and NEGATIVE LS 
LINE in the weighting capacitor quad LSWC.sub.i,j, and from the capacitive 
elements C25, C26, C27, and C28, which determine the differential 
capacitance to COMMON LINE from lines POSITIVE MS LINE and NEGATIVE MS 
LINE in the weighting capacitor quad MSWC.sub.i,j. The capacitive elements 
C25, C26, C27 and C28 in the weighting capacitor quad MSWC.sub.i,i have 
capacitances respectively similar to those of capacitive elements C21, 
C22, C23, and C24 in the weighting capacitor quad LSWC.sub.i,j. The other 
pair of capacitors in the quad is formed from the capacitive elements C30, 
C31, C32, C33, and C34, which determine the differential capacitance to 
signal ground from lines POSITIVE LS LINE and NEGATIVE LS LINE in the 
weighting capacitor quad LSWC.sub.i,j, and from the capacitive elements 
C35, C36, C37, and C38, which determine the differential capacitance to 
signal ground from lines POSITIVE MS LINE and NEGATIVE MS LINE in the 
weighting capacitor quad MSWC.sub.i,j. The capacitive elements C35, C36, 
C37 and C38 in the weighting capacitor quad MSWC.sub.i,j have capacitances 
respectively similar to those of capacitive elements C31, C32, C33, and 
C34 in the weighting capacitor quad LSWC.sub.i,j. Capacitive elements in 
the capacitor quad that have the same last digit in their call-out have 
capacitances that are similar to each other. The weighting capacitor quad 
MSWC.sub.i,j has no capacitive element corresponding to capacitive element 
C20 or C30 in the weighting capacitor quad LSWC.sub.i,j. 
FIG. 28 shows bit stores BS18, BS10, BS17, BS16, BS15, BS14, BS13, BS12 and 
BS11 which respectively store successive bits of a nine-bit weight in 
one's complement arithmetic, as respective square boxes arranged from left 
to right within the rectangular box representing the word storage element 
WSE.sub.i,j '. The leftmost bit of the nine-bit weighting word stored in 
bit store BS18 governs connection of capacitive element C28 to POSITIVE MS 
LINE or to NEGATIVE MS LINE by electronic switch SW28, depending whether 
that bit is a ONE or a ZERO; and that leftmost bit also governs connection 
of capacitive element C38 to POSITIVE MS LINE or to NEGATIVE MS LINE by 
electronic switch SW38, depending whether that bit is a ZERO or a ONE. The 
next to leftmost bit of the nine-bit weighting word stored in bit store 
BS10 governs connection of capacitive element C20 to POSITIVE LS LINE or 
to NEGATIVE LS LINE by electronic switch SW20, depending whether that bit 
is a ONE or a ZERO; and that next to leftmost bit also governs connection 
of capacitive element C30 to POSITIVE LS LINE or to NEGATIVE LS LINE by 
electronic switch SW30, depending whether that bit is a ZERO or a ONE. The 
three next most left bits of the nine-bit weighting word stored in bit 
stores BS17, BS16 and BS15 govern connection of capacitive elements C27, 
C26 and C25 to POSITIVE MS LINE or to NEGATIVE MS LINE by electronic 
switches SW27, SW26 and SW25, respectively, depending whether those bits 
are respectively each a ONE or a ZERO. The bits stored in bit stores BS17, 
BS16 and BS15 also govern connection of capacitive elements C37, C36 and 
C35 to POSITIVE MS LINE or to NEGATIVE MS LINE by electronic switches 
SW37, SW36 and SW35, respectively, depending whether those bits are 
respectively each a ZERO or a ONE. The four rightmost bits of the 
weighting word stored in bit stores BS11, BS12, BS13 and BS14 govern 
connections of capacitive elements C21, C22, C23 and C24 to POSITIVE LS 
LINE or to NEGATIVE LS LINE by electronic switches SW21, SW22, SW23 and 
SW24, respectively, depending whether those bits are respectively each a 
ONE or a ZERO. The four bits stored in bit stores BS11, BS12, BS13 and 
BS14 also govern connections of capacitive elements C31, C32, C33 and C34 
to POSITIVE LS LINE or to NEGATIVE LS LINE by electronic switches SW31, 
SW32, SW33 and SW34, respectively, depending whether those bits are 
respectively each a ZERO or a ONE. 
One skilled in the art and acquainted with the foregoing specification will 
be able to design numerous variants of the preferred embodiments of the 
invention described therein, and this should be borne in mind when 
construing the following claims. For example, the fixed-capacitance 
weighting capacitors in the matrix multipliers disclosed by W. E. Engeler 
in his U.S. Pat. No. 4,156,284 issued 22 May 1979 and entitled "SIGNAL 
PROCESSING APATUS" can be replaced in accordance with the invention 
with adjustable-capacitance weighting capacitors having their capacitances 
adjusted in accordance with digital signals stored in respective word 
storage elements of an interstitial memory array. As a further example, 
the fixed-capacitance switched capacitors described by Y. P. Tsividis and 
D. Anastassion in their letter "Switched-Capacitor Neural Networks" 
appearing in ELECTRONICS LETTERS, 27th Aug. 1987, Vol. 23, No. 18, pages 
958,959 (IEE) can be replaced in accordance with the invention with 
adjustable-capacitance switched capacitors having their capacitances 
adjusted in accordance with digital signals stored in respective word 
storage elements of an interstitial memory array.