Single layer neural network circuit for performing linearly separable and non-linearly separable logical operations

A neural network provides both linearly separable and non-linearly separable logic operations, including the exclusive-or operation, on input signals in a single layer of circuits. The circuit weights the input signals with complex weights by multiplication and addition, and provides weighted signals to a neuron circuit (a neuron body or soma) which provides an output corresponding to the desired logical operation.

The present invention relates to artificial neural network circuits, and 
more particularly to a circuit which uses complex weights to perform 
linearly separable or non-linearly separable logical operations in a 
single layer. 
BACKGROUND OF THE INVENTION 
A single neuron network, which may be configured in a single layer is known 
as a perceptron. In general, a perceptron accepts multiple inputs, 
multiplies each by a weight, sums the weighted inputs, subtracts a 
threshold, and limits the resulting signal, such as by passing it through 
a hard limiting nonlinearity. A perceptron is described in W. McCulloch 
and W. Pitts, "A Logical Calculus of the Ideas Immanent in Nervous 
Activity", Bulletin of Mathematical Biophysics, Vol. 5, pp. 115-133, 1943. 
This perceptron was not itself useful until combined into multiple 
perceptron networks known as artificial neural networks. Such networks 
generally comprised interconnected perceptron layers having multiple 
input, intermediate and output neurons. For an example of such networks, 
see the text by J. Anderson and E. Rosenfeld, Neurocomputing: Foundations 
of Research, the MIT Press, 1988. Artificial neural networks currently are 
used in such applications as pattern recognition and classification. 
While it is known to perform linearly separable functions using single 
layer neural networks, one shortcoming of such networks is that they 
heretofore have been thought to be inherently incapable of performing 
non-linearly separable functions. Specifically, it has been established 
that a two-input, single-layer, single-neuron network cannot implement 
such non-linearly separable functions as the exclusive-or ("XOR") and 
exclusive-nor functions ("XNOR"). See R. Hecht-Nielsen, Neurocomputing, 
17-18, Addison-Wesley, 1989. 
The rationale for the belief that single-layer neural networks incapable of 
performing non-linearly separable functions can be seen by examining the 
characteristics of a conventional neural network. Such a conventional 
neural network having two inputs, x.sub.1 and x.sub.2, can be represented 
by a hyperplane of w.sub.1 x.sub.1 +w.sub.2 x.sub.2 -.theta.=0, where 
w.sub.1 and w.sub.2 are weights applied to inputs x.sub.1 and x.sub.2 
before application to the neuron, and .theta. is a threshold. The decision 
region in the x.sub.1 -x.sub.2 plane, i.e., the region containing input 
coordinates which yield a common output, is only a half-plane bounded by 
the hyperplane. A logical function is said to be linearly separable if it 
conforms to the hyperplane criterion, i.e., all input pairs having 
coordinates on one side of the hyperplane yield one result and all inputs 
with coordinates on the other side yield a different result. Because there 
is no single straight line subdividing the x.sub.1 -x.sub.2 plane, such 
that the inputs (0,0) and (1,1) will fall on one side of the hyperplane 
and thus yield a logical zero, and (0,1) and (1,0) on the other to yield a 
logical one, it was supposed that no combination of values for w.sub.1 and 
w.sub.2 will produce the input/output relationship of the XOR function for 
a single layer neural network. That is to say, the XOR function is a 
non-linearly separable function. 
FIG. 11 shows the geometry of the plane/hyperplane discussed above. It can 
be seen that hyperplane 1 is incapable of separating the x.sub.1 -x.sub.2 
plane into a pair of half planes having (0,1) and (1,0) (which yields an 
XOR result of 1) on one side of the hyperplane, and (0,0) and (1,1) (which 
yields an XOR result of 0) on the other. 
The XOR operation is a common one in neural networks. To implement it, 
however, has required the use of multiple layers of neural networks. To 
reduce the number of components and hence the cost of networks, a need has 
arisen for a single layer neural network which can be trained to perform 
the XOR operation, as well as other non-linearly separable and separable 
operations, on one or more input signals. 
SUMMARY OF THE INVENTION 
It is an object of the present invention to provide a neuron circuit which 
can perform both linearly and non-linearly separable logical operations 
using fewer layers of components than are currently required. 
It is a further object of the present invention to provide a single layer 
neural network circuit which can perform such non-linearly separable 
logical operations as XOR and XNOR. 
A single layer, single artificial neuron circuit for performing linearly 
separable and non-linearly separable logical operations on one or more 
input signals in accordance with the invention includes synapse means for 
receiving the input signals and weighting the input signals with one or 
more complex weights to form a plurality of weighted input signals. The 
complex weights each have a real component and an imaginary component. The 
values of the real and imaginary components determine the logical 
operation to be performed on the one or more input signals. The circuit 
also includes neuron means for receiving the weighted input signals and 
producing a resultant electrical signal reflecting (corresponding to) the 
logical operation on the one or more input signals.

DETAILED DESCRIPTION 
FIG. 1 shows a neural network having a plurality of inputs, with two 
inputs, x.sub.1 and x.sub.2, being specifically shown. The inputs are 
electrical signals, analog or digital, upon which the desired logical 
function will be performed. These two inputs are applied to synapses 2, 
where they are multiplied by weights w.sub.1 and w.sub.2, respectively. 
The weights are complex and are of the form w.sub.n =a.sub.n +jb.sub.n. 
The factors a and b are the values of the real and imaginary components of 
the complex weight, respectively. The weighted signals are then applied to 
a neuron body 4. The neuron circuit, also called a neuron body or soma, 4 
performs an operation on the weighted input signals to produce an 
intermediate output y. The output y is obtained by squaring the magnitude 
of the sum of the weighted input signals. To achieve neuron output z of 
desired value for a given set of inputs, the intermediate output y is 
applied to a thresholding circuit 6. If a hard limiting nonlinear 
threshold is used, the neuron output z will be a logical 1 for all inputs 
y above a predetermined threshold amplitude and a logical 0 for all other 
inputs y (below the threshold). If a sigmoidal non-linear threshold is 
used, the neuron output z will be an analog value between 0 and 1. 
A discussion of the hardware implementation of the neural network of the 
present invention is now presented. FIG. 2 shows a preferred embodiment of 
the single layer neural network circuit for performing the XOR. As can be 
seen, inputs x.sub.1 and x.sub.2 each are applied to a separate input 
line. Each signal is split, and using a known multiplying device such as 
an Add-Shift multiplier (in digital design) or a Gilbert multiplier (in 
analog design), is multiplied with real-valued weights a.sub.1, b.sub.1, 
a.sub.2 and b.sub.2, by means of multipliers 15, 17, 19, and 21 
respectively. Each of the real-valued weights represents the real or 
imaginary component of its associated complex weight. To implement the 
neuron body 4 shown in FIG. 1, summers 5 and 7, multipliers 9 and 11, and 
summer 13 can be used as shown in FIG. 2. The weighted inputs 
corresponding to the real and imaginary portions of the complex weights 
are added separately. That is, the input signals weighted by the real 
portions of the complex weights are added by summer 5 and the input 
signals weighted by the imaginary portions of the complex weights are 
added by summer 7. Summers 5 and 7 preferably are carry-lookahead adders 
(in digital design) or a Kirchhoff adder (in analog design), but any 
suitable signal summing means will suffice. Each of the summed signals is 
then squared by first duplicating each summed signal and then applying 
them to multipliers 9 and 11. These multipliers preferably are either an 
Add-Shift multiplier (in digital design), or a Gilbert multiplier (in 
analog design), but any suitable multiplication device will suffice. 
Finally, the resultant products are added by summer 13, the output of 
which is the signal y. Summer 13 preferably is a carry-lookahead adder (in 
digital design), or a Kirchhoff adder (in analog design), although any 
suitable summing means will suffice. The circuit of FIG. 1 is shown 
generally in FIG. 6, for the case of more than two inputs. The operation 
of the circuit of FIG. 6 is identical to that of the circuit of FIG. 2. 
Complex weights are meant to include two dimensional weights with real and 
imaginary components and an alternative two dimensional weight having two 
real components. 
The non-linear separability problem which heretofore has prevented the 
implementation of such functions as XOR and XNOR by means of a single 
layer neural network is solved in accordance with the invention by using 
complex weights. Although complex weighting is used, the operation 
performed by neuron circuit 4 is implemented using real-valued electronic 
components. To achieve this without loss of generality, the square of the 
magnitude of the sum ("square of sum") of the weighted inputs is used. The 
output y can be obtained by multiplication in the synapses 2 and summing 
the squares of the synapse outputs. These operations can be expressed 
mathematically as follows: 
EQU y=.vertline.x.sub.1 (a.sub.1 +jb.sub.1)+x.sub.2 (a.sub.2 
+jb.sub.2).vertline..sup.2. (1) 
Expanding, 
EQU y=(x.sub.1 a.sub.1 +x.sub.2 a.sub.2).sup.2 +(x.sub.1 b.sub.1 +x.sub.2 
b.sub.2).sup.2. (2) 
As can be seen, the imaginary number drops out of equation (1) and, hence, 
equation (2) can be considered the transformation of (1) into an 
expression which includes only real numbers. 
With properly selected weights derived analytically or through training, 
the resultant signal y from neuron body 4 reflects the desired logical 
operation, e.g., XOR, XNOR, etc., for a given set of inputs. To convert 
the value of y to logical 1 or logical 0, the following hard limiting 
function is employed: 
##EQU1## 
To convert the value to an analog value between 0 and 1, the following 
sigmoid function is employed: 
##EQU2## 
where k is a squashing coefficient, and .crclbar. is a threshold value. 
The squashing coefficient, k, is a value that controls the slope of the 
function; large values of k squash the function. If k becomes infinitely 
high, the function becomes a hard limiting function. The threshold value, 
.theta., of the sigmoid controls the position of the center of the 
function. The squashing coefficient and threshold can be implemented as 
shown in FIGS. 13(b) and (c) by the circuit shown in FIG. 13(a). The 
sigmoid is a thresholding function which produces an analog value between 
0 and 1 in response to values of y. The characteristic curves for the hard 
limiting function and the sigmoid function are shown in FIG. 3(a) and 
3(b), respectively. 
To examine the decision boundary of a complex weight network, it is assumed 
that the output function is a hard-limiting function with a threshold of 
.theta.. Therefore, the decision boundary can be obtained by the equation 
EQU y-.theta.=0 
This equation can be written into matrix form using equation (2) 
EQU x.sup.t Cx+Kx.phi.+=0 (3) 
where, 
##EQU3## 
If a.sub.1.sup.2 +b.sub.1.sup.2, . . . , a.sub.n.sup.2 +b.sub.n.sup.2 are 
not all zero, (3) is a quadratic equation in n-space. Therefore, it is 
clear that the decision boundaries obtained from the complex weight neural 
networks are hyperquadrics. The details of the hyperquadrics are further 
analyzed by means of a discriminant, .delta.. For example, the 
discriminant of a two-dimensional space is .delta.=(a.sub.1 b.sub.2 
-a.sub.2 b.sub.1).sup.2. Hence, the decision boundary of a two-input 
complex weight neural network can be either elliptic (if .delta.&gt;0) or 
parabolic (if .delta.=0). Similarly, the discriminant in three-dimensional 
space is 
##EQU4## 
Note that for .delta.=0, at least one of the roots of the equation is 
zero. Therefore, the decision boundary of the three-input network is 
either elliptic paraboloid or hyperbolic paraboloid based on the complex 
weights. 
The choice of weights w.sub.1 and w.sub.2 in the implementation of the XOR 
as shown in FIG. 1, can be made analytically or by training (discussed 
later). To determine the values analytically, it must first be noted that 
inputs x.sub.1 and x.sub.2 will be either logical 1 or logical 0. Thus, 
whether or not the proper XOR result, z, is returned after thresholding 
depends upon the assignment of proper weights w.sub.1 and w.sub.2. The 
truth table for the XOR function is shown in Table 1. (It is assumed that 
y=z, the threshold result, for purposes of this analysis.) 
TABLE 1 
______________________________________ 
x.sub.1 x.sub.n 
y = z 
______________________________________ 
0 0 0 
0 1 1 
1 0 1 
1 1 0 
______________________________________ 
Solving equation (2) for the values of y shown in Table 1, it can readily 
be seen that y=0 when x.sub.1 =x.sub.2 =0. Solving for x.sub.1 =1, x.sub.2 
=0 and y=1, then 
EQU a.sub.1.sup.2 +b.sub.1.sup.2 =1. 
If x.sub.1 =0, x.sub.2 =1 and y=1, then 
EQU a.sub.2.sup.2 +b.sub.2.sup.2 =1. 
If x.sub.1 =x.sub.2 =1 and y=0, then 
EQU (a.sub.1 +a.sub.2).sup.2 +(b.sub.1 +b.sub.2).sup.2 =0. 
Assuming a.sub.1, a.sub.2, b.sub.1 and b.sub.2 are real numbers, then 
EQU a.sub.1 =-a.sub.2 
EQU b.sub.1 =-b.sub.2. 
One possible solution is 
EQU w.sub.1 =a.sub.1 +jb.sub.1 =.sqroot.2/2+.sub.j .sqroot.2/2, 
EQU w.sub.2 =a.sub.2 +jb.sub.2 =-.sqroot.2/2-.sub.j .sqroot.2/2. 
For the two-input network which implements the XOR function, shown in FIG. 
2, the output y resulting from the application of these complex weights 
(i.e., substituting w.sub.1 and w.sub.2 into equation (2)) is 
EQU y=(x.sub.1 -x.sub.2).sup.2. 
The preceding discussion shows that the proper complex weights can be 
determined analytically. The proper weights can also be obtained by 
iteratively training the neural network to deliver a desired result for a 
given set of inputs. Training involves the adjustment of weights based 
upon resultant outputs of prior training iterations. The training of the 
complex weight neural network circuit of the present invention can be 
carried out in a manner similar to conventional gradient descent and error 
back-propagation methods. Based on the proposed neural network, the output 
of the neuron 4 is represented as follows. 
##EQU5## 
The error function, E.sub.p, is defined to be proportional to the square 
of the difference between the target output, t.sub.p, and the actual 
output, z.sub.p, for each of the patterns, p, to be learned. 
##EQU6## 
To see the change of output with respect to the change of weight, partial 
derivatives of neuron output y.sub.p with respect to real weight a.sub.m 
and imaginary weight b.sub.m, where m=1, . . . , n, can be represented by 
##EQU7## 
respectively. Then, the partial derivatives of the sigmoid output, z, with 
respect to real and imaginary weights are represented by 
##EQU8## 
respectively. The changes of the error with respect to the change of the 
real and imaginary weights are 
##EQU9## 
where, .xi..sub.p=t.sub.p -z.sub.p. Then, the changes of the real and 
imaginary weights for each training epoch can be represented by 
##EQU10## 
and 
##EQU11## 
where g is a gain factor. 
Since the partial derivative of the sigmoid output, z, with respect to each 
weight is a function of the input and the weight, as shown in equations 
(4) and (5), the multiplication of the error and the partial derivatives, 
.xi..sub.p (.differential.z.sub.p /.differential.a.sub.m) and .xi.p.sub. 
(.differential.z.sub.p /.differential.b.sub.m), will maintain proper 
training. In other words, a positive .xi..sub.p means that the actual 
output is smaller than the target value, and vice versa. A positive 
.differential.z.sub.p /.differential.a.sub.m means that the output change 
will be positive proportional to the change of weight, and vice versa. For 
example, if .xi..sub.p &lt;0 and .differential.z.sub.p /.differential.a.sub.m 
&gt;0, then the result of the multiplication, .xi..sub.p 
(.differential.z.sub.p /.differential.a.sub.m) is negative, and eventually 
a.sub.m will be reduced. Training of a multi-layer network can be 
performed in the same manner as the conventional back-propagation method, 
which is described in D. Rumelhart, G. E. Hinton and R. J. Williams, 
"Learning Internal Representations by Error Propagation," in D. Rumelhart 
and J. McClelland, Parallel Distributed Processing: Explanations in the 
Microstructure of Cognition, Vol. 1: Foundations, Chap. 8., MIT Press, 
1986, incorporated herein by reference. 
If the weights w.sub.1 and w.sub.2 have been properly selected, signal y 
will be proportional to either a 1 or 0, depending on the values of inputs 
x.sub.1 and x.sub.2. Because the values of y corresponding to either 
logical 1 or 0 may not be equal for various input combinations, a 
thresholding means 6, which implements a threshold function is used to 
produce uniform output signals z. That is, thresholding means 6, which 
preferably is a magnitude comparator, produces one of two different output 
signals depending upon whether its input falls above or below a threshold 
level. Various implementations of the threshold functions can be found in 
C. Mead, Analog VLSI and Neural Systems, Addison Wesley, 1989, and 
"Digital Design Fundamentals" by K. J. Breeding, 2nd ed., Prentice Hall, 
1992, pp. 308-322." 
In addition to XOR, the non-linearly separable function XNOR can be 
implemented using a neural network in accordance with the present 
invention by using a threshold function to invert the output of the XOR. 
A single layer neural network using complex weights can also be used to 
implement linearly separable functions. This capability is useful so that 
production costs for a neural network can be minimized, since all 
functions can be performed with identical hardware. Distinct functions can 
be performed by altering the applied weights. Examples 1-4 illustrate the 
implementation of various linearly separable functions through the use of 
single-layer neural networks with complex weights. Examples 5-7 are 
additional examples of non-linearly separable functions. 
EXAMPLE 1 
NAND 
The single layer neural network using complex weights can be used to 
perform the NAND operation, as well. The NAND operation requires that an 
input offset, w.sub.o =a.sub.o +jb.sub.o, be applied to neuron body 4, as 
shown in FIG. 8. Thus, 
##EQU12## 
Transforming, 
EQU y=(a.sub.1 x.sub.1 +a.sub.2 x.sub.2 +a.sub.o).sup.2 +(b.sub.1 x.sub.1 
+b.sub.2 x.sub.2 +b.sub.o).sup.2. (6) 
in the NAND function, the output y=1, when x.sub.1 =x.sub.2 =0. Solving, 
EQU a.sub.o.sup.2 +b.sub.o.sup.2 =1. 
This is a unit circle in the complex plane, therefore, the input bias can 
be assumed to be 
w.sub.o =1. 
Solving equation (6) with W.sub.o =1, yields the following possible 
solutions: 
EQU a.sub.1 =-1/2, b.sub.1 =-.sqroot.3/2, a.sub.2 =-1/2, and b.sub.2 
=.sqroot.3/2. 
If we assume W.sub.o =-1, solving equation (6) yields the following 
possible result: 
EQU a.sub.1 =1/2, b.sub.1 =.sqroot.3/2, a.sub.2 =1/2, and b.sub.2 
=-.sqroot.3/2. 
EXAMPLE 2 
OR 
Solving equation (2) for the inputs and results of the OR operation yields 
the following possible weights for achieving the OR operation: 
EQU a.sub.1 =1/2b.sub.1 =.sqroot.3/2, a.sub.2 =1/2 and b.sub.2 =-.sqroot.3/2. 
EXAMPLE 3 
AND 
The AND function, shown in Table 2, can be implemented using a single layer 
neural network with complex weights. 
TABLE 2 
______________________________________ 
x.sub.1 x.sub.2 
y 
______________________________________ 
0 0 0 
0 1 0 
1 0 0 
1 1 1 
______________________________________ 
To achieve this operation, an output offset or threshold, .theta., can be 
applied to the output signal y in addition to an input offset W.sub.o. 
Output offset, .theta., added to both sides of equation (2), yields: 
EQU y+.theta.=(a.sub.1 x.sub.1 +a.sub.2 x.sub.2 +a.sub.o).sup.2 +(b.sub.1 
x.sub.1 +b.sub.2 x.sub.2 +b.sub.o).sup.2 +.theta. 
Solving for the inputs x.sub.1 and x.sub.2 shown in Table 2, yields 
For x.sub.1 =x.sub.2 =0, 
EQU a.sub.o.sup.2 +b.sub.o.sup.2 +.theta.=0; 
for x.sub.1 =0 and x.sub.2 =1, 
EQU (a.sub.2 32 a.sub.o).sup.2 =(b.sub.2 +b.sub.o).sup.2 +.theta.=0. 
Substituting, 
EQU a.sub.2.sup.2 +2a.sub.2 a.sub.o +b.sub.2.sup.2 +2b.sub.2 b.sub.o =0. 
For x.sub.1 =1 and x.sub.2 =0, 
EQU (a.sub.1 +a.sub.o).sup.2 +(b.sub.1 +b.sub.o).sup.2 +.theta.=0. 
Substituting, 
EQU a.sub.1.sup.2 +2a.sub.1 a.sub.o +b.sub.1.sup.2 +2b.sub.1 b.sub.o =0. 
For x.sub.1 =x.sub.2 =1, 
EQU (a.sub.1 +a.sub.2 +a.sub.o).sup.2 +(b.sub.1 +b.sub.2 +b.sub.o).sup.2 
+.theta.=1. 
Substituting, 
EQU a.sub.1 a.sub.2 +b.sub.1 b.sub.2 =1/2 
One of the solutions may be 
EQU a.sub.o =-1/2, b.sub.o =0, a.sub.1 =b.sub.1 a.sub.2 =b.sub.2 =1/2, and 
.theta.=-1/4. 
FIG. 9 shows a block diagram of the implementation an AND operation. 
EXAMPLE 4 
INVERTER 
The performance of the inverter operation can be accomplished by 
application of an input offset signal. This input offset will result in an 
output of logical 1 when the input signal, x, is 0. This can be shown with 
reference to the following equation: 
EQU y=.vertline.w.sub.1 x.sub.1 +W.sub.o .vertline..sup.2 =.vertline.a.sub.1 
x.sub.1 +jb.sub.1 x.sub.1 +a.sub.o +jb.sub.o .vertline..sup.2 
=.vertline.(a.sub.1 x.sub.1 +a.sub.o)+j(b.sub.1 x.sub.1 
+b.sub.o).vertline..sup.2. 
Taking the sum of the squares, yields 
EQU y=(a.sub.1 x.sub.1 +a.sub.o).sup.2 +(b.sub.1 x.sub.1 +b.sub.o).sup.2. 
The expected output, y, with input x=0, is 1. Substituting x=0 into the 
above equation, yields 
EQU a.sub.o.sup.2 +b.sub.o.sup.2 =1. 
This is a unit circle in the complex plane. Therefore, the input offset can 
be selected arbitrarily to be 
EQU W.sub.o =.+-.1 
The expected output, y, for an input x=1, is 0. Therefore, substituting in 
the above equation x=1 yields: 
EQU (a.sub.1 +a.sub.o).sup.2 +(b.sub.1 +b.sub.o).sup.2 =0, 
which yields 
EQU a.sub.1 =-1, b.sub.1 =0, if W.sub.o =1, or 
EQU a.sub.1 =1, b.sub.1 =0, if W.sub.o =-1. 
To summarize, 
EQU w.sub.1 =1, if W.sub.o =1; and 
EQU w.sub.1 =1, if W.sub.o =-1. 
An implementation of the inverter is shown in FIG. 10. So far, 
implementation examples of linearly separable logical functions using the 
complex weights have been described. The solutions have been obtained 
analytically by solving the proper equations. 
The next three examples show implementations of various non-linearly 
separable functions using complex weights. The solutions are obtained by 
training. 
EXAMPLE 5 
SYMMETRY 
A four-input symmetry function, shown in Table 3, and a six-input symmetry 
function (not shown), can be performed by properly adjusting the complex 
weights and adding additional input circuitry. 
TABLE 3 
______________________________________ 
x.sub.1 x.sub.2 
x.sub.3 x.sub.4 
y 
______________________________________ 
0 0 0 0 0 
0 0 0 1 1 
0 0 1 0 1 
0 0 1 1 1 
0 1 0 0 1 
0 1 0 1 1 
0 1 1 0 0 
0 1 1 1 1 
1 0 0 0 1 
1 0 0 1 0 
1 0 1 0 1 
1 0 1 1 1 
1 1 0 0 1 
1 1 0 1 1 
1 1 1 0 1 
1 1 1 1 0 
______________________________________ 
Through training, it has been determined that the following complex weights 
will yield the four-input symmetry function shown in Table 3: 
EQU w.sub.1 =0.8785+j0.6065 
EQU w.sub.2 =0.0852+j1.0676 
EQU w.sub.3 =-0.0914-j1.1862 
EQU w.sub.4 =-1.0343-j0.6850. 
A six input symmetry problem can also be obtained by training, as shown in 
FIG. 4. FIG. 4 shows the trajectory of each of the weights w.sub.1 
-w.sub.6 during training. Thus, an acceptable set of weights for 
performing the six-input symmetry function has been determined through 
training to be: 
EQU w.sub.1 =0.5546-j0.3157 
EQU w.sub.2 =-0.6434+j1.0052 
EQU w.sub.3 =0.6840+j0.7378 
EQU w.sub.4 =-0.5958-j0.5930 
EQU w.sub.5 =0.5687-j1.1013 
EQU w.sub.6 =-0.6208+j0.3433 
The four and six input symmetry operations can be implemented using a 
single layer neural network as shown in FIG. 6, with the number of inputs 
and synapses being either four or six. 
EXAMPLE 6 
ADD 
A single layer neural network using complex weights can also be used to 
perform a portion of a two 2-byte input and single 3-byte output summation 
problem. By using a single layer neural network with complex weights, the 
least significant bit of the sum can be obtained. Table 4 shows the inputs 
and desired outputs for this operation. 
TABLE 4 
______________________________________ 
x.sub.1 x.sub.2 
x.sub.3 x.sub.4 
y 
______________________________________ 
0 0 0 0 0 
0 0 0 1 1 
0 0 1 0 0 
0 0 1 1 1 
0 1 0 0 1 
0 1 0 1 0 
0 1 1 0 1 
0 1 1 1 0 
1 0 0 0 0 
1 0 0 1 1 
1 0 1 0 0 
1 0 1 1 1 
1 1 0 0 1 
1 1 0 1 0 
1 1 1 0 1 
1 1 1 1 0 
______________________________________ 
FIG. 5(a) and b) shows alternative block diagrams for performing this 
operation. FIG. 5(a) shows a four-input, single-layer neural network. FIG. 
6 shows a generalized block diagram capable of implementing the four-input 
ADD operation. Through training it has been determined that a complex 
weighting combination which will yield the desired result for the ADD is: 
EQU W.sub.1 =0.9724-j0.3085 
EQU W.sub.2 =0.0261-j0.0594 
EQU W.sub.2 =0.9576+j0.3874 
EQU W.sub.4 =0.0499+j0.2806. 
It should be noted that the four-input neural network for obtaining the 
least significant digit of the sum can be simplified to a two-input neural 
network, as shown in FIG. 5(b). This is so because the weights w.sub.2 and 
w.sub.4, determined through training, for inputs x.sub.2 and x.sub.4 of 
the neural network are negligible. Thus, the use of complex weights allows 
simplification of the four-input neural network to a two-input neural 
network in this case. The threshold values of the sigmoids in FIG. 5(a) 
and 5(b) are each 0.5. 
To obtain the second byte of the addition result, a two-layer network is 
required, as shown in FIG. 7. 
The circuit of the present invention is capable of implementing a 
quadrically separable function using a single layer. Because the function 
described in this example is not quadrically separable, two layers are 
required. Training of such a circuit can be done quickly using the 
training technique described above. 
EXAMPLE 7 
NEGATION 
A 4-byte input and 3-byte output negation function can also be performed by 
a single layer network, as shown in FIG. 12. 
The truth table for this operation is shown as Table 5. 
TABLE 5 
______________________________________ 
x.sub.4, x.sub.3, 
x.sub.2, x.sub.3 
z.sub.3, 
z.sub.2, 
z.sub.1 
______________________________________ 
0 0 0 0 0 0 0 
0 0 0 1 0 0 1 
0 0 1 0 0 1 0 
0 0 1 1 0 1 1 
0 1 0 0 1 0 0 
0 1 0 1 1 0 1 
0 1 1 0 1 1 0 
0 1 1 1 1 1 1 
1 0 0 0 1 1 1 
1 0 0 1 1 1 0 
1 0 1 0 1 0 1 
1 0 1 1 1 0 0 
1 1 0 0 0 1 1 
1 1 0 1 0 1 0 
1 1 1 0 0 0 1 
1 1 1 1 0 0 0 
______________________________________ 
As can be seen from the block diagram of FIG. 12, the negation operation 
can be performed by XOR'ing the input x.sub.4 with each of the inputs 
x.sub.1, x.sub.2 and x.sub.3. The three bit output, z.sub.1, z.sub.2, 
z.sub.3, is the result of these XOR operations. Through training, it has 
been determined that the complex weights w.sub.1 =w.sub.2 =w.sub.3 =-j and 
w.sub.4 =j can be used to perform the negation operation. 
From the foregoing description, it will be apparent that there has been 
provided an improved single layer neural network. While the above method 
and apparatus have been described with reference to the functions 
described in the examples, other variations and modifications in the 
herein described system, within the scope of the invention, will 
undoubtedly suggest themselves to those skilled in the art. Accordingly, 
the foregoing description should be taken as illustrative and not in a 
limiting sense.