Neural network

Abstract: (1) A data processing system comprising: a plurality of operational amplifiers; a plurality of input leading wires supplying input signals to each operational amplifier, respectively; a plurality of switching circuits connected with each output of the operational amplifier, respectively, when value of output signals for each operational amplifier exceeds the predetermined value; an output leading wire connected with the switching circuits for supplying output signals of each the switching circuit to input side of the operational amplifiers though variable resistance as well as outputting them to the out­ side.

Description: The present invention relates to a neural network. PRIOR ARTS Since Macalloch Pitts model, a fundamental model of neural network has an output of normalized digital value. A digital circuit for the model can easily be designated at the theoretical point of view. However, a huge size of circuit will be necessary for performing the following operation by digital circuit. The operation includes comparison, addition and multiplication of a lot of terms, as well as subtraction.  : Normalization function w i : Weight of the ith synapse A i : Input to the ith synapse  : Threshold of neuron Accordingly, it is difficult to expand the number of neurons up to the practical level in an IC and it is difficult to make such an IC due to the limitation in number of gates. Therefore, trial for the contraction of the neural network by analog circuit has been proposed by the U.S. patent No.4,660,166, No.4,719,591 and No.4,731,747. The neural network disclosed in the above publications has the structure to control each input to an operation amplifier at the condition of total unification by variable resistance, and evaluates the following energy formula: The above proposed neural network is effective for the calculation of exceedingly small value or exceedingly great value with respect to the variable given by the equivalent function to the above energy formula, so as to be used as, for example, a key to solution for the problems of traveling salesman. However, the neural network of this type lacks in normalization with respect to each neurons; i.e., the function outputting digital value according to the comparison result with threshold, so that the functions; information compression, arrangement and integration with respect to the neural network of bionic system cannot be realized. Therefore, it is impossible to obtain the essential effect of the neural network by the neural network of this type that an appropriate output pattern is generated in accordance with the input pattern. SUMMARY OF THE INVENTION The present invention is invented so as to solve the above problems of the prior art and has an object to provide a data processing system realizable of being an integrated circuit as well as comprising the function for the normalization. This object is solved by the neural network as claimed in claim 1. When value of output signal from each operational amplifier exceeds predetermined value, signal is output from switching circuit connected to the above operational amplifier, then such signal is input to each operational amplifier through variable resistance. Output signal from each switching circuit is to be input to operational amplifier as well as to be output to the outside of a data processing system. BRIEF DESCRIPTION OF THE DRAWINGS Fig.1 shows a circuit diagram of an embodiment of a neural network according to the present invention; Fig.2 shows a circuit diagram indicating the status of variable resistance. 10, 20, 30.....Operation Amplifier 11, 21, 31.....Switching Circuit 12, 22, 32.....Input Leading Wire 17, 27, 37.....Output Leading Wire 41 to 46.......Variable Resistance PREFERRED EMBODIMENT OF THE PRESENT INVENTION Hereinafter, an embodiment of the neural network according to the present invention is described with referring to the attached drawings. Although only three operational amplifiers are shown in this diagram for simplification, much more operational amplifiers can be provided, practically. According to the present embodiment, processing is performed onto three input data I1, I2 and I3, then three data O1, O2 and O3 are output. The combination of each operational amplifier 10, 20, 30 and switching circuit 11, 21, 31 corresponds to a neuron of the neural network of an organism, respectively. The connection structure among each operational amplifier corresponds to a so-called neural network. The connection structure of such operational amplifiers is basically equivalent to the circuit of hop-field disclosed by the specification of the U.S. patent 4,66,166 except switching circuits 11, 21 and 31. Hereinafter, peripheral structure of operational amplifier 10 is described. An input leading wire 12 which supplies input signal to operational amplifier 10 is connected to input terminal of the above operational amplifier 10. Resistance 13 and condenser 14 are connected to the input leading wire 12 in which time constant of operational amplifier 10 is set up. On the other hand, the base of transistor composing switching circuit 11 is connected to output terminal of the operational amplifier 10. Here, diode 15 for the prevention of a back current to the operational amplifier is provided in between the above base and the operational amplifier 10. Collector of transistor composing switching circuit 11 is connected to a power supply V, and emitter is connected to output leading wire 17 through resistance 16. Therefore, as much volume of electric current as it is determined by resistance 16 flows between collector and emitter of transistor when value of output signal of operational amplifier 11 exceeds the predetermined value determined by the characteristics of transistor of switching circuit 11; that is, when the base voltage exceeds the predetermined value. An electric current signal output from the above switching circuit 11 is input to operational amplifiers 20 and 30 though output leading wire 17. Output leading wire 17 is connected to input terminals of operational amplifiers 20 and 30 though variable resistances 41 and 42, respectively. This output leading wire is also lead to the outside of this neural network so as to take out output signal O1 of switching circuit 11 from the above output leading wire 17. Switching circuit 11 side of resistance 16 is branched at the middle of output leading wire 17 and connected through protection resistance 18 comprising bigger resistance value than that of resistance 16. Peripheral structures of the operation amplifiers 20 and 30 are the same as that of operational amplifier 10. Output leading wire 27 is connected to switching circuit 21 which is connected to output terminal of operational amplifier 20. Output leading wire 37 is connected to switching circuit 31 which is connected to output terminal of operational amplifier 30. Output leading wire 27 is connected to operational amplifiers 10 and 30 through variable resistances 43 and 44, respectively. Output leading wire 37 is connected to operational amplifiers 10 and 20 through variable resistances 45 and 46, respectively. It is possible to realize variable resistances 41 to 46 by the structure of, for example, connecting resistances r in parallel through diode d and transistor t, as shown in Fig.2. Resistance values of variable resistances 41 to 46 correspond to weight Wi of the above operation formula (1). An electric current signal supplied to operational amplifiers 10, 20 and 30 through each output leading wire 17, 27, 37 corresponds to input Ai of the formula (1). On the other hand, the characteristics of switching circuits 11, 21 and 31; that is, voltage of basic voltage required for outputting the signal corresponds to threshold of the formula (1). Therefore, normalization of neurons, i.e., the function to output signals in accordance with comparison result with threshold is completed. Accordingly, when input signals I1, I2 and I3 are given, each operational amplifier 10, 20, 30 outputs voltage signal according to the above input signal. Switching circuit outputs electric current signal when above voltage exceeds the predetermined base voltage. An electric signal output from switching circuit is input to operational amplifier, which is not connected to the base of the above switching circuit, through variable resistance. According to the above, output signal is controlled since input signal with respect to the operational amplifier is charged. Switching circuit connected to output terminal of the operational amplifier outputs the signal when the value of the above output signal exceed the predetermined value. Output signals O1, O2 and O3 from each switching circuit are output to the outside from this neural network. Control system prepared at the outside of this data processing system controls weights of each neuron, i.e., variable resistances from 41 to 46, according to output signals O1, O2 and O3. As a result, values of signals input to operational amplifiers 10, 20 and 30 are changed and output signals O1, O2 and O3 are changed as action status of switching circuit 11, 21 and 31 are changed. Accordingly, learning control is performed so as to control output signal to be the predetermined value. Therefore, a circuit comprising the similar ability to information processing function of the neural network of an organism is realized by a neural network according to the present embodiment. Although switching circuits 11, 21 and 31 consist of transistors, it is not restricted to transistors. It is possible to apply any circuit which comprises the equivalent function as that of transistors. As mentioned above, it is possible to realize the neural network comprising the normalization function with analog circuit so as to obtain a data processing system which prepares as many neurons as required at the practical level. Therefore, it is possible to make the neural network into an integrated circuit.

Claims: A neural network comprising: a plurality of interconnected neuron elements, each said element including: means (41-46, 10, 20, 30) for algebraically summing weighted output signals of at least two other neuron elements and the input signal to the respective neuron element and for amplifying the resultant sum signal, characterised in that each element further includes a threshold switching element (11, 21, 31) for receiving said amplified sum signal and for outputting an output signal if the amplified sum signal exceeds a predetermined threshold value. Neural network according to claim 1 wherein weighting of said output signals is effected by variable resistors (41-46), each of which coupling the output signal of a different neuron element to the input of the amplifying means (10,20,30). Neural network according to claim 2 wherein each said variable resistor comprises a plurality of resistor elements (r) in parallel with one another, and a plurality of switching means (t) in series between respective ones of said parallel resistor elements, so that different ones of said switching means can be actuated to connect different resistor elements to provide different equivalent resistance values. Neuron network according to one of the proceeding claims, wherein said threshold switching element is a transistor. Neural network according to claim 4 wherein a blocking diode (15) is coupled between the output of the amplifying means (10,20,30) and a base terminal of said transistor.

Inventors: KUMAGAI RYOHEI, MATSUMOTO KOJI, TAKATORI SUNAO, YAMAMOTO MAKOTO, KUMAGAI, RYOHEI, MATSUMOTO, KOJI, TAKATORI, SUNAO, YAMAMOTO, MAKOTO
Applicants: SHARP KK, YOZAN INC, SHARP KABUSHIKI KAISHA, YOZAN INC.
Application year: 1990
Main classification: G06F15
IPC codes: G06G7, G06N3, G06F15, G06N99