Source: http://www.google.com/patents/US5671336?ie=ISO-8859-1&dq=5,072,412
Timestamp: 2015-05-04 21:19:12
Document Index: 228238526

Matched Legal Cases: ['art 10', 'art 300', 'art 310', 'art 310', 'art 320', 'art 320', 'art 310', 'arts 310', 'art 320', 'art 310', 'art 300', 'art 310', 'art 320', 'art 200', 'art 300', 'art 320', 'art 310', 'art 210', 'art 300', 'art 320', 'art 310', 'art 210', 'art 300', 'art 310', 'art 320', 'art 300', 'arts 320', 'art 300', 'arts 320']

Patent US5671336 - Apparatus based on n-variable unlimited recurrent adjustable network - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsAn apparatus based on an n-variable unlimited recurrent adjustable network (URANn) comprises: two layers, each layer having the same number (n) of neuron elements; linear neuron elements xi constituting a first layer; nonlinear artificial neuron elements yj having respective temperature-dependent parameters...http://www.google.com/patents/US5671336?utm_source=gb-gplus-sharePatent US5671336 - Apparatus based on n-variable unlimited recurrent adjustable networkAdvanced Patent SearchPublication numberUS5671336 APublication typeGrantApplication numberUS 08/305,947Publication dateSep 23, 1997Filing dateSep 15, 1994Priority dateSep 16, 1993Fee statusLapsedPublication number08305947, 305947, US 5671336 A, US 5671336A, US-A-5671336, US5671336 A, US5671336AInventorsKazumi Yoshida, Norimasa KishiOriginal AssigneeNissan Motor Co., Ltd.Export CitationBiBTeX, EndNote, RefManPatent Citations (5), Non-Patent Citations (16), Referenced by (13), Classifications (10), Legal Events (4) External Links: USPTO, USPTO Assignment, EspacenetApparatus based on n-variable unlimited recurrent adjustable network
US 5671336 AAbstract
An apparatus based on an n-variable unlimited recurrent adjustable network (URANn) comprises: two layers, each layer having the same number (n) of neuron elements; linear neuron elements xi constituting a first layer; nonlinear artificial neuron elements yj having respective temperature-dependent parameters Tj and constituting a second layer. Each linear and nonlinear neuron element of the first and second layers is connected using a feedforward connection part, a recurrent connection part, and an auto connection part. A nonlinear oscillation apparatus having the recurrent neural network is generally operated in accordance with the equation (1) described below: ##EQU1##
1. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of neuron elements, and said recurrent neural network comprising:a) a first layer including a plurality of circuits of linear artificial neurons Xi (i=1, 2, - - - , i, - - - n); b) a second layer including a plurality of circuits of nonlinear artificial neurons yj (j=1, 2, - - - , j, - - - , n) and arranged to convert input signals thereto into output signals by means of given same nonlinear functions f, their temperature parameters being denoted by Tj ; c) a feedforward connection part arranged to multiply outputs of the first layer circuits of linear artificial neurons xi by feedforward connection coefficients wji and to output each of the multiplied outputs to a corresponding one of all of the circuits of nonlinear artificial neurons yj of said second layer; d) a recurrent connecting part arranged to multiply outputs of the respective circuits of nonlinear artificial neurons yj of said second layer by recurrent connection coefficients γj, to output first-order delays of time constants τj for the multiplied outputs of the respective circuits of nonlinear artificial neurons yj, and to input the delayed multiplied output of each respective circuit of onlinear artificial neurons yj to a corresponding one of the circuits of linear artificial neurons xj of said first layer; and e) an auto connection part arranged to multiply the outputs of the respective circuits of linear artificial neurons xi of the first layer, to output the first-order delays of the time constants τi for the multiplied outputs from said auto connection part, and to input again each of the first-order delayed outputs to the corresponding one of said circuits of linear artificial neurons xi, and wherein said recurrent neural network operates in accordance with the following equation (1) and said nonlinear oscillation apparatus generates a nonlinear oscillation when said recurrent neural network operates in accordance with the following equation (1), ##EQU18## wherein f denotes the given nonlinear function having saturation characteristics, and g denotes a characteristic function to discuss a stability of a differential equation and a nullcline when g=0. 2. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 1, wherein the nonlinear output function f having the saturation characteristics is expressed as f(x)≡tanh(x).
3. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 1, wherein a normal vector Ai is defined from either equation (3) or equation (7), said normal vector representing a direction of a tangent space of the nullcline at a coordinate origin of a state space in a spatial figure in a letter Z shape derived from the equation (1) as gi (x1, x2, . . . , xn)=0, wherein a distance Ci from the coordinate origin to an asymptote of said nullcline gi (x1, x2, . . . , xn)=0 is defined, and wherein said feedforward connection coefficients wji, said recurrent connection coefficients λi, said auto connection coefficients μi of the circuits of linear artificial neurons xi, and said temperature parameters Tj of the circuits of nonlinear artificial neurons yj of the second layer are determined according to a formulae group shown by either of equations (4) through (6) or equations (8) through (10), these equations (5) through (6) and (9) through (10) being derived from a limitation condition established between a virtual time constant εi corresponding to the time constant in a scaled equation shown in an equation (2) derived from the equation (1) and the time constant τi of the first-order time delay, thus said nonlinear oscillation apparatus having said recurrent neural network generating the nonlinear oscillation having a desired physical property, said equation (2), (3), (4), (5), (6), (7), (8), (9) and (10) being expressed as follows: ##EQU19##
4. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 3, wherein an n-th order nonlinear ordinary differential equation represents a dynamical system of said recurrent neural network, a stability of the equation (1) is determined according to a complex eigen value or a Jacobian matrix representing a linear approximation in the vicinity to a fixed point so as to derive an oscillation condition, the derived oscillation condition being corresponded to the geometrical shape of said nullcline so that said nonlinear oscillation apparatus having the recurrent neural network generates a desired nonlinear oscillation.
5. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 4, wherein said nonlinear oscillation is under a self-sustained oscillation condition and is provided with non-chaotic solution having either a periodic solution or quasiperiodic solution, a target time constant τ to achieve a target oscillation number ω of the desired nonlinear oscillation is adjusted by the recurrent neural network on the basis of the following equation (13): ##EQU20## wherein tm -to denotes a time difference between a time to at which the oscillation waveform in a graph in a xi -t space after crosses a time axis t after the nonlinear self-sustained oscillation starts and a sufficient time has passed to fall in an equilibrium and a time tm at which the oscillation waveform again crosses the time axis t after the m period, and wherein τo denotes an arbitrary time constant to a trial nonlinear self-sustained oscillation for the recurrent neural network by means of a numerical simulation.
6. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 4, wherein said nonlinear oscillation apparatus having recurrent neural network generates a nonlinear oscillation waveform corresponding to a geometrical shape of the nullclines and to a solution trajectory of the n-th order nonlinear ordinary differential equation within the n-dimensional state space extended by output of the corresponding circuits of the circuits of linear artificial neurons xi of the first layer and having a frequency corresponding to said virtual time constant εi so that said nonlinear oscillation circuit having said recurrent neural network generates the nonlinear oscillation from a limit cycle to a chaos.
7. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 6, wherein input signals are inputted to either the circuits of the linear artificial neurons xi of the first layer or the circuits of the nonlinear artificial neurons yj of the second layer, the outputs of either the circuits of the linear artificial neurons xi of the first layer or the circuits of the nonlinear artificial neurons yj of the second layer serving as output signals so that a nonlinear filtering of the input signals is carried out in said nonlinear oscillation apparatus having the recurrent neural network.
8. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 1, wherein the recurrent neural network system having each layer including n circuits of the artificial neuron elements is operated as a chaotic temporal signal generating apparatus as of said nonlinear oscillation apparatus, its output signal of the chaotic time series generating apparatus being provided with a 1/f (wherein f is frequency) fluctuation characteristic.
9. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 8, wherein a control input signal s(t) is inputted to either each of the circuits of the linear artificial neurons xi of the first layer or each of the circuits of the nonlinear artificial neurons yj of the second layer, the outputs of either the circuits of the linear artificial neurons xi of the first layer or the circuits of the nonlinear artificial neurons yj of the second layer serving as a control output signal r(t), said control output signal being outputted for a controlled object, the nonlinear oscillation apparatus thus including the recurrent neural network compensating the nonlinear characteristics that said controlled object inherently has.
10. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 1, wherein a controlled results S(t) of the controlled object is inputted to either each of the circuits of the linear artificial neurons xi of the first layer or each of the circuits of the nonlinear artificial neurons of the second layer, the respective outputs of the circuits of the linear artificial neurons xi of the first layer or of the circuits of the nonlinear artificial neurons yj of the second layer are fedback to the controlled object so that the recurrent neural network of said nonlinear oscillation apparatus carries out the nonlinear feedback control compensating the nonlinear characteristics that the controlled object inherently has.
11. A nonlinear oscillation apparatus having a recurrent neural network having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of elements as claimed in claim 1, wherein the same input signals p(t) are inputted to a plant having an unknown nonlinear characteristics and to said recurrent neural network so that internal parameters of the recurrent neural network are modified to eliminate an output error between the plant having the unknown nonlinear characteristics and the recurrent neural network, thus identifying the unknown nonlinear characteristics of the plant.
12. An apparatus based on an n-variable unlimited recurrent adjustable network, comprising:a) at least two hierarchically structured layers, each layer having the same number (n) of neuron circuit elements; b) a plurality of linear artificial neuron circuit elements xi (i1, 2, - - - , i, - - - , n) constituting a first layer of the two layers; c) a plurality of nonlinear artificial neuron circuit elements yj (j=1, 2, - - - , j, - - - , n) having respective temperature-dependent parameters Tj constituting a second layer of the two layers, each nonlinear artificial neuron circuit element yj converting an input signal thereto into an output signal using a predetermined nonlinear function f; d) a feedforward connection part arranged to multiply each output signal derived from each one of the linear artificial neuron circuit elements (xi) of the first layer by a feedforward connection coefficient wji and to output each of the multiplied output signals (w11, w12, - - - w1j, - - - , w1n, w21, w22, - - - , w2j, - - - , w2n, - - - , wn1, wn2, - - - , wnj, - - - , wnn) for all of the nonlinear artificial neuron circuit elements yj of the second layer; e) a recurrent connection part arranged to multiply each output signal derived from the nonlinear artificial neuron circuit elements yj of the second layer by a recurrent connection coefficient λj, to output a first-order time delay corresponding to a time constant τj for the λj multiplied output signal, and, thereafter, to provide the τj delayed and λj multiplied output signal again to the corresponding one xj of the linear artificial neuron circuit elements of the first layer; and f) an auto connection part arranged to multiply the output signal derived from each one xi of the respective linear artificial neuron circuit elements of the first layer by an auto connection coefficient μi, to output the first-order delay corresponding to the time constant τi for the μi multiplied signal, and to output the μi multiplied, μi delayed signal again for the corresponding one xi of the respective linear artificial neuron circuit elements. 13. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 12, wherein the apparatus is operated according to either of the following equations to generate a nonlinear oscillation, said apparatus, thus, being a nonlinear oscillation apparatus:in a continuous time system, ##EQU21## wherein f denotes the predetermined nonlinear function having satuaration characteristics and g denotes a characteristic function to discuss a stability of each corresponding differential equation and, when g=0, a nullcline is defined, and in a discrete time system, ##EQU22## 14. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 13, wherein said predetermined nonlinear function f is expressed as f(x)=tanh(x).
15. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 13, wherien said predetermined nonlinar function f is expressed as f(x)=1/ 1+exp(-x)!.
16. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 14, wherein said apparatus further comprises: g) defining means for defining a normal vector Ai (ai1, ai2, . . . , ain) representing a direction of a tangent space of a nullcline at a coordinate origin of a state space in a spatial figure of a letter Z shape derived when gi (x1 x2, - - - , xn)=0 in the equation (1) and for defining a distance Ci from the coordinate origin up to an asymptote of the nullcline gi (x1, x2, - - - , xn), said defining means defining the normal vector by either an equation (3) or an equation (7); and h) determining means for determining the respective internal parameters of the feedforward connection coefficient, recurrent connection coefficient, and auto connection coefficient according to formulae groups of either equations (4) through (6) or equations (8) through (10) derived under a limitation condition established between a virtual time constant εi corresponding to the time constant in a scaled equation shown in the following equation (2) derived from the equation (1) and the first-order time delay constant τi in the equation (1) so that the apparatus generates the nonlinear oscillation having a desired physical property, ##EQU23##
17. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 16, which further comprises: i) deriving means for determining a stability of the equation (1) according to an eigen value of a Jacobian matrix representing a linear approximation in the vicinity of a fixed point to derive an oscillation condition of the apparatus; and j) corresponding means for corresponding the derived oscillation condition to a geometrical shape of the nullcline gi (x1, x2, - - - , xn)=0, so that the apparatus to generate the nonlinear oscillation generates a target nonlinear oscillation.
18. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 17, wherein the stability of the equation (1) means an n-th order nonlinear ordinary differential equation representing a dynamical system of the recurrent neural network URANn.
19. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 18, wherein said apparatus comprises the nonlinear oscillation apparatus, said nonlinear oscillation apparatus being under a self-sustained oscillation condition and having a non-chaotic solution of a periodic solution and/or a quasi-periodic solution and said apparatus further comprises: k) adjusting means for adjusting a target time constant τ to achieve a target frequency ω of the nonlinear oscillation to be generated on the basis of the following equation (13): ##EQU24## wherein τo denotes an arbitrary time constant for the apparatus to generate a trial nonlinear self-sustained oscillation through a numerical simulation, tm -t0 denotes a time difference between a certain time t0 at which an oscillation waveform in a graph in a state space of xi -t crosses a time axis t after a sufficient time has passed when the nonlinear self-sustained oscillation has reached to a steady state upon a start of the nonlinear self-sustained oscillation and a time tm at which the oscillation waveform crosses again the time axis t after m periods, and τi of the equation (1) or corresponds to the virtual time constant εi of the scaled equation (2).
20. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 19, wherein said apparatus generates the nonlinear oscillation from a limit cycle to chaos having the waveform corresponding to a solution trajectory of the n-th order nonlinear ordinary differential equation within an n-variable state space extended by the output of the linear artificial neuron circuit elements xi of the first layer and having a frequency corresponding to the virtual time constant εi.
21. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 20, wherein an input signal Si (t) of the apparatus is received by either each of the linear artificial neuron circuit elements xi of the first layer or each of the nonlinear artificial neuron circuit elements yi of the second layer and an output signal ri (t) of the apparatus is derived from either each of the linear artificial neuron circuit elements xi of the first layer or each of the nonlinear artificial neuron circuit elements yi, said apparatus carrying out a nonlinear filtering for the input signal thereof.
22. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 20, wherein said apparatus functions as a chaotic temporal signal generating apparatus, an output signal of said chaotic temporal signal generating apparatus being a time series signal having a 1/f (where f is frequency) fluctuation characteristic.
23. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 20, which further comprises a controlled object and wherein a control input signal S(t) of the apparatus is received by either each of the linear artificial neuron circuit elements xi of the first layer or each of the nonlinear artificial circuit neuron elements yj of the second layer and a control output signal r(t) of the apparatus is derived from either each of the linear artificial neuron circuit elements xi of the first layer or each of the nonlinear artificial neuron circuit elements yj of the second layer, said apparatus outputting the control output signal r(t) for the controlled object so that said apparatus carries out a nonlinear feedforward control to compensate for nonlinear characteristics provided in said controlled object.
24. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 20, which further comprises a controlled object and wherein a controlled result z(t) of the controlled object is received by either each of the linear artificial neuron circuit elements xi of the first layer or each of the nonlinear artificial neuron circuit elements yj of the second layer and an output signal r(t) of the apparatus which is derived from either each of the linear artificial neuron circuit elements xi of the first layer or each of the nonlinear artificial neuron circuit elements yj of the second layer is fedback to the controlled object, said apparatus carrying out a nonlinear feedback control to compensate for nonlinear characteristics provided in said controlled object.
25. An apparatus based on an n-variable unlimited recurrent adjustable network as claimed in claim 20, which further comprises a plant having unknown nonlinear characteristics and wherein the same input signal s(t)=p(t) is supplied both to the plant and to said apparatus so that the internal parameters of said apparatus are modified to eliminate an error e(t) between output signals from said plant and apparatus, said apparatus functioning to identify the plant having the unknown nonlinear characteristics.
26. An apparatus based on an n-variable unlimited recurrent adjustable network, wherein n is an arbitrary number, and said apparatus comprising:a) at least two hierarchically structured layers, each layer having the same number (n) of neuron circuit elements; b) a plurality of linear artificial neuron circuit elements Xi (i=1 - - - n) constituting a first layer of the two layers and comprising first adders (100); c) a plurality of nonlinear artificial neuron circuit elements yj (j=1 - - - n), having respective temperature-dependent parameters Tj, constituting a second layer of the two layers, comprising second adders and nonlinear amplifiers (200), each nonlinear artificial neuron circuit element yj converting an input signal thereto into an output signal using a predetermined nonlinear function f to tanh(x); d) a feedforward connection part, connection between said linear artificial neuron circuit element and said nonlinear artificial neuron circuit element and comprising a plurality of multipliers, to multiply each output signal derived from each one of the linear artificial neuron circuit elements (xi) of the first layer by a feedforward connection coefficient wji and to output each of the multiplied output signals (wji xi) for all of the nonlinear artificial neuron circuit elements (yi) of the second layer; e) a recurrent connection part, connected between output ends of the nonlinear artificial neuron circuit element and input ends of the linear artificial neuron circuit element and comprising delay circuit elements (312) and associated multipliers (311), to multiply each output signal derived from the nonlinear artificial neuron circuit elements yj of the second layer by a recurrent connection coefficient λj, to output a first-order time delay corresponding to a time constant τj for the λj multiplied output signal, and, thereafter, to output the τj delayed and the λj multiplied output signal again to the corresponding one (xj) of the linear artificial neuron circuit elements of the first layer; and f) an auto connection part, connected between the linear artificial neuron circuit element and the same linear artificial neuron circuit element and comprising delay circuit elements (322) and multipliers (321), to multiply the output signal derived from each one xi of the respective linear artificial neuron circuit elements of the first layer by an auto connection coefficient μi, to output the first-order delay corresponding to the time constant τi for the μi multiplied signal, and to output the μ multiplied, τi delayed signal again for the corresponding one xi of the respective linear artificial neuron circuit elements. Description
The present invention relates generally to an apparatus based on a recurrent neural network (hereinafter, abbreviated as an RNN) and, more particularly, relates to the apparatus based on the RNN which carries out a generation of a nonlinear oscillation and/or its processing such as a nonlinear oscillation identification. The RNN in this specification is called URANn which will be described later.
The RNN is, generally, applicable to an information processing in a nonlinear system.
For example, when the RNN facilitates a design of circuit constants of a circuitry when a nonlinear oscillator is mounted in an electric or electronic circuit.
In addition, since the RNN (n-variable Unlimited Recurrent Adjustable Network; hereinafter, abbreviated as an URANn according to the present invention is capable of processing nonlinearly a time series (sequence) signal, URANn can effectively be utilized for a nonlinear filter and for a smooth control of an engineering system having remarkable nonlinearity characteristics such as a robot to which the nonlinear filter 1s applied.
Fluctuations (so called, 1/f fluctuations) involved in a chaotic temporal signal generated from the URANn can be used to provide waveform fluctuations and pitch fluctuations near to natural sounds for a sound source and rhythm source of such a composition and/or performing system of a computer music, a speech synthesis system, or voice converting system. Similarly, the temporal signal having such fluctuations as described above is applicable to control purposes of a vehicle or a building interior/exterior equipment such as an air conditioning system, lighting system, and body sonic sheet so as to contribute to an improvement of comfort in residential environments.
Furthermore, the temporal signal having such fluctuations as described above is again encoded in a spatial pattern so as to be permitted to assist in discussions on designs of an interior and exterior of the vehicle or the building and on the improvement in comfort of a light up pattern.
The temporal signal having the fluctuations and the spatial pattern having the fluctuations can effectively be utilized to enhance a stage effect on a dramatic performance (theatrical performance), concert, and so forth on a stage or, so-called, a live house.
FIG. 1 exemplifies a previously proposed nonlinear oscillation generating apparatus using URANn.
A structure described in FIG. 1 is called an excitatory, inhibitory two-cellular circuit model and is a nonlinear oscillation generating system constituted by a two-element artificial neurons (also called formal neurons or, simply, neurons, but hereinafter referred to the artificial neurons or artificial neuron elements) described, for example, in a Japanese Publication Book titled "An introduction to Neurocomputing" edited by Kauoru Nakano, page 143, published by Corona Publishing Co., Ltd. on Apr. 5, 1990 and in a IEEE's paper, Amari, S. (1992). Characteristics of random nets of analog neuron-like elements. IEEE Transactions on Systems, Mam and Cybernetics, SMC-2, 643-657.
Suppose that, in FIG. 1, average membrane potential of the artificial neurons x1 and x2 are denoted by u1 and u2, respectively. Here, a dynamical system can be described in equation of motion defined in equation (14) as a nonlinear function f having saturation characteristics. ##EQU2## wherein τ1 and τ2 denote time constants of the artificial neurons x1 and x2, respectively, μ1 and μ2 denote auto connection coefficients of the artificial neurons x1 and x2, respectively, w12 and w21 denote cross connection coefficients (in a case where the connections are inhibitory connections, these wji indicate negative values), and the above-described nonlinear function f is for example, f(x)≡tanh(x).
On the other hand, an APOLONN model reported by Masaaki Sato of ATR (Kabushiki Kaisha A.T.R, human information communication research institute) for learning a vocal fluctuation exemplifies another previously proposed neural network having a two-layer recurrent structure in which the excitatory, inhibitory two-cellular circuit models are connected in parallel to each other. The APOLONN model is, for example, described in pages 245 through 283, Chapter 7, "Learning of chaos dynamics in Recurrent Neural Network" of a Japanese Publication Book titled "Chaos in Neural System" edited by Kazuyuki Aibara and published by Tokyo Denki Daigaku Syuppan Kyoku in 1993 or one of international convention manuscripts of .left brkt-top.Masa-aki SATO, Kazuki Joe, Tatsuya HIRAHARA: APOLONN brings us to the real world: Learning nonlinear dynamics and fluctuations in nature", Proceedings of International Joint Conference on Neural Networks (IJCNN) 90 at San Diego, Vol. 1, pp. I581�I-587, 1990.right brkt-bot..
FIG. 2 shows a structure of the APOLONN model.
In FIG. 2, a part 10 enclosed by a broken line corresponds to the circuit model shown in FIG. 1. A part enclosed by a black disk corresponds to an output unit of the APOLONN model. White circles denoted by reference numerals 12 are paired with longitudinally aligned white circles 12 so as to constitute single units.
It is therefore a principal object of the present invention to provide URANn having a large degree of freedom in setting parameters, which is capable of generating and/or processing a nonlinear oscillation having a universal waveform and frequency, explicitly indicating a method of determining internal parameters thereof as formulae, and is capable of easily achieving the nonlinear oscillation having target or desired physical characteristics with less number of times trials are carried out.
The above-described object can be achieved by providing a recurrent neural network system having a two-layer hierarchical structure and a recurrent connection, each layer of said recurrent neural network including the same number of neuron elements, said recurrent neural network comprising: a) a first layer constituted by linear artificial neurons xi (i=1, 2, - - - , f, - - - n); b) a second layer constituted by nonlinear artificial neurons yj (j=1, 2, - - - , j, - - - , n) and which are constructed so as to convert input signals thereto into output signals by means of given same nonlinear functions f, their temperature parameters being denoted by Tj ; c) a feedforward connection part which is so constructed as to multiply outputs of the first layer artificial neuron xi by feedforward connection coefficients wji and so as to input each of the multiplied outputs to a corresponding one of all of the artificial neurons yj of said second layer; d) a recurrent connection part which is constructed so as to multiply outputs of the respective artificial neurons yj of said second layer by recurrent connection λj, so as to provide first-order delays of time constants τj for the multiplied outputs of the respective artificial neurons yj, and so as to input a corresponding one of the artificial neurons xj of said first layer; and e) an auto connection part which is so constructed as to multiply the outputs of the respective artificial neurons xi of the first layer, so as to provide the first-order delays of the time constants τi for the multiplied outputs from said auto connecting part, and so as to input again each of the first-order time delayed outputs to the corresponding one of said artificial neurons xi, and wherein said recurrent neural network operates in accordance with the following equation (1) and generates a nonlinear oscillation, ##EQU3## wherein f denotes the given nonlinear output function having saturation characteristics, and g denotes a characteristic function to discuss a stability of a differential equation system and an equilibrium state (or a stationary solution) when g=0.
The above-described object can also be achieved by providing an apparatus based on an n-variable unlimited recurrent adjustable network, comprising: a) at least hierarchically structured two layers, each layer having the same number (n) of neuron elements; b) a plurality of linear artificial neuron elements xi (i=1, 2, - - - , i, - - - , n) constituting a first layer of the two layers; c) a plurality of nonlinear artificial neuron elements yj (j=1, 2, - - - , j, - - - , n) having respective temperature-dependent parameters Tj and constituting a second layer of the two layers, each nonlinear artificial neuron element yj converting an input signal thereto into an output signal using a predetermined nonlinear function f; d) a feedforward connection part which is so constructed as to multiply each output signal derived from each one of the linear artificial neuron elements (xi) of the first layer by a feedforward connection coefficient wji and to provide each of the multiplied output signals (w11, w12, - - - w1j, - - - , w1n, w21, w22, - - - . w2i, - - - , w2n, - - - , wn1, wn2, - - - , wnj, - - - , wnn) for all of the nonlinear artificial neuron elements yj of the second layer: e) a recurrent connection part which is so constructed as to multiply each output signal derived from the nonlinear artificial neuron elements yj of the second layer by a recurrent connection coefficient λj, as to provide a first-order time delay corresponding to a time constant τj for the λj multiplied output signal, and, thereafter, as to provide the τj delayed and λj multiplied output signal again to the corresponding one xj of the linear artificial neuron elements of the first layer; and f) an auto connection part which is so constructed as to multiply the output signal derived from each one xj of the respective linear artificial neuron elements of the first layer by an auto connection coefficient μj, as to provide the first-order delay corresponding to the time constant τi for the μi multiplied signal, and as to provide the μi multiplied, μi delayed signal again for the corresponding one xi of the respective linear artificial neuron elements.
FIG. 1 is a circuit model view of an excitatory and inhibitory two-cellular circuit model described as a two-cellular neuron circuit generally disclosed as a nonlinear oscillation generating system in a Japanese Book titled "an introduction to neurocomputing" edited by Kaoru Nakano, page 143, published as a first publication at Apr. 5, 1990 from CORONA PUBLISHING CO., LTD.
FIG. 2 is a model view of an APOLONN MODEL reported by Masaaki Sato as a previously proposed example of a neural network having a two-layer recurrent network structure in which the excitatory and inhibitory two-cellular circuit models shown in FIG. 1 are connected in parallel to each other.
FIG. 3 is a neuron circuit model view of the recurrent neural network (URANn) (n≧2) according to the present invention which is capable of generating a nonlinear oscillation in an n-dimensional state space for explaining signal flows of the recurrent neural network (URANn) according to the present invention.
FIG. 4 is a neuron circuit model view of a 2-variable unlimited recurrent adjustable network (URANn) (n=2) as a specific example of the recurrent neural network (URANn) according to the present invention.
FIG. 5 is a detailed circuit block diagram of the example of URANn (n=2) shown in FIG. 4 in which the (URAN2) shown in FIG. 4 is constructed in the form of an analog (neuro)computing circuit.
FIG. 6 is a waveform chart of a temporal signal indicating limit cycle oscillations as a time evolution of output potentials of two linear artificial neurons xi (x1, x2) of the first layer 100 of FIG. 4.
FIG. 7 is a characteristic graph representing characteristics of an equilibrium state of a differential equation system representing a nonlinear oscillator constructed by the URAN2 shown in FIG. 4 so as to generate a nonlinear oscillation.
FIG. 8 is a characteristic graph for explaining calculations of parameters of connection coefficients between artificial neuron elements from parameters determining a geometric shape on the equilibrium state of URAN2 and internal parameters of the artificial neurons such as temperature(-dependent) constants.
FIG. 9 is a model view of an embodiment of one of application examples of the URANn in which a temporal signal is input to and output from a nonlinear oscillator using URANn so as to achieve a nonlinear filter.
FIG. 10 is a circuit block diagram of an 1/f fluctuations generating apparatus to which a chaotic temporal signal generating function of the nonlinear oscillator using URANn according to the present invention is applicable as another embodiment of the application examples of the URANn.
FIG. 11 is a circuit block diagram of a nonlinear feedforward control system to which the temporal processing function of the nonlinear oscillator using URANn according to the present invention is applicable as another embodiment of the application examples of URANn.
FIG. 12 is a circuit block diagram of a nonlinear feedback control system to which a temporal processing function of the nonlinear oscillator using the URANn according to the present invention is applicable as another embodiment of the application examples of URANn.
FIG. 13 is a circuit block diagram of a nonlinear characteristic identification apparatus to which the temporal processing function of the nonlinear oscillator using URANn according to the present invention is applicable to a still another embodiment of URANn.
FIGS. 14A, 14B, and 14C are waveform charts representing temporal waveforms in the generated nonlinear self-sustained oscillations in a space of xi -τ as a result of an actual numerical simulation in equations (7) through (12) and equation (29).
FIG. 15 is a characteristic graph of a "strange attractor" derived and expressed from a chaotic temporal signal having the same chaotic temporal signals as those shown in FIGS. 14A through 14C as a solution trajectory of a three-dimensional state space of x1 -x2 -x3, each internal parameter being a numerical value shown in equations (34) and defined in an equation (1).
Before explaining preferred embodiments of URANn according to the present invention, the following problems have occurred in the previously proposed nonlinear oscillation generation and/or processing apparatus using the recurrent neural network shown in FIGS. 1 and 2 and described in the BACKGROUND OF THE INVENTION.
(1) since, in the sole excitatory and inhibitory two-cellular neuron circuit model shown in FIG. 1, the number and quantity of the internal parameters are less, a degree of freedom of setting the parameters is small and waveforms and oscillation frequencies of the temporal signals which can be generated by the previously proposed neural network have been limited. Especially, since structures of a feedforward part and a feedback part are not separated from each other, it is impossible to determine connection coefficients of the feedforward part and connection coefficients of the feedback part, independently from each other, respectively, from observations on a geometrical shape of the equilibrium state defined in the above-described equation (14).
(2) Since the internal structures are too complex even in the recurrent neural network having the hierarchical structure, many recurrent neural networks described above cannot derive a method of designing the internal parameters to generate the nonlinear oscillation in the form of formulae. Therefore, the previously proposed recurrent neural networks, for example, shown in FIG. 2, require repetitive learnings for a long time so as to generate a desired nonlinear oscillation.
For example, in a case as shown in FIG. 2 of the APOLONN model proposed by Masaaki Sato et al., it is necessary to use the two-cellular neuron circuits which are combined with 30 of the two-cell neuron circuits connected in parallel to one another in order to succeed the learning although the APOLONN model can be simulated by means of a supervised learning even if the output signal thereof is a chaotic temporal signal. At this time, the connection coefficients having the number up to 1,000 must be optimized by updating 30,000 numbers of times the connection coefficients from a random initial value. Consequently, tremendous amounts of calculations such that a computer would be used and would be operated continuously for about one week would be required even if a parallel processing computer such as Aliant were used.
(3) Kenichi Funahashi of Toyohashi Technology and Science Institute showed that a solution trajectory of a differential equation system considered in the state space x1 -x2 - . . . -xi - . . . xn can approximate an arbitrary spatial curve (chaos, torus, and limit cycle inclusive) and showed that a theorem that a combination of the internal parameters which can achieve the above-described approximation of the arbitrary spatial curve is present. However, this theorem merely indicates the presence of the recurrent neural network and specific derivations of the values of the parameters are not carried out. In addition, the method of deriving the parameter values described above is not described therein any more. Hence, this proposed theorem cannot be applied to an actual engineering technology.
Referring to FIG. 3, FIG. 3 shows a presented embodiment of a recurrent neural network abbreviated as URANn and operated in accordance with an equation (1) representing a differential equation system.
An apparatus shown in FIG. 3 is provided with a two-layer hierarchically structured recurrent neural network (URANn) having a first layer element group 100 and a second layer element group 200, each element group including neuron circuit elements (so-called, neurons, or artificial neurons) and each layer having the same number (n) of neuron elements.
In addition, the first layer (element group) 100 is constituted by the plural linear artificial neurons xi (i=1, 2, . . . , n). It is noted that the artificial neurons are also called neurons, neuron elements, or formal neurons but hereinafter referred to as artificial neurons or artificial neuron elements. The linear artificial neurons xi have the same functions to add and/or subtract input signals and to output linearly amplified signals after linear amplifications (input sum of each thereof is xi, an amplification factor of each thereof is 1, and an output of each thereof is xi), respectively. The second layer (element group) 200 is constituted by the plural nonlinear artificial neurons yj. The nonlinear artificial neurons yj have the same functions to add and/or subtract input signals and to output nonlinearly calculated signals after nonlinear calculations of yj =f(uj), (input sum of each thereof is uj and output of each thereof is yj). It is noted that a reference numeral 210 denotes temperature(-dependent) parameters Tj (j=1, 2, . . . , j, . . . , n) of the nonlinear artificial neurons yj.
It is desirable to use a highly symmetrical nonlinear function f for the nonlinear output function to convert the input sum uj of each nonlinear artificial neuron yj of the second layer (element group) 200 into the output yj. In this embodiment shown in FIG. 3, the output yj =f(uj)=tanh(uj) is adopted. However, it is, alternatively, possible to make a mathematically equivalent discussion by carrying out a suitable change of variables if a generally known sigmoid function, i.e., f(uj)=1/ 1+exp(-uj)! is used.
It is noted that tanh(uj)=sinh(uj)/cosh(uj), sinh(uj)=(eu j-e-u j)/2, cosh(uj)=(eu j+e-u j)/2.
Referring to FIG. 3, a feedforward part 300 transmits its input signal from each of the linear artificial neurons xi, which is multiplied by wji therein, to each one of the nonlinear artificial neurons yj. That is to say, the feedforward part (section) 300 multiplies the respective outputs of the artificial neurons xi of the first layer 100 by feedforward connection coefficients wji and, thereafter, outputs the wji multiplied signals into all artificial neurons of the second layer 200, e.g., for y1, w11, w12, - - - , w1i, - - - , w1n being interconnected from x1, x2, - - - , xi, - - - , xn, for y2, w21, w22, - - - , w2i, - - - , w2n being interconnected from x1 through xn, and for yn, wn1, wn2, - - - , wni, - - - , wnn being interconnected from x1 through xn.
As appreciated from above, the coefficients wji denotes feedforward connection coefficients between the first layer 100 and the second layer 200.
Referring to FIG. 3, a recurrent connection part 310 includes excitatory or inhibitory connections and have a first-order time delay (lag).
That is to say, a signal of each artificial neuron yj of the second layer 200 is multiplied by λj therein, is provided with the first-order time delay (lag) τj, and is fedback to its corresponding one of the artificial neuron xj placed in the vertical direction with respect to the recurrent connection part 310. The symbol λj is the recurrent connection coefficient between the corresponding artificial neurons in the first layer 100 and the second layer 200.
Referring to FIG. 3, an auto connection part 320 includes excitatory or inhibitory connections and has the same first-order time delay (lag). That is to say, signals from the respective artificial neurons xi of the first layer 100 are multiplied by μi and are provided with the first-order time delay (lag) τi, and are fedback again to the artificial neurons xi of the first layer 100. The symbol μi denotes an auto connection coefficient of each artificial neuron xi of the first layer 100.
Herein, the following equation (1) is given: ##EQU4##
A time constant τi of the first-order delay (lag) of URANn expressed at a left side of the equation (1) is dependent upon a time constant that a device to which URANn is applicable inherently (naturally) has, the device being adopted, for example, when URANn is actually reduced into practice as a specific system. Therefore, the expression in the equation (1) is disadvantageous in an engineering application thereof to the system. Hence, in the present invention, to solve this problem, a newly expressed dynamical system obtained by a change of variables for the above-described equation (1) namely, the dynamical system expressed in an equation (2) is prepared. ##EQU5##
In the equation (2). 2Ci can represent a distance between asymptote of equilibrium state denoted by gi (x1, x2, . . . , xn)=0 within a state space.
In addition, the equation (2) represents a "standard" differential equation system to provide a good prospect of analysis for derivations of parameter determinative formulae as will be described later (in examples which will be described later, n=2 and n=3) and to facilitate their numerical calculus. Although, in the specification, only results of calculations are shown, when deriving the formulae to determine the internal parameters of URANn, it is more convenient to advance the observations (Considerations) with the state space represented by the equation of (2) (the state space described above is one of phase spaces extended by the outputs of the linear artificial neurons xi of the first layer 100) as a center of observations (considerations). It is noted that a "small parameter" εi which is possible to have an arbitrary value (refer to the equation (2)) can be interpreted as a virtual time constant in the n-dimensional state space: {x1, x2, . . . , xi, . . . , xn }. The small parameter "εi " can equally correspond to the time constant τi (having the value in the order corresponding to that inherently provided in the actual device to which URANn is applicable) of the equation (1) via the coefficients of the feedback connections (i.e., auto connection coefficient μi of the part 320 and recurrent connection coefficient λi of the part 310), as appreciated from the formulae (5) and (6) in the case where (n=2) or formulae (9) and (10) in the case where (n=3).
f(x)&#8801;tan h(x)                                        (15)
is adopted as a nonlinear saturation type output function, the following relationship (16) is established when a relationship between the geometrical shape of the equilibrium state gi (x1, x2, . . . , xn)=0 within the state space and connection coefficients in the feedback parts 310 and 320 shown in FIG. 3 is discussed:
&#955;i =Ci (1-&#956;i)                     (16).
The above-described equilibrium state gi (xi, x2, . . . , xn)=0 has a spatial figure derived from the equation (1) in which the right side thereof=0 and provides an effective tool for qualitatively discussions on the stability of the differential equation system. It is noted that, when n=2, the above-described equilibrium stare's curved surface is turned to be an equilibrium state's curved "line".
The change of variables to associate the small parameter (virtual time constant) ε1 in the state space with the first-order delay time constant τi is given by:
&#964;i =(1-&#956;i)&#949;i                 (17).
The given simultaneous equations (1) is uniformly transformed into the standard differential equation system (2) by using the combination of the equations (16) and (17).
By solving these equations (16) and (17) for the auto connection coefficients μi and for the recurrent connection coefficients λi, formulae to determine coefficients (auto connection coefficients μi of the part 320 and recurrent connection coefficients λi of the part 310, shown in FIG. 3) of the feedback connection parts can be derived from the time constant τi in the equation (1) and the small parameter εi in the equation (2). These formulae will be described later.
Next, FIG. 4 shows another preferred embodiment of URANn, wherein n=2, i.e., URAN2 according to the present invention.
The structure of URANn in the case where n=2 (URAN2) will be described below with reference to FIG. 4.
In the case where n=2, the dynamical system representing URAN2 will be described in a 2nd-order nonlinear ordinary differential equation expressed below. ##EQU6##
The above-described URAN2 can easily be realized by means of, for example, an analog (electronic computing) circuit, a digital (electronic) circuit, and/or a microcomputer system.
FIG. 5 shows a circuit block diagram of URAN2 when the structure of URAN2 shown in FIG. 4 is realized by the analog electronic (computing) circuit.
Summers (or adders) Σ101 shown in FIG. 5 constitute the linear artificial neurons xi of the first layer 100, the summers serving to add and/or subtract input signals thereof, respectively. The feedforward connection parts wji of the feedforward part 300 connecting the linear artificial neurons xi of the first layer 100 with the nonlinear artificial neurons yj of the second layer 200 are constituted by multipliers 301 or alternatively, linear amplifiers (may be constituted by operational amplifiers having gains of 1). The nonlinear artificial neurons yj of the second layer 200 can be constituted by adders (summers) 201 and nonlinear function generators 202, respectively, as shown in FIG. 5.
Referring to FIG. 5, each recurrent connection part 310 having the corresponding recurrent connection coefficient λj fedback from the nonlinear artificial neuron yj of the second layer 200 into the nonlinear artificial neuron xj of the first layer 100 is constituted by a delay circuit element 312 (for example, a charge coupled device or register) and its series-connected multiplier 311. An auto connection part 320 having the auto connection coefficient μi and which is started from the artificial neuron xi of the first layer 100 and is fedback again to the same artificial neuron xi can be constituted by the multiplier 321 and its series-connected delay element 322, as appreciated from FIG. 5.
On the other hand, URANn, in the case of n=2 can be realized by the microcomputer system, as described above.
That is to say, either the equation (1) or the equation (2) is transformed into time discrete values and the discrete values are transformed into difference equations so that a discrete system program corresponding to FIG. 4 or FIG. 5 can be realized. An equation (38) represents the difference equation in the time discrete system. The microcomputer system includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), a common bus, and I/O interfaces. In this case, for the nonlinear calculation in the nonlinear artificial neuron yj of the second layer, i.e., yj =f(uj), a magnitude of the input uj is made correspond to an address number using the ROM, a value corresponding to a magnitude of the output yj being stored into the content of the ROM whose address corresponds to the magnitude of the input uj, so that an arbitrary function can be generated.
The equation (38) will be described below: ##EQU7##
In addition, in a case where the circuit shown in FIG. 5 is used as a specific nonlinear oscillation generating (oscillating) apparatus, as the ouput of the nonlinear oscillatory signal, x1 (t) and x2 (t) output from the adder 101 (refer to FIG. 5), namely, the output signals of r1 (t) and r2 (t) (Rnn output 1 and Rnn output 2) can be used. Alternatively, y1 (t) and y2 (t) output from the nonlinear artificial neurons y1 and y2 of the part 200 may be used.
Furthermore, S1 (t) and S2 (t) of FIG. 5 (external input 1 and external output 2) serve as input signals when the whole circuit shown in FIG. 5 is used as a nonlinear filter. In this case, when the temporal signals S1 (t) and S2 (t) are input and either the above-described x1 (t) and x2 (t) or the above-described y1 (t) and y2 (t) are output from the circuit shown in FIG. 5, a signal conversion based on the input/output characteristics of URAN2 previously prepared is carried out. Consequently, the whole circuit shown in FIG. 5 operates as the nonlinear filter.
Next, an action of the embodiments of the URANn according to the present invention will be described below.
In order to discuss the stability of the equation (1) (namely, the nonlinear ordinary differential equation) representing the dynamical system of the given URAN2, a Jacobian matrix is, herein, defined in the following matrix of (19). The Jacobian matrix is derived by a linearization of the differential equation system in the vicinity of points (x1, x2) in the state space defined by the outputs x1 -x2 of the linear artificial neuron xi of the first layer 100. ##EQU8##
Here, each component in a left-sided matrix of the Jacobian matrix (19) is given by the following equations (20). ##EQU9##
In these embodiments described above, since the highly symmetrical nonlinear function as f(x)≡tanh(x) is adopted as the nonlinear output function f, a case where (x1, x2)≡(0, 0) can be considered in the equation (19) in order to discuss the stability of an equilibrium point are viewed from such a condition that an unsalability equilibrium point of the dynamical system of URAN2 is coincident with an origin at the state space of x1 -x2. Hence, the following equation (21) can be derived from the above-described equations of (19) and (20). ##EQU10##
Then, if an eigen value ψ.sub.� of J (0, 0) described above is established to be equal to the following equation (22), the eigen value ψ.sub.� can be calculated as in the next following equation (23) from the above equation (21). ##EQU11##
Here, the operation of the dynamical system of URAN2 before and after a Hopf bifurcation point is determined as follows depending upon a positive or negative of a real part of the eigen value ψ.sub.� of J (0, 0) described above.
That is to say, (1A) in a case where the real part of the eigen value ψ.sub.� of the Jacobian matrix (19) is negative (Re ψ.sub.� !=p<0), this URAN2 carries out a damped oscillation; and (2A) in a case where the real part of the eigen value ψ.sub.� is positive (Re ψ.sub.� =p>0), the URAN2 carries out a self-sustained oscillation (a self-sustained vibration but hereinafter referred to as a self-sustained oscillation).
Hence, in order to force URANn to oscillate in the self-sustained oscillation, the internal parameters such as interlayer connection coefficients wji, temperature(-dependent) parameters Tj, recurrent connection coefficients λi, and auto connection coefficients μi may be selected so as to satisfy the condition such that Re ψ.sub.� !>0.
FIG. 6 shows waveform charts of x1 (t) and x2 (t) representing a result of numerical simulation of the nonlinear self-sustained oscillation based on the above-described conclusion.
Characteristic graphs in FIG. 6 are graphs in the space xi -t as the result of numerical simulation in the case where the values of the respective internal parameters are set as those in the following equation (24).
w11 =1.2, w12 =-1.0, T1 =0.2, &#956;1 =-9.0, &#955;1 =10.0, &#964;1 =0.1,
w21 =1.0, w22 =1.2, T2 =0.2, &#956;2 =-9.0, &#955;2 =10.0, &#964;2 =0.1                    (24).
It is noted that, in a case where each internal parameter is set in the above-described numerical values of the equation (24),
Re &#968;.sub.� !=Re 50�50i!=50&gt;0,
Consequently, the condition of Re ψ.sub.� !>0 is satisfied.
As described above, it is appreciated that the URANn=2 according to the present invention functions as the nonlinear oscillator or nonlinear oscillation generating apparatus.
Next, as a method of giving the generating condition for the self-sustained oscillation, a limit cycle in the two-dimensional state space will be exemplified.
The limit cycle is defined as a closed curve in the state space of a particular dynamical system, from which state trajectories may recede, or which they may approach, for all initial states sufficiently close to the curve (refer to page 716 of the New IEEE Standard Dictionary of Electrical and Electronics Terms, Fifth edition).
A geometrical shape in the state space of x1 -x2 of the equilibrium state of the standard-type second-order nonlinear ordinary differential equation shown in the equation (2) is represented by a normal vector Ai on an equilibrium point (herein, it is coincident with the origin of the state space) of the two equilibrium solution states shown in the following equation (25), a distance Ci from the equilibrium point to an asymptotic line of the equilibrium states, and a temperature parameter Tj of the nonlinear artificial neuron yj of the second layer 200. The equilibrium states described above of the equation (2) is expressed as:
g1 (x1, x2)=0
g2 (x1, x2)=0                               (25).
Furthermore, using the normal vectors of the equilibrium states shown in the equation (3), the feedforward connection coefficient wji of the part 300, as shown in the following equation (4), can be determined: ##EQU12##
In addition, by arranging the two equations of (16) and (17), a formula (5) which gives the auto connection coefficient μi of the part 320 and a formula (6) which gives the recurrent connection coefficient λi of the part 310 can be derived.
The equation (16) shows the relationship of
&#955;i =Ci (1-&#956;i)                     (16)
derived according to the geometrical shape of the equilibrium state within the state space along with the adoption of
f(x)&#8801;tan h(x)                                        (15).
The equation (17) shows the change of coefficients to associate the small parameter (virtual time constant)εi in the state space with the first-order time delay constant τi, of URAN2. ##EQU13##
The second-order nonlinear ordinary differential equation shown in the equation (18) can be transformed into a standard equation (26) (corresponds to the case of n=2 in the equation (2)) described below if the formulae (5) and (6) derived above are used. ##EQU14##
As compared with the original equation (18), in the above-described equation (26) , the numbers of parameters are not only apparently reduced but also the small parameter εi within the two-dimensional state space is adopted as the coefficient corresponding to the time constant. Therefore, the small parameter εi can be operated independently of τi (having an order of the time constant that the device constituting URAN2 inherently has). Then, the standard differential equation (26) is considered in the two-dimensional state space and the internal parameters of URAN2 are determined in accordance with the limitation condition equations (3) through (6) so as to generate the nonlinear oscillation having the desired physical characteristics consequently, the coefficients of the given differential equation (18) are determined.
As described above, if the defined equation (3) and the derived equations (4), (5), and (6) are deemed to be the formulae, the parameters of the respective parts of URAN2 can be determined on the basis of the deemed formulae (3) through (6).
To generate the self-sustained oscillation (limit cycle), the normal vectors Ai and feedforward connection coefficients wji may be determined so as to satisfy, for example, the following conditions:
w12 &lt;0, w21 &lt;0, a11 a12 &lt;0, a21 a22 &gt;0(27).
In addition, if the waveform of the generated temporal signal is desired to be the symmetrical, an additive condition such that
A1, A2 0                                         (28)
may be added to the above-described conditions (27).
That is to say, if URAN2 is so designed as to satisfy the above-described equations (3) through (6) which represent the relationships between the internal parameters in URAN2 (in the case where the self-sustained oscillation having the symmetrical waveforms, the conditions (27) and (28) are included), the nonlinear oscillator having the structure of URAN2 can generate the nonlinear temporal signal having a desired physical property. It can be appreciated that this can, in general, determine the stability of the given dynamical system so that the real part of the complex eigen value ψ.sub.� of the Jacobian matrix J (0, 0) of the second-order nonlinear ordinary differential equation in the vicinity of the equilibrium point (in this case, the origin in the state space x1 -x2) gives a positive and the self-sustained oscillation can be generated from URAN2 in the embodiment when the above-described formulae (3) through (6) are applied to URAN2.
At this time, when the temperature parameter Tj of the part 210 has a value which is positive and sufficiently large, the equilibrium state is slowly bent. When the temperature parameter Tj approaches to zero which is limited value of Tj, the bent direction of the equilibrium state becomes steep.
FIG. 8 shows a characteristic graph representing the above-described relationship. This characteristic is such that the equilibrium state is in accordance with the above-described equation (25) and symbols of Ai and Ci are in accordance with the equations (3), (4), and (6).
It is noted that the time evolution solution of the second-order nonlinear differential equation which is derived according to the numerical simulation corresponds to FIGS. 6 and 7.
In FIGS. 7 and 8, although the calculated plotted points are in dotted forms, actually the time evolution solution described above is in the continuous curves for both x1 (t) and x2 (t).
Next, the operation of URANn (n=3, URAN3) will be described below.
In a case of either the equation (1) or equation (2), the same consideration as described in the case of URAN2 is given to the URAN3 so that equations (7) through (12) are derived.
That is to say, the normal vectors of the equilibrium states (surfaces) are defined by the following equation (7). The formula to give the feedforward connection coefficients wji of the part 300 is represented by the following equation (8). The formula to give the auto connection coefficients μi of the part 320 is represented by the following equation (9). The formula to give the recurrent connection coefficients λi of the part 310 is represented by the following equation (10). (Equations (11) and (12) will be described later). ##EQU15##
The internal parameters for the respective parts of URAN3 are determined on the basis of the above-described respective formulae (7) through (10).
Here, to generate the self-sustained oscillation, the normal vectors Ai of the equilibrium states shown in the following equation (30) and feedforward connection coefficients wji may be adjusted within a range which satisfies, for example,
w12 &lt;0, a11 a12 &lt;0, a13 a12 &lt;0,
w21 &gt;0, a22 a21 &gt;0, a23 a21 &gt;0,
w31 &gt;0, a32 =0, a33 a31 &lt;0             (29).
g1 (x1, x2, x3)=0
g2 (x1, x2, x3)=0
g3 (x1, x2, x3)=0                      (30).
When the conditions imposed on the normal vectors (refer to the above-described equation (7)) to satisfy the generating condition of the self-sustained oscillation are described by two of the following (I) and (II), the normal vectors with the following conditions of (I) and (II) can be represented by the following equations (11).
A1 O =(a11, a12, 0)
A2 O =(a21, a22, 0)
A3 O =(a31, 0, a33)                     (11).
"Condition (I): To adopt a singular perturbation form, a constraint surface is formed by the third equilibrium state denoted by
g3 (x1, x2, x3)=0                      (31)
and its normal vector A3 O is present within a plane of x1 -x3 ".
"Condition (II): To rotate the vector field about the x3 axis, the normal vectors A1 O and A2 O of the first and the second equilibrium states denoted by
g2 (x1, x2, x3)=0                      (32)
are present in the plane of x1 -x2.
Furthermore, it is necessary for a trajectory of the solution curve in the state space to have a "folding structure" in order to generate a chaotic temporal signal. To achieve this, the normal vectors A1 O and A2 O may be changed so that a cross line of the above-described two equilibrium states playing a role of the rotation of a vector field (the cross line described above equals a line of inflection) is not in parallel to the x3 axis, the cross line being defined as
g1 (x1, x2, x3)=g2 (x1, x2, x3)(33).
Here, suppose that offset coefficients at a cross point of the cross line with respect to the plane x3 =C3 in directions of x1 and x2 are denoted by k1 and k2, respectively.
The normal vectors after the above-described shift are expressed in the following equations (12). ##EQU16##
Hence, fine adjustments of the parameters such as the offset parameters k1 and k2, the temperature parameters Tj of the part 210, and/or small parameters εi can cause the URAN3 to generate the chaotic temporal signal having the desired physical property.
When the chaotic temporal signal is generated from URAN3 whose internal parameters are determined using the above-described equations (7) through (12) and the equation (29), various combinations of the parameters can be considered.
In the embodiment of URAN3, the numerical values of these parameters described in the following equations (34) are adopted as one example of the parameters of URAN3.
w11 =-1.1, w12 =-1.0, w13 =0.085.
T1 =1.0, &#956;1 =0.0, &#955;i =1.0, &#964;1 =1.0.
w21 =1.0, w32 =1.1, w23 =0.1175.
T2 =1.0, &#956;2 =0.0, &#955;2 =1.0, &#964;2 =1.0.
w31 =-1.0, w32 =0.0, w33 =0.35.
T3 =0.2, &#956;3 =-19.0, &#955;3 =20.0, &#964;3 =1.0(34).
In addition, the equilibrium point is given:
g1 (x1, x2, x3)=g2 (x1, x2, x3)=g3 (x1, x2, x3)              (35).
Here, since the eigen value of the differential equation (1) at the equilibrium point of x3 ≠0 takes a form of:
(a+bi, a-bi,-c)                                  (36),
when a>0, b>0, and c>0, it can be appreciated that this equilibrium point is an unstability fixed point indicating a one-dimensional stability and a two-dimensional unstability.
FIGS. 14A through 14C show characteristic graphs of the temporal waveforms of the generated nonlinear self-sustained oscillations in a space of xi -t(i=1, 2, and 3) as the results of actual numerical simulations in the equations (7) through (12) and the equation (29).
At the times of simulations, the solution curves in the state space x1 -x2 -x3 take a form of an attractor structure in FIG. 15.
It is noted that the characteristic graphs shown in FIGS. 14A through 14C and FIG. 15 are the characteristics when the respective internal parameters indicate the numerical values of the equations (34) which are substituted for the equation (1).
As far as the qualitative appreciation is concerned, since in the attractor shown in FIG. 15, a "folding structure" is observed In the solution trajectory and, in turn, in the temporal waveforms shown in FIGS. 14A through 14C, "intermittent jumps" are observed, the generated temporal signals can be said to considerably be chaotic.
Furthermore, in order to make a quantitative determination for the chaotic characteristics, when Lyapunov (Liapunove or Liapounoff) Exponents are calculated, the following numerical values are established:
(0.08, 0.00, -16.40)                                       (37).
It is, therefore, concluded that the derived attractor shown in FIG. 18 is a "strange attractor". Furthermore, such generated temporal signals as shown in FIGS. 14A through 14C are "chaotic temporal signals".
URAN3 in this embodiment functions as the nonlinear oscillator (of nonlinear oscillation generating apparatus) which is capable of generating a variety of temporal signals including the chaotic temporal signals.
Next, a method of adjusting the first-order time delay constant τi of URANn so that the number of times oscillations occur per rime (oscillation frequency) of the generated nonlinear oscillation achieves a target state, in the state space of xi -t represented by output potentials xi of the linear artificial neurons xi of the first layer 100 and time axis t, will be described below.
This adjustment operation is executed, for example, for a periodic (not chaotic) nonlinear oscillation waveform in the state space of xi -t as shown in FIG. 8.
That is to say, referring back to FIG. 6, when a sufficient time has passed after the start of the nonlinear oscillation and the oscillation has arrived at a steady state in FIG. 6, suppose that a point at which the nonlinear oscillation waveform derived by observing the output potential of the linear artificial neuron xi of the first layer 100 of URANn first crosses the time axis t is denoted by (t0, 0) and a point at which it crosses after an m period is denoted by (tm, 0).
When the first-order time delay constant of the dynamical system of URANn is previously set as τO, an arbitrary time constant τO is given according to the following equation (13) so that this URANn is caused to generate a trial nonlinear oscillation. Thereafter, the method of adjusting time constant τ can be provided so that URANn executes the self-sustained oscillation at a target oscillation frequency ω. ##EQU17##
That is to say, the method of adjusting the time constant to calculate the target time constant τ is provided according to the equation (13) from the previously set time constant τ0 so that the generated nonlinear oscillation provides the target oscillation frequency ω on the basis of an actual measurement of the passed time duration on the time axis from a time at which the oscillation waveform in the state space of xi -t of the output potential of xi has crossed first the time axis t to a time at which it has crossed again the time axis t after m period.
It is noted that the above-described τ may be considered to be the first-order time delay constant τi in the equation (1) and to be the small parameter εi in the equation (2).
It is also noted that an application range of the equation (13) excludes a chaotic region as apparently from a standpoint of the physical property of the nonlinear dynamical system.
As described above, the cases of n=2 and n=3 have been explained for URANn.
In general, URANn having each layer including the n-th neuron elements (n≧2) functions as the generating apparatus of the nonlinear oscillation in the self-sustained oscillation state. As the unstability of the dynamical system (differential equation system) becomes increased, a solution trajectory in the state space extended by the output potentials xi of the linear artificial neurons of the first layer 100 repeats bifurations such as Hopf bifurcation. The solution orbit can easily be estimated from the standpoint of the physical property of the nonlinear oscillator that it is often and finally transited into the strange attractor, i.e., chaos via the limit cycle (periodic orbit) and/or torus (in the two-dimensional phase space, quasiperiodic orbit).
The above-described embodiments indicate the generations of the limit cycle (refer to FIGS. 6 through 8) and of the chaos (refer to FIGS. 14A through 14C and FIG. 15), particularly, by giving the specific numerical values to the internal parameters of URANn.
Hence, URANn according to the present invention can effectively be applied to the variety of systems as the variety of nonlinear oscillation generating apparatus.
In this case, the output temporal signals from URANn may be defined by the output potential xi of the linear artificial neuron xi of the first layer 100 or may be defined by the internal potential uj of the nonlinear artificial neuron yj of the second layer 200, or may be defined by the output potential yj thereof.
The nonlinear oscillator achieved by URANn having each layer including the n-th elements (n≧2) can exhibit functions of, so-called, "neural cocktail party effect" and/or "hyper chaos generation" due to mutual attractions between the nonlinear oscillations such as the limit cycle (periodic trajectory), torus (quasiperiodic orbit), and chaos in a mutually synthesized oscillation state of a plurality of oscillators. This can easily be estimated from physical characteristics of the nonlinear oscillators.
Hereinafter, application examples of URANn will be described below.
First, FIG. 9 shows one embodiment of the application examples of URANn which carries out the temporal signal processing.
It is noted that a specific circuit structure of URANn shown in FIG. 9 is already shown in FIG. 5 (n=2).
In the structure of the URANn shown in FIG. 9, each temporal input signal Si (t) (i=1, 2, - - - , n) is input to the corresponding one of the artificial neurons xi of the first layer 100 and each output xi (t) of the artificial neuron of the first layer 100 serves as each temporal output signal ri (t) of URANn, in addition to the basic structure shown in FIG. 3.
The circuit structure of FIG. 9 can be used as the nonlinear filter for the purpose of the temporal signal processing.
It is noted that each temporal signal denoted by Sj (t) received by URANn may, alternatively, be input to the corresponding one of the nonlinear artificial neurons yj of the second layer 200. It is also noted that the internal potential uj of, alternatively, the output potential yj (t) of the nonlinear artificial neuron yj of the second layer 200 may, alternatively, be used as the temporal signal rj (t) output from URANn.
In addition, in a case where the structure of FIG. 9 is used as the nonlinear filter to convert the temporal signals, a suitable input element Si (for example, xi or yi) is selected and the temporal signal Si (t) is input to the selected input element Si. Here, the signal conversion is carried out on the basis of previously set input/output characteristics of URANn and, thereafter, the output temporal signal ri (t) can be derived from the selected output element ri (for example, xi or yi). Here, which of the output signals is used may properly be selected depending upon desired input/output relationships.
FIG. 10 shows a schematic circuit block diagram of an embodiment of the application examples of the structure of URANn shown in FIG. 9 to a practical system.
A temporal signal generator 1000 to which the structure of URANn according to the present invention is utilized is provided and operates as a fluctuation generator of such as, so-called, 1/f (f: frequency) fluctuations (also called, in an electric resistor, flicker noises, or pink noises).
As shown in FIG. 10, any one of several temporal output signals output from the temporal signal generator 1000 is selected by means of, for example, a multiplexer from among several temporal output signals x1 through xn and the selected temporal signal is retrieved and converted into a corresponding voltage by means of a frequency/voltage converter 420. The corresponding voltage signal derived from the frequency/voltage converter 420 serves to actuate a driving unit 430 such as a motor or another actuator. Consequently, the fluctuations (for example, 1/f fluctuations) involved in the chaotic temporal signal generated from URANn (1000) can be used to provide waveform fluctuations and/or pitch fluctuations near to natural sounds for a sound source or a rhythm source of a composition/performance system, a speech synthesis system, or voice (vocal, or utterance) conversion (or translation) system.
In the same way, the temporal signal derived from URANn having the fluctuations can be applied to control operations for a vehicle, building interior/exterior equipment such as an air conditioner, lighting system, and a body sonic sheet so as to contribute to an improvement of comfort in residential environments.
In addition, if the temporal signal having the fluctuations are modified and encoded into a spatial pattern, it can contribute to the discussions on the design of the vehicle and the building interior/exterior equipment and to the improvement in the comfort in a light up pattern.
Furthermore, the temporal signal and the spatial pattern having the fluctuations can effectively be utilized to enhance a stage effect on a theatrical performance and/or musical concert in a stage or a live house.
It is noted that reference numeral 410 shown in FIG. 10 denotes a switch selectively set to activate or to stop the temporal signal generator 1000.
Next, FIG. 11 shows another embodiment of the application examples of URANn according to the present invention to a feedforward control system.
In FIG. 11, the reference numeral 100 denotes URANn shown in FIG. 9, reference numeral 2000 denotes a controlled object of the feedforward control system shown in FIG. 11, a reference numeral 3000 denotes a feedback circuit (k denotes a feedback gain), and reference numeral 4000 denotes an adder.
According to the structure shown in FIG. 11, a result of the signal conversion of the input control signal S(t) on the basis of the input/output characteristics of the prepared URANn 1000 is provided for the controlled object 2000 as r(t) so that it becomes possible to compensate for a nonlinear behavior that the controlled object 2000 inherently has. The input signal z(t) of the controlled object 2000 can be expressed as z(t)=-kz(t)+r(t)=-kz(t)+x(S(t)).
Next, FIG. 12 shows another embodiment of the application examples of URANn shown in FIG. 9 according to the present invention to a feedback control system.
In FIG. 12, the reference numeral 1000 denotes URANn shown in FIG. 9, the reference numeral 2000 denotes the controlled object 2000, a reference numeral 3100 denotes an input gain (input amplifier) having the gain k, and the reference numeral 4000 denotes an adder (summer).
The structure shown in FIG. 12 is a system to compensate for the nonlinear behaviour of the controlled object 2000 in the feedback control mode.
As described in FIG. 12, a subtraction result of the feedback output r(t) of URANn 1000 from the input signal p(t) multiplied by k by means of the input gain 3100 is input to the controlled object 2000.
next, FIG. 13 shows another embodiment of the application examples of URANn according to the present invention to a nonlinear characteristic identification apparatus for an unknown plant 2000.
The structure shown in FIG. 13 represents the apparatus for identifying a nonlinear characteristics of the plant 2000 whose characteristics (transfer function) is unknown.
In operation, the input signal p(t) is passed through URANn 1000 as its input of s(t) to provide the value of r(t). Here, the output z(t) of the unknown plant 2000 is added to the adder (4100). In addition, The adder 4100 receives the value of r(t) from URANn together with z(t) so that the internal parameters of URANn 1000 are modified to give an output error signal e(t) of both signals z(t) and r(t) zero, In this way, the internal parameters of URANn in the case where the output error signal e(t) gives zero correspond to internal parameters (,i.e., the nonlinear characteristics) of the unknown plant 2000.
According to the identification apparatus shown in FIG. 13, the internal parameters of the identified unknown plant 2000, which is the nonlinear dynamical system, such as a viscous resistance having a nonlinear temperature characteristics in, for example, a multi-axis robot (or manipulator), a Coriolis force in proportion to a velocity, mass and inertia moment which determine an acceleration performance of the multi-axis robot can be identified.
In addition, if the various embodiments of the application examples are combined systematically, a system having high-level functions can be achieved.
For example, when an utterance (vocal sound) is identified and learned by the nonlinear characteristic identifying apparatus shown in FIG. 13 and the 1/f fluctuation generating apparatus shown in FIG. 10 gives the waveform fluctuations and pitch fluctuations to reproduced utterance (reproduced vocal sound), the speech synthesis system generating the vocal sound (utterance) having natural fluctuations can be achieved.
As described hereinabove, the structure of URANn according to the present invention is summarized as follows:
1) The two-layer hierarchical structure recurrent neural network (URANn) is provided, each layer having the same number (n) of the neuron elements.
2) The first layer 100 (element group) (xi) is constituted by the linear artificial neurons x1 which add and/or subtract the respective input signals and output signals after the linear amplifications (input sum xi, output xi). In addition, the second layer (element group) yj of the second layer 200 is constituted by the nonlinear artificial neurons yj which add and subtract their respective input signals and output them after the nonlinear calculations (input sum uj and output yj) of yj =f(uj). The nonlinear artificial neurons yj have the nonlinear output functions f, respectively.
3) It is desirable for the nonlinear output function f to convert the input sum uj of the nonlinear artificial neurons yj of the element group of the second layer 200 to the outputs to have the high symmetry. For example, although yj =f(uj)=tanh(uj) has been adopted, the generally known output function of f(uj)=1/ 1+exp(-uj)! (sigmoid function) may, alternatively, be used in place of the former. In the latter case, a suitable change of variables permits the mathematical equivalent discussion to the nonlinear function of tanh(uj).
4) The feedforward connection part 300 multiplies the signal xi by wji and transmits the multiplied signal to each one of the artificial neurons yj of the second layer 200. In details, the outputs of the artificial neurons xi of the first layer 100 are multiplied by wji and, thereafter, are input to each one of all artificial neurons yj of the second layer 200.
5) The recurrent connection part 310 includes the excitatory or inhibitory connections and is provided with the first-order time delay. In details, the signals from the nonlinear artificial neurons yj of the second layer 200 are multiplied by λj, the time delay τj are provided for the λj multiplied signals of the second layer artificial neurons yj and, finally, fedback again into the artificial neurons xj of the first layer 100.
6) The auto connection part 320 includes the excitatory or inhibitory connections and is provided with the first order time delay. In details, the signals from the artificial neurons xi of the first layer 100 are multiplied by μi, the time delays τi are provided so as to be fedback again into the respectively corresponding artificial neurons xi of the first layer 100.
Next, the following advantages can be achieved by URANn summarized above according to the present invention.
(1B) Since URANn has a great margin in settings of the internal parameters, the degree of freedom in design in the internal parameters is so large that URANn can generate the nonlinear oscillations having universal (freely variable) waveforms and frequencies.
(2B) Since the nonlinear function having the high symmetry, for example, f(x)≡tanh(x) has been adopted as the output function of the nonlinear artificial neuron yj of the second layer 200 within the URANn, the analysis of the dynamics in the recurrent neural network (URANn) and the derivations of the formulae can be executed with the good prospect. If f(x)=1/ 1+exp(-x)! is selected as the output function, the suitable change of the variables permits the mathematically equivalent discussion in the same way as the case f(x)=tanh(x).
(3) Since the method of determining the internal parameters of URANn is explicitly indicated, the nonlinear oscillations having the desired physical characteristics can easily be generated with few trial number of times before the nonlinear oscillations occur.
Particularly, since URANn can be structured and the functions of the feedforward part and feedback part are explicitly separated from each other, the formulae determining the respective connection coefficients can be derived independently of each other.
For example, in the case of (n=2) where URANn is URAN2, the connection coefficients of the feedforward part 300 can be given by the equation (4) and the connection coefficients of the connection parts 320 and 310 are given by equations (5) and (6), respectively.
On the other hand, in the case of (n=3) where URANn is URAN3, the connection coefficients of the feedforward part 300 can be given by the equation (8) and the connection coefficients of the feedback parts 320 and 310 can be given by the equations (9) and (10), respectively.
(4B-1) In URAN2 (2-variable Unlimited Recurrent Adjustable Network), by considering the relationships between the physical property of the nonlinear oscillation to be generated and the geometrical shape of the solution trajectory of the differential equation (1) mainly in the state space of x1 -x2 (this state space is a two-dimensional phase space extended by the output potential xi of the corresponding linear artificial neuron x1 of the first layer 100 shown in FIG. 3) and the equilibrium state gi (x1, x2, . . . , xn)=0, the formulae groups of (3) through (6) (27), and (28) to determine the group of the internal parameters of URAN2 are derived so that the desired nonlinear oscillations can be generated. It is noted that the self-sustained oscillation is the limit cycle according to a Poincare'.Bendixon's theorem.
For example, if the group of the internal parameters given as the equation (24) are used, the limit cycle temporal signal (refer to FIG. 6) in the space of xi =t and a trajectory of the limit cycle (refer to FIGS. 7 and 8) in the state space of x1 -x2 can be observed.
(4B-2) In the case where URANn =URAN3 (3-variable Unlimited Recurrent Adjustable Network), the desired nonlinear oscillation (the self-sustained oscillations are such as the limit cycle, torus, and chaos) can be generated by determining the group of the internal parameters from the formulae (7) through (12), and (29) derived in the same way as the case where n=2.
For example, in the case where the group of the internal parameters such as those given by the equations (34) are used, the chaotic temporal signal (refer to FIG. 14) can be observed in the space of xi =t and the strange attractor (refer to FIG. 15) can be observed in the state space of x1 -x2 -x3, respectively.
The following advantages and application examples are considered in the respective embodiments described above.
(1C) In URANn constituted by the respective layers having the n-th number of elements, in the case where the real part of the complex eigen value in the Jacobian matrix (19) of the differential equation in the vicinity of the equilibrium point is negative, this URANn carries out the damped oscillation.
(2C) In URANn constituted by the respective layers having the n-th number of elements, in the case where the real part of the complex eigen value in the Jacobian matrix (19) of the differential equation in the vicinity of the equilibrium point is positive, this URANn carries out the self-sustained oscillation.
(3) In the URAN2 having each layer including the two elements, the URAN2 functions as the nonlinear self-sustained oscillation generating apparatus due to unstability in the equilibrium point caused by the Hopf bifurcation during the relaxation oscillation. The generated oscillation indicates the limit cycle (periodic) trajectory) in the two-dimensional state space. Here, from the above-described item (2C), the real part of the complex eigen value of the Jacobian matrix in the vicinity of the equilibrium point is positive.
(4C) Since the shape of the generated periodic trajectory in the two-dimensional state space under the condition described in the item (3C) corresponds to the stability of the nonlinear ordinary differential equation representing URAN2 expressed by the geometrical shape of the equilibrium state and the parameters of the geometrical shape of the limit cycle and internal parameters of the temperature coefficients and first-order time delays are supposed, the method for calculating the system parameters such as the connection coefficients between the artificial neurons of the URAN2 are derived (refer to FIG. 7).
(5C) In URANn having each layer constituted by the equal n-th number of elements, RANn approximately functions as the conversion apparatus of the nonlinear temporal signal.
In this case, as the amplitude of the forced oscillation which excites URANn from its external is increased, the solution repeats the bifurcation such as the Hopf bifurcation so that its oscillation is transited finally into the chaos via the limit cycle (periodic trajectory) and torus (quasiperiodic trajectory). This can easily be estimated from the physical characteristics of the nonlinear oscillator.
(8C) In URANn having each layer including the n-th number of elements (n≧3), the URANn can exhibit the performance of "neural cocktail party effect" and/or "hyper chaos generation" due to the mutual entertainment between the nonlinear oscillations such as the limit cycle (periodic trajectory) and torus (quasiperiodic trajectory) in the mutually synthesized state of the plurality of oscillators. This can easily be estimated from the physical characteristics of the nonlinear oscillators.
(7C) In URANn having each layer including the n-th number of elements, the method for inputting and outputting the temporal signals to URANn to apply the temporal signal processing function (for example, the application thereof to the nonlinear filter) is shown in FIG. 9.
(8C) In URANn having each layer including the n-th number of elements, the URANn to be applied as the chaotic temporal signal generating system (for example, the application thereof to the 1/f fluctuation generation system) is shown in FIG. 10.
(9C) In URANn having each layer including the n-th number of elements, the temporal signal processing function of URANn which is applied to the feedforward control system is shown in FIG. 11.
(10C) In URANn having each layer including the n-th number of elements, the temporal signal processing function of URANn to the nonlinear feedback control system is shown in FIG. 12.
(11C) In URANn having each layer including the n-th number of elements, the temporal signal processing function of URANn which is applied to the nonlinear characteristic identification apparatus is shown in FIG. 13.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5371834 *Aug 28, 1992Dec 6, 1994The United States Of America As Represented By The Administrator Of The National Aeronautics And Space AdministrationAdaptive neuron model--an architecture for the rapid learning of nonlinear topological transformationsUS5408424 *May 28, 1993Apr 18, 1995Lo; James T.Optimal filtering by recurrent neural networksUS5412754 *Jun 30, 1992May 2, 1995At&T Corp.Reverse time delay neural network for pattern generationJPH04149661A * Title not availableJPH04149663A * Title not available* Cited by examinerNon-Patent CitationsReference1"An Introduction to Neurocomputing", edited by Kaoru Nakano, p. 143, published by Corona Publishing, Apr. 5, 1990.2"Chaos In Neural System", Chapter 7, 1993, pp. 245-283.3"On the Recurrent Neural Networks", Kenu-ichi Funahashi, Toyohashi University of Technology, The Institute of Electronics, Information and Communication Engineers, 1992.4"The New IEEE Standard Dictionary of Electrical and Electronics Terms", 5th edition, p. 716, 1992.5 *An Introduction to Neurocomputing , edited by Kaoru Nakano, p. 143, published by Corona Publishing, Apr. 5, 1990.6 *Chaos In Neural System , Chapter 7, 1993, pp. 245 283.7deFigueiredo, "A neural network model for nonlinear predictive coding in Fock space"; 1993 IEEE International symposium on circuits and systems, pp. 2164-2167 vol. 4, 3-6 May 93.8 *deFigueiredo, A neural network model for nonlinear predictive coding in Fock space ; 1993 IEEE International symposium on circuits and systems, pp. 2164 2167 vol. 4, 3 6 May 93.9Hakim et al, "Volterra characterization of neural networks"; Conference record of the twenty-fifth Asilomar conference on signals, systems and computers, pp. 1128-1132 vol. 2, 4-6 Nov. 1991. Nov. 1991.10 *Hakim et al, Volterra characterization of neural networks ; Conference record of the twenty fifth Asilomar conference on signals, systems and computers, pp. 1128 1132 vol. 2, 4 6 Nov. 1991. Nov. 1991.11 *On the Recurrent Neural Networks , Kenu ichi Funahashi, Toyohashi University of Technology, The Institute of Electronics, Information and Communication Engineers, 1992.12 *The New IEEE Standard Dictionary of Electrical and Electronics Terms , 5th edition, p. 716, 1992.13Yoshida et al, "On the architecture of a recurrent neural network as a chaos 1/f generator"; Proceedings of hte 1995 IEEE Internatonal Conference on Fuzzy Systems. The International joint conference of the fourth IEEE international conference on fuzzy sy Mar. 1995.14 *Yoshida et al, On the architecture of a recurrent neural network as a chaos 1/f generator ; Proceedings of hte 1995 IEEE Internatonal Conference on Fuzzy Systems. The International joint conference of the fourth IEEE international conference on fuzzy sy Mar. 1995.15Youlal et al, "Fast diagonal recurrent neural networks for identification and control of non-linear systems"; International conference on control 94', pp. 104-109 vol. 1, 21-24 Mar. 1994.16 *Youlal et al, Fast diagonal recurrent neural networks for identification and control of non linear systems ; International conference on control 94 , pp. 104 109 vol. 1, 21 24 Mar. 1994.* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS6216118Oct 30, 1997Apr 10, 2001Kabushiki Kaisha MeidenshaApparatus and method for discriminating a time series dataUS6453309 *May 12, 1999Sep 17, 2002Nokia Mobile Phones LimitedMethod for correcting errors in parallel A/D conversion, corrector and parallel A/D converterUS6463424 *Apr 23, 1998Oct 8, 2002Taiko PharmaAssociation unit, association apparatus and method for the sameUS6505182 *Oct 9, 1998Jan 7, 2003Van Den Heuvel Raymond C.Recognition engine with time referenced neuronsUS6601054 *Aug 11, 2000Jul 29, 2003Maryland Technology CorporationActive acoustic and structural vibration control without online controller adjustment and path modelingUS6915283 *Jul 2, 2001Jul 5, 2005Sony CorporationInformation processing apparatus and method, and recording mediumUS7072875Mar 25, 2005Jul 4, 2006Sony CorporationInformation processing apparatus and method, and recording mediumUS7082421Mar 25, 2005Jul 25, 2006Sony CorporationInformation processing apparatus and method, and recording mediumUS7089219Mar 25, 2005Aug 8, 2006Sony CorporationInformation processing apparatus and method, and recording mediumUS7321882 *Oct 5, 2001Jan 22, 2008Fraunhofer-Gesellschaft Zur Foederung Der Angewandten Forschung E.V.Method for supervised teaching of a recurrent artificial neural networkUS7353088 *Oct 25, 2004Apr 1, 2008Ford Global Technologies, LlcSystem and method for detecting presence of a human in a vehicleUS7401058 *Apr 29, 2005Jul 15, 2008University Of MassachusettsArtificial neuron with phase-encoded logicWO2004104917A1 *May 20, 2003Dec 2, 2004Fujitsu LtdSensor and sensor program* Cited by examinerClassifications U.S. Classification706/30, 706/23, 706/41International ClassificationG06N3/04, G06N3/063, G06F15/18Cooperative ClassificationG06N3/0635, G06N3/063European ClassificationG06N3/063, G06N3/063ALegal EventsDateCodeEventDescriptionNov 27, 2001FPExpired due to failure to pay maintenance feeEffective date: 20010923Sep 23, 2001LAPSLapse for failure to pay maintenance feesApr 17, 2001REMIMaintenance fee reminder mailedMar 16, 1995ASAssignmentOwner name: NISSAN MOTOR CO., LTD., JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YOSHIDA, KAZUMI;KISHI, NORIMASA;REEL/FRAME:007387/0893Effective date: 19941018RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services