Noise assisted signal processor with nonlinearly coupled arrays of nonlinear dynamic elements

A signal processor utilizes a globally nonlinearly coupled array of nonlinear dynamic elements. In one embodiment of the invention, these elements take the form of bistable overdamped oscillators. The processor exploits the phenomenon of stochastic resonance to amplify a weak periodic signal embedded in noise. In this signal processor, a system or plurality of nonlinearly coupled overdamped oscillators is subject to a weak periodic signal embedded in a noise background. For communication or detection applications, this weak signal component is the signal of interest. A reference oscillator is chosen from the plurality of overdamped oscillators, and is given a time scale for relaxation that is longer than the remaining oscillators. The output of the reference oscillator is analyzed for signal processing purposes in response to the signal and noise. A detailed numerical analysis of the full dynamics of the bistable element represented by the reference oscillator has shown that the signal-to-noise ratio (SNR) of the entire processor system reaches a maximum at a critical noise variance value. By using a number of overdamped oscillators working together, an enhancement of SNR can be achieved over that of the use of a single oscillator.

BACKGROUND OF THE INVENTION 
The following document is hereby incorporated by reference into this 
specification: Bulsara, A. R. and G. Schmera, May 1993, "Stochastic 
Resonance in Globally Coupled Nonlinear Oscillators", Physical Review E, 
Vol. 47, no. 5, pp. 3734-3737. 
This invention pertains broadly to the field of signal processing. More 
particularly, the invention pertains to a signal processor that exploits 
noise to amplify a signal of interest. In greater particularity, but 
without limitation thereto, the signal processor of the invention utilizes 
the phenomenon of stochastic resonance in a nonlinear dynamic system to 
transfer power from a noise background to a signal of interest. 
Traditional signal processing has relied on various combinations of linear 
filters, including numerical techniques such as the Fast Fourier Transform 
(FFT), that are realizable in both hardware and software. Though the FFT 
is applicable to signals of any frequency, its use requires significant 
computation. 
Hardware filters or processors for very low frequencies can be difficult to 
design. Such filters are, typically, tuned inductor-capacitor resistor 
(LCR) circuits, the resonant frequency of which is changed by capacitor 
and inductor adjustment. For very low frequencies, practical limitations 
exist on the magnitudes of the circuit inductance and capacitance one can 
realize while still producing a high quality circuit. 
Software filters have been developed to overcome the deficiencies of many 
analog filters, but implementing software filters can require complex 
hardware and have significant computational requirements. 
In the above-described conventional signal processing methods, noise, 
whether created naturally or intentionally, is usually considered a 
disruption or a hindrance to communication. This noise is usually 
eliminated or substantially reduced through filtering. In fact, ever since 
the advent of telephone and radio, engineers have devoted tremendous 
efforts to eliminating or minimizing the effects of noise. As a result, an 
entire discipline known as linear filter theory has evolved and has become 
standard teaching to electrical engineering and/or communication students. 
In the cognitive and neural science areas, a nonlinear filtering process 
known as stochastic resonance (SR) has been investigated. To those 
schooled in linear doctrine, filtering with SR begins with a radical 
premise: that noise, either inherent or generated externally, can be used 
to enhance the flow of information through certain nonlinear systems. 
Stochastic resonance is a nonlinear stochastic phenomenon which can 
effectively cause a transfer of energy from a random process (noise) to a 
periodic signal over a certain range of signal and system parameters. It 
has been observed in natural and physical systems and may be one means by 
which biological sensor systems amplify weak sensory signals for 
detection. 
Stochastic resonance has actually been demonstrated in a variety of 
physical experiments ranging from ring lasers to a number of solid state 
devices including SQUIDs (super-conducting quantum interference devices) 
and tunnel diodes. 
Peter Jung, Ulrich Behn, Eleni Pantazelou and Frank Moss have proposed that 
in a network consisting of an infinite number of linearly coupled bistable 
oscillators with linear mean-field interaction, the stochastic resonance 
effect is enhanced over what would be expected for a single isolated 
element of the network. In examining output signal only, the theory 
proposed is confined to the bifurcation point of the effective bistable 
potential that describes the network dynamics (in the large N limit) and 
appears inapplicable away from the bifuraction critical point. In this 
linear coupling theory, the strength and signs of all coupling 
coefficients must be the same. 
SUMMARY OF THE INVENTION 
The invention is a signal processor utilizing a globally nonlinearly 
coupled array of nonlinear dynamic bistable elements. These elements can 
take the form of overdamped oscillators, "overdamped" in this sense 
meaning that the oscillators essentially possess no second time derivative 
in their dynamics. The processor exploits the phenomenon of stochastic 
resonance to amplify a weak periodic signal embedded in noise. The 
processor is applicable beyond critical bifurcation points of a bistable 
potential, and can be used as a way of enhancing the output 
signal-to-noise ratio (SNR) over its value for a single bistable element. 
It can be used with as few as two bistable elements, and can be optimized 
through the adjustment of system parameters. 
In a preferred embodiment of the invention, a system or plurality of 
nonlinearly coupled overdamped nonlinear oscillators is subject to a weak 
periodic signal embedded in a noise background. For communication or 
detection applications, this weak signal component is the signal of 
interest. A reference oscillator is chosen from the plurality of 
oscillators, and is given a time scale for relaxation that is longer than 
the remaining oscillators. The output of the reference oscillator is 
analyzed for signal processing purposes in response to the signal and 
noise. 
A detailed numerical analysis of the full dynamics of the bistable element 
represented by the overdamped reference oscillator has shown that the SNR 
of the entire processor system is maxmized when noise reaches a critical 
value. By coupling a number of overdamped oscillators together, an 
enhancement of SNR can be achieved over the use of a single oscillator. 
OBJECTS OF THE INVENTION 
It is an object of this invention to provide a signal processor that is an 
improvement over hardware or software linear filters including linear 
filtering techniques such as the Fast Fourier Transform. 
Another object of this invention is to provide a signal processor in which 
noise is a feature to be utilized rather than to be suppressed. 
Still a further object of this invention is to provide a signal processor 
that utilizes the phenomenon of stochastic resonance to enhance the 
detection or other processing of a signal of interest. 
Other objects, advantages and new features of the invention will become 
apparent from the following detailed description of the invention when 
considered in conjunction with the accompanying drawings.

DESCRIPTION OF THE PREFERRED EMBODIMENT 
A plurality of nonlinearly coupled, nonlinear dynamic elements subject to a 
weak periodic signal embedded in noise can be described by the system: 
##EQU1## 
In Equation (1) it is assumed that each element of this system is subject 
to Gaussian delta-correlated noise having zero mean and variance D. It 
should be noted, however, that the noise is not restricted to be Gaussian 
or delta-correlated but could be other forms of noise as well. 
In one embodiment of the invention, the nonlinear dynamic elements are 
nonlinear overdamped oscillators to be described in greater detail. The 
index i=1 is taken to denote a reference element/oscillator and the 
indices i=2, . . . N denote the remaining plurality of 
elements/oscillators. C.sub.i and R.sub.i. denote the input capacitance 
and resistance of the oscillator with index i. The term u.sub.i denotes an 
activation or forcing function input that may take any of a number of 
forms, e.g. voltage, sound intensity, light intensity, magnetic flux. The 
term J is a coupling coefficient to be discussed. The hyperbolic tangent 
(tanh) is a nonlinear coupling function. The hyperbolic tangent of u.sub.i 
is computed to produce a nonlinear element coupling. It should be noted 
that other nonlinear coupling functions such as trigonometric functions, 
other hyperbolic functions and powers can also be used. The term F(t) is 
noise and q sin.omega.t is a weak periodic signal in which q is the 
amplitude of the signal and .omega. is the frequency of the signal. FIGS. 
1A and 1B show a block diagram of how a system described by equation (1) 
can be realized in analog circuitry. 
In FIGS. 1A and 1B, a signal processor 10 is shown incorporating a 
representative plurality (N) of nonlinear bistable overdamped oscillators 
12. Each oscillator 12 is globally connected to every other oscillator 12 
within processor 10. What is meant by "globally connected" is that each 
oscillator feeds/interacts with every other oscillator or, put another 
way, each oscillator receives an output derived from every other 
oscillator. Each oscillator 12 is also connected to receive and output 
derived from itself. 
In FIGS. 1A and 1B the blocks containing "J" are "coupling coefficients" 
that are the result of a chosen function the preferred form to be 
discussed. These coupling coefficients, designated as 14 in FIGS. 1A and 
1B, are shown with a first digit indicating the coefficient's destination 
oscillator and a second digit indicating its source oscillator. For 
example, J.sub.ij has an output going to destination oscillator T.sub.i 
and has an input received from source oscillator T.sub.j. 
The summers 16, indicated symbolically with a sigma, each serve to sum 
those oscillator outputs, appropriately multiplied by (adjusted by) 
coupling coefficients 14, desired to be input back to a particular 
oscillator 12. For example, output summation S.sub.1 is the sum of all 
oscillator 12 outputs T(u) as multiplied by coefficients J.sub.11 . . . 
J.sub.1N. Similarly, summation S.sub.2 is the summation of all oscillator 
12 outputs T(u) times coefficients J.sub.21 . . . J.sub.2N. The summation 
output is shown by the equation 
##EQU2## 
According to the invention, the output of the "reference" oscillator, 
u.sub.1 (t), of the plurality of overdamped oscillators, is measured in 
response to the signal (q sin.omega.t) and noise (F(t)). For communication 
or detection applications, the signal component q sin.omega.t is the 
signal of interest. 
In FIG. 2 a representative oscillator is shown. This oscillator is 
identified as T.sub.1, which, in this description of the invention, is the 
oscillator chosen as a reference oscillator. It should be understood that 
the reference oscillator could be any of the overdamped oscillators 
described by equation (1) or shown in FIGS. 1A and 1B. As with all the 
oscillators 12 of FIGS. 1A and 1B, a "coupling summation" input, in this 
case S.sub.1, will be input to the oscillator. In addition, each 
oscillator will have two additional inputs, the noise input F(t) and the 
weak periodic source signal (q sin.omega.t), see FIG. 2. It should be 
noted that what is meant by weak in this case is that q is small compared 
to F (the intensity of F). The input signal to noise ratio (SNR) 
referenced to noise power in the frequency range .DELTA..nu. is defined 
as: 
##EQU3## 
where D equals the variance of noise F and q is the amplitude of the 
"weak" periodic signal. The noise source F(t) and the weak periodic signal 
source q sin.omega.t are summed in summer 18 and are then combined with 
coupling summation S.sub.1, shown in equation form as 
##EQU4## 
and -1/R.sub.1 in a summer 20. This sum is then multiplied by 1/C.sub.1. 
Again, R.sub.1 and C.sub.1 in this instance equal the capacitance and 
resistance, respectively, of the oscillator having the index i=1. The 
output of summer 20, shown in equation form as 
##EQU5## 
is then integrated in integrator 22. Oscillator output signal u.sub.1 (t), 
from integrator 22, is in turn processed by nonlinear coupling function 24 
shown here as the hyperbolic tangent (tanh (u.sub.1)). This nonlinear 
coupling function 24 computes a nonlinear function of integrator 22 output 
u.sub.1 (t). As mentioned earlier, other nonlinear coupling functions 
could be used, such as trigonometric functions, other hyperbolic functions 
as well as powers, etc. It should be noted that the nonlinear term does 
not need to be incorporated within oscillator element 12 but could be 
utilized as a separate element. Nonlinear output signal T(u.sub.1) is then 
forwarded to the appropriate coupling coefficients 14 shown in FIGS. 1A 
and 1B. 
Oscillator output signal u.sub.1 (t) is an oscillator output taken 
immediately after integrator 22 and before the nonlinear coupling term is 
applied. This output signal, in one use of the invention, can be analyzed 
for communication or detection purposes. 
According to a preferred embodiment of the invention, the time scale for 
relaxation of the reference oscillator should be made longer than the rest 
of the overdamped oscillators to which it is connected: 
EQU C.sub.i R.sub.i &lt;C.sub.1 R.sub.1 (i&gt;1). (2) 
From the oscillator system described in Equation (1), the dynamics of the 
reference oscillator that is characterized by state variable u.sub.1 (T) 
can be extracted and cast in the form: 
##EQU6## 
where, 
##EQU7## 
and F(t) is, in this preferred embodiment, now Gaussian delta-correlated 
noise having zero mean and unit variance. Equation (3) represents a 
one-variable nonlinear dynamic system describable by a "potential 
function" 
##EQU8## 
which has turning points at u.sub.1 =0 and u.sub.1 
=.+-.c.apprxeq..beta./.alpha. tanh(.beta./.alpha.). If one numerically 
integrates (3) and computes the power spectral density S(.OMEGA.) of the 
solution u.sub.1 (t), and the signal-to-noise ratio (SNR), it will be 
observed that the SNR reaches a maximum as the noise variance D.sub.e 
reaches a critical value D.sub.c. Typically, one obtains D.sub.c 
.apprxeq.U.sub.0 .ident.U(0)-U(c), the height of the potential barrier. 
By utilizing a plurality of nonlinearly coupled nonlinear overdamped 
oscillators it is possible to enhance the SNR over what would be observed 
for a single (isolated) oscillator case. Referring to FIG. 3, the 
enhancement of SNR by using multiple nonlinearly coupled bistable 
overdamped oscillators can be seen. 
In this figure, the double thin line curve shows input SNR. The remaining 
curves show theoretical predictions for output SNR, for one oscillator 
element (dashed-exed), and two (dashed-diamond), three (dashed-dotted), 
and four (connected dots) coupled oscillator elements. Data points (shown 
as dots) give results of numerical simulations. System parameters are: 
EQU .omega.=0.1, q=0.1, R.sub.1 =1, C.sub.1 =1, J.sub.11 =2.5, R.sub.i 
=0.1,C.sub.i =1,J.sub.ii =1, J.sub.12 =2, J.sub.13 =2.1,J.sub.14 
=1.9,J.sub.i1 =-1,i&gt;1 
(For FIG. 3, the width of each FFT bin is 0.00015955 Hertz, and the FFT 
processing gain, using a Welch windowing function, is 0.8333). 
From FIG. 3 it is easy to see that the signal-to-noise ratio reaches a 
maximum at a particular noise variance value D and that this effect is 
enhanced through the use of multiple coupled overdamped oscillators. 
The invention exploits the nonlinear dynamic characteristics of a system 
such as that described by Equation (1). A number of specific advantages of 
this invention are worth indicating. Though the system of the invention 
may be used to detect and process signals at almost any frequency, it 
works extremely well at low frequencies. The only requirement for 
optimization of the invention in this regard is that the characteristic 
noise-induced hopping frequency of the processor (Kramers frequency 
r.sub.0) be comparable to the modulation frequency (.omega.) such as by a 
factor of 2 and that the noise be white over a bandwidth at least two 
orders of magnitude greater than the modulation frequency. To optimize the 
detection process for a given input frequency, the system parameters 
should be adjusted such that the Kramers rate r.sub.0 is of the same order 
of magnitude as the signal frequency .omega. for the composite system (3). 
However, the stochastic resonance effect is obtained even when r.sub.0 and 
.omega. differ by several orders of magnitude. Here, r.sub.0 is given by: 
##EQU9## 
where U.sub.0 is the potential barrier height defined above, and we define 
U.sup.(2) (0).ident.[d.sup.2 U/du.sub.1.sup.2 ].sub.u.sbsb.1.sub.=0 and 
U.sup.(2) (c).ident.[d.sup.2 U/du.sub.1.sup.2 ].sub.u.sbsb.1.sub.=c 
where c is the location of the rightmost minimum of the effective potential 
(5). Note that r.sub.0 is adjustable by changing system and noise 
parameters. It is independent of .omega.. 
A further advantage of the signal processor of the invention is that, 
noise, whether arising inherently or superimposed on the signal itself, 
can actually be used to enhance signal detection. In fact, for extremely 
weak signals, injecting carefully controlled amounts of noise into the 
system can actually enhance the system's signal-to-noise ratio output. 
This effect does not occur in conventional signal processing. 
Finally, it has been realized that by nonlinearly coupling large numbers of 
nonlinear circuit elements, constrained, for example, by Equation (2), 
larger output signal-to-noise ratios will be possible than with a single 
(isolated) oscillator. Ultimately, for large numbers of coupled elements 
subject to the constraint (2), the output SNR approaches (even at low 
noise) the input SNR. The system of the invention thus performs like an 
"optimal" linear system. 
For the special case in which all the oscillator resistors R.sub.i are set 
the same, the system output SNR becomes nearly equal to the input SNR. 
This effect can be realized even with as few as two coupled overdamped 
oscillators. For this case, however, a theoretical characterization of the 
form of equation (3) is not known to exist. 
Superior results appear to be obtainable for coupling coefficients J 
subject to the conditions: 
(i) J.sub.11 &gt;(R.sub.1 C.sub.1).sup.-1 &gt;0 . . . this ensures bistability in 
the composite dynamics (3) and 
(ii) J.sub.1i &gt;0, J.sub.i1 &lt;0 with .vertline.J.sub.1i .vertline., 
.vertline.J.sub.i1 .vertline.&lt;J.sub.11. 
An estimate of the output SNR referenced to noise power in the angular 
frequency range .DELTA..omega. may be obtained using the approximate 
expression 
##EQU10## 
The other quantities in (7) are defined as: 
EQU N.sub.0 =[1-8Z(r.sub.0 .zeta.).sup.2 ](8Zr.sub.0 c.sup.2) (8a) 
EQU S.sub.0 =16Z.pi.(r.sub.0 .zeta.c).sup.2 (8b) 
where we define Z=(4r.sub.0.sup.2 +.omega..sup.2).sub.-1 and 
.zeta.=.delta.c/D.sub.e is a perturbation theory expansion parameter. This 
expression may be used to approximately calculate the output SNR for a 
given set of coupling coefficients and compare it to the input SNR. Hence, 
it serves as a guide to the best selection of the coupling coefficients. 
To use the invention as a signal processor/detector, one would first make a 
determination of a band of frequencies and a range of amplitudes (q) in 
which a signal of interest (.omega.) would fall. Further, the noise 
intensity background (F(t)) should be surmised. The capacitance terms, 
resistance terms and the coupling coefficients (J) of Equation (1) would 
then be chosen. In particular, the capacitance, resistance and coupling 
term J of the reference and remaining overdamped oscillators would be 
chosen so that a peak in signal-to-noise ratio, such as determined through 
equation (7), is achieved close to the frequency of interest. The 
nonlinear function of Equation (1) could, of course, be changed; however, 
the hyperbolic tangent has been chosen as this function closely resembles 
that of biological systems thought to employ stochastic resonance 
efficiently. 
Obviously, many modifications and variations of the invention are possible 
in light of the above teachings. It is therefore to be understood that 
within the scope of the appended claims the invention may be practiced 
otherwise than as has been specifically described.